What is the state of art on implementing a p2p message-broadcast network?

Question

I'm aware a quick Google wields tons of results and the literature on this topic is very rich, and that is exactly the problem. Among the universe of possible solutions I'm not sure which specifically is the best, up-to-date choice for my specific needs.

I'm trying to implement a p2p network over the internet with the only and one feature of broadcasting messages to online nodes. In order to connect to the network, you must be able to point to existing IPs. When you do, you discover some peers and keep an active connection with them. You then can send messages to every other node. There is no direct communication, every message sent is received by everyone else. I want this network to be as performant as possible, and work on top of UDP.

What are the names of some state of art algorithms to solve this problem on the shape here specified?

Answer 1

A broadcast of a single message in an overlay network over the internet is a fairly simple affair. You join the network, build a randomized or structured routing table of neighbors, flood neighbors with the message, they do the same with their neighbors, minus some pruning to avoid forwarding loops.

Complexity arises from additional constraints and requirements which vary from network to network. E.g. trust problems in anonymous settings, latency optimizations, optimizing high-bandwidth streams of many messages (p2p-assisted live video broadcasting), etc.

Answer 2

If you want to learn about the concepts for setting up decentralized P2P architecture, you could take a look at the Dat Foundation, who provide an ecosystem of modules for P2P data sharing over the internet, such as Hypercore (raw P2P streams) and Hyperdrive (file transfer on top of hypercore). They have some good documentation on technical concepts, challenges and how they tackled them.

Some of the challenges in P2P designs:

• NAT routers and firewalls: overcome by implementing NAT Traversal and UDP hole punching (see: The State of NAT Traversal by ZeroTier)
• Discovering peers: Dat Project uses Gossiping as an efficient way for a swarm of peers to find each other on the network, and a simple (protobuf) wire protocol for communication (see: Hyperdiscovery and Hypercore Protocol).
• P2P data communication / synchronization: Hypercore implements append-only logs on each peer, and Merkle trees for aggregating and deduplicating data chunks from other peers in correct order

Both TCP, UDP, WebRTC and BittorrentDHT are supported by Dat Project.

Please read some of their specification docs for in-depth information on these concepts and the protocol design:

• Dat - Distributed Dataset Synchronization And Versioning (pdf)
• How Dat works
• Hypercore Protocol