IPFS - Content Addressed, Versioned, P2P File System

Reference: Juan Benet (2014). arXiv:1407.3561 (DRAFT 3). Source file: downloads/ipfs_benet.pdf. URL

Summary

Original whitepaper introducing the InterPlanetary File System (IPFS), a peer-to-peer hypermedia protocol that synthesises ideas from Git (Merkle-DAG object model), BitTorrent (incentivised block exchange via BitSwap), DHTs (Kademlia-based routing), and self-certifying file systems (SFS) into a single content-addressed distributed file system. Objects are referenced by the cryptographic hash of their contents, giving intrinsic integrity, deduplication, and location independence.

The design stack layers an identity/peer-routing layer (S/Kademlia DHT), a block exchange layer (BitSwap with tit-for-tat credit), an object graph layer (Merkle-DAG with links), a naming layer (IPNS for mutable pointers signed by keys), and application-level filesystem semantics. The paper positions IPFS as the substrate for a “permanent web” where content survives independent of any particular host.

Key Ideas

Content addressing: name = hash(content) -> intrinsic integrity, dedup, cacheability
Merkle-DAG as universal object model spanning files, directories, commits
BitSwap: BitTorrent-like block trading generalised beyond single swarms
Kademlia DHT for peer and content routing
IPNS: mutable pointer namespace via signed records keyed to public keys
Self-certifying paths and location-independent fetching

Connections

Conceptual Contribution

Claim: A single peer-to-peer protocol layered over Merkle-DAG content addressing can unify the roles played today by HTTP, Git, BitTorrent, and CDNs, yielding a permanent, decentralised, versioned web.
Mechanism: Five-layer stack — S/Kademlia DHT for identity/routing, BitSwap for incentive-compatible block exchange, Merkle-DAG for the object graph, IPNS for mutable naming via signed records, and filesystem-level conventions above that. All objects are referenced by their SHA-2 multihash, so any peer can serve any object and clients can verify it independently.
Concepts introduced/used: Content-addressed Storage, Merkle-DAG, Gossip Protocols, BitSwap, IPNS, Self-certifying paths, Peer Sampling Service, Hypermedia
Stance: engineering / systems
Relates to: Directly underpins the distribution story for Protocol Documents in A Scalable Communication Protocol for Networks of LLMs — Agora PDs are hash-identified and can be served over any content-addressed store, of which IPFS is the canonical example. Shares the decentralised-naming motivation with Decentralized Identifiers and the self-organising-overlay motivation with Myconet Fungi Inspired Superpeer Overlay.

Tags

#content-addressing #p2p #distributed-systems #protocols #merkle-dag

Summary

Myconet is a biologically inspired self-organizing overlay-construction protocol for superpeer-based P2P networks, drawing its design from the growth patterns of fungal mycelia. Each node is modeled as a “biomass peer” with a capacity; hyphal peers extend links toward high-capacity regions, branch when saturated, and become immobile superpeers at full utilization. The protocol layers these stigmergic transitions on top of Newscast-style gossip to move biomass toward the highest-capacity peers and dynamically maintain a near-optimal number of well-connected superpeers.

Evaluation shows Myconet converges to approximately optimal superpeer counts across power-law and uniform capacity distributions, reaches ~95% utilization, and self-heals quickly after catastrophic loss of 30-50% of superpeers. It outperforms or matches comparable approaches (SG-1, ERASP) while providing stronger topology stability and resistance to targeted attack on superpeers.

Key Ideas

Biomass peers + hyphal peers + immobile (superpeer) peers with state transitions.

Stigmergy over Newscast gossip for local-only self-organization.

Extension, branching, and absorption rules tuned to heterogeneous capacities.

Near-optimal superpeer count and high utilization under realistic distributions.

Resilience: rapid reconvergence after catastrophic peer loss.

Conceptual Contribution

Claim: Fungal mycelium growth — biomass flowing along hyphae toward productive regions — is a productive metaphor for self-organising superpeer overlays, yielding topologies that are near-optimal in superpeer count, highly utilised, and resilient to targeted attack.

Mechanism: Models each peer with a capacity and three states (biomass, hyphal extending/branching, immobile superpeer); state transitions driven by local biomass comparisons with neighbours obtained via Newscast gossip; branching hyphae explore new capacity, immobile hyphae absorb biomass up to saturation. Simulation on 10^3–10^6 peers with power-law and uniform capacity distributions shows convergence in ~30-35 rounds and recovery from 50% catastrophic peer loss.

Concepts introduced/used: Stigmergy, Mycelium Model, Hyphal Peer, Biomass, Superpeer Overlay, Newscast Gossip, Self-Organising Topology, Catastrophic Failure Recovery

Stance: empirical / engineering (bio-inspired)

Relates to: Uses the peer-sampling/gossip substrate formalised in Gossiping in Distributed Systems and Gossip-based Aggregation in Large Dynamic Networks; exemplifies the structural/ensemble self-organisation advocated by Self-Adaptation Self-Expression Self-Awareness ASCENS and the bio-inspired cognition of Computational Boundary of a Self.

Summary

Introduces Agora, a meta-protocol for inter-agent communication in large heterogeneous networks of LLM-powered agents. Agora frames the design space as the Agent Communication Trilemma — versatility, efficiency, portability — and argues no single format (natural language, structured APIs like REST, or semantic-web RDF) can satisfy all three simultaneously.

Agora’s trick is to use different formats for different traffic volumes: rare/novel messages flow as natural language handled by LLMs; frequent patterns are formalised into Protocol Documents (PDs) negotiated between agents and then served by cheap LLM-written routines. A 100-agent demo shows emergent self-organising protocols and ~5x cost reduction over natural-language-only communication.

Key Ideas

Agent Communication Trilemma: versatility vs efficiency vs portability

Protocol Documents (PDs): hash-identified, agent-negotiated, machine-readable specs

Hybrid hierarchy: NL bootstrap -> PD negotiation -> LLM-written routines -> traditional protocols

Fully decentralised, hash-addressed storage (IPFS-compatible)

Emergent protocols among 100 heterogeneous LLM agents without central coordination

Conceptual Contribution

Claim: No single communication format can simultaneously satisfy versatility, efficiency, and portability (the Agent Communication Trilemma) at scale; a meta-protocol that dynamically mixes natural language, structured data, and LLM-written routines can sidestep the trilemma.

Mechanism: Agora uses hash-identified Protocol Documents (PDs) — plain-text, implementation-agnostic specs — negotiated on demand between LLM agents. Frequent traffic is handled by cheap LLM-written routines implementing a PD; rare or novel traffic falls back to LLMs with natural language. Decentralised, content-addressed (IPFS-style) PD distribution; demonstrated on a 100-agent heterogeneous network showing emergent protocols and ~5× cost reduction.

Concepts introduced/used: Agent Communication Trilemma, Protocol Documents, Meta-protocol, Emergent Protocols, Emergent Communication, LLM Agents, Content-addressed Storage, Negotiated Protocols, Negotiation, Agent Communication Languages

Stance: engineering / systems

Relates to: Modern successor to KQML as an Agent Communication Language and FIPA-ACL, replacing stipulated performatives with negotiated PDs. Echoes the emergent-language findings of Multi-Agent Cooperation and the Emergence of Natural Language and Emergence of Grounded Compositional Language in Multi-Agent Populations at the protocol-document level. Overlaps the design space of Model Context Protocol, Agent-to-Agent Protocol, Agent Network Protocol. Its bottom-up spirit mirrors The Extensible Language - Graham.

IPFS - Content Addressed, Versioned, P2P File System

Summary

Key Ideas

Connections

Conceptual Contribution

Tags

Backlinks