Iroh Primitives: How Data and Peers Connect

Iroh’s Content-Addressed Data: Why Immutability Matters

Iroh refers to all data by a cryptographic hash of its content, not by its location or a mutable name. This approach, known as content addressing, ensures that every distinct piece of data has a unique, verifiable identifier. The hash acts as a fingerprint; even a single bit change in the data results in a completely different hash.

This content-addressing model fundamentally implies immutability. Once data is stored and its hash is computed, that specific data cannot be altered. Any modification creates new data with a new hash. The original data, identified by its initial hash, remains unchanged. This property simplifies data management in distributed environments.

Immutability offers several advantages within Iroh:

• Verifiability: The content hash serves as a built-in integrity check. Any peer receiving data can recompute its hash and compare it to the expected value, immediately detecting corruption or tampering.
• Deduplication: Identical data, regardless of where it originates, will always produce the same content hash. This allows Iroh to store and transfer data efficiently, avoiding redundant copies across the network or on disk.
• Referential Integrity: Links to content-addressed data are stable. A hash always points to the exact same data, ensuring that references remain valid over time, even if the data moves between storage locations or peers.

Iroh stores all raw data as immutable blobs, each identified by its content hash. This hash is a 32-byte Blake3 digest, represented as a 64-character hexadecimal string.

Adding text to Iroh generates a unique hash:

iroh blob add --text "Hello, Iroh!"

ba32a7629634d40974ed2e8b610c49740523b038029d95f8e5f22f7e41103f6f

Adding the exact same text again produces the identical hash, demonstrating deduplication:

iroh blob add --text "Hello, Iroh!"

ba32a7629634d40974ed2e8b610c49740523b038029d95f8e5f22f7e41103f6f

A slight modification, however, results in a completely new hash, confirming immutability:

iroh blob add --text "Hello, Iroh!!"

6791e81255767b090a194520937a0928923a1a0f9185a67f10b89231f85b62b0

This foundational approach to data ensures consistency and reliability across Iroh’s distributed network, simplifying operations that would otherwise require complex synchronization and conflict resolution mechanisms.

How Iroh Blobs & Docs Structure Data

Iroh structures data using two primitives: Blobs and Docs. These distinct types address different data persistence needs within a distributed system.

Blobs store immutable content. Each blob is identified by its cryptographic hash, ensuring that its content cannot change once created. This content-addressing property makes blobs suitable for static files, media assets, or any data that should remain unaltered. Retrieving content requires only its hash.

To add a file as a blob, use the iroh add command:

echo "Hello, Iroh!" > greeting.txt
iroh add greeting.txt

Hash: bafyreib7w54z73j62l47e7d5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a
Size: 14

Any modification to greeting.txt would result in a different hash, creating a new blob. Blobs offer simplicity and content integrity. They do not natively support in-place updates; a new version of data always means a new blob hash.

Docs provide mutable, versioned collections of key-value pairs. Each Doc has a unique identifier (DocId) and maintains a history of changes. This structure is designed for dynamic data, such as application state or collaborative documents, where content updates are frequent and tracking history is beneficial.

Creating a new Doc generates its unique identifier:

iroh doc create

Doc ID: doc_bafyreib7w54z73j62l47e7d5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a

Data within a Doc is stored as key-value pairs, where keys are byte arrays and values are Blobs. Setting a key in a Doc updates its content and creates a new version, preserving the previous state. This allows for conflict resolution and historical queries across connected peers.

# Assuming DOC_ID is set from the 'iroh doc create' output
iroh doc set <DOC_ID> my_key "First value"
iroh doc set <DOC_ID> another_key "Second value"

Docs provide mutability and version history, which introduces more complexity with synchronization and conflict resolution mechanisms compared to simple blob transfer. The overhead of managing versions and merging concurrent updates is the cost for dynamic data capabilities.

Iroh Tickets & Peer Discovery Internals

Iroh tickets encapsulate the necessary information for a peer to locate and retrieve data from another node. A ticket combines a content address, the publishing node’s identifier, and potential relay addresses. This structure enables secure, decentralized data exchange without requiring a central directory.

A ticket contains three primary components: the content hash, the peer’s Node ID, and a list of relay addresses. The content hash, typically a BLAKE3 digest, verifies data integrity. The Node ID is a cryptographic identifier for the publishing peer, ensuring the connection is established with the correct source. Relay addresses provide alternative connection points for peers behind Network Address Translators (NATs).

Consider generating a ticket for a blob of data:

iroh share /path/to/my/data.txt
# Example output: iroh:brp546s5... (a long string representing the ticket)

The output will be an Iroh ticket string. This string is shareable and contains all the information another Iroh node needs to fetch the data.

Peer discovery begins when a node receives an Iroh ticket. The receiving node first attempts a direct connection to the publishing peer using any known IP addresses or through the peer’s advertised network addresses. This is the most efficient method, but often fails due to NATs or firewalls preventing direct inbound connections.

If a direct connection is not possible, the receiving node uses the relay addresses embedded in the ticket. Relays are public Iroh nodes that facilitate connections between peers unable to connect directly. This adds latency and bandwidth overhead compared to a direct connection but provides connectivity in most network environments.

For scenarios where only the Node ID is known, or when initial connection attempts fail, Iroh uses a Kademlia-like Distributed Hash Table (DHT) for peer discovery. Nodes advertise their presence and network addresses on the DHT. A node can query the DHT for a specific Node ID, retrieving its current network addresses, including any available relays. This mechanism is more resilient to network changes but involves higher latency due to DHT lookups.

The choice between direct connection, relay, or DHT lookup involves tradeoffs between speed, reliability, and network overhead. Direct connections are fast but fragile. Relays offer higher reliability but introduce an intermediary. The DHT provides robust discovery but at the cost of increased initial latency.

Ship Iroh Data: Blobs, Docs, and Tickets in Practice

Iroh uses content-addressed blobs as its fundamental data unit. These blobs are organized within mutable documents (Docs), and shared via Tickets. This example demonstrates creating a blob, adding it to a Doc, and sharing it between two Iroh nodes.

Start an Iroh Node

All Iroh operations require a running node. This node manages local data and peer connections. For this example, start an Iroh node listening on an ephemeral port.

iroh start --listen 127.0.0.1:0

The output confirms the node’s peer ID and address, which other peers use to connect.

INFO  iroh::node > Started Iroh node with peer ID: 16Uiu2HAmN8K... and listening on: /ip4/127.0.0.1/tcp/51234

Add a Blob

First, create a simple text file. Then, add this file to your Iroh node. Iroh stores its content as a blob and returns its content hash.

echo "Hello, Iroh data!" > greeting.txt
iroh add greeting.txt

The iroh add command computes a content hash for greeting.txt and stores the data locally.

Hash: 4u6gbx...
Size: 18

Create a Doc and Insert the Blob

Docs provide a mutable key-value store where keys map to blob hashes. Create a new Doc to get its unique identifier.

iroh doc create

This command outputs a unique identifier for the new Doc.

Doc ID: doc_qr7h...

Insert the blob into this Doc. Replace <doc_id> and <blob_hash> with the values from your terminal output. This operation updates the Doc, linking the key “greeting” to the blob’s content hash.

iroh doc set <doc_id> greeting <blob_hash>

Generate a Ticket

To share this Doc, generate a Ticket. A Ticket encapsulates the Doc ID, the sharing node’s address, and a secret for verification. The output is a single, base32-encoded string representing the Ticket, which you then share with another peer.

iroh doc share <doc_id>

ticket_s7v...

Retrieve the Doc with a Ticket

On a separate Iroh node, use the generated Ticket to retrieve the Doc. For local demonstration, open a new terminal and run iroh start there before executing the iroh doc get command. This command connects to the sharing node, retrieves the Doc’s state, and downloads any associated blobs not already present locally.

iroh doc get <ticket_string>

Doc ID: doc_qr7h...
Key: greeting, Hash: 4u6gbx..., Size: 18

To view the content of the retrieved blob, specify the Doc ID and key. This completes the round trip: data created, added to Iroh, organized in a Doc, shared via a Ticket, and retrieved by another peer.

iroh doc get <doc_qr7h...> greeting

Hello, Iroh data!

Iroh Primitives: What Breaks and Why

Iroh peers often fail to connect initially due to incorrect addresses or network restrictions. An iroh-net address uniquely identifies an Iroh peer and its listening endpoints. These addresses include a peer ID, an optional iroh-net ticket, and potentially direct IP addresses and ports.

A common mistake is assuming a peer ID alone is sufficient for connection. A peer ID identifies who a peer is, but not where or if it is currently online and listening. To establish a connection, you need an active iroh-net address. Firewalls or Network Address Translators (NATs) frequently block the necessary UDP and TCP ports, preventing peers from discovering each other or forming direct connections.

When attempting to fetch a blob, receiving a hash does not guarantee data availability. A hash is a content identifier; it does not contain location information. The peer that originally published the blob, or any other peer currently holding a copy, must be online and reachable for a fetch operation to succeed. If no peer with the data is online, the fetch will eventually time out.

Practice: Observing a Fetch Timeout

This exercise demonstrates a fetch failure when the source peer is offline.

1. Start two Iroh nodes in separate terminals.

   iroh start --rpc-port 8001 --web-port 8081 --store-dir ~/.iroh/node_a

   iroh start --rpc-port 8002 --web-port 8082 --store-dir ~/.iroh/node_b

2. Add a blob to node_a and note its hash.

   echo "hello iroh" > test.txt
   iroh --rpc-port 8001 doc new
   # Output includes a Document ID, e.g., doc-d4b...
   iroh --rpc-port 8001 doc add doc-d4b... test.txt
   # Added blob with hash: blb-e3b...

   Replace doc-d4b... and blb-e3b... with the actual values from your output.

3. Stop node_a. This simulates an offline peer. Use Ctrl+C in the terminal running node_a.

4. Attempt to fetch the blob from node_b using the hash.

   iroh --rpc-port 8002 get blb-e3b...

   This command will hang for a period (typically 30-60 seconds) and then report a timeout or “NotFound” error, as node_b cannot find any online peer holding the data.

5. Restart node_a and establish a connection from node_b to node_a. First, get node_a’s iroh-net address.

   iroh start --rpc-port 8001 --web-port 8081 --store-dir ~/.iroh/node_a
   iroh --rpc-port 8001 node addr

   Peer ID: p-b42...
   Addrs: iroh-net-q7g...

   Then, connect node_b to node_a using the address starting with iroh-net-.

   iroh --rpc-port 8002 node connect iroh-net-q7g...

6. Fetch the blob again from node_b.

   iroh --rpc-port 8002 get blb-e3b...

   The data hello iroh will now be retrieved successfully. This demonstrates that a live connection to a data-holding peer is required for blob retrieval.