Skip to main content

Concepts

This page explains the concepts of IOTA Storage with the values of the current testnet network (100 shards, 1-day epochs).

Blob​

A blob is any sequence of bytes stored on the network: a file, an image, a web page, a dataset. The network does not interpret it.

  • Blobs are public: anyone who knows a blob ID can read the blob. Encrypt your data before storing it if it must stay private.
  • A blob is stored for a number of epochs that you pay for, and can be renewed.
  • The largest blob depends on the number of shards. With 100 shards it is 149,286,452 bytes (about 142 MiB). The publisher accepts at most 10 MiB per blob.

Blob ID​

The blob ID identifies a blob by its content. It is a BLAKE2b-256 hash of the encoding type, the blob length and the Merkle root of the hashes of its slivers. Its text form is the base64url encoding (without padding) of those 32 bytes: 43 characters.

  • The client computes it locally, before contacting the network. The contract recomputes it when the blob is registered (register_blob) and rejects a blob ID that does not match the root and the size.
  • The same bytes always give the same blob ID on this network. The blob ID depends on the number of shards, so a network with a different number of shards would give a different one. For example, the 25-byte text Hello from IOTA Storage! followed by a newline has the blob ID OshHTDIFITnA3F-HioxOxCmRkWq4AO1wwZVLnWd_gec with 100 shards.

Encoding, slivers and shards​

Blobs are encoded with RS2, an erasure coding based on Reed–Solomon codes, with primary and secondary slivers. With n shards, the network tolerates f = ⌊(n − 1) / 3⌋ faulty shards: 33 with 100 shards.

  • Sliver. The encoding splits the blob into n sliver pairs, each with a primary sliver and a secondary sliver. A reader only needs n − 2f primary slivers (34 with 100 shards) to rebuild the blob.
  • Shard. The unit of responsibility for storage. Each sliver pair goes to one shard (the assignment is rotated by the blob ID), and each storage node holds a set of shards. Today there are 4 nodes with 25 shards each.
  • Encoded size. What is stored, and paid for, is the encoded size: with 100 shards, about 4.4 times the original size plus about 0.6 MiB of metadata per blob. That fixed part is why small files are better grouped in quilts.

Epoch​

An epoch is the period during which the set of storage nodes and their shards does not change. On testnet an epoch lasts 1 day.

  • Storage is bought by epochs: from 1 to 53 epochs per purchase. A blob is available until its end epoch; after that, nodes delete its data.
  • The storage nodes advance the epochs themselves on IOTA (voting_end, initiate_epoch_change_v2 and epoch_sync_done).

Committee​

The committee is the set of storage nodes of an epoch, with their shards and their BLS public keys. It is chosen by stake and kept in the System and Staking objects of the almacen package. Clients read it from IOTA to know which node holds each shard.

Certificate and quorum​

After receiving the metadata and the slivers of its shards, each node returns a confirmation signed with BLS over the epoch and the blob ID (and, for a deletable blob, the ID of its Blob object).

  • The client aggregates the signatures of nodes that together hold a quorum of shards: 2f + 1 (exactly, weight w such that 3w ≥ 2n + 1, that is, 67 of 100 shards).
  • It sends the aggregated signature and a bitmap of the signers to certify_blob. The contract verifies both on-chain. From then on the blob is certified and available.
  • With 4 nodes of 25 shards, blobs can be stored and read with one node down.

The Blob object​

Each stored blob has a Blob object on IOTA (<almacen>::blob::Blob), owned by your address. It records the blob ID, the size, the epochs when it was registered and certified, its Storage resource and whether it is deletable. As its owner you can:

OperationContract functionEffect
Renewsystem::extend_blobPays more epochs. The nodes see the new end epoch.
TransferIOTA object transferThe Blob has store, so it can go to any address or contract.
Deletesystem::delete_blobOnly deletable blobs. Returns the Storage to the owner; the nodes delete the data if no other Blob registers the same blob ID.
Burnblob::burnDestroys the object (meant for expired blobs; recovers its storage deposit). It emits no event, so the nodes keep the data until its end epoch.

A deletable blob can be deleted by its owner before it expires. A permanent blob cannot be deleted and stays available until its end epoch. Either way, deleting does not guarantee that no copy of the data remains elsewhere.

Storage​

A Storage resource is a reservation of encoded bytes from a start epoch to an end epoch, bought with reserve_space. Each Blob carries one. When you delete a deletable blob, its Storage comes back to your wallet as an object. The current clients do not reuse it yet: they always buy new space.

Quilts and patches​

A quilt stores many small files as a single blob, in the QuiltV1 format. Each file is a patch, with an identifier (for example its path) and optional tags. Since every blob pays a fixed amount of metadata, quilts make small files much cheaper.

  • With 100 shards a quilt holds up to 66 files, and the smallest quilt is 4,556 bytes.
  • A quilt patch ID identifies one file inside a quilt: base64url of the blob ID, a version and the start and end columns of the patch. The aggregator serves a patch by its quilt patch ID or by the quilt blob ID and the identifier.

Site​

A Site is an IOTA object of the sitios package that indexes a static website. It has a name, optional metadata (shown through Display) and one Resource per path, stored as a dynamic field, with:

  • the HTTP headers of the file;
  • the blob ID of the quilt that contains it, and the patch inside it (x-quilt-patch-internal-id header);
  • the SHA-256 of its content, which the portal checks before serving it.

A Site can also have routes and redirects. See Sites.

Treasury​

The Treasury is a shared object of the almacen::treasury module. It receives:

  • 3 % of the storage rewards of each epoch;
  • the commission of a slashed storage node.

Anyone can deposit into the treasury; only the holder of the TreasuryAdminCap (the project) can withdraw. See Economics.

Services​

  • Aggregator. An HTTP service that reads blobs from the storage nodes, decodes them and checks their blob ID. Public at https://aggregator.iotastorage.com.
  • Publisher. An HTTP service that encodes, pays and stores a blob on behalf of a client, and sends the Blob to the address the client chooses. The public one requires a token.
  • Portal. An HTTP service that serves Site objects as websites, at https://<site ID in base36>.iotastorage.com/ or through IOTA Names.

See the HTTP API.