Storage & Serving

What this layer does. It keeps contributed content retrievable and verifiable. Payloads are required to be encrypted before they arrive and nothing here does it for them: bytes are stored as handed over, and integrity comes from the on-chain hash rather than from any transformation this layer applies. The metadata that describes a payload is public by design — that is deliberate, and the distinction between it and the payload is the most useful thing on this page.

Metadata, manifest and data are different artifacts

For a dataset version, the chain points at public metadata. That document references an encrypted chunk manifest and a separate public CF list. The manifest lists the data chunks; its hash does not directly hash all those chunks. The diagram describes the dataset document format; production dataset-package creation and authorised key delivery are not running yet.

The on-chain metadataHash covers the metadata document. Its manifestHash verifies the encrypted manifest's exact bytes; cfListHash independently verifies the CF-list bytes. contributorsMerkleRoot is a different commitment, used to verify ownership claims. Verifying metadata and manifest does not by itself verify the data chunks: those must be checked against the manifest's entries. These are integrity checks, not confidentiality guarantees. A dataset version's document must declare an encryptionSuite, but declaring it does not prove encryption happened; the CF document has no encryption field.

Each reference separates identity from retrieval: the hash fixes the content, the URI locates it. A replacement retrieval endpoint must return the same committed bytes. This does not imply that every registry permits its anchored URI to be edited.

Where it is stored

The protocol does not mandate a backend. What it anchors is a hash and a URI, and several schemes resolve: an owner publishes through Arweave, Filecoin, IPFS, Google Cloud Storage, or a hybrid of them, and the reader follows whatever the URI says.

Of those, two are worth describing because their guarantees differ:

• Arweave takes uploads as ANS-104 data items signed with a key held in a cloud KMS, never as a plaintext key file.
• Filecoin is the planned cold archive for encrypted dataset packages, staged in private GCS and archived through Synapse. It does not store individual CF metadata or payloads. No production service currently creates or archives these packages; complete archival verification is a release condition, not a running guarantee.

Google Cloud Storage is a primary custodian too, not a cache — in a hybrid configuration a payload's only copy can be a world-readable GCS object, with integrity coming from the on-chain hash rather than from the transport. That is why the requirement below sits on the producer and matters: a payload uploaded in the clear is a payload published.

Arweave uploads have a per-caller daily ceiling. The Filecoin quota field is reserved, not enforced, and the former Filecoin upload route has been removed. Public artifact URLs need no credentials; private origins and authenticated gateway routes have their own access boundaries.

What is encrypted, and what is not

The payload is required to arrive as ciphertext, and nothing in this layer produces or checks it. The storage design is explicit that every tier sees only ciphertext for contribution payloads. But that is a requirement on producers, not a property this layer establishes: no service here encrypts, decrypts, inspects, or refuses a payload, so a producer that ignores it is not stopped — and the published CF document schema does not express the requirement either, describing an inline payload as raw bytes and defining no encryption field.

Encryption is designed to live one layer up, at the dataset version — one key per version, wrapped per grant, held custodially — and that model is not implemented; see Access Control for its state and for the proxy-re-encryption design that was rejected rather than deferred. A legacy Python SDK does carry AES-256-GCM, but it is on no deployed path.

The metadata document and the CF list are public, deliberately. They live in buckets readable by anyone: that is how the indexer enriches records and how a reader verifies a dataset without asking permission. They contain references, roots and counts — not contributed content.

So the precise statement is about references, and only that: a payload reference being visible tells you nothing about whether you can read the payload. Hashes, URIs and structure are public. Whether the bytes behind them are readable is not this layer's guarantee to give.

Interfaces

• In: payload bytes, required to be ciphertext and taken on trust, plus the metadata document and CF list that describe it, referenced from a Contribution Fingerprint or a dataset version from Data Assembly.
• Out: stored bytes without transformation; metadata is public, while retrieval of a payload depends on its storage location and access configuration.
• Cross-links: key release is designed to come from Access Control and runs nowhere yet; provenance runs back through the CFs.

Invariants

• Verify the hash for the artifact it names. Metadata, encrypted manifest and CF list have separate commitments; a hash mismatch means the bytes are not the committed artifact.
• No service here encrypts, decrypts, or checks whether a payload was encrypted. It stores the bytes it is given. Ciphertext is required of the producer and unverified here, so this layer can neither add to a payload's confidentiality nor report on it.
• Cold-ready is a planned publication gate. The dataset archive path must verify complete storage before anchoring; this path is not serving production uploads.
• Public retrieval is not access control. A plaintext object published at a public URL is readable by anyone holding it. Private-origin access restrictions are separate from the planned grant-based key delivery.

Not here yet

Confidential and distributed compute — running work against this data without any party seeing it in the clear — is a direction rather than a capability. See Future Directions.