Upgrades

This page explains how the Interfold versions ciphernode releases and how to install a new release. It also explains how the on-chain release gate decides which nodes can serve on committees. It is for operators who run a ciphernode. At the end, you can tell which kind of release you have and install it. You can also reset the node state when a release changes the storage format.

Release versions

Each release has three values. Only the two compatibility values decide eligibility.

The release ID is keccak256("interfold.node.release:v1:" + version). The node records it on-chain, but the ID does not decide eligibility.

Recent releases

Nodes on releases v0.15.1 through v0.18.0 share protocol_version 4 and the same P2P wire versions, so they can connect to each other. A node on generation 1 can connect, but it cannot be active when the chain requires generation 2. The storage schema changes at v0.18.0, so a node that ran v0.15.1 through v0.17.0 must reset its state before v0.18.0 can start.

The v0.19 DKG message format requires storage schema 8 and gossip/sync wire versions 5 and 4. It uses protocol version 8 and node generation 2. Complete active E3 work before the storage reset. The signed DKG messages cannot reuse schema 7 state.

The bb 6.0.0 toolchain uses protocol version 9. Its proofs and verification keys do not verify against bb 5.2.0 nodes or verifiers, so upgrade every node together during the paused cutover, after active E3 work completes. No storage reset is needed.

The on-chain release gate

The Interfold contract points to a node release registry (INodeReleaseRegistry). The registry stores the required values in requiredProtocolVersion() and requiredNodeGeneration(). An operator is release-ready (isNodeReleaseReady) when both of these conditions are true:

• The acknowledged protocol_version is equal to the required value.
• The acknowledged node_generation is equal to or larger than the required value.

A release with a higher protocol_version than the required value is also not ready. The BondingRegistry active check includes release readiness, so a node that is not release-ready cannot enter new committees. See When a ciphernode is active.

Startup check

Before the node starts its protocol actors, it checks each enabled chain:

1. It reads nodeReleaseRegistry(), bondingRegistry(), and ciphernodeRegistry() from the Interfold contract. The configured addresses must match.
2. It compares its own compatibility values with the required values.
3. If the operator already acknowledged this release, and the operator is active or not registered, the node logs Ciphernode release is accepted.
4. Otherwise, the node logs Acknowledging ciphernode release and sends acknowledgeNodeRelease(releaseId, protocolVersion, nodeGeneration) from the operator key. The registry emits OperatorNodeReleaseAcknowledged and refreshes the operator status.

If a read in step 3 or the transaction in step 4 fails, the node runs steps 3 and 4 again, up to three attempts, after about 2 s and then 4 s. Each attempt runs the check of step 3 again, so after a mined acknowledgment it sends nothing more when the operator is active or not registered.

The acknowledgment is a transaction, so the operator wallet needs ETH for gas. These errors stop the startup:

The release acknowledgment is a statement by the operator. It prevents the accidental use of old software. It does not prove which executable runs. Threshold cryptography, proof verification, and slashing still protect the protocol against a malicious operator.

Check the required values

Read the values with cast. Get the Interfold address from Contracts and addresses:

cast call <Interfold> "nodeReleaseRegistry()(address)" --rpc-url <RPC_URL>
cast call <NodeReleaseRegistry> "requiredProtocolVersion()(uint32)" --rpc-url <RPC_URL>
cast call <NodeReleaseRegistry> "requiredNodeGeneration()(uint32)" --rpc-url <RPC_URL>
cast call <NodeReleaseRegistry> "operatorNodeRelease(address)((bytes32,uint32,uint32))" \
  0xOPERATOR --rpc-url <RPC_URL>

P2P compatibility

Every P2P protocol name includes the network ID and the protocol_version. The Kademlia, sync, gossip, and Identify names all use the prefix interfold/<network_id>/protocol/<protocol_version>. Nodes with different protocol versions therefore do not connect to each other.

The node_generation is not part of these names. A node-only cutover removes old nodes from new committees, but old and new nodes stay on one P2P network. A bug that changes message parsing, message meaning, or peer safety therefore needs a new protocol_version.

Release types

Compatible release

A compatible release keeps both values. Old and new nodes stay eligible, so you can install the release when you want. If you run several nodes, restart one node at a time. Keep enough committee members online so that each active committee keeps its threshold.

A release that changes the sortition seed or the ticket score that nodes compute is an exception. While old and new nodes run together, they select different nodes to submit tickets. Fewer nodes than the committee size can then submit, and the E3 fails with InsufficientCommitteeMembers. No honest node is slashed for this. For such a release, governance pauses new E3 requests until the operators have upgraded, then resumes them.

Mandatory release

Governance uses a cutover for a mandatory node release or a protocol release:

1. PauseGovernance pauses new E3 requests.
3. DrainActive E3s and committees reach a terminal state.
5. RaiseGovernance raises the required version or generation.
7. RestartOperators start the new release. Each node acknowledges it.
9. ResumeGovernance resumes requests when enough nodes are active.

setRequiredNodeRelease reverts with NodeReleasePolicyRequiresPause while requests are open. It reverts with NodeReleasePolicyInUse while an E3 is active or a committee is not released. A new requirement makes every operator inactive at once. Each upgraded node acknowledges its release at startup and becomes active again.

Before governance resumes requests, it refreshes every registered operator and checks that committeeOwnerCapacity covers the largest committee. For a protocol release, operators upgrade at least one configured bootstrap peer first, so that new nodes can find each other.

Upgrade a ciphernode

Use this procedure for every release. The release notes tell you if a release changes the storage schema or the configuration.

curl -fsSL https://raw.githubusercontent.com/theinterfold/interfold/main/install | bash -s -- --version v0.18.0

If the startup fails with a release error, leave the node stopped. Install the release that the chain requires, or wait for the protocol upgrade to complete.

For Docker, run the interfold commands of this procedure in a one-off container from the new image, through the image entrypoint. Mount the same volumes, as in Validate the state. Do not use --entrypoint interfold.

Upgrade a node that serves an active E3

Use this procedure when the new release keeps the storage schema of the running release. The node keeps its state, so it keeps its key share for each active E3. v0.19.0 raises the schema, so this procedure does not apply to an upgrade to v0.19.0. Upgrade a node to v0.19.0 only when it serves no active E3. Then reset its state as Reset node state describes.

1. Stop the node gracefully, and wait until the process exits. Do not send SIGKILL. For Docker, run docker stop --time 65 <container>.
2. Back up the complete data directory and the complete configuration directory. interfold config get db_file shows a path in the data directory. interfold config get key_file shows a path in the configuration directory. For Docker, back up the complete volume. A container from an image earlier than v0.19.0 keeps its state inside the container, not on the volume. For such a container, see Move state from an earlier image.
3. Run interfold node validate with the new binary. The report must show on-disk schema version <n> matches this binary, where <n> is the schema of the running release. It must end with VALIDATION PASSED — state is intact and consistent. If one of these lines is missing, do not start the new binary. The line empty store will be initialized means that the command found no state at this location.
4. Start the new binary. Make sure that the log shows Ciphernode release is accepted.

To roll back, stop the node, restore both directories from the backup, and start the previous binary.

Never run interfold node reset-data or interfold nodes purge on a member of a committee with an active E3. Both commands delete its key shares, and the chain cannot restore them.

v0.17.0 uses storage schema 6, v0.18.0 uses schema 7, and v0.19.0 uses schema 8. A binary halts on state of another schema, so every node resets its state when it upgrades to v0.19.0. Keep each node on its current release until its E3s are complete or failed, and their report deadlines are past. If the command refuses shows the checks.

Reset node state

The storage schema is a marker in the key/value store of the node. The binary loads existing state only when the marker is equal to its own schema version. It refuses older state and newer state, because the stored event log and actor snapshots contain protocol data in a binary format. These messages stop the startup:

The operator key and the libp2p keypair live in the same key/value store as the protocol state. A manual delete of the data directory therefore also deletes the identity that holds the bond. interfold node reset-data removes the state and keeps the identity. See the CLI reference.

reset-data refuses when the node holds key-share state for an E3 that it has not seen complete. It also refuses for an E3 that shows Failed, because the node records its own local failures that way while the E3 can continue on chain. The refusal lists each E3 and its stage, and the command deletes nothing. It also refuses while the node holds slash reports that it has not submitted, also for an E3 that is complete, because the chain cannot restore their evidence. That refusal lists each chain and the number of its reports. --allow-active-e3s overrides both refusals.

A container from a CLI image earlier than v0.19.0 keeps its state inside the container, not on the volume. For such a container, follow Move state from an earlier image. It copies the state to the volume, and then it resets the state.

The DAppNode package stores its state on the /data volume. Keep that volume. On a release with interfold node reset-data --if-schema-older, the package runs that command before each start, so an update resets an older state by itself. A refusal stops the package with the same lists. Do the checks in If the command refuses. For a package, the release that the node ran before is the previous package version. Set ALLOW_ACTIVE_E3S_RESET to true in the package Config only where those checks say to add --allow-active-e3s, and set it back to false after the start. On v0.19.0, use the one-off container commands below instead of the CLI image migration procedure.

interfold node reset-data --name <node> --yes

docker run --rm --entrypoint interfold <new-dappnode-image> --version

docker run --rm --volumes-from <stopped-dappnode-container> --workdir /data \
  --entrypoint interfold <new-dappnode-image> wallet get --config /data/config.yaml
docker run --rm --volumes-from <stopped-dappnode-container> --workdir /data \
  --entrypoint interfold <new-dappnode-image> net get-peer-id --config /data/config.yaml

docker run --rm --volumes-from <stopped-dappnode-container> --workdir /data \
  --entrypoint interfold <new-dappnode-image> \
  node reset-data --name _default --yes --config /data/config.yaml

The operator key and libp2p identity were preserved. Verify with: interfold wallet get --name <node>

cast call <Interfold> "getE3Stage(uint256)(uint8)" <e3_id> --rpc-url <RPC_URL>
cast call <Interfold> "getE3LifecycleDeadline(uint256)(uint256)" <e3_id> --rpc-url <RPC_URL>
cast block latest --field timestamp --rpc-url <RPC_URL>

Use the same --name as the node. Without --name, the CLI uses the _default node profile. For a node that runs under a named profile, the command then works on the _default profile instead, and interfold wallet get reports No wallet has been set.

The backup file is identity-backup-<node>-<unix-time>.json, next to the db directory. The command keeps this file after a successful reset. It contains the encrypted operator key, so keep it as safe as the data directory, or delete it after the node runs correctly.

The full sync reads the contract logs from each deploy_block, so the RPC endpoint must serve the complete log history. See RPC endpoints. The node exits when the startup takes longer than node.startup_timeout_secs (default 1800 s).

Do not use these commands for a reset

• Do not delete the data directory by hand. The operator key and the libp2p keypair are in it.
• Do not use interfold nodes purge or interfold purge-all. Both delete .interfold/data and .interfold/config in the current directory, which includes the key file.
• Do not change the stored schema marker.
• Do not clear only the key/value store. The node then halts with On-disk state has no schema marker.

interfold node validate --repair is a different tool. It can repair an event-log tail that was provably not committed, and rebuild a stale derived sortition projection. It does not repair a corrupt indexed record or an unsupported schema.

Startup and validation check the storage schema before they read or repair event logs. An older schema reports the schema mismatch and reset command, even when its event format cannot decode. Validation skips the event and snapshot checks for that store, including with --repair.

Configuration for protocol version 4

Each enabled chain that configures the interfold contract needs a data_availability block. A configuration from interfold ciphernode setup of an earlier release does not have this block, so add it before the first start. The node reads proof-backed ciphertext outputs from the data-availability layer. Without the block, the node exits after it accepts the release:

chain 'mainnet' has Interfold enabled but no data_availability reader; this protocol release must retrieve proof-backed ciphertext outputs

Add the block inside the chain entry:

chains:
  - name: mainnet
    chain_id: 1
    data_availability:
      mode: avail
      rpc_url: 'https://avail-rpc.publicnode.com/'

The DAppNode package uses https://avail-rpc.publicnode.com/ for chain 1 and https://turing-rpc.avail.so/rpc for Sepolia. The mock_http mode is for local tests. Make the change while the node is stopped. See Data availability for the keys.

Next steps