Skip to content

fix(sync): authenticate heights and exclude rejected block suppliers - #1036

Open
evan-forbes wants to merge 21 commits into
mainfrom
security/validated-parent-height
Open

evan-forbes wants to merge 21 commits into
mainfrom
security/validated-parent-height

Conversation

@evan-forbes

@evan-forbes evan-forbes commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

A V5+ body can change its coinbase height without changing its header hash, because ZIP 244 leaves the coinbase input out of the transaction ID. Legacy sync and gossip read that height before full validation. A peer can set it to 1 or to a far-ahead value, and the behind-tip or lookahead policy then drops the requested hash: unscored and not requeued. A V4 body works too, because the downloader drops the block before anything checks its Merkle root.

V12 finding #297531 reported this on main. Zebra fixed the behind-tip half in v6.3.0 (GHSA-g95h-hw6g-pvgv) with a parent-height check. The lookahead half is still open upstream.

Solution

  • Authenticate the claimed height without chain state. The header commits to every transaction ID through the Merkle root. A V1–V4 transaction ID commits to the coinbase input. A V5+ transaction ID commits to the expiry height, which must equal the block height from NU5 (ZIP 203). So a body whose Merkle root matches, and whose V5+ coinbase expiry equals its input height, carries the height its header commits to.
  • Run the expiry comparison on every downloaded block. It costs nothing.
  • In the syncer, hash the transactions only when the claimed height would make the policy drop or pause the block, or would clear an active lookahead pause (V12 #297532). The verifiers already check the Merkle root of every block they accept, so an honest block in the download window pays for that hashing once, as before.
  • In gossip, hash every body. Gossip carries about one block per target spacing, and rejecting an in-window rewrite here lets it be re-requested.
  • Attach local feedback that identifies the actual TCP or v2 supplier. Exclude rejected hash/supplier pairs before retrying. Keep feedback off the wire.
  • Bound supplier exclusions and retry budgets. Preserve unrelated downloads when a rejected-body retry budget expires.

The earlier revision checked the claimed height against the committed parent. That needed one state read per downloaded block, and it could not check blocks whose parent had not arrived yet: during checkpoint sync, that is most of them. This revision drops the parent lookup and the AnyChainHeight state request, so zakura-state no longer changes.

Testing

  • auth_download_height: canonical V4 and V5 heights are bound. A V5 input-height rewrite fails the free expiry check. A V5 input-and-expiry rewrite and a V4 input-height rewrite fail the Merkle check.
  • unknown_parent_rejects_forged_heights: a forged low or high height on a block whose parent is unknown is rejected and attributed to its supplier.
  • forged_reset_height_keeps_the_lookahead_pause: while the pause is active, a forged in-window height is rejected and leaves the pause set. Without the fix, the body reaches the verifier.
  • gossiped_non_tip_child_rejects_poisoned_coinbase_height covers the same cases on the gossip path. inbound_poisoned_coinbase_height_scores_and_requeues still passes for an in-window V1 rewrite, because gossip checks the Merkle root of every body.
  • The existing tip-child, retry-budget, and behind-tip tests pass unchanged apart from the renamed error.
  • Rust 1.98.0: zakura library 454 passed, zakura-network library 1514 passed (3 ignored). Strict Clippy for zakura, zakura-network, and zakura-state with all targets, cargo fmt, and ./scripts/changelog.py check pass.

Changelog

Updates the changelog fragment and the parameter-ledger entries.

Follow-up

#1038 and #1039 are stacked on this branch and still build on the parent lookup. They need a merge from this branch before review.

Provenance

Ported from the private security review, then reworked for the stateless check.

@evan-forbes
evan-forbes marked this pull request as ready for review September 17, 2026 22:27
@evan-forbes
evan-forbes requested a review from a team September 17, 2026 22:27
@v12-auditor

v12-auditor Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

Note

Complete: Audit complete. V12 found 11 issues worth reviewing.

Open the full results here.

FindingSeverityDetails
F-281608 🟡 Medium
Missing-height bodies evade supplier exclusion

A supplier can answer a BlocksByHash request with the exact requested header and a malformed body whose transaction list is empty or does not begin with a coinbase transaction. The block identifier is derived from the header, so the response still satisfies the requested hash while coinbase_height() returns None. When the parent height is unavailable or the local lookup fails, parent_height_mismatch returns neutral, after which the changed code constructs InvalidHeight with advertiser_addr: None. It also discards the attached BlockFeedback without calling reject(). Downstream handling therefore neither scores nor excludes the supplier and treats the error as a non-restarting drop without immediately requeueing the required hash.

F-281611 🟠 High
Retry saturation drops required blocks

Untrusted peers can fill poisoned_block_retry_counts without supplying valid proof-of-work blocks because legacy FindBlocks accepts distinct unknown hashes before their headers or ancestry are authenticated. For each filler hash, a peer can provide an arbitrary header pointing to an old committed parent and a wrong-height V5+ body, causing ParentHeightMismatch before consensus verification and inserting the hash into the retry table. A colluding peer can then return the same header hash with the corrected old height, which is dropped as BehindTipHeightLimit; that non-fatal completion frees the in-flight slot but does not remove the retry-table entry. Repeating the two-response sequence fills all 4,096 entries. Once full, a mismatch for a real required hash returns Ok(()) before insertion or requeue, even though dispatch already removed the hash from the reserve and task completion removed it from in-flight state.

F-281612 🟠 High
Consensus-invalid suppliers remain eligible

Single-hash TCP and Zakura responses are wrapped with BlockFeedback, but both downloaders call reject() only for the new parent-height mismatch. A supplier can instead provide a hash-matching body with the correct claimed height whose V5 authorizing data or other body contents fail full consensus verification. The verifier then proves the supplied body invalid, yet the error mapping discards the feedback without recording the (hash, supplier) pair. This is especially consequential for Zakura responses, which carry no TCP address and rely on the feedback capability for exact supplier identity. Subsequent requests therefore keep the malicious Zakura peer eligible for the same hash.

F-281613 🟡 Medium
In-flight retries bypass supplier rejection

ZakuraRequestClient::request snapshots ready handles and filters rejected suppliers only once before its first request. If that request returns missing inventory or fails, fallback selection reuses the original handle vector. The source-missing retry loop similarly calls one saved handle repeatedly after sleeps. A concurrent download can reject the same (hash, peer_id) while either path is in flight, but neither path rechecks the shared registry before the next actual send. Requests can therefore be dispatched to a supplier after its body for that hash was already proven invalid.

F-281614 🟠 High
Duplicate responses retain malicious suppliers

The Zakura response decoder accepts each available block independently when its hash belongs to the requested set, but it does not enforce uniqueness. For a single-hash request, a malicious peer can return the same requested block twice. request_one attaches feedback for the selected peer, but the sync consumer rejects the two-entry vector as a generic DownloadFailed without calling reject(). The transient retry path immediately requeues the hash, while the supplier registry still considers the attacker eligible. The same peer can repeat the duplicate response on every retry and subsequent sync round.

F-281615 🟡 Medium
V2 poisoners remain eligible across hashes

During legacy recovery on a dual-stack node, an authenticated Zakura peer can answer each requested V5+ hash with a same-header body whose coinbase height is rewritten. Zakura decoding stores no socket address in the block entry and preserves the authenticated peer only inside BlockFeedback. The proven mismatch rejects only the exact (hash, peer_id) pair, while the resulting error has no legacy address and emits no peer-wide punishment. An honest alternate peer can satisfy the retry for that hash, but source-less requests select the stable first ready handle again for the next hash. The malicious authenticated peer can therefore poison every successive block while remaining connected and eligible across hashes.

F-281616 🟡 Medium
Global cache churn evicts live exclusions

All rejected (hash, supplier) pairs share one 4,096-entry IndexMap. When full, each unique new rejection unconditionally removes insertion index zero even if that exclusion is still within its 16-minute lifetime, then inserts the attacker-controlled pair. Re-rejecting an existing pair updates its timestamp without moving it to the back, so a freshly renewed but old-index entry is still the next one evicted. A supplier or coalition can therefore churn unique poisoned pairs until an unrelated live exclusion disappears. TCP and Zakura routing both interpret the missing entry as renewed eligibility, and the Zakura compatibility path has no socket-address score as a secondary defense.

F-281618 🟡 Medium
Stale mismatches amplify useless downloads

The committed-parent mismatch check executes before determining whether the authenticated child height is within the node's accepted reorg window. An attacker can create distinct alleged children of any retained historical parent and supply wrong-height or missing-height bodies. The mismatch is proven before proof of work or other consensus verification and is classified as a recoverable poisoned body. Sync then requests each irrelevant hash up to eight more times, and inbound up to three more times. If the authenticated expected height had first been compared with min_accepted_height, all of this work would have been discarded immediately as permanently stale.

F-281619 🟠 High
Retry exhaustion abandons required hashes

After a hash consumes all eight sync poisoned-body retries, handle_block_response_with_missing_retry returns Ok(()) without preserving or propagating the terminal ParentHeightMismatch. The completed downloader task has already removed its queue entry, and the handler neither requeues the hash nor records a deferred retry. This makes the separate should_restart_sync arm for an exhausted mismatch unreachable. The sync loop treats the swallowed completion as progress and can declare the round exhausted or leave descendants parked. A Sybil set can repeat the same attack after the next round clears the exhausted budget.

F-281620 🔵 Low
Idle inbound service delays poison recovery

Inbound scoring and poisoned-body recovery run only when Inbound::poll_ready drains the download stream. The advertisement call queues the asynchronous download after its preceding readiness poll and immediately returns. If the poisoned response completes and no later inbound request causes another readiness poll, the ready mismatch remains unconsumed. retry_poisoned is therefore not called, and peer scoring is delayed. Honest advertisements that arrived while the hash occupied the dedupe slot were already refused, so recovery depends on unrelated future traffic or another sync subsystem.

F-281621 🟡 Medium
Tip-scoped retries mishandle historical poisoned blocks

PoisonedRetryBudgets is keyed by block hash but globally scoped to the mutable best-tip hash, although parent_height_mismatch can authenticate an alleged child of any committed parent. Distinct historical hashes with malformed claimed heights can therefore occupy the current tip epoch's shared table; entries remain after failed downloads because only a successful completion releases them, eventually preventing a replacement download for a real current-tip hash. Conversely, every best-tip change clears the per-hash counts and re-arms the same historical hash, even when its body remains unavailable. Pruning can preserve historical height evidence without retaining the block body, while an otherwise honest historical body is rejected by the behind-tip policy, so these entries do not reliably reach the success path that releases them. Automatic poisoned-body retries should be restricted and accounted for according to whether the hash is a live current-tip recovery candidate, preventing irrelevant historical mismatches from either consuming or regaining that recovery budget.

And two more auto-invalidated findings.

Analyzed 16 files, diff cc2b5b8...3ed1502.

The audit of #1036 reported two ways a supplier escapes the exclusion this pull
request introduces.

A body whose transaction list is empty or does not start with a coinbase
transaction has no coinbase height. The block identifier comes from the header,
so such a response still satisfies the requested hash, but the download failed
with `InvalidHeight` and discarded the attached feedback, leaving the supplier
eligible for the same hash on every retry. It is now rejected. The advertiser
stays unattributed, because it did not choose the body.

The Zakura response decoder checked only that each delivered hash was requested,
not that it was delivered once. A peer could answer a single-hash request twice;
the consumer saw a response that did not match its request and reported a
generic download failure, which retried without excluding anyone. The decoder
now accepts one answer per requested hash, and a decode failure rejects the
supplier that produced it.
evan-forbes added a commit that referenced this pull request Sep 18, 2026
evan-forbes added a commit that referenced this pull request Sep 18, 2026
The arm claimed it is only reached once the requeue budget is exhausted. The
retry handler returns success on exhaustion so the round keeps its other
downloads, so the arm is unreachable from there. The round's stall deadline is
what recovers an exhausted hash.
evan-forbes added a commit that referenced this pull request Sep 18, 2026
evan-forbes added a commit that referenced this pull request Sep 18, 2026
Main's #940 landed an overlapping narrower check: `tip_child_mismatch`
authenticates a claimed height only when the block's parent is the current tip.
This branch's `parent_height_mismatch` authenticates against any committed
parent through the new `AnyChainHeight` query, so its `ParentHeightMismatch`
replaces `TipChildHeightMismatch` and keeps main's parameterised retry test
covering the transaction-commitment and merkle-root kinds #940 added.
evan-forbes added a commit that referenced this pull request Sep 18, 2026
…tment branch

Main's #940 added `forged_expiry_with_authentic_height_is_a_payload_mismatch`,
which caught a real regression in this branch: requesting the parent's context
before the body's Merkle root meant an unavailable parent reported
`MissingParentContext` and masked a malleated body. That classifies a permanently
invalid body retryable and leaves its supplier unscored against a header the node
keeps re-requesting. Every Merkle-root outcome is decided from the header's
commitment and the delivered list, so the check now runs before the
pending-parent error is reported.

`is_poisoned_body` from #940 keeps its shape but widens to every body-commitment
kind, because the early commitment check proves an authorizing-data mismatch that
an alternate body for the same header can repair. Two test mocks answer the
requests the other side of the merge introduced.
Replace the per-block parent-height lookup with a stateless check. The
header commits to every transaction ID through the Merkle root, a V1-V4
transaction ID commits to the coinbase input height, and a V5+ coinbase
expiry height must equal the block height (ZIP 203). So a body whose
Merkle root matches, and whose V5+ expiry equals its input height,
carries the height its header commits to.

The parent lookup cost one state read per downloaded block and could not
check blocks whose parent had not arrived yet, which is most blocks
during checkpoint sync. That left the lookahead drop open (V12 #297531).

The syncer runs the free expiry check on every block and hashes the
transactions only when the claimed height would drop or pause the
block; the verifiers already check the Merkle root of every accepted
block. Gossip checks every body, because it carries about one block per
target spacing and a rejection there can be re-requested.

Drop the AnyChainHeight state request, so zakura-state is unchanged.
evan-forbes added a commit that referenced this pull request Oct 3, 2026
…nt branch

#1036 now authenticates coinbase heights without chain state. Main added
NSM and ZIP 234 verifier tests whose state mocks did not answer this
branch's parent context request; they now get a bound stand-in context.
The checkpoint handoff router test uses a V4 coinbase, so its height-gap
child still reaches state instead of failing at the parent context check.
A body at or below the lookahead reset height clears the pause flag,
but heights inside the window skipped the Merkle check. A forged
in-window height could therefore release backpressure the syncer still
needed. Run the check for that case too; it costs one txid pass per
pause cycle, because the first honest block below the reset height
clears the flag.

Fixes V12 #297532.
@czarcas7ic
czarcas7ic dismissed a stale review October 20, 2026 02:59

Reposting as comment-only feedback.

Comment thread crates/zakurad/src/components/sync.rs Outdated
Comment on lines +2698 to +2701
if !self.poisoned_block_retry_counts.contains_key(&hash)
&& self.poisoned_block_retry_counts.len() >= MAX_POISONED_BLOCK_RETRY_HASHES
{
return Ok(());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI comment


I reproduced this with both a full retry table and an exhausted per-block budget. With mocked services and a 200 ms stall timeout, unrelated tip extensions kept the round alive for five deadlines without recovering the required block. Restarting on either limit made the tests pass. Can we keep a bounded recovery path for the missing hash, or restart when recovery can't continue?

Comment thread crates/zakurad/src/components/sync/downloads.rs
Comment on lines +721 to +723
let Some(block_height) = block.coinbase_height() else {
if let Some(feedback) = &supplier_feedback { feedback.reject(); }
return Err(BlockDownloadVerifyError::InvalidHeight { hash, advertiser_addr: None });

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI comment


If a supplier returns the requested header without a coinbase, we exclude it here but drop the required hash through InvalidHeight. Gossip also loses both exclusion and recovery on this case. Can we route missing-height bodies through the bounded poisoned-body recovery path in both downloaders?

Comment on lines +213 to +215
if self.counts.len() >= MAX_INBOUND_CONCURRENCY && !self.counts.contains_key(&hash) {
return None;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI comment


In a downloader test with mocked services, the next block recovered after 199 unrelated failed downloads but got no retry after 200, despite an empty queue and an unused per-block budget. The junk arrived sequentially from one source and never reached verification. One extra table slot restored recovery. Could we keep unverified junk from using up recovery capacity for the next block?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants