fix(mining): prefer the first received block on equal work

[czarcas7ic]

Motivation

Equal-work forks currently select the greatest raw tip hash. If A arrives before B but B finishes verification first, mining should switch to A once it passes validation. A chain with more work still wins.

Solution

Record receipt order before asynchronous verification and preserve it through duplicates, forks, and reconsideration. Native sync serves retained bodies on the header-selected branch, so a node mining A can still serve retained side-fork B to a peer requesting that branch. A block redelivered after its earlier attempts finish gets a new receipt, while a body waiting in the state queue keeps its own. The orphan-priority policy remains deferred to #1127, making header selection agree on equal work is tracked in #1137, and the pre-existing same-height body-download gap is tracked separately in #1131. Fork eviction recovery and receipt metadata for secondary nodes are already on main. Restored forks use the hash fallback after restart. Upgrade secondaries alongside the primary when enabling the new policy. The version metadata also covers the breaking stream API inherited from main, with dependent patch bumps so published packages use matching network and state types.

Testing

All 690 state library tests pass, with four ignored. The serving regression fails on the previous implementation and covers different mining/download tips, missing selected bodies, greater-work switches, and an in-flight range retaining its original branch. All-target state Clippy, formatting, Markdown lint, and changelog checks pass. Earlier validation covered the verifier, router, header-chain, and conformance suites. Native E2E validation passes block-sync fuzz, cluster integration, and no-checkpoint-long. Its three header-lifecycle/reopen assertion failures also occur on the earlier main run. The trace confirms all 400 headers were admitted before body requests, but that assertion incorrectly requires a single response instead of the actual 250/150 batches. Used Codex for implementation and tests. The version correction passes locked dependency resolution, the crates.io publish-graph dry run, formatting, Markdown lint, and changelog validation. Fresh CI at 9ec0d6db4 passes every semver check, crate builds, all unit-test shards, configuration compatibility, lint, and integration tests. The optional Docker image check is still building.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-28T10:38:28.057383Z	67f4556	New commits

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Chef's kiss.

Reviewed commit: 28f86b0916

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[czarcas7ic]

Fixed the three issues introduced here, each in its own commit:

• F-283601: fixed in 7f1877c. Temporary failures keep receipt priority in a bounded retry cache.
• F-283602: fixed in b669c06. Empty legacy failover clears the old fork and keeps the finalized tip current.
• F-283603: fixed in b40a5ac. Mirrors publish complete snapshots and discard interrupted batches.
• F-283605: deferred. The ten-versus-eleven fork retention mismatch already exists on main and needs a separate fix to align header and full-state retention.

[evan-forbes]

Makes sense, this change has grown on me. I still don't like the inevitable increase in one block reorgs, but as mentioned in the body / by ValarDragon its worth the extra selfish mining protection.

I have two requests.

1. Make the header engine and full state agree on equal work. Right now header_best breaks ties by raw hash and verified_best by receipt order. A node can mine on A while it downloads and serves B's branch. Supporting that split takes the verified_tip override, InvalidVerifiedPreference, the header-branch serving path in block_range.rs, and the two-comparator spec text. It also defeats the policy when both headers arrive before either body: native sync downloads only the higher-hash body, so A's body may never reach the verifier (#1131).

Headers are different because a node can hold a header without its body. So the rule should be that the first body received wins, and having only a header never wins a tie. On equal work, the header engine would:

• prefer a tip whose body full state has verified over a header-only tip, and
• between two verified tips, take full state's receipt-order choice.

The raw hash would then break ties only between header-only tips. Selection stays a deterministic function of the graph plus body-verification evidence, and the engine already consumes that evidence. header_best and verified_best then agree on every tie where a body exists. Native sync downloads and serves the mining branch, so the serving change should no longer be needed.

2. Drop the retry receipt cache (RetryReceipts, allow_retry, retains_error, and the body digest). Without it, a block that fails before it reaches the state queue gets a fresh receipt when someone redelivers it.

• Most of the audit churn comes from the cache. Of the 27 V12 findings across runs 8107–8207, 14 concern retry-receipt retention: F-283675, F-283679, F-283740, F-283741, F-283742, F-283743, F-283789, F-283794, F-283795, F-284871, F-284872, F-284875, F-284876, and F-284877. Each fix added another classification rule or bound: the error allowlist, the Merkle-before-retry reorder, the per-header variant cap, the TTL, and the pending-duplicate exception. Run 8207 still lists F-284875 (a resubmission renews the TTL forever) and F-284877 (the global lock hashes 2 MB bodies) against the current design.
• The cache decides #1127 early. It exists mainly so that a block which fails because its parent is missing keeps its priority. That is the orphan-priority question the PR defers to #1127. The design doc also notes that it lets a miner reserve priority for a tip while withholding an ancestor. zcashd assigns priority only once the block and its ancestors are available, so a fresh receipt on redelivery is closer to zcashd, not further from it.
• Without the cache, the common paths still work. Overlapping deliveries still share one receipt through the active registry. A block waiting for its parent in the state queue still keeps its receipt on the prepared block. Transient local failures (StateService, Io, QueueFull) are rare, and losing a tie there costs little.

Removing the cache deletes most of receipt.rs, the retry tests in block/tests.rs, and the three params.md rows. It also closes every open retry finding at once. The queue behavior from p0mvn's thread belongs in #1130, as you said.

[czarcas7ic]

Will drop the retry cache here; for (1), serving would still need to change because mismatched bodies are penalized (

zakura/crates/zakura-network/src/zakura/block_sync/peer_routine.rs

Lines 1592 to 1602 in 1b8b57c

	match outstanding.request.expected_hash(height) {
	Some(requested) if requested != hash => {
	let mismatch = header_hash_payload_mismatch(
	outstanding.request.owner,
	self.source,
	requested,
	hash,
	);
	self.report_misbehavior(BlockSyncMisbehavior::BodyPayloadMismatch(mismatch))
	.await;
	return;

), so can we make it a follow-up that lands before 9.0.0 along with #1131?

[czarcas7ic]

Retry cache removed in 86cc137, and header selection agreement is tracked in #1137.

[evan-forbes]

LGTM, thanks for dropping the retry cache, and you're right about serving on (1).

[evan-forbes]

trying this out atm on the 80 node cluster