Repository navigation
Look up workers after dial failures; vendor patched libp2p-stream - #217
Merged
Merged
Conversation
mo4islona
force-pushed
the
vendor-libp2p-stream
branch
2 times, most recently
from
October 15, 2026 10:43
21c35c1 to
3231c67
Compare
The reconnect path fires only on ConnectionClosed, and at that point the address cache still holds the dead address, so it is dialed instead of looked up. That dial fails and evicts the entry, and nothing re-arms: a failed dial establishes no connection, so no close event follows, and the whitelist sweep is suppressed while the registered set is unchanged. The worker stayed undialable until unrelated DHT traffic happened to supply an address. NoAddresses retries too, after the same cooldown. A lookup returns only peers that answered, so it cannot find a worker that is still offline, and the dial after it has no address; stopping there left the worker unreachable after it came back. A worker waits for one reconnect at a time, across the cooldown and the lookup queue, so failures while the lookup cap is reached no longer queue it again. Tests that read or set PRIVATE_NETWORK now hold a lock: addr_is_reachable reads it on every call and a binary's tests run as threads in one process. Part of NET-384 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The kalabukdima/rust-libp2p fork patched nothing but protocols/stream: an unbounded dial channel and Control::accept_with_capacity, both fixing streams that upstream silently drops under concurrency. Carrying them as a fork of the whole workspace also forced a [patch.crates-io] for libp2p-identity on every downstream crate. Copy the one patched crate into the transport crate instead and take the rest of libp2p from crates.io at the version the fork is based on. - Add crates/transport/src/libp2p_stream: upstream rust-libp2p aefbfbdc (MIT) plus the two patches, and upstream's tests for them; unchanged apart from module paths, rand::rng() and this repo's rustfmt - Point libp2p and friends at crates.io 0.56.0, drop the identity patch - Drop the now-unreachable OpenStreamError arm and derive ClientBehaviour's Default: #[non_exhaustive] and clippy treat the types as local now The fork branched 26 commits past the v0.56.0 release, so the released crates are not identical to its tree: libp2p-core 0.43.1 -> 0.43.2, libp2p-swarm 0.47.0 -> 0.47.1, libp2p-quic 0.13.0 -> 0.13.1, and libp2p-kad and libp2p-gossipsub back to the released 0.48.0 and 0.49.5. That picks up the fixes for GHSA-5hq8-qhww-jm7q (quic panicking on certificate validation failure) and for three gossipsub advisories published after the fork's base, at the cost of the unreleased gossipsub 0.50.0 changes the fork carried. The one behavioural delta that reaches us: pubsub builds gossipsub with the default subscription filter, which is now capped at 2000 topics. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Dial errors beyond the four upstream handles, a Behaviour dropped with dials pending, and a request that raced a new connection each left open_stream waiting on a channel nothing would drain; the last became reachable once dials were de-duplicated per peer. Closed connections also kept their senders for the life of the process. Co-Authored-By: Codex <codex@openai.com> Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
mo4islona
force-pushed
the
vendor-libp2p-stream
branch
from
October 15, 2026 11:49
3231c67 to
76a6dff
Compare
This was referenced Oct 15, 2026
mo4islona
added a commit
to subsquid/sqd-portal
that referenced
this pull request
Oct 15, 2026
f3564b4 never looks a worker up again after a failed redial evicts its cached address, so requests to that worker fail with "no addresses for peer" until unrelated DHT traffic or a restart brings the address back. e1460fd has subsquid/sqd-network#217 and vendors libp2p-stream, so the libp2p-identity patch for the fork goes away. sqd-assignments stays at f3564b4: the transport does not use it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
mo4islona
added a commit
to subsquid/sqd-portal
that referenced
this pull request
Oct 15, 2026
f3564b4 never looks a worker up again after a failed redial evicts its cached address, so requests to that worker fail with "no addresses for peer" until unrelated DHT traffic or a restart brings the address back. e1460fd has subsquid/sqd-network#217 and vendors libp2p-stream, so the libp2p-identity patch for the fork goes away. The tests and the harness now pass typed chunk ids to the assignment builders. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
mo4islona
added a commit
to subsquid/sqd-portal
that referenced
this pull request
Oct 15, 2026
f3564b4 never looks a worker up again after a failed redial evicts its cached address, so requests to that worker fail with "no addresses for peer" until unrelated DHT traffic or a restart brings the address back. e1460fd has subsquid/sqd-network#217 and vendors libp2p-stream, so the libp2p-identity patch for the fork goes away. The tests and the harness now pass typed chunk ids to the assignment builders. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Supersedes #216: its commits are folded in, so the lookup fix reaches
maintogether with the vendored stream crate.Why
Actors that keep connections to all workers (portal, pings collector, logs collector, SQL client) can lose a worker for good after it is unreachable for a few minutes. A failed redial evicts the worker's cached address and nothing starts a new lookup, so every request to it fails with
Dial error: no addresses for peeruntil unrelated DHT traffic supplies the address or the process restarts.Changes
Look up a worker after a failed dial (from #216)
Transport,WrongPeerIdorNoAddressesdial failure to a registered worker schedules a Kademlia lookup afterreconnect_cooldown.NoAddressesis included because a lookup returns only peers that answered: it cannot find a worker that is still offline, and the dial after it has no address. The cooldown paces these lookups.schedule_reconnect, and a worker waits for one reconnect at a time across the cooldown and the lookup queue, so it is not queued again while the lookup cap holds it back.PRIVATE_NETWORKhold a lock, since they share the process environment.Vendor
libp2p-streaminstead of forking libp2pprotocols/stream. That crate is now copied intocrates/transport/src/libp2p_stream(upstreamaefbfbdc, MIT), and the rest of libp2p comes from crates.io 0.56.0, which drops the[patch.crates-io]forlibp2p-identity.Behaviour of the vendored crate, on top of the fork's two patches (unbounded dial channel,
Control::accept_with_capacity)open_streamcalls, now includingAbortedandLocalPeerId.open_streamreturns an error once itsBehaviouris gone instead of hanging, and dropping theBehaviourcancels the requests still waiting for a dial.DialPeerConditionFalse) hands its queued requests to that connection. The swarm builds a connection's handler before it reports the connection, and anopen_streamin between queues behind a dial that is then skipped. With de-duplication that pending entry used to outlive the connection, and after the next disconnect requests to the peer waited on it instead of dialing.Testing
cargo test --workspace --all-features --locked,cargo clippy --workspace --all-targets --all-featuresand nightlyrustfmtpass on the branch rebased onmain.🤖 Generated with Claude Code