feat(durable-streams-rust): --io-uring write paths (experimental, default off — v1 measured as a regression)#4709
Open
balegas wants to merge 11 commits into
Open
feat(durable-streams-rust): --io-uring write paths (experimental, default off — v1 measured as a regression)#4709balegas wants to merge 11 commits into
balegas wants to merge 11 commits into
Conversation
…r telemetry Skip the per-append .meta sidecar flush for plain appends (memory + wal), eliminating the memory-mode cardinality cliff (+104% @ 50k local, +14-16% NVMe) and reducing wal checkpoint work. Adds --server-stats (SRV_STATS: cpu/inflight/ svc/durwait) and --wal-fsync-parallel telemetry/knobs used to characterize the wal bottleneck as device-fdatasync-bound (not shards/CPU). Findings in CARDINALITY_CLIFF_FIX.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y3x7bcT9vLGiT4tXZeQpnk
…er vs O(N) per-stream fdatasync) Cardinality-cliff #2. The checkpoint made each touched stream's data file durable via one fdatasync per stream — O(N_touched) device barriers (measured ~29s at 200k streams), which saturate the disk and stall commit fsyncs (the cliff). --wal-checkpoint-syncfs on replaces that loop with a single syncfs() on the data fs: one barrier flushes every touched file. Linux-only (falls back to the per-stream loop elsewhere); default off. Durability-before-recycle ordering unchanged; covered by e2e_checkpoint_syncfs_recovers_acked_records. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y3x7bcT9vLGiT4tXZeQpnk
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y3x7bcT9vLGiT4tXZeQpnk
…knob + retained-WAL size budget Replace the hardcoded 3s all-shards checkpoint wave with a per-shard dual trigger: --wal-checkpoint-interval-ms (time, default 3000 = old behavior) and --wal-checkpoint-wal-bytes (size, 0 = off): checkpoint a shard as soon as its retained WAL exceeds the budget. Rationale (wal-decomp-lane0, 2026-07-13): acks never gate on checkpoint and reads never touch the WAL, so checkpoint frequency is purely a replay-time budget — a size trigger makes that budget explicit and lets shards self-stagger by their own write rates instead of storming the device together. Each shard's clock restarts when its checkpoint finishes, an in-flight guard prevents re-firing a slow shard, and due shards still checkpoint concurrently. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y3x7bcT9vLGiT4tXZeQpnk
…-path configuration Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y3x7bcT9vLGiT4tXZeQpnk
…ed; note the separate ~1M-stream wall Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y3x7bcT9vLGiT4tXZeQpnk
…cross per-device lane dirs The ~1M-stream wall (wal-1m-diag): with every stream file on ONE data device, a checkpoint's syncfs writeback of ~200k dirty files took 60-74s per shard — tiny appends to distinct files amplify to ~8-12KB of data+inode+journal writeback each, saturating the single device and starving append staging (batch_avg collapsed 53->2-6). --stream-lanes N hashes stream files across streams/<0..N>/ subdirs, one per device: N× writeback capacity, N parallel syncfs barriers (one per lane, threaded), and no single ext4 dir holds every stream. Default 1 = byte-identical historical layout. Recovery walks all lanes; e2e covers lanes=3 crash recovery + layout spread. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y3x7bcT9vLGiT4tXZeQpnk
…n (Linux), drop dead fsync fan-out, refresh WAL_TUNING - --wal-checkpoint-syncfs defaults ON for Linux (validated +51% at the cliff, neutral below; off = escape hatch). Non-Linux unchanged (no syncfs). - Remove FSYNC_FANOUT / --wal-fsync-parallel: parallel per-file checkpoint fsync regressed in every controlled test (f8 -11% NVMe, f16 -19% constrained) and the syncfs barrier supersedes it. Flag accepted as a warning no-op so old deploy scripts don't crash. Checkpoint step 2 is now two modes: per-lane syncfs (Linux default) or the serial per-file fallback. - e2e resets restore the platform default instead of hard 'false'. - WAL_TUNING.md: stream-lanes results (1M wall broken), cardinality-based lane-split guidance, updated flag set, fd-ceiling note (#4706). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y3x7bcT9vLGiT4tXZeQpnk
Config-surface audit removals (all confirmed unexercised or A/B residue of already-shipped defaults): - --wal-fsync-parallel (was already a no-op stub) — gone entirely. - --wal-checkpoint-syncfs — gone; the per-lane syncfs barrier is unconditional on Linux, the serial per-file loop is the non-Linux path. - --wal-meta-gate / --mem-meta-gate — gone; the gated (default) behavior is now the only behavior, dead A/B baseline branches deleted. - --meta-sweep-disable / --meta-sweep-stats — gone (bench-only diagnostics; sweep_meta_once simplified). - --tier local + --tier-local-dir CLI — gone (--tier is off|s3); TierKind::Local stays crate-internal for unit tests. - --durability memory + --tier is now rejected at startup (memory-mode acks are not durable; tiering presumes durable segments). - WAL codec: PAYLOAD_CHECKSUMMED is mandatory — a flag-clear header decodes as Torn (the only writer that ever left it clear was the removed splice relay); dropped the legacy decode branch and its pinning test. - Stale zero-copy-era comment on DurabilityMode::Memory corrected. Kept deliberately (audit showed live use + CI coverage): --tail-cache-bytes (fan-out dedup lever), --read-offload (cold-read isolation), --wal-stats / --server-stats (bottleneck telemetry). Docs (README, WAL_TUNING) updated. 105/105 tests pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y3x7bcT9vLGiT4tXZeQpnk
…unt guard, panic-safe checkpoint ticker Self-review findings on #4697: - Restore packages/durable-streams-rust/Dockerfile (deleted collaterally by a bench image-build flow; docker_multiarch_image.yml builds from this path). - Persist the stream-lane count in streams/.lanes and REFUSE a mismatched --stream-lanes at open (mirrors the WAL shard count contract): a silent mismatch made every existing stream invisible. Also refuses lanes>1 over an existing flat layout; recovery skips the marker; e2e asserts the rejection. - Checkpoint ticker: map task-id -> shard so a PANICKED checkpoint task still clears its in-flight guard (previously a panic silenced that shard's checkpoints forever = unbounded retained WAL). - Drop unused AtomicBool import; mark TierKind::Local as the test-only backend (allow(dead_code)); merge the orphaned checkpoint doc comments; drop the removed-knob tombstone comment; whitespace nit. 105/105 tests pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y3x7bcT9vLGiT4tXZeQpnk
… and memory modes New src/uring.rs: thread-local rings (committer thread + each tokio worker), submit-and-wait per op, routed into (a) WAL segment positioned writes + the group-commit IORING_FSYNC_DATASYNC via the SegmentWriter seam, and (b) the stream data-file append in write_wire — the persistence write both durability modes execute (memory mode's whole disk path). Linux-only with a one-time runtime probe and transparent syscall fallback (flag off by default; non-Linux and seccomp-blocked kernels fall back silently). Durability contract unchanged; e2e crash-recovery test with the flag on. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y3x7bcT9vLGiT4tXZeQpnk
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
balegas
added a commit
that referenced
this pull request
Jul 14, 2026
…yncfs checkpoints, per-shard triggers, stream lanes (#4697) Eliminates the WAL write cardinality cliff end-to-end. Formerly a 4-PR stack (#4704, #4705, #4707 folded in here). Full deployment guidance: `WAL_TUNING.md` (in this PR). **Validated by the canonical benchmark campaign 2026-07-14** (ds-bench#6: `results/REPORT.md` + `PROVENANCE.md`) run against this PR's head. ## The problem WAL-durable write throughput collapsed with stream count: **10.4k ops/s @100k streams** (vs ~110k @10k), degrading further under load. Two compounding causes: 1. The checkpoint issued **one `fdatasync` per touched stream** — an O(N) barrier storm sharing the device with commit fsyncs. 2. At extreme cardinality (~1M), checkpoint **writeback amplification** (~40× for small appends: dirty page + inode + extents + journal per file) saturated the single data device — one `syncfs` measured at 60–74 s. ## The changes | commit | what | measured effect | |---|---|---| | syncfs checkpoint | one `syncfs` barrier per stream lane instead of O(N-touched) `fdatasync` (unconditional on Linux) | +51% at the 100k cliff | | per-shard checkpoint triggers | `--wal-checkpoint-interval-ms` + `--wal-checkpoint-wal-bytes` (retained-WAL/replay budget; shards self-stagger, in-flight guard, panic-safe) | checkpoint cost ≈ 0 | | `--stream-lanes N` | hash stream data files across per-device lane dirs; N parallel per-lane syncfs; lane count persisted + validated on open | 1M wall broken: 68k → **212k** (syncfs 60–74s → 6–11s) | | cleanup | dead knobs removed (`--wal-fsync-parallel`, meta gates, meta-sweep toggles, `--tier local`); memory×tier rejected at startup; flag-clear WAL records decode as torn | smaller, safer surface | ## Results — canonical campaign 2026-07-14 (this PR's head, ideal configuration) **Write saturation** (`canonical-write`, 256 B appends, c4d-standard-64-lssd, 3 stream lanes + 3 WAL lanes, pinned cores): | streams | wal-ideal | memory (no-fsync ceiling) | |---|---|---| | 10k | 417k | 680k | | 100k | **382k (−8%, no cliff)** | 632k | vs 10.4k @100k pre-campaign = **37×**. At extreme cardinality (same config): 285k @500k, 212k @1m. **Reads** (unchanged by this PR, revalidated): catch-up 2.27–2.74 GiB/s, cardinality-flat 10→100 streams; SSE tail p99 2–3 ms up to 2048 connections. **Mixed interference**: 100k concurrent catch-up readers cost a pinned 50k ops/s write load nothing (303 MiB/s replay served alongside). The 2026-07-02 memory-mode delivery collapse no longer reproduces on this build: delivery tracks writes 1:1 to 127k/s (wal: keeps pace to ~40k, caps ~64k del/s at full write saturation). ## Correctness - The recoverability contract is unchanged: ack = WAL group-commit fdatasync; a WAL segment is recycled only after its records' stream bytes are fsynced AND the durable-tail map is persisted. All knobs only change *when/how* that sequence runs. - 105/105 tests: syncfs crash-recovery e2e, 3-lane crash-recovery + layout-spread + lane-mismatch rejection, memory-mode sidecar-skip regression, randomized crash sim. - CI green including all four conformance configs (`memory`, `wal-default`, `wal-read-offload-always`, `wal-resident-cache`). - Defaults preserve behavior except syncfs-on-Linux (extensively measured); `--stream-lanes` defaults to 1 = byte-identical layout. Changeset: patch. ## Follow-ups (tracked) - #4695 log-structured store (removes per-file writeback amplification structurally — the residual 100k→1M slope) - #4696 cold tier as durable store - #4706 lazy fd management (1 fd/live stream = 96% of the 1,048,576 ceiling at 1M streams) - #4709 io_uring experiment (measured as a regression in v1 form; default off, not for merge) 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01Y3x7bcT9vLGiT4tXZeQpnk --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds io_uring support for both durability modes' write paths, flag-gated and default off. Opening with honest benchmark results so the seam is reviewable; the v1 submission strategy is a measured regression and must NOT be enabled as-is.
What's implemented
src/uring.rs: thread-local rings (dedicated committer thread + each tokio worker), one-time runtime probe, transparent syscall fallback (non-Linux, seccomp-blocked kernels).IORING_FSYNC_DATASYNC, via the pre-existingSegmentWriterseam.write_wirestream data-file append (memory's entire persistence path); WAL-mode data-file writes get the same routing.--io-uring on|off(default off). e2e crash-recovery test with the flag on; 109/109 on Linux (real ring verified engaged — no fallback warning), 106/106 macOS (fallback).Benchmark (ds-bench
wal-uring-ab, canonical ideal config, c4d-standard-64-lssd)Real ring path confirmed engaged on GKE (pods run seccomp Unconfined; no fallback warning in server logs). Controls reproduced the canonical references, so the measurement is sound.
Why v1 loses, and what a v2 would need
Submit-and-wait per op keeps one
io_uring_enterper operation (no syscall elided), and buffered regular-file writes through io_uring punt to io-wq kernel workers — extra context switches per 256 B write vs the plainpwritefast path. A genuine win requires batched/linked submission (stage N record writes + linked fsync in one submit from the committer) and/or SQPOLL + registered buffers — an architectural change to the append path, not a drop-in. This PR's value is the seam + the negative result; recommend merging only if we want the flag available for future iteration, otherwise close and keep the branch.🤖 Generated with Claude Code
https://claude.ai/code/session_01Y3x7bcT9vLGiT4tXZeQpnk