Repository navigation
Relink dry run: take present paths from the inventory already in memory - #646
Merged
Merged
Conversation
Every dry run checked every known path with a realpath plus a stat, even when the walk came from the #641 cache: 5.5 s of an 8.0 s dry run for 6,374 paths on a CIFS share. A path listed by an in-memory inventory (walk cache, or the remembered no-rules answer, each inside its TTL) is now present without a stat; every other path is still checked, so the inventory can only make a row look present, never missing. Apply keeps checking every path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ShzWm3urD55AUk5yrSgXsi
coaxk
force-pushed
the
perf/relink-stale-from-inventory
branch
from
October 31, 2026 04:23
27df184 to
8928d33
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #645. Retarget to
mainafter #645 merges.What
The relink dry run no longer stats a known path that an inventory already in memory lists as present. Every other path is still checked, so the inventory can only make a row look present, never missing. Apply (
use_no_rules_cache=False) still checks every path.Why
Measured on the maintainer's install, inside the container: 6,374 known paths on a CIFS share.
canonical_to_fs(realpath, which lstats each component) plusexists()took 5.5 s of an 8.0 s dry run, even when the walk came from the #641 cache. The orphans GET pays the same ~5.75 s, but that path deletes data and keeps checking files on disk.Expected: only the paths the inventory lacks are stat'd (about 1,980 of 6,374 on that install), so roughly 8 s → 2.5 s. This is a projection; measure after deploy.
How
relink.peek_inventory(): the newest inventory in memory for the current library key, either the walk cache (INVENTORY_TTL_S, 300 s) or the remembered no-rules answer (NO_RULES_TTL_S), each inside its own TTL. Never walks._relink_analysisskips the stat for listed paths only whenuse_no_rules_cacheis set.Trade-off
A file deleted since that walk reads as present until the inventory ages out. This is the same blind spot the #641 cache already accepts, and it is in the safe direction: a stale row is kept, nothing is deleted.
Tests
tests/test_relink_stale_inventory.py(12; 8 red before the change). Covers: no inventory means a full stat; a second dry run stats only unlisted paths, with identical results; a file appearing after the walk is stat'd and is not stale; a file deleted after the walk reads present, then stale once the TTL passes; the no-rules inventory is used after the walk cache expires; apply stats everything; an unresolvable path is stat'd and counts present;peek_inventoryreturns None when empty, never walks, prefers the newer walk, ignores another library config and ignores an expired entry. All 184 relink and orphan tests pass locally.🤖 Generated with Claude Code
https://claude.ai/code/session_01ShzWm3urD55AUk5yrSgXsi