Repository navigation
perf(probe): build the Library Probe payload off the event loop (#626) - #654
Merged
Merged
Conversation
GET /api/probe/library hydrated every media_probe row on the loop and blocked it for 1.1-1.5 s per call at 40k rows (loop_lag warned on 3 of 3 calls; 0 of 3 after). The coverage refresh already hydrates in a worker thread and stays under the 0.5 s threshold at 40k, so it is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ShzWm3urD55AUk5yrSgXsi
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #626.
What the measurement showed
Step 1 of #626, measured with a synthetic
media_probe(2 audio + 3 subtitle streams per row), Python 3.12,--cpus 2(the maintainer instance's limit),loop_lagat its default 0.5 s threshold:Coverage refresh (
build_coverage, 3 builds per size)all_entries()in the buildloop_lagwarningsThe hydration already runs in
asyncio.to_thread, and CPython hands the GIL back every 5 ms, so the loop is slowed, not frozen. The issue's acceptance (noloop_lagwarning from a coverage refresh at 40k) already holds onmain, so no caching, lazy hydration or chunked decode is added there: each would add memory bounds or invalidation logic to remove a gap that is under the threshold.GET /api/probe/library(the Library Probe tab) calledall_entries()directly on the event loop:loop_lagwarningsThe warning's stack named
routers/probe.pylibrary->probe_store.pyall_entries.Change
_library_payloadand runs inasyncio.to_thread. Response shape and every filter are unchanged.Tests
tests/test_probe_library_off_loop_626.py:all_entries()and the per-row classification must never run on the loop. Fails onmain; a mutant that moves only the hydration off-loop fails the classification assert.limitvstotal_cached,only_without_eng(partial English counts as English), combinedfilter_text+sub_kind, and the empty store. These had no test before; they pass onmainand after.tests/test_library_probe.pyunchanged and green.🤖 Generated with Claude Code
https://claude.ai/code/session_01ShzWm3urD55AUk5yrSgXsi