Repository navigation
perf(classify): pack move tables; vote once per read - #481
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #479
(1) Packed move table: shipped (bar cleared)
AnchoredRead.seq_to_sig: Vec<i64>(8 B/base, held for the whole BAM) becomesmoves: PackedMoves(one bit per stride) withAnchoredRead::seq_to_sig()deriving the map where used. A unit test pins the packed expansion equal to the oldseq_to_sig_mapover 200 random move arrays (values 0/1/2, trailing zeros, non-byte lengths).Measured on
ldx14_frozen_edx06(70,188 anchored reads, 65,821 classified, 2 POD5,--threads 8,profilingprofile), anon RSS peak from/proc/PID/smaps_rollup(Anonymous:, polled 0.2 s), arms interleaved, 3 reps on one node:About 125 MB saved, 32% of peak (bar: 10% at test scale or 1 GB at 57 M reads), ~1.8 KB/read, so ~100 GB extrapolated at 57 M reads. Wall unchanged within noise (9.4-10.5 s warm; the first main rep was cold, 38 s).
Identity:
cltags and TSV byte-for-byte equal after sorting in all 3 reps (BAM record bodies equal in order, 73,551 records; TSV 70,189 lines).(2) One vote per read: shipped (correctness fix)
Votes are now tallied from the deduped map after the scan (
BamScan::tally_votes), not per record.a_read_with_several_alignments_votes_once_on_the_one_keptfails with the old per-record counting (time: 3, reversed: 1, total 4 instead of 2). On the real run the frame vote is unchanged (time=0, reversed=66943on both): secondary and supplementary records are already skipped, so multi-anchored reads do not occur in this set.Not changed
Scorers, GPU, the waveform path's layout. Library callers:
AnchoredRead::seq_to_sigis a method now; the field ismoves. Benches updated.CI-equivalent run clean (fmt, workspace clippy
--all-targets, gpu x3, waveform-onnx, models-download, train, crf-decode, classify benches; nextest workspace/cli experimental/classify waveform-onnx; doctests).