Repository navigation
perf(neon): block scanners for fixed-width arrays and pairwise kernels - #59
Merged
Merged
Conversation
📊 Benchmark Comparison Results✅ All benchmarks within noise104 benchmarks compared, no significant changes detected. Legend: ✅ Faster (>5%) · |
mrecachinas
force-pushed
the
perf/neon-block-scanners
branch
from
September 23, 2026 01:43
4373817 to
695a87a
Compare
mrecachinas
force-pushed
the
perf/neon-block-scanners
branch
from
September 23, 2026 01:54
695a87a to
2f5b6d7
Compare
mrecachinas
force-pushed
the
perf/neon-block-scanners
branch
from
September 23, 2026 02:08
2f5b6d7 to
9843637
Compare
Replace the per-record NEON fixed-width scanners with block scanners that compute distances for a whole block of records at once, reduce them with an ADDP tree, compare against a single limit and iterate hits from a one-bit-per-lane mask: width records/block reduction 8 B 16 3-level ADDP, byte totals 16 B 16 15 ADDP, nibble mask 32 B 8 ADDP tree + vpaddlq_u8, u16 totals 64 B 4 ADDP tree + vpaddlq_u8/u16, u32 totals Widths 8 and 64 were previously not specialized at all (generic loop with an indirect kernel call per record). `best` keeps its lowest-index tie-break and exact-match early exit; `all` stays in ascending order. Pairwise distances get const-generic kernels for widths 8/16/20/32/64, used by bytes_pairwise_distances and bytes_pairwise_distances_into when the algorithm is native or neon. Interleaved A/B in one process (Apple M4 Max, CPython 3.14, ns/call, previous layer -> this layer): first/1024x8/nomatch 1593.0 -> 122.9 12.96x best/1024x8/nomatch 1863.3 -> 138.3 13.48x all/1024x16/nomatch 366.6 -> 231.0 1.59x best/1024x16/nomatch 306.8 -> 237.9 1.29x best/1024x32/nomatch 552.0 -> 459.7 1.20x best/1024x64/nomatch 1583.5 -> 935.9 1.69x best/16384x64/nomatch 24.38us -> 13.98us 1.74x pairwise_into/10000x8 17.83us -> 3.40us 5.24x pairwise_into/10000x20 27.99us -> 5.41us 5.17x pairwise_into/10000x32 22.54us -> 5.60us 4.02x pairwise_list/10000x20 50.36us -> 33.10us 1.52x many_all/1000q x 16384x16 5.32ms -> 3.29ms 1.62x Tests: Rust oracle tests for every width across block-boundary counts, limits, duplicates and complements; the Python randomized oracle test now covers widths 8/16/32/64 and limits at the width boundary. Criterion benches cover widths 8/64 and add a pairwise_into group. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c7edd725-5714-4044-acf4-c9afc6ec8cf8
Found while re-measuring the README: dense all_within_dist results (packed/into) were 6-9% slower than before the block scanners. Every match went through Vec::push, and a block whose records all matched was walked lane by lane. Each block with matches now reserves its records once and writes into spare capacity, and a fully matching block is copied out in one straight loop. Rust level, 1024 records, best of 7: width all match half match no match 8 B 1111 -> 720 ns 804 -> 679 ns unchanged 16 B 1200 -> 725 ns 898 -> 713 ns unchanged 32 B 1423 -> 900 ns 1008 -> 845 ns unchanged 64 B 1539 -> 1329 ns 1289 -> 1297 ns unchanged Python, 1024x16 all matching, vs main: packed 1944 -> 1629 ns, into 1291 -> 957 ns (the previous commit: 2043 and 1364 ns). A test checks that each scanner's full-block mask is exactly what mask() reports when every lane matches. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c7edd725-5714-4044-acf4-c9afc6ec8cf8
mrecachinas
force-pushed
the
perf/neon-block-scanners
branch
from
September 23, 2026 11:59
9843637 to
e4deeb8
Compare
23 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces the per-record NEON fixed-width scanners with block scanners. Each block of records gets its distances computed at once and reduced with an ADDP tree. One compare against a single limit then produces a one-bit-per-record hit mask.
vpaddlq_u8(u16 totals)vpaddlq_u8/u16(u32 totals)Widths 8 and 64 previously had no specialization: a generic loop made an indirect kernel call per record. Semantics are unchanged:
firstreturns the lowest index.bestreturns the minimum distance, breaking ties by lowest index, and returns immediately on an exact match.allreturns hits in ascending index order.Pairwise distances get const-generic kernels for widths 8/16/20/32/64.
Interleaved A/B in one process (Apple M4 Max, CPython 3.14):
best1024×8 B, no matchall1024×16 Bbest1024×32 Bbest16384×64 Bpairwise_into10000×8 Bpairwise_into10000×32 Ball, 1000 q × 16384×16 BDense matches. A follow-up commit writes each matching block's results into reserved capacity instead of pushing them one by one, and copies a block whose records all match in one straight loop. Before it, dense
allresults were 6–9% slower than onmain: Python_into, 1024×16 with every record matching, went 1291 → 1364 ns. It now takes 957 ns. At the Rust level, 1024 records: all matching is 1.16–1.66× faster, half matching 1.0–1.26×, and no match is unchanged.Tests
pairwise_intogroup.mask()reports when every lane matches.