Repository navigation
perf(x86): AVX-512 block scanners, VBMI hex parse, short-input POPCNT - #63
Merged
Merged
Conversation
📊 Benchmark Comparison Results
➖ 74 benchmarks within noise (click to expand)
Legend: ✅ Faster (>5%) · |
mrecachinas
force-pushed
the
perf/x86-avx512
branch
from
September 23, 2026 02:08
95bde08 to
1ba753d
Compare
Fixed-width scanners for 8/16/32/64-byte records on AVX-512 VPOPCNTDQ hosts. Each block of 16 records is XORed with the broadcast query and counted per 8-byte lane with VPOPCNTQ. A tree of VPERMT2D even/odd selections and adds then folds the partial counts into one ZMM of 16 per-record u32 distances in record order. One VPCMPUD gives a 16-bit hit mask, and blocks without hits need no scalar work. first, best and all semantics are unchanged; the tail after the last full block uses scalar POPCNT. per 16 records old (fixed4) new 16 B ~80 uops, 16 cmp/branch 16 uops, 1 kortest 8 / 64 B no AVX-512 scanner ~6 / ~80 uops Emitted x86-64 loops were checked instruction by instruction: no calls, one vpermt2d/vpaddd fold tree, one vpcmpleud + kortestw per block. algo=native falls through to the AVX2 scanners when VPOPCNTDQ is missing. Also: - hamming_distance_bytes_avx512 sends inputs under 32 bytes to the direct POPCNT path. That is at most three 64-bit POPCNTs plus a byte tail, instead of mask synthesis, two masked loads, VPOPCNTB, VPSADBW and a horizontal reduce. Results are identical (exact distance, or u64::MAX over max_dist). - Full-distance hex on AVX-512 VBMI parses 64 chars per VPERMI2B from a 128-entry table: nibble for hex digits, 0x80 tag otherwise. Tags and raw high bits are ORed and checked once per call. The masked tail masks its contribution so unloaded lanes cannot flag an invalid char. _with_max keeps the existing parser and check points. Tests (run under the SDE icl job): the fold tree against scalar distances with planted exact and complement records; scanner oracles for all four widths, with thresholds up to 8*width+1; VBMI hex against the previous AVX-512 kernel for lengths 0..=600 with and without invalid bytes; the short-input shortcut; and public-API dispatch oracles. This host cannot execute x86. Everything above is static analysis; runtime validation comes from the Intel SDE CI matrix. An AVX-512 (VPOPCNTDQ/VBMI) hardware benchmark is needed before merge. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c7edd725-5714-4044-acf4-c9afc6ec8cf8
mrecachinas
force-pushed
the
perf/x86-avx512
branch
from
September 23, 2026 11:59
1ba753d to
20e8caf
Compare
mrecachinas
added a commit
that referenced
this pull request
Sep 23, 2026
The Benchmark PR run for #63 landed on an AVX-512 host (Xeon Platinum 8573C), comparing the AVX-512 block scanners with the AVX2 ones on the same machine: 2.5-4.8x faster for 16-, 32- and 64-byte records. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c7edd725-5714-4044-acf4-c9afc6ec8cf8
mrecachinas
added a commit
that referenced
this pull request
Sep 23, 2026
* docs: refresh README benchmarks for the optimized stack Re-measure every table in the benchmark section on the final stack and add scripts/readme_benchmarks.py, which regenerates the Python tables as reStructuredText (median of seven runs, each the best of seven timeit repeats). It replaces the .benchmarks/batch_measure.py the README cited, which is not in the repository. * Rust (Criterion) and Python tables re-measured; the malformed matrix tables are rebuilt with column widths that fit their headers. * New "Large catalogs" table for single-query scans on the thread pool. * Catalog index table re-measured against the now-parallel free function, noting the planner's linear choice at radius 12. * The check_hexstrings_within_dist row uses the same early-exit inputs as the test suite's benchmark, sliced once outside the timed call. * The x86 notes describe the SDE and Benchmark PR CI jobs; the AVX-512 table is labelled as measured before the current block scanners. * Batch-API interpretation updated for the faster per-call path and the serial prefix of early-exit batches. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c7edd725-5714-4044-acf4-c9afc6ec8cf8 * docs: note the AVX-512 block scanner speedup measured on Emerald Rapids The Benchmark PR run for #63 landed on an AVX-512 host (Xeon Platinum 8573C), comparing the AVX-512 block scanners with the AVX2 ones on the same machine: 2.5-4.8x faster for 16-, 32- and 64-byte records. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c7edd725-5714-4044-acf4-c9afc6ec8cf8 --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c7edd725-5714-4044-acf4-c9afc6ec8cf8
23 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixed-width block scanners for 8/16/32/64-byte records on AVX-512 VPOPCNTDQ hosts. VPOPCNTDQ ships on every BITALG CPU (Ice Lake and later, Zen 4).
How it works. For each block of 16 records:
u32distances, in record order.vpcmpleudproduces a 16-bit hit mask. Blocks without hits cost no scalar work.fixed4)kortestwThe emitted loops were checked instruction by instruction: no calls, one fold tree, one compare +
kortestwper block.nativefalls back to the AVX2 scanners (#62) without VPOPCNTDQ.first/best/allsemantics are unchanged.Also in this PR
hamming_distance_bytes_avx512sends inputs under 32 bytes to direct POPCNT, skipping the masked-tail setup. Results are identical._with_maxkeeps its existing parser and check points.icl.Tests (run under the SDE
icljob)8*width+1.Measured on AVX-512 hardware. The
Benchmark PRrun landed on an Intel Xeon Platinum 8573C (Emerald Rapids: AVX-512 with VPOPCNTDQ, BITALG and VBMI). It compared this PR against #62 (AVX2) on the same host:best/all/first16384×64, no early exitfirst16384×64, match at midfirst/best/allfirst/best/allwithin_distcallsSDE validates semantics on the Skylake-SP and Ice Lake models.