Skip to content

perf(x86): AVX-512 block scanners, VBMI hex parse, short-input POPCNT - #63

Merged
mrecachinas merged 1 commit into
perf/x86-avx2from
perf/x86-avx512
Sep 23, 2026
Merged

mrecachinas merged 1 commit into
perf/x86-avx2from
perf/x86-avx512

Conversation

@mrecachinas

@mrecachinas mrecachinas commented Sep 23, 2026 •

Copy link
Copy Markdown
Owner

Fixed-width block scanners for 8/16/32/64-byte records on AVX-512 VPOPCNTDQ hosts. VPOPCNTDQ ships on every BITALG CPU (Ice Lake and later, Zen 4).

How it works. For each block of 16 records:

  1. XOR with the broadcast query, then VPOPCNTQ per 8-byte lane.
  2. A VPERMT2D even/odd fold tree reduces the counts to one ZMM of 16 per-record u32 distances, in record order.
  3. One vpcmpleud produces a 16-bit hit mask. Blocks without hits cost no scalar work.
Per 16 records Before (fixed4) After
16 B ~80 uops, 16 compare/branch 16 uops, 1 kortestw
8 B / 64 B no AVX-512 scanner ~6 / ~80 uops

The emitted loops were checked instruction by instruction: no calls, one fold tree, one compare + kortestw per block. native falls back to the AVX2 scanners (#62) without VPOPCNTDQ. first/best/all semantics are unchanged.

Also in this PR

  • hamming_distance_bytes_avx512 sends inputs under 32 bytes to direct POPCNT, skipping the masked-tail setup. Results are identical.
  • Full-distance hex on AVX-512 VBMI: one VPERMI2B per 64 chars from a 128-entry tagged table, validated once per call. _with_max keeps its existing parser and check points.
    • This also fixes a bug in the first draft: its masked tail flagged unloaded lanes as invalid characters. The new tests catch it under SDE icl.

Tests (run under the SDE icl job)

  • Fold tree vs scalar distances, with planted exact and complement records.
  • Scanner oracles for all four widths, with thresholds up to 8*width+1.
  • VBMI hex vs the previous AVX-512 kernel for lengths 0..=600, with and without invalid bytes.
  • The short-input shortcut and public-API dispatch oracles.

Measured on AVX-512 hardware. The Benchmark PR run landed on an Intel Xeon Platinum 8573C (Emerald Rapids: AVX-512 with VPOPCNTDQ, BITALG and VBMI). It compared this PR against #62 (AVX2) on the same host:

Case #62 (AVX2) This PR Speedup
best/all/first 16384×64, no early exit 0.066–0.068 ms 0.014 ms 4.6–4.8×
first 16384×64, match at mid 0.031 ms 0.0073 ms 4.2×
1024×32 matrix, first/best/all 0.0017–0.0021 ms 0.0005–0.0006 ms 3.0–3.6×
1024×16 matrix, first/best/all 0.0008–0.0011 ms 0.0002–0.0004 ms 2.5–3.4×
matches at record 0, small within_dist calls ~0.1 µs ~0.1 µs within noise

SDE validates semantics on the Skylake-SP and Ice Lake models.

@mrecachinas
mrecachinas added this pull request to stack #60 September 23, 2026 01:55
@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

📊 Benchmark Comparison Results

Status Benchmark Base PR Delta
✅ check_bytes_arrays_best_within_dist[16384 elems,s=64,at 0] 0.0679ms 0.0142ms 79.2% faster
✅ check_bytes_arrays_all_within_dist[16384 elems,s=64,end] 0.0683ms 0.0143ms 79.1% faster
✅ check_bytes_arrays_best_within_dist[16384 elems,s=64,end] 0.0673ms 0.0142ms 79.0% faster
✅ check_bytes_arrays_best_within_dist[16384 elems,s=64,mid] 0.0659ms 0.0142ms 78.5% faster
✅ check_bytes_arrays_all_within_dist[16384 elems,s=64,at 0] 0.0660ms 0.0143ms 78.4% faster
✅ check_bytes_arrays_all_within_dist[16384 elems,s=64,mid] 0.0661ms 0.0143ms 78.4% faster
✅ check_bytes_arrays_first_within_dist[16384 elems,s=64,end] 0.0616ms 0.0141ms 77.1% faster
✅ check_bytes_arrays_first_within_dist[16384 elems,s=64,mid] 0.0306ms 0.0073ms 76.2% faster
✅ check_bytes_arrays_first_within_dist[ 1024 elems,s=32,end] 0.0018ms 0.0005ms 71.9% faster
✅ fixed_width_array_scan_matrix[all-threshold_d_minus_1-32] 0.0018ms 0.0005ms 71.9% faster
✅ fixed_width_array_scan_matrix[best-threshold_d_minus_1-32] 0.0018ms 0.0005ms 71.8% faster
✅ fixed_width_array_scan_matrix[best-threshold_d-32] 0.0019ms 0.0005ms 71.7% faster
✅ check_bytes_arrays_all_within_dist[ 1024 elems,s=32,mid] 0.0021ms 0.0006ms 71.7% faster
✅ fixed_width_array_scan_matrix[all-threshold_d-32] 0.0021ms 0.0006ms 71.6% faster
✅ fixed_width_array_scan_matrix[all-random_no_match-32] 0.0018ms 0.0005ms 71.3% faster
✅ fixed_width_array_scan_matrix[best-exact_late-32] 0.0018ms 0.0005ms 71.2% faster
✅ fixed_width_array_scan_matrix[best-random_no_match-32] 0.0018ms 0.0005ms 71.1% faster
✅ fixed_width_array_scan_matrix[first-threshold_d_minus_1-32] 0.0017ms 0.0005ms 71.1% faster
✅ fixed_width_array_scan_matrix[all-exact_mid-32] 0.0021ms 0.0006ms 70.8% faster
✅ fixed_width_array_scan_matrix[first-exact_late-32] 0.0017ms 0.0005ms 70.5% faster
✅ fixed_width_array_scan_matrix[first-random_no_match-32] 0.0017ms 0.0005ms 70.4% faster
✅ fixed_width_array_scan_matrix[all-threshold_d_plus_1-32] 0.0019ms 0.0006ms 69.8% faster
✅ fixed_width_array_scan_matrix[all-exact_late-32] 0.0019ms 0.0006ms 69.4% faster
✅ check_bytes_arrays_best_within_dist[ 1024 elems,s=32,at 0] 0.0020ms 0.0006ms 69.2% faster
✅ fixed_width_array_scan_matrix[all-exact_early-32] 0.0019ms 0.0006ms 68.4% faster
✅ check_bytes_arrays_best_within_dist[ 1024 elems,s=32,mid] 0.0020ms 0.0006ms 68.1% faster
✅ check_bytes_arrays_best_within_dist[ 1024 elems,s=32,end] 0.0019ms 0.0006ms 67.2% faster
✅ fixed_width_array_scan_matrix[best-threshold_d_plus_1-32] 0.0019ms 0.0006ms 67.2% faster
✅ check_bytes_arrays_all_within_dist[ 1024 elems,s=32,at 0] 0.0020ms 0.0007ms 66.9% faster
✅ check_bytes_arrays_all_within_dist[ 1024 elems,s=32,end] 0.0020ms 0.0007ms 65.5% faster
➖ 74 benchmarks within noise (click to expand)
Benchmark Base PR Delta
check_bytes_arrays_all_within_dist[ 512 elems,s=16,at 0] 0.0007ms 0.0003ms 54.0% faster
check_bytes_arrays_all_within_dist[ 512 elems,s=16,end] 0.0006ms 0.0003ms 55.8% faster
check_bytes_arrays_all_within_dist[ 512 elems,s=16,mid] 0.0006ms 0.0002ms 63.0% faster
check_bytes_arrays_best_within_dist[ 512 elems,s=16,at 0] 0.0001ms 0.0001ms 6.8% slower
check_bytes_arrays_best_within_dist[ 512 elems,s=16,end] 0.0005ms 0.0002ms 61.0% faster
check_bytes_arrays_best_within_dist[ 512 elems,s=16,mid] 0.0003ms 0.0001ms 53.9% faster
check_bytes_arrays_first_within_dist[ 512 elems,s=16,at 0] 0.0001ms 0.0001ms 2.6% slower
check_bytes_arrays_first_within_dist[ 512 elems,s=16,end] 0.0004ms 0.0002ms 61.5% faster
check_bytes_arrays_first_within_dist[ 512 elems,s=16,mid] 0.0002ms 0.0001ms 58.7% faster
check_bytes_arrays_first_within_dist[ 1024 elems,s=32,at 0] 0.0001ms 0.0001ms 6.0% slower
check_bytes_arrays_first_within_dist[ 1024 elems,s=32,mid] 0.0009ms 0.0003ms 69.1% faster
check_bytes_arrays_first_within_dist[16384 elems,s=64,at 0] 0.0002ms 0.0002ms 0.1% slower
check_bytes_within_dist[127 bytes,d=500] 0.0001ms 0.0001ms 3.2% slower
check_bytes_within_dist[16 bytes,d=0] 0.0001ms 0.0001ms 6.8% slower
check_bytes_within_dist[64 bytes,d=100] 0.0001ms 0.0001ms 4.5% slower
hex_within_dist 0.0001ms 0.0001ms 0.9% slower
fixed_width_array_scan_matrix[all-exact_early-16] 0.0010ms 0.0004ms 66.1% faster
fixed_width_array_scan_matrix[all-exact_late-16] 0.0011ms 0.0004ms 65.4% faster
fixed_width_array_scan_matrix[all-exact_mid-16] 0.0011ms 0.0004ms 66.7% faster
fixed_width_array_scan_matrix[all-random_no_match-16] 0.0011ms 0.0003ms 71.5% faster
fixed_width_array_scan_matrix[all-threshold_d-16] 0.0011ms 0.0004ms 66.4% faster
fixed_width_array_scan_matrix[all-threshold_d_minus_1-16] 0.0010ms 0.0003ms 69.8% faster
fixed_width_array_scan_matrix[all-threshold_d_plus_1-16] 0.0011ms 0.0004ms 66.6% faster
fixed_width_array_scan_matrix[best-exact_early-16] 0.0001ms 0.0001ms 6.6% slower
fixed_width_array_scan_matrix[best-exact_early-32] 0.0001ms 0.0001ms 12.8% slower
fixed_width_array_scan_matrix[best-exact_late-16] 0.0009ms 0.0003ms 65.2% faster
fixed_width_array_scan_matrix[best-exact_mid-16] 0.0005ms 0.0002ms 60.8% faster
fixed_width_array_scan_matrix[best-exact_mid-32] 0.0010ms 0.0003ms 66.1% faster
fixed_width_array_scan_matrix[best-random_no_match-16] 0.0009ms 0.0003ms 66.0% faster
fixed_width_array_scan_matrix[best-threshold_d-16] 0.0009ms 0.0003ms 66.0% faster
fixed_width_array_scan_matrix[best-threshold_d_minus_1-16] 0.0009ms 0.0003ms 66.2% faster
fixed_width_array_scan_matrix[best-threshold_d_plus_1-16] 0.0009ms 0.0003ms 65.2% faster
fixed_width_array_scan_matrix[first-exact_early-16] 0.0001ms 0.0001ms 3.7% slower
fixed_width_array_scan_matrix[first-exact_early-32] 0.0001ms 0.0001ms 4.7% slower
fixed_width_array_scan_matrix[first-exact_late-16] 0.0008ms 0.0003ms 67.3% faster
fixed_width_array_scan_matrix[first-exact_mid-16] 0.0004ms 0.0002ms 64.9% faster
fixed_width_array_scan_matrix[first-exact_mid-32] 0.0009ms 0.0003ms 65.9% faster
fixed_width_array_scan_matrix[first-random_no_match-16] 0.0008ms 0.0002ms 70.2% faster
fixed_width_array_scan_matrix[first-threshold_d-16] 0.0005ms 0.0002ms 61.2% faster
fixed_width_array_scan_matrix[first-threshold_d-32] 0.0009ms 0.0003ms 66.6% faster
fixed_width_array_scan_matrix[first-threshold_d_minus_1-16] 0.0008ms 0.0003ms 68.8% faster
fixed_width_array_scan_matrix[first-threshold_d_plus_1-16] 0.0005ms 0.0002ms 61.2% faster
fixed_width_array_scan_matrix[first-threshold_d_plus_1-32] 0.0009ms 0.0003ms 66.1% faster
hd_bytes[1000-diff] 0.0001ms 0.0001ms 3.6% slower
hd_bytes[1000-same] 0.0001ms 0.0001ms 3.0% slower
hd_bytes[1024-diff] 0.0001ms 0.0001ms 4.0% slower
hd_bytes[1024-same] 0.0001ms 0.0001ms 3.3% slower
hd_bytes[3-diff] 0.0001ms 0.0001ms 4.4% slower
hd_bytes[3-same] 0.0001ms 0.0001ms 3.7% slower
hd_bytes[64-diff] 0.0001ms 0.0001ms 2.4% slower
hd_bytes_buffer[bytearray] 0.0001ms 0.0001ms 2.7% slower
hd_bytes_buffer[memoryview] 0.0001ms 0.0001ms 2.2% slower
hd_bytes_gil_boundary[16376] 0.0004ms 0.0003ms 23.4% faster
hd_bytes_gil_boundary[16384] 0.0004ms 0.0003ms 9.8% faster
hd_bytes_gil_boundary[16392] 0.0004ms 0.0005ms 24.5% slower
hd_bytes_random[1024] 0.0001ms 0.0001ms 5.3% slower
hd_bytes_random[16] 0.0001ms 0.0001ms 3.0% slower
hd_bytes_random[64] 0.0001ms 0.0001ms 1.8% slower
hd_string[1000-diff] 0.0001ms 0.0001ms 16.2% faster
hd_string[1000-same] 0.0001ms 0.0001ms 18.9% faster
hd_string[1024-diff] 0.0001ms 0.0001ms 18.6% faster
hd_string[1024-same] 0.0001ms 0.0001ms 22.2% faster
hd_string[3-diff] 0.0002ms 0.0002ms 2.6% slower
hd_string[3-same] 0.0001ms 0.0001ms 6.9% slower
hd_string[64-diff] 0.0001ms 0.0001ms 4.1% slower
hd_string_random[1024] 0.0001ms 0.0001ms 16.9% faster
hd_string_random[16] 0.0002ms 0.0001ms 56.6% faster
hd_string_random[64] 0.0001ms 0.0001ms 8.0% slower
stdlib_bytes_distance_random[1024] 0.0027ms 0.0027ms 0.7% slower
stdlib_bytes_distance_random[16] 0.0004ms 0.0004ms 0.7% slower
stdlib_bytes_distance_random[64] 0.0004ms 0.0004ms 0.1% faster
stdlib_hex_distance_random[1024] 0.0034ms 0.0033ms 1.8% faster
stdlib_hex_distance_random[16] 0.0003ms 0.0003ms 0.6% faster
stdlib_hex_distance_random[64] 0.0004ms 0.0004ms 0.2% slower

Legend: ✅ Faster (>5%) · ⚠️ Slower (5-30%) · ❌ Regression (>30%) · 🆕 New · 🗑️ Removed

Fixed-width scanners for 8/16/32/64-byte records on AVX-512 VPOPCNTDQ
hosts. Each block of 16 records is XORed with the broadcast query and
counted per 8-byte lane with VPOPCNTQ. A tree of VPERMT2D even/odd
selections and adds then folds the partial counts into one ZMM of 16
per-record u32 distances in record order. One VPCMPUD gives a 16-bit
hit mask, and blocks without hits need no scalar work. first, best
and all semantics are unchanged; the tail after the last full block
uses scalar POPCNT.

  per 16 records   old (fixed4)             new
  16 B             ~80 uops, 16 cmp/branch  16 uops, 1 kortest
  8 / 64 B         no AVX-512 scanner       ~6 / ~80 uops

Emitted x86-64 loops were checked instruction by instruction: no
calls, one vpermt2d/vpaddd fold tree, one vpcmpleud + kortestw per
block. algo=native falls through to the AVX2 scanners when VPOPCNTDQ
is missing.

Also:
- hamming_distance_bytes_avx512 sends inputs under 32 bytes to the
  direct POPCNT path. That is at most three 64-bit POPCNTs plus a byte
  tail, instead of mask synthesis, two masked loads, VPOPCNTB, VPSADBW
  and a horizontal reduce. Results are identical (exact distance, or
  u64::MAX over max_dist).
- Full-distance hex on AVX-512 VBMI parses 64 chars per VPERMI2B from
  a 128-entry table: nibble for hex digits, 0x80 tag otherwise. Tags
  and raw high bits are ORed and checked once per call. The masked
  tail masks its contribution so unloaded lanes cannot flag an invalid
  char. _with_max keeps the existing parser and check points.

Tests (run under the SDE icl job): the fold tree against scalar
distances with planted exact and complement records; scanner oracles
for all four widths, with thresholds up to 8*width+1; VBMI hex against
the previous AVX-512 kernel for lengths 0..=600 with and without
invalid bytes; the short-input shortcut; and public-API dispatch
oracles.

This host cannot execute x86. Everything above is static analysis;
runtime validation comes from the Intel SDE CI matrix. An AVX-512
(VPOPCNTDQ/VBMI) hardware benchmark is needed before merge.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c7edd725-5714-4044-acf4-c9afc6ec8cf8
mrecachinas added a commit that referenced this pull request Sep 23, 2026
The Benchmark PR run for #63 landed on an AVX-512 host (Xeon Platinum
8573C), comparing the AVX-512 block scanners with the AVX2 ones on the
same machine: 2.5-4.8x faster for 16-, 32- and 64-byte records.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c7edd725-5714-4044-acf4-c9afc6ec8cf8
@mrecachinas
mrecachinas merged commit 90d8f12 into main Sep 23, 2026
31 checks passed
@mrecachinas
mrecachinas deleted the perf/x86-avx512 branch September 23, 2026 13:32
mrecachinas added a commit that referenced this pull request Sep 23, 2026
* docs: refresh README benchmarks for the optimized stack

Re-measure every table in the benchmark section on the final stack and
add scripts/readme_benchmarks.py, which regenerates the Python tables as
reStructuredText (median of seven runs, each the best of seven timeit
repeats). It replaces the .benchmarks/batch_measure.py the README cited,
which is not in the repository.

* Rust (Criterion) and Python tables re-measured; the malformed matrix
  tables are rebuilt with column widths that fit their headers.
* New "Large catalogs" table for single-query scans on the thread pool.
* Catalog index table re-measured against the now-parallel free
  function, noting the planner's linear choice at radius 12.
* The check_hexstrings_within_dist row uses the same early-exit inputs as
  the test suite's benchmark, sliced once outside the timed call.
* The x86 notes describe the SDE and Benchmark PR CI jobs; the AVX-512
  table is labelled as measured before the current block scanners.
* Batch-API interpretation updated for the faster per-call path and the
  serial prefix of early-exit batches.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c7edd725-5714-4044-acf4-c9afc6ec8cf8

* docs: note the AVX-512 block scanner speedup measured on Emerald Rapids

The Benchmark PR run for #63 landed on an AVX-512 host (Xeon Platinum
8573C), comparing the AVX-512 block scanners with the AVX2 ones on the
same machine: 2.5-4.8x faster for 16-, 32- and 64-byte records.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c7edd725-5714-4044-acf4-c9afc6ec8cf8

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c7edd725-5714-4044-acf4-c9afc6ec8cf8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant