Skip to content

perf(x86): AVX2+POPCNT fixed-width scanners and pairwise kernels - #62

Merged
mrecachinas merged 1 commit into
perf/neon-hex-lutfrom
perf/x86-avx2
Sep 23, 2026
Merged

mrecachinas merged 1 commit into
perf/neon-hex-lutfrom
perf/x86-avx2

Conversation

@mrecachinas

@mrecachinas mrecachinas commented Sep 23, 2026 •

Copy link
Copy Markdown
Owner

On AVX2-only hosts (no AVX-512), fixed-width scans had no specialized scanner:

  • Every record took an indirect byte-kernel call.
  • Inputs under 64 bytes then re-ran SSE4.1/POPCNT feature detection and tail-called the SSE kernel.
  • Static analysis puts that at about 26 uops per 16-byte record.

This PR adds:

  • Scanners for 8/16/32/64-byte records: unrolled POPCNT loops with no per-record call, used for native/avx2 when AVX2+POPCNT are present. first/best/all semantics are unchanged.
  • Pairwise fast paths for the same widths. The list API writes straight into its u64 buffer.
  • Short-input shortcut: hamming_distance_bytes_avx2 sends inputs under 64 bytes to an inline POPCNT path. It returns the same values as the SSE/native kernels it replaces: the exact distance, or u64::MAX once over max_dist.
  • Hex: full-distance AVX2 hex checks invalid lanes once after the vector body, and uses 128-char batches to cut register pressure. _with_max is unchanged, so invalid-vs-threshold precedence is preserved.

⚠️ x86 validation: this was written on Apple Silicon, so the evidence here is static (assembly and uop counts).

  • Execution is covered by the x86-emulated SDE matrix: hsw runs these AVX2 paths, and skx/icl run them as fallbacks.
  • Real x86 timings come from the Benchmark PR workflow comment on this PR.

@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

📊 Benchmark Comparison Results

Status Benchmark Base PR Delta
✅ fixed_width_array_scan_matrix[first-threshold_d_minus_1-16] 0.0065ms 0.0007ms 89.7% faster
✅ fixed_width_array_scan_matrix[first-random_no_match-16] 0.0065ms 0.0007ms 89.6% faster
✅ fixed_width_array_scan_matrix[first-exact_late-16] 0.0065ms 0.0007ms 89.4% faster
✅ fixed_width_array_scan_matrix[all-threshold_d_minus_1-16] 0.0065ms 0.0007ms 88.5% faster
✅ fixed_width_array_scan_matrix[all-random_no_match-16] 0.0065ms 0.0007ms 88.5% faster
✅ fixed_width_array_scan_matrix[first-threshold_d_plus_1-16] 0.0034ms 0.0004ms 88.1% faster
✅ check_bytes_arrays_first_within_dist[ 512 elems,s=16,end] 0.0034ms 0.0004ms 88.0% faster
✅ fixed_width_array_scan_matrix[first-threshold_d-16] 0.0034ms 0.0004ms 88.0% faster
✅ fixed_width_array_scan_matrix[first-exact_mid-16] 0.0034ms 0.0004ms 88.0% faster
✅ fixed_width_array_scan_matrix[all-exact_early-16] 0.0066ms 0.0008ms 87.6% faster
✅ fixed_width_array_scan_matrix[all-threshold_d-16] 0.0066ms 0.0008ms 87.3% faster
✅ fixed_width_array_scan_matrix[all-exact_mid-16] 0.0066ms 0.0008ms 87.3% faster
✅ fixed_width_array_scan_matrix[all-threshold_d_plus_1-16] 0.0066ms 0.0008ms 87.3% faster
✅ check_bytes_arrays_first_within_dist[ 512 elems,s=16,mid] 0.0017ms 0.0002ms 86.4% faster
✅ fixed_width_array_scan_matrix[best-threshold_d-16] 0.0069ms 0.0009ms 86.4% faster
✅ fixed_width_array_scan_matrix[best-threshold_d_plus_1-16] 0.0069ms 0.0009ms 86.4% faster
✅ fixed_width_array_scan_matrix[best-random_no_match-16] 0.0069ms 0.0009ms 86.2% faster
✅ fixed_width_array_scan_matrix[best-threshold_d_minus_1-16] 0.0069ms 0.0009ms 86.2% faster
✅ fixed_width_array_scan_matrix[best-exact_late-16] 0.0069ms 0.0010ms 86.1% faster
✅ fixed_width_array_scan_matrix[all-exact_late-16] 0.0066ms 0.0010ms 85.5% faster
✅ check_bytes_arrays_all_within_dist[ 512 elems,s=16,at 0] 0.0034ms 0.0005ms 85.4% faster
✅ check_bytes_arrays_all_within_dist[ 512 elems,s=16,end] 0.0035ms 0.0005ms 85.2% faster
✅ check_bytes_arrays_all_within_dist[ 512 elems,s=16,mid] 0.0035ms 0.0005ms 85.2% faster
✅ check_bytes_arrays_best_within_dist[ 512 elems,s=16,end] 0.0036ms 0.0005ms 84.8% faster
✅ fixed_width_array_scan_matrix[best-exact_mid-16] 0.0036ms 0.0005ms 84.7% faster
✅ check_bytes_arrays_best_within_dist[ 1024 elems,s=32,mid] 0.0078ms 0.0014ms 82.1% faster
✅ check_bytes_arrays_best_within_dist[ 512 elems,s=16,mid] 0.0018ms 0.0003ms 81.9% faster
✅ check_bytes_arrays_best_within_dist[ 1024 elems,s=32,at 0] 0.0078ms 0.0014ms 81.9% faster
✅ fixed_width_array_scan_matrix[best-exact_mid-32] 0.0040ms 0.0007ms 81.8% faster
✅ fixed_width_array_scan_matrix[best-random_no_match-32] 0.0078ms 0.0014ms 81.8% faster
✅ check_bytes_arrays_best_within_dist[ 1024 elems,s=32,end] 0.0078ms 0.0014ms 81.6% faster
✅ fixed_width_array_scan_matrix[best-exact_late-32] 0.0078ms 0.0014ms 81.6% faster
✅ check_bytes_arrays_first_within_dist[ 1024 elems,s=32,end] 0.0075ms 0.0014ms 81.3% faster
✅ fixed_width_array_scan_matrix[best-threshold_d-32] 0.0078ms 0.0015ms 81.1% faster
✅ fixed_width_array_scan_matrix[first-threshold_d_plus_1-32] 0.0039ms 0.0007ms 80.9% faster
✅ fixed_width_array_scan_matrix[first-threshold_d-32] 0.0038ms 0.0007ms 80.8% faster
✅ fixed_width_array_scan_matrix[first-exact_mid-32] 0.0039ms 0.0007ms 80.8% faster
✅ fixed_width_array_scan_matrix[first-threshold_d_minus_1-32] 0.0075ms 0.0014ms 80.7% faster
✅ fixed_width_array_scan_matrix[first-random_no_match-32] 0.0075ms 0.0014ms 80.7% faster
✅ fixed_width_array_scan_matrix[first-exact_late-32] 0.0075ms 0.0015ms 80.5% faster
✅ check_bytes_arrays_first_within_dist[ 1024 elems,s=32,mid] 0.0038ms 0.0008ms 78.1% faster
✅ fixed_width_array_scan_matrix[all-random_no_match-32] 0.0075ms 0.0017ms 77.7% faster
✅ fixed_width_array_scan_matrix[all-threshold_d_minus_1-32] 0.0075ms 0.0017ms 77.6% faster
✅ fixed_width_array_scan_matrix[all-exact_early-32] 0.0076ms 0.0018ms 76.8% faster
✅ check_bytes_arrays_all_within_dist[ 1024 elems,s=32,at 0] 0.0076ms 0.0018ms 76.8% faster
✅ fixed_width_array_scan_matrix[all-threshold_d_plus_1-32] 0.0076ms 0.0018ms 76.5% faster
✅ fixed_width_array_scan_matrix[all-exact_late-32] 0.0076ms 0.0018ms 76.5% faster
✅ check_bytes_arrays_all_within_dist[ 1024 elems,s=32,mid] 0.0076ms 0.0018ms 76.4% faster
✅ fixed_width_array_scan_matrix[all-exact_mid-32] 0.0076ms 0.0018ms 76.4% faster
✅ fixed_width_array_scan_matrix[all-threshold_d-32] 0.0076ms 0.0018ms 76.2% faster
✅ check_bytes_arrays_all_within_dist[ 1024 elems,s=32,end] 0.0076ms 0.0018ms 76.2% faster
✅ check_bytes_arrays_best_within_dist[16384 elems,s=64,at 0] 0.1181ms 0.0359ms 69.6% faster
✅ check_bytes_arrays_best_within_dist[16384 elems,s=64,mid] 0.1188ms 0.0365ms 69.2% faster
✅ check_bytes_arrays_first_within_dist[16384 elems,s=64,end] 0.1086ms 0.0339ms 68.8% faster
✅ check_bytes_arrays_first_within_dist[16384 elems,s=64,mid] 0.0542ms 0.0173ms 68.1% faster
✅ check_bytes_arrays_best_within_dist[16384 elems,s=64,end] 0.1137ms 0.0372ms 67.3% faster
✅ check_bytes_arrays_all_within_dist[16384 elems,s=64,at 0] 0.1091ms 0.0388ms 64.5% faster
✅ check_bytes_arrays_all_within_dist[16384 elems,s=64,end] 0.1090ms 0.0389ms 64.4% faster
✅ check_bytes_arrays_all_within_dist[16384 elems,s=64,mid] 0.1086ms 0.0389ms 64.2% faster
➖ 45 benchmarks within noise (click to expand)
Benchmark Base PR Delta
check_bytes_arrays_best_within_dist[ 512 elems,s=16,at 0] 0.0001ms 0.0001ms 4.5% faster
check_bytes_arrays_first_within_dist[ 512 elems,s=16,at 0] 0.0001ms 0.0001ms 3.7% faster
check_bytes_arrays_first_within_dist[ 1024 elems,s=32,at 0] 0.0001ms 0.0001ms 6.8% faster
check_bytes_arrays_first_within_dist[16384 elems,s=64,at 0] 0.0002ms 0.0002ms 2.6% slower
check_bytes_within_dist[127 bytes,d=500] 0.0001ms 0.0001ms 1.0% slower
check_bytes_within_dist[16 bytes,d=0] 0.0001ms 0.0001ms 3.8% slower
check_bytes_within_dist[64 bytes,d=100] 0.0001ms 0.0001ms 3.4% slower
hex_within_dist 0.0001ms 0.0001ms 1.1% slower
fixed_width_array_scan_matrix[best-exact_early-16] 0.0001ms 0.0001ms 2.1% faster
fixed_width_array_scan_matrix[best-exact_early-32] 0.0001ms 0.0001ms 2.0% slower
fixed_width_array_scan_matrix[best-threshold_d_minus_1-32] 0.0078ms 0.0014ms 81.8% faster
fixed_width_array_scan_matrix[best-threshold_d_plus_1-32] 0.0078ms 0.0015ms 81.1% faster
fixed_width_array_scan_matrix[first-exact_early-16] 0.0001ms 0.0001ms 3.9% faster
fixed_width_array_scan_matrix[first-exact_early-32] 0.0001ms 0.0001ms 5.7% faster
hd_bytes[1000-diff] 0.0001ms 0.0001ms 5.3% faster
hd_bytes[1000-same] 0.0001ms 0.0001ms 1.8% faster
hd_bytes[1024-diff] 0.0001ms 0.0001ms 0.8% slower
hd_bytes[1024-same] 0.0001ms 0.0001ms 0.7% slower
hd_bytes[3-diff] 0.0002ms 0.0001ms 58.1% faster
hd_bytes[3-same] 0.0001ms 0.0001ms 1.2% slower
hd_bytes[64-diff] 0.0001ms 0.0001ms 4.7% faster
hd_bytes_buffer[bytearray] 0.0001ms 0.0001ms 4.4% faster
hd_bytes_buffer[memoryview] 0.0001ms 0.0001ms 3.2% slower
hd_bytes_gil_boundary[16376] 0.0006ms 0.0006ms 3.4% slower
hd_bytes_gil_boundary[16384] 0.0007ms 0.0007ms 4.3% slower
hd_bytes_gil_boundary[16392] 0.0007ms 0.0007ms 2.9% slower
hd_bytes_random[1024] 0.0001ms 0.0001ms 7.9% faster
hd_bytes_random[16] 0.0001ms 0.0001ms 3.3% faster
hd_bytes_random[64] 0.0001ms 0.0001ms 5.7% slower
hd_string[1000-diff] 0.0002ms 0.0002ms 24.1% faster
hd_string[1000-same] 0.0002ms 0.0002ms 12.7% faster
hd_string[1024-diff] 0.0002ms 0.0001ms 21.8% faster
hd_string[1024-same] 0.0002ms 0.0001ms 16.1% faster
hd_string[3-diff] 0.0001ms 0.0002ms 111.7% slower
hd_string[3-same] 0.0001ms 0.0001ms 2.6% slower
hd_string[64-diff] 0.0001ms 0.0001ms 1.4% slower
hd_string_random[1024] 0.0002ms 0.0001ms 21.3% faster
hd_string_random[16] 0.0001ms 0.0001ms 9.3% slower
hd_string_random[64] 0.0001ms 0.0001ms 1.5% slower
stdlib_bytes_distance_random[1024] 0.0030ms 0.0029ms 1.0% faster
stdlib_bytes_distance_random[16] 0.0005ms 0.0004ms 24.5% faster
stdlib_bytes_distance_random[64] 0.0005ms 0.0005ms 2.6% slower
stdlib_hex_distance_random[1024] 0.0035ms 0.0035ms 0.0% slower
stdlib_hex_distance_random[16] 0.0004ms 0.0003ms 4.4% faster
stdlib_hex_distance_random[64] 0.0005ms 0.0005ms 0.6% faster

Legend: ✅ Faster (>5%) · ⚠️ Slower (5-30%) · ❌ Regression (>30%) · 🆕 New · 🗑️ Removed

On AVX2-only hosts (no AVX-512 BITALG), fixed-width scans had no
specialized scanner. Every record took an indirect byte-kernel call,
and inputs under 64 bytes then re-ran SSE4.1/POPCNT feature detection
and tail-called the SSE kernel: about 26 uops per 16-byte record
(analysis in x86.md).

- Fixed-width scanners for 8/16/32/64-byte records. They are unrolled
  POPCNT record loops with no per-record call, selected for algo
  native/avx2 when AVX2 and POPCNT are present. first, best and all
  keep their semantics: lowest index; minimum distance with
  lowest-index ties and exact-match exit; ascending order.
- Pairwise fast paths for the same widths. The list API writes
  straight into its u64 buffer.
- hamming_distance_bytes_avx2 sends inputs under 64 bytes to an inline
  POPCNT path. It returns the same values as the SSE/native kernels it
  replaces: the exact distance, or u64::MAX once over max_dist.
- Full-distance AVX2 hex accumulates invalid lanes and checks them once
  after the vector body. Batches drop from 256 to 128 chars to cut
  register pressure. _with_max is unchanged, so invalid-vs-threshold
  precedence stays at its existing check points.

This host cannot execute x86. The evidence here is static (assembly
and uop counts); execution is validated by the Intel SDE CI matrix
(hsw/skx/icl), and x86 timings come from the Benchmark PR workflow.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c7edd725-5714-4044-acf4-c9afc6ec8cf8
@mrecachinas
mrecachinas merged commit a2fc546 into main Sep 23, 2026
31 checks passed
@mrecachinas
mrecachinas deleted the perf/x86-avx2 branch September 23, 2026 13:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant