Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,8 @@ The project uses GitHub Actions with maturin for cross-platform wheel building:
every x86 dispatch path runs regardless of runner hardware.
`HEXHAMMING_EXPECT_X86_FEATURES` (checked by `tests/x86_cpu_features.rs`) fails the job if the
emulated CPU does not report the expected features
- `pool-sanitizers`: the scan thread pool's tests (`par::`) under ThreadSanitizer and Miri
with `HEXHAMMING_NUM_THREADS=4`
- `Benchmark PR` (`benchmark.yml`): benchmarks every PR against its base branch on x86 and
comments the comparison
- Builds wheels for Linux (manylinux), macOS, Windows
Expand Down
33 changes: 31 additions & 2 deletions .github/workflows/pythonpackage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,33 @@ jobs:
- name: Run Rust tests
run: cargo test --no-default-features --release

pool-sanitizers:
name: Thread pool under ThreadSanitizer and Miri
runs-on: ubuntu-latest
timeout-minutes: 45
env:
# Enough workers to exercise the pool on any runner.
HEXHAMMING_NUM_THREADS: 4
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

- name: Install Rust nightly
uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9 # v1
with:
toolchain: nightly
components: rust-src, miri

- name: ThreadSanitizer
env:
RUSTFLAGS: -Zsanitizer=thread
run: cargo test -Zbuild-std --target x86_64-unknown-linux-gnu --no-default-features 'par::'

- name: Miri
env:
# The pool's workers outlive the tests by design.
MIRIFLAGS: -Zmiri-disable-isolation -Zmiri-ignore-leaks -Zmiri-env-forward=HEXHAMMING_NUM_THREADS
run: cargo miri test --no-default-features 'par::'

x86-emulated:
name: x86 tests under Intel SDE (${{ matrix.cpu }})
runs-on: ubuntu-latest
Expand Down Expand Up @@ -136,10 +163,12 @@ jobs:

- name: Python tests
# Emulation makes wall-clock assertions meaningless, so skip the one
# timing test along with the benchmarks.
# timing test along with the benchmarks, and SDE cannot follow the
# execve of the test that runs Python in a subprocess.
run: >-
sde64 -${{ matrix.cpu }} -- python -m pytest -q -p no:cacheprovider test/
-k "not bench and not test_check_hexstrings_within_dist_long_random_fast"
-k "not bench and not test_check_hexstrings_within_dist_long_random_fast
and not test_num_threads_environment_variable"

build-and-test:
name: Testing on ${{ matrix.os }} with Python ${{ matrix.python-version }}
Expand Down
1 change: 0 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ python = ["pyo3", "pyo3-build-config"]

[dependencies]
pyo3 = { version = "0.29", features = ["extension-module"], optional = true }
rayon = "1.10"

[build-dependencies]
pyo3-build-config = { version = "0.29", optional = true }
Expand Down
15 changes: 8 additions & 7 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -428,9 +428,8 @@ array all [100000×128, parallel] 99,266.0
================================================ ===========

On AArch64, LLVM's auto-vectorized native byte loop is faster than the
hand-written NEON byte kernel for these sizes, while hexadecimal strings still
use the packed NEON implementation. Large array workloads use four balanced
Rayon jobs to avoid oversubscribing the memory-bound scan.
hand-written NEON byte kernel for these sizes. Large array scans are split into
chunks that a small built-in thread pool claims dynamically.

Python API (via PyO3)
~~~~~~~~~~~~~~~~~~~~~
Expand Down Expand Up @@ -505,10 +504,12 @@ dominates (roughly 30–40 ns on this machine). For large inputs
(1024+ chars, 16384-element arrays), computation dominates and Python overhead
is negligible. Byte operations release the GIL at 16 KiB, while immutable
strings use a zero-copy detached path from 4 KiB. Array wrappers release the GIL
at 64 KiB; generic scans parallelize with Rayon at 5 MiB, while the optimized
16/32-byte NEON scanners use a measured 16 MiB crossover. The ``first`` variant
additionally short-circuits on the first hit, so a match near the start is much
faster than one near the end.
at 64 KiB. Array scans run on a built-in thread pool from 512 KiB for widths
with a block scanner (8/16/32/64 bytes) and from 64 KiB for other widths, the
measured crossovers on this machine. Set ``HEXHAMMING_NUM_THREADS`` to limit
the pool (``1`` disables it). The ``first`` and ``best`` variants scan a short
prefix serially and stop early, so a match near the start is much faster than
one near the end.

Batch APIs vs. Python for-loops
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Expand Down
17 changes: 9 additions & 8 deletions src/api.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,14 @@ use crate::par;
use std::sync::atomic::{AtomicUsize, Ordering};

/// Scans of at least this many bytes are split across threads when each record
/// costs a kernel call (widths without a block scanner). Measured crossover
/// (Apple M4 Max, rayon): 1.6x at 1 MiB for 20-byte records.
pub(crate) const PAR_THRESHOLD_BYTES: usize = 1024 * 1024;
/// costs a kernel call (widths without a block scanner). Measured on an Apple
/// M4 Max with the scan pool: 20-byte records are 2.0-3.5x faster at 128 KiB
/// and ~3x at 256 KiB.
pub(crate) const PAR_THRESHOLD_BYTES: usize = 64 * 1024;
/// Block scanners are cheaper per byte, so they need a larger scan before
/// waking the thread pool pays off: 0.6x at 2 MiB, 1.2-1.3x at 4 MiB and 1.8x
/// at 8 MiB for 16- and 64-byte records.
pub(crate) const FIXED_WIDTH_PAR_THRESHOLD_BYTES: usize = 4 * 1024 * 1024;
/// splitting pays off: 16- and 64-byte records are 0.8x at 256 KiB, 1.4-1.7x
/// at 512 KiB and 2.5x at 1 MiB.
pub(crate) const FIXED_WIDTH_PAR_THRESHOLD_BYTES: usize = 512 * 1024;
/// `first` and `best` scan a serial prefix before going parallel, so early
/// matches (and early exact matches for `best`) never pay for waking workers.
/// It is 1/64 of the scan, clamped to 16-256 KiB, which keeps its cost on scans
Expand Down Expand Up @@ -239,7 +240,7 @@ impl<'a> Scan<'a> {
let prefix_bytes = (records.len() / SERIAL_PREFIX_SHARE)
.clamp(SERIAL_PREFIX_MIN_BYTES, SERIAL_PREFIX_MAX_BYTES);
let prefix = (prefix_bytes / width).next_multiple_of(16).min(count);
par::plan(count - prefix, width, 0).map(|plan| (prefix, plan))
par::split(count - prefix, width, min_bytes).map(|plan| (prefix, plan))
}

/// `first_parallel` with an explicit minimum scan size for splitting.
Expand Down Expand Up @@ -690,7 +691,7 @@ mod tests {
}

// -----------------------------------------------------------------------
// Wave 2b: rayon parallelization regression tests
// Parallel scan regression tests
// -----------------------------------------------------------------------

/// Build a big array of `num_elements` chunks of size `elem_size`, all filled
Expand Down
10 changes: 5 additions & 5 deletions src/catalog.rs
Original file line number Diff line number Diff line change
Expand Up @@ -33,17 +33,17 @@ const MAX_PROBE_RADIUS: usize = 4;
// (Apple M4 Max, block scanners): a bucket probe, including verifying its
// ~N/2^s records, costs 1.2-5.6 ns, rising with index size as probes miss
// cache and TLB. A linear scan costs about 0.11 ns per record or 0.0133 ns
// per byte, whichever is larger; split across threads it runs about 1.2x
// faster at 4 MiB, 1.8x at 8 MiB and ~3x from 16 MiB. Slower linear scans
// elsewhere only make the planner more conservative.
// per byte, whichever is larger; split across threads it runs about 1.4x
// faster at 512 KiB, 2.5x at 1 MiB and 4-6x from 2 MiB (modelled as up to 4x).
// Slower linear scans elsewhere only make the planner more conservative.
const QUERY_OVERHEAD_NS: f64 = 50.0;
const PROBE_BASE_NS: f64 = 1.5;
const PROBE_CACHE_MISS_NS: f64 = 4.0;
const PROBE_MISS_SATURATION_BYTES: f64 = (128u64 << 20) as f64;
const SCAN_NS_PER_RECORD: f64 = 0.11;
const SCAN_NS_PER_BYTE: f64 = 0.0133;
const PARALLEL_SPEEDUP_BYTES: f64 = (5u64 << 20) as f64;
const MAX_PARALLEL_SPEEDUP: f64 = 3.0;
const PARALLEL_SPEEDUP_BYTES: f64 = (400u64 << 10) as f64;
const MAX_PARALLEL_SPEEDUP: f64 = 4.0;

/// Estimated cost of scanning `n` records of `width` bytes linearly.
fn linear_scan_ns(n: usize, width: usize) -> f64 {
Expand Down
Loading
Loading