Skip to content

ci: test every stacked PR and run x86 paths under Intel SDE - #57

Merged
mrecachinas merged 7 commits into
mainfrom
ci/stacked-pr-checks
Sep 23, 2026
Merged

mrecachinas merged 7 commits into
mainfrom
ci/stacked-pr-checks

Conversation

@mrecachinas

@mrecachinas mrecachinas commented Sep 23, 2026 •

Copy link
Copy Markdown
Owner

CI groundwork for this stack of performance PRs.

  • Stacked PRs get CI. build and Benchmark PR now run for every pull request, not just those into main. Each layer is tested and benchmarked on x86 against the layer below it.

  • New lint job: ruff check, ruff format --check, cargo fmt --check.

  • New rust-tests job (Linux/macOS/Windows): cargo test --no-default-features --release. It logs which SIMD extensions the runner has.

  • New x86-emulated job: runs the Rust and Python suites under Intel SDE as Nehalem (SSE4.2), Haswell (AVX2), Skylake-SP (AVX-512 without BITALG/VBMI) and Ice Lake (AVX-512 with BITALG/VBMI). Every x86 dispatch path now runs, whatever the runner's hardware.

    • tests/x86_cpu_features.rs fails the job if the emulated CPU doesn't report the expected features.
    • The SDE archive's SHA-256 is pinned. It matches Intel's mirror, the setup-sde copy and the AUR PKGBUILD.
  • Docs: .github/copilot-instructions.md now says to use cargo test --no-default-features, because plain cargo test can't link the PyO3 extension. It also describes the new jobs.

  • New PyPy leg in the test matrix (Linux, PyPy 3.11). Releases build PyPy wheels (3.0.0 shipped pp310/pp311 for Linux), but no PR check built for PyPy.

One library fix, found by that leg. main no longer compiles for PyPy: PyO3 0.29 (#49) dropped PyObject_CheckBuffer on PyPy, and python.rs used it to choose an error message after a failed buffer request. The next release could not have built its PyPy wheels. It now probes with the loosest read-only buffer request, which every interpreter supports. The wall-clock test is skipped on PyPy, where the C-extension call alone takes about as long as its threshold. Locally the suite passes on PyPy 3.11 (223 passed, 5 skipped).

mrecachinas and others added 2 commits September 22, 2026 21:00
The build workflow only ran for pull requests targeting main, so PRs
stacked on another branch got no CI at all, and Rust unit tests (the
only place the SIMD kernels are differentially tested against scalar
oracles) never ran in CI.

- Trigger on every pull request.
- Add a lint job: ruff check, ruff format --check, cargo fmt --check.
- Add rust-tests on Linux, macOS, and Windows running
  `cargo test --no-default-features --release`, logging which x86 SIMD
  extensions the runner exposes so skipped backends are visible.
- Fix the contributor docs: plain `cargo test` cannot link because the
  default `python` feature builds an extension module.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c7edd725-5714-4044-acf4-c9afc6ec8cf8
GitHub-hosted runners only execute whichever SIMD backend their CPU
happens to support, so most x86 dispatch paths were never run in CI.
The new x86-emulated job runs the Rust and Python test suites under
Intel SDE on four CPU models:

  nhm  SSE4.2 + POPCNT
  hsw  AVX2
  skx  AVX-512 F/BW without BITALG/VBMI
  icl  AVX-512 with BITALG and VBMI

tests/x86_cpu_features.rs checks HEXHAMMING_EXPECT_X86_FEATURES against
runtime detection, so a misconfigured emulator fails the job instead of
silently skipping backends. The SDE archive is fetched from Intel's
mirror (GitHub mirror as fallback) and verified by SHA-256; the hash
matches Intel's mirror, the setup-sde vendored copy and the AUR
PKGBUILD.

The Benchmark PR workflow now runs for every pull request rather than
only those targeting main, so each stacked layer is benchmarked on x86
against the layer below it.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c7edd725-5714-4044-acf4-c9afc6ec8cf8
@mrecachinas
mrecachinas added this pull request to stack #60 September 23, 2026 01:29
@mrecachinas mrecachinas changed the title ci/stacked pr checks ci: test every stacked PR and run x86 paths under Intel SDE Sep 23, 2026
@github-actions

Copy link
Copy Markdown

📊 Benchmark Comparison Results

✅ All benchmarks within noise

104 benchmarks compared, no significant changes detected.

Legend: ✅ Faster (>5%) · ⚠️ Slower (5-30%) · ❌ Regression (>30%) · 🆕 New · 🗑️ Removed

mrecachinas and others added 5 commits September 22, 2026 21:42
ruff 0.16 reports EXE001 for the shebang on test/test_hexhamming.py,
a pytest module that is never executed directly, so drop the shebang
and pin ruff so future releases cannot change the rules under a PR.
Under Intel SDE the only failure was the wall-clock assertion in
test_check_hexstrings_within_dist_long_random_fast; deselect it there.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c7edd725-5714-4044-acf4-c9afc6ec8cf8
The AVX-512 block scanners are gated on VPOPCNTDQ, so the SDE job must
fail if emulation does not expose (icl) or hide (skx) it.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c7edd725-5714-4044-acf4-c9afc6ec8cf8
The single 50k-call average failed intermittently on shared macOS
runners (0.161 us and 1.880 us against a 0.15 us limit) because
another tenant only has to slow part of the run. A real regression
slows every round, so taking the best of 7 rounds of 10k calls keeps
the guard while removing most of the noise.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c7edd725-5714-4044-acf4-c9afc6ec8cf8
PyO3 0.29 (#49) no longer exposes PyObject_CheckBuffer on PyPy, so main
fails to compile for PyPy and the next release could not build the PyPy
wheels 3.0.0 shipped (pp310/pp311, Linux). It only picks an error message
after a buffer request failed; probe with the loosest read-only request
instead, which every interpreter supports.

The wall-clock test is skipped on PyPy, where the C-extension call alone
takes about as long as its threshold. The suite otherwise passes on PyPy
3.11 (223 passed, 5 skipped).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c7edd725-5714-4044-acf4-c9afc6ec8cf8
Releases build PyPy wheels for Linux with --find-interpreter, but no pull
request check built for PyPy, so a dependency bump broke it unnoticed.
Add a Linux PyPy 3.11 leg to the test matrix.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c7edd725-5714-4044-acf4-c9afc6ec8cf8
@mrecachinas
mrecachinas merged commit 36593ff into main Sep 23, 2026
31 checks passed
@mrecachinas
mrecachinas deleted the ci/stacked-pr-checks branch September 23, 2026 13:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant