Repository navigation
build: profile-guided optimization for natively built release wheels - #68
Merged
Merged
Conversation
mrecachinas
force-pushed
the
build/pgo
branch
from
September 23, 2026 03:52
6aaefb9 to
c78757d
Compare
📊 Benchmark Comparison Results✅ All benchmarks within noise104 benchmarks compared, no significant changes detected. Legend: ✅ Faster (>5%) · |
The Linux x86_64, macOS arm64 and Windows wheels are now PGO builds: an instrumented wheel runs scripts/pgo_train.py, llvm-profdata merges the profile, and every wheel is rebuilt with it. The pipeline lives in the reusable pgo-wheels.yml: - The release calls it on tags. Cross-built targets (Linux aarch64, macOS x86_64) keep plain builds, since the training cannot run on their runner. - On a pull request it runs only with the `ci:wheels` label. It builds and tests the PGO wheels, then compares them with a plain build on the same runner (scripts/pgo_bench.py, best of 5 alternating runs) in the job summary. Nothing is published from pull requests. - The manylinux container installs the same toolchain version as the host, so llvm-profdata matches the compiler. - Training uses the runner's default interpreter, so the set of published wheels is unchanged. The training mix mirrors typical use: small single calls, str/bytes/ bytearray/memoryview inputs, fixed-width and generic scans (serial and pooled), pairwise and many-query batches, catalogs with and without an index, and error paths. Profile-guided size optimization is disabled (-Cllvm-args=-pgso=false). With it, paths the training never ran became size-optimized: pairwise_into at 8 B was 0.47x and set_algo 0.90x. Measured on the full stack (Apple M4 Max, same toolchain for both builds, interleaved A/B): hamming_distance_string/16..64 1.08-1.11x hamming_distance_bytes/16,64 1.04-1.11x check_*_within_dist, scans, pairwise, many-query: neutral The ~1.2-1.6x PGO gains in the original analysis came mostly from inlining PyO3's argument glue, which the raw FASTCALL layer already removed. x86 numbers will come from the `ci:wheels` run. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c7edd725-5714-4044-acf4-c9afc6ec8cf8
With fail-fast, the PyPy compile error on Linux cancelled the macOS job, hiding whether it would have passed. Publishing already waits for every platform, so letting the others finish costs nothing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c7edd725-5714-4044-acf4-c9afc6ec8cf8
mrecachinas
force-pushed
the
build/pgo
branch
from
September 23, 2026 11:59
c78757d to
1e25208
Compare
23 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Linux x86_64, macOS arm64 and Windows release wheels become profile-guided (PGO) builds:
scripts/pgo_train.pyagainst it.llvm-profdata.How the pipeline works (in the reusable
pgo-wheels.yml)ci:wheelslabel. It builds and tests the PGO wheels, then compares them with a plain build on the same runner. The comparison usesscripts/pgo_bench.py(best of 5 alternating runs) and is written to the job summary. Nothing is published from PRs.llvm-profdatamatches the compiler.Training and flags
str/bytes/bytearray/memoryviewinputs; fixed-width and generic scans, both serial and pooled; pairwise and many-query batches; catalogs with and without an index; and error paths.-Cllvm-args=-pgso=false). Without that, paths the training never ran were size-optimized:pairwise_intoat 8 B was 0.47× andset_algo0.90×.Measured on the full stack (Apple M4 Max, same toolchain for both builds, interleaved A/B):
hamming_distance_string16–64 charshamming_distance_bytes16/64 BThe 1.2–1.6× PGO gains from the original analysis came mostly from inlining PyO3's argument glue, which #64 already removed. So this is a modest win on the hottest small calls.
ci:wheelslabel, so its checks include the PGO build on all three platforms and x86 PGO-vs-plain timings (see the job summaries). Before the next release, check that the wheel set and timings look right.x86 (this PR's CI, linux-x86_64, one run each):
best/allover 1024×16 records are 1.17–1.23× faster with PGO, and 16384×64 scans are 1.04× faster. Small calls land at 0.94–1.05×, within single-run noise.PyPy: the first PGO run failed compiling the PyPy 3.11 wheel (
--find-interpreterincludes PyPy). That was a real release blocker, now fixed in #57 (PyO3 0.29 onmain) and #64 (the raw bindings). The matrix no longer fails fast, so one platform's failure doesn't hide the others' results.Found while here: PyPI 3.0.0 ships only one CPython wheel for macOS (cp314) and one for Windows (cp312). Those jobs build for the runner's default interpreter, and this PR keeps that behavior. Adding
--find-interpreter(withsetup-pythonfor 3.10–3.14) would publish wheels for every supported Python.