Sync mutate_core.py: refuse a file the binary under test is not built from - #186
Merged
Merged
Conversation
… from Re-copied from unordered_dense, where the change was made and where scripts/test_mutate.py covers it. A file no test binary includes scores every mutant `survived`, so the report reads as an indictment of the tests when nothing was measured at all - the flattering direction, which is the one this tool must never fail in. Nanobench cannot hit that case today: src/include/nanobench.h is included by every translation unit in the suite, so the probe accepts it. Verified end to end here rather than assumed, since this is the cmake backend and the change was developed against make: the probe passes, the baseline builds, and mutants score `caught` exactly as before.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Re-copied from unordered_dense, where the change was made and where
scripts/test_mutate.pycovers it (martinus/unordered_dense#206).A file no test binary includes scores every mutant
survived, so the reportreads as an indictment of the tests when nothing was measured at all — the
flattering direction, which is the one this tool must never fail in. It was
found in oans, whose
src/tests.c#includes most ofsrc/*.cbut deliberatelynot
oans.c: sweeping that planned 1,394 mutants that would all have come backsurvived, presented as a 0% kill rate.The probe appends an
#errorto the target and builds. If the build stillsucceeds, nothing the test binary is made of ever included the file — which
needs to know nothing about the build system, and so works the same under cmake
here, meson there and make in oans.
Nanobench cannot hit that case today:
src/include/nanobench.his included byevery translation unit in the suite, so the probe accepts it. Verified end to end
here rather than assumed, since this is the cmake backend and the change was
developed against make — the probe passes, the baseline builds, and mutants score
caughtexactly as before.Why this is a separate PR rather than nothing
This is the gap vendoring introduces, and it bit. The file was copied into all
three working trees weeks ago, but only oans's copy reached a default branch —
so this repository has been running the old core with
lint-mutate-core.pygreen, because it records the hash of whatever it has committed. No lint in
any of the three can see the other two.
Only
mutate_core.pyand its.sha256change; the adapter beside it isuntouched, and nothing in
src/include/or the test suite is affected.Generated by Claude Code