test(ci): complete the macOS 3.5.0 matrix with both root legs - #7692
Merged
Conversation
oferchen
force-pushed
the
feat/uts-macos-root-legs
branch
from
September 5, 2026 12:52
78839de to
86b60fe
Compare
Linux has run {nonroot,root} x {pipe,tcp} since PR #7408. macOS ran nonroot
only, so the privilege axis - the cells that need real root - was covered on
one platform. This adds the two missing legs and consolidates the duplication
that adding them would otherwise have created.
WHY THE PRIVILEGE AXIS IS NOT REDUNDANT WITH LINUX
Crossing the committed Linux manifests puts ~29 cells in the set a macOS root
leg would newly RUN. They are not the chown/devices cells the old header
assumed: the set is dominated by the symlink-race and confinement family -
backup-dir-symlink-race, symlink-race-dest, temp-dir-symlink-injection,
daemon-module-chdir-symlink, operator-path-backup-symlink, and all five
chroot-*-inner-module. Those are exactly the cells whose Linux passes are
known to rest on Landlock rather than on the userspace confinement guard; that
guard was measured failing on BOTH platforms under a symlinked ancestor. macOS
has no Landlock, so it is the only place the guard is exercised unaided.
THE ~29 IS A PREDICTION AND IS SHIPPED AS ONE
Neither leg names a manifest yet. Each runs the whole corpus and emits its own
via EMIT_EXPECT_RESULT as an artifact, so the manifests come from an executed
run of the leg they describe. That is deliberate: the equivalent prediction for
the macOS tcp leg said 52 cells and the executed run gave 32, because cells
skip on macOS for platform reasons the Linux manifests cannot express. Until
the manifests are committed these two checks report the suite's genuine result,
which on a suite with known divergences means red. Both are non-required,
matching the macOS pipe leg.
CONSOLIDATION, BECAUSE OTHERWISE THIS WOULD ADD TWO MORE COPIES
The nonroot and root legs differ only in the sudo re-exec and their output
paths; six setup steps and a ~60-line summary renderer are identical. Adding
root by copy would have made a third and fourth copy of each - the Linux
workflow already carries two divergent copies of its own preamble.
- .github/actions/upstream-testsuite-macos-setup: the six shared setup steps,
now parameterised. Checkout stays in the caller, because a local `uses:`
action cannot check out the tree it is loaded from.
- tools/ci/summarize_upstream_testsuite.sh: the summary renderer, taking the
log path and a heading. It exits 0 on a missing log on purpose - it runs
under `if: always()` after a step that may have failed, and the job's
conclusion must stay the testsuite's.
Net: one new job and its full setup cost 4 lines of YAML each rather than ~190.
NO NEW CALLER JOB, AND THAT WAS A CORRECTION
`variant` defaults to `all`, so the two existing macOS callers each publish
BOTH legs. This mirrors `_upstream-testsuite.yml`, whose callers pass no
variant at all.
The first version of this change instead added a caller per privilege leg with
`variant: 'root'` pinned. That works, but a caller pinned to one variant makes
the reusable workflow's OTHER job report a permanently SKIPPED check - four of
them across the two transports, which the check-name lint confirmed by counting
84 names instead of 80. Skipped checks nobody can act on are noise, and the
Linux idiom already avoided them. Corrected before landing.
VERIFICATION
The setup extraction was done by script, and the script asserts that every
non-trivial line of the removed block survives in the action. 120 of 129 lines
moved verbatim; the 9 it flagged are exactly the intended parameterisation -
three env.* -> inputs.*, the rust-cache key, and the reworded cache-key comment.
That comment is load-bearing and was updated with the code it explains: the
upstream-source cache key hashes the file that supplies CPPFLAGS/LDFLAGS,
because run_upstream_testsuite.sh skips ./configure when shconfig is present,
so a tree cached under a different flag set would be reused with its ORIGINAL
feature set and no step would notice. The flags now live in the action, so the
key hashes the action too.
Workflows verified by parsing the written YAML, not by re-reading the intended
edit: two jobs, correct `if:` guards, and artifact names, emitted manifest
paths and rust-cache keys all pairwise distinct - each is a per-run namespace,
so a collision would let one leg silently overwrite another's outcomes. The
check also asserts every composite `run:` step declares `shell:`, without which
the action fails to load at runtime naming no step.
Summariser exercised against a synthetic log carrying PASS/FAIL/UPASS/SKIP/
XFAIL rows: correct sections, exit 0. Check-name uniqueness lint: 84 names,
each with one publisher. tools tests: 86 passed, exit 0.
oferchen
force-pushed
the
feat/uts-macos-root-legs
branch
from
September 5, 2026 13:30
86b60fe to
47eecb5
Compare
…runs The two macOS root legs landed in bootstrap: no `expect_result_macos_root`, so each ran the WHOLE corpus and reported its genuine result while EMIT_EXPECT_RESULT wrote the manifest as an artifact. Those runs have now happened, so this converts the bootstrap into a real gate. MEASURED, each from a run of the leg it describes: root/pipe 266 pass / 2 fail / 77 skip (345 selected) root/tcp 124 pass / 12 fail / 19 skip (155 selected) NEITHER was predicted from the Linux legs, and that is deliberate: the one time a macOS leg WAS predicted that way the forecast said 52 cells and the executed run gave 32. A cell can skip on macOS for platform reasons no Linux manifest can express. THE root/tcp RESULT IS THE INTERESTING ONE Its twelve failures are IDENTICAL to the Linux root/tcp set, cell for cell: the four daemon-chroot/-munge/-private-parent/-symlink-escape-matrix rows, daemon-zstd-thread-exhaustion, malicious-server-partial-basis-symlink- overwrite, nonroot-restrictive-perms, and the five proto-* rows. Zero macOS-only failures, zero regressions against the macOS nonroot legs. So adding the privilege axis on macOS introduced no divergence of its own - it re-runs exactly the known-failing set, on a platform with no Landlock to mask a userspace confinement gap. That is what the leg is for. root/pipe's two are `daemon-module-private-parent` and `partial-protected-regular-retry-policy`, both pre-existing and both owned. WHY THE DEFAULT MOVES RATHER THAN THE CALLERS `expect_result_macos_root` now defaults to the root/pipe manifest, mirroring `expect_result_macos`, which already defaults to the nonroot/pipe one. The tcp caller names both tcp manifests explicitly for the same reason it already had to: `--daemon-tests-only` selects a smaller set, so a shared file would fail the driver's coverage guard outright. One idiom, not two. The input keeps its bootstrap contract documented, because that is how the NEXT new leg obtains its first manifest: set it to the empty string, run, take the artifact. Every divergence stays recorded as a `fail` row rather than hidden, exactly as the Linux legs record theirs, so each gate fires on DRIFT from this measured state.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Completes the upstream 3.5.0 testsuite matrix on macOS and consolidates the
duplication that adding the legs would otherwise have created.
Coverage
Why the privilege axis is not redundant with Linux
Crossing the committed Linux manifests puts ~29 cells in the set a macOS
root leg would newly RUN. They are not the chown/devices cells the workflow
header used to assume — the set is dominated by the symlink-race and
confinement family:
backup-dir-symlink-race,symlink-race-dest,temp-dir-symlink-injection,daemon-module-chdir-symlink,operator-path-backup-symlink, and all fivechroot-*-inner-module.Those are exactly the cells whose Linux passes are known to rest on Landlock
rather than on the userspace confinement guard — that guard was measured failing
on both platforms under a symlinked ancestor. macOS has no Landlock, so it is
the only place the guard is exercised unaided.
⚠ The ~29 is a prediction, and ships as one
Neither leg names a manifest. Each runs the whole corpus and emits its own via
EMIT_EXPECT_RESULTas an artifact, so the manifests come from an executed runof the leg they describe.
That is deliberate: the same prediction for the macOS tcp leg said 52 cells
and the executed run gave 32, because cells skip on macOS for platform
reasons the Linux manifests cannot express. Until the manifests are committed
these two checks report the suite's genuine result — which on a suite with known
divergences means red. Both are non-required, matching the macOS pipe leg.
No new caller job — and that was a correction
variantdefaults toall, so the two existing macOS callers each publishboth legs. This mirrors
_upstream-testsuite.yml, whose callers pass novariantat all.The first version of this change added a caller per privilege leg with
variant: 'root'pinned. That works, but a caller pinned to one variant makesthe reusable workflow's other job report a permanently SKIPPED check —
four of them across the two transports, which
check_pr_check_name_uniqueness.pyconfirmed by counting 84 names insteadof 80. Skipped checks nobody can act on are noise, and the Linux idiom already
avoided them. Corrected before landing; the lint is back to 80.
Consolidation — otherwise this would add two more copies
The nonroot and root legs differ only in the sudo re-exec and their output
paths. Six setup steps and a ~60-line summary renderer are identical, and the
Linux workflow already carries two divergent copies of its own preamble.
.github/actions/upstream-testsuite-macos-setup— the shared setup steps.Checkout stays in the caller: a local
uses:action cannot check out the treeit is loaded from.
tools/ci/summarize_upstream_testsuite.sh— the summary renderer. Exits 0on a missing log on purpose: it runs under
if: always()after a step that mayhave failed, and the job's conclusion must stay the testsuite's.
A new job and its full setup now costs 4 lines of YAML instead of ~190.
Verification
of the removed block survives in the action. 120 of 129 moved verbatim; the
9 it flagged are exactly the intended parameterisation (three
env.*→inputs.*, the rust-cache key, and the reworded cache-key comment).cache key hashes the file supplying
CPPFLAGS/LDFLAGS, becauserun_upstream_testsuite.shskips./configurewhenshconfigis present —a tree cached under a different flag set would be reused with its original
feature set and no step would notice. The flags now live in the action, so the
key hashes the action too.
edit. Confirms two jobs, correct
if:guards, and that artifact names, emittedmanifest paths and rust-cache keys are all pairwise distinct — each is a
per-run namespace, so a collision would let one leg silently overwrite
another's outcomes. It also asserts every composite
run:step declaresshell:, without which the action fails to load naming no step.rows: correct sections, exit 0.
check_pr_check_name_uniqueness.py: 80 names, one publisher each.run_tools_tests.sh: 86 passed, exit 0.Follow-up
Once both manifests land from these runs, the residual consolidation is the
Linux workflow's own two copies of the same preamble and summary block — it can
adopt both shared pieces without changing any required check context.