WORKFLOW.md section 5 audit procedure: guarantees it never reaches, instructions that fail literally, checks that fail conforming content

[ptr727]

WORKFLOW.md section 5 is the procedure an agent runs to decide whether a repository's pipeline is operational. Twelve of its instructions cannot be followed as written, or would record a conforming repository as drifted. Found by the whole-unit canonical passes on PR #1198 and deferred out of it, whose scope was section 4's contract claims.

WORKFLOW.md is carried at intent, so every fleet repository runs this procedure.

Guarantees the procedure never reaches

5A's core bullets are per-domain summaries rather than per-item assertions, so several MUST-level items are asserted by no 5A bullet and exercised by no 5B scenario: D1.6 entirely (the coverage invocation, the upload step, both secret stores, global.json, the CodeCoverage floor, codecov.yml, the .gitignore entries, every part statically checkable), D3.3, D4.5 (no 5B scenario has a failing build at all), D7.4, D8.1 beyond the concurrency key, D8.2 entirely, D8.4 entirely, and D9.5. A repository can be pronounced operational with coverage silently unreported, which is what #1199 turned out to be.

The Assessment nonetheless says "operational iff every applicable 5A item passes", so it claims a verdict over items the procedure has no check for.

Instructions that fail literally

• 5C bullet 2 says to "Drive a smoke: true push-probe of the build task". build-release-task.yml declares on: workflow_call: only, and the canonical caller's smoke-build fires on pull_request. There is no push path into the task to probe.
• 5C bullet 1 says to "Open a trivial-change PR" on the audited repository, and another says to re-run a real publish. Both are outward-facing writes, with none of the authorization carve-out bullet 5 spells out one paragraph later for the deploy dispatch ("The agent prepares the command and reads all four back afterwards. It does not fire it.").
• Procedure step 3 scopes 5C to "live version classification, registry state, artifact lifecycle", which omits the deploy ref gate that 5C's own fourth bullet declares mandatory and un-substitutable, and includes S1, which the static trace already settles.
• 5C tells a PyPI auditor to inspect the Compute PyPI version step log. That step name exists only in the hub default action; a repository carrying its own build-pypi hook replaces it wholesale and has no step by that name.

Checks that fail conforming content

• 5A's D5 bullet says "no .artifacts[].id blanket delete exists anywhere", mechanizing D5.5 as one literal jq spelling. A blanket delete written .artifacts[] | .id or .artifacts | map(.id) passes the check and violates the rule. The same bullet drops D5.3's "tolerates a failed listing", which the canonical step implements.
• 5A's D4 bullet permits a dispatch-only publisher to "omit the gate and the exists-check entirely", which contradicts D6.4's "The github-release job body stays verbatim" and spec/fidelity-model.md, where that job region is a verbatim unit. A repository taking the permission fails the fidelity check the same document mandates.
• 5A's D6 target-set list is shorter than D6.4's: it omits the enable_<target> input, the build-<target> job, and the changes filter's output. A dropped output or a leftover build-<target> job is what "startup-fails on a missing leaf", and neither is reached.
• The static-site addendum requires concurrency "keyed on the environment", while D7.1 requires "a global, ref-independent concurrency group", and S12 cites D7.1 for exactly that shape. Nothing says the addendum overrides D7.1, so the two checks cannot both pass.
• The PyPI addendum asserts "The pypi environment has a deployment-branch rule", a requirement no section 4 guarantee states, appearing for the first time inside the audit checklist. 5A's declared evidence set is "the workflow files plus version.json", which contains neither that rule, nor the ruleset payload D1.5's name match needs, nor the project files the NuGet symbols check needs, so several items have no citable file:line under the stated scope.
• The expect_release_assets: false requirement is asserted only in the Docker addendum, though D4.3 applies it to Docker-only, PyPI-only and source-only repositories alike. PR Settle Seven Section 4 Contract Claims the Workflows Falsify #1198 added it to the PyPI addendum; source-only still has no addendum.

Scenario rows

• S13 cites D2.3 for a check D2.3 would pass: a production deploy dispatched from develop satisfies "a dispatch from any ref other than main or develop fails fast" while violating the addendum's "production gated to the default branch".
• The Exercises columns disagree across S2, S3 and S4 for identical asserted outputs. S2 asserts "validation runs" and cites D1.1 and D1.5, never D1.2. S3 and S4 both assert "aggregator success" and neither cites D1.5. S4 cites D1.3, which its expected output says nothing about, while its "smoke versions as prerelease" is D3.2's and is cited by nothing.
• The Assessment says "observed" where 5B is titled "No Execution" and instructs emitting a predicted table.

Undefined terms

• "golden-list length floors" appears once in the whole document, in 5C's terminal-check ordering, and is defined only in a comment inside deploy-site-task.yml, which a repository auditing itself has no reason to open.
• "any aggregation job", in 5A's smoke-gating clause, is defined nowhere and collides with D1.5's ruleset-bound aggregator, which must be if: always() and must not be smoke-gated.
• 5B's N/A rule is "Scenarios that exercise an absent target are N/A", narrower than section 1's own applicability rule, and it leaves a dispatch-only publisher with no route to N/A for S5, S6 and S9, which 5A separately says can never fire there.

Related

#1200 covers section 4's own claims. #1199 is the live CI defect that this section's gap allowed through.

[ptr727]

Two more 5A gaps, raised by CodeRabbit on #1203 and verified against the current tree. Both fall inside this issue's "Guarantees the procedure never reaches" section and sharpen it.

D4.5's guards are asserted nowhere in section 5. The D4 static-audit item covers target_commitish, prerelease, the release-create gate, the asset-delete condition, and the dispatch-only exception, and says nothing about D4.5. Checking the whole of section 5, !failure() appears only at line 180 and always() only at lines 156, 162 and 204, all inside section 4. So a workflow can pass the audit while its github-release or Docker job carries no status-check function and skips on a disabled target.

Two related halves are asserted, but not where a reader looks for them. The needs: coverage is at line 236 under D6, "the target set is consistent across the release task (both the github-release and build-docker needs: lists)". The publish-<target> dependency on the release-task call is in the NuGet addendum only, "is gated by that job's needs: on the release-task call, never on an existence check", and the PyPI addendum has no equivalent clause even though D4.5 covers both.

D1's item omits three checks, and #1203 widened one of the gaps. The D1 static-audit item does not require smoke-build to needs: the changes job, does not require the aggregator's if: always(), and reaches no part of D1.6. Codecov appears once in the whole file, at line 157, so nothing in section 5 checks the coverage invocation, the upload step, either secret store, global.json, the CodeCoverage floor, codecov.yml, or the .gitignore entries. That last part is already this issue's opening finding, and it is what #1199 turned out to be.

The new part is the first: #1203's own rewrite of the D1 item added smoke-build to the aggregator's needs: list, changing it from "the aggregator needs: the changes and validation jobs" to "needs: the validation job, and the changes and smoke-build jobs too wherever the repo has a smoke build", without adding the smoke-build needs: changes assertion that rewrite makes newly relevant. So the audit now names smoke-build as an aggregator dependency while still never checking the edge D1.2 requires under it.

Both were deferred out of #1203 rather than fixed, since section 5 wants the one coherent pass this issue scopes rather than two more per-item patches.

[ptr727]

One more scenario-row gap, surfaced by #1215 and reinforcing this issue's own point that section 5 never reaches D4.5.

5B's minimum set has no row for a failed package push. S9 is the closest, and it is scoped to a re-run whose predecessor push succeeded: its expected output is "NuGet/PyPI pushes no-op (server dedupe)". So the entire failed-push branch of D4.5, the branch that carries a recovery an operator is meant to follow under pressure, is in no deterministic trace at all.

That now matters more than it did. #1215 gave D4.5 a second recovery, Re-run all jobs for the case where the branch tip has moved, derived under 5B's own "deterministic from the YAML" standard. A recovery route derived under 5B and then absent from 5B's scenario set is exactly the gap this issue names: nothing an auditor walks would notice if the route stopped holding.

There is a second-order trap in adding the row, worth stating so a fix does not create a false finding. An auditor tracing S9 against a recovery re-run records a real push where S9 predicts a no-op, and would report a defect that is not one. So either S9 gains the failed-push branch explicitly, or a new row carries it and S9 states that its input presupposes a successful predecessor. Leaving both implicit is what makes the two indistinguishable today.

Related: #1216 records the one mechanism that could falsify the new route, an artifact surviving the failed run and colliding with the re-run's upload, which is likewise untested and likewise reaches no scenario row.