Skip to content

validate-task.yml: three gaps blocking the last carried-copy repo from adopting it (no ref input, root-relative Python detection, no coverage assertion) #1134

Description

@ptr727

Found preparing ptr727/PlexCleaner to adopt the hub's reusable validator, against hub f3b4cc9. Three independent gaps in one file, filed together because one exercise surfaced them. Nothing here is a defect in PlexCleaner.

Context: PlexCleaner is currently the last repo carrying its own validate-task.yml, which spec/divergences.json gives a retire disposition. Now that #1107 migrated the hub's unit-test step to MTP, the coverage blocker is gone, and these three are what remain between it and deleting its copy.

1. No ref input, so a caller cannot validate a named commit

The task declares exactly two inputs:

$ sed -n '/^on:/,/^jobs:/p' .github/workflows/validate-task.yml | grep -E "^      [a-z-]+:"
      markdown-exclude-globs:
      repo-gate-exclude-globs:

and every actions/checkout in it is bare, with no ref:.

A publisher that pins the commit it releases cannot use this. PlexCleaner's publish-release.yml validate job passes ref: ${{ github.sha }} precisely so the publish gate validates the exact commit being published, which is WORKFLOW.md D4.6 "Publish is tested as built" and its own D4.2 pin-the-commit rule. Adopting the hub task leaves two options, both wrong:

  • keep the input, and the call fails workflow_call validation on an undeclared input
  • drop the input, and the gate silently validates the caller's default ref instead of the commit being released

This is already live, not hypothetical. PhotoCleaner is the pilot consumer and its publish-release.yml validate job calls the hub task with no ref, so its publish gate validates whatever the default ref resolves to rather than the commit its publish job pins with ref: ${{ github.sha }}. On a quiet branch those are the same commit; on a branch that advances mid-run they are not, which is the case D4.6 exists for.

An optional ref input threaded to each actions/checkout would close it, and would be inert for a caller that omits it.

2. Python lint gates on a root-relative hashFiles, while the C# steps recurse

Within the same lint job:

167:        if: hashFiles('**/*.csproj') != ''          # C#, recursive
187:        if: hashFiles('**/*.csproj') != ''          # C#, recursive
193:        if: hashFiles('pyproject.toml') != ''       # Python, root only
203:        if: hashFiles('pyproject.toml') != ''       # Python, root only
213:        if: hashFiles('pyproject.toml') != ''       # Python, root only
226:        if: hashFiles('pyproject.toml') != ''       # Python, root only

hashFiles is workspace-relative and does not recurse without **, so a repo whose Python lives in a subdirectory matches nothing and every Python step, ruff check, ruff format, and the type check, skips. The job still reports success, so the gate goes dark rather than red.

PlexCleaner is that shape: its only Python is the stdlib-only tooling under RegressionTests/, with RegressionTests/pyproject.toml and no root pyproject.toml. Its registry driftNotes already records this as the reference csharp+python layout, and spec/project-types.json's python profileNote describes the lint-only profile it uses, so it is a sanctioned fleet shape rather than a PlexCleaner quirk.

Detection and execution both need to move together: **/pyproject.toml would find it, and the steps would then need to run in that directory, since mypy resolves its config from the working directory. PlexCleaner's carried copy currently does this with working-directory: RegressionTests.

3. Nothing asserts a coverage report was actually produced

      - name: Run unit tests step
        if: hashFiles('**/*Tests*.csproj') != ''
        run: |
          set -Eeuo pipefail
          dotnet test --coverage --coverage-output-format cobertura --results-directory ./coverage
          for report in ./coverage/*.cobertura.xml; do
            [ -e "$report" ] || continue
            mv "$report" "./coverage/coverage-$(basename "$report")"
          done

[ -e "$report" ] || continue is correct handling for the no-match glob, but it also makes "the run produced no coverage at all" a silent no-op: dotnet test has already exited 0, the loop does nothing, and the upload's fail_ci_if_error: false then swallows codecov-cli's own "no coverage reports found". Coverage stops being reported on a fully green gate.

The guard cannot distinguish the two cases it currently merges. A post-loop assertion can, for example a compgen -G "./coverage/coverage-*.cobertura.xml" test that emits ::error:: and exits non-zero when it matches nothing.

That keeps fail_ci_if_error: false doing its intended job, tolerating a Codecov outage, while a repo-side regression that stops emission fails loudly. D1.6's own stated purpose, "Prevents: coverage silently going unreported", is the case this closes.

Worth weighing against the if: hashFiles(...) guard on the step: a caller with no test project skips the step entirely, so the assertion would only run where a test project exists and coverage is genuinely expected.

Not filed here

PlexCleaner's own convergence issue is separate. No audit report or registry change is being written back to this repo.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    scriptA defect in hub tooling

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions