Skip to content

fix(vscode): persist the post-processed Cobertura document before the coverage-threshold assertion (#565) - #753

Merged
drmoisan merged 14 commits into
mainfrom
bug/invoke-mstestwithcoverage-threshold-before-setcontent-565
Sep 3, 2026
Merged

fix(vscode): persist the post-processed Cobertura document before the coverage-threshold assertion (#565)#753
drmoisan merged 14 commits into
mainfrom
bug/invoke-mstestwithcoverage-threshold-before-setcontent-565

Conversation

@drmoisan

@drmoisan drmoisan commented Sep 3, 2026

Copy link
Copy Markdown
Owner

Suggested title

fix(vscode): persist the post-processed Cobertura document before the coverage-threshold assertion (#565)

Summary

  • scripts/vscode/Invoke-MSTestWithCoverage.ps1 previously called Assert-CoberturaLineCoverageThreshold before Set-Content persisted the post-processed Cobertura XML, so a failing (sub-threshold) run threw before the judged document was ever written to disk, leaving the raw, un-post-processed dotnet-coverage output at the -CoverageOutput path instead.
  • Fix is a pure two-statement reorder inside Invoke-MSTestWithCoverageMain: Set-Content now runs immediately after $processedXmlContent is computed and before Assert-CoberturaLineCoverageThreshold is called. No logic, threshold value, or message text changed.
  • Adds one new Pester regression test to tests/scripts/vscode/Invoke-MSTest.RunSettings.Tests.ps1 proving Set-Content is invoked exactly once before the threshold assertion can throw on a sub-threshold run.
  • This branch was cut before sibling issue Bug: coverage-cobertura-mstest-powershell-tooling-defects #733 (already merged to main) refactored this same file, splitting Assert-CoberturaLineCoverageThreshold out into a new file, scripts/vscode/Invoke-MSTestWithCoverage.Threshold.ps1. The branch was reconciled against the current main, the defect was re-confirmed still present at the drifted line numbers, and the fix was executed and verified against the reconciled tree.
  • Threshold value (80%) and message text are unchanged and independently confirmed unmodified in both Invoke-MSTestWithCoverage.Helpers.ps1 and the relocated Invoke-MSTestWithCoverage.Threshold.ps1 (empty diffs against main for both files).
  • 6/6 acceptance criteria in spec.md verified and checked off; zero blocking findings in policy-audit, code-review, and feature-audit.

Why

Only the failure path was affected — a passing run always wrote the correct document. But the failure path is exactly when someone reads the artifact on disk to diagnose a coverage shortfall, and what they found was a document with different numbers than the one that produced the failure message (absolute paths, unmerged third-party packages, duplicate classes, and the double-counted line totals that issue #441 had already corrected). A failed gate also left behind an artifact that, if fed to any downstream consumer, silently reported the pre-#441 inflated denominator. Issue #563 (a separate, unrelated contradiction in the threshold value) is explicitly out of scope for this fix, which touches statement order only.

What Changed

Core logic:

  • scripts/vscode/Invoke-MSTestWithCoverage.ps1 — swapped the Set-Content and Assert-CoberturaLineCoverageThreshold calls inside Invoke-MSTestWithCoverageMain (net +2/-2 lines).

Tests:

  • tests/scripts/vscode/Invoke-MSTest.RunSettings.Tests.ps1 — added one new It to the existing Describe 'Invoke-MSTestWithCoverageMain' block (net +8 lines), inserted between the pre-existing 'fails when the search root cannot be found' test and the sibling-issue-Bug: coverage-cobertura-mstest-powershell-tooling-defects #733-added '.claude worktree' exclusion test.

Docs/evidence:

  • docs/features/active/2026-09-02-invoke-mstestwithcoverage-threshold-before-setcontent-565/ — issue, spec, research, atomic plan, full Phase 0-6 evidence trail, and policy-audit/code-review/feature-audit artifacts.

Architecture / How It Fits Together

Invoke-MSTestWithCoverageMain collects coverage via dotnet-coverage, post-processes the raw Cobertura XML for Koverage compatibility (ConvertTo-KoverageCoberturaXml), and previously validated the resulting line-rate before writing it to disk. The fix only reorders when the write happens relative to the validation; Assert-CoberturaLineCoverageThreshold remains a pure read-and-throw function with no side effect that could observe or be observed by Set-Content, so the reorder is behaviorally safe under both the passing and failing case.

Verification

Completed (from evidence trail):

  • RED before / GREEN after: pre-fix simulation showed the new test failing (Set-Content invoked 0 times before the throw); post-fix run showed 28/28 tests passed in the target file.
  • Full PowerShell suite (10 files, 93 tests): 93 passed, 0 failed, 0 skipped.
  • PoshQC format: no rewrites needed on either owned file.
  • PSScriptAnalyzer: 0 diagnostics on either owned file (matches baseline).
  • Coverage on scripts/vscode/Invoke-MSTestWithCoverage.ps1: 90.09% before and after (no regression).
  • git diff against main confirmed empty for both Invoke-MSTestWithCoverage.Helpers.ps1 and Invoke-MSTestWithCoverage.Threshold.ps1 (threshold value/message untouched).
  • feature-review: policy-audit PASS, code-review PASS (0 blocking), feature-audit PASS (6/6 AC verified).

Recommended (CI):

  • PowerShell workflow(s) covering scripts/vscode/** and tests/scripts/vscode/**.

Backward Compatibility / Migration Notes

None. No function signature, parameter, output format, or configuration surface changed. Passing runs (coverage at or above 80%) are unaffected because both statements always executed in that case; only the failing-run artifact-on-disk behavior changes, and only for the better (it now matches the document that produced the failure).

Risks and Mitigations

  • Risk: none identified beyond the reordered two statements themselves — this is a minimal, self-contained fix confined to a single production file plus one test file.
  • Mitigation / rollback: the change is a direct two-line swap; rollback is a straightforward revert of the diff.

Review Guide

Suggested order:

  1. scripts/vscode/Invoke-MSTestWithCoverage.ps1 — the actual fix (4-line diff).
  2. tests/scripts/vscode/Invoke-MSTest.RunSettings.Tests.ps1 — the new regression test (8-line diff).
  3. docs/features/active/2026-09-02-invoke-mstestwithcoverage-threshold-before-setcontent-565/spec.md — acceptance criteria and their check-off state.
  4. Remaining files are evidence artifacts (baseline/QA-gate/regression-testing captures) and the three review artifacts (policy-audit, code-review, feature-audit); skim for the pass/fail signal in each rather than reading line by line.

Follow-ups

GitHub Auto-close

drmoisan and others added 14 commits September 2, 2026 09:33
…rdering fix

Promotes existing issue #565 into an active full-bug feature folder
(issue.md, spec.md, research), and authors an atomic plan reordering
Invoke-MSTestWithCoverage.ps1's Set-Content ahead of the coverage
threshold assertion so a failing gate leaves the post-processed
Cobertura document on disk instead of the raw pre-#441 output.
Preparation-mode only: atomic execution, PR authoring, and CI
monitoring are deferred to parallel-orchestrator.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LTjXvNFHVh7Fo7kYGgWsx2
…r stops misreading them as writes

The parallel-run blast-radius harvester extracts every backtick-delimited,
whitespace-free, repository-relative path token from spec.md and the plan and
treats each as a write target, with no notion of surrounding polarity. spec.md
and the plan quoted several scope-EXCLUSION paths inside backticks (files this
fix must not touch, plus the .claude/**, .codex/**, .agents/**,
config/blast-radius.json, and config/orchestration-routing.json shared-surface
excludes), so the harvester believed this item writes those shared trees,
causing false contention with every other item in the bugs-2026-09-02 run.

- Added an explicit "## Write Set" section to spec.md naming the plan's two
  actual write targets: scripts/vscode/Invoke-MSTestWithCoverage.ps1 and
  tests/scripts/vscode/Invoke-MSTest.RunSettings.Tests.ps1.
- Rewrote every exclusion/context-only path mention in spec.md and the plan
  (Invoke-MSTestWithCoverage.Helpers.ps1, Invoke-MSTestWithCoverage.ClosureFilter.ps1,
  the non-modified sibling Pester test files, the Phase 0 policy-read list, and
  the five shared-surface excludes) as plain prose, with backticks removed.
- No task text, acceptance-criterion substance, command, evidence path, or the
  80% coverage threshold value was changed. Text-presentation revision only.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LTjXvNFHVh7Fo7kYGgWsx2
Records policy reads, feature-document reads (with #733/#748 drift notes),
branch/commit baseline (BASELINE_SHA=dc5e8c0f...), citation verification
(drifted-anchor branch taken for both checks), and PoshQC format/analyze/test
baselines for the ordering fix in Invoke-MSTestWithCoverage.ps1.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…efect

Adds the new It block to Describe 'Invoke-MSTestWithCoverageMain', inserted before
the #733 '.claude worktree' test per the plan's primary-anchor resolution (the
secondary anchor is now ambiguous after #733/#748 added an intervening test).
Confirms RED against the pre-fix order: only the new test fails
(Set-Content invoked 0 times, not 1), all 27 other tests in the file pass.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Pure statement reorder in Invoke-MSTestWithCoverageMain: Set-Content now persists
the post-processed Cobertura document immediately after it is computed, before
Assert-CoberturaLineCoverageThreshold can throw on a sub-threshold run. Applied at
the drifted-anchor location (actual lines 342/344, post-#733 Get-Content insertion)
per re-derived P0-T4 citations. No change to Helpers.ps1, ClosureFilter.ps1, or
Threshold.ps1 (confirmed by empty diffs against BASELINE_SHA).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…y tests

Invoke-MSTest.RunSettings.Tests.ps1: 28/28 passed post-fix, including the new
regression test and the pre-existing 5-test Describe 'Invoke-MSTestWithCoverageMain'
block. Invoke-MSTestWithCoverage.Threshold.Tests.ps1 (the #733-relocated home of
the five Assert-CoberturaLineCoverageThreshold boundary tests, not
Helpers.Tests.ps1 as the plan originally assumed): 5/5 passed, confirming the
threshold logic needed no change.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…le pass

PoshQC format: no rewrites on either owned file. PSScriptAnalyzer: 0 diagnostics on
both files (matches P0-T6 baseline). Full Pester suite (10 files, 93 tests):
Failed=0. Coverage of Invoke-MSTestWithCoverage.ps1 unchanged at 90.09%
(111 commands, 100 executed) vs the P0-T7 baseline — no regression.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Each AC checked off individually, citing its corresponding evidence artifact.
AC 3 and AC 5 carry inline drift notes recording that the threshold logic and its
boundary tests now live in Invoke-MSTestWithCoverage.Threshold.ps1 /
.Threshold.Tests.ps1 post-#733, not the originally cited Helpers.ps1 files.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
spec.md Status -> Implemented, both Test Strategy "Seeded from issue" checkboxes
checked. AC status summary: 6/6 checked off. Issue #565 comment posted
summarizing the fix, mirrored locally.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
git status --porcelain is empty at this point (recorded before this commit
itself); this commit adds only the recording artifact and the final [P6-T4]
plan check-off.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…565

Feature-review pass over the Set-Content/threshold-assertion reorder fix.
All three artifacts PASS with zero blocking findings; 6/6 spec.md
acceptance criteria independently verified.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Adds artifacts/pr_body_565.md and its sibling SHA-256 receipt via the
pr-author skill, and refreshes the PR context bundle (which, until now,
still described a stale merge-base after main advanced past this
branch's reconciliation point).

Issue #565 was independently confirmed OPEN via `gh issue view` before
authoring the single `- Closes #565` directive; the context collector's
"author-asserted autoclose issues" list included noise (#441, #563,
#733, and a non-issue "#ISO-8601" token) that this body does not close.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@drmoisan
drmoisan merged commit 87cb4df into main Sep 3, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Invoke-MSTestWithCoverage.ps1 asserts coverage threshold before Set-Content, leaving the raw un-post-processed Cobertura on disk when the gate fails

1 participant