Skip to content

fix(vscode): correct Cobertura merge arithmetic, MSTest assembly discovery, and closure-filter documentation (#733) - #748

Merged
drmoisan merged 6 commits into
mainfrom
bug/coverage-cobertura-mstest-powershell-tooling-defects-733
Sep 3, 2026
Merged

fix(vscode): correct Cobertura merge arithmetic, MSTest assembly discovery, and closure-filter documentation (#733)#748
drmoisan merged 6 commits into
mainfrom
bug/coverage-cobertura-mstest-powershell-tooling-defects-733

Conversation

@drmoisan

@drmoisan drmoisan commented Sep 3, 2026

Copy link
Copy Markdown
Owner

Suggested title

fix(vscode): correct Cobertura merge arithmetic, MSTest assembly discovery, and closure-filter documentation (#733)

Summary

  • Recomputes <package>-level line-rate and branch-rate after a class merge, so merged Cobertura documents no longer report the stale input rate at package scope.
  • Unions <methods> entries across every member of a merged filename group instead of cloning only the primary class's subtree, so method entries contributed by closure classes survive the merge.
  • Excludes assemblies discovered beneath a .claude path segment from MSTest assembly discovery, so sibling agent worktrees no longer pollute the discovered test-assembly set.
  • Extracts the Invoke-MSTest.ps1 discovery pipeline into Get-MSTestAssemblyPathList, returning an array at every cardinality so a single-match run no longer throws under Set-StrictMode -Version Latest.
  • Extracts the host-bound top-level body of Invoke-MSTest.ps1 into Invoke-MSTestMain, raising that file's command coverage from 68.89 percent to 94.00 percent without excluding it from measurement.
  • Documents two asserted design choices in the closure filter: the local-function exclusion, and the bare-name overload collision together with its fail-safe direction.

Why

Issue #733 consolidates seven independently reported findings against the scripts/vscode PowerShell MSTest and Cobertura tooling (source issues 529, 530, 531, 537, 559, 560, 713). All seven are small fixes confined to one subsystem, which is why they were merged into a single issue rather than pursued separately.

Two of the findings are arithmetic defects in Merge-CoberturaClassesByFilename: after merging duplicate class entries that share a filename, the enclosing <package> node kept the rate values from its input rather than the values implied by its merged contents, and the merge cloned only the primary class's <methods> subtree, silently discarding method entries contributed by the other members of the group.

One finding is an environment defect: assembly discovery walked into .claude/worktrees/, so a developer or agent with sibling worktrees checked out would discover another worktree's built test assemblies.

One finding is a strict-mode defect: the discovery pipeline in Invoke-MSTest.ps1 was not array-wrapped, so a run matching exactly one assembly produced a bare string, and the subsequent .Count access threw.

The remaining findings are documentation-only. A signature-based re-key of the closure presence set was evaluated and rejected as infeasible during research, because the closure-naming convention does not carry enough information to recover a parameter signature; the existing bare-name key is therefore documented along with the direction in which it fails.

What Changed

Cobertura merge arithmetic

  • scripts/vscode/Invoke-MSTestWithCoverage.PackageRate.ps1 (new): Get-CoberturaPackageLineSummary, a pure per-package accumulator returning the same shape and using the same rounding and zero-denominator fallback as the existing document-level summarizer.
  • scripts/vscode/Invoke-MSTestWithCoverage.Helpers.ps1: the document-level summarizer now calls the new helper per package rather than accumulating inline; Merge-CoberturaClassesByFilename gained the <methods> union and the package-rate recomputation.
  • scripts/vscode/Invoke-MSTestWithCoverage.Threshold.ps1 (new): Assert-CoberturaLineCoverageThreshold relocated verbatim out of the helpers file, which had reached the 500-line ceiling. Behavior is unchanged; the function is dot-sourced back so every existing caller and mock still resolves.

MSTest discovery

  • scripts/vscode/Invoke-MSTestWithCoverage.ps1: a fourth clause in the discovery Where-Object predicate excluding paths containing a .claude segment.
  • scripts/vscode/Invoke-MSTest.ps1: new Get-MSTestAssemblyPathList and Get-VsTestConsolePath seams, and the former top-level body moved into Invoke-MSTestMain. The top level is now a single guarded invocation.

Documentation

  • scripts/vscode/Invoke-MSTestWithCoverage.ClosureFilter.ps1: two comment-based-help addenda. No behavior change.

Tests

Five new Pester files under tests/scripts/vscode/: Invoke-MSTestWithCoverage.PackageRate.Tests.ps1, .Merge.Tests.ps1, .Threshold.Tests.ps1, Invoke-MSTest.AssemblyDiscovery.Tests.ps1, and Invoke-MSTest.Main.Tests.ps1. Three existing test files were extended.

Architecture / How It Fits Together

Invoke-MSTestWithCoverage.Helpers.ps1 remains the entry point that callers dot-source; it now dot-sources three siblings rather than one (ClosureFilter, PackageRate, Threshold). That keeps the public surface unchanged while holding every file under the 500-line ceiling.

The Cobertura pipeline order is unchanged and remains load-bearing: the exempt-closure filter runs before Merge-CoberturaClassesByFilename, because after the merge the surviving node is named for the declaring type and no longer carries the closure marker the filter matches on. An existing regression test pins that ordering.

Invoke-MSTest.ps1 now follows the same shape its sibling Invoke-MSTestWithCoverage.ps1 already used: a Main function holding the workflow, thin wrapper seams around each external executable, and a top-level guard that runs Main only when the script is invoked rather than dot-sourced. That guard is what allows the test file to dot-source the script and exercise the guards without launching vswhere.exe or vstest.console.exe.

Verification

Completed

  • Pester: 92 passed, 0 failed, 0 skipped across tests/scripts/vscode, against a 70-passing baseline. Each of the ten test files also passes standalone and in reverse-alphabetical order, so there is no inter-file order dependence.
  • PSScriptAnalyzer: 16 diagnostics (13 Warning, 3 Information, 0 Error), set-identical to the pre-change baseline in both directions. No new diagnostic was introduced and none was silently resolved.
  • PoshQC format: clean; the final iteration rewrote no file, verified by comparing SHA-256 hashes of all 21 in-scope files before and after.
  • Coverage, per production file, command coverage: Invoke-MSTestWithCoverage.Helpers.ps1 90.84, Invoke-MSTestWithCoverage.ps1 90.09, Invoke-MSTestWithCoverage.ClosureFilter.ps1 100, Invoke-MSTest.ps1 94.00 (from 68.89 at baseline), Invoke-MSTestWithCoverage.PackageRate.ps1 100, Invoke-MSTestWithCoverage.Threshold.ps1 88.24. Aggregate 93.10 percent over 565 commands. No production file was excluded from measurement and no threshold was changed.
  • Branch coverage: not emitted by Pester 5. This is a measured fact, not an omission; the uniform branch threshold does not apply to PowerShell for that reason.
  • Every fix carries fail-before and pass-after evidence recorded under the feature folder's evidence/regression-testing/ directory.
  • Feature review: zero blocking findings, zero FAIL, zero blocking-PARTIAL. All 8 acceptance criteria verified against their cited evidence.

Recommended

  • mcp__drm-copilot__run_poshqc_format, then run_poshqc_analyze, then run_poshqc_test, scoped to scripts/vscode and tests/scripts/vscode.
  • A direct Pester run over tests/scripts/vscode with code coverage enabled over the six production files, to reproduce the per-file figures above.

Backward Compatibility / Migration Notes

No breaking change to any public surface. Invoke-MSTestWithCoverage.Helpers.ps1 exposes the same functions to its callers; Assert-CoberturaLineCoverageThreshold moved file but is dot-sourced back, so its production caller and its existing mock both still resolve.

One behavior change is intentional and worth calling out for reviewers: Merge-CoberturaClassesByFilename previously retained only the primary class's <methods> subtree, and an existing test asserted exactly that. That test's assertion was deliberately reversed to assert the union, and its comment updated to say so. This is the fix for one of the seven findings, not an unintended regression.

Invoke-MSTest.ps1 runs its workflow only when invoked, not when dot-sourced. Any caller that relied on dot-sourcing the script to execute it must invoke it instead.

Risks and Mitigations

  • Risk: the <methods> union could duplicate a method entry if two members of a merged group legitimately contributed the same method name. Mitigation: research established that distinct group members do not share an identical method name under the closure-naming convention; a three-way merge fixture asserts all three distinct names appear with no duplication. Deduplication was deliberately not added, since a dedup key would mask the case rather than surface it.
  • Risk: the unary comma in Get-MSTestAssemblyPathList is easy to remove during a later cleanup, which would silently reintroduce the strict-mode defect. Mitigation: two assertions read the returned value's array shape without re-wrapping it, and were demonstrated to fail with the comma removed. The reason is also recorded in the function's comment-based help.
  • Risk: the Invoke-MSTestMain extraction is the largest structural change and touches the script's execution path. Mitigation: every guard, message, throw text, and ordering was moved verbatim, and the extracted path is now covered by tests that assert each failure mode.
  • Rollback: each phase is independently revertible; the documentation-only closure-filter change carries no behavior risk.

Review Guide

Suggested order:

  1. scripts/vscode/Invoke-MSTestWithCoverage.PackageRate.ps1 — small, self-contained, and the basis for the package-rate fix.
  2. Merge-CoberturaClassesByFilename in scripts/vscode/Invoke-MSTestWithCoverage.Helpers.ps1 — the union loop and the rate recomputation.
  3. scripts/vscode/Invoke-MSTest.ps1 — the two seams and the Main extraction. The unary comma on the Get-MSTestAssemblyPathList return is load-bearing; the comment-based help explains why.
  4. The remaining test files and the documentation addenda.

Mechanical, low-signal moves: Assert-CoberturaLineCoverageThreshold moving to Invoke-MSTestWithCoverage.Threshold.ps1, and two Describe blocks moving out of Invoke-MSTestWithCoverage.Helpers.Tests.ps1 into sibling files. Both were forced by the 500-line ceiling and neither changes behavior.

The bulk of the diff by line count is evidence artifacts under the feature folder, not code.

Follow-ups

GitHub Auto-close

drmoisan and others added 6 commits September 2, 2026 12:53
- Add issue documentation with consolidated findings
- Add feature specification and requirements
- Add implementation research findings
- Add atomic plan (preflight-cleared)

Refs: #733

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LTjXvNFHVh7Fo7kYGgWsx2
…ilter docs

Addresses the seven consolidated findings tracked under issue 733, all
confined to the scripts/vscode PowerShell MSTest/Cobertura tooling.

Production changes:
- Merge-CoberturaClassesByFilename now recomputes package-level line-rate
  and branch-rate after a class merge, via a new pure per-package helper
  Get-CoberturaPackageLineSummary, and unions <methods> entries across the
  merged group instead of cloning only the primary class's subtree.
- Invoke-MSTestWithCoverageMain excludes assemblies discovered beneath a
  .claude path segment, so sibling agent worktrees no longer pollute the
  discovered test-assembly set.
- Invoke-MSTest.ps1 extracts its discovery pipeline into
  Get-MSTestAssemblyPathList. The unary comma on the return is load-bearing:
  a function return enumerates its output, so a plain @(...) is unwrapped at
  the boundary and a single-match run yields a bare string whose .Count
  throws under Set-StrictMode -Version Latest.
- Invoke-MSTest.ps1 additionally extracts its host-bound top-level body into
  Invoke-MSTestMain, mirroring Invoke-MSTestWithCoverageMain, so the guards
  and error paths become reachable from Pester. Behavior is unchanged.
- ClosureFilter.ps1 receives documentation only: the local-function
  exclusion is ratified as intentional, and the bare-name overload collision
  is documented as failing in the safe under-exclusion direction.

Coverage: scripts/vscode/Invoke-MSTest.ps1 rises from 68.89 to 94.00 percent
command coverage. No production file was excluded from measurement and no
threshold was changed. All six production files are at or above 85 percent.

Tests: 92 passing, 0 failed, 0 skipped across tests/scripts/vscode, up from
a 70-passing baseline.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0195CL93aqgKF19nj9h6trbt
Records the review pass and closes the last acceptance criterion.

- policy-audit, code-review, and feature-audit artifacts: zero blocking
  findings, zero FAIL, zero blocking-PARTIAL.
- AC4 scope-boundary evidence recording the anchored diff against
  origin/main alongside the plan's literal porcelain check, with the
  reasoning for why the anchored form is the stronger check once a commit
  exists.
- spec.md AC4 and plan task P5-T9 checked off by the review pass, bringing
  acceptance criteria to 8 of 8.

The review reproduced the test, analyzer, coverage, and line-count figures
independently rather than accepting them, and recounted the AC4 prefix
distribution. That recount corrected this orchestrator's figures from
51/6/6 to the actual 49/6/8; the correction is recorded in the evidence
artifact rather than silently applied. The verdict was unaffected, since it
turns on the count of paths outside the allowed prefixes, which is zero.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0195CL93aqgKF19nj9h6trbt
@drmoisan
drmoisan merged commit b13d5b7 into main Sep 3, 2026
5 checks passed
drmoisan added a commit that referenced this pull request Sep 3, 2026
Records policy reads, feature-document reads (with #733/#748 drift notes),
branch/commit baseline (BASELINE_SHA=dc5e8c0f...), citation verification
(drifted-anchor branch taken for both checks), and PoshQC format/analyze/test
baselines for the ordering fix in Invoke-MSTestWithCoverage.ps1.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
drmoisan added a commit that referenced this pull request Sep 3, 2026
…efect

Adds the new It block to Describe 'Invoke-MSTestWithCoverageMain', inserted before
the #733 '.claude worktree' test per the plan's primary-anchor resolution (the
secondary anchor is now ambiguous after #733/#748 added an intervening test).
Confirms RED against the pre-fix order: only the new test fails
(Set-Content invoked 0 times, not 1), all 27 other tests in the file pass.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: coverage-cobertura-mstest-powershell-tooling-defects

1 participant