Fold repetitive fan-out output (run/reboot/perform)

[plusky]

Summary

Folds repetitive success spam in parallel fan-out outputs (run, reboot, perform diagnostics, FanOut errors) so multi-host runs no longer dump N identical blocks. Verdicts and all failure output are never folded and stay at the head of the buffer.

Changes

• New crates/mtui-core/src/fold.rs: fold_output/can_fold_block — identical clean blocks share one banner plus …[output identical on N hosts folded]; repeated lines fold with counts. A block folds only if exit is 0, stderr is empty, every line is foldable (error keywords block), and output is non-empty.
• run.rs: per-host fold gated on can_fold_block; failed-host stdout/stderr always verbatim. reboot.rs: successes combine into one line, failures stay per-host. perform.rs: degradations print verbatim in place, clean sections fold. error.rs: identical fan-out messages group (a, b: boom, first-seen order).
• Grep-surface note: multi-host h1, h2:-> replaces per-host h1:->; rebooted & reconnected on h1, h2 replaces per-host lines; grouped FanOut may reorder RRIDs (first-seen). In-repo tests updated; docs/src/mcp.md shows verdicts only.

Checklist

• Commits follow Conventional Commits.
• cargo fmt --all --check is clean.
• cargo clippy --workspace --all-targets --all-features -- -D warnings is clean.
• cargo test --workspace passes.
• Feature matrix builds (--no-default-features, --all-features, compile-only).
• New/changed code covered (cargo llvm-cov -p mtui-core --lib: fold 100%, error 100%, run 99.3%, reboot 96.5%, perform 99.4%; misses pre-existing).
• User-visible changes recorded in CHANGELOG.md.
• Docs updated where relevant (no docs/src change needed; verdict-first ordering unchanged).

Related issues

Part 2 of 4 in the MCP token-reduction stack. Merge order: log-fold → json-crush → reread-dedup → token-nudges. Expected trivial conflicts with siblings on CHANGELOG.md and crates/mtui-mcp/tests/it.rs (one mod line each); resolved at merge time in that order.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.87%. Comparing base (98f3fa0) to head (da75ab6).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #608      +/-   ##
==========================================
+ Coverage   96.83%   96.87%   +0.03%     
==========================================
  Files         214      215       +1     
  Lines       62943    63743     +800     
==========================================
+ Hits        60951    61751     +800     
  Misses       1992     1992              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

[mimi1vx]

Blocker

• [crates/mtui-core/src/fold.rs:75] Empty-output commands never fold banners. In can_fold_block, empty stdout returns false. When executing quiet commands like run true across a 50-host fleet, 50 per-host banners are printed (h1:-> true [0], h2:-> true [0], etc.). Clean empty stdout (exit 0, empty stderr) should fold into a shared banner.
• [crates/mtui-core/src/fold.rs:45] Substring matching in is_foldable blocks clean output. is_foldable searches for lowercase substrings like "error" or "timeout". Benign package names (e.g. liberror) or innocuous messages containing these substrings prevent line folding. Use word-boundary or prefix checks instead.

[mimi1vx]

Blockers

1. [P0] commands/perform.rs::render_diagnostics — color is applied before the fold-safety keyword scan, so a colorized warning line escapes its own never-fold guarantee. highlight_warning splices ANSI codes (\x1b[33mwarning\x1b[39m) into the line before it is pushed into clean and scanned by fold::is_foldable. The byte immediately preceding "warning" is then m (a word byte), so contains_bounded's start-boundary check fails and the line is treated as ordinary foldable spam. Under ColorMode::Always/auto-tty, 3+ hosts hitting the same warning (e.g. an "Additional rpm output" zypper section — exactly this PR's target scenario) will silently fold, contradicting the CHANGELOG-documented invariant that warning/error/trace lines never fold. The added regression test (repetitive_diagnostics_fold_but_warnings_survive) only runs under ColorMode::Never, where yellow() is a no-op, so it cannot catch this. Fix: scan the pre-color text for foldability, or decide foldability before coloring. Add a ColorMode::Always regression test with 3+ identical highlighted warnings.
2. [P1] commands/run.rs block-grouping has no 3-host non-adjacent-regroup test. The algorithm (and its error.rs sibling, which does have fanout_grouping_keeps_first_seen_order) is documented to support h1, h3 regrouping around a differing h2, but every run.rs test uses exactly 2 hosts. A regression to "merge only with the immediately preceding group" would pass the entire current suite while breaking this guarantee. Add a 3-host test (h1/h3 match, h2 differs) asserting the combined banner reads h1, h3:-> with h2 printed separately.
3. [P2] error.rs::CommandError::FanOut Display format changed for a documented-frozen string. The module doc states these strings are pinned/grepped by tests; this PR changes the multi-failure aggregate from per-host a: msg; b: msg to deduped a, b: msg. Reasonable change, disclosed in the CHANGELOG, but confirm no downstream tooling parses this format positionally before merging.

[plusky]

CI fix (c4f3d93): the typos job flagged nd in the new word-boundary matcher — renamed to needle. No behavior change; fold tests green, clippy clean.

[mimi1vx]

BLOCKING: crates/mtui-core/src/fold.rs:787-805 — word-boundary keyword matching misses error/trace inside traceback/keyerror, so a Python traceback with exit 0 and empty stderr won't trip the fold-safety keyword net. Add traceback (and similar) to the keyword list.

BLOCKING: fold.rs:872-896 — runs of identical blank lines never fold since is_foldable("") returns false, undercutting the token-budget goal for blank-line-heavy output.

BLOCKING: crates/mtui-core/src/error.rs:689-703 — fanout_detail groups by to_string() equality, so unrelated CommandError variants with identical rendered text get merged into one group. Document this explicitly in code, not only the CHANGELOG.

[plusky]

Review-sweep responses (2026-09-12) — all findings addressed, each rework adversarially re-reviewed (GO) with red-proven tests, full workspace gates green. Cross-PR summary (per-PR diffs carry the detail):

• Fold repetitive fan-out output (run/reboot/perform) #608 log-fold: traceback-family keywords (traceback/stacktrace/keyerror/assertionerror), blank runs ≥3 fold, fanout_detail grouping-key doc+test.
• MCP token nudges: core profile, terser heavy-tool helps #609 token-nudges: get "full" restored, patch/write relpath examples, --json help made truthful for the pre-Row-budget crush for updates/list_refhosts/openqa_overview #610 tree (limit-capped array + limit hint, zero truncation claims); hunk yields to Row-budget crush for updates/list_refhosts/openqa_overview #610's exact text at land time.
• Row-budget crush for updates/list_refhosts/openqa_overview #610 json-crush: severity-ordered anomalies (u8, K/L kept counts in notice), window-before-probe in list_refhosts with pre-window totals in notice/footer.
• Collapse exact testreport_read re-reads to a notice #611 reread-dedup: additive deduped bool on read output (true on collapse, false on file text), documented; notice text unchanged.
• fix(mcp,testreport): stop steering transport unlock failures at --force #618 unlock-force: two-foreign-owners cancel test; ContendedSurface enum replaces paired check/scope strings, byte-identical rendering.
• feat(mcp): durable audit record of tool calls #620 audit-log+OTLP: real Debug negative pins, OTLP config_set body test, Full-via-dispatch test, always-fingerprint put/write/patch payloads as {bytes,sha256} (sha2, already in lock), resolve gated on auditing(), atomic 0600 create + tightening pre-existing sinks.
• fix(mcp): filter config_show on the headless surface #621 config-show: reuses pub sanitize_url (local dup deleted), edge-case parity tests, const subset assertions + url-defaults coverage test.

One systemic thread across #610/#611/#620: new I/O or truncation reuses the project primitives (spawn_blocking for blocking I/O, in-band truncation notices) — applied in each.

[mimi1vx]

Blocker (P0)

• crates/mtui-core/src/commands/perform.rs:43-52, fold.rs:24-46: Diagnostic warnings fold away under ANSI color due to escape codes defeating word-boundary matching. In render_diagnostics, diag.text.replace("warning", &session.display.yellow("warning")) colors the text before passing it to clean and fold_output. In fold.rs:24, is_word_char considers 'm' a word character (b.is_ascii_alphanumeric()). Because the SGR escape sequence immediately preceding "warning" ends in 'm' (e.g. \x1b[33mwarning\x1b[39m), contains_bounded evaluates is_word_char(hay[i - 1]) as true and fails to match "warning" or "warn". As a result, is_foldable returns true, and $\ge 3$ identical warnings from hosts fold away into …[N identical lines folded] under ColorMode::Always or Auto. Perform folding on the clean, uncolored diagnostic text before applying color formatting, or strip ANSI escape sequences in is_foldable before checking word boundaries.
• crates/mtui-core/src/commands/perform.rs:305-320: Regression test masks the ANSI color folding bug. repetitive_diagnostics_fold_but_warnings_survive hardcodes session_with_color(ColorMode::Never). Under ColorMode::Never, yellow("warning") is a no-op, allowing the test to pass. Under ColorMode::Always, identical highlighted warnings fold. Add a test case asserting that repeated highlighted warnings survive under ColorMode::Always.

Blocker (P1)

• crates/mtui-core/src/fold.rs:24-46: Remote command output containing ANSI color codes before signal keywords bypasses fold safety. Any subprocess output containing colored error messages (e.g. \x1b[31merror\x1b[0m) fails contains_bounded word-boundary checks because the byte preceding the keyword is 'm'. Strip ANSI escape sequences before running keyword checks in is_foldable.

Blocker (P2)

• crates/mtui-core/src/fold.rs:70: Python exception formats without traceback headers bypass keyword detection. Single-line exception formatters printing bare ValueError: ... without the traceback header evade contains_bounded(&lower, "error") because 'e' is preceded by 'u'. Consider adding common exception names (valueerror, typeerror, runtimeerror) to is_foldable.

[mimi1vx]

Blockers

• [P0] crates/mtui-core/src/fold.rs:29-46 — contains_bounded's word-boundary check fails on ordinary inflected forms of its own protected words: "3 errors found", "warnings: 2 issues", and Rust's real panic message "thread 'main' panicked at ..." all fail to match error/warning/panic and fold away silently, with exit 0. No test covers this.
• [P0] crates/mtui-core/src/fold.rs:100-124 — the fix for the traceback/keyerror blocker is an enumerated compound list, not a boundary-rule fix: IndexError, AttributeError, ImportError, FileNotFoundError, PermissionError, ConnectionError, NotImplementedError, ZeroDivisionError, NameError, SyntaxError, ModuleNotFoundError, OSError, LookupError still bypass detection and fold away with exit 0/empty stderr — same bug class as originally reported, still open for the general case.
• [P1] crates/mtui-core/src/fold.rs:55-74 — strip_ansi only strips CSI sequences, not OSC (ESC ] ... ESC \); harmless today by coincidence, fragile if that changes.
• [P1] crates/mtui-core/src/fold.rs:100-124 — critical (a common syslog level) is missing from the keyword list alongside panic/fatal/exception.
• [P1] crates/mtui-core/src/commands/run.rs:160-181 — per-host grouping loop is O(n²) worst case (all-distinct outputs); pre-existing, unlikely at real fleet sizes, flagging for awareness.
• [P2] Consider replacing the enumerated compound list with a general rule (e.g. treat a keyword match followed by an uppercase letter in the original-case text as still bounded) to close the bug class instead of just the named instances.

The two P0s are why this stays blocking: this PR's stated goal is "never fold a real failure," and both are live counterexamples with exit 0 that fold away today.

[mimi1vx]

Reviewed at head da75ab6 (full diff + changed files, not diff hunks alone).

No blockers.

Priority: Medium

• crates/mtui-core/src/error.rs:14-19 — fanout_detail's doc comment isn't separated from CommandError's, so rustdoc attaches the block to the helper and the public enum ends up undocumented. Add a blank line or move the fn below the enum.
• CHANGELOG.md:22 vs crates/mtui-core/src/fold.rs:144-145 — changelog says cancel/timeout lines never fold, but the denylist only has cancelled/canceled, not bare cancel; gerund/present-tense cancellation text would still fold.

Priority: Low

• fold.rs:70-89 — an unterminated ANSI CSI/OSC sequence at end of a truncated line drains to end-of-line instead of falling back to literal text.
• fold.rs:183-201 — fold-threshold tests cover n=2/n=5 but not the exact boundary n=3.
• fold.rs:110 — line.contains(":->") scans the un-stripped line, inconsistent with the rest of is_foldable.

Approving; happy to see the Medium items in a quick follow-up.