Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions skills/ce-babysit-pr/references/tick.md

Large diffs are not rendered by default.

9 changes: 5 additions & 4 deletions skills/ce-babysit-pr/references/watch-loop.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,8 @@ The round/time budget above is a **blunt cost floor**, not a convergence detecto

A loop can churn without finishing: CI **ping-pong** (fix A surfaces B, fix B brings A back — often an emergent trade-off), a review-bot **treadmill** (each commit spawns fresh nits), or **wrong-approach whack-a-mole** (each nit is valid but the approach, e.g. a regex, is the problem). A raw attempt counter can't tell these from *legitimate progress* (four independent failures each fixed once) — so the decision is **agent reasoning over the trajectory**, and the split is strict:

- **`pr-snapshot` (babysit) ships facts.** The `trajectory` block is deterministic and coarse: `check_recur_max`/`recurring_checks` (a check that failed → cleared → failed again on a *new* head; same-head flapping is excluded, so this is not flaky noise), `unresolved_trend` + `new_threads_this_tick` (backlog growing / fresh threads arriving), `stream_alternations` (ci↔review bouncing — cross-stream churn only babysit can see), `heads_since_progress` (heads moved without a new low in open problems). Babysit **never** labels this "non-convergence."
- **The leaf judges.** When a trigger fires (the thresholds are in SKILL.md Step 2 — the single source of truth; do not re-list them here), pass the trajectory into that tick's `ce-debug`/`ce-resolve-pr-feedback` as **mandatory input**. It must either demonstrate progress (name the invariant the next bounded fix resolves) or return a `needs-human` that **parks the whole stream** with a `decision_context` (the tension/root, options, tradeoffs, its lean).
- **`pr-snapshot` (babysit) ships facts.** The `trajectory` block is deterministic and coarse: `check_recur_max`/`recurring_checks` (a check that failed → cleared → failed again on a *new* head; same-head flapping is excluded, so this is not flaky noise), `unresolved_trend` + `new_threads_this_tick` (backlog growing / fresh threads arriving), `stream_alternations` (ci↔review bouncing — cross-stream churn only babysit can see), `heads_since_progress` (heads moved without a new low in open problems), `invariant_rounds` (resolver-supplied `invariant_key` values counted per unique head — the script never infers a key from paths, regexes, or comment text). Babysit **never** labels this "non-convergence."
- **The leaf judges.** When a trigger fires (the thresholds are in SKILL.md Step 2 / `references/tick.md` — the single source of truth; do not re-list them here), pass the trajectory into that tick's `ce-debug`/`ce-resolve-pr-feedback` as **mandatory input**. It must either demonstrate progress (name the invariant the next bounded fix resolves) or return a `needs-human` that **parks the whole stream** with a `decision_context` (the tension/root, options, tradeoffs, its lean). On a **fix** outcome, the leaf **returns** a stable `invariant_key` for each root it fixed, associated with the items that root covered; it does not own `pr-snapshot` and must not mark. Babysit persists each key at its existing atomic dispatched-mark boundary (`--thread` / `--comment` plus `--invariant-key`). A pass that would begin a third unique-head round for the same key (`invariant_rounds[].rounds >= 2`, since a round is recorded only after its fix completes) is itself a trigger: route the trajectory **before** another fix/commit/push so the leaf returns one approach-level `needs-human` instead of mutating again. Unrelated keys stay independently actionable. One- and two-round progress is not a blocker.

**The anti-cry-wolf line (put it to the leaf):** *progressive failure migration* — A fixed → B appears once → B fixed → done — is ordinary repair; **do not park.** *Oscillation* — A returns after B's fix, the failing set cycles, defects migrate X→Y→Z with the same invariant unsatisfied, or fix size grows superlinearly — is non-convergence; park. "We've tried a lot" is never enough.

Expand Down Expand Up @@ -141,7 +141,8 @@ State lives at `<scratch-root>/ce-babysit-pr/<host>-<owner>-<repo>-<pr>/state.js
"unresolved_series": [2, 3, 4],
"stream_series": ["ci", "review", "ci"],
"min_open_problems": 1,
"heads_since_progress": 0
"heads_since_progress": 0,
"invariant_heads": {}
}
}
```
Expand All @@ -154,7 +155,7 @@ A `check_key` is `"<workflow>/<name>"` (or `"<name>"` when there is no workflow)

The rule that makes ticks idempotent *and* crash-safe: **the snapshot never marks an item handled just from observing it.** An item leaves the actionable set only when the agent confirms it acted (via `mark`) or when remote truth removes it. So if a resolve/debug pass crashes, errors, or returns without finishing, the item is still actionable on the next tick — the loop cannot silently drop work.

- **Review threads.** A thread is actionable while it is unresolved and you have not recorded acting on it. After a resolve pass, `mark --thread <id> --disposition dispatched` handles an ordinary unresolved thread. The shared residual mark freezes its complete source observations without changing their dispositions. A later reviewer comment invalidates any covering decision and reopens every surviving sibling; it does not answer the decision. Every mark carries the active invocation tuple, so stale ticks cannot silence work in a newer invocation.
- **Review threads.** A thread is actionable while it is unresolved and you have not recorded acting on it. After a resolve pass, `mark --thread <id> --disposition dispatched` handles an ordinary unresolved thread; when the leaf returned an `invariant_key`, that same mark carries `--invariant-key`. The shared residual mark freezes its complete source observations without changing their dispositions. A later reviewer comment invalidates any covering decision and reopens every surviving sibling; it does not answer the decision. Every mark carries the active invocation tuple, so stale ticks cannot silence work in a newer invocation.
- **Non-thread feedback candidates** (top-level PR comments + review-submission bodies). These appear as `actionable.comments` when feedback has no inline thread. The detector excludes only empty bodies and never classifies content, authors, or posting surfaces; `ce-resolve` owns that judgment. Because there is no remote resolve, every passed candidate must either be marked `dispatched` or be covered by one validated current decision. A dispatched candidate stays silent across body edits because status bots routinely rewrite comments; an edit to a covered candidate invalidates the decision. A new comment has a new ID and is actionable. Both feedback surfaces remain one review stream for trajectory and backlog accounting.
- **CI checks.** A failing check on the current head is actionable until you `mark --check <key>`. A typed decision enters the same canonical set through the shared residual mark, never through check-specific decision state. A new head clears dispatch state, invalidates check-sourced residuals, and re-evaluates the decision against the new commit.

Expand Down
36 changes: 36 additions & 0 deletions skills/ce-babysit-pr/scripts/pr-snapshot
Original file line number Diff line number Diff line change
Expand Up @@ -1334,6 +1334,7 @@ def _empty_trajectory():
"heads_since_progress": 0, # head changes since progress (a new low OR something cleared)
"last_head": None, # head as of the last AGENT tick — hsp counts moves between ticks,
# NOT poll-observed head moves (state["head_sha"] advances on polls)
"invariant_heads": {}, # resolver-supplied invariant_key -> unique heads it was marked on
}


Expand All @@ -1350,6 +1351,36 @@ def _load_trajectory(state):
return tj


_INVARIANT_KEY_RE = re.compile(r"^[A-Za-z0-9._:-]{1,120}$")


def _record_invariant_round(state, key, head):
"""Count a resolver-supplied invariant against unique heads. Never infer keys."""
if not key:
return
if not _INVARIANT_KEY_RE.match(key):
raise SystemExit("--invariant-key must be 1-120 chars of A-Za-z0-9._:-")
if not head:
return
tj = _load_trajectory(state)
heads = tj.setdefault("invariant_heads", {})
seen = heads.setdefault(key, [])
if not isinstance(seen, list):
seen = []
heads[key] = seen
if head not in seen:
seen.append(head)


def _invariant_rounds_view(tj):
"""Public trajectory view: resolver keys counted per unique head."""
items = []
for key, heads in sorted((tj.get("invariant_heads") or {}).items()):
if isinstance(heads, list):
items.append({"key": key, "rounds": len(heads)})
return items


def _push_bounded(lst, item, cap):
"""Append to a sliding window that keeps only the last `cap` items."""
lst.append(item)
Expand Down Expand Up @@ -1484,6 +1515,7 @@ def _update_trajectory(state, head, new_checks, new_threads, new_feedback, actio
"new_threads_this_tick": len(new_arrivals),
"stream_alternations": _stream_alternations(tj["stream_series"]),
"heads_since_progress": tj["heads_since_progress"],
"invariant_rounds": _invariant_rounds_view(tj),
}


Expand Down Expand Up @@ -3431,6 +3463,8 @@ def cmd_mark(args):
entry["acted_identity"] = [args.acted_edit_id]
state["last_action"] = f"{args.disposition} {label} {item_id}"
marked = marked or item_id
if getattr(args, "invariant_key", None) and args.disposition == DISPOSITION_DISPATCHED:
_record_invariant_round(state, args.invariant_key, state.get("head_sha"))
if isinstance(marked, dict):
print(json.dumps({"marked": marked.get("id"), "decision": marked}, indent=2))
else:
Expand Down Expand Up @@ -3523,6 +3557,8 @@ def main():
help="exact current decision ID whose human answer is being recorded")
m.add_argument("--answer-file", default=None,
help="file containing the human answer to preserve until covered work moves")
m.add_argument("--invariant-key", default=None,
help="resolver-supplied review invariant; counted per unique head, never inferred")
m.set_defaults(func=cmd_mark)

w = sub.add_parser("watch")
Expand Down
2 changes: 1 addition & 1 deletion skills/ce-resolve-pr-feedback/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Evaluate and fix PR review feedback, then reply and resolve threads. The orchest

**Escalations never block.** `needs-human` is the escalation channel: leave the thread open with a natural reply and report the structured `decision_context`. Never pause mid-run to ask. That is what lets an autonomous caller — `ce-babysit-pr` running unattended, for example — loop this skill. Items that need a human decision come back as `needs-human` results for the caller to surface, rather than stalling the run; that includes a fix that would change behavior the author chose deliberately (see the rubric).

**`mode:pipeline`** (set by an orchestrator like `ce-babysit-pr` or `lfg`): the run is unattended, so **never call the blocking-question tool for any reason**, and read `references/pipeline-mode.md` before acting. It owns the two things ordinary mode leaves open. First, the open thread is the escalation ledger, so never write a PR-body residual section of your own. Second, the caller may pass a `trajectory` (`unresolved_trend`, `new_threads_this_tick`); when it shows that the feedback is not converging, answer with one approach-level `needs-human` rather than fixing nit after nit.
**`mode:pipeline`** (set by an orchestrator like `ce-babysit-pr` or `lfg`): the run is unattended, so **never call the blocking-question tool for any reason**, and read `references/pipeline-mode.md` before acting. It owns the two things ordinary mode leaves open. First, the open thread is the escalation ledger, so never write a PR-body residual section of your own. Second, the caller may pass a `trajectory` (`unresolved_trend`, `new_threads_this_tick`, `invariant_rounds`); when it shows that the feedback is not converging, or `invariant_rounds[].rounds >= 2` for a key this pass would continue (the next fix would be that key's third round) and that key's escalation is unanswered, answer with one approach-level `needs-human` rather than fixing nit after nit — an answered escalation authorizes the next action instead. On a fix outcome, return a stable `invariant_key` per fixed root; do not run `pr-snapshot`.

**Authority in pipeline mode.** Being invoked by an orchestrator is **not** itself authorization. You act under the **inherited** scope it holds from the user: **actions** = fix / commit / push / reply / resolve on the PR head, plus ticking a `## Unapplied review findings` bullet a committed fix closed (below); **exclusions** = merge, rebase, force-push, approve CI. You may *narrow* this (decline a fix, defer a `needs-human`) but never *broaden* it — if resolving a thread would require an excluded action, defer it as `needs-human` rather than perform it.

Expand Down
8 changes: 6 additions & 2 deletions skills/ce-resolve-pr-feedback/references/pipeline-mode.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,10 @@ Return the exact typed residual defined by the rubric: `type: "needs-human"`, `s

## 3. Non-convergence (wrong-approach cluster / treadmill)

When the caller passes a `trajectory` (rising `unresolved_trend`, `new_threads_this_tick > 0` across passes), check whether the feedback is *not converging*: several nits that share a **root** — the approach itself is the problem (canonical: "your regex misses case X" repeated for X after X, an unbounded whack-a-mole) — or a bot re-posting fresh nits every commit without end. If so, raise **one** approach-level `needs-human` about the root decision (e.g. "regex is the wrong tool here — options: exhaustive table / a real parser / accept known limits; lean: …") and stop fixing the individual instances, rather than dutifully fixing nit after nit.
When the caller passes a `trajectory` (rising `unresolved_trend`, `new_threads_this_tick > 0` across passes, or any `invariant_rounds[].rounds >= 2`), decide each root's standing before fixing anything on it:

Hold the anti-cry-wolf line: this fires only on a *demonstrated* shared root or a *demonstrated* treadmill across passes — a normal batch of unrelated valid nits is just fixed, one pass, as usual.
- **Escalate** — raise **one** approach-level `needs-human` about the root decision (e.g. "regex is the wrong tool here — options: exhaustive table / a real parser / accept known limits; lean: …") **before** any fix/commit/push — when the root's feedback is *demonstrably* not converging (several nits sharing one root, "your regex misses case X" repeated for X after X; or a bot re-posting fresh nits every commit without end), or when a fix would begin the root's third recorded round (`invariant_rounds[].rounds >= 2` for a key this pass would continue; rounds are recorded after a fix completes).
- **Execute an answered escalation** — when the open thread already carries a human's decision on the root, that answer authorizes the next action; apply it. Re-raising the same `needs-human` is rejected by the persistence layer.
- **Otherwise fix as usual** — a normal batch of unrelated valid nits is just fixed, one pass.

On a **fix** outcome, return a stable `invariant_key` (1–120 chars of `A-Za-z0-9._:-`) for **each** root a fix resolved, associated with the threads/comments that root covered — unrelated roots fixed in one pass carry distinct keys, so each accumulates its own rounds. Do not run `pr-snapshot`; the caller persists each key on that item's dispatched mark.
4 changes: 3 additions & 1 deletion tests/ce-babysit-pr-contract.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ const TRAJECTORY_FIELDS = [
"new_threads_this_tick",
"stream_alternations",
"heads_since_progress",
"invariant_rounds",
]
const BABYSIT_TRAJECTORY_REFS = [
"check_recur_max",
Expand All @@ -78,8 +79,9 @@ const BABYSIT_TRAJECTORY_REFS = [
"new_threads_this_tick",
"stream_alternations",
"heads_since_progress",
"invariant_rounds",
]
const CERESOLVE_TRAJECTORY_REFS = ["unresolved_trend", "new_threads_this_tick"]
const CERESOLVE_TRAJECTORY_REFS = ["unresolved_trend", "new_threads_this_tick", "invariant_rounds"]

function emittedTrajectoryKeys(script: string): string[] {
const fn = script.slice(script.indexOf("def _update_trajectory"))
Expand Down
21 changes: 21 additions & 0 deletions tests/ce-babysit-pr-snapshot.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3137,6 +3137,27 @@ print(json.dumps({
expect(d.trajectory.heads_since_progress).toBe(1) // head moved s1->s2 between agent ticks; not starved by the poll
}, 15000)

test("invariant_rounds count unique heads for a resolver-supplied key (#1575)", () => {
const dir = mkdtempSync(path.join(tmpdir(), "inv-rounds-"))
const state = path.join(dir, "state")
const t = (head: string) => ({
...FAILING,
head_sha: head,
checks: [],
threads: [{ thread_id: "T1", last_comment_id: head, last_comment_at: head }],
})
snapshot(state, fetchFile(dir, "h1.json", t("h1")))
mark(state, ["--thread", "T1", "--invariant-key", "golden-boundary"])
snapshot(state, fetchFile(dir, "h2.json", t("h2")))
mark(state, ["--thread", "T1", "--invariant-key", "golden-boundary"])
// Two recorded rounds is the trigger state: the next fix would be the third.
const atTrigger = snapshot(state, fetchFile(dir, "h3.json", t("h3")))
expect(atTrigger.trajectory.invariant_rounds).toEqual([{ key: "golden-boundary", rounds: 2 }])
mark(state, ["--thread", "T1", "--invariant-key", "golden-boundary"])
const d = snapshot(state, fetchFile(dir, "h3b.json", { ...FAILING, head_sha: "h3", checks: [], threads: [] }))
expect(d.trajectory.invariant_rounds).toEqual([{ key: "golden-boundary", rounds: 3 }])
})

test("check recurrence catches a CLEAR observed only on a watch poll (C1)", () => {
const sd = path.join(dir, "recurwatch")
const RED = { key: "CI/x", name: "x", status: "COMPLETED", conclusion: "FAILURE", details_url: "u" }
Expand Down