Skip to content

Fix interrupted sweep presentation recovery - #13

Closed
spiky02plateau wants to merge 1 commit into
EveryInc:mainfrom
spiky02plateau:codex/recover-incomplete-sweeps
Closed

spiky02plateau wants to merge 1 commit into
EveryInc:mainfrom
spiky02plateau:codex/recover-incomplete-sweeps

Conversation

@spiky02plateau

Copy link
Copy Markdown
Contributor

Summary

  • stage source checkpoints until every kept judgment has a durable card
  • use deterministic run and batch identities so interrupted retries cannot duplicate presentation
  • validate complete source-run provenance for stable kept-judgment cardId obligations
  • surface pending recovery cards and add sweep:status, including legacy incomplete-batch detection
  • commit cards, checkpoints, source-run receipts, and current sweep state in one SQLite transaction
  • defer readable source mirrors until the SQLite transaction succeeds

Incident addressed

An Inbox source run and sweep batch could become current and advance the Gmail checkpoint before later card:upsert commands completed.
If the agent backend failed while preparing card payloads, kept judgments remained durable but no user-visible card represented them.

The new two-phase protocol keeps the prior batch and checkpoints current until all declared presentations exist.
It creates one recovery card for missing presentations and refuses collection work completion or a newer batch while recovery remains outstanding.

Compatibility

  • existing CLI commands remain available
  • new source runs require a stable, file-safe cardId on every keep judgment
  • sweep:status is additive and the CLI contract advances to 0.5
  • legacy run and batch payloads remain readable
  • pre-recovery current runs with kept judgments and no referencing card report attention_required
  • no SQLite schema migration is required because the new state is stored in existing JSON payload columns

Verification

  • pnpm install --frozen-lockfile
  • pnpm check - 221 passed, 1 expected local-Supabase skip
  • pnpm build
  • pnpm tend:build
  • pnpm tend:smoke
  • pnpm tend:package
  • pnpm audit - no known vulnerabilities

Regression coverage injects failures during recovery-card staging, final card persistence, and checkpoint commit.
It verifies SQLite rollback, unchanged checkpoints, visible pending state, deterministic retry IDs, duplicate-free recovery, evidence completeness, work-completion fencing, legacy incident detection, legitimate empty sweeps, and multi-source batches.

Tend feedback: feedback_a4d71304-f6c9-457e-9965-f4b012f9d220

dshipper added a commit that referenced this pull request Sep 13, 2026
…rds (#37)

A source run recorded for claimed recollection work advanced the source checkpoint before any card existed, so an agent that died between judging and presenting silently checkpointed kept mail as handled (the incident behind #13). Checkpoints recorded with --work now stay on the run until work:complete succeeds, inside the same transaction; completion is refused until every review judgment has a card the user can see now or has acted on since the run, and routine_action judgments are covered by cards or by routine action groups proposed since the sweep. A recollection that fails after recording its batch re-offers searching again; the newest run per source owns its checkpoint and a held checkpoint is not written over one that changed after the run was recorded; source and run mirrors are routed through the MirrorWriteCoordinator so refusals leave files untouched. Work-less runs and runs from older builds behave as before. Verified: CI green (build/test/smoke, Supabase, iOS); local replica with Bun 1.3.11 (466 tests on main plus this branch); four Codex review passes with all findings addressed.
@dshipper

Copy link
Copy Markdown
Collaborator

Thanks for the detailed report and PR. The incident was real: an agent that died between judging and presenting had already checkpointed the kept mail as handled, and nothing pointed at it afterwards. We fixed the underlying problem on main with a narrower change in #37: checkpoints recorded for claimed recollection work now stay on the run until work:complete succeeds, completion is refused until every review judgment has a card the user can actually see (and routine items have a card or a proposed routine group), a recollection that fails after recording its batch re-offers searching again, and a held checkpoint is never written over one that moved in the meantime. We deliberately kept the existing contract rather than requiring a cardId on every judgment, adding recovery cards, or blocking other work while a sweep is pending, which is why we went with our own fix instead of rebasing this one. Closing in favor of that; if you hit a case the narrower fix doesn't cover, please open an issue and we'll look.

@dshipper dshipper closed this Sep 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants