[earakely-scale]

What

A step completion that the task store refuses because a later attempt owns the instance now raises SupersededAttemptError instead of returning without a trace.

Before this, record_step_complete_sync declined such a write silently in two cases:

• the step was dispatched under an attempt_epoch older than the instance's;
• it journaled before a re-register moved the run's generation on.

The async record_step_complete also swallowed every exception. A dispatcher that recorded steps under a stale epoch therefore lost every step's writes with no error and no log at all. Its dependents then ran on a context without them, and failed far from the cause with messages like "sandbox not found in context".

The easiest way to get there is legitimate-looking: create the instance, then register it again before running it. The re-register advances the instance's attempt_epoch, which was undocumented, and a dispatcher that keeps its own epoch counter starting at 0 is fenced off from its first step.

Changes

• TaskInstanceStore.record_step_complete_sync: both refusals (stale epoch, stale generation) raise SupersededAttemptError, naming the instance, the step and the reason, and log it as a warning. The existing cleanup of the orphaned journal row still runs first. If that cleanup also fails, the refusal is still what's raised. The orphan row is harmless, because replay drops a step with no completion.
• record_step_complete (async): lets SupersededAttemptError through and still only logs any other store error, so it's fail-open as before. Retrying a superseded write can't land, so a dispatcher should end the step rather than retry it.
• register_task_instance: documents the contract. Re-registering advances attempt_epoch, and a caller that dispatches steps itself must record them under the returned instance's epoch.
• SupersededAttemptError is exported from agent_env.task.

Effect on Task.run

Task.run takes its epoch from its own registration and from each rollback, so a run of its own never trips this.

• A rolled-back straggler: its late write still lands in a future the drain already cancelled, and is discarded as before. Only the new warning notes it.
• A run superseded by a re-register elsewhere used to keep executing steps with every write dropped. It now stops at its first refused write. It still writes nothing to the run that replaced it, since the failure record stays fenced.
• The scheduler treats a SupersededAttemptError as terminal, even for a step with fail_task_on_error=False or a retry budget, since neither can make this run's writes land.
  • Without this, a tolerant step would let the superseded run carry on into its dependents.
  • A retry already failed, because the undo is fenced by the same epoch. It now fails without first attempting a futile rollback.

Tests

• Updated to assert the error instead of silence:
  • the zombie write after a rollback;
  • the writer that passed the fence before the undo;
  • the completion journaled before a re-register;
  • the superseded Task.run, which now ends with SupersededAttemptError at s0 instead of running on to s1.
• Fixed: test_a_retired_entry_is_never_replayed_into_the_new_run recorded its step under epoch 0 after two registrations. So the step never landed, and the test passed without exercising the replay it describes. It now records under the new run's epoch and asserts the step landed before undoing it.
• New:
  • a dispatcher still on the old epoch after a re-register is refused, and the registered epoch lands;
  • the async wrapper raises a fenced write but still only logs a store error;
  • the same refusal through the configured instance store, in test_record_step_concurrency.py;
  • a superseded run stops at a tolerant or retrying step instead of running its dependents;
  • a refusal whose journal-row cleanup also fails is still raised.
• Against the old code:
  • the six tests that assert the error fail;
  • the superseded run's test shows it running on into s1;
  • the tolerant-step and failed-cleanup tests fail too.

On a Linux VM (Ubuntu 24.04, Python 3.12) at the head commit, running CI's own commands:

• Lockfile and lint: uv sync --locked --extra dev, uv lock --check (before and after a version bump) and ruff check are clean.
• Unit tier (tst/unit plus the protocol tests, -n auto): 9351 passed, 0 failed. One earlier run hit test_cancelling_the_caller_of_abandon_still_abandons_the_run, which asserts on an async teardown's timing under full parallel load. It flakes on main too: 1 failure in 15 isolated runs there, versus 15/15 passing on this branch.
• Integration tier (not slow, with the registry service): 168 passed, 5 failed. The same 5 fail identically on main: the eval CLI tests and a bundle-init scopes test, none of which touch the task store.
• Plugin API check against main: no break to the plugin surface.

🤖 Generated with Claude Code

Confidence Score: 5/5

The PR appears safe to merge; both previous findings are fixed.

Summary

Fenced step completions now raise SupersededAttemptError rather than disappearing silently.

• The task store now reports step completions it refuses.
• Task runs stop when a step belongs to a newer attempt.

Verification was limited to reading code and tests; no tests were run.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Step completion] --> B{Later attempt owns instance?}
    B -- No --> C[Save completion]
    B -- Yes --> D[Try to remove orphan journal row]
    D --> E[Raise SupersededAttemptError]
    E --> F[Task.run stops new work]
    F --> G[Skip tolerance and retries]

Loading

_{Reviews (2) · Last reviewed commit: "End a superseded run at its step, and ke..." · Reviewed by Greptile}