Repository navigation
fix(strands): let the agent trim the conversation on a context overflow (breaking) - #279
Closed
campoy wants to merge 3 commits into
Conversation
…ow (breaking) A ContextWindowOverflowError from the model activity reached the agent as an ActivityFailure, which Strands wraps in a plain ModelError. The conversation manager only trims on ContextWindowOverflowError, so it never ran: the error left agent.invoke and, uncaught, failed the workflow task on every attempt. TemporalModel.stream now rethrows a failure of type ContextWindowOverflowError, which the plugin's activity interceptor produces, as Strands' own ContextWindowOverflowError. The conversation manager then trims and calls the model again, as outside Temporal. Breaking change: a run whose workflow code caught that overflow before this version calls the model again on replay and fails with a nondeterminism error. A run whose workflow task kept failing on it continues and trims.
…places-caller-failure-converter' into fix/strands-typescript-context-overflow-recovery-skipped # Conflicts: # typescript/strands-agents/README.md # typescript/strands-agents/src/__tests__/workflows/strands.ts
…places-caller-failure-converter' into fix/strands-typescript-context-overflow-recovery-skipped
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was changed
TemporalModel.streamrethrows a model Activity failure of typeContextWindowOverflowErroras Strands' ownContextWindowOverflowError. The agent's conversation manager then trims the conversation and calls the model again, as it does outside Temporal. The README's Retries section says so, with an upgrade note.Stacked on #200, whose Activity interceptor sends the overflow as a non-retryable failure of that type. Review base: #200's branch.
Why?
Strands' conversation managers recover from a context overflow by trimming the oldest messages and retrying, but only when the model call fails with
ContextWindowOverflowError. Through the plugin, the overflow reached the agent as anActivityFailure, which Strands wraps in a plainModelError. Nothing was trimmed: the error leftagent.invokeand, uncaught, failed the Workflow Task on every attempt, so a chat whose history outgrew the model's context stopped answering. A code owner's review of #200 pointed out that nothing decoded the typed failure back.Breaking change
[TMPRL1100].Upgrade: let runs that caught a context overflow finish before you upgrade, or keep them on the earlier version with Worker Versioning.
Checklist
No issue of its own; follow-up to fix(strands): keep the caller's failure converter and turn Strands errors into Temporal failures (breaking) #200 ([Bug] strands-agents (TypeScript): StrandsPlugin replaces the caller's failure converter #199).
How was this tested:
a context overflow in the model activity lets the agent trim the conversation: a stub model holds six messages. Three turns fill them, and the fourth overflows; the agent trims and answers, keeping six messages. With fix(strands): keep the caller's failure converter and turn Strands errors into Temporal failures (breaking) #200's head (cb4cfa4) the Workflow Task fails withActivity task faileduntil the run times out.cb4cfa4without an overflow replay on this change, and the other way around.cb4cfa4run that caught the overflow fails with[TMPRL1100]on this change.cb4cfa4run whose Workflow Task kept failing on the overflow completes on this change with the trimmed conversation, and its history replays here.cb4cfa4: a trimmed history fails replay there, since that code lets the overflow escape again.[TMPRL1100].make sync,make lint,make test(39 tests; 40 after merging fix(strands): keep the caller's failure converter and turn Strands errors into Temporal failures (breaking) #200'scaae774into this branch asf1744e4),make build, the npm package check, the smoke install andcheck_conventions.pypass, and Prettier with the SDK settings reports the changed files clean. The Node 22, macOS and Windows cells did not run locally.Any docs updates needed?
The plugin README is updated. Release notes must say that context overflows are now trimmed and which runs started before the upgrade do not replay on it.
Not covered
agent.invoke. fix(strands): keep the caller's failure converter and turn Strands errors into Temporal failures (breaking) #200's Workflow interceptor then fails the run with a non-retryableContextWindowOverflowError. Not run.invokeModelStreaming) goes through the same catch. By source, not run.