Skip to content

fix(openai-agents): run agents as written, support agent.asTool() and remove agentAsTool (breaking) - #270

Open
campoy wants to merge 10 commits into
mainfrom
fix/openai-agents-typescript-native-as-tool
Open

campoy wants to merge 10 commits into
mainfrom
fix/openai-agents-typescript-native-as-tool

Conversation

@campoy

@campoy campoy commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

What was changed

  • TemporalOpenAIRunner runs the caller's agents as they are written. It no longer clones the agent graph to swap in Activity-backed models. Its inner Runner gets a model provider, ActivityBackedModelProvider, that sends every model name to the plugin's model Activity with the runner's Activity options, and the run's model.
  • Upstream passes the runner's config to nested agent.asTool() runs, so the framework's own asTool() now works in a Workflow. Nested runs take the run's model (for agents that set none), its modelSettings without toolChoice and parallelToolCalls, its context and sandbox, and the runner's Activity options.
  • agentAsTool, AgentAsToolOptions and the hook that passed the sandbox to it are removed.
  • Because agents aren't cloned, agent.on(...) listeners on the caller's agents fire, and a custom Handoff hands off to the agent its callback returns.
  • convertAgent becomes checkAgent, which checks the graph without changing it: raw functions in tools, a missing model, and Model objects, through handoffs and asTool() tools. It also rejects asTool() options that would run model calls or sandbox operations in the Workflow: runConfig.modelProvider (even undefined, which upstream spreads over the runner's provider), a Model object as runConfig.model, a sandbox client or live session not from temporalSandboxClient(), and a MemorySession.
  • The plugin also checks each agent just before it calls its model, by wrapping Agent.prototype.getSystemPrompt, which upstream calls once per model call, after it picks the model. This covers agents the check before the run can't see: one that a custom Handoff's callback returns, or one that an agent.on(...) listener changes. Its model must be a name, and its asTool() tools must pass the checks above. In the outer run, the run fails. In a nested run, the error goes back to the parent model as the tool's output, and the model isn't called. An agent from another copy of @openai/agents-core, such as one a dependency brings, is checked the same way: the plugin wraps that copy's Agent.prototype.getSystemPrompt before the agent's first model call, from the check before the run, from each run's start and from the runner's agent_start event.
  • Upstream reads an asTool() tool's runConfig and runOptions only when the nested run starts, after the tool's parameters schema, its inputBuilder and any agent_tool_start listener ran. So the plugin also checks each Runner run as it starts, by wrapping Runner.prototype.run in the Workflow: the run's model provider must be the runner's, its runConfig.model a name, its session not a MemorySession, and its sandbox config must pass the checks above. A nested run that fails this doesn't start, and the error is the tool's output. The run then uses a frozen copy of its sandbox config, which upstream reads as it goes, so a later change to the caller's object, for example in an agent_start listener, doesn't reach it.
  • A streamed model call with useLocalActivity now fails in ActivityBackedModel, not only in run(), so a nested asTool() run with onStream or on() handlers is rejected too.
  • As upstream, an asTool() tool returns a nested run's failure as its output, including a cancelled model Activity. If the run's cancellation scope was cancelled, run() now throws the CancelledFailure instead of returning, and a streamed run fails with it, even when the run ended on that output, for example with toolUseBehavior: 'stop_on_first_tool'.
  • To find the agent behind an asTool() tool, which upstream keeps private, the plugin wraps Agent.prototype.asTool in the Workflow before the Workflow module loads, and records each tool's agent and the runConfig and runOptions it was made with. The returned tool is not modified. A tool that looks like an asTool() tool but wasn't recorded, such as a copy, is rejected.
  • A ModelSummaryProvider still gets the running agent, tracked from the inner Runner's agent_start and agent_handoff events. A model call made during a tool call belongs to a nested asTool() run and gets agent: undefined, which its type allows.
  • The README shows agent.asTool(), lists the rejected nested setups, the check before each model call (including agents from another copy of @openai/agents-core) and the check when each run starts, explains cancellation when a run ends on a tool's output, adds agent lifecycle hooks to the feature table, and says runConfig.model applies to agents that set none.

Migration: replace agentAsTool(agent, { toolName, toolDescription }) with agent.asTool({ toolName, toolDescription }), created in Workflow code.

Why?

Three defects had the same root, the plugin's copies of the framework's objects:

With a provider on the runner, upstream's code does the work, including fields it passes to nested runs later.

Breaking changes

  • agentAsTool and AgentAsToolOptions are gone.

  • runConfig.model now applies only to agents that set no model of their own, as upstream. Before, it replaced every agent's model. Runs in progress and RunState saved before the upgrade follow the new rule from their next model call: an agent that sets its own model switches to it, without an error. Let those runs finish before upgrading, or use Worker Versioning.

  • Nested runs behave like upstream asTool(). A nested run's failure goes back to the parent model as the tool's output instead of failing the run. A nested approval pauses the parent run instead of failing it with NestedAgentInterruption. Nested model calls use the runner's Activity options.

  • A SandboxAgent returned by a custom Handoff's callback, in a run without runConfig.sandbox, fails the run with upstream's error, an AgentsWorkflowError with SandboxAgent execution requires RunConfig.sandbox.

  • An agent that a custom Handoff's callback returns, or that an agent.on(...) listener changes, now fails the run if its model is a Model object or one of its asTool() tools is rejected, including an agent from another copy of @openai/agents-core. An asTool() tool whose runConfig or runOptions a listener, its inputBuilder or its parameters schema changes to a rejected setup doesn't start its nested run; the error is the tool's output, as upstream.

  • An upstream Runner, or run(), can't run agents in a Workflow: its run is rejected unless its model provider is the plugin's. It had no Activity-backed provider before, so it failed with No default model provider set or ran models in the Workflow. Use TemporalOpenAIRunner.

  • A runConfig.sandbox.session that isn't from a Temporal sandbox client is rejected with SandboxConfigurationError.

  • asTool() tools must be created in Workflow code, so the plugin can record them.

  • Replay: runs that started before the upgrade keep working after users switch from agentAsTool to asTool(), except five kinds, which fail replay with [TMPRL1100]:

    • runs whose runner sets useLocalActivity: the nested model call was a regular Activity and is now local;
    • runs that called agent.asTool() before this change: the nested agent never ran, and now it schedules a model Activity;
    • agentAsTool runs whose nested run failed and whose Workflow recorded anything after the failure, such as recovery work in a catch: the failure now goes back to the parent model, which schedules a model Activity;
    • runs whose commands depend on agent.on(...) listeners, for example an agent_end listener that schedules an Activity: the listeners didn't fire before;
    • runs through a custom Handoff whose callback returns a different agent from the one it was declared with, when the two agents call different Activities: the run went to the declared agent before.

    A run still waiting after such a failure, for example in its catch handler, replays without an error but changes its result: it continues through the parent model and skips its recovery. A replay check of old histories doesn't find these runs, or the model switch above. Let all of these runs finish before upgrading, or use Worker Versioning.

Checklist

  1. Closes [Bug] openai-agents (TypeScript): agentAsTool runs drop the run's model, model settings and runner defaults #243, closes [Bug] openai-agents (TypeScript): agent.on(...) lifecycle hooks never run through TemporalOpenAIRunner #235, closes [Bug] openai-agents (TypeScript): TemporalOpenAIRunner ignores the agent a custom Handoff's callback returns #237. Replaces fix(openai-agents): give agentAsTool runs the run model, model settings and runner defaults #244, Run agent.on(...) lifecycle hooks in TemporalOpenAIRunner #236 and fix(openai-agents): hand off to the agent a custom Handoff returns #238.

  2. How was this tested:

    • New Worker tests:

      Each of these fails without the change.

    • A ModelSummaryProvider gets the running agent; nested agent.asTool() calls get none passes before and after the clone removal, with the same labels.

    • Tests from the review rounds, which pass on this design:

      • a test that each rejected nested setup fails before any Activity is scheduled;
      • Agents picked or changed during the run are checked before they call their model. It covers a Model object from a custom Handoff's callback, a callback-returned agent with an asTool() tool that sets runConfig.modelProvider, and agent_start listeners in the outer and a nested run. It also covers an asTool() tool whose captured runConfig an agent_tool_start listener changes to a modelProvider, to modelProvider: undefined, or to a Model object for a model-less nested agent; an inputBuilder that sets a modelProvider; and asTool({ runConfig: { modelProvider: undefined } }), with a default model provider registered. Each is rejected before the nested model is called, and each replays. Without the checks, the inline model runs in the Workflow. A control, a listener that changes the tool's runConfig.model to another name, schedules two model Activities and the nested call uses the new name. A parameters Zod refinement that sets a modelProvider was checked the same way in a local probe (the plugin has no direct zod dependency): the inline model ran without the run check and was rejected with it;
      • Agents from another copy of @openai/agents-core are checked before they call their model bundles a second, unmodified copy of @openai/agents-core 0.14.3. It covers three ways such an agent gets a Model object: an agent_start listener, a custom Handoff's callback, and a tool between two turns. Each is rejected before the model is called (0, 1 and 1 model Activities), and each replays. Without the fix, all three ran the inline model in the Workflow. A control agent from that copy with a string model completes through one model Activity;
      • A sandbox client that an agent_start listener replaces with a raw one never runs: the run completes through sandbox Activities; without the frozen copy, the raw client ran in the Workflow. The agent_tool_start variant, on an asTool() tool's runOptions.sandbox, is rejected as the tool's output without calling the backend. Both replay. A raw runConfig.sandbox.session is rejected; without the check it ran in the Workflow;
      • cancelling the Workflow during a nested model call ends it CANCELLED with stop_on_first_tool, plain and streamed. Without the fix, both completed with ActivityFailure: Activity cancelled as output. Two controls, the default loop and stop_on_first_tool without cancelling, pass with and without the fix;
      • resuming after continueAsNew with an agent that is both a handoff target and an asTool() source;
      • a nested approval;
      • nested streaming to onStream and to on() handlers, including a handler registered after the run starts;
      • nested streaming with useLocalActivity: the nested model isn't called, and the tool's output is the StreamingLocalActivityUnsupported message. Without the check, the nested model streamed through a local Activity;
      • reusing an asTool() options object.
    • check-agent.test.ts covers the checks, the asTool() record, the check before each model call (also on another copy's Agent class, installed once) and the check when each run starts (each installed once, inactive outside a Workflow).

    • Locally, in a Linux container with Node 24 and pnpm 10: make lint (the 8 warnings already on main, none in changed code), make test (227 tests), make build, and Prettier with the SDK settings pass. The Node 22, macOS and Windows cells did not run locally.

    • Replay check, with Workers with no Workflow cache, on runs with no run config, a run model and settings, a local-Activity runner and native asTool():

      • plain and run-config runs replay across main (agentAsTool) and this change (asTool()), in both directions;
      • local-Activity and native-asTool() runs fail with [TMPRL1100], as listed above;
      • a run whose nested model call failed with HTTP 400, and whose Workflow caught the failure and started a recovery Timer, fails with [TMPRL1100] in both directions; the same run held in its catch handler on main completed on this change through the parent model, without its recovery;
      • a run held inside its nested run carried over in both directions;
      • a control that switches to local Activities fails on both;
      • in review, an agent_end listener that schedules an Activity, and a custom Handoff that picks an agent with a different Activity tool, failed with [TMPRL1100] in both directions, while their self-replays passed; they are listed above.

      The same histories, and runs held across versions, replay between the first commit and the clone-free code before the review-round merge. The merge doesn't change which commands are scheduled, and neither do the checks added since: they schedule nothing for valid setups.

    • Cancelling a Workflow during a nested model call fails it on main and cancels it with this change.

  3. Any docs updates needed?
    The README is updated here. The docs page and the samples are updated in Show agent.asTool() instead of agentAsTool on the TypeScript OpenAI Agents page documentation#5597 and Use agent.asTool() in the OpenAI Agents agent-patterns sample samples-typescript#534, drafts until this is released.

Not covered:

  • The check before each model call wraps getSystemPrompt. An Agent subclass that overrides it without calling super skips the check.
  • The check when each run starts wraps Runner.prototype.run. Code that replaces run on the prototype or on a Runner instance skips it.
  • A Handoff object from another copy of @openai/agents-core in an agent's handoffs: upstream's run loop doesn't recognize it (it checks instanceof Handoff), so the run fails in upstream before any model call.
  • Usage across nested runs.

…agentAsTool (breaking)

TemporalOpenAIRunner now gives its inner Runner a model provider that
sends every model name to the model Activity, plus the run's model.
Upstream passes the runner's config to nested agent.asTool() runs, so
the framework's own asTool() works in a Workflow: its model calls are
model Activities with the run's model and model settings and the
runner's Activity options.

agentAsTool and AgentAsToolOptions are removed. Use agent.asTool().
@campoy

campoy commented Oct 29, 2026

Copy link
Copy Markdown
Contributor Author

Let's take option 2: keep native agent.asTool(), with a supported upstream hook as the longer-term replacement for prototype interception.

For now, wrap Agent.prototype.asTool from the plugin's Workflow module to record each tool's source agent and options. Use that information to validate nested configurations and rebuild tools against a consistent converted agent graph.

The main benefit is the developer experience: applications moving from native execution to Temporal can keep their agent-tool declarations. We also reuse upstream's tool behavior rather than maintain a separate implementation. The plugin still needs its Temporal runner, Worker setup and Activity boundaries; native syntax does not remove those requirements. Existing plugin users would still migrate from agentAsTool to agent.asTool() as described in this PR.

The tradeoff is that the plugin owns a temporary patch to a shared upstream class in the Workflow runtime. That makes initialization order and SDK compatibility our responsibility. The implementation must install interception before tools are created, handle repeated initialization safely, and account for subclasses and duplicate SDK module copies. Tools whose source was not recorded must be rejected with a clear error.

This choice does not accept the two review findings as documented limitations. Before merging, we need to:

  • Reject nested model objects and provider overrides that would bypass model Activities.
  • Keep identities consistent when an agent is both a handoff target and a tool, including approval restoration and older saved state.
  • Verify that rebuilding tools preserves the run's model/settings, Activity defaults and supported upstream tool behavior, including approvals and streaming. Replay compatibility needs its own checks.

In parallel, request a supported upstream way to inspect an agent tool's source and options and rebuild or rebind it while preserving identity and approval restoration. Once that exists, we can replace the interception inside the plugin while keeping the same native API and tested behavior. Callers should not need another migration for that internal change.

campoy and others added 3 commits October 28, 2026 23:56
TemporalOpenAIRunner no longer clones the agent graph to swap in
Activity-backed models. Its inner Runner's model provider already sends
every model name to the model Activity, so agents run as the caller
wrote them. That also fixes two defects of the clones: agent.on(...)
listeners on the caller's agents now fire (#235), and a custom Handoff
now hands off to the agent its callback returns (#237).

convertAgent becomes checkAgent, which runs the same checks, including
the nested agent.asTool() checks, without changing the graph. asTool()
tools are no longer rebuilt, so the asTool() record keeps only the agent
and the runConfig and runOptions the tool was made with. The same-name
check goes: without clones, handoff and nested agents keep their own
string models.

A ModelSummaryProvider still gets the running agent, tracked from the
inner Runner's agent_start and agent_handoff events; a model call during
a tool call belongs to a nested agent.asTool() run and gets none.

As upstream, runConfig.model now applies only to agents that set no
model of their own.
@campoy campoy changed the title fix(openai-agents): run agent.asTool() through the plugin and remove agentAsTool (breaking) fix(openai-agents): run agents as written, support agent.asTool() and remove agentAsTool (breaking) Oct 29, 2026
@campoy
campoy marked this pull request as ready for review October 29, 2026 19:26
@campoy
campoy requested a review from a team as a code owner October 29, 2026 19:26

@brianstrauch brianstrauch left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lint, build, and all 261 existing tests pass on Node 24. A focused Worker regression probe reproduces the nested-streaming restriction bypass noted inline.

const innerRunner = new Runner({
...runnerConfigOverrides,
model: modelOverride,
modelProvider: new ActivityBackedModelProvider(this.modelParams, () =>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Enforce the local Activity streaming restriction for nested tools

With useLocalActivity: true and a nested agent.asTool({ onStream }), the child run goes directly through upstream Runner, bypassing the StreamingLocalActivityUnsupported guard in TemporalOpenAIRunner.run(). This provider inherits the local Activity options and schedules invokeModelStreamActivity locally. I reproduced this by running the existing agentToolStreamWorkflow with modelParams.useLocalActivity: true: the provider's getStreamedResponse() runs once, both stream handlers receive the answer, and the parent succeeds instead of rejecting the unsupported combination. Please enforce the restriction in ActivityBackedModel.getStreamedResponse() or the shared run check, and add nested regression coverage.

@campoy campoy Oct 30, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, thanks. Fixed in 3f11f6c: ActivityBackedModel.getStreamedResponse() now rejects useLocalActivity, next to its streaming-topic check, so every streamed model call has the check, including nested agent.asTool() runs. The runner's own check stays, so a top-level streamed run still fails before it starts.

One difference from the top-level case: as upstream, asTool() returns a nested run's failure to the parent model as the tool's output, so the parent run goes on. The nested model is never called. This matches the other checks that run inside a nested run. Rejecting the parent before it starts would need every on() handler up front, and handlers can be added after the run starts.

The new Worker test, A nested agent.asTool() run fails to stream when model calls are local Activities, runs agentToolStreamWorkflow with useLocalActivity: true. It fails without the check (the nested model streams) and passes with it.

…agent.asTool() runs

A nested agent.asTool() run with onStream or on() handlers streams through
upstream's Runner, so it skipped the runner's useLocalActivity check and ran
the streamed model call as a local Activity. Check in
ActivityBackedModel.getStreamedResponse(), which every streamed model call
reaches. As upstream, the nested failure is the tool's output.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants