Skip to content

fix(autogen-ext): skip LangChain callback-manager (run_manager) when inferring tool args schema - #7994

Open
godququ5-code wants to merge 4 commits into
microsoft:mainfrom
godququ5-code:fix/langchain-adapter-skip-run-manager
Open

fix(autogen-ext): skip LangChain callback-manager (run_manager) when inferring tool args schema#7994
godququ5-code wants to merge 4 commits into
microsoft:mainfrom
godququ5-code:fix/langchain-adapter-skip-run-manager

Conversation

@godququ5-code

Copy link
Copy Markdown

Summary

Fixes #6385.

LangChainToolAdapter inferred a pydantic args model from the callable's signature when a tool provided no args_schema. LangChain injects a run_manager (CallbackManagerForToolRun / its async variant) into a tool's _run/_arun, and pydantic cannot generate a schema for that type, so constructing the adapter raised Unable to generate pydantic-core schema for ...CallbackManagerForToolRun (e.g. GoogleDriveSearchTool).

  • In _langchain_adapter.py, when inferring fields, skip parameters whose annotation resolves to a LangChain callback-manager type. Detection unwraps Optional[...]/Union[...] and also falls back to the conventional run_manager parameter name for unresolvable (string) annotations (the module uses from __future__ import annotations).
  • The adapter never passed a callback manager to the underlying tool, so omitting run_manager from the schema is correct — the tool still runs without it (it defaults to None).
  • Adds a regression test (test_langchain_tool_adapter_skips_run_manager) using a tool without an explicit args_schema.

Test plan

  • pytest python/packages/autogen-ext/tests/tools/test_langchain_tools.py — passes (2 tests).

…inferring tool args schema

LangChainToolAdapter inferred a pydantic args model from the callable's
signature when no args_schema was provided. LangChain injects a
`run_manager` (CallbackManagerForToolRun / its async variant) into a tool's
`_run`/\_arun`, and pydantic cannot generate a schema for that type, so
construction raised 'Unable to generate pydantic-core schema for
CallbackManagerForToolRun' (e.g. GoogleDriveSearchTool, microsoft#6385).

Skip parameters whose annotation resolves to a LangChain callback-manager
type (also by the conventional 'run_manager' name as a fallback for
unresolvable string annotations). The adapter never passed a callback
manager to the underlying tool, so omitting it from the schema is correct.

Adds a regression test using a tool without an explicit args_schema.

Fixes microsoft#6385

@gnanirahulnutakki gnanirahulnutakki left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing this adapter failure. I reproduced the original path and found two follow-ups before the change is complete: LangChain also reserves callbacks, which still triggers the same schema-generation crash, and the new test needs presence guards for optional schema keys to pass Pyright. I included minimal fixes and focused validation results inline.

# fallback for unresolvable (string) annotations, by the
# conventional parameter name.
annotation = type_hints.get(k, v.annotation)
if _is_callback_manager_annotation(annotation) or k == "run_manager":

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we filter callbacks by name here as well? The pinned langchain-core defines FILTERED_ARGS = ("run_manager", "callbacks"), and Callbacks is a sequence of callback handlers, so _is_callback_manager_annotation does not match it. I reproduced this with a no-schema tool whose _run accepts callbacks: Callbacks = None: constructing LangChainToolAdapter raises PydanticSchemaGenerationError for BaseCallbackHandler. Using k in ("run_manager", "callbacks") and extending this regression test to include callbacks fixes the crash; focused pytest, Pyright, mypy, and Ruff all pass.


schema = adapter.schema
assert schema["name"] == "NoSchema"
props = schema["parameters"]["properties"]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pyright reports three reportTypedDictNotRequiredAccess errors here because parameters, properties, and required are optional TypedDict keys. The earlier test in this file already uses the safe pattern: assert each key is present before indexing it. Adding assert "parameters" in schema, assert "properties" in schema["parameters"], and assert "required" in schema["parameters"] makes targeted Pyright pass with 0 errors.

…nal schema keys in test

Address review feedback on microsoft#7994:
- Filter the reserved 'callbacks' parameter (in addition to 'run_manager')
  when inferring the args schema. LangChain's FILTERED_ARGS is
  ('run_manager', 'callbacks'); a 'Callbacks' annotation is not matched by
  _is_callback_manager_annotation, so a no-schema tool whose _run accepts
  'callbacks' raised PydanticSchemaGenerationError. Filter it by name.
- Add presence assertions for the optional TypedDict keys ('parameters',
  'properties', 'required') in the existing test to satisfy Pyright
  (reportTypedDictNotRequiredAccess), matching the safe pattern used
  earlier in the file.
- Add a regression test (test_langchain_tool_adapter_skips_callbacks)
  covering the 'callbacks' case.

@gnanirahulnutakki gnanirahulnutakki left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed exact head c1b1fc24cfb4. Both findings are resolved: callbacks is filtered alongside run_manager, and the tests now guard optional TypedDict keys before indexing. The three focused tests, targeted Pyright, targeted mypy, and Ruff all pass locally.

@ErenAta16 ErenAta16 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified the detection logic against the cases it needs to cover, replicating the helper standalone:

bare CallbackManagerForToolRun          -> detected
Optional[CallbackManagerForToolRun]     -> detected
Union[AsyncCallbackManagerForToolRun, None] -> detected
int                                     -> not detected  (control)
Optional[int]                           -> not detected  (control)
inspect.Parameter.empty                 -> not detected

The Union unwrapping matters because Optional[CallbackManagerForToolRun] is how LangChain's own tool templates declare it, so handling only the bare type would have missed the common case. Covering types.UnionType alongside typing.Union also picks up X | None syntax, which is what a tool written on 3.10+ will use.

The name-based fallback is doing real work rather than being defensive padding, and it's worth spelling out why in case someone later reads it as redundant. This module has from __future__ import annotations, so annotations arrive as strings, and a string annotation can't be resolved by issubclass:

is_cb("Optional[CallbackManagerForToolRun]") -> False

That's precisely the situation the run_manager name check catches. Without it the type check would silently do nothing in exactly the module the fix targets, which would be a nasty way to have a "working" fix that isn't. Worth a brief comment saying the fallback exists because of the postponed-annotations import, since that link isn't obvious from the code alone.

Wrapping the langchain_core.callbacks.manager import in try/except ImportError is also right: this adapter is an optional extra, so the helper shouldn't hard-require the symbols to exist at import time.

Two small things:

The test tool declares run_manager on both _run and _arun. Worth confirming the fix path is exercised for the sync and async signatures separately, since _arun takes AsyncCallbackManagerForToolRun and a fix that only unwrapped the sync type would still pass a test that only checks the adapter constructs.

Since the reported failure is a hard exception at adapter-construction time (Unable to generate pydantic-core schema for ...CallbackManagerForToolRun), it'd be worth asserting the inferred schema doesn't contain run_manager in addition to asserting construction succeeds. Constructing successfully proves the crash is gone; asserting the field is absent proves it was excluded rather than coerced into something the model would then be asked to supply.

…allback

- assert the inferred schema explicitly excludes run_manager/callbacks
- add a test case covering the async CallbackManagerForToolRun variant so
  both sync and async callback-manager types are exercised separately
- document why the run_manager/callbacks name fallback exists: this module
  uses from __future__ import annotations, so annotations arrive as strings
  and get_type_hints can fail to resolve them, leaving string annotations
  that the issubclass check cannot match

Signed-off-by: godququ5-code <godququ5@gmail.com>
return a + b

async def _arun(
self, a: int, b: int, run_manager: Optional[AsyncCallbackManagerForToolRun] = None

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The repository-pinned formatter still fails on exact head 315e4f04: uv run --frozen ruff format --check packages/autogen-ext/tests/tools/test_langchain_tools.py reports this file would be reformatted and collapses both new _run/_arun signatures onto one line. Please run the pinned formatter before CI. The 4 focused tests, targeted Pyright and mypy, and Ruff lint otherwise pass locally.

Reformat the test file with the repo-pinned ruff (0.4.8). Pure
formatting change (signature line collapsing); the callback-manager
filtering logic and sync/async regression coverage are unchanged.

Signed-off-by: godququ5-code <godququ5@gmail.com>
@godququ5-code

Copy link
Copy Markdown
Author

Fixed the pinned Ruff formatting issue in test_langchain_tools.py. The callback-manager filtering logic and sync/async regression coverage are unchanged. Targeted tests and type checks pass locally.

@gnanirahulnutakki gnanirahulnutakki left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-verified exact head 7b5e64de. The incremental delta only applies the repository-pinned formatting: ruff format --check and Ruff lint now pass, all 4 focused adapter tests pass, Pyright reports 0 errors, and mypy reports no issues in the adapter and its test. The callback-manager filtering logic and sync/async regression coverage are unchanged from the previously verified head, so the formatter finding is resolved.

@ErenAta16

Copy link
Copy Markdown

Formatting fix noted, and good that the detection logic and the sync/async coverage stayed untouched in that pass. My earlier review points still stand as written; nothing in a Ruff-only commit changes them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Error in LangChainToolAdapter GoogleDriveSearchTool: Unable to generate pydantic-core schema

3 participants