fix(autogen-ext): skip LangChain callback-manager (run_manager) when inferring tool args schema - #7994
Conversation
…inferring tool args schema LangChainToolAdapter inferred a pydantic args model from the callable's signature when no args_schema was provided. LangChain injects a `run_manager` (CallbackManagerForToolRun / its async variant) into a tool's `_run`/\_arun`, and pydantic cannot generate a schema for that type, so construction raised 'Unable to generate pydantic-core schema for CallbackManagerForToolRun' (e.g. GoogleDriveSearchTool, microsoft#6385). Skip parameters whose annotation resolves to a LangChain callback-manager type (also by the conventional 'run_manager' name as a fallback for unresolvable string annotations). The adapter never passed a callback manager to the underlying tool, so omitting it from the schema is correct. Adds a regression test using a tool without an explicit args_schema. Fixes microsoft#6385
gnanirahulnutakki
left a comment
There was a problem hiding this comment.
Thanks for fixing this adapter failure. I reproduced the original path and found two follow-ups before the change is complete: LangChain also reserves callbacks, which still triggers the same schema-generation crash, and the new test needs presence guards for optional schema keys to pass Pyright. I included minimal fixes and focused validation results inline.
| # fallback for unresolvable (string) annotations, by the | ||
| # conventional parameter name. | ||
| annotation = type_hints.get(k, v.annotation) | ||
| if _is_callback_manager_annotation(annotation) or k == "run_manager": |
There was a problem hiding this comment.
Could we filter callbacks by name here as well? The pinned langchain-core defines FILTERED_ARGS = ("run_manager", "callbacks"), and Callbacks is a sequence of callback handlers, so _is_callback_manager_annotation does not match it. I reproduced this with a no-schema tool whose _run accepts callbacks: Callbacks = None: constructing LangChainToolAdapter raises PydanticSchemaGenerationError for BaseCallbackHandler. Using k in ("run_manager", "callbacks") and extending this regression test to include callbacks fixes the crash; focused pytest, Pyright, mypy, and Ruff all pass.
|
|
||
| schema = adapter.schema | ||
| assert schema["name"] == "NoSchema" | ||
| props = schema["parameters"]["properties"] |
There was a problem hiding this comment.
Pyright reports three reportTypedDictNotRequiredAccess errors here because parameters, properties, and required are optional TypedDict keys. The earlier test in this file already uses the safe pattern: assert each key is present before indexing it. Adding assert "parameters" in schema, assert "properties" in schema["parameters"], and assert "required" in schema["parameters"] makes targeted Pyright pass with 0 errors.
…nal schema keys in test Address review feedback on microsoft#7994: - Filter the reserved 'callbacks' parameter (in addition to 'run_manager') when inferring the args schema. LangChain's FILTERED_ARGS is ('run_manager', 'callbacks'); a 'Callbacks' annotation is not matched by _is_callback_manager_annotation, so a no-schema tool whose _run accepts 'callbacks' raised PydanticSchemaGenerationError. Filter it by name. - Add presence assertions for the optional TypedDict keys ('parameters', 'properties', 'required') in the existing test to satisfy Pyright (reportTypedDictNotRequiredAccess), matching the safe pattern used earlier in the file. - Add a regression test (test_langchain_tool_adapter_skips_callbacks) covering the 'callbacks' case.
gnanirahulnutakki
left a comment
There was a problem hiding this comment.
Re-reviewed exact head c1b1fc24cfb4. Both findings are resolved: callbacks is filtered alongside run_manager, and the tests now guard optional TypedDict keys before indexing. The three focused tests, targeted Pyright, targeted mypy, and Ruff all pass locally.
ErenAta16
left a comment
There was a problem hiding this comment.
Verified the detection logic against the cases it needs to cover, replicating the helper standalone:
bare CallbackManagerForToolRun -> detected
Optional[CallbackManagerForToolRun] -> detected
Union[AsyncCallbackManagerForToolRun, None] -> detected
int -> not detected (control)
Optional[int] -> not detected (control)
inspect.Parameter.empty -> not detected
The Union unwrapping matters because Optional[CallbackManagerForToolRun] is how LangChain's own tool templates declare it, so handling only the bare type would have missed the common case. Covering types.UnionType alongside typing.Union also picks up X | None syntax, which is what a tool written on 3.10+ will use.
The name-based fallback is doing real work rather than being defensive padding, and it's worth spelling out why in case someone later reads it as redundant. This module has from __future__ import annotations, so annotations arrive as strings, and a string annotation can't be resolved by issubclass:
is_cb("Optional[CallbackManagerForToolRun]") -> False
That's precisely the situation the run_manager name check catches. Without it the type check would silently do nothing in exactly the module the fix targets, which would be a nasty way to have a "working" fix that isn't. Worth a brief comment saying the fallback exists because of the postponed-annotations import, since that link isn't obvious from the code alone.
Wrapping the langchain_core.callbacks.manager import in try/except ImportError is also right: this adapter is an optional extra, so the helper shouldn't hard-require the symbols to exist at import time.
Two small things:
The test tool declares run_manager on both _run and _arun. Worth confirming the fix path is exercised for the sync and async signatures separately, since _arun takes AsyncCallbackManagerForToolRun and a fix that only unwrapped the sync type would still pass a test that only checks the adapter constructs.
Since the reported failure is a hard exception at adapter-construction time (Unable to generate pydantic-core schema for ...CallbackManagerForToolRun), it'd be worth asserting the inferred schema doesn't contain run_manager in addition to asserting construction succeeds. Constructing successfully proves the crash is gone; asserting the field is absent proves it was excluded rather than coerced into something the model would then be asked to supply.
…allback - assert the inferred schema explicitly excludes run_manager/callbacks - add a test case covering the async CallbackManagerForToolRun variant so both sync and async callback-manager types are exercised separately - document why the run_manager/callbacks name fallback exists: this module uses from __future__ import annotations, so annotations arrive as strings and get_type_hints can fail to resolve them, leaving string annotations that the issubclass check cannot match Signed-off-by: godququ5-code <godququ5@gmail.com>
| return a + b | ||
|
|
||
| async def _arun( | ||
| self, a: int, b: int, run_manager: Optional[AsyncCallbackManagerForToolRun] = None |
There was a problem hiding this comment.
The repository-pinned formatter still fails on exact head 315e4f04: uv run --frozen ruff format --check packages/autogen-ext/tests/tools/test_langchain_tools.py reports this file would be reformatted and collapses both new _run/_arun signatures onto one line. Please run the pinned formatter before CI. The 4 focused tests, targeted Pyright and mypy, and Ruff lint otherwise pass locally.
Reformat the test file with the repo-pinned ruff (0.4.8). Pure formatting change (signature line collapsing); the callback-manager filtering logic and sync/async regression coverage are unchanged. Signed-off-by: godququ5-code <godququ5@gmail.com>
|
Fixed the pinned Ruff formatting issue in test_langchain_tools.py. The callback-manager filtering logic and sync/async regression coverage are unchanged. Targeted tests and type checks pass locally. |
gnanirahulnutakki
left a comment
There was a problem hiding this comment.
Re-verified exact head 7b5e64de. The incremental delta only applies the repository-pinned formatting: ruff format --check and Ruff lint now pass, all 4 focused adapter tests pass, Pyright reports 0 errors, and mypy reports no issues in the adapter and its test. The callback-manager filtering logic and sync/async regression coverage are unchanged from the previously verified head, so the formatter finding is resolved.
|
Formatting fix noted, and good that the detection logic and the sync/async coverage stayed untouched in that pass. My earlier review points still stand as written; nothing in a Ruff-only commit changes them. |
Summary
Fixes #6385.
LangChainToolAdapterinferred a pydantic args model from the callable's signature when a tool provided noargs_schema. LangChain injects arun_manager(CallbackManagerForToolRun/ its async variant) into a tool's_run/_arun, and pydantic cannot generate a schema for that type, so constructing the adapter raisedUnable to generate pydantic-core schema for ...CallbackManagerForToolRun(e.g.GoogleDriveSearchTool)._langchain_adapter.py, when inferring fields, skip parameters whose annotation resolves to a LangChain callback-manager type. Detection unwrapsOptional[...]/Union[...]and also falls back to the conventionalrun_managerparameter name for unresolvable (string) annotations (the module usesfrom __future__ import annotations).run_managerfrom the schema is correct — the tool still runs without it (it defaults toNone).test_langchain_tool_adapter_skips_run_manager) using a tool without an explicitargs_schema.Test plan
pytest python/packages/autogen-ext/tests/tools/test_langchain_tools.py— passes (2 tests).