[integrations][java][python] Apply Anthropic native structured output

[weiqingy]

Linked issue: #280

Purpose of change

A caller can now pass an output schema to the Anthropic connection and get back a response the provider itself enforces.

Where the schema cannot be sent natively, the request goes out unconstrained and the existing prompt-engineering fallback takes over. Before this change both languages rejected the schema outright, so the capability was out of reach even on models that support it.

The same code path also decides json_prefill. That parameter is now an explicit opt-in with one contract in both languages: off by default, and applied only when nothing on the request conflicts with it.

Runtime flow

chat hands the schema to buildRequest, which resolves the effective model and then makes two decisions in order. First, whether the schema can be sent natively, in which case it is translated into an output_config. Second, whether to prefill, judged on what the request ended up carrying rather than on what was supplied. The table below gives the conditions on both.

buildRequest returns that prefill decision along with the request, so the response conversion reconstructs the leading { from the decision the request was actually built with. Python makes both decisions inside the single call that builds the request and converts the response.

Key decisions

Which models are capable is a generational rule, not a list of snapshots. Anthropic documents structured output as generally available from Claude 4.5 onward, plus Mythos Preview. That splits the allowlist in two.

Names from the 4.6 generation on, such as claude-opus-4-6, carry no date, so the name is itself the snapshot and is matched exactly. The three 4.5-generation names, claude-opus-4-5, claude-sonnet-4-5 and claude-haiku-4-5, are aliases that front a dated snapshot, so each one matches either the alias itself or a name that continues it after a -. That admits claude-sonnet-4-5-20250929. It excludes a name that extends the alias without the separator, which is a different minor version.

The prefill guard keeps its own list of models, rather than deriving from the structured-output allowlist whose names it currently duplicates. The two encode different boundaries: structured output starts at 4.5, prefill rejection at 4.6. A shared list would withdraw the prefill from the whole 4.5 generation the first time either boundary moves.

A caller's own output_config wins. When a caller supplies one, no derived config is written, and the prefill is suppressed as well. Overwriting the exact parameter a caller set would throw away a deliberate choice, without an error and without any other trace.

Implementation Description

Interaction decisions

A schema means a POJO Class in Java, or a BaseModel wrapped in OutputSchema in Python. json_prefill is off by default, so every row below assumes the caller turned it on.

Output schema	Effective model	Caller output_config	Tools	Request carries	Prefill
POJO / BaseModel	capable	none	none	derived output_config	suppressed
POJO / BaseModel	capable	supplied	none	the caller's config, nothing derived	suppressed
POJO / BaseModel	not capable	none	none	no config, prompt fallback	applied
RowTypeInfo or another shape	4.5 generation	none	none	no config, prompt fallback	applied
none	any	supplied	none	the caller's config	suppressed
any	any	any	present	schema handling unchanged	suppressed
any	rejects prefill	any	any	schema handling unchanged	suppressed

The models that reject prefilling are the 4.6 generation and later, Mythos Preview, Fable 5 and Mythos 5. The RowTypeInfo row is where the two lists visibly differ: a 4.5-generation model such as claude-sonnet-4-5 is capable of structured output and still accepts a prefill, so a schema that cannot be sent natively there keeps the prefill its fallback depends on.

Behavioral contracts

1. A POJO or BaseModel schema, on a capable model, with no caller output_config, produces a request carrying the derived config.
2. A schema of any other shape produces no derived config.
3. A model name in neither allowlist, including null, counts as not capable and produces no derived config.
4. A caller-supplied output_config is preserved unchanged, and no derived config is written alongside it.
5. json_prefill is off unless the caller turns it on. An opted-in prefill is applied only when the request has no tools, has no output_config from either source, and targets a model that accepts prefilling.
6. The response reconstructs the leading { exactly when the request carried the prefill.
7. Java and Python read the same allowlists and apply the same rules, so the same model name and request shape decide the same way in both.

Failure behavior

No configuration on this path raises. An incapable model, an untranslatable schema, or a caller-supplied output_config each only change what the request carries, so the risks here are silent ones.

An unrecognized model name is treated differently by the two guards, and that is deliberate. For structured output it counts as not capable, so the request falls back to prompt engineering. For prefill it counts as supported, so an opted-in request is sent with the prefill, and a model that does reject prefilling answers with a 400. The two documented rules have opposite shapes, which is why the two defaults point in opposite directions.

A name that arrives with a provider prefix or a version suffix, such as a Bedrock anthropic.claude-opus-5, is compared against bare names. It falls to both of those defaults.

A caller who forces the NATIVE strategy on a rejected model gets the prompt fallback rather than an error. The requested strategy is not visible at this layer, so the code cannot tell an explicit NATIVE from one that merely resolved that way. Marked TODO(#912) in both languages, matching the OpenAI and Azure connections.

Provider errors are unchanged. Java wraps them, Python lets the SDK exception through. That difference predates this change.

Tests

The suites run offline: 60 Java cases and 66 Python. The Java side uses no mocking framework, and builds its response objects from the SDK's own public builders.

Every contract above is pinned in both languages:

Contract	Java	Python
1. Derived config on a capable model	✅	✅
2. No derived config for another schema shape	✅	✅
3. Unknown or null model is not capable	✅	✅
4. A caller's output_config is preserved	✅	✅
5. Prefill opt-in, and what withholds it	✅	✅
6. Leading { reconstructed exactly when prefilled	✅	✅
7. Java and Python decide alike	✅	✅

Every row of the decision table has a case in both suites, built through the real request builder and asserted on what the request ends up carrying. Three things that table does not show:

• The capable case runs against claude-sonnet-4-5 and claude-opus-4-6, one name from each matching rule, so narrowing the check to either branch fails a test.
• Every prefill case asserts the converted response as well as the request, so the decision and the reconstruction are never checked apart.
• Both model lists are enumerated name by name, as literals written independently of the production ones, because a wrong entry is otherwise invisible until a provider rejects a request.

Not covered: no test makes a live request, so these pin what the connection sends and never that the provider accepts it. The Python dependency floor is unenforced, because the client is mocked. Provider-prefixed model names have no test either.

Mutation notes and implementation invariants

Mutation notes. Each of the following was applied to a private copy and run, in both languages: deriving the prefill guard from the structured-output allowlist, dropping the output_config conjunct from the prefill decision, reading json_prefill by key presence rather than by value, narrowing the capability call to the alias prefixes only, and forwarding json_prefill to the provider instead of consuming it. Every one fails a test rather than passing quietly.

Implementation invariants, not caller-observable:

• The capability predicate reads no instance state, so capability stays answerable independently of how the connection was configured. testCapabilityReadsNoInstanceState, test_capability_reads_no_instance_state.
• The Java 3-arg chat forwards all four arguments to the 4-arg form with a null schema. testThreeArgChatForwardsNoSchema.
• Both languages carry the same twelve structured-output identifiers and the same nine prefill-rejecting names, in the same order, as independent literals in production and in tests.
• A null effective model returns false rather than propagating the NullPointerException that Set.contains(null) raises on an immutable allowlist.
• The prefill decision is returned from buildRequest alongside the request it was made for, so the response conversion reads that decision instead of recomputing a flag that can drift from the request it describes.
• Local schema validation is off on the Java translator, so the provider rather than the client is the authority on which schemas it accepts.
• The Java translator extracts the config from a throwaway request that is never sent. The Kotlin facade outputFormatFromClass would produce it directly, but it is compiled ACC_SYNTHETIC and cannot be named from Java, and the typed outputConfig(Class) overload retypes both the request and the response while the deserialized POJO is discarded anyway. If the SDK returns no config for a schema it accepted, Java raises IllegalStateException, which the public API cannot reach because the same call sets the config two lines earlier.
• testNativePathSendsNoBetaHeader pins that the native path adds no anthropic-beta header.

API

No new public API in Java. Both connections override methods the foundation already defines, and the 3-arg chat now delegates to the 4-arg one. Python gains a single setup parameter, json_prefill, to match the Java one.

Two things change for a caller who does nothing differently. The Java json_prefill default moves from true to false, so a response no longer begins with a forced {; setting the parameter to true asks for it back, though it stays withheld on a model that rejects prefilling and on any request carrying an output_config. And a request carrying an output_config supplied through additional_kwargs no longer carries the prefill as well, which Anthropic documents as incompatible. The markdown extraction that already handles fenced JSON is untouched.

Two dependency floors rise, each to the first release that exposes the parameter on the non-beta client: com.anthropic:anthropic-java from 2.11.1 to 2.12.0, and anthropic from 0.64.0 to 0.77.0. The Java bump forces no other source change and leaves transitive dependencies unchanged. NOTICE is updated to match.

Documentation

• doc-needed
• doc-not-needed
• doc-included

The Anthropic parameter tables gain a json_prefill row on the Python tab. The Java row's default changes to false, and both name the conditions that withhold the prefill. The Java additional_kwargs row records that an output_config supplied there takes precedence over one derived from a schema.

Which providers fulfil output_schema natively rather than by prompt engineering stays undocumented, for every provider. It belongs in the integration support matrix as one change, rather than a third of it landing here.

Was this patch authored or co-authored using generative AI tooling?

• Yes
• No

Generated-by: Claude Code 2.1.226

[weiqingy]

Hi @wenjin272 , could you take a look when you get a chance? Third sample for the Implementation Description experiment on #894.

This one speaks to the size question you raised: 919 lines across two languages, where #952 was 123. Same format, nothing changed in how it is written.

It came out at 7.5k characters rather than the 6k I aimed for. I compressed twice and stopped, since what was left was the contracts and the failure paths. So either 6k is too tight for a two-language change, or the tests table should move to a comment sooner.

On whether writing it was worth it: this time it surfaced nothing new about the code, unlike #930 where it turned up the refusal gap that became #936. Worth recording, since a format that only pays off sometimes is a different proposition.

Still not touching the PR template until you have reviewed one.

[wenjin272]

I think json_prefill should be an explicit opt-in and have the same contract in Java and Python.

Currently Java defaults it to true, while the Python Anthropic integration does not implement json_prefill at all. This creates both a surprising default for ordinary Java chat requests and a Java/Python behavior difference.

Could we align both implementations as follows?

1. Default json_prefill to false.
2. Add first-class json_prefill support to the Python setup and connection.
3. Apply prefill only when all of these conditions hold:
   • the caller explicitly enabled it;
   • no tools are present;
   • the final request contains no output_config, whether derived from output_schema or supplied by the caller;
   • the effective model supports assistant-message prefilling.
4. Reconstruct the leading { only when the request actually carried the prefill.

The model check is important: Anthropic documents that Claude 4.6 and later models, as well as Mythos Preview, reject assistant-message prefill with a 400 because the request must end with a user message:
https://platform.claude.com/docs/en/build-with-claude/working-with-messages#prefilling-claudes-response

This means the Java implementation also needs an additional model-capability guard. For example, a schema-free request to Claude 4.6 currently retains the default prefill and can be rejected even though no native schema is involved.

The caller-supplied output_config case should also suppress prefill. testCallerOutputConfigWinsOverSchema currently expects jsonPrefillApplied to remain true, but output_config.format and message prefilling are incompatible, so that expectation should be reversed.

Suggested parity tests:

• absent/default value does not apply prefill;
• an explicit true applies it on a supported model;
• tools suppress it;
• any caller- or framework-supplied output_config suppresses it;
• Claude 4.6+, Claude 5, and Mythos models suppress or reject it consistently;
• response reconstruction prepends { only when prefill was actually applied.

[wenjin272]

Thanks @weiqingy for putting this together and for trying the Implementation Description experiment across a larger, two-language change. Overall, the description was useful: it let me establish the runtime flow and intended contracts without first reconstructing them from the diff. I think it could be made more review-oriented in four ways:

1. State the purpose more directly. The current section mainly explains why the previous implementation rejected the schema, but the intended user-visible outcome is less immediate. A shorter version could be:

   This PR makes the existing provider-independent output_schema API work with Anthropic in both Java and Python. Supported schemas are translated into Anthropic's native output_config; unsupported combinations continue through the existing prompt-based fallback. The goal is consistent structured-output behavior across providers and across the Java and Python integrations, without introducing a new public API.

2. Add a small interaction/decision table. The numbered contracts describe individual paths well, but make cross-feature combinations harder to inspect. A table covering model capability, schema type, caller output_config, tools, and JSON prefill would make conflicts more visible. The output_config plus prefill case discussed above is a good example of something such a table could surface.

3. Separate behavioral contracts from implementation invariants. User-observable behavior should remain in the main contract section, while details such as the predicate reading no instance state, 3-arg forwarding, and identical model-list ordering could move to implementation notes.

4. Keep the test-to-contract mapping, but move the detailed test names and mutation notes into a <details> block or follow-up comment. The main body could summarize coverage by risk and explicitly list what was not verified. This would reduce the reading cost without losing the evidence useful for Stage 2.

My takeaway is that the main description should prioritize the information needed for human review: the purpose, interaction decisions, behavioral contracts, risks, and coverage. Detailed implementation invariants and test mappings can remain available through a <details> block or a follow-up comment without competing for the reviewer's attention, allowing us to keep the main description as concise as possible.

[weiqingy]

Thanks @weiqingy for putting this together and for trying the Implementation Description experiment across a larger, two-language change. Overall, the description was useful: it let me establish the runtime flow and intended contracts without first reconstructing them from the diff. I think it could be made more review-oriented in four ways:

1. State the purpose more directly. The current section mainly explains why the previous implementation rejected the schema, but the intended user-visible outcome is less immediate. A shorter version could be:

   This PR makes the existing provider-independent output_schema API work with Anthropic in both Java and Python. Supported schemas are translated into Anthropic's native output_config; unsupported combinations continue through the existing prompt-based fallback. The goal is consistent structured-output behavior across providers and across the Java and Python integrations, without introducing a new public API.

2. Add a small interaction/decision table. The numbered contracts describe individual paths well, but make cross-feature combinations harder to inspect. A table covering model capability, schema type, caller output_config, tools, and JSON prefill would make conflicts more visible. The output_config plus prefill case discussed above is a good example of something such a table could surface.
3. Separate behavioral contracts from implementation invariants. User-observable behavior should remain in the main contract section, while details such as the predicate reading no instance state, 3-arg forwarding, and identical model-list ordering could move to implementation notes.
4. Keep the test-to-contract mapping, but move the detailed test names and mutation notes into a <details> block or follow-up comment. The main body could summarize coverage by risk and explicitly list what was not verified. This would reduce the reading cost without losing the evidence useful for Stage 2.

My takeaway is that the main description should prioritize the information needed for human review: the purpose, interaction decisions, behavioral contracts, risks, and coverage. Detailed implementation invariants and test mappings can remain available through a <details> block or a follow-up comment without competing for the reviewer's attention, allowing us to keep the main description as concise as possible.

Thanks for the feedback. I have added them to my notes for the experiment.

The description on this PR was auto-generated by the agent from an earlier version of the template. Once the template improves with the feedback, hope the descriptions will come out more readable for a human reviewer.

[weiqingy]

I think json_prefill should be an explicit opt-in and have the same contract in Java and Python.

Currently Java defaults it to true, while the Python Anthropic integration does not implement json_prefill at all. This creates both a surprising default for ordinary Java chat requests and a Java/Python behavior difference.

Could we align both implementations as follows?

1. Default json_prefill to false.

2. Add first-class json_prefill support to the Python setup and connection.

3. Apply prefill only when all of these conditions hold:

   • the caller explicitly enabled it;
   • no tools are present;
   • the final request contains no output_config, whether derived from output_schema or supplied by the caller;
   • the effective model supports assistant-message prefilling.

4. Reconstruct the leading { only when the request actually carried the prefill.

The model check is important: Anthropic documents that Claude 4.6 and later models, as well as Mythos Preview, reject assistant-message prefill with a 400 because the request must end with a user message: https://platform.claude.com/docs/en/build-with-claude/working-with-messages#prefilling-claudes-response

This means the Java implementation also needs an additional model-capability guard. For example, a schema-free request to Claude 4.6 currently retains the default prefill and can be rejected even though no native schema is involved.

The caller-supplied output_config case should also suppress prefill. testCallerOutputConfigWinsOverSchema currently expects jsonPrefillApplied to remain true, but output_config.format and message prefilling are incompatible, so that expectation should be reversed.

Suggested parity tests:

• absent/default value does not apply prefill;
• an explicit true applies it on a supported model;
• tools suppress it;
• any caller- or framework-supplied output_config suppresses it;
• Claude 4.6+, Claude 5, and Mythos models suppress or reject it consistently;
• response reconstruction prepends { only when prefill was actually applied.

@wenjin272 Agreed on all four, and thanks for catching this. The prefill check only looked at whether we had added an output_config ourselves, so it missed the one a caller supplies.

There was one more case than you named. That check sat inside the outputSchema instanceof Class branch, so passing output_config with no schema at all still got the prefill. Fixed as well.

All four are in the PR now. The default is false, Python has the parameter, we only prefill on models that accept it, and we only add the { back when we actually sent it. Your six tests are in too, both languages where they apply.

One call I would like your view on. The prefill gate has its own model list instead of reusing the structured-output one, because the two do not line up. 4.5 supports structured output but still accepts prefill, and only 4.6 onward rejects it, so sharing a list would quietly switch prefill off for all of 4.5. There is also nothing to query for it, the Models API has capabilities.structured_outputs but nothing for prefill, so it comes down to a hand-maintained list. Does that sound right? wdyt?

[Zhuoxi2000]

Also reviewed this with the previous response-parsing issues (#914, #946) in mind. Token usage is still preserved across the new 4-arg flow, and the first-content-block handling is unchanged. Moving the prefill decision into BuiltRequest also looks cleaner and avoids the old flag/request drift. Left two inline comments: one on the alias boundary and one on the default change.

[Zhuoxi2000]

The alias-prefix logic has the same edge case one level down: startsWith("claude-sonnet-4-5") would also match something like claude-sonnet-4-50. Since the intent is to match either the alias itself or a dated snapshot, name.equals(prefix) || name.startsWith(prefix + "-") seems safer. The same applies to Python’s _NATIVE_STRUCTURED_OUTPUT_ALIAS_PREFIXES.

[weiqingy]

Thanks for the review. Improved this in commit a72fbbf4.

[Zhuoxi2000]

The Java default changing from true to false is also a behavior change for existing setups that relied on implicit prefilling. The new default makes sense given 4.6+ models reject prefilling, but should this get a release-note / BREAKING entry for 0.4 rather than only the docs update?

[weiqingy]

You're right, this breaks existing setups. json_prefill defaulted to true in 0.2.0 through 0.3.1, so callers without tools lose the prefilled { on 0.4.

Currently I don't think it needs a separate BREAKING entry. code_review.md says breaking changes are acceptable in beta when they prevent an old bug path staying available, which is this one: the old gate never checked the model, so a 4.6+ model returned a 400 on every tool-free request. It also asks that such a change be intentional, documented and tested, and this one is, in the chat_models.md table and the setup tests in both languages. I have also fixed the PR description, which still claimed nothing changed.

Once we reach 1.0 we should guarantee backward compatibility. Having somewhere to record breaks like this before then would be worth doing, and this is a good example of why.

[weiqingy]

Updated the PR body to make it easier to read, applying the points from the review feedback.