[java][python] IBM watsonx.ai chat model integration

[cansuk99]

Linked issue: #902

Purpose of change

Chat model integration for watsonx.ai models provided by IBM Cloud.

• Adds a new watsonx integration module for Java chat models.
• Adds a new watsonx integration module for Python chat models.
• Adds watsonx resource names and YAML alias mappings in both Java and Python.
• Updates distribution and build wiring to include the new integration artifact.
• Adds documentation for watsonx setup, parameters, model usage, and YAML alias support.

Tests

Java:

• Watsonx connection behavior and payload conversion tests.
• Watsonx setup validation tests.
• Alias resolution tests for watsonx.

Python:

• Watsonx chat behavior tests (mocked tool-call, retry flows).
• Alias resolution tests for watsonx.

Both Java and Python integrations were tested live with IBM Cloud credentials with flink jobs running on k8s.

Documentation

• doc-needed
• doc-not-needed
• doc-included

[weiqingy]

Thanks for taking this on. A few questions inline.

[weiqingy]

_chat_with_retry retries over RETRYABLE_STATUS_CODES = {408, 429, 500, 502, 503, 504} with max_retries defaulting to 3. The SDK already retries underneath: ibm_watsonx_ai/_wrappers/httpx/rate_limited_retry/rate_limited_retry_decorator.py:40-44 sets MAX_RETRIES = 10 over (429, 503, 504, 520), wired into every chat POST at base_model_inference.py:63-76, same on the 1.3.42 pin. A sustained 429 then costs up to 44 HTTP requests and minutes of blocking time.sleep (:327) on a Flink async-executor thread for one record, with a third layer above both at plan/actions/chat_model_action.py:304-360.

Java's sendWithRetry (WatsonxChatModelConnection.java:290-325) is the only layer there, so the same max_retries=3 means about 3 retries on Java and up to about 43 on Python, while the max_retries row in chat_models.md describes both identically ("Maximum retries for transport failures and HTTP 408, 429, 500, 502, 503, and 504 responses"). AGENTS.md asks for semantic alignment, and here the key, the default and the docs all match while behavior differs by an order of magnitude.

Was the SDK's built-in retry known about when the wrapper was written? ModelInference.__init__ takes max_retries, delay_time and retry_status_codes directly (model_inference.py:162-164, both pins), so the configured policy could be handled by one layer with the same user-visible semantics.

[cansuk99]

Hey @weiqingy he SDK retry layer was not accounted for in the initial implementation - thank you for catching this! I have disabled SDK retries by passing max_retries=0 to ModelInference and kept _chat_with_retry as the single retry owner. Here I optimized for behavioural parity between the Java and Python implementation, as otherwise if we prioritize the SDK built-in retry which does not exist in Java, the connectors would differ.

Does this approach make sense to you, or would you prefer delegating retries to the native SDK in Python while retaining the connector-level implementation in Java?

[weiqingy]

Connector-level in both makes sense to me, for the reason you gave. It keeps the two languages on one retry set and one Retry-After policy.

I checked that max_retries=0 really does switch the SDK's own retry off, and it does on every version in the range pinned at python/pyproject.toml:59-60. Worth checking on more than the newest, since that code was reworked between releases.

The one thing still open is that nothing tests it. test_watsonx_chat_model.py replaces _get_model outright (:89-91, :146-148, :184-186), so the ModelInference constructor is never seen, and deleting max_retries=0 would leave the suite green. The five new reserved keys sit in the same spot, since test_configuration_contract (:355-360) only exercises temperature.

Would asserting the constructor kwargs in that mocked test be enough to pin both, or would you rather keep this for a follow-up?

[cansuk99]

Agreed. I updated the existing mocked chat test to exercise the real _get_model() path and assert that ModelInference is constructed with max_retries=0. I also added parameterized coverage for model_id, messages, tools, project_id, and space_id in additional_kwargs.

[wenjin272]

Thanks for taking this on @cansuk99. Overall looks good to me. I left two minor comments.

[cansuk99]

Hey @weiqingy @wenjin272 , the Flink Agents CI / it-java [ubuntu-latest] [java-21] [flink-2.1] check appears to have failed in TokenMetricsE2ETest, which does the OpenAI mock connector and is outside the Watsonx.ai changes in this PR. Could one of you please rerun the failed job? The failure may be transient and a re-run shall suffice.

I don’t have permission to rerun GitHub jobs on this repository without having to push another commit and re-run all jobs.

[wenjin272]

LGTM. No further comments from my side. Let's wait for @weiqingy to take a look.

[weiqingy]

Thanks for resolving the comments. Just a few small ones left from me. Nothing here blocks.

nit, and the only thing here that is not inline: the provider matrix in docs/content/docs/faq/faq.md:98-107 lists eight chat-model providers and has no watsonx row. That table has a Python and a Java column, and it is alphabetical, so a watsonx row would sit right after Tongyi with a check in both columns, since this ships both languages. For the pairing precedent, 3cb28967 ("[docs][integration] Document Gemini chat model integration (#898)") touched exactly two files, chat_models.md and faq.md, the second being just the new row.

[weiqingy]

Returning content here means an empty or whitespace-only block keeps its tags: <think></think>Answer comes back unchanged at head, where before it returned Answer. Being on the shared base, this reaches OllamaChatModelConnection too (ollama_chat_model.py:138).

Would return cleaned, None work? cleaned still equals content when nothing matched, so the no-tag case you're protecting is untouched.