Skip to content

fix(ai): forward request timeout caps through intent classification - #373

Merged
brokermr810 merged 1 commit into
OpenByteInc:mainfrom
AmazingZ-sys:fix/intent-request-timeout
Nov 13, 2026
Merged

brokermr810 merged 1 commit into
OpenByteInc:mainfrom
AmazingZ-sys:fix/intent-request-timeout

Conversation

@AmazingZ-sys

Copy link
Copy Markdown
Contributor

Summary

call_llm_api already accepts a caller-specific timeout, but safe_call_llm cannot forward it. Intent classification therefore uses the provider's full request timeout before returning its existing fallback plan. Add an optional keyword-only timeout and use a 20-second request cap for classification.

Related issue

Small internal timeout propagation fix; no matching open PR found.

Changes

  • Preserve the existing call behavior when no timeout is supplied.
  • Apply a supplied cap without increasing a stricter provider timeout; forward it through both existing alternative-provider error paths.
  • Keep intent normalization and fallback behavior. The cap applies to each individual request: multiple model/provider attempts may take longer than 20 seconds in total.

Test plan

  • python -m pytest tests/test_llm_request_timeout.py tests/test_llm_litellm_provider.py tests/test_ai_copilot_latency.py -q — 62 passed.
  • Tests omitted/explicit timeout forwarding, stricter provider limits, every model attempt, structured/HTTP alternative-provider paths, and intent fallback/context after a timeout. Uses synthetic responses without external model calls.
  • ruff check app/services/llm.py app/routes/ai_chat.py tests/test_llm_request_timeout.py — passed.
  • python scripts/backend_quality_check.py and git diff --check — passed.

API documentation

No public HTTP contract, migration, or runtime dependency changes.

Signed-off-by: Amazingz <amazingzsys@gmail.com>
@sonarqubecloud

Copy link
Copy Markdown

@codecov

codecov Bot commented Nov 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@brokermr810 brokermr810 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed timeout propagation through safe_call_llm and intent-classification coverage. Each intent-classification request is capped at 20 seconds; this is intentionally a per-request cap rather than a total multi-provider deadline. Backend tests and security checks passed; the pip-audit failure is the pre-existing soupsieve 2.9 baseline on main.

@brokermr810
brokermr810 merged commit 4cc8e65 into OpenByteInc:main Nov 13, 2026
14 of 15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants