Skip to content

[feature] Add OrcaRouter as a named model provider #574

Description

@XiaoHuo888-hue

What problem does this solve?

config.yaml and the Settings UI both treat model endpoints as a generic OpenAI-compatible pair of base_url + model. That works, but for users who route through OrcaRouter — an OpenAI-compatible model gateway with smart routing across many upstream models — it means hand-typing the endpoint every time and re-discovering the right model IDs. The repo already sets the precedent for a named provider (provider: AzureOpenAIChatCompletionClient in src/magentic_ui/_ai_client.py, the model_config_azure_example block in config.yaml.example, and the Azure bullet in docs/configuration.md). A named OrcaRouter provider would mirror that precedent so routing through the gateway is a one-block setup.

It also runs gateway-level, zero-trust security for AI agents on the same endpoint — screening every prompt/response and governing every tool call on a default-deny basis, with no application code changes.

Proposed solution

Add OrcaRouter as a named model provider, mirroring the existing Azure named-provider pattern:

  1. Backend registrysrc/magentic_ui/_ai_client.py:

    • Add an _ORCAROUTER_PROVIDERS set and a _create_orcarouter() factory alongside _create_azure(), dispatching on provider: OrcaRouterChatCompletionClient.
    • The factory returns the standard AsyncOpenAI client with base_url=https://api.orcarouter.ai/v1, an ORCAROUTER_API_KEY env-var fallback, and a default model of orcarouter/auto (OrcaRouter's smart-routing model). Everything downstream (the ChatClient wrapper, OmniAgent, FaraWebSurfer) is already OpenAI-compatible and needs no changes.
  2. Config exampleconfig.yaml.example: add a model_config_orcarouter anchor block for both the orchestrator and web-surfer roles (mirroring the Azure example), so users can route either agent through the gateway.

  3. Docsdocs/configuration.md: add an OrcaRouter bullet and YAML example in the "Model clients" notes (mirroring the Azure note), including that the browser-use model should be a vision-capable gateway model (e.g. orcarouter/fusion).

  4. Optional UI polish — the model cards in ModelSettings.tsx / onboarding currently accept free-form input; a provider preset that pre-fills https://api.orcarouter.ai/v1 + orcarouter/auto would make adoption one click.

I verified the gateway's OpenAI-compatible surface with a live probe:

  • GET https://api.orcarouter.ai/v1/models200, 208 models, orcarouter/auto present.
  • POST https://api.orcarouter.ai/v1/chat/completions with model: "orcarouter/auto"200 with a normal chat.completion response (routed to deepseek-v4-pro), which is exactly the shape MagenticLite's _verify_single_endpoint probe and create_openai_client expect.

Alternatives considered

  • Generic OpenAI-compatible passthrough only (current state): already works if users type https://api.orcarouter.ai/v1 by hand, but there's no named entry point, no preset, and no documentation — the endpoint stays invisible and each user re-discovers it. A named provider gives the same one-block config the repo already offers for Azure.
  • Frontend-only dropdown: would be user-visible but without the backend provider dispatch the saved config still records a generic client; the backend named provider is the foundation, so it should come first.

Additional context

I'm an engineer on the OrcaRouter team — happy to turn this into a PR (the code is small and mirrors the Azure wiring), and to adjust the model IDs, defaults, or docs based on your preferences. I understand from CONTRIBUTING that code PRs are currently limited to vetted contributors; opening this issue first per the repo's issues-first guidance.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions