diff --git a/CHANGELOG.md b/CHANGELOG.md index 778d36fc..888f141a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,37 @@ ### Added +- **Atlas Cloud LLM provider** — `AtlasCloudLLM` lands in both SDKs, adding an + OpenAI-compatible aggregator gateway (`https://api.atlascloud.ai/v1`) that + fronts 100+ open and frontier models behind one key, so swapping models is a + `model` string rather than a new provider, key and pricing table: + - **`AtlasCloudLLMProvider` / `AtlasCloudLLM`** (Python + `getpatter.llm.atlascloud`, TS `getpatter/llm/atlascloud`, plus the flat + `AtlasCloudLLM` alias on the package root). It is `OpenAILLMProvider` with + an Atlas base URL, so streaming, tool calling and every OpenAI sampling + knob (`response_format`, `parallel_tool_calls`, `tool_choice`, `seed`, + `top_p`, `frequency_penalty`, `presence_penalty`, `stop`) are inherited + unchanged. Credentials come from `ATLASCLOUD_API_KEY` when `api_key` / + `apiKey` is omitted. Default model `openai/gpt-4.1-mini` — it answered a + tool-calling turn in 14 completion tokens with **zero reasoning tokens**, + where the reasoning-tuned models in the catalog spent 28–64 first; on a + phone call those are dead air. + - **Pricing for all eight `AtlasCloudModel` ids**, read from the gateway's + own catalog (`GET /v1/models` → `pricing.prompt` / `pricing.completion` / + `pricing.input_cache_read`) rather than a docs snapshot. `cache_read` is + populated even where it equals `input` (the gateway applies no discount for + those models) — omitting the key would have billed cached tokens at $0. + Both pricing tests are driven off the enum, so adding a model without a + rate fails CI instead of silently under-billing. + - **`atlascloud` extra** (`pip install 'getpatter[atlascloud]'`), an + `ATLASCLOUD_API_KEY` line in both `.env.example` files, an **Atlas Cloud + entry in `patter init`**'s pipeline LLM table (both runtimes), and + `atlascloud` added to the telemetry vendor allowlist so the stack field + reports the real vendor instead of collapsing to `other`. + - Docs: `providers/atlascloud.mdx` in both SDK sections (model + live + pricing table, prompt-cache semantics) and an `AtlasCloudLLM` entry in each + `llm.mdx`. + - **xAI (Grok) voice catalog wired into both SDKs** — the 26 built-in Grok voices (`eve` default, `ara`, `rex`, `sal`, `leo`, `carina`, ...) are now a typed, immutable catalog instead of docs-only prose: diff --git a/docs/docs.json b/docs/docs.json index 40c24d10..013cf985 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -138,6 +138,7 @@ "pages": [ "python-sdk/llm", "python-sdk/providers/anthropic", + "python-sdk/providers/atlascloud", "python-sdk/providers/cerebras", "python-sdk/providers/google", "python-sdk/providers/groq" @@ -278,6 +279,7 @@ "pages": [ "typescript-sdk/llm", "typescript-sdk/providers/anthropic", + "typescript-sdk/providers/atlascloud", "typescript-sdk/providers/cerebras", "typescript-sdk/providers/google", "typescript-sdk/providers/groq" diff --git a/docs/python-sdk/llm.mdx b/docs/python-sdk/llm.mdx index a5ab1efa..d210a3fb 100644 --- a/docs/python-sdk/llm.mdx +++ b/docs/python-sdk/llm.mdx @@ -53,6 +53,7 @@ Tool calling works across every provider — each adapter normalizes its vendor- | `AnthropicLLM` | `getpatter.llm.anthropic.LLM` | `ANTHROPIC_API_KEY` | `getpatter[anthropic]` | | `GroqLLM` | `getpatter.llm.groq.LLM` | `GROQ_API_KEY` | `getpatter[groq]` | | `CerebrasLLM` | `getpatter.llm.cerebras.LLM` | `CEREBRAS_API_KEY` | `getpatter[cerebras]` | +| `AtlasCloudLLM` | `getpatter.llm.atlascloud.LLM` | `ATLASCLOUD_API_KEY` | `getpatter[atlascloud]` | | `GoogleLLM` | `getpatter.llm.google.LLM` | `GEMINI_API_KEY` (falls back to `GOOGLE_API_KEY`) | `getpatter[google]` | All classes accept `api_key: str | None = None` and fall back to the listed env var when it is omitted. @@ -103,6 +104,22 @@ llm = groq.LLM(api_key="gsk_...", model="llama-3.3-70b-versatile") Install: `pip install 'getpatter[groq]'`. +### AtlasCloudLLM + +[Atlas Cloud](https://www.atlascloud.ai) aggregator (OpenAI-compatible) at `https://api.atlascloud.ai/v1` — one key and one endpoint in front of 100+ open and frontier models, so changing model is a `model=` string rather than a new provider. Default model `"openai/gpt-4.1-mini"`; `GET /v1/models` lists every routable id with its live per-token rates. + +Streaming, tool calling and all OpenAI sampling kwargs (`response_format`, `parallel_tool_calls`, `tool_choice`, `seed`, `top_p`, `frequency_penalty`, `presence_penalty`, `stop`) come from `OpenAILLMProvider` unchanged. Prompt-cache hits are reported in `usage.prompt_tokens_details.cached_tokens` and billed once at the `cache_read` rate. + +```python +from getpatter import AtlasCloudLLM # flat +from getpatter.llm import atlascloud # namespaced + +llm = AtlasCloudLLM() # reads ATLASCLOUD_API_KEY +llm = atlascloud.LLM(api_key="apikey-...", model="deepseek-ai/deepseek-v3.2") +``` + +Install: `pip install 'getpatter[atlascloud]'`. See the [Atlas Cloud provider page](/python-sdk/providers/atlascloud) for the model/pricing table. + ### CerebrasLLM Cerebras Inference API (OpenAI-compatible) at `https://api.cerebras.ai/v1`. Default model `"gpt-oss-120b"` — production tier, ~3000 tok/sec on WSE-3, no deprecation date. Pass `model="llama3.1-8b"` (8B params, sub-100ms TTFT) for the smaller free-tier alternative. The 404 `model_not_found` error includes a recovery hint listing other valid IDs (`qwen-3-235b-a22b-instruct-2507`, `llama-3.3-70b` on paid tier). @@ -218,7 +235,7 @@ from getpatter import LLMChunk, DefaultToolExecutor - `LLMChunk` — the streaming-output type yielded by every `LLMProvider.stream(...)` implementation. Carries either a partial text delta, a tool-call delta, or a stream-end marker. - `DefaultToolExecutor` — the default tool dispatcher used by `LLMLoop`. Constructs from a `tools=` list and resolves both Python `handler=` callables and `webhook_url=` HTTP tools. Override its hooks to swap in custom error handling, telemetry, or authentication. -- `OpenAILLMProvider` — the parent class shared by `OpenAILLM`, `GroqLLM`, `CerebrasLLM`. Sampling kwargs (`temperature`, `top_p`, `seed`, `tool_choice`, `response_format`, …) live here and are forwarded by every subclass. +- `OpenAILLMProvider` — the parent class shared by `OpenAILLM`, `GroqLLM`, `CerebrasLLM`, `AtlasCloudLLM`. Sampling kwargs (`temperature`, `top_p`, `seed`, `tool_choice`, `response_format`, …) live here and are forwarded by every subclass. These are stable public symbols mirrored byte-for-byte by the TypeScript SDK. diff --git a/docs/python-sdk/providers/atlascloud.mdx b/docs/python-sdk/providers/atlascloud.mdx new file mode 100644 index 00000000..e0e2e278 --- /dev/null +++ b/docs/python-sdk/providers/atlascloud.mdx @@ -0,0 +1,161 @@ +--- +title: "Atlas Cloud" +description: "Atlas Cloud LLM provider — one OpenAI-compatible endpoint in front of 100+ open and frontier models." +icon: "cloud" +--- + +# Atlas Cloud LLM + +`AtlasCloudLLM` plugs Atlas Cloud's OpenAI-compatible Chat Completions API at `https://api.atlascloud.ai/v1` into Patter's pipeline mode. It is a thin wrapper around `OpenAILLMProvider` with an Atlas-specific base URL, so streaming, tool calling and every OpenAI sampling parameter behave exactly as they do on `OpenAILLM`. + +## Why an aggregator for voice + +A phone agent usually needs one model, but rarely the *same* model forever: you tune for latency, then a cheaper open-weights model lands, then one vendor rate-limits you at 6pm. Atlas Cloud fronts 100+ models — OpenAI, DeepSeek, Qwen, GLM, Kimi, MiniMax — behind a single key and a single endpoint, so switching is a `model=` string rather than a new provider, a new key and a new pricing table. + +Two things matter specifically for voice: + +- **Tool calling works across the catalog.** The models listed below were all verified to emit a streaming `tool_calls` delta for a `transfer_call` function — the pattern Patter's builtin tools rely on. +- **Reasoning tokens are dead air.** On a phone call, tokens the caller never hears are silence. The default `openai/gpt-4.1-mini` answered a transfer request in 14 completion tokens with zero reasoning tokens; the reasoning-tuned models in the catalog (`kimi-k2.5`, `minimax-m2.5`, `qwen3.5-35b-a3b`) spent 28–64 reasoning tokens first. Pick those deliberately, for the turns that need them. + +## Install + +```bash +pip install "getpatter[atlascloud]" +``` + +```bash +npm install getpatter +``` + +## Usage + + + +```python Python +# Namespaced import +from getpatter.llm import atlascloud + +llm = atlascloud.LLM() # reads ATLASCLOUD_API_KEY +llm = atlascloud.LLM(api_key="apikey-...", model="deepseek-ai/deepseek-v3.2") +llm = atlascloud.LLM( + model="Qwen/Qwen3-235B-A22B-Instruct-2507", + temperature=0.4, + response_format={"type": "json_object"}, # OpenAI-style structured outputs +) + +# Flat alias (equivalent) +from getpatter import AtlasCloudLLM + +llm = AtlasCloudLLM() +``` + +```ts TypeScript +// Namespaced import +import * as atlascloud from "getpatter/llm/atlascloud"; + +const llm = new atlascloud.LLM(); // reads ATLASCLOUD_API_KEY +const llm2 = new atlascloud.LLM({ apiKey: "apikey-...", model: "deepseek-ai/deepseek-v3.2" }); +const llm3 = new atlascloud.LLM({ + model: "Qwen/Qwen3-235B-A22B-Instruct-2507", + temperature: 0.4, + responseFormat: { type: "json_object" }, +}); + +// Flat alias (equivalent) +import { AtlasCloudLLM } from "getpatter"; + +const llm4 = new AtlasCloudLLM(); +``` + + + + +The namespaced import (`from getpatter.llm import atlascloud` / `import * as atlascloud from "getpatter/llm/atlascloud"`) auto-resolves the API key from `ATLASCLOUD_API_KEY` and exposes a uniform `LLM` class — the same pattern Patter uses for STT and TTS namespaces. + + +Plug it into an agent: + + + +```python Python +import asyncio +from getpatter import Patter, Twilio, DeepgramSTT, AtlasCloudLLM, ElevenLabsTTS + +phone = Patter(carrier=Twilio(), phone_number="+15550001234") + +agent = phone.agent( + stt=DeepgramSTT(), + llm=AtlasCloudLLM(), # ATLASCLOUD_API_KEY from env + tts=ElevenLabsTTS(voice_id="rachel"), + system_prompt="You are a helpful assistant.", + first_message="Hi, how can I help?", +) + +asyncio.run(phone.serve(agent)) +``` + +```ts TypeScript +import { Patter, Twilio, DeepgramSTT, AtlasCloudLLM, ElevenLabsTTS } from "getpatter"; + +const phone = new Patter({ carrier: new Twilio(), phoneNumber: "+15550001234" }); + +const agent = phone.agent({ + stt: new DeepgramSTT(), + llm: new AtlasCloudLLM(), // ATLASCLOUD_API_KEY from env + tts: new ElevenLabsTTS({ voiceId: "rachel" }), + systemPrompt: "You are a helpful assistant.", + firstMessage: "Hi, how can I help?", +}); + +await phone.serve(agent); +``` + + + +Or scaffold a project with it preselected: + +```bash +getpatter init --mode pipeline --llm atlascloud +``` + +## Supported models + +These are the ids in the `AtlasCloudModel` enum and the ones carrying rates in `LLM_PRICING` / `llmPricing`. Pricing is USD per 1M tokens, read from the gateway's own catalog (`GET /v1/models` → `pricing.prompt` / `pricing.completion` / `pricing.input_cache_read`, in USD per token). + +| Model | Input | Output | Cache read | Context | Notes | +|---|---|---|---|---|---| +| `openai/gpt-4.1-mini` (default) | $0.40 | $1.60 | $0.10 | 1,047,576 | No reasoning tokens — lowest dead air per turn. | +| `openai/gpt-5.4-mini` | $0.75 | $4.50 | $0.075 | 400,000 | Deepest cache discount (10× off input). | +| `deepseek-ai/deepseek-v3.2` | $0.26 | $0.38 | $0.13 | 163,840 | Cheapest output in the set. | +| `Qwen/Qwen3-235B-A22B-Instruct-2507` | $0.20 | $0.88 | $0.20 | 131,072 | Cheapest input; no cache discount. | +| `qwen/qwen3.5-35b-a3b` | $0.225 | $1.80 | $0.225 | 262,144 | Reasoning-tuned; no cache discount. | +| `zai-org/GLM-4.6` | $0.60 | $2.20 | $0.11 | 202,752 | | +| `moonshotai/kimi-k2.5` | $0.49 | $2.50 | $0.20 | 262,144 | Reasoning-tuned. | +| `minimaxai/minimax-m2.5` | $0.295 | $1.20 | $0.06 | 196,608 | Reasoning-tuned. | + + +The gateway routes far more models than these — `GET https://api.atlascloud.ai/v1/models` lists everything currently available, with its live per-token rates. Any id there works as `model=`; models outside the table above simply bill $0 in the dashboard until you add a `pricing={...}` override. + + +## Environment variables + +| Variable | Required | Notes | +|---|---|---| +| `ATLASCLOUD_API_KEY` | yes | Auto-loaded when `api_key` / `apiKey` is omitted. | + +## Options + +| Option | Default | Notes | +|---|---|---| +| `api_key` / `apiKey` | `None` | Reads from `ATLASCLOUD_API_KEY` when omitted. | +| `model` | `"openai/gpt-4.1-mini"` | Any id from `GET /v1/models`. | +| `base_url` / `baseUrl` | `https://api.atlascloud.ai/v1` | Override the Atlas Cloud endpoint (rarely needed). | +| `temperature`, `max_tokens`, `top_p`, `seed`, `frequency_penalty`, `presence_penalty`, `stop`, `response_format`, `parallel_tool_calls`, `tool_choice` | unset | All forwarded to `chat.completions.create`. | + +## Prompt caching + +The gateway reports cached input in the standard OpenAI field (`usage.prompt_tokens_details.cached_tokens`). Patter subtracts it from `prompt_tokens` before billing, so cached tokens are charged once at the `cache_read` rate rather than at both rates. Where a model's `cache_read` equals its `input` rate the gateway is applying no cache discount for that model — the entry is still present in the pricing table, because omitting it would bill those tokens at $0. + +## Error handling + +Errors surface exactly as they do on `OpenAILLM` — the provider is the same engine with a different `base_url`. Failures retry once with exponential backoff; terminal errors raise `PatterError` so the LLM fallback chain can fail over and the spoken error fallback can fire. diff --git a/docs/typescript-sdk/llm.mdx b/docs/typescript-sdk/llm.mdx index ee17e704..12958ea2 100644 --- a/docs/typescript-sdk/llm.mdx +++ b/docs/typescript-sdk/llm.mdx @@ -50,6 +50,7 @@ Tool calling works across every provider — each adapter normalizes its vendor- | `AnthropicLLM` | `ANTHROPIC_API_KEY` | included | | `GroqLLM` | `GROQ_API_KEY` | included | | `CerebrasLLM` | `CEREBRAS_API_KEY` | included | +| `AtlasCloudLLM` | `ATLASCLOUD_API_KEY` | included | | `GoogleLLM` | `GEMINI_API_KEY` (falls back to `GOOGLE_API_KEY`) | included | All classes accept an options object with `apiKey?: string` and fall back to the listed env var when it is omitted. @@ -93,6 +94,21 @@ const llm = new GroqLLM(); // reads GROQ_API_KEY const llm2 = new GroqLLM({ apiKey: "gsk_...", model: "llama-3.3-70b-versatile" }); ``` +### AtlasCloudLLM + +[Atlas Cloud](https://www.atlascloud.ai) aggregator (OpenAI-compatible) at `https://api.atlascloud.ai/v1` — one key and one endpoint in front of 100+ open and frontier models, so changing model is a `model` string rather than a new provider. Default model `"openai/gpt-4.1-mini"`; `GET /v1/models` lists every routable id with its live per-token rates. + +Streaming, tool calling and all OpenAI sampling options (`responseFormat`, `parallelToolCalls`, `toolChoice`, `seed`, `topP`, `frequencyPenalty`, `presencePenalty`, `stop`) behave as on `OpenAILLM`. Prompt-cache hits are reported in `usage.prompt_tokens_details.cached_tokens` and billed once at the `cache_read` rate. + +```typescript +import { AtlasCloudLLM } from "getpatter"; + +const llm = new AtlasCloudLLM(); // reads ATLASCLOUD_API_KEY +const llm2 = new AtlasCloudLLM({ apiKey: "apikey-...", model: "deepseek-ai/deepseek-v3.2" }); +``` + +See the [Atlas Cloud provider page](/typescript-sdk/providers/atlascloud) for the model/pricing table. + ### CerebrasLLM Cerebras Inference API (OpenAI-compatible) at `https://api.cerebras.ai/v1`. Default model `"gpt-oss-120b"` — production tier, ~3000 tok/sec on WSE-3, no deprecation date. Pass `model: "llama3.1-8b"` for the smaller free-tier alternative. The 404 `model_not_found` error includes a recovery hint listing other valid IDs. @@ -196,7 +212,7 @@ import { LLMChunk, DefaultToolExecutor, LLMLoop, OpenAILLMProvider } from "getpa - `LLMChunk` — the streaming-output type yielded by every `LLMProvider.stream(...)` implementation. Carries either a partial text delta, a tool-call delta, or a stream-end marker. - `DefaultToolExecutor` — the default tool dispatcher used by `LLMLoop`. Constructs from a `tools` array and resolves both inline `handler` callables and `webhookUrl` HTTP tools. Override its hooks to swap in custom error handling, telemetry, or authentication. -- `OpenAILLMProvider` — the parent class shared by `OpenAILLM`, `GroqLLM`, `CerebrasLLM`. Sampling options (`temperature`, `topP`, `seed`, `toolChoice`, `responseFormat`, …) live here and are forwarded by every subclass. +- `OpenAILLMProvider` — the parent class shared by `OpenAILLM`, `GroqLLM`, `CerebrasLLM`, `AtlasCloudLLM`. Sampling options (`temperature`, `topP`, `seed`, `toolChoice`, `responseFormat`, …) live here and are forwarded by every subclass. - `LLMLoop` — the orchestration loop wiring an `LLMProvider`, a `DefaultToolExecutor`, and the streaming output back to TTS. These are stable public symbols mirrored byte-for-byte by the Python SDK. diff --git a/docs/typescript-sdk/providers/atlascloud.mdx b/docs/typescript-sdk/providers/atlascloud.mdx new file mode 100644 index 00000000..db24b84b --- /dev/null +++ b/docs/typescript-sdk/providers/atlascloud.mdx @@ -0,0 +1,161 @@ +--- +title: "Atlas Cloud" +description: "Atlas Cloud LLM provider — one OpenAI-compatible endpoint in front of 100+ open and frontier models." +icon: "cloud" +--- + +# Atlas Cloud LLM + +`AtlasCloudLLM` plugs Atlas Cloud's OpenAI-compatible Chat Completions API at `https://api.atlascloud.ai/v1` into Patter's pipeline mode. It is a thin wrapper around the OpenAI Chat Completions client with an Atlas-specific base URL, so streaming, tool calling and every OpenAI sampling parameter behave exactly as they do on `OpenAILLM`. + +## Why an aggregator for voice + +A phone agent usually needs one model, but rarely the *same* model forever: you tune for latency, then a cheaper open-weights model lands, then one vendor rate-limits you at 6pm. Atlas Cloud fronts 100+ models — OpenAI, DeepSeek, Qwen, GLM, Kimi, MiniMax — behind a single key and a single endpoint, so switching is a `model=` string rather than a new provider, a new key and a new pricing table. + +Two things matter specifically for voice: + +- **Tool calling works across the catalog.** The models listed below were all verified to emit a streaming `tool_calls` delta for a `transfer_call` function — the pattern Patter's builtin tools rely on. +- **Reasoning tokens are dead air.** On a phone call, tokens the caller never hears are silence. The default `openai/gpt-4.1-mini` answered a transfer request in 14 completion tokens with zero reasoning tokens; the reasoning-tuned models in the catalog (`kimi-k2.5`, `minimax-m2.5`, `qwen3.5-35b-a3b`) spent 28–64 reasoning tokens first. Pick those deliberately, for the turns that need them. + +## Install + +```bash +npm install getpatter +``` + +```bash +pip install "getpatter[atlascloud]" +``` + +## Usage + + + +```ts TypeScript +// Namespaced import +import * as atlascloud from "getpatter/llm/atlascloud"; + +const llm = new atlascloud.LLM(); // reads ATLASCLOUD_API_KEY +const llm2 = new atlascloud.LLM({ apiKey: "apikey-...", model: "deepseek-ai/deepseek-v3.2" }); +const llm3 = new atlascloud.LLM({ + model: "Qwen/Qwen3-235B-A22B-Instruct-2507", + temperature: 0.4, + responseFormat: { type: "json_object" }, +}); + +// Flat alias (equivalent) +import { AtlasCloudLLM } from "getpatter"; + +const llm4 = new AtlasCloudLLM(); +``` + +```python Python +# Namespaced import +from getpatter.llm import atlascloud + +llm = atlascloud.LLM() # reads ATLASCLOUD_API_KEY +llm = atlascloud.LLM(api_key="apikey-...", model="deepseek-ai/deepseek-v3.2") +llm = atlascloud.LLM( + model="Qwen/Qwen3-235B-A22B-Instruct-2507", + temperature=0.4, + response_format={"type": "json_object"}, # OpenAI-style structured outputs +) + +# Flat alias (equivalent) +from getpatter import AtlasCloudLLM + +llm = AtlasCloudLLM() +``` + + + + +The namespaced import (`import * as atlascloud from "getpatter/llm/atlascloud"` / `from getpatter.llm import atlascloud`) auto-resolves the API key from `ATLASCLOUD_API_KEY` and exposes a uniform `LLM` class — the same pattern Patter uses for STT and TTS namespaces. + + +Plug it into an agent: + + + +```ts TypeScript +import { Patter, Twilio, DeepgramSTT, AtlasCloudLLM, ElevenLabsTTS } from "getpatter"; + +const phone = new Patter({ carrier: new Twilio(), phoneNumber: "+15550001234" }); + +const agent = phone.agent({ + stt: new DeepgramSTT(), + llm: new AtlasCloudLLM(), // ATLASCLOUD_API_KEY from env + tts: new ElevenLabsTTS({ voiceId: "rachel" }), + systemPrompt: "You are a helpful assistant.", + firstMessage: "Hi, how can I help?", +}); + +await phone.serve(agent); +``` + +```python Python +import asyncio +from getpatter import Patter, Twilio, DeepgramSTT, AtlasCloudLLM, ElevenLabsTTS + +phone = Patter(carrier=Twilio(), phone_number="+15550001234") + +agent = phone.agent( + stt=DeepgramSTT(), + llm=AtlasCloudLLM(), # ATLASCLOUD_API_KEY from env + tts=ElevenLabsTTS(voice_id="rachel"), + system_prompt="You are a helpful assistant.", + first_message="Hi, how can I help?", +) + +asyncio.run(phone.serve(agent)) +``` + + + +Or scaffold a project with it preselected: + +```bash +npx getpatter init --mode pipeline --llm atlascloud +``` + +## Supported models + +These are the ids in the `AtlasCloudModel` enum and the ones carrying rates in `llmPricing` / `LLM_PRICING`. Pricing is USD per 1M tokens, read from the gateway's own catalog (`GET /v1/models` → `pricing.prompt` / `pricing.completion` / `pricing.input_cache_read`, in USD per token). + +| Model | Input | Output | Cache read | Context | Notes | +|---|---|---|---|---|---| +| `openai/gpt-4.1-mini` (default) | $0.40 | $1.60 | $0.10 | 1,047,576 | No reasoning tokens — lowest dead air per turn. | +| `openai/gpt-5.4-mini` | $0.75 | $4.50 | $0.075 | 400,000 | Deepest cache discount (10× off input). | +| `deepseek-ai/deepseek-v3.2` | $0.26 | $0.38 | $0.13 | 163,840 | Cheapest output in the set. | +| `Qwen/Qwen3-235B-A22B-Instruct-2507` | $0.20 | $0.88 | $0.20 | 131,072 | Cheapest input; no cache discount. | +| `qwen/qwen3.5-35b-a3b` | $0.225 | $1.80 | $0.225 | 262,144 | Reasoning-tuned; no cache discount. | +| `zai-org/GLM-4.6` | $0.60 | $2.20 | $0.11 | 202,752 | | +| `moonshotai/kimi-k2.5` | $0.49 | $2.50 | $0.20 | 262,144 | Reasoning-tuned. | +| `minimaxai/minimax-m2.5` | $0.295 | $1.20 | $0.06 | 196,608 | Reasoning-tuned. | + + +The gateway routes far more models than these — `GET https://api.atlascloud.ai/v1/models` lists everything currently available, with its live per-token rates. Any id there works as `model=`; models outside the table above simply bill $0 in the dashboard until you add a `pricing={...}` override. + + +## Environment variables + +| Variable | Required | Notes | +|---|---|---| +| `ATLASCLOUD_API_KEY` | yes | Auto-loaded when `apiKey` / `api_key` is omitted. | + +## Options + +| Option | Default | Notes | +|---|---|---| +| `apiKey` / `api_key` | `undefined` | Reads from `ATLASCLOUD_API_KEY` when omitted. | +| `model` | `"openai/gpt-4.1-mini"` | Any id from `GET /v1/models`. | +| `baseUrl` / `base_url` | `https://api.atlascloud.ai/v1` | Override the Atlas Cloud endpoint (rarely needed). | +| `temperature`, `maxTokens`, `topP`, `seed`, `frequencyPenalty`, `presencePenalty`, `stop`, `responseFormat`, `parallelToolCalls`, `toolChoice` | unset | All forwarded to `chat.completions.create`. | + +## Prompt caching + +The gateway reports cached input in the standard OpenAI field (`usage.prompt_tokens_details.cached_tokens`). Patter subtracts it from `prompt_tokens` before billing, so cached tokens are charged once at the `cache_read` rate rather than at both rates. Where a model's `cache_read` equals its `input` rate the gateway is applying no cache discount for that model — the entry is still present in the pricing table, because omitting it would bill those tokens at $0. + +## Error handling + +Errors surface exactly as they do on `OpenAILLM` — the provider is the same engine with a different `baseUrl`. Failures retry once with exponential backoff; terminal errors raise `PatterError` so the LLM fallback chain can fail over and the spoken error fallback can fire. diff --git a/libraries/python/.env.example b/libraries/python/.env.example index 6193ae71..3ea08683 100644 --- a/libraries/python/.env.example +++ b/libraries/python/.env.example @@ -35,6 +35,7 @@ # LLM providers (pipeline mode) # ------------------------------------------------------------ # ANTHROPIC_API_KEY=sk-ant-... +# ATLASCLOUD_API_KEY=apikey-... # CEREBRAS_API_KEY= # GROQ_API_KEY= # GEMINI_API_KEY= diff --git a/libraries/python/getpatter/__init__.py b/libraries/python/getpatter/__init__.py index 00585437..869010da 100644 --- a/libraries/python/getpatter/__init__.py +++ b/libraries/python/getpatter/__init__.py @@ -160,6 +160,7 @@ from getpatter.llm.inworld import LLM as InworldLLM from getpatter.llm.google import LLM as GoogleLLM from getpatter.llm.litellm import LLM as LiteLLMLLM +from getpatter.llm.atlascloud import LLM as AtlasCloudLLM from getpatter.llm.openai_compatible import ( LLM as OpenAICompatibleLLM, OpenAICompatibleLLMProvider, @@ -577,6 +578,7 @@ def mix_pcm(agent: bytes, bg: bytes, ratio: float) -> bytes: "InworldLLM", "GoogleLLM", "LiteLLMLLM", + "AtlasCloudLLM", "OpenAICompatibleLLM", "OpenAICompatibleLLMProvider", "CustomLLM", diff --git a/libraries/python/getpatter/init/cli.py b/libraries/python/getpatter/init/cli.py index 147f36a4..d6e1b082 100644 --- a/libraries/python/getpatter/init/cli.py +++ b/libraries/python/getpatter/init/cli.py @@ -170,6 +170,12 @@ "env": ("GROQ_API_KEY",), "extra": "groq", }, + "atlascloud": { + "class": "AtlasCloudLLM", + "label": "Atlas Cloud", + "env": ("ATLASCLOUD_API_KEY",), + "extra": "atlascloud", + }, "google": { "class": "GoogleLLM", "label": "Google Gemini", diff --git a/libraries/python/getpatter/llm/__init__.py b/libraries/python/getpatter/llm/__init__.py index 2ae6b346..dc7032b8 100644 --- a/libraries/python/getpatter/llm/__init__.py +++ b/libraries/python/getpatter/llm/__init__.py @@ -21,6 +21,7 @@ "cerebras", "google", "litellm", + "atlascloud", "openai_compatible", "custom", "hermes", diff --git a/libraries/python/getpatter/llm/atlascloud.py b/libraries/python/getpatter/llm/atlascloud.py new file mode 100644 index 00000000..257de49c --- /dev/null +++ b/libraries/python/getpatter/llm/atlascloud.py @@ -0,0 +1,40 @@ +"""Atlas Cloud LLM for Patter pipeline mode.""" + +from __future__ import annotations + +import os +from typing import ClassVar + +from getpatter.providers.atlascloud_llm import AtlasCloudLLMProvider as _AtlasCloudLLM + +__all__ = ["LLM"] + + +class LLM(_AtlasCloudLLM): + """Atlas Cloud LLM provider (OpenAI-compatible Chat Completions). + + Example:: + + from getpatter.llm import atlascloud + + llm = atlascloud.LLM() # reads ATLASCLOUD_API_KEY + llm = atlascloud.LLM(api_key="apikey-...", model="deepseek-ai/deepseek-v3.2") + """ + + provider_key: ClassVar[str] = "atlascloud" + + def __init__( + self, + api_key: str | None = None, + *, + model: str = "openai/gpt-4.1-mini", + **kwargs, + ) -> None: + key = api_key or os.environ.get("ATLASCLOUD_API_KEY") + if not key: + raise ValueError( + "Atlas Cloud LLM requires an api_key. Pass " + "api_key='apikey-...' or set ATLASCLOUD_API_KEY in the " + "environment." + ) + super().__init__(api_key=key, model=model, **kwargs) diff --git a/libraries/python/getpatter/pricing.py b/libraries/python/getpatter/pricing.py index 31700003..26043df7 100644 --- a/libraries/python/getpatter/pricing.py +++ b/libraries/python/getpatter/pricing.py @@ -686,6 +686,27 @@ def calculate_realtime_cached_savings( "mixtral-8x7b-32768": {"input": 0.27, "output": 0.27}, "gemma2-9b-it": {"input": 0.20, "output": 0.20}, }, + "atlascloud": { + # USD per 1M tokens, as of 2026-09-15. Read from the gateway's own + # catalog rather than a docs page: ``GET + # https://api.atlascloud.ai/v1/models`` returns per-model + # ``pricing.prompt`` / ``pricing.completion`` / + # ``pricing.input_cache_read`` in USD *per token* — multiply by 1e6. + # That endpoint lists every routable model (118 at the time of + # writing); the entries below are the ones in ``AtlasCloudModel``. + # Unlisted models bill $0 here, so add an entry when you point Patter + # at one. Where ``cache_read`` equals ``input`` the gateway applies no + # cache discount for that model — the key is still required, because + # omitting it bills cached tokens at $0. + "openai/gpt-4.1-mini": {"input": 0.40, "output": 1.60, "cache_read": 0.10}, + "openai/gpt-5.4-mini": {"input": 0.75, "output": 4.50, "cache_read": 0.075}, + "deepseek-ai/deepseek-v3.2": {"input": 0.26, "output": 0.38, "cache_read": 0.13}, + "Qwen/Qwen3-235B-A22B-Instruct-2507": {"input": 0.20, "output": 0.88, "cache_read": 0.20}, + "qwen/qwen3.5-35b-a3b": {"input": 0.225, "output": 1.80, "cache_read": 0.225}, + "zai-org/GLM-4.6": {"input": 0.60, "output": 2.20, "cache_read": 0.11}, + "moonshotai/kimi-k2.5": {"input": 0.49, "output": 2.50, "cache_read": 0.20}, + "minimaxai/minimax-m2.5": {"input": 0.295, "output": 1.20, "cache_read": 0.06}, + }, "cerebras": { # Rates as of 2026-05-11 verified against the canonical per-model docs # pages at ``https://inference-docs.cerebras.ai/models/``. The @@ -716,7 +737,7 @@ def calculate_llm_cost( Args: provider: Provider key in :data:`LLM_PRICING` (``"anthropic"``, - ``"google"``, ``"groq"``, ``"cerebras"``). + ``"google"``, ``"groq"``, ``"cerebras"``, ``"atlascloud"``). model: Model identifier under the provider (e.g. ``"claude-haiku-4-5"``). input_tokens: Non-cached input tokens billed at the full rate. diff --git a/libraries/python/getpatter/providers/__init__.py b/libraries/python/getpatter/providers/__init__.py index cac09047..fd32927d 100644 --- a/libraries/python/getpatter/providers/__init__.py +++ b/libraries/python/getpatter/providers/__init__.py @@ -217,6 +217,12 @@ def _load_groq_llm(): return GroqLLMProvider +def _load_atlascloud_llm(): + from getpatter.providers.atlascloud_llm import AtlasCloudLLMProvider + + return AtlasCloudLLMProvider + + def _load_cerebras_llm(): from getpatter.providers.cerebras_llm import CerebrasLLMProvider @@ -240,6 +246,7 @@ def __getattr__(name: str): loaders = { "AnthropicLLMProvider": _load_anthropic_llm, "GroqLLMProvider": _load_groq_llm, + "AtlasCloudLLMProvider": _load_atlascloud_llm, "CerebrasLLMProvider": _load_cerebras_llm, "GoogleLLMProvider": _load_google_llm, "LiteLLMProvider": _load_litellm, @@ -272,6 +279,7 @@ def __getattr__(name: str): "sarvam", "AnthropicLLMProvider", "GroqLLMProvider", + "AtlasCloudLLMProvider", "CerebrasLLMProvider", "GoogleLLMProvider", "LiteLLMProvider", diff --git a/libraries/python/getpatter/providers/atlascloud_llm.py b/libraries/python/getpatter/providers/atlascloud_llm.py new file mode 100644 index 00000000..01caccd4 --- /dev/null +++ b/libraries/python/getpatter/providers/atlascloud_llm.py @@ -0,0 +1,113 @@ +"""Atlas Cloud LLM provider for Patter's pipeline mode. + +Atlas Cloud's Chat Completions API is OpenAI-compatible, so this provider is a +thin wrapper around :class:`getpatter.services.llm_loop.OpenAILLMProvider` +that points at ``https://api.atlascloud.ai/v1``. All sampling kwargs +(``response_format``, ``parallel_tool_calls``, ``tool_choice``, ``seed``, +``top_p``, ``frequency_penalty``, ``presence_penalty``, ``stop``, +``temperature``, ``max_tokens``) are inherited from the parent and +forwarded to ``chat.completions.create`` automatically. +""" + +from __future__ import annotations + +import logging +import os +from enum import StrEnum +from typing import ClassVar + +from getpatter.services.llm_loop import OpenAILLMProvider + +__all__ = ["AtlasCloudLLMProvider", "AtlasCloudModel"] + +logger = logging.getLogger("getpatter.providers.atlascloud_llm") + + +class AtlasCloudModel(StrEnum): + """Known Atlas Cloud Chat Completions models. The gateway serves many more; + ``GET /v1/models`` lists what is currently routable.""" + + GPT_4_1_MINI = "openai/gpt-4.1-mini" + GPT_5_4_MINI = "openai/gpt-5.4-mini" + DEEPSEEK_V3_2 = "deepseek-ai/deepseek-v3.2" + QWEN_3_235B_INSTRUCT = "Qwen/Qwen3-235B-A22B-Instruct-2507" + QWEN_3_5_35B_A3B = "qwen/qwen3.5-35b-a3b" + GLM_4_6 = "zai-org/GLM-4.6" + KIMI_K2_5 = "moonshotai/kimi-k2.5" + MINIMAX_M2_5 = "minimaxai/minimax-m2.5" + + +_ATLASCLOUD_BASE_URL = "https://api.atlascloud.ai/v1" +_DEFAULT_MODEL = AtlasCloudModel.GPT_4_1_MINI.value + + +class AtlasCloudLLMProvider(OpenAILLMProvider): + """LLM provider backed by Atlas Cloud's OpenAI-compatible Chat Completions API. + + Streams in the same ``{"type": "text" | "tool_call" | "done"}`` chunk + format as :class:`OpenAILLMProvider`. All OpenAI-spec sampling kwargs + accepted by the parent (``response_format``, ``parallel_tool_calls``, + ``tool_choice``, ``seed``, ``top_p``, ``frequency_penalty``, + ``presence_penalty``, ``stop``, ``temperature``, ``max_tokens``) are + forwarded transparently — see :class:`OpenAILLMProvider` for details. + + Args: + api_key: Atlas Cloud API key. If omitted, ``ATLASCLOUD_API_KEY`` is read from + the environment. + model: Atlas Cloud model ID. Defaults to ``openai/gpt-4.1-mini``. + base_url: Optional Atlas Cloud base URL override. + **kwargs: Sampling kwargs forwarded to + :class:`OpenAILLMProvider`. + """ + + #: Stable pricing/dashboard key — read by stream-handler/metrics. + provider_key: ClassVar[str] = "atlascloud" + + def __init__( + self, + api_key: str | None = None, + model: AtlasCloudModel | str = _DEFAULT_MODEL, + base_url: str = _ATLASCLOUD_BASE_URL, + **kwargs, + ) -> None: + try: + from openai import AsyncOpenAI + except ImportError as e: + raise RuntimeError( + "The 'openai' package is required for AtlasCloudLLMProvider. " + "Install it with: pip install 'getpatter[atlascloud]'" + ) from e + + resolved_key = api_key or os.environ.get("ATLASCLOUD_API_KEY") + if not resolved_key: + raise ValueError( + "Atlas Cloud API key is required, either as the 'api_key' " + "argument or via the ATLASCLOUD_API_KEY environment variable." + ) + + # Initialise parent state (model, sampling kwargs, _user_agent) + # without using its OpenAI-pointed client. We swap in an Atlas-pointed + # client below using the same User-Agent the parent computed. + super().__init__(api_key=resolved_key, model=model, **kwargs) + self._client = AsyncOpenAI( + api_key=resolved_key, + base_url=base_url, + default_headers={"User-Agent": self._user_agent}, + ) + + def _record_completion_cost( + self, *, prompt_tokens: int, completion_tokens: int + ) -> None: + """Stamp ``patter.cost.llm_*_tokens`` with the Atlas Cloud provider tag.""" + try: + from getpatter.observability.attributes import record_patter_attrs + + record_patter_attrs( + { + "patter.cost.llm_input_tokens": prompt_tokens, + "patter.cost.llm_output_tokens": completion_tokens, + "patter.llm.provider": "atlascloud", + } + ) + except Exception: # pragma: no cover — defense in depth + logger.debug("_record_completion_cost failed", exc_info=True) diff --git a/libraries/python/getpatter/stream_handler.py b/libraries/python/getpatter/stream_handler.py index e9b31a7a..5787f818 100644 --- a/libraries/python/getpatter/stream_handler.py +++ b/libraries/python/getpatter/stream_handler.py @@ -1273,6 +1273,7 @@ def _infer_llm_provider(self) -> str: "anthropic", "cerebras", "groq", + "atlascloud", "google", "gemini", "openai", diff --git a/libraries/python/getpatter/telemetry/stack.py b/libraries/python/getpatter/telemetry/stack.py index cd9e6d9b..cf305a54 100644 --- a/libraries/python/getpatter/telemetry/stack.py +++ b/libraries/python/getpatter/telemetry/stack.py @@ -32,6 +32,7 @@ class constant (minification-safe, the canonical vendor id) and stores its "google", "cerebras", "groq", + "atlascloud", "deepgram", "elevenlabs", "cartesia", diff --git a/libraries/python/pyproject.toml b/libraries/python/pyproject.toml index d41120c8..cc336eb5 100644 --- a/libraries/python/pyproject.toml +++ b/libraries/python/pyproject.toml @@ -139,6 +139,10 @@ anthropic = [ groq = [ "openai>=1.0", ] +# Atlas Cloud LLM provider (OpenAI-compatible aggregator gateway). +atlascloud = [ + "openai>=1.0", +] # Cerebras LLM provider (OpenAI-compatible with optional msgpack/gzip payload compression). cerebras = [ "openai>=1.0", diff --git a/libraries/python/tests/test_pricing.py b/libraries/python/tests/test_pricing.py index b2de8d18..47493de2 100644 --- a/libraries/python/tests/test_pricing.py +++ b/libraries/python/tests/test_pricing.py @@ -474,6 +474,17 @@ def test_cerebras_all_supported_models_billed(self): assert cost > 0.0, f"cerebras model {model!r} silently billed $0" assert model in LLM_PRICING["cerebras"], f"missing rate for {model!r}" + def test_atlascloud_all_models_billed(self): + """Every model in the AtlasCloudModel enum has a pricing entry (no $0 holes).""" + # Driven off the enum itself so adding a model without a rate fails here + # rather than silently billing $0 in production. + from getpatter.providers.atlascloud_llm import AtlasCloudModel + + for model in (m.value for m in AtlasCloudModel): + cost = calculate_llm_cost("atlascloud", model, 10_000, 10_000) + assert cost > 0.0, f"atlascloud model {model!r} silently billed $0" + assert model in LLM_PRICING["atlascloud"], f"missing rate for {model!r}" + def test_groq_all_models_billed(self): """Every model in the GroqModel enum has a pricing entry (no $0 holes).""" # Mirrors libraries/python/getpatter/providers/groq_llm.py GroqModel. diff --git a/libraries/python/tests/unit/test_atlascloud_llm.py b/libraries/python/tests/unit/test_atlascloud_llm.py new file mode 100644 index 00000000..3aae827e --- /dev/null +++ b/libraries/python/tests/unit/test_atlascloud_llm.py @@ -0,0 +1,124 @@ +"""Tests for the Atlas Cloud LLM provider (OpenAI-compatible gateway). + +Atlas Cloud fronts many upstream model vendors behind one OpenAI-compatible +Chat Completions endpoint, so the provider is a thin +:class:`OpenAILLMProvider` subclass that only swaps the client's ``base_url`` +and credential source. These tests pin the three things that swap can get +wrong: the endpoint actually points at Atlas, the parent's User-Agent survives +the client replacement (the gateway rejects the default ``python-httpx`` UA +family at the edge), and cost attribution is tagged ``atlascloud`` rather than +inheriting ``openai`` from the parent. +""" + +from __future__ import annotations + +from unittest.mock import patch + +import pytest + +from getpatter.llm import atlascloud as atlascloud_ns +from getpatter.providers.atlascloud_llm import AtlasCloudLLMProvider, AtlasCloudModel + + +def _provider(**kwargs) -> AtlasCloudLLMProvider: + kwargs.setdefault("api_key", "apikey-test") + return AtlasCloudLLMProvider(**kwargs) + + +def test_default_model_is_gpt_4_1_mini() -> None: + """Default is the cheapest broadly-available chat model on the gateway. + + Voice turns are short and latency-sensitive; the larger reasoning models in + :class:`AtlasCloudModel` stay reachable via ``model=``. + """ + assert _provider()._model == "openai/gpt-4.1-mini" + + +def test_explicit_model_override_is_honoured() -> None: + assert _provider(model="deepseek-ai/deepseek-v3.2")._model == "deepseek-ai/deepseek-v3.2" + + +def test_model_enum_values_are_accepted() -> None: + """``AtlasCloudModel`` is a ``StrEnum``, so members pass straight through.""" + provider = _provider(model=AtlasCloudModel.GLM_4_6) + assert provider._model == "zai-org/GLM-4.6" + + +def test_client_points_at_the_atlas_cloud_gateway() -> None: + provider = _provider() + assert str(provider._client.base_url).rstrip("/") == "https://api.atlascloud.ai/v1" + + +def test_base_url_override_is_honoured() -> None: + provider = _provider(base_url="https://gateway.internal/v1") + assert str(provider._client.base_url).rstrip("/") == "https://gateway.internal/v1" + + +def test_client_carries_the_patter_user_agent() -> None: + """The swapped-in client must keep the User-Agent the parent computed. + + Replacing ``self._client`` drops the parent's default headers, so the UA + has to be re-applied explicitly — otherwise Patter traffic becomes + indistinguishable from a bare SDK call at the gateway. + """ + provider = _provider() + assert provider._client.default_headers["User-Agent"] == provider._user_agent + assert "getpatter" in provider._user_agent.lower() + + +def test_api_key_is_read_from_the_environment(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setenv("ATLASCLOUD_API_KEY", "apikey-from-env") + provider = AtlasCloudLLMProvider() + assert provider._client.api_key == "apikey-from-env" + + +def test_explicit_api_key_wins_over_the_environment( + monkeypatch: pytest.MonkeyPatch, +) -> None: + monkeypatch.setenv("ATLASCLOUD_API_KEY", "apikey-from-env") + assert _provider(api_key="apikey-explicit")._client.api_key == "apikey-explicit" + + +def test_missing_api_key_raises_with_a_named_env_var( + monkeypatch: pytest.MonkeyPatch, +) -> None: + monkeypatch.delenv("ATLASCLOUD_API_KEY", raising=False) + with pytest.raises(ValueError, match="ATLASCLOUD_API_KEY"): + AtlasCloudLLMProvider() + + +def test_provider_key_is_atlascloud() -> None: + """``provider_key`` drives pricing lookup and the dashboard breakdown.""" + assert AtlasCloudLLMProvider.provider_key == "atlascloud" + + +def test_cost_attribution_is_tagged_atlascloud() -> None: + """Without the override, spend would be attributed to ``openai`` (parent).""" + recorded: dict = {} + + def _capture(attrs): + recorded.update(attrs) + + with patch( + "getpatter.observability.attributes.record_patter_attrs", new=_capture + ): + _provider()._record_completion_cost(prompt_tokens=11, completion_tokens=22) + + assert recorded["patter.llm.provider"] == "atlascloud" + assert recorded["patter.cost.llm_input_tokens"] == 11 + assert recorded["patter.cost.llm_output_tokens"] == 22 + + +def test_namespace_llm_is_the_same_provider(monkeypatch: pytest.MonkeyPatch) -> None: + """``from getpatter.llm import atlascloud`` mirrors the other adapters.""" + monkeypatch.setenv("ATLASCLOUD_API_KEY", "apikey-from-env") + llm = atlascloud_ns.LLM() + assert isinstance(llm, AtlasCloudLLMProvider) + assert llm.provider_key == "atlascloud" + assert llm._model == "openai/gpt-4.1-mini" + + +def test_namespace_llm_raises_without_a_key(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.delenv("ATLASCLOUD_API_KEY", raising=False) + with pytest.raises(ValueError, match="ATLASCLOUD_API_KEY"): + atlascloud_ns.LLM() diff --git a/libraries/typescript/.env.example b/libraries/typescript/.env.example index 21ea0b49..0a7dc565 100644 --- a/libraries/typescript/.env.example +++ b/libraries/typescript/.env.example @@ -35,6 +35,7 @@ # LLM providers (pipeline mode) # ------------------------------------------------------------ # ANTHROPIC_API_KEY=sk-ant-... +# ATLASCLOUD_API_KEY=apikey-... # CEREBRAS_API_KEY= # GROQ_API_KEY= # GEMINI_API_KEY= diff --git a/libraries/typescript/src/index.ts b/libraries/typescript/src/index.ts index 0193f37a..dcdc5867 100644 --- a/libraries/typescript/src/index.ts +++ b/libraries/typescript/src/index.ts @@ -267,6 +267,8 @@ export { LLM as AnthropicLLM } from "./llm/anthropic"; export type { AnthropicLLMOptions } from "./llm/anthropic"; export { LLM as GroqLLM } from "./llm/groq"; export type { GroqLLMOptions } from "./llm/groq"; +export { LLM as AtlasCloudLLM } from "./llm/atlascloud"; +export type { AtlasCloudLLMOptions } from "./llm/atlascloud"; export { LLM as CerebrasLLM } from "./llm/cerebras"; export type { CerebrasLLMOptions } from "./llm/cerebras"; export { LLM as InworldLLM } from "./llm/inworld"; diff --git a/libraries/typescript/src/init/cli.ts b/libraries/typescript/src/init/cli.ts index 78057775..5dfd9eb4 100644 --- a/libraries/typescript/src/init/cli.ts +++ b/libraries/typescript/src/init/cli.ts @@ -160,6 +160,12 @@ const PIPELINE_LLM: Record = { env: ['GROQ_API_KEY'], extra: 'groq', }, + atlascloud: { + cls: 'AtlasCloudLLM', + label: 'Atlas Cloud', + env: ['ATLASCLOUD_API_KEY'], + extra: 'atlascloud', + }, google: { cls: 'GoogleLLM', label: 'Google Gemini', diff --git a/libraries/typescript/src/llm/atlascloud.ts b/libraries/typescript/src/llm/atlascloud.ts new file mode 100644 index 00000000..8e40cafa --- /dev/null +++ b/libraries/typescript/src/llm/atlascloud.ts @@ -0,0 +1,69 @@ +/** Atlas Cloud LLM for Patter pipeline mode. */ +import { AtlasCloudLLMProvider as _AtlasCloudLLM } from "../providers/atlascloud-llm"; + +/** Constructor options for the Atlas Cloud `LLM` adapter. */ +export interface AtlasCloudLLMOptions { + /** API key. Falls back to ATLASCLOUD_API_KEY env var when omitted. */ + apiKey?: string; + /** Model id (e.g. ``"openai/gpt-4.1-mini"``). */ + model?: string; + /** Override the OpenAI-compatible base URL (rarely needed). */ + baseUrl?: string; + /** Sampling temperature [0, 2]. */ + temperature?: number; + /** Max tokens in the assistant response (sent as ``max_completion_tokens``). */ + maxTokens?: number; + /** OpenAI-style ``response_format`` for JSON mode / structured outputs. */ + responseFormat?: Record; + /** Whether to allow parallel tool calls. */ + parallelToolCalls?: boolean; + /** ``"auto" | "none" | "required"`` or a specific tool object. */ + toolChoice?: string | Record; + /** Sampling seed. */ + seed?: number; + /** Nucleus sampling cutoff in [0, 1]. */ + topP?: number; + /** Penalty in [-2, 2] applied to repeated tokens. */ + frequencyPenalty?: number; + /** Penalty in [-2, 2] applied to seen tokens. */ + presencePenalty?: number; + /** Stop sequence(s). */ + stop?: string | string[]; +} + +/** + * Atlas Cloud LLM provider (OpenAI-compatible Chat Completions, streaming). + * + * @example + * ```ts + * import * as atlascloud from "getpatter/llm/atlascloud"; + * const llm = new atlascloud.LLM(); // reads ATLASCLOUD_API_KEY + * const llm = new atlascloud.LLM({ apiKey: "apikey-...", model: "deepseek-ai/deepseek-v3.2" }); + * ``` + */ +export class LLM extends _AtlasCloudLLM { + static readonly providerKey = "atlascloud"; + constructor(opts: AtlasCloudLLMOptions = {}) { + const key = opts.apiKey ?? process.env.ATLASCLOUD_API_KEY; + if (!key) { + throw new Error( + "Atlas Cloud LLM requires an apiKey. Pass { apiKey: 'apikey-...' } or set ATLASCLOUD_API_KEY.", + ); + } + super({ + apiKey: key, + model: opts.model, + baseUrl: opts.baseUrl, + temperature: opts.temperature, + maxTokens: opts.maxTokens, + responseFormat: opts.responseFormat, + parallelToolCalls: opts.parallelToolCalls, + toolChoice: opts.toolChoice, + seed: opts.seed, + topP: opts.topP, + frequencyPenalty: opts.frequencyPenalty, + presencePenalty: opts.presencePenalty, + stop: opts.stop, + }); + } +} diff --git a/libraries/typescript/src/pricing.ts b/libraries/typescript/src/pricing.ts index 6e855624..bd1f2aff 100644 --- a/libraries/typescript/src/pricing.ts +++ b/libraries/typescript/src/pricing.ts @@ -754,6 +754,25 @@ export const llmPricing: Record> = { 'gemini-live-2.5-flash-native-audio': { input: 0.30, output: 2.50 }, 'gemini-3.1-flash-live-preview': { input: 0.30, output: 2.50 }, }, + atlascloud: { + // USD per 1M tokens, as of 2026-09-15. Read from the gateway's own catalog + // rather than a docs page: GET https://api.atlascloud.ai/v1/models returns + // per-model pricing.prompt / pricing.completion / pricing.input_cache_read + // in USD *per token* — multiply by 1e6. That endpoint lists every routable + // model (118 at the time of writing); the entries below are the ones in + // AtlasCloudModel. Unlisted models bill $0 here, so add an entry when you + // point Patter at one. Where cache_read equals input the gateway applies no + // cache discount for that model — the key is still required, because + // omitting it bills cached tokens at $0. + 'openai/gpt-4.1-mini': { input: 0.4, output: 1.6, cache_read: 0.1 }, + 'openai/gpt-5.4-mini': { input: 0.75, output: 4.5, cache_read: 0.075 }, + 'deepseek-ai/deepseek-v3.2': { input: 0.26, output: 0.38, cache_read: 0.13 }, + 'Qwen/Qwen3-235B-A22B-Instruct-2507': { input: 0.2, output: 0.88, cache_read: 0.2 }, + 'qwen/qwen3.5-35b-a3b': { input: 0.225, output: 1.8, cache_read: 0.225 }, + 'zai-org/GLM-4.6': { input: 0.6, output: 2.2, cache_read: 0.11 }, + 'moonshotai/kimi-k2.5': { input: 0.49, output: 2.5, cache_read: 0.2 }, + 'minimaxai/minimax-m2.5': { input: 0.295, output: 1.2, cache_read: 0.06 }, + }, groq: { // Rates as of 2026-05-08; verify against groq.com/pricing. // ``llama-3.3-70b-versatile`` is the Patter default for Groq. The diff --git a/libraries/typescript/src/providers/atlascloud-llm.ts b/libraries/typescript/src/providers/atlascloud-llm.ts new file mode 100644 index 00000000..01856496 --- /dev/null +++ b/libraries/typescript/src/providers/atlascloud-llm.ts @@ -0,0 +1,188 @@ +/** + * Atlas Cloud LLM provider for Patter's pipeline mode. + * + * Atlas Cloud exposes an OpenAI-compatible Chat Completions API. We reuse the + * streaming code path by implementing the same SSE parser as + * ``OpenAILLMProvider`` but pointed at ``api.atlascloud.ai``. Defaults to + * ``openai/gpt-4.1-mini``. + */ + +import type { LLMChunk, LLMProvider, LLMStreamOptions } from "../llm-loop"; +import { + mergeAbortSignals, + createStreamIdleWatchdog, + LLM_STREAM_IDLE_TIMEOUT_MS, +} from "../llm-loop"; +import { parseOpenAISseStream } from './groq-llm'; +import { getLogger } from '../logger'; +import { VERSION } from '../version'; +import { PatterConnectionError } from '../errors'; + +const ATLASCLOUD_BASE_URL = 'https://api.atlascloud.ai/v1'; + +/** Known Atlas Cloud Chat Completions models. Availability depends on account tier. */ +export const AtlasCloudModel = { + GPT_4_1_MINI: 'openai/gpt-4.1-mini', + GPT_5_4_MINI: 'openai/gpt-5.4-mini', + DEEPSEEK_V3_2: 'deepseek-ai/deepseek-v3.2', + QWEN_3_235B_INSTRUCT: 'Qwen/Qwen3-235B-A22B-Instruct-2507', + QWEN_3_5_35B_A3B: 'qwen/qwen3.5-35b-a3b', + GLM_4_6: 'zai-org/GLM-4.6', + KIMI_K2_5: 'moonshotai/kimi-k2.5', + MINIMAX_M2_5: 'minimaxai/minimax-m2.5', +} as const; +/** Union of {@link AtlasCloudModel} string values. */ +export type AtlasCloudModel = (typeof AtlasCloudModel)[keyof typeof AtlasCloudModel]; + +const DEFAULT_MODEL: string = AtlasCloudModel.GPT_4_1_MINI; + +/** Constructor options for {@link AtlasCloudLLMProvider}. */ +export interface AtlasCloudLLMOptions { + apiKey: string; + model?: string; + baseUrl?: string; + /** Sampling temperature [0, 2]. */ + temperature?: number; + /** Max tokens in the assistant response (sent as ``max_completion_tokens``). */ + maxTokens?: number; + /** OpenAI-style ``response_format`` for JSON mode / structured outputs. */ + responseFormat?: Record; + /** Whether to allow parallel tool calls. */ + parallelToolCalls?: boolean; + /** ``"auto" | "none" | "required"`` or a specific tool object. */ + toolChoice?: string | Record; + /** Sampling seed. */ + seed?: number; + /** Nucleus sampling cutoff in [0, 1]. */ + topP?: number; + /** Penalty in [-2, 2] applied to repeated tokens. */ + frequencyPenalty?: number; + /** Penalty in [-2, 2] applied to seen tokens. */ + presencePenalty?: number; + /** Stop sequence(s). */ + stop?: string | string[]; +} + +/** LLM provider backed by Atlas Cloud's OpenAI-compatible Chat Completions API. */ +export class AtlasCloudLLMProvider implements LLMProvider { + /** Stable pricing/dashboard key — read by stream-handler/metrics. */ + static readonly providerKey = 'atlascloud'; + private readonly apiKey: string; + readonly model: string; + private readonly baseUrl: string; + private readonly temperature?: number; + private readonly maxTokens?: number; + private readonly responseFormat?: Record; + private readonly parallelToolCalls?: boolean; + private readonly toolChoice?: string | Record; + private readonly seed?: number; + private readonly topP?: number; + private readonly frequencyPenalty?: number; + private readonly presencePenalty?: number; + private readonly stop?: string | string[]; + + constructor(options: AtlasCloudLLMOptions) { + if (!options.apiKey) { + throw new Error( + 'Atlas Cloud API key is required. Pass it via { apiKey } or read ATLASCLOUD_API_KEY from the environment.', + ); + } + this.apiKey = options.apiKey; + this.model = options.model ?? DEFAULT_MODEL; + this.baseUrl = options.baseUrl ?? ATLASCLOUD_BASE_URL; + this.temperature = options.temperature; + this.maxTokens = options.maxTokens; + this.responseFormat = options.responseFormat; + this.parallelToolCalls = options.parallelToolCalls; + this.toolChoice = options.toolChoice; + this.seed = options.seed; + this.topP = options.topP; + this.frequencyPenalty = options.frequencyPenalty; + this.presencePenalty = options.presencePenalty; + this.stop = options.stop; + } + + /** + * Pre-call DNS / TLS warmup for the Atlas Cloud inference endpoint. + * Best-effort: 5 s timeout, all exceptions swallowed at debug level. + */ + async warmup(): Promise { + try { + await fetch(`${this.baseUrl}/models`, { + method: 'GET', + headers: { Authorization: `Bearer ${this.apiKey}` }, + signal: AbortSignal.timeout(5_000), + }); + } catch (err) { + getLogger().debug(`Atlas Cloud LLM warmup failed (best-effort): ${String(err)}`); + } + } + + /** Stream Patter-format LLM chunks from the Atlas Cloud chat completions API. */ + async *stream( + messages: Array>, + tools?: Array> | null, + opts?: LLMStreamOptions, + ): AsyncGenerator { + const body: Record = { + model: this.model, + messages, + stream: true, + stream_options: { include_usage: true }, + }; + if (this.temperature !== undefined) body.temperature = this.temperature; + if (this.maxTokens !== undefined) { + // Atlas Cloud accepts both, but ``max_completion_tokens`` is the modern OpenAI + // spec name and matches Cerebras/OpenAI parity. + body.max_completion_tokens = this.maxTokens; + } + if (this.responseFormat !== undefined) body.response_format = this.responseFormat; + if (this.parallelToolCalls !== undefined) body.parallel_tool_calls = this.parallelToolCalls; + if (this.toolChoice !== undefined) body.tool_choice = this.toolChoice; + if (this.seed !== undefined) body.seed = this.seed; + if (this.topP !== undefined) body.top_p = this.topP; + if (this.frequencyPenalty !== undefined) body.frequency_penalty = this.frequencyPenalty; + if (this.presencePenalty !== undefined) body.presence_penalty = this.presencePenalty; + if (this.stop !== undefined) body.stop = this.stop; + if (tools) body.tools = tools; + + // Idle watchdog (re-armed per chunk) instead of a whole-stream ceiling — + // see llm-loop.ts createStreamIdleWatchdog. + const idle = createStreamIdleWatchdog(); + const response = await fetch(`${this.baseUrl}/chat/completions`, { + method: 'POST', + headers: { + 'Content-Type': 'application/json', + Authorization: `Bearer ${this.apiKey}`, + 'User-Agent': `getpatter/${VERSION}`, + }, + body: JSON.stringify(body), + signal: mergeAbortSignals(opts?.signal, idle.signal), + }); + + if (!response.ok) { + const errText = await response.text(); + // Cap the logged/thrown body — provider 401 bodies have been observed to + // embed the rejected API-key prefix, which would otherwise land in logs. + getLogger().error(`Atlas Cloud API error: ${response.status} ${errText.slice(0, 200)}`); + // Throw (don't return silently) so the LLM fallback chain fails over and + // the spoken error fallback can fire — a silent return looks like success. + throw new PatterConnectionError( + `Atlas Cloud API returned ${response.status}: ${errText.slice(0, 200)}`, + ); + } + + try { + yield* parseOpenAISseStream(response, idle.touch); + } catch (err) { + if (idle.fired && !opts?.signal?.aborted) { + throw new PatterConnectionError( + `Atlas Cloud stream idle timeout — no data for ${LLM_STREAM_IDLE_TIMEOUT_MS / 1000}s`, + ); + } + throw err; + } finally { + idle.clear(); + } + } +} diff --git a/libraries/typescript/src/telemetry/stack.ts b/libraries/typescript/src/telemetry/stack.ts index 25860c8e..34ebaf90 100644 --- a/libraries/typescript/src/telemetry/stack.ts +++ b/libraries/typescript/src/telemetry/stack.ts @@ -24,6 +24,7 @@ export const STACK_VENDORS: ReadonlySet = new Set([ 'google', 'cerebras', 'groq', + 'atlascloud', 'deepgram', 'elevenlabs', 'cartesia', diff --git a/libraries/typescript/tests/atlascloud-llm.mocked.test.ts b/libraries/typescript/tests/atlascloud-llm.mocked.test.ts new file mode 100644 index 00000000..9081b2f3 --- /dev/null +++ b/libraries/typescript/tests/atlascloud-llm.mocked.test.ts @@ -0,0 +1,213 @@ +/** + * Tests for the Atlas Cloud LLM provider (OpenAI-compatible gateway). + * + * Mocked at the paid boundary only: ``globalThis.fetch`` stands in for the Atlas + * Cloud HTTP endpoint. Everything else — the provider, the shared SSE parser, + * the namespace adapter, the pricing lookup — is the real code path. + * + * Atlas Cloud fronts many upstream model vendors behind one OpenAI-compatible + * Chat Completions endpoint, so the provider only swaps the base URL and + * credential source. These tests pin what that swap can get wrong: the request + * actually goes to Atlas with the Patter User-Agent, the shared SSE parser is + * reused (cached prompt tokens must not be billed twice), and a non-2xx + * response throws instead of ending the stream silently — a silent end looks + * like a successful empty turn and leaves the caller in dead air. + */ + +import { describe, expect, it, vi, afterEach } from 'vitest'; +import { AtlasCloudLLMProvider, AtlasCloudModel } from '../src/providers/atlascloud-llm'; +import { LLM as AtlasCloudLLM } from '../src/llm/atlascloud'; +import { PatterConnectionError } from '../src/errors'; +import type { LLMChunk } from '../src/llm-loop'; + +const originalFetch = globalThis.fetch; +const originalKey = process.env.ATLASCLOUD_API_KEY; + +afterEach(() => { + globalThis.fetch = originalFetch; + if (originalKey === undefined) delete process.env.ATLASCLOUD_API_KEY; + else process.env.ATLASCLOUD_API_KEY = originalKey; + vi.restoreAllMocks(); +}); + +/** Build a streaming SSE Response from raw `data:` payload lines. */ +function sseResponse(payloads: string[]): Response { + const body = payloads.map((p) => `data: ${p}\n\n`).join('') + 'data: [DONE]\n\n'; + return new Response( + new ReadableStream({ + start(controller) { + controller.enqueue(new TextEncoder().encode(body)); + controller.close(); + }, + }), + { status: 200 }, + ); +} + +async function drain( + gen: AsyncGenerator, +): Promise { + const out: LLMChunk[] = []; + for await (const chunk of gen) out.push(chunk); + return out; +} + +describe('AtlasCloudLLMProvider construction', () => { + it('defaults to openai/gpt-4.1-mini', () => { + expect(new AtlasCloudLLMProvider({ apiKey: 'apikey-test' }).model).toBe( + 'openai/gpt-4.1-mini', + ); + }); + + it('honours an explicit model override', () => { + const provider = new AtlasCloudLLMProvider({ + apiKey: 'apikey-test', + model: AtlasCloudModel.DEEPSEEK_V3_2, + }); + expect(provider.model).toBe('deepseek-ai/deepseek-v3.2'); + }); + + it('requires an API key', () => { + expect(() => new AtlasCloudLLMProvider({ apiKey: '' })).toThrow(/ATLASCLOUD_API_KEY/); + }); + + it('exposes the atlascloud pricing key', () => { + expect(AtlasCloudLLMProvider.providerKey).toBe('atlascloud'); + }); +}); + +describe('AtlasCloudLLMProvider request shape', () => { + it('posts to the Atlas Cloud gateway with bearer auth and the Patter UA', async () => { + const fetchMock = vi.fn().mockResolvedValue(sseResponse([])); + globalThis.fetch = fetchMock as unknown as typeof fetch; + + const provider = new AtlasCloudLLMProvider({ apiKey: 'apikey-test', maxTokens: 64 }); + await drain(provider.stream([{ role: 'user', content: 'hi' }])); + + const [url, init] = fetchMock.mock.calls[0] as [string, RequestInit]; + expect(url).toBe('https://api.atlascloud.ai/v1/chat/completions'); + const headers = init.headers as Record; + expect(headers.Authorization).toBe('Bearer apikey-test'); + expect(headers['User-Agent']).toMatch(/^getpatter\//); + + const body = JSON.parse(init.body as string); + expect(body.model).toBe('openai/gpt-4.1-mini'); + expect(body.stream).toBe(true); + expect(body.stream_options).toEqual({ include_usage: true }); + // Modern OpenAI spec name — Atlas accepts both, parity with Cerebras/OpenAI. + expect(body.max_completion_tokens).toBe(64); + expect(body).not.toHaveProperty('temperature'); + }); + + it('honours a base URL override', async () => { + const fetchMock = vi.fn().mockResolvedValue(sseResponse([])); + globalThis.fetch = fetchMock as unknown as typeof fetch; + + const provider = new AtlasCloudLLMProvider({ + apiKey: 'apikey-test', + baseUrl: 'https://gateway.internal/v1', + }); + await drain(provider.stream([{ role: 'user', content: 'hi' }])); + + expect(fetchMock.mock.calls[0][0]).toBe('https://gateway.internal/v1/chat/completions'); + }); +}); + +describe('AtlasCloudLLMProvider streaming', () => { + it('yields text and tool-call chunks from the shared OpenAI SSE parser', async () => { + globalThis.fetch = vi.fn().mockResolvedValue( + sseResponse([ + JSON.stringify({ choices: [{ delta: { content: 'Hel' } }] }), + JSON.stringify({ choices: [{ delta: { content: 'lo' } }] }), + JSON.stringify({ + choices: [ + { + delta: { + tool_calls: [ + { index: 0, id: 'call_1', function: { name: 'lookup', arguments: '{"q":' } }, + ], + }, + }, + ], + }), + ]), + ) as unknown as typeof fetch; + + const provider = new AtlasCloudLLMProvider({ apiKey: 'apikey-test' }); + const chunks = await drain(provider.stream([{ role: 'user', content: 'hi' }])); + + expect(chunks.filter((c) => c.type === 'text').map((c) => c.content)).toEqual(['Hel', 'lo']); + const toolCall = chunks.find((c) => c.type === 'tool_call'); + expect(toolCall).toMatchObject({ index: 0, id: 'call_1', name: 'lookup' }); + }); + + it('subtracts cached prompt tokens so they are not billed twice', async () => { + globalThis.fetch = vi.fn().mockResolvedValue( + sseResponse([ + JSON.stringify({ + choices: [], + usage: { + prompt_tokens: 1000, + completion_tokens: 40, + prompt_tokens_details: { cached_tokens: 900 }, + }, + }), + ]), + ) as unknown as typeof fetch; + + const provider = new AtlasCloudLLMProvider({ apiKey: 'apikey-test' }); + const chunks = await drain(provider.stream([{ role: 'user', content: 'hi' }])); + + // prompt_tokens includes the cached portion; billing the full 1000 at the + // input rate AND 900 at the cache-read rate would double-count. + expect(chunks.find((c) => c.type === 'usage')).toMatchObject({ + inputTokens: 100, + outputTokens: 40, + cacheReadInputTokens: 900, + }); + }); +}); + +describe('AtlasCloudLLMProvider error handling', () => { + it('throws (not silently ends) on a non-2xx response', async () => { + globalThis.fetch = vi + .fn() + .mockResolvedValue(new Response('{"error":"invalid api key"}', { status: 401 })) as unknown as typeof fetch; + vi.spyOn(console, 'error').mockImplementation(() => {}); + + const provider = new AtlasCloudLLMProvider({ apiKey: 'apikey-bad' }); + await expect(drain(provider.stream([{ role: 'user', content: 'hi' }]))).rejects.toThrow( + PatterConnectionError, + ); + }); + + it('caps the error body it logs and rethrows', async () => { + globalThis.fetch = vi + .fn() + .mockResolvedValue(new Response('x'.repeat(5000), { status: 500 })) as unknown as typeof fetch; + const logs: string[] = []; + vi.spyOn(console, 'error').mockImplementation((...args: unknown[]) => { + logs.push(args.map(String).join(' ')); + }); + + const provider = new AtlasCloudLLMProvider({ apiKey: 'apikey-test' }); + await expect(drain(provider.stream([{ role: 'user', content: 'hi' }]))).rejects.toThrow( + /Atlas Cloud API returned 500/, + ); + expect(logs.join('\n')).not.toContain('x'.repeat(201)); + }); +}); + +describe('llm/atlascloud namespace adapter', () => { + it('reads ATLASCLOUD_API_KEY from the environment', () => { + process.env.ATLASCLOUD_API_KEY = 'apikey-from-env'; + const llm = new AtlasCloudLLM(); + expect(llm).toBeInstanceOf(AtlasCloudLLMProvider); + expect(llm.model).toBe('openai/gpt-4.1-mini'); + }); + + it('throws a named error when no key is available', () => { + delete process.env.ATLASCLOUD_API_KEY; + expect(() => new AtlasCloudLLM()).toThrow(/ATLASCLOUD_API_KEY/); + }); +}); diff --git a/libraries/typescript/tests/pricing.test.ts b/libraries/typescript/tests/pricing.test.ts index 28ef3e5b..e9e128d1 100644 --- a/libraries/typescript/tests/pricing.test.ts +++ b/libraries/typescript/tests/pricing.test.ts @@ -10,6 +10,7 @@ import { calculateLlmCost, llmPricing, } from '../src/pricing'; +import { AtlasCloudModel } from '../src/providers/atlascloud-llm'; describe('DEFAULT_PRICING', () => { it('includes all expected providers', () => { @@ -419,6 +420,16 @@ describe('LLM cost billing — Cerebras + Groq silent under-billing regression', } }); + it('every AtlasCloudModel enum value has a pricing entry (no $0 holes)', () => { + // Driven off the enum itself so adding a model without a rate fails here + // rather than silently billing $0 in production. + for (const model of Object.values(AtlasCloudModel)) { + const cost = calculateLlmCost('atlascloud', model, 10_000, 10_000); + expect(cost, `atlascloud model ${model} silently billed $0`).toBeGreaterThan(0); + expect(llmPricing.atlascloud[model], `missing rate for ${model}`).toBeDefined(); + } + }); + it('every GroqModel enum value has a pricing entry (no $0 holes)', () => { // Mirrors libraries/typescript/src/providers/groq-llm.ts GroqModel. const groqModels = [