Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,37 @@

### Added

- **Atlas Cloud LLM provider** — `AtlasCloudLLM` lands in both SDKs, adding an
OpenAI-compatible aggregator gateway (`https://api.atlascloud.ai/v1`) that
fronts 100+ open and frontier models behind one key, so swapping models is a
`model` string rather than a new provider, key and pricing table:
- **`AtlasCloudLLMProvider` / `AtlasCloudLLM`** (Python
`getpatter.llm.atlascloud`, TS `getpatter/llm/atlascloud`, plus the flat
`AtlasCloudLLM` alias on the package root). It is `OpenAILLMProvider` with
an Atlas base URL, so streaming, tool calling and every OpenAI sampling
knob (`response_format`, `parallel_tool_calls`, `tool_choice`, `seed`,
`top_p`, `frequency_penalty`, `presence_penalty`, `stop`) are inherited
unchanged. Credentials come from `ATLASCLOUD_API_KEY` when `api_key` /
`apiKey` is omitted. Default model `openai/gpt-4.1-mini` — it answered a
tool-calling turn in 14 completion tokens with **zero reasoning tokens**,
where the reasoning-tuned models in the catalog spent 28–64 first; on a
phone call those are dead air.
- **Pricing for all eight `AtlasCloudModel` ids**, read from the gateway's
own catalog (`GET /v1/models` → `pricing.prompt` / `pricing.completion` /
`pricing.input_cache_read`) rather than a docs snapshot. `cache_read` is
populated even where it equals `input` (the gateway applies no discount for
those models) — omitting the key would have billed cached tokens at $0.
Both pricing tests are driven off the enum, so adding a model without a
rate fails CI instead of silently under-billing.
- **`atlascloud` extra** (`pip install 'getpatter[atlascloud]'`), an
`ATLASCLOUD_API_KEY` line in both `.env.example` files, an **Atlas Cloud
entry in `patter init`**'s pipeline LLM table (both runtimes), and
`atlascloud` added to the telemetry vendor allowlist so the stack field
reports the real vendor instead of collapsing to `other`.
- Docs: `providers/atlascloud.mdx` in both SDK sections (model + live
pricing table, prompt-cache semantics) and an `AtlasCloudLLM` entry in each
`llm.mdx`.

- **xAI (Grok) voice catalog wired into both SDKs** — the 26 built-in Grok
voices (`eve` default, `ara`, `rex`, `sal`, `leo`, `carina`, ...) are now a
typed, immutable catalog instead of docs-only prose:
Expand Down
2 changes: 2 additions & 0 deletions docs/docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,7 @@
"pages": [
"python-sdk/llm",
"python-sdk/providers/anthropic",
"python-sdk/providers/atlascloud",
"python-sdk/providers/cerebras",
"python-sdk/providers/google",
"python-sdk/providers/groq"
Expand Down Expand Up @@ -278,6 +279,7 @@
"pages": [
"typescript-sdk/llm",
"typescript-sdk/providers/anthropic",
"typescript-sdk/providers/atlascloud",
"typescript-sdk/providers/cerebras",
"typescript-sdk/providers/google",
"typescript-sdk/providers/groq"
Expand Down
19 changes: 18 additions & 1 deletion docs/python-sdk/llm.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ Tool calling works across every provider — each adapter normalizes its vendor-
| `AnthropicLLM` | `getpatter.llm.anthropic.LLM` | `ANTHROPIC_API_KEY` | `getpatter[anthropic]` |
| `GroqLLM` | `getpatter.llm.groq.LLM` | `GROQ_API_KEY` | `getpatter[groq]` |
| `CerebrasLLM` | `getpatter.llm.cerebras.LLM` | `CEREBRAS_API_KEY` | `getpatter[cerebras]` |
| `AtlasCloudLLM` | `getpatter.llm.atlascloud.LLM` | `ATLASCLOUD_API_KEY` | `getpatter[atlascloud]` |
| `GoogleLLM` | `getpatter.llm.google.LLM` | `GEMINI_API_KEY` (falls back to `GOOGLE_API_KEY`) | `getpatter[google]` |

All classes accept `api_key: str | None = None` and fall back to the listed env var when it is omitted.
Expand Down Expand Up @@ -103,6 +104,22 @@ llm = groq.LLM(api_key="gsk_...", model="llama-3.3-70b-versatile")

Install: `pip install 'getpatter[groq]'`.

### AtlasCloudLLM

[Atlas Cloud](https://www.atlascloud.ai) aggregator (OpenAI-compatible) at `https://api.atlascloud.ai/v1` — one key and one endpoint in front of 100+ open and frontier models, so changing model is a `model=` string rather than a new provider. Default model `"openai/gpt-4.1-mini"`; `GET /v1/models` lists every routable id with its live per-token rates.

Streaming, tool calling and all OpenAI sampling kwargs (`response_format`, `parallel_tool_calls`, `tool_choice`, `seed`, `top_p`, `frequency_penalty`, `presence_penalty`, `stop`) come from `OpenAILLMProvider` unchanged. Prompt-cache hits are reported in `usage.prompt_tokens_details.cached_tokens` and billed once at the `cache_read` rate.

```python
from getpatter import AtlasCloudLLM # flat
from getpatter.llm import atlascloud # namespaced

llm = AtlasCloudLLM() # reads ATLASCLOUD_API_KEY
llm = atlascloud.LLM(api_key="apikey-...", model="deepseek-ai/deepseek-v3.2")
```

Install: `pip install 'getpatter[atlascloud]'`. See the [Atlas Cloud provider page](/python-sdk/providers/atlascloud) for the model/pricing table.

### CerebrasLLM

Cerebras Inference API (OpenAI-compatible) at `https://api.cerebras.ai/v1`. Default model `"gpt-oss-120b"` — production tier, ~3000 tok/sec on WSE-3, no deprecation date. Pass `model="llama3.1-8b"` (8B params, sub-100ms TTFT) for the smaller free-tier alternative. The 404 `model_not_found` error includes a recovery hint listing other valid IDs (`qwen-3-235b-a22b-instruct-2507`, `llama-3.3-70b` on paid tier).
Expand Down Expand Up @@ -218,7 +235,7 @@ from getpatter import LLMChunk, DefaultToolExecutor

- `LLMChunk` — the streaming-output type yielded by every `LLMProvider.stream(...)` implementation. Carries either a partial text delta, a tool-call delta, or a stream-end marker.
- `DefaultToolExecutor` — the default tool dispatcher used by `LLMLoop`. Constructs from a `tools=` list and resolves both Python `handler=` callables and `webhook_url=` HTTP tools. Override its hooks to swap in custom error handling, telemetry, or authentication.
- `OpenAILLMProvider` — the parent class shared by `OpenAILLM`, `GroqLLM`, `CerebrasLLM`. Sampling kwargs (`temperature`, `top_p`, `seed`, `tool_choice`, `response_format`, …) live here and are forwarded by every subclass.
- `OpenAILLMProvider` — the parent class shared by `OpenAILLM`, `GroqLLM`, `CerebrasLLM`, `AtlasCloudLLM`. Sampling kwargs (`temperature`, `top_p`, `seed`, `tool_choice`, `response_format`, …) live here and are forwarded by every subclass.

These are stable public symbols mirrored byte-for-byte by the TypeScript SDK.

Expand Down
161 changes: 161 additions & 0 deletions docs/python-sdk/providers/atlascloud.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
---
title: "Atlas Cloud"
description: "Atlas Cloud LLM provider — one OpenAI-compatible endpoint in front of 100+ open and frontier models."
icon: "cloud"
---

# Atlas Cloud LLM

`AtlasCloudLLM` plugs Atlas Cloud's OpenAI-compatible Chat Completions API at `https://api.atlascloud.ai/v1` into Patter's pipeline mode. It is a thin wrapper around `OpenAILLMProvider` with an Atlas-specific base URL, so streaming, tool calling and every OpenAI sampling parameter behave exactly as they do on `OpenAILLM`.

## Why an aggregator for voice

A phone agent usually needs one model, but rarely the *same* model forever: you tune for latency, then a cheaper open-weights model lands, then one vendor rate-limits you at 6pm. Atlas Cloud fronts 100+ models — OpenAI, DeepSeek, Qwen, GLM, Kimi, MiniMax — behind a single key and a single endpoint, so switching is a `model=` string rather than a new provider, a new key and a new pricing table.

Two things matter specifically for voice:

- **Tool calling works across the catalog.** The models listed below were all verified to emit a streaming `tool_calls` delta for a `transfer_call` function — the pattern Patter's builtin tools rely on.
- **Reasoning tokens are dead air.** On a phone call, tokens the caller never hears are silence. The default `openai/gpt-4.1-mini` answered a transfer request in 14 completion tokens with zero reasoning tokens; the reasoning-tuned models in the catalog (`kimi-k2.5`, `minimax-m2.5`, `qwen3.5-35b-a3b`) spent 28–64 reasoning tokens first. Pick those deliberately, for the turns that need them.

## Install

```bash
pip install "getpatter[atlascloud]"
```

```bash
npm install getpatter
```

## Usage

<CodeGroup>

```python Python
# Namespaced import
from getpatter.llm import atlascloud

llm = atlascloud.LLM() # reads ATLASCLOUD_API_KEY
llm = atlascloud.LLM(api_key="apikey-...", model="deepseek-ai/deepseek-v3.2")
llm = atlascloud.LLM(
model="Qwen/Qwen3-235B-A22B-Instruct-2507",
temperature=0.4,
response_format={"type": "json_object"}, # OpenAI-style structured outputs
)

# Flat alias (equivalent)
from getpatter import AtlasCloudLLM

llm = AtlasCloudLLM()
```

```ts TypeScript
// Namespaced import
import * as atlascloud from "getpatter/llm/atlascloud";

const llm = new atlascloud.LLM(); // reads ATLASCLOUD_API_KEY
const llm2 = new atlascloud.LLM({ apiKey: "apikey-...", model: "deepseek-ai/deepseek-v3.2" });
const llm3 = new atlascloud.LLM({
model: "Qwen/Qwen3-235B-A22B-Instruct-2507",
temperature: 0.4,
responseFormat: { type: "json_object" },
});

// Flat alias (equivalent)
import { AtlasCloudLLM } from "getpatter";

const llm4 = new AtlasCloudLLM();
```

</CodeGroup>

<Note>
The namespaced import (`from getpatter.llm import atlascloud` / `import * as atlascloud from "getpatter/llm/atlascloud"`) auto-resolves the API key from `ATLASCLOUD_API_KEY` and exposes a uniform `LLM` class — the same pattern Patter uses for STT and TTS namespaces.
</Note>

Plug it into an agent:

<CodeGroup>

```python Python
import asyncio
from getpatter import Patter, Twilio, DeepgramSTT, AtlasCloudLLM, ElevenLabsTTS

phone = Patter(carrier=Twilio(), phone_number="+15550001234")

agent = phone.agent(
stt=DeepgramSTT(),
llm=AtlasCloudLLM(), # ATLASCLOUD_API_KEY from env
tts=ElevenLabsTTS(voice_id="rachel"),
system_prompt="You are a helpful assistant.",
first_message="Hi, how can I help?",
)

asyncio.run(phone.serve(agent))
```

```ts TypeScript
import { Patter, Twilio, DeepgramSTT, AtlasCloudLLM, ElevenLabsTTS } from "getpatter";

const phone = new Patter({ carrier: new Twilio(), phoneNumber: "+15550001234" });

const agent = phone.agent({
stt: new DeepgramSTT(),
llm: new AtlasCloudLLM(), // ATLASCLOUD_API_KEY from env
tts: new ElevenLabsTTS({ voiceId: "rachel" }),
systemPrompt: "You are a helpful assistant.",
firstMessage: "Hi, how can I help?",
});

await phone.serve(agent);
```

</CodeGroup>

Or scaffold a project with it preselected:

```bash
getpatter init --mode pipeline --llm atlascloud
```

## Supported models

These are the ids in the `AtlasCloudModel` enum and the ones carrying rates in `LLM_PRICING` / `llmPricing`. Pricing is USD per 1M tokens, read from the gateway's own catalog (`GET /v1/models` → `pricing.prompt` / `pricing.completion` / `pricing.input_cache_read`, in USD per token).

| Model | Input | Output | Cache read | Context | Notes |
|---|---|---|---|---|---|
| `openai/gpt-4.1-mini` (default) | $0.40 | $1.60 | $0.10 | 1,047,576 | No reasoning tokens — lowest dead air per turn. |
| `openai/gpt-5.4-mini` | $0.75 | $4.50 | $0.075 | 400,000 | Deepest cache discount (10× off input). |
| `deepseek-ai/deepseek-v3.2` | $0.26 | $0.38 | $0.13 | 163,840 | Cheapest output in the set. |
| `Qwen/Qwen3-235B-A22B-Instruct-2507` | $0.20 | $0.88 | $0.20 | 131,072 | Cheapest input; no cache discount. |
| `qwen/qwen3.5-35b-a3b` | $0.225 | $1.80 | $0.225 | 262,144 | Reasoning-tuned; no cache discount. |
| `zai-org/GLM-4.6` | $0.60 | $2.20 | $0.11 | 202,752 | |
| `moonshotai/kimi-k2.5` | $0.49 | $2.50 | $0.20 | 262,144 | Reasoning-tuned. |
| `minimaxai/minimax-m2.5` | $0.295 | $1.20 | $0.06 | 196,608 | Reasoning-tuned. |

<Note>
The gateway routes far more models than these — `GET https://api.atlascloud.ai/v1/models` lists everything currently available, with its live per-token rates. Any id there works as `model=`; models outside the table above simply bill $0 in the dashboard until you add a `pricing={...}` override.
</Note>

## Environment variables

| Variable | Required | Notes |
|---|---|---|
| `ATLASCLOUD_API_KEY` | yes | Auto-loaded when `api_key` / `apiKey` is omitted. |

## Options

| Option | Default | Notes |
|---|---|---|
| `api_key` / `apiKey` | `None` | Reads from `ATLASCLOUD_API_KEY` when omitted. |
| `model` | `"openai/gpt-4.1-mini"` | Any id from `GET /v1/models`. |
| `base_url` / `baseUrl` | `https://api.atlascloud.ai/v1` | Override the Atlas Cloud endpoint (rarely needed). |
| `temperature`, `max_tokens`, `top_p`, `seed`, `frequency_penalty`, `presence_penalty`, `stop`, `response_format`, `parallel_tool_calls`, `tool_choice` | unset | All forwarded to `chat.completions.create`. |

## Prompt caching

The gateway reports cached input in the standard OpenAI field (`usage.prompt_tokens_details.cached_tokens`). Patter subtracts it from `prompt_tokens` before billing, so cached tokens are charged once at the `cache_read` rate rather than at both rates. Where a model's `cache_read` equals its `input` rate the gateway is applying no cache discount for that model — the entry is still present in the pricing table, because omitting it would bill those tokens at $0.

## Error handling

Errors surface exactly as they do on `OpenAILLM` — the provider is the same engine with a different `base_url`. Failures retry once with exponential backoff; terminal errors raise `PatterError` so the LLM fallback chain can fail over and the spoken error fallback can fire.
18 changes: 17 additions & 1 deletion docs/typescript-sdk/llm.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ Tool calling works across every provider — each adapter normalizes its vendor-
| `AnthropicLLM` | `ANTHROPIC_API_KEY` | included |
| `GroqLLM` | `GROQ_API_KEY` | included |
| `CerebrasLLM` | `CEREBRAS_API_KEY` | included |
| `AtlasCloudLLM` | `ATLASCLOUD_API_KEY` | included |
| `GoogleLLM` | `GEMINI_API_KEY` (falls back to `GOOGLE_API_KEY`) | included |

All classes accept an options object with `apiKey?: string` and fall back to the listed env var when it is omitted.
Expand Down Expand Up @@ -93,6 +94,21 @@ const llm = new GroqLLM(); // reads GROQ_API_KEY
const llm2 = new GroqLLM({ apiKey: "gsk_...", model: "llama-3.3-70b-versatile" });
```

### AtlasCloudLLM

[Atlas Cloud](https://www.atlascloud.ai) aggregator (OpenAI-compatible) at `https://api.atlascloud.ai/v1` — one key and one endpoint in front of 100+ open and frontier models, so changing model is a `model` string rather than a new provider. Default model `"openai/gpt-4.1-mini"`; `GET /v1/models` lists every routable id with its live per-token rates.

Streaming, tool calling and all OpenAI sampling options (`responseFormat`, `parallelToolCalls`, `toolChoice`, `seed`, `topP`, `frequencyPenalty`, `presencePenalty`, `stop`) behave as on `OpenAILLM`. Prompt-cache hits are reported in `usage.prompt_tokens_details.cached_tokens` and billed once at the `cache_read` rate.

```typescript
import { AtlasCloudLLM } from "getpatter";

const llm = new AtlasCloudLLM(); // reads ATLASCLOUD_API_KEY
const llm2 = new AtlasCloudLLM({ apiKey: "apikey-...", model: "deepseek-ai/deepseek-v3.2" });
```

See the [Atlas Cloud provider page](/typescript-sdk/providers/atlascloud) for the model/pricing table.

### CerebrasLLM

Cerebras Inference API (OpenAI-compatible) at `https://api.cerebras.ai/v1`. Default model `"gpt-oss-120b"` — production tier, ~3000 tok/sec on WSE-3, no deprecation date. Pass `model: "llama3.1-8b"` for the smaller free-tier alternative. The 404 `model_not_found` error includes a recovery hint listing other valid IDs.
Expand Down Expand Up @@ -196,7 +212,7 @@ import { LLMChunk, DefaultToolExecutor, LLMLoop, OpenAILLMProvider } from "getpa

- `LLMChunk` — the streaming-output type yielded by every `LLMProvider.stream(...)` implementation. Carries either a partial text delta, a tool-call delta, or a stream-end marker.
- `DefaultToolExecutor` — the default tool dispatcher used by `LLMLoop`. Constructs from a `tools` array and resolves both inline `handler` callables and `webhookUrl` HTTP tools. Override its hooks to swap in custom error handling, telemetry, or authentication.
- `OpenAILLMProvider` — the parent class shared by `OpenAILLM`, `GroqLLM`, `CerebrasLLM`. Sampling options (`temperature`, `topP`, `seed`, `toolChoice`, `responseFormat`, …) live here and are forwarded by every subclass.
- `OpenAILLMProvider` — the parent class shared by `OpenAILLM`, `GroqLLM`, `CerebrasLLM`, `AtlasCloudLLM`. Sampling options (`temperature`, `topP`, `seed`, `toolChoice`, `responseFormat`, …) live here and are forwarded by every subclass.
- `LLMLoop` — the orchestration loop wiring an `LLMProvider`, a `DefaultToolExecutor`, and the streaming output back to TTS.

These are stable public symbols mirrored byte-for-byte by the Python SDK.
Expand Down
Loading