Skip to content

feat(python): InworldRealtime speech-to-speech engine - #224

Open
fra-shipper wants to merge 1 commit into
mainfrom
feat/py-inworld-realtime
Open

fra-shipper wants to merge 1 commit into
mainfrom
feat/py-inworld-realtime

Conversation

@fra-shipper

Copy link
Copy Markdown
Collaborator

Summary

  • Adds InworldRealtime, a speech-to-speech engine for Inworld's Realtime API
    (getpatter.engines.inworld.InworldRealtime, flat alias from the package
    root), so a Python caller can do InworldRealtime(voice="Ashley") the same
    way they'd use XaiRealtime or OpenAIRealtime2.
  • Adds InworldRealtimeAdapter (getpatter/providers/inworld_realtime.py),
    which subclasses the v1 OpenAIRealtimeAdapter and overrides only the
    WebSocket endpoint (wss://api.inworld.ai/v1/realtime) and provider_key
    ("inworld") — Inworld advertises an OpenAI-Realtime migration path with a
    matching event schema, so no new session-handling code is needed.
  • Threads a new inworld_key parameter through server.py,
    stream_handler.py, and the three telephony bridges (twilio.py,
    telnyx.py, plivo.py), and treats inworld_realtime as g711_ulaw
    pass-through in each bridge's mulaw-detection branch, matching the OpenAI
    realtime modes.
  • Adds "inworld_realtime" to the ProviderMode literal (additive — no
    existing members removed) and a new Agent.inworld_realtime: dict | None = None field carrying engine-supplied session knobs to the stream-handler.
  • Wires cost telemetry so Inworld calls report zero AI cost deliberately:
    CallMetricsAccumulator now excludes inworld_realtime (alongside the
    existing xai_realtime exclusion) from the OpenAI-rate-table token
    accounting, and the _compute_cost branch explicitly zeroes stt/tts/llm
    cost with a comment citing Inworld's published pricing page (no Realtime
    rate as of 2026-08-24) rather than silently defaulting to $0 by omission.
  • Adds LocalConfig.inworld_key and docs
    (docs/python-sdk/engines.mdx overview section,
    docs/python-sdk/providers/inworld-realtime.mdx full reference,
    docs/docs.json nav entry).

Defaults changed: none — every new field/parameter is additive with a
safe default (inworld_key: str = "", Agent.inworld_realtime: dict | None = None, LocalConfig.inworld_key: str = "").

Removed ids: none — ProviderMode only gains "inworld_realtime".

Scope note: Python only. libraries/typescript/ has no equivalent
changes in this diff, which is a sdk-parity rule violation as written; flag
for a follow-up PR or confirm this is an intentional staged Python-first
landing before merge.

Verification

  • New unit test file libraries/python/tests/unit/test_inworld_realtime.py:
    21 test functions covering the adapter (endpoint/provider_key, subclassing,
    defaults, base_url override + trailing-slash trim, session-config shape,
    transcription-language pinning, connect/auth, setup-error surfacing, audio
    send/receive, tool-call round trip, warmup no-op + unsupported parking), the
    engine marker (unpacking, env-key fallback, missing-key error), agent
    dispatch selecting the Inworld provider, and the cost-telemetry exclusions
    (meters against the existing Inworld pricing entry, does not price against
    the OpenAI rate table, reports zero AI cost deliberately).
  • CI: see checks.

Notes / follow-ups

  • Python-only change — no TypeScript port in this PR (see Scope note
    above); needed to satisfy the repo's sdk-parity rule before merge, per
    .claude/rules/sdk-parity.md.
  • The adapter's own docstring/comments flag that "the exact production wire
    details (whether an Inworld account requires a session-scoped
    ?key=<session-id> obtained from a prior REST call, and which audio
    formats are accepted) are not fully public" — defaults are the
    OpenAI-compatible ones, override base_url / audio_format per account.
    Docs mark the feature Beta — spec-validated, not yet live-call-validated.
  • Inworld sessions are not parked during ringing (Patter._park_provider_connections
    only parks OpenAI realtime modes), so the first word on an Inworld call
    always pays the cold-connect path — called out explicitly in
    docs/python-sdk/engines.mdx.
  • Cost telemetry zeroes Inworld AI cost by design pending a published Inworld
    Realtime rate; services/metrics.py comments point at re-metering this
    once Inworld ships a rate and a matching pricing.py entry exists.
  • All touched/added files stay well under the repo's file-size caps (largest
    new file is the 445-line test file; client.py/server.py/metrics.py
    edits are incremental diffs, not full-file rewrites).

🤖 Generated with Claude Code

@mintlify

mintlify Bot commented Aug 24, 2026 •

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
patter-06b046ce 🟢 Ready View Preview Aug 24, 2026, 10:46 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

This branch was successfully deployed

1 active deployment
staging - docs — 46fcd281 Deployed Aug 24, 2026 by mintlify[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant