Skip to content

feat(worker)!: transcribe at an endpoint instead of in the worker's own memory - #183

Merged
TheMeinerLP merged 3 commits into
mainfrom
docs/openai-transcription-spec
Oct 14, 2026
Merged

TheMeinerLP merged 3 commits into
mainfrom
docs/openai-transcription-spec

Conversation

@TheMeinerLP

Copy link
Copy Markdown
Contributor

Moves transcription out of the worker's own memory and into an HTTP call against an OpenAI-compatible /audio/transcriptions endpoint. The worker stops owning a model and starts owning a client.

This branch carries three commits: the design document (docs), the implementation (feat(worker)!), and the adoption of OpenSpec (chore) with this change recorded as its first spec under openspec/changes/. The squash merge lands as the feat commit; the other two are the reasoning and the process that produced it.

What it does

The worker no longer loads a CTranslate2 conversion of Whisper and decodes each speaker's track on a thread. It splits a track into windows, uploads each window to the configured endpoint, and assembles the returned segments into the same protocol as before. The image loses CTranslate2 and faster-whisper; the chart loses the model-cache PVC, its volume and mount, and the memory the weights required, so the worker pod is stateless and rolls instead of recreating. /about, the consent prompt and the README now say that plaintext audio leaves the pod, and name the configured host.

Why it is shaped this way

  • The engine is removed, not made optional. A backend chosen by environment variable is two live code paths, two test suites and two sets of failure modes for a project with one deployment's worth of operational attention. A self-hoster who wants local computation runs a vLLM or Speaches container and points the base URL at it: a second container, not a second code path, and one somebody else maintains.
  • STURNUS_TRANSCRIPTION_BASE_URL has no default, on purpose. It is a consent control. Without it the settings fail to construct and the process does not start, so there is no configuration under which a recording leaves the pod because nobody chose otherwise. The bot needs it as well as the worker, although the bot never calls it: the consent prompt has to name the host before somebody agrees to be recorded, and a bot naming one endpoint while the worker uploads to another would make that prompt a false statement. The chart therefore carries it once, in commonEnv.
  • Every window boundary lands inside a silence the speech gate reported. The old engine dropped the gaps and concatenated the speech, then attributed each decoded segment to a clip by Segment.seek, because a decoded end routinely overruns its window; resolving the clip from that end adds the next clip's offset, measured at 20 to 31 seconds of displacement on a 100-minute recording. An HTTP response carries no seek, so the only safe arrangement is one where attribution is unnecessary.
  • Two of the three hallucination vetoes move from decoder arguments to rules over the returned segments, as constants carrying their own measurements. avg_logprob is deliberately not a third: an invented subtitle credit is a fluent, high-probability sequence. A response that omits a figure means the rule reading it does not apply, never that the job fails (vLLM omits no_speech_prob; docs/operations.md says which endpoint is the safer one).
  • The model registry follows the endpoint. whisper-1 is the only OpenAI model that returns segment timestamps, and a protocol interleaving speakers chronologically cannot be assembled without them. Its entries lose their size and summary, which were honest while the worker downloaded the weights and cannot be written truthfully for a model somebody else serves.
  • The stall alert moves with the window cap. Progress now advances once per completed window, so the documented 600 second threshold would page on every healthy multi-window job against the 720 second cap; it is 1800 and documented as a function of that cap.

What it deliberately refuses to do

  • No in-process engine, no fallback, no "local" mode behind a flag.
  • No default endpoint, and no silent choice of a vendor for the operator.
  • The API key reaches no user-facing text, log field, span attribute or metric label. Only the host is ever shown.
  • It does not fail a job over a missing optional figure in the response.

Checks

pytest -q 2731 passed · ruff check · ruff format --check · mypy (293 files) · console vitest run 1719 passed (56 files) · vue-tsc --noEmit — all clean.

BREAKING CHANGE: STURNUS_TRANSCRIPTION_BASE_URL is now required by both the bot and the worker and has no default; set it (chart: commonEnv.STURNUS_TRANSCRIPTION_BASE_URL) before upgrading, or neither process starts. The in-process faster-whisper engine is removed; STURNUS_WHISPER_MODEL and STURNUS_MODEL_CACHE_DIR no longer exist, STURNUS_WHISPER_DEFAULT_LANGUAGE is renamed to STURNUS_TRANSCRIPTION_DEFAULT_LANGUAGE, and the chart's worker persistence block and model-cache PVC are removed. STURNUS_TRANSCRIPTION_API_KEY is the new worker secret, and the default model is whisper-1.

Records the decision to delete sturnus.infrastructure.whisper and call an
OpenAI-compatible /audio/transcriptions endpoint instead, so a deployment no
longer needs a local model, a model-cache PVC or CTranslate2 in the image.

Covers the chunking a 25 MB request limit forces, why every cut lands in
silence (an HTTP response carries no Segment.seek, so whisper.py's restore
cannot be expressed and attributing a segment by its reported times is the
20-to-31-second defect that file documents), which decoding vetoes move from
decode time to the response and which are simply lost, and the three text
changes that stop README.md, /about and CONSENT_STRINGS from claiming audio
never leaves the pod.

Section 6 is marked cuttable: making the model registry configurable is the
only part that reaches outside the worker.
…n memory

The worker loaded a CTranslate2 conversion of Whisper into its own process
and decoded each speaker's track on a thread. That one decision is why the
image carried CTranslate2, why the chart had a model-cache volume, why a
first start downloaded several gigabytes, and why somebody with a Discord
server and no GPU-adjacent hardware could not stand this project up at all.
Transcription is now an HTTP call to an OpenAI-compatible
/audio/transcriptions endpoint: the worker stops owning a model and starts
owning a client.

The engine is removed rather than made optional, and that was the choice
rather than the consequence. A backend selected by environment variable
would be two live code paths, two sets of tests and two sets of failure
modes for a project with one deployment's worth of operational attention.
A self-hoster who wants local computation runs a vLLM or Speaches container
and points STURNUS_TRANSCRIPTION_BASE_URL at it -- a second container, not
a second code path, and one somebody else maintains.

STURNUS_TRANSCRIPTION_BASE_URL has no default, and that is a consent
control rather than an oversight. A worker without it fails to construct
its settings and does not start, so there is no configuration under which a
recording leaves the pod because nobody chose otherwise. Operators must set
it before upgrading; STURNUS_WHISPER_MODEL and STURNUS_MODEL_CACHE_DIR are
gone, and STURNUS_WHISPER_DEFAULT_LANGUAGE is renamed to
STURNUS_TRANSCRIPTION_DEFAULT_LANGUAGE with unchanged meaning.

A track is divided into windows because the endpoint has a request size
limit, and every cut lands inside a silence the speech gate reported. The
cheaper shape -- dropping the gaps and concatenating the speech, which the
old engine did -- cannot be expressed here. That engine attributed each
decoded segment to a clip by Segment.seek precisely because a decoded end
routinely overruns its window, and resolving the clip from that end adds
the next clip's offset: measured at 20 to 31 seconds of displacement on a
100-minute recording. An HTTP response carries no seek, so the only safe
arrangement is one where attribution is unnecessary.

Two of the three decode-time hallucination vetoes move from decoder
arguments to rules applied to the returned segments, as constants carrying
their own measurements. avg_logprob is deliberately not a third: an
invented subtitle credit is a fluent, high-probability sequence, which is
exactly how " Untertitelung des ZDF, 2020" reached a production protocol
with no_speech_threshold already set. A response omitting a figure means
the rule that reads it does not apply, never that the job fails -- vLLM
omits no_speech_prob, and docs/operations.md says which endpoint is the
safer one to point at.

Because plaintext audio now leaves the pod, three texts that would
otherwise be false are corrected: the README's "never leave the pod"
sentence is narrowed, and /about and the consent prompt name the configured
host before anybody agrees to be recorded. The host only -- the key reaches
no user-facing text, log field, span attribute or metric label.

The registry follows the endpoint: whisper-1 is the only OpenAI model that
returns segment timestamps, and a protocol interleaving several speakers
chronologically cannot be assembled without them. Its entries lose their
size and summary, which were honest while the worker downloaded the weights
and cannot be written truthfully for a model somebody else serves.

The stall alert moves with the window cap. Progress advances once per
completed window rather than once per decoded segment, so the documented
threshold of 600 seconds would have paged on every healthy multi-window job
against a 720-second cap; it is now 1800 and documented as a function of
that cap.

The chart loses the model-cache PVC, the volume and mount, and the memory
the weights required; the worker pod is stateless and rolls rather than
recreates. The image loses CTranslate2 and faster-whisper.
Design rounds in this repository have lived as dated documents under
docs/superpowers/specs/ and as prose in commit messages. Both stay, but
neither answers "what does this system currently claim to do" without
reading every round in order and reconciling them by hand. OpenSpec adds
that missing half: openspec/specs/ holds what is true now, and a change
under openspec/changes/ holds the delta until it is archived into it.

openspec/config.yaml carries the project context every artifact is
generated against -- the stack, the layering tests/test_architecture.py
enforces, and the two constraints that decide designs here: that nobody is
recorded without consent, and that no raw value reaches a log field a
Sentry integration forwards verbatim. Its per-artifact rules ask for the
prose this repository already writes: the decision and the failure it
prevents, the alternatives that were rejected, and the test that proves a
task done.

The endpoint change is archived as the first one and leaves nine
requirements in openspec/specs/transcription/spec.md -- where the audio is
sent, how a track too large for one request is divided, which returned
segments are refused as invented, how a track's language is decided, and
what a person is told before consenting. One task in it is unfinished and
is archived that way rather than ticked: walking the end-to-end checklist
against a deployment pointed at a real endpoint, with one session over
twelve minutes, is the only test that exercises the window boundaries
against material nobody constructed.

The .claude/ command and skill definitions are committed because they are
what makes the workflow available to anybody who clones this; the settings,
worktrees and lock file beside them are per-machine and are ignored.
@TheMeinerLP
TheMeinerLP merged commit b8473e4 into main Oct 14, 2026
13 checks passed
@TheMeinerLP
TheMeinerLP deleted the docs/openai-transcription-spec branch October 15, 2026 11:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant