Repository navigation
feat(worker)!: transcribe at an endpoint instead of in the worker's own memory - #183
Merged
Merged
Conversation
Records the decision to delete sturnus.infrastructure.whisper and call an OpenAI-compatible /audio/transcriptions endpoint instead, so a deployment no longer needs a local model, a model-cache PVC or CTranslate2 in the image. Covers the chunking a 25 MB request limit forces, why every cut lands in silence (an HTTP response carries no Segment.seek, so whisper.py's restore cannot be expressed and attributing a segment by its reported times is the 20-to-31-second defect that file documents), which decoding vetoes move from decode time to the response and which are simply lost, and the three text changes that stop README.md, /about and CONSENT_STRINGS from claiming audio never leaves the pod. Section 6 is marked cuttable: making the model registry configurable is the only part that reaches outside the worker.
…n memory The worker loaded a CTranslate2 conversion of Whisper into its own process and decoded each speaker's track on a thread. That one decision is why the image carried CTranslate2, why the chart had a model-cache volume, why a first start downloaded several gigabytes, and why somebody with a Discord server and no GPU-adjacent hardware could not stand this project up at all. Transcription is now an HTTP call to an OpenAI-compatible /audio/transcriptions endpoint: the worker stops owning a model and starts owning a client. The engine is removed rather than made optional, and that was the choice rather than the consequence. A backend selected by environment variable would be two live code paths, two sets of tests and two sets of failure modes for a project with one deployment's worth of operational attention. A self-hoster who wants local computation runs a vLLM or Speaches container and points STURNUS_TRANSCRIPTION_BASE_URL at it -- a second container, not a second code path, and one somebody else maintains. STURNUS_TRANSCRIPTION_BASE_URL has no default, and that is a consent control rather than an oversight. A worker without it fails to construct its settings and does not start, so there is no configuration under which a recording leaves the pod because nobody chose otherwise. Operators must set it before upgrading; STURNUS_WHISPER_MODEL and STURNUS_MODEL_CACHE_DIR are gone, and STURNUS_WHISPER_DEFAULT_LANGUAGE is renamed to STURNUS_TRANSCRIPTION_DEFAULT_LANGUAGE with unchanged meaning. A track is divided into windows because the endpoint has a request size limit, and every cut lands inside a silence the speech gate reported. The cheaper shape -- dropping the gaps and concatenating the speech, which the old engine did -- cannot be expressed here. That engine attributed each decoded segment to a clip by Segment.seek precisely because a decoded end routinely overruns its window, and resolving the clip from that end adds the next clip's offset: measured at 20 to 31 seconds of displacement on a 100-minute recording. An HTTP response carries no seek, so the only safe arrangement is one where attribution is unnecessary. Two of the three decode-time hallucination vetoes move from decoder arguments to rules applied to the returned segments, as constants carrying their own measurements. avg_logprob is deliberately not a third: an invented subtitle credit is a fluent, high-probability sequence, which is exactly how " Untertitelung des ZDF, 2020" reached a production protocol with no_speech_threshold already set. A response omitting a figure means the rule that reads it does not apply, never that the job fails -- vLLM omits no_speech_prob, and docs/operations.md says which endpoint is the safer one to point at. Because plaintext audio now leaves the pod, three texts that would otherwise be false are corrected: the README's "never leave the pod" sentence is narrowed, and /about and the consent prompt name the configured host before anybody agrees to be recorded. The host only -- the key reaches no user-facing text, log field, span attribute or metric label. The registry follows the endpoint: whisper-1 is the only OpenAI model that returns segment timestamps, and a protocol interleaving several speakers chronologically cannot be assembled without them. Its entries lose their size and summary, which were honest while the worker downloaded the weights and cannot be written truthfully for a model somebody else serves. The stall alert moves with the window cap. Progress advances once per completed window rather than once per decoded segment, so the documented threshold of 600 seconds would have paged on every healthy multi-window job against a 720-second cap; it is now 1800 and documented as a function of that cap. The chart loses the model-cache PVC, the volume and mount, and the memory the weights required; the worker pod is stateless and rolls rather than recreates. The image loses CTranslate2 and faster-whisper.
Design rounds in this repository have lived as dated documents under docs/superpowers/specs/ and as prose in commit messages. Both stay, but neither answers "what does this system currently claim to do" without reading every round in order and reconciling them by hand. OpenSpec adds that missing half: openspec/specs/ holds what is true now, and a change under openspec/changes/ holds the delta until it is archived into it. openspec/config.yaml carries the project context every artifact is generated against -- the stack, the layering tests/test_architecture.py enforces, and the two constraints that decide designs here: that nobody is recorded without consent, and that no raw value reaches a log field a Sentry integration forwards verbatim. Its per-artifact rules ask for the prose this repository already writes: the decision and the failure it prevents, the alternatives that were rejected, and the test that proves a task done. The endpoint change is archived as the first one and leaves nine requirements in openspec/specs/transcription/spec.md -- where the audio is sent, how a track too large for one request is divided, which returned segments are refused as invented, how a track's language is decided, and what a person is told before consenting. One task in it is unfinished and is archived that way rather than ticked: walking the end-to-end checklist against a deployment pointed at a real endpoint, with one session over twelve minutes, is the only test that exercises the window boundaries against material nobody constructed. The .claude/ command and skill definitions are committed because they are what makes the workflow available to anybody who clones this; the settings, worktrees and lock file beside them are per-machine and are ignored.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Moves transcription out of the worker's own memory and into an HTTP call against an OpenAI-compatible
/audio/transcriptionsendpoint. The worker stops owning a model and starts owning a client.This branch carries three commits: the design document (
docs), the implementation (feat(worker)!), and the adoption of OpenSpec (chore) with this change recorded as its first spec underopenspec/changes/. The squash merge lands as thefeatcommit; the other two are the reasoning and the process that produced it.What it does
The worker no longer loads a CTranslate2 conversion of Whisper and decodes each speaker's track on a thread. It splits a track into windows, uploads each window to the configured endpoint, and assembles the returned segments into the same protocol as before. The image loses CTranslate2 and faster-whisper; the chart loses the model-cache PVC, its volume and mount, and the memory the weights required, so the worker pod is stateless and rolls instead of recreating.
/about, the consent prompt and the README now say that plaintext audio leaves the pod, and name the configured host.Why it is shaped this way
STURNUS_TRANSCRIPTION_BASE_URLhas no default, on purpose. It is a consent control. Without it the settings fail to construct and the process does not start, so there is no configuration under which a recording leaves the pod because nobody chose otherwise. The bot needs it as well as the worker, although the bot never calls it: the consent prompt has to name the host before somebody agrees to be recorded, and a bot naming one endpoint while the worker uploads to another would make that prompt a false statement. The chart therefore carries it once, incommonEnv.Segment.seek, because a decoded end routinely overruns its window; resolving the clip from that end adds the next clip's offset, measured at 20 to 31 seconds of displacement on a 100-minute recording. An HTTP response carries no seek, so the only safe arrangement is one where attribution is unnecessary.avg_logprobis deliberately not a third: an invented subtitle credit is a fluent, high-probability sequence. A response that omits a figure means the rule reading it does not apply, never that the job fails (vLLM omitsno_speech_prob;docs/operations.mdsays which endpoint is the safer one).whisper-1is the only OpenAI model that returns segment timestamps, and a protocol interleaving speakers chronologically cannot be assembled without them. Its entries lose their size and summary, which were honest while the worker downloaded the weights and cannot be written truthfully for a model somebody else serves.What it deliberately refuses to do
Checks
pytest -q2731 passed ·ruff check·ruff format --check·mypy(293 files) · consolevitest run1719 passed (56 files) ·vue-tsc --noEmit— all clean.BREAKING CHANGE:
STURNUS_TRANSCRIPTION_BASE_URLis now required by both the bot and the worker and has no default; set it (chart:commonEnv.STURNUS_TRANSCRIPTION_BASE_URL) before upgrading, or neither process starts. The in-process faster-whisper engine is removed;STURNUS_WHISPER_MODELandSTURNUS_MODEL_CACHE_DIRno longer exist,STURNUS_WHISPER_DEFAULT_LANGUAGEis renamed toSTURNUS_TRANSCRIPTION_DEFAULT_LANGUAGE, and the chart's workerpersistenceblock and model-cache PVC are removed.STURNUS_TRANSCRIPTION_API_KEYis the new worker secret, and the default model iswhisper-1.