Skip to content

Add natural local voice and contextual journey replies - #3

Merged
SayamDev merged 1 commit into
mainfrom
codex/natural-contextual-voice
Oct 12, 2026
Merged

SayamDev merged 1 commit into
mainfrom
codex/natural-contextual-voice

Conversation

@SayamDev

Copy link
Copy Markdown
Owner

Spoken journey replies repeated the departure time and read ISO dates/ranges as written. This adds shorter contextual replies (pickup, drop-off, wait/get-ready, late/passed times and low confidence) and an optional natural British Kokoro voice. Every time, window and confidence still comes from the existing deterministic engine; neither prediction nor the text form changes.

Design decisions:

  • English understanding stays in the existing deterministic action/parser pipeline. Speech templates read the engine directly, including data-limit warnings; no LLM interprets journeys or changes numbers.
  • Kokoro generates audio locally in a separate worker after explicit opt-in. Supported GPUs use fp32/WebGPU (~350 MB); other browsers use q8/WASM (~115 MB), plus the speech runtime. Screen results remain immediate. British/enhanced local browser voices provide the default and failure/loading fallback.
  • Short chunks prevent tokenizer truncation. Cancellation, stale request ids, bounded load/generation/playback and late voice discovery protect interruptions and lifecycle changes. Cached files recreate the runtime on reopening; readiness is never inferred from a saved preference. The UI identifies the voice actually used and explains playback/timeout fallback.
  • No audio or transcripts are stored/logged. Only the voice preference and isolated model/style caches persist. Hugging Face sees public file requests/IP, not journey text. Clear all data also removes model/style files; the service worker preserves these and only removes obsolete shell caches.
  • VoiceStudio was considered but requires visitors to run a local API. No hosted voice API, account, key, cloning or new backend is introduced. Original app source remains MIT; the optional combined worker contains GPL eSpeak. Licence notices, source archives and a worker banner accompany it. Node-only dependency overrides resolve sharp/global-agent advisories. CSP now permits self/blob audio and same-origin workers without allowing remote scripts/audio.

Validation: lint, strict TypeScript, production build and npm audit (zero vulnerabilities) pass. The expanded suite has 398 tests covering contextual numbers/dates, midnight windows, warnings, cache/runtime restoration, intentional cancellation, stale audio, device fallback and voice discovery, plus the existing full utterance-to-engine path. An evening run exposed two existing timetable fixtures whose arrivals moved past midnight; they now use a fixed midday test clock, with no production change. Chrome production preview verified model download/warm-up, actual WebGPU natural playback, cached reload, Stop, 390px layout and 48px touch targets with no horizontal overflow.

Limits: CPU speech can miss the deadline and fall back; older phones may have memory/battery constraints and gaps between sentences. Physical iOS/Android autoplay/lockscreen, noisy-car recognition and subjective listening tests remain pending. Free-data/callsign limits remain unchanged. Further speech quality/latency work needs device testing and an efficient on-device model; missing flight/queue/traffic facts need reliable licensed data. Rollback is a PR revert, with no journey-schema migration.

@SayamDev
SayamDev merged commit 6d46ef0 into main Oct 12, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant