Podcast voices: natural provider TTS, and system-voice hygiene

Two reports, one root: the platform's speech engine.

"Loud cracking at the end of every sentence" is the browser/OS TTS popping
between its internal sentence chunks (an idle cancel makes it worse). Hygiene
for that path: never cancel when nothing is pending, normalise the spoken text
(control characters, ellipses, dashes), and a 140ms beat between sections so
speech never runs into the engine's tail.

The real fix is a second voice source that does not use the platform engine:
- lib/providers carries TTS profiles (OpenAI gpt-4o-mini-tts, Groq
  playai-tts, any OpenAI-compatible /audio/speech behind the custom provider),
  male voices first.
- /api/tts proxies them with the same guards and the key still in headers.
- the picker gains System / Natural, with the provider's voice list.
- lib/podcast-audio plays one clip per section through a hidden <audio>
  element, measures the clip's real duration and gives each word a slice of it
  proportional to its spoken length - the highlight follows the audio exactly,
  and pause/seek/chapters become real audio operations. Clips cache per
  session, the next section generates while the current plays, and a voice
  change disposes the element and its blobs. Speed applies live (setRate)
  without dropping the cache.

podcast-test stubs /api/tts with a real silent WAV and pins the path: the
voice list, the request body, exactly one playing element, and the highlight
riding currentTime. 43 checks. tests/lib.mjs launches Chromium with the
autoplay policy relaxed (a real device has the listener's gesture).
Version 0.7.6, SW v98.
This commit is contained in:
Martin Dvořák 2026-10-16 16:34:46 +00:00
commit 8e1cd0855d
13 changed files with 997 additions and 63 deletions

View file

@ -1492,6 +1492,26 @@ Things I chose while building, with the reasoning:
into view as you type. No strings, so no i18n; the additive change sits on top of the merged
scene-heat rework unchanged.
104. **Natural voices — and the crack at the end of every sentence.** Two reports, one root:
the platform's speech engine. "There's a loud cracking sound at the end of every sentence" is
the browser/OS TTS (iOS and several Linux engines pop between their internal sentence chunks,
and an unnecessary `cancel()` makes it worse) — nothing in our code touches audio there. Both
halves are answered: the system path got hygiene (never cancel while idle, strip control
characters and normalise ellipses/dashes before speaking, and a 140 ms beat between sections
so speech never runs into the engine's tail), and the real fix is a second voice source that
does not use the platform engine at all. `lib/providers` now carries TTS profiles (OpenAI
`gpt-4o-mini-tts`, Groq `playai-tts`, and any OpenAI-compatible `/audio/speech` the custom
provider points at; male voices first), `/api/tts` proxies them so the key still never leaves
the browser, and the player's picker gains a **System / Natural** switch with the provider's
voice list. The natural engine (`lib/podcast-audio`) plays one clip per section through an
`<audio>` element: the clip's real duration is measured and each word gets a slice of it
proportional to its spoken length, so the highlight follows the audio exactly instead of
estimating — and pause, seek and chapter jumps become real audio operations. Clips are cached
per session, the next section is generated while the current one plays, and switching voice
disposes the old element and its blobs. The suite stubs the TTS endpoint with a real silent
WAV and pins the whole path (voice list, request body, one playing element, the highlight
riding `currentTime`) — 43 checks.
103. **Podcast mode: your notes, read aloud, highlighted word by word.** The third segment of
the note-powered wizards (Study deck ⇄ Homework ⇄ Podcast) takes the same intake — photos,
documents, pasted text, the share target — and turns it into a spoken episode instead of