Podcast voices: natural provider TTS, and system-voice hygiene

Two reports, one root: the platform's speech engine.

"Loud cracking at the end of every sentence" is the browser/OS TTS popping
between its internal sentence chunks (an idle cancel makes it worse). Hygiene
for that path: never cancel when nothing is pending, normalise the spoken text
(control characters, ellipses, dashes), and a 140ms beat between sections so
speech never runs into the engine's tail.

The real fix is a second voice source that does not use the platform engine:
- lib/providers carries TTS profiles (OpenAI gpt-4o-mini-tts, Groq
  playai-tts, any OpenAI-compatible /audio/speech behind the custom provider),
  male voices first.
- /api/tts proxies them with the same guards and the key still in headers.
- the picker gains System / Natural, with the provider's voice list.
- lib/podcast-audio plays one clip per section through a hidden <audio>
  element, measures the clip's real duration and gives each word a slice of it
  proportional to its spoken length - the highlight follows the audio exactly,
  and pause/seek/chapters become real audio operations. Clips cache per
  session, the next section generates while the current plays, and a voice
  change disposes the element and its blobs. Speed applies live (setRate)
  without dropping the cache.

podcast-test stubs /api/tts with a real silent WAV and pins the path: the
voice list, the request body, exactly one playing element, and the highlight
riding currentTime. 43 checks. tests/lib.mjs launches Chromium with the
autoplay policy relaxed (a real device has the listener's gesture).
Version 0.7.6, SW v98.
This commit is contained in:
Martin Dvořák 2026-10-16 16:34:46 +00:00
commit 8e1cd0855d
13 changed files with 997 additions and 63 deletions

View file

@ -87,6 +87,64 @@ export const PROVIDERS: ProviderInfo[] = [
},
];
/**
* Text-to-speech for the podcast player. Only OpenAI-compatible
* `/audio/speech` endpoints are wired (OpenAI, Groq and any local server the
* custom provider points at); the rest simply do not offer TTS. Voice ids come
* from each provider's catalogue — the male ones lead because a podcast host
* was the ask.
*/
export type TtsVoice = { id: string; label: string; male?: boolean };
export type TtsProfile = {
/** Default model for /audio/speech. */
model: string;
voices: TtsVoice[];
/** Response format to request (Groq speaks WAV, everything else MP3). */
format: "mp3" | "wav";
};
export const TTS_PROFILES: Record<string, TtsProfile> = {
openai: {
model: "gpt-4o-mini-tts",
format: "mp3",
voices: [
{ id: "onyx", label: "Onyx", male: true },
{ id: "echo", label: "Echo", male: true },
{ id: "ash", label: "Ash", male: true },
{ id: "fable", label: "Fable" },
{ id: "nova", label: "Nova" },
{ id: "coral", label: "Coral" },
{ id: "shimmer", label: "Shimmer" },
],
},
groq: {
model: "playai-tts",
format: "wav",
voices: [
{ id: "Fritz-PlayAI", label: "Fritz", male: true },
{ id: "Thunder-PlayAI", label: "Thunder", male: true },
{ id: "Mason-PlayAI", label: "Mason", male: true },
{ id: "Celeste-PlayAI", label: "Celeste" },
{ id: "Cheyenne-PlayAI", label: "Cheyenne" },
],
},
custom: {
model: "tts-1",
format: "mp3",
voices: [
{ id: "onyx", label: "Onyx", male: true },
{ id: "echo", label: "Echo", male: true },
{ id: "alloy", label: "Alloy" },
{ id: "nova", label: "Nova" },
],
},
};
export function ttsProfile(providerId: string): TtsProfile | null {
return TTS_PROFILES[providerId] ?? null;
}
export function providerInfo(id: string): ProviderInfo | undefined {
return PROVIDERS.find((provider) => provider.id === id);
}