Podcast mode: notes read aloud with a lyric-style word highlight

The third segment of the note-powered wizards (Study deck / Homework /
Podcast) takes the same intake - photos, documents, pasted text, the share
target - and turns it into a spoken episode.

- prompts: PODCAST_SYSTEM_PROMPT writes for the ear (plain prose, no markdown,
  no visual references), chaptered into 3-8 sections around a length target.
- /api/podcast mirrors /api/generate's plumbing (rate limit, model gate,
  provider keys, heartbeats, the 2x deadline, the optional vision pass).
- storage: frezmo.podcasts.v1, zod-validated, included in the backup and the
  encrypted sync payload; a new library tab lists episodes with listen /
  rename / delete-with-undo, and the wizard switch deep-links in.
- player: the transcript is word-wrapped spans and the word under the voice
  rides a lyric-style highlight that keeps itself centred. Two clocks: exact
  onboundary events where the browser reports them, a per-word length timer
  where it does not (iOS Safari); the first boundary retires the timer.
  Pause = cancel + re-speak from the word, which makes resume, chapter jumps
  and tap-any-word-to-seek the same operation; the position saves on pause,
  page-hide and unmount, so reopening resumes. Speed rebuilds the engine and
  keeps playing; the Aero soundtrack pauses itself when the voice starts; a
  browser with no voice gets an honest note and a readable transcript.
- the script sits on a glass pane (plain paragraphs over Aero's grass failed
  contrast; the scan now covers the player too).

tests/podcast-test.mjs (17th suite) stubs the model and the speech engine and
drives the highlight by hand: 26 checks. All suites green, contrast clean.
Version 0.7.3, SW v95.
This commit is contained in:
Martin Dvořák 2026-10-16 11:42:15 +00:00
commit d95c5c7151
30 changed files with 2575 additions and 26 deletions

View file

@ -20,6 +20,8 @@ import {
HOMEWORK_CHECK_SYSTEM_PROMPT,
HOMEWORK_GUIDE_SYSTEM_PROMPT,
HOMEWORK_MAKE_SYSTEM_PROMPT,
PODCAST_SYSTEM_PROMPT,
buildPodcastUserPrompt,
REFINE_SYSTEM_PROMPT,
VISION_SYSTEM_PROMPT,
VISION_USER_INSTRUCTION,
@ -27,6 +29,7 @@ import {
import {
extractedNotesSchema,
generatedActivitySchema,
generatedPodcastSchema,
gradeResultSchema,
homeworkCheckSchema,
homeworkMakeSchema,
@ -38,6 +41,7 @@ import {
type HomeworkCheck,
type HomeworkGuide,
type HomeworkMake,
type GeneratedPodcast,
} from "./schemas";
import { parseJsonLoose } from "./utils";
import type { OllamaAuth } from "./config";
@ -401,6 +405,58 @@ export async function generateActivities(input: {
return { deck: value, requested };
}
/* ------------------------------------------------------------------ */
/* Podcast */
/* ------------------------------------------------------------------ */
const PODCAST_SHAPE_HINT = `{"title":"Nothing but the sky","language":"English","summary":"One sentence.","sections":[{"heading":"Why blue","text":"Sixty to a hundred and forty words of narration."}]}`;
export async function generatePodcast(input: {
model: string;
auth: OllamaAuth;
provider?: ProviderTarget;
notes: string;
minutes: number;
language?: string;
focus?: string;
signal?: AbortSignal;
}): Promise<{ podcast: GeneratedPodcast }> {
const minutes = Math.min(Math.max(input.minutes, 1), 15);
const { value } = await runJsonStep<GeneratedPodcast>({
model: input.model,
auth: input.auth,
provider: input.provider,
system: PODCAST_SYSTEM_PROMPT,
user: buildPodcastUserPrompt({
notes: input.notes,
minutes,
language: input.language,
focus: input.focus,
}),
/* One usable section beats a retry loop: the repair pass fills gaps. */
validate: (parsed) => {
const result = generatedPodcastSchema.safeParse(parsed);
if (!result.success) return { ok: false as const, error: summariseZodError(result.error) };
const sections = result.data.sections
.map((section) => ({
heading: section.heading.trim(),
text: section.text.trim().replace(/\s+/g, " "),
}))
.filter((section) => section.text.length > 0);
if (sections.length === 0) {
return { ok: false as const, error: "No sections with spoken text were found." };
}
return { ok: true as const, value: { ...result.data, sections } };
},
shapeHint: PODCAST_SHAPE_HINT,
label: "Writing the script",
signal: input.signal,
temperature: 0.75,
});
return { podcast: value };
}
/* ------------------------------------------------------------------ */
/* Grading */
/* ------------------------------------------------------------------ */