Chat mode, phase 1: grounded streaming chat over your notes
Entry points (no sixth tab): Ask in the Library header (desktop button, phone icon), Ask about this deck in the deck menu, Ask a question on study/homework results (compact result travels as context), and a Chat row in the desktop sidebar. Phones get a full-screen route with back + New chat; the tab bar steps aside. - Client-side retrieval: chunk scoped decks (~800 tokens), BM25 against the question + recent turns, top chunks under ~7k tokens, each labelled with deck + section. Notes come from the deck's stored sourceText (already kept for Regenerate); decks without it fall back to generated content and the UI says so. - Stateless server: /api/chat validates against the CHAT_MODELS allowlist, trims the history to ~6k tokens, caps context at ~9k and streams token/thinking/done/error NDJSON. /api/chat/extract reduces attachments to text through the existing vision step. - UI: user bubbles + full-width assistant text, streaming with stop, auto-scroll that pauses with a jump-to-bottom button, safe markdown (React nodes, no HTML injection; KaTeX dynamically imported only when a message has math), Thinking row for reasoning, copy/regenerate/ edit-and-resend, model + context pickers (last model remembered), composer with visualViewport keyboard handling, safe-area, 16px textarea, Enter-to-send on desktop. - Usage-limit failures keep the draft and show the shared message with Settings/Ollama actions; stopping maps the reader abort to a clean stopped state; offline disables sending with a reason. - Settings: Keep notes text for chat toggle; deck menu: Delete notes text. RATE_LIMIT_CHAT + CHAT_MODELS in .env.example and README. - Tests: chat suite (63 checks, stub Ollama) wired into run-all. Full set: 12/12 suites, 487 checks. SW -> freezmo-v32. Also: tests/model-compare.mjs (diagnostic) benchmarks two models on the same notes (latency, repair rounds, activity types) so the gpt-oss vs gemma default can be decided on numbers; defaults unchanged.
This commit is contained in:
parent
6b6a6ea158
commit
756a6257a2
46 changed files with 3840 additions and 56 deletions
95
README.md
95
README.md
|
|
@ -29,6 +29,7 @@ no database, no data leaving your machine except the requests you send to Ollama
|
|||
- [Install as an app (PWA)](#install-as-an-app-pwa)
|
||||
- [Security notes](#security-notes)
|
||||
- [How it works](#how-it-works)
|
||||
- [Chat](#chat)
|
||||
- [Project structure](#project-structure)
|
||||
- [API routes](#api-routes)
|
||||
- [Scripts](#scripts)
|
||||
|
|
@ -65,6 +66,11 @@ no database, no data leaving your machine except the requests you send to Ollama
|
|||
study sessions and homework guides; wizard drafts survive a reload or an OS kill; re-tapping
|
||||
the active tab scrolls to the top; an offline pill appears and generation is disabled with
|
||||
a reason until the connection is back.
|
||||
- **Chat with your notes**: an LLM-style conversation grounded in your decks. Ask from the
|
||||
Library, a deck's menu, or a result; pick the scope (all notes, folders/decks, or none) and the
|
||||
model; photos and documents are transcribed first, so any model works. Answers stream with stop,
|
||||
copy, regenerate and edit-and-resend, render safe markdown + math, and hide reasoning behind a
|
||||
**Thinking** row. See [Chat](#chat).
|
||||
- **Learning engine**: every answer is recorded (deck, activity, topic, correct/wrong, time) in an
|
||||
IndexedDB log, and each activity gets a light **SM-2 schedule** — correct answers come back after
|
||||
1 day, then 3, then interval × ease; wrong ones after ten minutes. The Library shows a **Smart
|
||||
|
|
@ -441,6 +447,7 @@ client IP with a sliding window. Everything is configurable in `.env.local`:
|
|||
| `POST /api/homework` | 12 per 5 minutes | `RATE_LIMIT_HOMEWORK` |
|
||||
| `POST /api/grade` | 40 per 5 minutes | `RATE_LIMIT_GRADE` |
|
||||
| `POST /api/homework/refine` | 30 per 5 minutes | `RATE_LIMIT_REFINE` |
|
||||
| `POST /api/chat`, `POST /api/chat/extract` | 60 per 5 minutes | `RATE_LIMIT_CHAT` |
|
||||
| `GET /api/health` | 90 per 5 minutes | `RATE_LIMIT_HEALTH` |
|
||||
| `POST/GET/DELETE /api/sync` | 60 per 5 minutes | `RATE_LIMIT_SYNC` |
|
||||
| `GET/POST/DELETE /api/push`, `POST /api/push/test` | 30 per 5 minutes | `RATE_LIMIT_PUSH` |
|
||||
|
|
@ -455,6 +462,10 @@ Caveats, stated honestly: counters live **in memory**, so they reset on restart
|
|||
between multiple instances or Node workers. That is fine for a self-hosted single-instance app; if
|
||||
you put Freezmo behind a reverse proxy, keep the proxy's own limits in place too.
|
||||
|
||||
Chat has its own allowlist: `CHAT_MODELS` (comma-separated) lists what the model chip may offer,
|
||||
with the first entry as the default; when unset it falls back to `TEXT_MODEL` and `VISION_MODEL`.
|
||||
The server rejects any model name outside that list.
|
||||
|
||||
Model names are normalised automatically: when the base URL points at `ollama.com`, Freezmo strips
|
||||
the local routing suffix (`gemma4:31b-cloud` → `gemma4:31b`, `gpt-oss:120b-cloud` → `gpt-oss:120b`)
|
||||
because the cloud API expects plain library names, while a local daemon wants the `-cloud` form.
|
||||
|
|
@ -672,6 +683,47 @@ Limits and behaviour worth knowing:
|
|||
|
||||
---
|
||||
|
||||
## Chat
|
||||
|
||||
Chat mode is an LLM-style conversation that knows your notes — ask about a deck, get quizzed, or
|
||||
have a step explained without leaving the app.
|
||||
|
||||
- **Entry points.** The Library header has **Ask** (a labelled button on desktop, an icon on
|
||||
phones), every deck's menu has **Ask about this deck** (opens scoped to that deck), the desktop
|
||||
sidebar has a **Chat** row, and study/homework results have **Ask a question** (the compact
|
||||
result travels along as extra context). On phones the chat is a full-screen route with a back
|
||||
button and **New chat** — the tab bar steps aside so the composer gets the space. Desktop keeps
|
||||
the sidebar.
|
||||
- **Grounded answers, retrieved in the browser.** Chat reads each deck's stored notes text
|
||||
(`sourceText` — the same text *Regenerate* uses, already kept and size-capped per deck). The
|
||||
client chunks the scoped decks (~800 tokens per chunk), ranks the chunks against your question
|
||||
and the recent turns with a small BM25 scorer, and sends only the top chunks (~7k tokens, each
|
||||
labelled with its deck and section). The server stays stateless: the chosen context and the
|
||||
conversation window travel with every request, exactly like the API key. Decks without notes text
|
||||
fall back to their generated study content, and the chat says so.
|
||||
- **Scope.** The context chip picks **All notes**, **selected folders/decks** or **No notes**
|
||||
(general chat); opening from a deck defaults to that deck. A Settings toggle (**Keep notes text
|
||||
for chat**) makes chat ignore the notes text, and each deck's menu has **Delete notes text** to
|
||||
remove it from that deck entirely.
|
||||
- **Models.** The model chip lists the server's allowlist (`CHAT_MODELS`, defaulting to the
|
||||
configured text and vision models) with text/vision badges and availability from the health
|
||||
check; the last choice is remembered. The server validates every request against the same
|
||||
allowlist and never accepts a model name from the client.
|
||||
- **Attachments.** Photos, PDFs and documents go through the existing vision/extraction step first
|
||||
(plain text), so any chat model works — text-only included.
|
||||
- **Streaming and controls.** Tokens stream with a caret and a stop button; auto-scroll pauses when
|
||||
you scroll up and a jump-to-bottom button appears; answers render safe markdown (lists, tables,
|
||||
code blocks with copy — no HTML injection) and KaTeX loads dynamically only when a message
|
||||
contains math; reasoning models' thinking is hidden behind a collapsible **Thinking** row and
|
||||
never mixed into the answer. Every answer offers copy / regenerate, the last question edit and
|
||||
resend. The composer auto-grows at 16px, sends on Enter on desktop (newline on phones), tracks
|
||||
the on-screen keyboard through `visualViewport` and respects reduced motion.
|
||||
- **Privacy.** Notes text and conversations stay in this browser; nothing is stored server-side.
|
||||
The API key still only ever travels in the `x-ollama-key` header to Freezmo's own route for that
|
||||
single request.
|
||||
|
||||
---
|
||||
|
||||
## Project structure
|
||||
|
||||
```
|
||||
|
|
@ -684,15 +736,19 @@ app/
|
|||
create/page.tsx upload screen
|
||||
deck/[id]/page.tsx study feed for a saved deck
|
||||
homework/page.tsx “Make my homework” (assignment in, homework or guide out)
|
||||
chat/page.tsx chat mode (full-screen on phones, sidebar on desktop)
|
||||
api/generate/route.ts images/text in → NDJSON progress → deck JSON out
|
||||
api/homework/route.ts assignment in → NDJSON progress → homework/guide JSON out
|
||||
api/grade/route.ts short-answer grading
|
||||
api/chat/route.ts grounded chat turns → NDJSON token/thinking stream
|
||||
api/chat/extract/route.ts chat attachments → plain text (vision step)
|
||||
api/health/route.ts Ollama reachability + model availability
|
||||
components/
|
||||
activities/ one component per activity type + shared shell
|
||||
Flashcard.tsx MultipleChoice.tsx TrueFalse.tsx FillBlank.tsx
|
||||
Matching.tsx Ordering.tsx ShortAnswer.tsx shared.tsx index.tsx
|
||||
homework/ make-my-homework screens (input, result, guide feed, history)
|
||||
chat/ chat screen, composer, message rows, markdown + math, pickers
|
||||
UploadScreen.tsx StudyFeed.tsx ResultsScreen.tsx DeckCard.tsx ApiKeyGate.tsx
|
||||
InstallSheet.tsx TabBar.tsx ServiceWorkerRegistrar.tsx FolderBar.tsx
|
||||
LibraryScreen.tsx SettingsDialog.tsx AppHeader.tsx ThemeToggle.tsx icons.tsx ui.tsx
|
||||
|
|
@ -717,6 +773,11 @@ lib/
|
|||
clipboard.ts copy helper with a non-secure-context fallback
|
||||
image.ts client-side validation, resize and compression
|
||||
client-api.ts browser client for the API routes (NDJSON reader)
|
||||
chat.ts chat types + last-model memory
|
||||
chat-context.ts client-side chunking + BM25 retrieval over the scoped decks
|
||||
chat-client.ts streaming chat client + attachment extraction
|
||||
chat-seed.ts one-shot hand-off from results into the chat
|
||||
chat-trim.ts token budgeting + the sliding history window (server + client)
|
||||
xp.ts utils.ts deck-ops.ts confetti.ts
|
||||
public/
|
||||
sw.js service worker (production only)
|
||||
|
|
@ -740,7 +801,9 @@ scripts/
|
|||
| `POST /api/homework` | `{ mode: "make" \| "guide" \| "check", images?: string[], text?, solution?, language? }` | NDJSON stream: `{type:"stage"}`, `{type:"homework",homework,meta}`, `{type:"error"}` |
|
||||
| `POST /api/homework/refine` | `{ instruction: "shorter" \| "more" \| "explain", text, context?, language? }` | `{ text }` — one rewritten piece, plain text |
|
||||
| `POST /api/grade` | `{ question, referenceAnswer, keyPoints, userAnswer, language? }` | `{ correct, score, feedback, missing }` |
|
||||
| `GET /api/health` | – | `{ ok, reachable, mode, baseUrl, hasApiKey, vision, text, latencyMs, modelCount, error }` |
|
||||
| `POST /api/chat` | `{ model?, messages: [{role,content}], context?: [{deckId,deckTitle,section,text}] }` | NDJSON stream: `{type:"token"}`, `{type:"thinking"}`, `{type:"done",model}`, `{type:"error"}` |
|
||||
| `POST /api/chat/extract` | `{ images?: string[], text? }` | `{ text }` — attachments reduced to plain text via the vision step |
|
||||
| `GET /api/health` | – | `{ ok, reachable, mode, baseUrl, hasApiKey, vision, text, chatModels, latencyMs, modelCount, error }` |
|
||||
| `POST /api/sync` | `{ id, envelope }` (opaque AES-GCM blob) | `{ updatedAt }` — 413 over `SYNC_MAX_BYTES`, 507 over `SYNC_TOTAL_BYTES` |
|
||||
| `GET /api/sync?id=` | – | `{ updatedAt, envelope }` or 404 |
|
||||
| `DELETE /api/sync?id=` | – | `{ ok: true }` (revokes the synced copy) |
|
||||
|
|
@ -749,7 +812,7 @@ scripts/
|
|||
| `DELETE /api/push?endpoint=` | – | `{ ok, count }` |
|
||||
| `POST /api/push/test` | `{ endpoint }` | `{ ok: true }` or 502 when the push service refuses |
|
||||
|
||||
All four routes accept an optional **`x-ollama-key`** header (the browser's own key; required in
|
||||
The model routes accept an optional **`x-ollama-key`** header (the browser's own key; required in
|
||||
`user-key` auth mode). Missing keys answer **401** with `kind: "needs_key"`, a rejected key answers
|
||||
`kind: "auth"` (or `kind: "no_credits"` for HTTP 402), and exceeding a rate limit answers **429**
|
||||
with `Retry-After`. Keys are used for the single request and discarded.
|
||||
|
|
@ -1072,8 +1135,8 @@ Things I chose while building, with the reasoning:
|
|||
(native), i18n, the freeze loading screen, Smart Review, and phases 1–3 (backup/sync/usage
|
||||
limits, the learning engine, homework check/follow-ups). Suites that need a stub Ollama or
|
||||
VAPID keys start their own `next start` instance on a spare port, verify the port is free and
|
||||
kill the process group on exit; test scratch stays in `tests/.tmp/` (gitignored). Eleven suites,
|
||||
424 checks.
|
||||
kill the process group on exit; test scratch stays in `tests/.tmp/` (gitignored). Twelve suites,
|
||||
487 checks.
|
||||
|
||||
46. **The ice orbs are rising bubbles now, composited and cheaper to read through.** The glossy
|
||||
orbs (the glacier blobs stay exactly as they were) rise from below the bottom edge to above the
|
||||
|
|
@ -1096,6 +1159,30 @@ Things I chose while building, with the reasoning:
|
|||
tab-hidden parking) plus the 16-run axe contrast matrix (360/390/1280 × dark/light × EN/CS).
|
||||
`tests/bubble-perf.mjs` keeps the measurement repeatable.
|
||||
|
||||
47. **Chat grounds on the notes that were already there, and the server stays stateless.** The
|
||||
reconnaissance before this feature found the extracted notes text already kept per deck
|
||||
(`sourceText`, size-capped, used by Regenerate), so the "store it in IndexedDB" branch of the
|
||||
brief didn't apply — chat reads the deck record instead, with the generated study content as
|
||||
the documented fallback for old or imported decks. Retrieval is client-side and dependency-free:
|
||||
~800-token chunks, BM25 (k1 = 1.2, b = 0.75, stopwords, diacritics folded) scored against the
|
||||
question plus the last turns, top chunks under a ~7k-token budget, each labelled with deck and
|
||||
section. The request carries the chosen context and the conversation window (like the key
|
||||
does); the server trims the window to ~6k tokens, caps the context at ~9k, validates the model
|
||||
against `CHAT_MODELS` and streams `token` / `thinking` / `done` / `error` NDJSON. Reasoning is a
|
||||
separate event, so it can be hidden behind the Thinking row and never mixed into the answer.
|
||||
Attachments reuse the vision step (`/api/chat/extract`) so text-only chat models work. Markdown
|
||||
is a small React-node parser (no HTML injection, no dependency); KaTeX is dynamically imported
|
||||
only when a message contains math and writes into its container imperatively — there is still
|
||||
no `dangerouslySetInnerHTML` anywhere in the app. Stopping mid-stream aborts the upstream call;
|
||||
the abort arrives as a reader rejection and is mapped back to a clean "stopped" state instead of
|
||||
an error card. Usage-limit failures keep the draft and show the shared message with
|
||||
Settings/Ollama actions in an inline banner. The Settings toggle "Keep notes text for chat"
|
||||
makes chat ignore notes text, and a deck's menu can delete the notes text outright. Verified by
|
||||
the chat suite (63 checks against a stub Ollama: validation and the allowlist, context travel,
|
||||
the history window, thinking separation, usage-limit mapping, attachment extraction, entry
|
||||
points, streaming, stop, markdown + KaTeX, the model and scope pickers, edit/regenerate,
|
||||
offline, attachments and reduced motion).
|
||||
|
||||
36. **The mobile declutter pass reshapes, it doesn't fork.** Phones (≤639px) get a 3-step deck
|
||||
flow and a 2-step homework flow that share the exact state and handlers with the desktop
|
||||
cards (one `UploadScreen` / one `HomeworkInput`, two layouts switched by `useIsPhone()`);
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue