dvorakmortonxd/freezmo
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
freezmo/README.md
Martin Dvorak 26e36bc0c0 Add the Install the app bottom sheet for mobile browsers 
- New InstallSheet mounted in the root layout: appears ~1s after load on
  iOS/Android phones and tablets, over any screen including the key gate,
  with a dimmed backdrop. Never on desktop, never in standalone mode, and
  at most once per 7 days after a dismissal (localStorage).
- Platform/browser detection (lib/install.ts): iPhone/iPad incl. iPadOS
  reporting as Mac with touch, Android, Safari vs Chrome vs other vs
  in-app browsers, with step 1 adapted accordingly.
- iOS: numbered steps with chips (Safari, Share, ... , More, Add to Home
  Screen) and the accent Add pill. Android: captured beforeinstallprompt
  turns into a native Install button; fallback steps use Install app /
  Add to Home screen chips and an accent Install pill.
- Close via X, backdrop, Escape or swipe-down on the handle; the site is
  untouched afterwards. Anchored to 100dvh so browser toolbars never hide
  the last step; pb-safe respects the home indicator.
- Dark sheet by default; light only when the app theme was explicitly set
  to light. Settings gains a mobile-only Install app row that reopens it.
- The inline head script captures beforeinstallprompt before hydration and
  records appinstalled. Old InstallHint banner removed.
- Tests: new install-sheet suite (45 checks across iPhone Safari/Chrome/
  in-app, Android Chrome/Samsung, desktop, prompt path, dismissal, 7-day
  expiry, standalone, Settings row, both themes); pwa suite updated.
2026-10-07 08:27:43 +00:00

792 lines
48 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# ❄️ Freezmo
**Turn photos of your study notes into a swipeable feed of mini study games.**
Freezmo is a TikTok/Gizmo-style study app. You paste or upload photos of your notes, an AI reads
them and builds a deck of 8–15 interactive activities — flashcards, quizzes, matching games and
more — that you swipe through vertically like a feed. Every answer gives instant feedback, XP and
streaks, and the deck is saved in your browser so you can retry it later.
All AI runs through **your own Ollama setup** (a local daemon or the Ollama cloud API). No accounts,
no database, no data leaving your machine except the requests you send to Ollama.
---
## Table of contents
- [Features](#features)
- [Quick start](#quick-start)
- [Supported input formats](#supported-input-formats)
- [Ollama setup](#ollama-setup)
- [Rate limits](#rate-limits)
- [Mode 1 — local daemon (recommended)](#mode-1--local-daemon-proxying-cloud-models-recommended)
- [Mode 2 — direct Ollama cloud API](#mode-2--direct-ollama-cloud-api-api-key)
- [Changing models](#changing-models)
- [Checking the connection](#checking-the-connection)
- [Running on your network / from your phone](#running-on-your-network--from-your-phone)
- [Folders](#folders)
- [Install as an app (PWA)](#install-as-an-app-pwa)
- [Security notes](#security-notes)
- [How it works](#how-it-works)
- [Project structure](#project-structure)
- [API routes](#api-routes)
- [Scripts](#scripts)
- [Data, storage and privacy](#data-storage-and-privacy)
- [Troubleshooting](#troubleshooting)
- [Decisions and notes](#decisions-and-notes)
---
## Features
**Capture & generate**
- Drag & drop, file picker, **mobile camera capture** and **clipboard paste** (Ctrl/⌘+V)
- **Photos and documents**: JPG/PNG/WEBP/HEIC images plus **PDF, DOCX, PPTX, ODT/ODP/ODS, TXT, MD, CSV**
- Documents are read **in the browser**: PDF text layers are extracted, scanned PDF pages are rendered
as images for the vision model, DOCX/PPTX text, speaker notes and embedded images are pulled out.
The raw file never leaves your device.
- Up to 8 photos + 6 documents per deck (max 12 images reach the model); photos are resized to max
1600 px and re-encoded as JPEG before upload
- Optional extra text (pasted notes) and a topic hint
- Easy / Medium / Hard difficulty and 8 / 10 / 12 / 15 activities per deck
- Fun staged loading state (`Reading your notes… → Finding the key ideas… → Building your activities…`)
- Friendly error cards with a **Retry** button that point at Settings when a key is needed
**Language**
- The notes' language is detected and **preserved end to end**: transcription, deck title, questions,
options, explanations, short-answer grading and later *regenerate* / *add more* all stay in that
language. Czech notes produce a Czech deck; the app never translates them to English.
- **Language guard:** after the vision step, the transcription is checked (script + language markers).
If it clearly came back in another language, `TEXT_MODEL` (which is much stronger multilingually)
restores it before any activities are written — the UI shows *“Restoring the notes' language…”*.
On good transcriptions the guard stays silent and costs nothing.
- The deck stores its language, so short answers are graded and feedback written in the same language.
- The Freezmo interface itself stays English (only your content is language-matched).
**Study feed**
- Full-screen vertical scroll-snap feed, one activity per screen
- Arrow-key / PageUp / PageDown / Home / End navigation, plus swipe on touch devices
- Progress bar, activity counter, XP counter and answer streak in the chrome
- Instant feedback with a short explanation for every activity, confetti when you finish
**Activity types** (each one is its own React component, rendered from JSON by a `type` field)
| Type | Interaction |
| --- | --- |
| `flashcard` | Tap to flip, then self-assess *I knew it / Not yet* |
| `multiple_choice` | 4 options, keys 1–4, correct answer + explanation |
| `true_false` | Tap the buttons **or** swipe the card right/left |
| `fill_blank` | Inline input (one typo forgiven) or a word bank |
| `matching` | Tap a term, then its definition; perfect match for full XP |
| `ordering` | Drag with the mouse/finger, or use ▲▼ buttons, then check |
| `short_answer` | Typed answer graded by the AI via `/api/grade`, with a self-check fallback |
**First run**
- A full-screen **API-key gate** replaces the app until a key is saved (in `user-key` mode): one
masked input with a show/hide toggle, a single **Continue**, and a "?" sheet with the four steps
to a free key. The key is validated with a real 1-token probe before it is accepted — invalid
keys and network trouble get different inline messages. Removing the key in Settings brings the
gate back; `server` and `local` installs skip it entirely.
**Library & folders**
- Decks saved locally with title, subject, folder, activity count, date, play count and best score
- **Folders**: create named groups ("History", "Biology", …) from the sidebar or the create screen.
New decks freeze straight into the selected folder; anything unfiled lives in **Unorganized**.
Rename, delete (decks survive — they return to Unorganized) and move decks between folders
- Rename, delete (with confirm), regenerate, and **add more activities** to an existing deck
- Export a deck as JSON and import it back — the folder name travels with the file and is recreated
on import
**Design**
- **ChatGPT-style shell**: a persistent left **sidebar** (brand, a quiet "New deck" row, a *Folders*
list with per-folder counts, Settings/Dark mode pinned to the bottom). It **collapses** to a 68px
icon rail (remembered in `localStorage`) and becomes a slide-in **drawer** behind a hamburger on
phones; mobile also keeps the frosted header + iOS tab bar
- **One neutral palette, both themes**: light page `#ffffff` / sidebar `#f9f9f9` / border `#e5e5e5`;
dark page `#212121` / sidebar `#171717` / surfaces `#2f2f2f` / border `rgba(255,255,255,.1)`.
Text tiers are documented AA ratios (see below), and the cards fill the row with
`repeat(auto-fill, minmax(280px, 1fr))` — no stranded cards, no dead space
- Deck cards: title → muted metadata → neutral tags (difficulty carries the only colour:
green/amber/red) → summary → a single **Study** button, with everything else in the `⋯` menu
- **Frosted UI**: the sticky header, tab bar, sheets and panels use real `backdrop-filter` blur over
a faint viewport-fixed aurora wash (sky/lilac/mint) — depth without noise
- **Pill language**: primary CTAs are near-black in light mode and white in dark mode; secondaries
are outlined or ghost. Colour is reserved for meaning — activity tints (sky/lilac/mint/peach/rose),
correct/wrong feedback, key status
- Micro-interactions: 160–200ms colour/transform transitions, `active:scale-[0.985]`, skeleton
shimmer, hairline hover washes on every row and card
- **iOS-flavoured PWA**: standalone mode, safe-area insets, bottom tab bar, sheet-style dialogs with
a grabber, inset grouped lists, segmented controls, "Add to Home Screen" hint — see
[Install as an app](#install-as-an-app-pwa)
- Mobile-first, dark mode, keyboard accessible, focus rings, ARIA labels, `prefers-reduced-motion` support
---
## Quick start
Requirements: Node.js 20+ (tested on 24) and npm.
```bash
cd freezmo
./setup.sh # install + .env.local + production build
npm start # http://0.0.0.0:3207
```
Or do it by hand (and use the dev server):
```bash
cd freezmo
npm install
cp .env.example .env.local # first time only
npm run dev # http://0.0.0.0:3207
```
`setup.sh` flags: `--dev` installs and prepares `.env.local` without building, `--no-build` skips the
build too.
Open <http://localhost:3207>. The app runs without any Ollama configuration — you'll just need a
key (or a signed-in daemon) before it can generate a deck. Complete one of the Ollama setups below,
then paste the key on the gate (or later in **Settings**).
`npm run dev` and `npm run build` also copy the pdf.js worker into `public/` automatically (see
`scripts/copy-pdf-worker.mjs`), which is what lets the browser parse PDFs.
Production:
```bash
npm run build
npm start # http://0.0.0.0:3207
```
Production is also where the PWA comes alive: the service worker registers, the app can be installed
to the home screen and the `/offline` fallback works. In `npm run dev` none of that runs, so you
won't fight a stale cache while developing.
---
## Ollama setup
Freezmo talks to **`POST /api/chat`** and **`GET /api/tags`** of whatever URL is in
`OLLAMA_BASE_URL`. All calls happen in Next.js route handlers on the server — the browser never
talks to Ollama directly.
Who provides the API key is controlled by **`OLLAMA_AUTH`**:
| `OLLAMA_AUTH` | Key comes from | Use it when |
| --- | --- | --- |
| **`user-key`** (default) | each browser, pasted on the key gate (or in Settings) | several people/devices share the app, or you don't want a key on the server |
| `server` | `OLLAMA_API_KEY` in `.env.local` | a single-user install |
| `local` | nothing — a daemon signed in with `ollama signin` | you run Ollama locally |
### Mode 1 (default) — bring your own key
```bash
OLLAMA_AUTH=user-key
OLLAMA_BASE_URL=https://ollama.com
VISION_MODEL=gemma4:31b-cloud
TEXT_MODEL=gpt-oss:120b-cloud
```
1. Open the app — the **key gate** asks for a key on first visit.
2. Create one at <https://ollama.com/settings/keys>, paste it in and press **Continue**.
3. The gate validates it with a real 1-token request, then drops you into the icebox.
**Settings → Test connection** re-checks any time and shows the model list.
Where the key lives:
- **In the browser only.** It is stored in this browser's `localStorage` under `freezmo.settings.v1`
and sent with each API call in the `x-ollama-key` header.
- **Never on the server:** the route handlers read the header, use it for that one request and drop
it. It is not written to `.env.local`, a database, a cookie, a log line or a JS bundle. Restarting
the server does not remember any key, and every browser/device needs its own (or the same) key.
- You can remove it any time with **Remove key from this browser**.
- Because the browser hands the key to your own server, use HTTPS if the LAN is not trusted — on a
plain-HTTP LAN somebody sniffing traffic could read it. The key never goes to any third party.
Keys are verified with a real (tiny, 1-token, 5-minute-cached) chat request, because model listing
on `ollama.com` is public and would happily accept a bogus key.
### Mode 2 — local daemon (no key at all)
Runs cloud models through your local Ollama daemon, signed in once with `ollama signin`.
```bash
ollama # start the app/daemon
ollama signin # approve the printed link in any browser
ollama pull gemma4:31b-cloud # vision model, reads the photos of your notes
ollama pull gpt-oss:120b-cloud # text model, builds the activities and grades
```
`.env.local`:
```bash
OLLAMA_AUTH=local
OLLAMA_BASE_URL=http://localhost:11434
VISION_MODEL=gemma4:31b-cloud
TEXT_MODEL=gpt-oss:120b-cloud
```
Restart `npm run dev` after editing `.env.local`. Verify with the **⚙️ settings dialog** in the
app or `curl localhost:3207/api/health`.
**Free tier.** `gemma4:31b-cloud` and `gpt-oss:120b-cloud` are included in the Ollama **Free** plan
(verified October 2026). Other free models: `gpt-oss:20b-cloud`, `nemotron-3-super:cloud`,
`nemotron-3-nano:30b-cloud`. Models such as `glm-5.3-flash:cloud`, `deepseek-v4.1-flash`,
`minimax-m2.7` and `mistral-large-3` reply with *"this model is not included in your free
usage…"* — Freezmo detects that (HTTP 402) and says so instead of pretending the key is broken.
### Mode 3 — one key for the whole server
```bash
OLLAMA_AUTH=server
OLLAMA_BASE_URL=https://ollama.com
OLLAMA_API_KEY=your_key_here
```
The key stays in `.env.local` (gitignored) and is never sent to the browser. Requests from any
device use it.
#### Installing without root (user-space)
The official installer needs root. If there is no `sudo` on the machine (like the box Freezmo was
built on), Ollama can be installed entirely under your home directory:
```bash
cd ~ && mkdir -p .local/ollama && cd .local/ollama
# grab the tarball for your arch from https://github.com/ollama/ollama/releases/latest
# (ollama-linux-amd64.tar.zst) and extract it here, so you end up with
# ~/.local/ollama/bin/ollama and ~/.local/ollama/lib/ollama/
# No zstd CLI on Debian? Stream it with Python's zstandard module instead of tar --zstd.
export PATH="$HOME/.local/ollama/bin:$PATH" # add to ~/.profile too
ollama serve & # daemon on 127.0.0.1:11434
ollama signin
ollama pull gemma4:31b-cloud && ollama pull gpt-oss:120b-cloud
```
On the development machine the daemon runs as a **systemd user service**
(`~/.config/systemd/user/ollama.service`):
```bash
systemctl --user status ollama # is it up?
systemctl --user restart ollama # restart it
journalctl --user -u ollama -f # logs
```
If the user has lingering enabled (`loginctl show-user $USER | grep Linger`) the service also
starts at boot with nobody logged in; otherwise it starts when you log in. Use the system-wide
installer if you want a boot service without a login session.
### Changing models
Any model that your Ollama instance exposes works. Edit `.env.local`:
```bash
VISION_MODEL=<model used to transcribe the photos>
TEXT_MODEL=<model used to build activities and grade short answers>
```
Current cloud models that work well (verified against the Ollama cloud in October 2026):
| Purpose | Model | Free tier? | Notes |
| --- | --- | --- | --- |
| Vision (default) | `gemma4:31b-cloud` | ✅ | Native multimodal, 256K context, current generation |
| Text (default) | `gpt-oss:120b-cloud` | ✅ | OpenAI open-weights, excellent at strict JSON, quick |
| Text, light | `gpt-oss:20b-cloud` | ✅ | Smaller/faster, slightly weaker |
| Text, cheap | `nemotron-3-super:cloud` | ✅ | 120B MoE with 12B active, very inexpensive |
| Text, smallest | `nemotron-3-nano:30b-cloud` | ✅ | Lowest cost option |
| Text, faster | `glm-5.3-flash:cloud` | 💳 credits | 18B active, 1M context, very fast |
| Text, best quality | `kimi-k3:cloud` | 💳 credits | Strongest open-weight model, slowest/most expensive |
| Vision, larger | `mistral-large-3:675b`, `deepseek-v4.1-flash` | 💳 credits | Multimodal alternatives |
> The previously suggested `qwen3-vl:235b-cloud` has been retired. The current defaults —
> `gemma4:31b-cloud` (vision) and `gpt-oss:120b-cloud` (text) — are both free-tier friendly and
> were verified end-to-end on a real photo of notes. The full list is one click away in the
> settings dialog or at <https://ollama.com/search?c=cloud>.
If your vision model is also good at generation you can point both variables at the same model —
the two pipeline steps stay separate either way.
### Checking the connection
- **In the app:** click the ⚙️ button — it shows mode, base URL, API key status, latency and
whether each configured model is available, with a hint for what to fix.
- **From the shell:** `curl http://localhost:3207/api/health`
- **Directly against Ollama:** `curl http://localhost:11434/api/tags`
---
## Rate limits
Generation is expensive (two model calls per deck), so the API routes are limited per client IP with
a sliding window. Everything is configurable in `.env.local`:
| Route | Default | Env var |
| --- | --- | --- |
| `POST /api/generate` | 12 per 5 minutes | `RATE_LIMIT_GENERATE` |
| `POST /api/grade` | 40 per 5 minutes | `RATE_LIMIT_GRADE` |
| `GET /api/health` | 90 per 5 minutes | `RATE_LIMIT_HEALTH` |
| window for all of them | 5 minutes | `RATE_LIMIT_WINDOW_MS` |
When a limit is hit the route answers **429** with a `Retry-After` header and a friendly message
(“Too many requests — try again in 42s”); the UI shows that message and keeps the Retry button. The
client IP is taken from `x-forwarded-for` / `x-real-ip` when present (reverse proxy), otherwise all
local clients share one bucket.
Caveats, stated honestly: counters live **in memory**, so they reset on restart and are not shared
between multiple instances or Node workers. That is fine for a self-hosted single-instance app; if
you put Freezmo behind a reverse proxy, keep the proxy's own limits in place too.
Model names are normalised automatically: when the base URL points at `ollama.com`, Freezmo strips
the local routing suffix (`gemma4:31b-cloud` → `gemma4:31b`, `gpt-oss:120b-cloud` → `gpt-oss:120b`)
because the cloud API expects plain library names, while a local daemon wants the `-cloud` form.
The same `.env` therefore works in every mode.
## Running on your network / from your phone
The dev and start scripts already bind to `0.0.0.0` on port **3207**:
```bash
npm run dev # next dev -H 0.0.0.0 -p 3207
npm start # next start -H 0.0.0.0 -p 3207
```
1. Find your machine's LAN IP: `hostname -I` (Linux) or `ipconfig getifaddr en0` (macOS).
2. On your phone open `http://<your-lan-ip>:3207`.
3. Allow port 3207 through the firewall if needed
(e.g. `sudo ufw allow 3207/tcp`, or the equivalent Windows rule).
4. Ollama only needs to be reachable from the Next.js server, not from the phone.
Notes for phones:
- **The camera button works over plain HTTP** because it uses `<input type="file" capture="environment">`,
not `getUserMedia` (which would require HTTPS). Tapping **Camera** opens the phone's camera app
and the photo comes back as a file.
- Clipboard paste works best on desktop; on mobile use the camera or photo picker.
- Add the page to your home screen for a full-screen, app-like experience.
---
## Folders
Decks are organised into folders — the same way you'd file paper notes.
- **Unorganized** is where every deck starts. It is virtual: it simply means "no folder yet".
- Create a folder from the library's folder bar (**New folder**) or from the create screen's
**Freeze into folder** row. Names are unique, case-insensitive, and can be renamed later.
- **Freezing a deck into a folder:** pick the folder before generating (create screen) or with the
folder row on the create screen; or use a deck's **⋯ menu → Folder** chips afterwards. The library
shows which folder each deck is in, and groups the list by folder.
- **Filtering:** the folder bar filters the library to one folder; *All decks* shows every folder as
its own section, Unorganized last, with per-folder counts on each chip.
- **Deleting a folder never deletes decks** — they move back to Unorganized. The confirmation says
exactly how many decks will be re-homed.
- Folders live beside the decks in `localStorage` (`freezmo.folders.v1`), so they are per-browser
and need no account. Exporting a deck records its folder name, and importing recreates or reuses
that folder.
## Install as an app (PWA)
Freezmo ships as an installable Progressive Web App with iOS in mind.
Phones and tablets get an **"Install the app" bottom sheet** about a second after load (never on
desktop, never when already installed, and at most once per 7 days after a dismissal). It is a
native-feeling sheet with the Freezmo card, numbered steps and chips for the exact buttons to tap,
plus an accent confirmation pill. Close it with the X, the backdrop, Escape, or a swipe down — the
site keeps working normally. On iOS the steps adapt to the browser (Safari, Chrome, in-app), and
on Android Chrome a captured `beforeinstallprompt` turns into a real **Install** button. The sheet
can be reopened any time from **Settings → Install app** (mobile only).
It launches full-screen in standalone mode (no browser chrome), respects the notch/home-indicator
safe areas, and has an iOS-style bottom tab bar (Library · New deck · Settings) with sheet-style
dialogs. Android Chrome also offers "Install app" from the ⋮ menu; Freezmo works offline for the
parts that don't need a model.
What's included:
| Piece | Detail |
| --- | --- |
| `app/manifest.ts` | `display: standalone`, portrait, theme/background colours, app shortcuts |
| Icons | 192/512 px, a maskable 512 px icon, and 180/167/152 px `apple-touch-icon`s |
| iOS meta | `apple-mobile-web-app-capable`, `-title`, `-status-bar-style`, `viewport-fit=cover` |
| Service worker | `public/sw.js`, **production only**: cached app shell, stale-while-revalidate for static assets, network-first navigations with an `/offline` fallback. `/api/*` is never cached |
| Offline page | `/offline` — "your decks are still here, reconnect to generate" |
| Safe areas | `pb-safe` / `pt-safe` utilities plus `pb-tabbar` so nothing hides behind the tab bar |
The icons are generated by `scripts/make-icons.mjs` (headless Chromium); it is a one-off tool and
is not part of the app's dependencies.
## Security notes
An honest review of what Freezmo does and does not defend against. It is a single-user,
self-hosted study app with no accounts, no database and no server-side state.
**What is hardened**
- **No secrets on the client.** The API key lives only in the browser's `localStorage` and travels
in the `x-ollama-key` header per request. Route handlers use it for that one call and discard it —
no disk, no log, no cookie, no environment write. Verified by grepping the whole project and the
production `.next` output for a test key, and by restarting the server (which remembers nothing).
- **No server-side injection surface.** The Ollama endpoint and model names come only from the
server's `.env`; the client cannot point the server at another host. Model output is validated with
zod and rendered as text — there is no `dangerouslySetInnerHTML` except two static inline scripts
(the pre-paint theme choice and the `beforeinstallprompt` capture), so a deck cannot inject markup
or script (React escapes everything).
- **Key-header sanitisation.** Keys from the client are stripped of control characters/whitespace
and capped at 512 chars before they are used in an outbound header, so a crafted key cannot inject
headers. Verified with a raw-socket CRLF attempt.
- **Cross-origin requests are refused** on `/api/generate` and `/api/grade` (`Origin` must match the
host), so a random website in your browser cannot make your Freezmo spend your rate limit or, in
`server` auth mode, your credits. Origin-less clients (curl) still work.
- **Body-size limits** (32 MB for generation, 64 KB for grading) are rejected with 413 before any
parsing, on top of the zod limits on image count, text length and field sizes.
- **Rate limiting** per client IP; see [Rate limits](#rate-limits). Auth is checked *before* parsing
request bodies, so unauthenticated traffic costs almost nothing.
- **No CSRF surface**: no cookies, no sessions, no server-side mutations.
- **Prompt injection is low impact by design.** Your notes could contain text that tries to
manipulate the model, but the model can only return a JSON deck (validated per activity, dropped if
invalid) — it has no tools, no file access and no network access beyond the model call.
**Known limitations (by design)**
- The key travels from your browser to your own Freezmo server, so on an untrusted network it needs
HTTPS to stay private. On a plain-HTTP LAN someone sniffing traffic could read it, along with your
notes.
- Anything stored in `localStorage` is readable by JavaScript on the same origin. Freezmo has no XSS
vector today, but that is the standard risk of browser-stored credentials.
- Rate limiting trusts `x-forwarded-for` when present (correct behind a proxy, spoofable when
exposed directly) and keeps counters in memory: single instance only, reset on restart.
- Ollama cloud usage is billed to whoever's key is used, so treat a shared Freezmo URL as shared key
access in `server` mode.
- No accounts means no per-user isolation: anyone who can reach the URL can use the app (and, in
`user-key` mode, only with their own key).
## Supported input formats
Everything is parsed **client-side**, so documents are never uploaded — only the extracted text and
page images are sent to `/api/generate`.
| Format | What Freezmo reads | Sent to the model as |
| --- | --- | --- |
| JPG / PNG / WEBP / HEIC / GIF | the photo itself, resized to 1600 px JPEG | images (vision model) |
| **PDF, text layer** | text of every page (up to 80 pages) | text |
| **PDF, scanned / image-only** | pages with little or no text are rendered to images | images (vision model) |
| **DOCX** | document text + embedded images | text + images |
| **PPTX** | slide text + speaker notes + slide images | text + images |
| **ODT / ODP / ODS** | text from the OpenDocument `content.xml` | text |
| TXT / MD / CSV / TSV / LOG | the file's text | text |
Limits and behaviour worth knowing:
- up to 6 documents and 8 photos per deck, 40 MB per document, 20 MB per photo
- at most 12 images are sent (photos, rendered scanned pages and images embedded in documents)
- each document is capped at 60 000 characters and the merged text at 100 000; longer documents show
a “truncated” note in the UI
- the activity generator reads up to 48 000 characters of notes per deck — plenty for lecture notes,
but a whole book will only be sampled
- password-protected or corrupted files report a clear error on their card and can be removed
## How it works
```
photos + text /api/generate
─────────────┐
▼
┌──────────────────────────┐ step 1 (if images) ┌───────────────────────────┐
│ browser: resize/compress │ ─────────────────────► │ VISION_MODEL │
│ max 1600px JPEG data URL │ │ transcribe + extract │
└──────────────────────────┘ │ notes/concepts/formulas │
│ └────────────┬──────────────┘
│ NDJSON progress stream │ clean structured text
▼ ▼
┌──────────────────────────┐ ┌───────────────────────────┐
│ browser: study feed │ ◄───────────────────── │ TEXT_MODEL │
│ JSON validated by zod │ deck JSON │ design 8–15 activities │
└──────────────────────────┘ └───────────────────────────┘
```
- The prompts live in [`lib/prompts.ts`](lib/prompts.ts). They instruct the model to use **only the
student's notes**, never invent facts, vary the activity types, write a short explanation for each
activity and answer with JSON only.
- Every response is parsed (markdown ``` fences stripped), validated with **zod**
([`lib/schemas.ts`](lib/schemas.ts)) and — if validation fails — automatically retried **once**
with the validation error fed back to the model (`lib/ai.ts`).
- Individual broken activities are dropped rather than failing the whole deck; if fewer than half
the requested activities survive, the step is retried too.
- Requests use `stream: false` with per-request timeouts and a hard overall deadline
(two sequential model calls). Errors surface as friendly messages
("Ollama isn't reachable, check your settings") with a retry button.
- `POST /api/generate` answers with newline-delimited JSON (`stage` / `deck` / `error` events) so the
UI can show real progress between the vision and generation steps.
---
## Project structure
```
app/
layout.tsx root layout, iOS meta tags + theme bootstrap script
manifest.ts web app manifest (installable PWA)
offline/page.tsx offline fallback served by the service worker
globals.css Tailwind v4 theme, frosty palette, animations, reduced-motion
page.tsx library ("Your icebox")
create/page.tsx upload screen
deck/[id]/page.tsx study feed for a saved deck
api/generate/route.ts images/text in → NDJSON progress → deck JSON out
api/grade/route.ts short-answer grading
api/health/route.ts Ollama reachability + model availability
components/
activities/ one component per activity type + shared shell
Flashcard.tsx MultipleChoice.tsx TrueFalse.tsx FillBlank.tsx
Matching.tsx Ordering.tsx ShortAnswer.tsx shared.tsx index.tsx
UploadScreen.tsx StudyFeed.tsx ResultsScreen.tsx DeckCard.tsx ApiKeyGate.tsx
InstallSheet.tsx TabBar.tsx ServiceWorkerRegistrar.tsx FolderBar.tsx
LibraryScreen.tsx SettingsDialog.tsx AppHeader.tsx ThemeToggle.tsx icons.tsx ui.tsx
lib/
ollama.ts server-only HTTP client (timeouts, auth, error mapping)
prompts.ts the vision, generation, grading and repair prompts
ai.ts the three pipeline steps with zod validation + retry
schemas.ts zod schemas and shared types
config.ts env config, cloud/direct model-name normalisation
documents.ts client-side PDF/DOCX/PPTX/ODT/text extraction
folders.ts folder library (create/rename/delete) in localStorage
install.ts install-prompt platform/browser detection + 7-day dismissal
settings.ts browser-side settings (the user's API key) + auth headers
use-health.ts client hook for /api/health, re-runs when the key changes
rate-limit.ts in-memory sliding-window limiter for the API routes
http-guard.ts same-origin check, body-size cap, API-key sanitiser
language.ts script/language checks that catch translated transcriptions
ui-events.ts tiny event bus (any screen can open the settings dialog)
storage.ts localStorage deck library, import/export
image.ts client-side validation, resize and compression
client-api.ts browser client for the API routes (NDJSON reader)
xp.ts utils.ts deck-ops.ts confetti.ts
public/
sw.js service worker (production only)
icons/ PWA + iOS icons and splash screens
scripts/
copy-pdf-worker.mjs copies the pdf.js worker to /public (runs on predev/prebuild)
make-icons.mjs regenerates the PWA/iOS icons (one-off tool)
```
---
## API routes
| Route | Body | Returns |
| --- | --- | --- |
| `POST /api/generate` | `{ images?: string[], text?, difficulty, count?, focus?, language?, mode?, existing? }` | NDJSON stream: `{type:"stage"}`, `{type:"deck",deck,meta}`, `{type:"error"}` |
| `POST /api/grade` | `{ question, referenceAnswer, keyPoints, userAnswer, language? }` | `{ correct, score, feedback, missing }` |
| `GET /api/health` | – | `{ ok, reachable, mode, baseUrl, hasApiKey, vision, text, latencyMs, modelCount, error }` |
All three routes accept an optional **`x-ollama-key`** header (the browser's own key; required in
`user-key` auth mode). Missing keys answer **401** with `kind: "needs_key"`, a rejected key answers
`kind: "auth"` (or `kind: "no_credits"` for HTTP 402), and exceeding a rate limit answers **429**
with `Retry-After`. Keys are used for the single request and discarded.
Images are sent as base64 data URLs; the server strips the prefix before handing them to Ollama.
Uploads are validated for type (photos and the document formats above), count (max 8 photos,
6 documents, 12 images) and size (max ~8 MB per image after compression, 40 MB per document) and the
client compresses aggressively, so a typical upload is 1–3 MB. `language` is optional on both routes:
the vision step fills it in automatically, and the UI passes the stored deck language for grading,
regeneration and “add more”.
---
## Scripts
| Script | What it does |
| --- | --- |
| `npm run dev` | `next dev -H 0.0.0.0 -p 3207` |
| `npm run build` | production build (type-check + lint) |
| `npm start` | `next start -H 0.0.0.0 -p 3207` |
| `npm run lint` | ESLint |
| `node scripts/make-icons.mjs` | regenerate the PWA/iOS icons (needs a Playwright Chromium) |
| `node scripts/copy-pdf-worker.mjs` | copy the pdf.js worker (also runs on predev/prebuild) |
---
## Data, storage and privacy
- Decks live in **`localStorage`** under `freezmo.decks.v1`, folders under `freezmo.folders.v1` (no
accounts, no server database).
Stored per deck: title, subject, summary, language, difficulty, the extracted **source text**,
activities, play count and best score.
- **Original files are never stored or uploaded** — photos and documents are parsed in the browser,
and only the extracted text (plus page images for the vision model) is sent to your Ollama
endpoint. That is why *regenerate* and *add more activities* work from the source text rather than
the original files. Imported decks that contain no source text can't be regenerated; the UI says so.
- Documents are parsed locally by `pdfjs-dist` (PDF), `mammoth` (DOCX), `jszip` (PPTX/OpenDocument)
and the browser's own `File.text()` — nothing is uploaded for parsing.
- Images and rendered PDF pages are sent to whatever Ollama endpoint you configure. In local-daemon
mode they go to your machine first; in direct-cloud mode they are processed by Ollama's cloud.
- In the default `user-key` mode your Ollama API key is also kept in `localStorage` (key
`freezmo.settings.v1`) and sent with each API call so the server can forward it to Ollama. It is
never written to the server, a log, a cookie or the environment — see
[Ollama setup](#ollama-setup) for the full story.
- Clearing browser data deletes the library and the saved key. Use **Export JSON** for anything you
want to keep.
---
## Troubleshooting
| Symptom | Fix |
| --- | --- |
| “Ollama isn't reachable at http://localhost:11434” | Start the daemon (`ollama serve`, or `systemctl --user start ollama` for a user-space install), or fix `OLLAMA_BASE_URL`. Check `curl http://localhost:11434/api/tags`. |
| A document card is stuck on “Reading page …” | The pdf.js worker may be missing: run `npm run dev`/`npm run build` (they copy it to `public/pdf.worker.min.mjs`) or `node scripts/copy-pdf-worker.mjs` manually. |
| PDF uploaded but no activities about its content | The PDF is probably image-only (a scan). Freezmo renders such pages and sends them to the vision model — check the card note (“N scanned pages were turned into images”); if it says “No readable text”, the scan may be too low quality. |
| Deck came back in English even though the notes are not | Make sure both model calls succeed (a failed vision step falls back to raw text). The prompts force the notes' language; if you still see translation, switch `TEXT_MODEL` to a stronger instruction-follower such as `gpt-oss:120b-cloud`. |
| “Add your Ollama API key first” | Bring-your-own-key mode is active. Paste a key from <https://ollama.com/settings/keys> on the key gate (or in the ⚙️ dialog) — it is stored only in your browser. |
| “Ollama rejected this API key” | The key is invalid or revoked. Create a new one and paste it again; the check uses a real 1-token request, so a green tick means it truly works. |
| “this model is not included in your free usage” | The key works, but the model is outside your plan. Use one of the ✅ models in the table above, or add credits at <https://ollama.com/settings>. |
| “Too many requests — try again in Ns” | You hit a rate limit (12 decks / 40 grades per 5 min by default). Wait, or raise `RATE_LIMIT_*` in `.env.local`. |
| “set to OLLAMA_AUTH=local but points at https://ollama.com” | Fix the combination: use `OLLAMA_AUTH=user-key`/`server` for the cloud URL, or point `OLLAMA_BASE_URL` at `http://localhost:11434`. |
| Settings dialog says a model is “not found” | Local daemon: pull it once — `ollama pull gemma4:31b-cloud` (cloud pulls are just manifests). Direct cloud: check the model name and that your key is valid. |
| 401/403 from `https://ollama.com` | Create a key at <https://ollama.com/settings/keys> and put it in `OLLAMA_API_KEY` (no quotes), then restart. |
| Daemon logs / it keeps dying | User-space install: `journalctl --user -u ollama -f`. Manual start: `~/.local/ollama/bin/ollama serve`. |
| Generation times out | Use fewer/smaller photos, choose a faster `TEXT_MODEL` (e.g. `gpt-oss:20b-cloud`), or raise `OLLAMA_TIMEOUT_MS`. |
| Responses are slow | Thinking models spend tokens on reasoning. Set `OLLAMA_THINK=false` to send `think: false`, and/or use a smaller model. |
| “did not match the expected format” | Freezmo retried once automatically. If it still fails, the model is poor at JSON — switch to a stronger `TEXT_MODEL` (e.g. `gpt-oss:120b-cloud`). |
| Blank/boxed icons on Linux | The app chrome uses inline SVG, but activity badges use emoji. Install an emoji font (`fonts-noto-color-emoji`). |
| Port 3207 already in use | `lsof -i :3207` and stop the other process, or change the port in both scripts in `package.json`. |
| Camera button missing on mobile | It's in the *Add your notes* card; some in-app browsers hide file inputs — open the page in Chrome/Safari directly. |
| Hydration warning in dev | Usually a stale dev artifact: hard-reload. Decks load in `useEffect`/`useSyncExternalStore`, so no library content is server-rendered. |
---
## Decisions and notes
Things I chose while building, with the reasoning:
1. **Defaults follow the current Ollama catalogue and the free tier.** The suggested
`qwen3-vl:235b-cloud` has been retired, so the defaults are `gemma4:31b-cloud` (vision) and
`gpt-oss:120b-cloud` (text/JSON/grading) — both verified as included in the free plan on a real
photo of notes. Paid/credit models such as `glm-5.3-flash:cloud` and `kimi-k3:cloud` are
documented as alternatives.
2. **One `.env` for both modes.** Direct-cloud names differ from daemon names (`gemma4:31b` vs
`gemma4:31b-cloud`), so `lib/config.ts` strips the `-cloud` / `:cloud` suffix automatically when
`OLLAMA_BASE_URL` points at `ollama.com`.
3. **Two separable AI steps.** Photos always go through `VISION_MODEL` for transcription/extraction;
the structured text then goes to `TEXT_MODEL`. Pointing both variables at one model works too.
Text-only requests skip the vision step entirely.
4. **NDJSON progress stream instead of a single long request.** The two model calls are sequential
and slow, so `/api/generate` streams `stage` events; the UI shows real stages rather than a fake
timer. `stream: false` is still used for each Ollama call for simplicity and reliability.
5. **zod + one repair retry, plus per-activity salvage.** A single bad activity shouldn't cost you
the whole deck: valid activities are kept, and the repair round is only used when too few survive.
6. **Flashcards count as "correct" via self-assessment** (I knew it / Not yet) so that scoring, XP
and "retry missed ones" apply to every activity type.
7. **Fill-in-the-blank is graded locally** (case/punctuation-insensitive, one typo forgiven,
`/` alternatives in the answer) — no round trip needed. Only short answers call `/api/grade`.
8. **Matching requires a perfect match for full XP**, otherwise it's marked as missed with an
explanation of how many wrong attempts you made.
9. **Source text is stored with each deck** so *regenerate* and *add more* work without keeping the
original photos (storage/privacy win). Imported decks have no source text, so those two actions
are disabled for them.
10. **XP is assigned server-side** (easy 10 / medium 15 / hard 20, +5 for matching, ordering and
short answer) so the model can't invent inconsistent rewards.
11. **Documents are parsed in the browser, not on the server.** pdf.js, mammoth and jszip run
client-side, so the raw file never leaves the device and the API stays a thin text/image pipe.
Scanned PDF pages are rendered to JPEG for the vision model, while PDFs with a text layer skip
the vision step entirely (faster and exact).
12. **The notes' language is detected once and then carried everywhere.** The vision step reports a
`language` field, the deck stores it, and generation, grading, regenerate and add-more all
receive it explicitly — plus “never translate” rules in every prompt. Verified with Czech
notes across all three input paths (text PDF, scanned PDF, DOCX + PPTX).
13. **Keys are per browser and never stored server-side.** Freezmo defaults to
`OLLAMA_AUTH=user-key`: the key is typed into the UI, kept in `localStorage` and sent in the
`x-ollama-key` header, and the route handlers use it for that request only — nothing is written
to disk, logs, cookies or the environment (verified by grepping the whole project for a test key
and by restarting the server). `server` and `local` modes remain available by env var.
14. **Key verification uses a 1-token chat probe, cached for 5 minutes.** Model listing on
`ollama.com` is public, so a bogus key would look valid; only a real authenticated request can
tell. HTTP 402 (“not included in your free usage”) is reported as a *plan* problem rather than a
broken key.
15. **Rate limits are per client IP, in memory, and generous by default** (12 decks / 5 min, 40
grades / 5 min). They exist to stop runaway loops and accidental abuse, not to be a security
boundary — the README says so, and the counters are intentionally not persisted.
16. **A misconfiguration is caught early:** `OLLAMA_AUTH=local` combined with `https://ollama.com`
returns an explanatory error instead of a confusing 401.
17. **Folders are a first-class, non-destructive idea.** `UNORGANIZED_ID` is virtual (decks simply
have no `folderId`), deleting a folder re-homes its decks instead of deleting them, and folder
names travel in exports so an imported deck lands where it came from. 13 browser checks cover
create → freeze → move → rename → delete → reload → import.
18. **The visual language is glassmorphism, with restraint.** One accent colour drives the buttons
and highlights; the backdrop is a soft aurora wash; surfaces are frosted and translucent. Colour
appears in activity tints and semantic feedback only — pretty, but still readable and quiet.
19. **Custom CSS lives in `@layer components`, and glass uses Tailwind's backdrop utilities.**
Two build-level traps, both found by looking at real screenshots: unlayered rules beat Tailwind
utilities (which silently broke `fixed` on the feed), and a hand-written `backdrop-filter` gets
minified down to `-webkit-` only — which painted no blur at all. `@apply backdrop-blur-*`
emits both properties correctly.
20. **There is no demo/sample deck.** Every activity comes from the user's own notes; the empty
state points at the create flow and Settings instead of shipping fake content. Browser tests seed
a fixture deck into `localStorage` rather than relying on app code.
21. **Dark mode is token-driven, and contrast is audited.** Text colours come from
`--text` / `--text-muted` / `--text-subtle` (never hardcoded), and a `contrast.mjs` audit checks
every pairing at WCAG AA — currently **20/20 pairs pass** (e.g. dark secondary `#b4b4b4` on
`#212121` = 7.8:1). Difficulty colours were darkened until they cleared 4.5:1 on their pills.
This was a real bug: an earlier edit silently failed, so `.dark` never overrode `--text`, leaving
near-black text on a dark page.
22. **The dashboard follows a neutral, ChatGPT-style system.** Near-white/near-black surfaces,
hairline borders, pill CTAs, 18px radii and restrained shadows; the frost survives in the chrome
(header, tab bar, sheets) rather than in every card, which keeps text crisp and the layout
premium. Deck cards got real hierarchy: title → muted metadata → tags → summary → prominent
Study CTA, with secondary actions behind a quiet menu.
23. **One fixed backdrop layer instead of per-page blobs.** The decorative aurora used to be
rendered inside the content column, so on wide screens it was clipped into a visible hard edge.
It now lives in a single `position: fixed` layer (`.app-backdrop`, `z-index: -10`) with very low
opacities, and `body`'s colour matches its bottom so mobile overscroll shows no seam.
24. **The UI was deliberately made quieter.** One accent colour (`accent-*`), a desaturated
`frost-*` scale, hairline borders and flat surfaces replaced the saturated gradients and the
seven-colour activity badges. Semantic colours (green/amber/red) survive only where they carry
meaning: correct/wrong feedback, key status, rate limits.
25. **PWA instead of a native wrapper.** A manifest, iOS meta tags, generated icons, a
production-only service worker, an offline page and a standalone-aware layout give the
install-to-home-screen experience without adding a runtime dependency — and the app still works
as a plain website.
26. **The API is defended like a LAN service.** Cross-origin POSTs are refused, bodies are capped
before parsing, keys are sanitised before they reach a header, and auth is checked before any
body work. See [Security notes](#security-notes) for the full list and the known limitations.
27. **A language guard repairs accidentally translated transcriptions.** `TEXT_MODEL` (`gpt-oss:120b`)
is the strongest multilingual model in the free tier, so when the transcription check
(`lib/language.ts`: expected script vs. actual script, distinctive diacritics, English stopword
density) flags a mismatch, the text model rewrites the JSON back into the original language
before generation. Verified both ways: it fires on a synthetic English-but-declared-Czech
transcription and never fires on real Czech notes (no extra calls, no added latency).
28. **The first screen is the API-key gate.** In `user-key` mode a browser without a key gets a
full-screen, monochrome gate instead of the dashboard: one masked input, one Continue button,
and a "?" sheet with the steps to a free key. The gate reuses the cached 1-token probe through
`/api/health`, so an invalid key and an unreachable Ollama are worded differently. The old
dashboard banner is gone; key management stays in Settings, and removing the key brings the
gate back. The gate covers `/`, `/create` and deck study, but never `/offline`.
29. **The install prompt is a teaching sheet, not a nag.** It appears once, phones/tablets only,
a second after load, and never in standalone mode; dismissing it silences it for 7 days. It
detects iOS vs Android and Safari vs Chrome vs an in-app browser so step 1 always matches the
device, uses Chrome's deferred `beforeinstallprompt` for a one-tap install where available, and
stays reachable from Settings. It is mounted in the root layout, so it can appear over any
screen — including the key gate — without blocking the page underneath.
14. **`gpt-oss:120b` does all content work — but it cannot read images** (`this model does not
support image input`), so `VISION_MODEL` has to stay a multimodal model (`gemma4:31b-cloud`).
Text-only inputs — pasted text, PDFs with a text layer, DOCX/PPTX/ODT — never touch the vision
model at all, so those decks are produced by `gpt-oss:120b` start to finish.
13. **No auth, no server storage.** Everything personal lives in `localStorage`; the only server
state is the request in flight.
14. **`canvas-confetti` and `framer-motion` are the only UI dependencies.** Document parsing adds
`pdfjs-dist`, `mammoth` and `jszip`, all dynamically imported (the `/create` page grew ~3 kB;
the parsers load only when a document is added). Animations are skipped when
`prefers-reduced-motion` is set.
15. **Tailwind v4 class-based dark mode** with a pre-paint script so there is no theme flash, and
`suppressHydrationWarning` on `<html>`.
16. **Testing.** The pipeline was first tested against a mock Ollama HTTP server (including a
deliberately invalid first response to exercise the repair path) with the UI driven by headless
Chromium, then re-run against the real Ollama cloud. Current suites: 13/13 document extraction
(one fixture per format, including a text PDF and a scanned PDF), 15/15 language preservation,
24/24 activity/scoring/feed checks and 11/11 real-model browser checks.
Reference in a new issue View git blame Copy permalink