Generation guards: cards with repeated wording never ship

A generated Czech literature deck reached a phone with three identical
matching definitions — the card could not be solved. The pipeline only
deduped matching terms (and ordering items), so repeated definitions and
options sailed through.

- parseDeck now rejects any card whose options, items, terms or definitions
  repeat (case-insensitive), through the same drop-or-repair path as the
  ordering giveaway guard: the card is dropped, or a deck left with too few
  cards gets one repair round with a clear reason.
- The matching dedupe covers definitions too, and both it and the ordering
  one only trim when the card still meets the schema minimum afterwards —
  a shorter card would fail validation on the next read and take the whole
  deck down with it (latent data-loss bug).
- The generation prompt states the uniqueness rule.
- phase-1 suite: a stub Ollama returns a deck with a repeated-definition
  matching card (the reported shape), a repeated-option card and a
  salvageable one; asserts the first two are dropped, the third is trimmed
  to three pairs and the good cards ship (43 checks).
- README: decision 51.

Server-side only (lib/ai.ts, lib/prompts.ts) — no client bundle change, no
SW bump.
This commit is contained in:
Martin Dvořák 2026-10-12 05:37:03 +00:00
commit f202653017
4 changed files with 177 additions and 22 deletions

View file

@ -310,6 +310,14 @@ function validateDeck(parsed: unknown, minimum: number, requested: number) {
);
return;
}
// Repeated wording makes a card impossible to answer: two identical
// options, items, terms or definitions cannot be told apart.
if (repeatedEntries(activity)) {
rejected.push(
`#${index + 1} (${activity.type}): repeated entries — every option, item, term and definition must be unique.`,
);
return;
}
activities.push(activity);
} else {
const type = (item as { type?: unknown } | null)?.type;
@ -581,42 +589,71 @@ export async function refineHomeworkText(input: {
/* ------------------------------------------------------------------ */
/**
* Small fixes so the UI can trust the data: ordering items must be unique
* (they are used as React keys and are ambiguous otherwise), fill_blank
* sentences must actually contain a blank, and word banks must contain
* the answer.
* Small fixes so the UI can trust the data: ordering items and matching pairs
* must be unique (they are used as React keys, and a repeat makes the card
* ambiguous), fill_blank sentences must actually contain a blank, and word
* banks must contain the answer.
*
* A salvage only happens when the card still meets the schema's minimum after
* deduping — a shorter list would fail validation on the next read and take
* the whole deck down with it.
*/
function sanitiseActivity(activity: GeneratedActivity): GeneratedActivity {
switch (activity.type) {
case "ordering": {
const seen = new Set<string>();
const items = activity.items.filter((item) => {
const key = item.trim().toLowerCase();
if (seen.has(key)) return false;
seen.add(key);
return true;
});
return items.length === activity.items.length ? activity : { ...activity, items };
const items = dedupeBy(activity.items, (item) => item);
return items.length === activity.items.length || items.length < 3 ? activity : { ...activity, items };
}
case "fill_blank": {
if (/_{2,}/.test(activity.sentence)) return activity;
return { ...activity, sentence: `${activity.sentence.trim()} ____` };
}
case "matching": {
const seen = new Set<string>();
const pairs = activity.pairs.filter((pair) => {
const key = pair.term.trim().toLowerCase();
if (seen.has(key)) return false;
seen.add(key);
return true;
});
return pairs.length === activity.pairs.length ? activity : { ...activity, pairs };
// Terms first, then definitions: one definition per term, or the learner
// cannot tell the pairs apart.
const pairs = dedupeBy(dedupeBy(activity.pairs, (pair) => pair.term), (pair) => pair.definition);
return pairs.length === activity.pairs.length || pairs.length < 3 ? activity : { ...activity, pairs };
}
default:
return activity;
}
}
/** Case-insensitive, trimmed dedupe that keeps the first occurrence. */
function dedupeBy<T>(items: T[], key: (item: T) => string): T[] {
const seen = new Set<string>();
return items.filter((item) => {
const value = key(item).trim().toLowerCase();
if (seen.has(value)) return false;
seen.add(value);
return true;
});
}
/** True when a list repeats an entry (case-insensitive, trimmed). */
function hasDuplicates(values: string[]): boolean {
return dedupeBy(values, (value) => value).length !== values.length;
}
/** The values a card needs to keep distinct to stay answerable. */
function repeatedEntries(activity: GeneratedActivity): boolean {
switch (activity.type) {
case "multiple_choice":
return hasDuplicates(activity.options);
case "fill_blank":
return activity.options ? hasDuplicates(activity.options) : false;
case "ordering":
return hasDuplicates(activity.items);
case "matching":
return (
hasDuplicates(activity.pairs.map((pair) => pair.term)) ||
hasDuplicates(activity.pairs.map((pair) => pair.definition))
);
default:
return false;
}
}
/** Compact, human-readable zod error for the repair prompt. */
function summariseZodError(error: { issues: Array<{ path: PropertyKey[]; message: string }> }): string {
return error.issues

View file

@ -82,6 +82,7 @@ export const ACTIVITY_JSON_SPEC = `Each activity is one JSON object with a "type
"options" is optional but preferred: 3-6 plausible choices including the answer.
5. matching - 3 to 6 term/definition pairs
Every term and definition must be distinct: a definition describes exactly one term, so never reuse a definition for two pairs.
{ "type": "matching", "topic": "short label", "instruction": "Match each term to its definition.", "pairs": [{ "term": "Osmosis", "definition": "diffusion of water across a membrane" }], "explanation": "..." }
6. ordering - 3 to 7 items given in the CORRECT order (the app shuffles them)
@ -105,6 +106,8 @@ Hard rules:
- Vary the activity types. Use a healthy mix of all seven types below, and never put more than two activities of the same type in a row.
- Cover the whole notes, not just the first paragraph. Spread activities across the different topics, definitions, formulas and facts.
- Make the wrong options plausible (common mix-ups from the same notes), never silly filler.
- Every option, item, term and definition on a card must be unique: never repeat the same wording
twice on one card — a repeated option or definition makes the card impossible to solve.
- Ordering items must never reveal their position: no dates, years, times, step numbers or ordinals
("first", "next", "finally", "step 2"). Describe the event or stage itself.
- Keep questions short enough to read on a phone screen. No walls of text.