Grade against the question: stop requiring details the question never asked for

A short-answer case: "Why was Kryštof Harant executed?" with the student
answering "because he took part in the anti-Habsburg uprising" was failed
for not naming the year 1621 — a detail only present in the reference
answer, never asked by the question. The generator had put the date into
keyPoints, and the grader treated every key point as mandatory.

- Grading prompt: the QUESTION defines what is required. A key point
  counts only when the question asks for it or the answer is wrong or
  incomplete without it; extra detail in the reference answer (dates,
  places, names, numbers) is a bonus — never listed as missing, never a
  reason to fail. An accepted answer must return an empty "missing"
  array.
- Generation spec: short_answer keyPoints must contain only the ideas
  the question asks for (a "why" question requires the reason, not the
  year).
- gradeShortAnswer drops any stray "missing" entries when correct is
  true, so the UI can never show a gap on an accepted answer.

Verified: mock-Ollama integration test 9/9 through the real /api/grade
route (new rules sent upstream, incorrect answers keep their missing
point, accepted answers normalise to missing: []), regression 74/74,
mobile 51/51, i18n 27/27. Lint + build clean. SW freezmo-v18.
This commit is contained in:
Martin Dvořák 2026-10-09 18:13:29 +00:00
commit 6fb11fa405
3 changed files with 10 additions and 5 deletions

View file

@ -396,7 +396,9 @@ export async function gradeShortAnswer(input: {
signal: input.signal,
temperature: 0.1,
});
return value;
// An accepted answer never has "missing" points — anything the question did
// not ask about is a bonus, and the UI must not show it as a gap.
return value.correct ? { ...value, missing: [] } : value;
}
/* ------------------------------------------------------------------ */