Grade against the question: stop requiring details the question never asked for
A short-answer case: "Why was Kryštof Harant executed?" with the student answering "because he took part in the anti-Habsburg uprising" was failed for not naming the year 1621 — a detail only present in the reference answer, never asked by the question. The generator had put the date into keyPoints, and the grader treated every key point as mandatory. - Grading prompt: the QUESTION defines what is required. A key point counts only when the question asks for it or the answer is wrong or incomplete without it; extra detail in the reference answer (dates, places, names, numbers) is a bonus — never listed as missing, never a reason to fail. An accepted answer must return an empty "missing" array. - Generation spec: short_answer keyPoints must contain only the ideas the question asks for (a "why" question requires the reason, not the year). - gradeShortAnswer drops any stray "missing" entries when correct is true, so the UI can never show a gap on an accepted answer. Verified: mock-Ollama integration test 9/9 through the real /api/grade route (new rules sent upstream, incorrect answers keep their missing point, accepted answers normalise to missing: []), regression 74/74, mobile 51/51, i18n 27/27. Lint + build clean. SW freezmo-v18.
This commit is contained in:
parent
829a71f4ff
commit
6fb11fa405
3 changed files with 10 additions and 5 deletions
|
|
@ -396,7 +396,9 @@ export async function gradeShortAnswer(input: {
|
|||
signal: input.signal,
|
||||
temperature: 0.1,
|
||||
});
|
||||
return value;
|
||||
// An accepted answer never has "missing" points — anything the question did
|
||||
// not ask about is a bonus, and the UI must not show it as a gap.
|
||||
return value.correct ? { ...value, missing: [] } : value;
|
||||
}
|
||||
|
||||
/* ------------------------------------------------------------------ */
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue