Make a quiz from lecture notes, with a source quote per answer

Ask for "5 multiple-choice questions from these notes" and you get a clean quiz with an answer key. What you don't get is the sentence in your notes that each answer rests on, so checking the key means rereading all the notes. The night before an exam, nobody wants that.

[1] Put the notes in a file. These are made-up notes for an intro psychology course: seven concepts, with a deliberate caveat in the last one ("this course only covers everyday shopping"). Only part of the file is shown.

[Intro Psychology, Week 3 notes] 1. Anchoring. The first number a person sees becomes a reference point, and the final estimate gets pulled toward it. A $70 item placed next to a $100 list-price item looks cheap. The effect appears even when the anchor is a randomly chosen number. 2. Loss aversion. For the same amount, a loss hurts more than a gain feels good. The lecture put the ratio at roughly 2 to 1. This is why "You lose $10 if you skip this" works harder than "Save $10". … 7. Framing effect. The same content leads to different choices depending on how it is worded. "90% survival" and "10% mortality" carry the same information, but the first sounds more positive. It also shows up in real decisions such as surgery, but this course only covers everyday shopping situations.

[2] Add a quote requirement to the prompt. Each question gets a quote field holding one sentence copied from the notes, and a question with no such sentence is dropped. This is the approach in Anthropic's guide to reducing hallucinations (find a supporting quote for each claim, retract the claim if none exists), applied to quiz writing. The output is a JSON array so the script in step 4 can read it.

Write 5 multiple-choice practice questions in English from the lecture notes inside <notes>. Rules:- Ask only about content that is written in the notes. Do not use outside knowledge.- Each question has 4 choices (A-D) and exactly one correct answer. The wrong choices must be clearly wrong according to the notes, not just "less right".- For every question give a "quote": one sentence copied character for character from the notes that supports the correct answer. The quote must not also fit a wrong choice.- If you cannot find such a quote, drop the question. Fewer than 5 is fine.- Cover 5 different concepts.- Output only a JSON array, no other text. Each item: {"q": "...", "choices": {"A": "...", "B": "...", "C": "...", "D": "..."}, "answer": "A", "quote": "..."} <notes>{your lecture notes}</notes>

[3] Run it twice.

$ (cat head.txt notes.en.txt; echo "</notes>") > prompt.en.txt$ claude -p --model claude-sonnet-5-5 --max-turns 1 --tools "" --setting-sources "" --strict-mcp-config < prompt.en.txt > out.en.1.txt

One question from the first run:

[  {    "q": "According to the lecture, roughly what is the ratio of how much a loss hurts to how good an equal gain feels?",    "choices": {      "A": "1 to 1",      "B": "1 to 2",      "C": "3 to 1",      "D": "2 to 1"    },    "answer": "D",    "quote": "The lecture put the ratio at roughly 2 to 1."  },  …]

[4] Check the quotes. The script collapses whitespace in the notes, counts how often each quote appears, and confirms the answer letter is one of the four choices. Anything other than exactly one hit exits with 1.

#!/usr/bin/env python3"""usage: check.py notes.txt quiz.json"""import json, sysnotes = " ".join(open(sys.argv[1], encoding="utf-8").read().split())bad = 0for i, it in enumerate(json.load(open(sys.argv[2], encoding="utf-8")), 1):    n = notes.count(" ".join(it["quote"].split()))    ok = n == 1 and it["answer"] in it["choices"] and len(it["choices"]) == 4    bad += not ok    print("OK  " if ok else "BAD ", f"Q{i}: quote found {n}x, answer {it['answer']}, choices {len(it['choices'])}")print(f"{i} questions, {bad} to review")sys.exit(1 if bad else 0)

$ python3 -I check.py notes.en.txt out.en.1.txt; echo "exit $?"OK   Q1: quote found 1x, answer B, choices 4OK   Q2: quote found 1x, answer D, choices 4OK   Q3: quote found 1x, answer A, choices 4OK   Q4: quote found 1x, answer C, choices 4OK   Q5: quote found 1x, answer C, choices 45 questions, 0 to reviewexit 0

The second run passed all five as well. A quote that isn't in the notes gets caught like this. Here Q2's quote was edited by hand to "roughly 3 to 1" (the model did not make this error).

$ python3 -I check.py notes.en.txt out-bad.en.json | grep -v '^OK'; echo "exit ${PIPESTATUS[0]}"BAD  Q2: quote found 0x, answer D, choices 45 questions, 1 to reviewexit 1

[5] Read the rest yourself. The script only proves a quote exists. Whether that sentence supports the keyed answer and no other choice means putting each question next to its quote and reading both. Then take the quiz without the key, and any question where your answer differs from the key probably has two defensible answers.

Where it went wrong

• The right choice repeats the quote. In both runs, all 10 correct choices are a phrase lifted from the question's own quote. In the second run, Q3's correct choice "Only future costs and future benefits" is the tail of the quote word for word, so you can pick it by matching phrases without understanding anything. The quote check passes it. Add a rule such as "Choices must not copy the quote" and read the result again; that rule was not run in English.
• Coverage skews. Across the two runs, status quo bias appeared once and the framing effect never did, because the prompt asks for "5 different concepts" without naming them. Name the missing ones and run it again.

The quote check filters out questions that go beyond the notes. It cannot tell you whether the notes are right: a wrong sentence in your notes becomes a well-sourced wrong answer. Treat the course's textbook or instructor as the authority on what is in scope. Planning study hours around the exam date is a separate job; see study plan from exam date and hours.