I stopped letting the AI model write facts in my flashcard app
I made a flashcard app while studying for my boards. The design decision that may be useful outside flashcards is how it outputs generated cards: as character ranges into the source document, instead of free text from the model.
Spaced repetition works. In medical education, many people depend on it for studying. The main pain point is that making the cards is slow and cumbersome. This is one of the main reasons why AI generated flashcards have become so popular over the last couple of years.
When trying out these AI tools for my own studying, I ran into a problem that is seemingly inherently part of AI generated material. They hallucinated, and I couldn’t reliably spot the hallucinations while reviewing a list of generated cards. A flashcard app is an especially bad place for this because you learn this incorrect hallucinated information and confidently recall it as fact later down the line. In my case, that might have happened on a board exam or during patient care. The model being factual only most of the time, simply wasn’t good enough.
Putting “be accurate” in the prompt does roughly nothing. Asking for citations helps less than you’d expect. Every model I’ve tried will paraphrase a sentence into something subtly wrong and then cite the page it came from, making the card seem safer than it is. Searching the document by hand to fact check every generated answer defeats much of the point of generating the cards in the first place.
Selection, not writing
So, I removed fact-writing from the model’s job.
When you generate cards in Clozely, the client extracts the document text along with its character positions. The model chooses sentences worth turning into cards and quotes the answer text that should be hidden. Each quoted answer then goes through a grounding step that finds its exact character range in the source.
The grounding match has to use the source’s own words. It can normalize formatting differences such as case, whitespace, or an erroneous line break but it never uses fuzzy similarity. If the quoted answer cannot be anchored to the text, the card is dropped before the user sees it.
Internally, the card is stored as ranges within the document. The model decides which sentence matters and which words to hide. It doesn’t get to supply the answer from memory. If the system cannot point to the answer in the source, then I don’t trust the card enough to learn from.
There is one deliberate exception around the card stem. The main card text allows lazy matching and very light rewording because source material often makes poor flashcards as written. Textbooks are full of wording that would make awkward flashcards, in-line references, and sentences that only make sense next to the previous paragraph. The cloze answer, meaning the hidden part the user is tested on, still has to be verbatim source text. The framing can move a little, but the tested fact cannot.
Given this sentence:
Under stress, the adrenal cortex secretes cortisol in response to ACTH from the pituitary gland.
The model’s output amounts to “use this sentence and hide cortisol.” The system resolves the sentence and answer to exact ranges.
Reviewing the cards in context
Once a card has a position in the document, the client can draw it on the rendered page. Generated cards appear as highlights on the source document or webpage, just like highlights made by hand in Clozely.
This makes reviewing them for accuracy after generation much faster. I don’t have to work through a list of cards while hunting through a document for whatever might support them. I scroll through the document and see the proposed cards right on the sentences they came from.
The link to the source remains after the card is accepted. Clicking it later, either from Clozely’s built-in library or from inside Anki during review, opens the source and scrolls back to the highlight.
What this doesn’t solve
“Can’t hallucinate” has a narrow meaning here.
The constraint prevents fabrication, but it does not prevent bad judgment. The model can choose an unimportant sentence or choose to hide the wrong word. Model choice and prompt engineering still matter for those problems.
Getting models to follow this contract reliably was its own project. The validation layer took a lot of trial and error.
In my testing on ordinary prose, about 83% of the answers the model quotes anchor to the source verbatim on the first attempt. Another 1 to 2% anchor after formatting normalization: differences in case, collapsed whitespace, or a line break that a scan inserted in the middle of a phrase. The roughly 15% that can’t be anchored are dropped, not repaired.
Every rejected answer is logged with a reason. Some have been stitched together from text that is not contiguous anywhere in the document. Some quotes get anchored to the wrong row of a table, and get rejected. Other rejected cards merge two separate facts into one sentence.
This method of constraint could be useful anywhere a model is supposed to extract facts from a document and hallucinations are unusually costly. Make the output point back into the source, then discard anything that cannot be grounded there.
What failure modes do you see with this approach?