How to Fact-Check AI Answers: A Hallucination Checklist

The answer reads well, the numbers look plausible, and you have ten minutes before it goes to a client. That is the moment when most people either send it as is or reread it and call that a check. Neither works. To fact-check AI answers you need a different routine from proofreading: pull the claims out of the prose, sort them by how much damage they would do if wrong, and check the dangerous ones against something outside the chat. This piece gives you that routine, a copy-paste audit prompt, and a ten-minute checklist.

Why a confident answer tells you nothing

With a human colleague, hesitation is a signal. A chatbot sounds the same whether it knows or guesses.

OpenAI's own researchers explained why in a September 2025 paper, Why Language Models Hallucinate. Models are trained and graded like students on a test where a blank answer scores zero and a guess sometimes scores a point. Guessing wins, so the model produces "plausible yet incorrect statements" in the same voice it uses for things it knows.

You can see the effect in the Tow Center study published by Columbia Journalism Review in March 2025. Researchers ran 1,600 queries across eight AI search tools, asking each to identify the source of a news excerpt. More than 60 percent of the answers were wrong. ChatGPT Search got 134 of its 200 wrong and signaled any doubt in only 15. The tools have been updated many times since and the percentages will have moved. The pattern has not: a wrong answer looks exactly like a right one.

Sort the claims before you check any of them

Checking everything takes longer than writing the text yourself. Checking nothing is how invented case law ends up in court filings. The working middle is triage: some kinds of claims fail far more often than others, and they fail in recognizable ways.

Claim type	How it usually goes wrong	Fastest check
Citations and links	The source does not exist, or exists and says something else	Open the link, search the page for the claim
Direct quotes	A paraphrase dressed in quotation marks, or the wrong speaker	Search the exact phrase in quotation marks
Statistics	A real number with the wrong year, sample, or publisher	Find the original report, not an article repeating it
Prices, plans, laws, "current" facts	Correct as of the model's training data, outdated today	The vendor's or regulator's own page, with today's date
Names, titles, dates	Two people or events blended into one	One independent source
Summary of a document you supplied	Softened wording, missing exceptions	Spot-read the original sections
Calculations	Right method with a wrong input, or the reverse	Recompute the one number that matters
General explanations of well-known concepts	Rarely wrong in ways that matter	Read once with your own expertise

The first four rows are where most of our checking time goes.

Step 1: Make the model list its own claims

A typical case. A marketing manager asks for a one-page brief on email benchmarks for a client deck. The draft comes back with six percentages, two named industry reports, and a quote from an executive. In flowing prose it is hard to see how many separate facts you are about to vouch for. So the first move is to take the prose apart.

Below is a draft. Do not rewrite it and do not add new facts.

1. List every checkable claim as a numbered table with three columns:
   - the claim in one sentence
   - its type: number, date, name or title, direct quote, citation,
     cause-and-effect claim, legal or regulatory statement
   - the exact sentence in the draft it comes from
2. For each claim, state where it came from:
   (a) a document I gave you in this chat: quote the passage
   (b) a web page you opened in this chat: give the URL and quote the passage
   (c) your training data: write "from memory, unverified"
3. Mark the three claims that would do the most damage if wrong,
   given that this goes to [a client / my manager / a public post].
4. If you cannot trace a claim, say so. "I can't trace this" is a useful answer.

DRAFT:
[paste the text]

What comes back is a worklist. For the email brief, expect something like fourteen rows, with most of the percentages labeled "from memory, unverified." Those rows are your job now. The claims traced to a document or URL are faster: you only need to confirm that the quoted passage is really there.

How to adjust it. If the model labels everything as sourced, ask for the quoted passage for each row; a source it cannot quote goes back to "unverified." If the table is long, tell it who the reader is. Damage depends on the audience, and a wrong open rate matters less than a wrong claim about a named competitor. The same prompt works on text a colleague generated and forwarded to you.

Step 2: Open the source and find the sentence

For every high-damage claim, three questions.

Does the source exist? Click the link. Search the title of the report. A report that cannot be found on the publisher's own site in two minutes should be treated as invented until proven otherwise.

Does it say this? Use Ctrl+F for the number or a distinctive phrase. Real pages get cited for things they never said. In the Tow Center study, more than half of the responses from Gemini and Grok 3 cited fabricated or broken URLs.

Is it the original? Statistics travel. A figure attributed to a well-known research firm often turns out to come from a vendor's blog post that cites another blog post. Follow the chain until you reach the organization that collected the data, then read the year and the sample. If the chain ends nowhere, cut the number.

Quotes get the same treatment: paste the exact wording into a search engine in quotation marks. No results usually means the model wrote the quote.

Step 3: Get a second opinion that can disagree

Asking the same chat "are you sure?" is the most common check and the weakest. The model either apologizes and changes a correct answer, or defends a wrong one in more detail. It is rereading its own text with the same blind spots.

A better second opinion has two properties: a fresh context and access to the web. Open a new chat, ideally in a different tool with search turned on, and ask it to attack the claims instead of confirming them.

Here are five claims from a draft. For each one, search the web and try to prove it wrong.

Report one of three results per claim:
- CONFIRMED by a primary source: link plus the quoted passage
- CONTRADICTED: link plus the quoted passage
- COULD NOT VERIFY

Prefer the organization that published the number over articles that repeat it.
Do not answer from memory. If you did not open a page, say so.

CLAIMS:
[paste the high-damage rows from the table]

Treat the result as a set of leads. "Confirmed" still means you click the link and see the passage yourself, which now takes seconds because you know where to look. "Could not verify" on a statistic is a strong hint to delete it. We compared how the three main assistants handle search and citations in ChatGPT vs Claude vs Gemini for Work.

The ten-minute checklist

• The claims are in a table, each with a type and an origin.
• The three highest-damage claims are marked for this specific reader.
• Every cited source was opened, and the claim was found on the page with Ctrl+F.
• Every statistic is traced to the organization that produced it, with year and sample.
• Every direct quote was searched as an exact phrase.
• Every "current" fact (price, plan, deadline, rule) was checked on the official page today.
• One number that matters was recomputed outside the chat.
• A fresh chat with web search tried to disprove the high-damage claims.
• Anything that could not be verified is deleted or openly marked as an estimate.
• For summaries: two sections of the original were read in full and compared.

Scale it to the stakes. An internal brainstorm needs the first two lines. A client deliverable needs all ten. Anything going to a court, a regulator, or the press needs a person with domain expertise reading every line, whatever the checklist says.

Where this breaks

Omissions. The checklist catches false statements. It does not catch what a summary left out, and a summary of a 40-page report that drops the one exception clause contains no false sentence. Ask separately: "What did this summary leave out that a [CFO / lawyer / client] would want to know?" Then read those sections yourself. We use the same compare-with-the-original pass for contracts in our contract review prompt.

Fields where you cannot judge the answer. Verification assumes you can recognize a good source. In law, medicine, and tax, that takes training. Damien Charlotin's public database of court decisions involving AI-fabricated material listed 2,097 cases as of October 1, 2026. It counts decisions where a court or tribunal found, or implied, that a party had relied on invented content.

Internal facts. If the claim is about your own company's numbers or a client's history, the web has nothing to check against. The only source is the document or the person who knows.

Numbers. Arithmetic and data analysis fail in their own ways and need their own checks. We cover them in Is ChatGPT Accurate with Numbers? and How to Analyze a Spreadsheet with ChatGPT.

Vague prompts. A model given no source material fills the gaps from memory, which is where invented details come from. Supplying the documents and telling it to answer only from them removes much of the checking load. The reasons are in Why AI Answers Are Generic.

Take this further in the practicum

Verification is part of Session 3 of AI system in two weeks, the session on documents, data, and visuals. We work through where AI is reliable and where it is confidently wrong, on files and questions the participants bring from their own jobs, and the course materials include a full answer-verification checklist that this article shortens. By the end of the session you have your own version of the routine, fitted to the kind of claims your work produces. The program and upcoming cohorts are on the course page.