Field note · Everyday AI

How to Spot AI Hallucinations

Published June 28, 2026 · Empire Publishing

Short answer: An AI "hallucination" is a confident, fluent, completely invented answer. You can't tell it's wrong by how it sounds — that's the whole trap. You catch it by watching for a few specific tells and running a 30-second check before you act on anything that matters.

Why fluent and wrong look identical

A large language model doesn't look facts up. It predicts the next most plausible words, one after another, based on patterns in its training. Most of the time plausible and true line up, which is why the tool is useful. But when they diverge — an obscure detail, a recent event, a citation it never actually saw — the model doesn't stop and say "I don't know." It generates something that sounds right with the exact same confidence it uses for things that are right. There is no wobble in its voice. That's why you can't catch a hallucination by reading harder.

The five tells

No single one is proof. Two or more together means stop and verify:

• Suspiciously specific, with no source. Exact figures, dates, names, or quotes presented with zero attribution — the more precise the claim and the vaguer the origin, the more suspect it is.
• Perfect confidence on the obscure or the recent. The more niche the topic, or the closer to "today," the more likely the model is filling a gap with invention rather than knowledge.
• Citations you can't find. A real-looking study, case, book, or URL that returns nothing when you actually search for it. Fabricated references are the single most common hallucination — and the most dangerous, because they look like proof.
• The answer changes when you ask again. Pose the same question in a fresh chat. If the "facts" shift, they were generated, not retrieved.
• Claims past the training cutoff. Confident statements about events after the model was trained are guesses dressed as reporting.

The 30-second habit that catches most of it

You don't need to fact-check everything — only the things you're about to act on. Before a checkable claim leaves your hands, do one of three things: ask the model for a source and open it; paste the claim into a search engine and see if anything independent backs it; or ask the same question again, cold, and watch whether the answer holds. Thirty seconds, and most invented answers fall apart. The habit isn't paranoia — it's the difference between AI as a fast first draft and AI as a liability.

The short list: what to never let AI decide

Some answers you can afford to be wrong about. Some you can't. The rule is simple — don't let AI have the final word on anything you can't undo. Medical and medication decisions, legal filings, money moves, anything safety-related, and anything you'd be unable to walk back: these get a human and a real source, every time. Advice you can ignore is fine. An action you can't take back is not.

A real one, so it's not abstract

In Mata v. Avianca, two New York lawyers filed a brief built on court cases ChatGPT had invented whole — fake names, fake quotes, fake citations, all fluent and authoritative. Nobody checked. The court sanctioned them, and the story went around the world. That's the shape of every hallucination that hurts someone: the model sounded sure, a person believed it, and the cost landed on the human who acted. The fix wasn't a smarter model. It was thirty seconds of verification that never happened.

Frequently asked

What is an AI hallucination?

A confident, fluent, false answer — a made-up fact, quote, case, or citation, written with the same authority as a true one because the model is predicting plausible text, not recalling verified knowledge.

How do you tell if AI is making something up?

Watch the five tells — unsourced specifics, confidence on obscure or recent topics, citations you can't find, answers that change on a re-ask, and claims past the training cutoff — and verify before you act.

Can you stop AI from hallucinating completely?

No, but you can cut it sharply: ask for checkable sources, ask it to admit uncertainty, feed it the source material, and never make it the final authority on anything irreversible.

← More field notes