Why Your AI Keeps Making Things Up (and the Defenses That Actually Work)
A chatbot invented a 20% discount and 22 customers asked for it. Why "don’t hallucinate" fails — and the five layered defenses that actually reduce AI fabrications.
A Bangkok tour operator's AI chatbot invented a promotion. "Mention this chat for 20% off sunset cruises" — a policy that had never existed. Twenty-two customers asked for the discount before anyone noticed. The owner's reaction: "I told it not to make things up! Rule six, right there in the prompt!"
It was. And it didn't matter — because "don't hallucinate" buried in a long instruction list is one of the weakest controls that exists. 
This guide explains why confident AI mistakes happen even when you forbade them, and the layered defenses that actually work for a business that lets AI talk to customers.
AI hallucinations happen because language models are optimized to produce coherent, fluent output — when data is thin or the task rewards "engaging" writing, sounding right beats being right. You reduce them structurally, not by prohibition: break work into small steps, frame accuracy as the primary mission ("if unsure, say it's missing"), require a citation or quote for every claim, add a separate QA pass that verifies before sending, and include examples where the correct answer was "not enough data." Layered together these cut fabrications dramatically — no single control is enough. :::
Why "don't make things up" fails
Four behaviors explain most hallucinations in the wild:
Narrative pressure
The model has the facts but "spices them up." Ask for a vivid, engaging reply and it may add flourishes the source never contained. It's not lying; it's fulfilling your request for better writing.
The spirit-vs-letter trap
The model decides a slightly-off sentence "captures the spirit" of your dry price sheet. To the AI, it made the content more usable. To your customer, it's a promised discount that doesn't exist.
The proximity problem
Instructions aren't weighed equally. Rule six of a forty-rule prompt is nowhere in the model's attention when it's mid-paragraph composing a persuasive reply. What's closest to the task wins.
The polished-example trap
If your few-shot examples all show rich, detailed answers, the model treats that richness as a requirement — and invents details to match, even when the current data is thin. Your "golden" examples teach it that a good answer is a full answer.
| Failure | What it looks like | Structural fix |
|---|---|---|
| Narrative pressure | Invented promos, embellished claims | Accuracy-first framing + QA pass |
| Spirit vs letter | "Smoothed" prices, softened policies | Exact-extraction wording, citations |
| Proximity | Rules ignored mid-task | Small steps, rule next to the task |
| Example pressure | Fabricated details to match examples | Include "insufficient data" examples |
The five defenses that work
1. Break the workflow down
The single most effective change: narrow the scope of each AI call. One job per step — extract, then classify, then draft — instead of one mega-prompt juggling goals. Fewer simultaneous objectives means fewer places to wander. (This is the same small-steps principle behind the Loader-Worker analysis pattern and eval suites.)
2. Frame accuracy as the mission, not a prohibition
"Don't hallucinate" is a negative command the model can't operationalize. Give it a positive identity instead:
ACCURACY FIRST — your most important mandate.
Accuracy beats a vivid narrative. If a fact is missing,
state that it is missing. A plain honest answer is better
than a polished one containing errors.
Mission-framing outperforms rule-listing because it stays active at the moment of composition.
3. Force citations
Require a source for every claim: "Every quote must include [Source: doc ID]." The act of having to locate an ID for each sentence makes inventing one dramatically harder — the model has to look back at the data to finish its own sentence.
4. Add a separate QA pass
After drafting, switch roles — the model reviews its own output as a critic:
{
"qa_step": {
"run_after": "draft",
"instructions": "Verify every quote is verbatim. Verify every
number matches the source sheet. If unsure of any claim,
remove it. Paraphrase rather than quote when exactness
cannot be verified."
}
}
Draft and critique as separate calls with separate criteria — a model asked to simultaneously write persuasively and verify skeptically does neither well.
5. Show it "I don't know" examples
Curate examples where the right answer was "the data doesn't say" — a reply that declines to quote, a field left empty. You're teaching the acceptable shape of an honest gap. Prompts whose examples are all perfect teach the opposite: that completeness is mandatory, whatever the data says.
The wording triggers nobody warns you about
Certain verbs invite fabrication. If you want literal output, avoid the creativity vocabulary:
# verbs that invite "improvement" → use instead
summarize → extract the following fields
rewrite nicely → reproduce with these exact substitutions
make it catchy → format per the template
polish this → correct grammar only, change no facts
"Extract," "transcribe," "list exactly" anchor the model to the source. "Summarize," "rewrite," "make it engaging" hand it a license to smooth.
Managing expectations honestly
Hallucination is inherent to current architectures — reducible, not removable. For customer-facing deployments, say so plainly: outputs pass automated checks and a QA pass, a human reviews high-stakes replies, and nothing irreversible executes without approval. That transparency builds more trust than pretending perfection.
A concrete hardening checklist
hallucination_hardening:
workflow: one task per model call
mission: accuracy-first framing at the TOP of every prompt
evidence: quote-or-empty rule on every factual field
qa: separate critic pass before anything customer-facing
examples: include 2 "insufficient data" cases in every prompt
evals: 5 must-never-say scenarios run on every prompt change
humans: final send on pricing/policy/reviews stays manual
Run that checklist against any AI that touches your customers and most invented-promo disasters become structurally impossible. And while you're hardening the conversation layer, check the discovery layer too: a free geo-grid scan at https://gbppeak.com/free-maps shows exactly where your Google Maps ranking stands across your service area.
Frequently Asked Questions
Why does the AI still hallucinate when I told it not to?
"Don't hallucinate" is a buried negative command that loses to nearer, positive goals like "write an engaging reply." Models respond better to a positive mission ("accuracy first; say when data is missing") placed close to the task, plus structural checks — citations and a QA pass — that catch what framing misses.
Is one complex prompt better than many small ones?
For accuracy, many small ones win. Each narrow step gives the model fewer objectives to juggle and lets you place a guardrail at every stage. Reserve the big single prompt for low-stakes creative work where a flourish costs nothing.
How do I stop the AI from "improving" my data?
Change the verbs. "Summarize," "rewrite," and "make it catchy" license smoothing; "extract exactly," "transcribe," and "reproduce with these substitutions" anchor it to the source. Pair with a quote-verification QA pass for anything customer-facing.
Can hallucinations be eliminated completely?
Not with current architectures — they're a byproduct of how language models generate fluent text. The realistic goal is layered reduction: small steps, accuracy framing, citations, QA passes, and human review on anything irreversible. Businesses that deploy that stack see fabrication drop to a rare, caught event.
Final note
The goal isn't a system that sounds good — it's one that's reliably useful. Every defense in this guide costs minutes to add and pays off the first week. Start with the mission statement and the QA pass; they're the two highest-leverage lines you'll ever add to a prompt.