Lesson 5 of 23 · Part 0: Foundations

Hallucination

When AI confidently says things that aren't true — and how to fight it.

Last time: The system prompt, messages[0], is the job briefing that sets the AI's role and rules.
Today: Sometimes AI sounds sure but is wrong. We learn why, and five ways to fight it.
ASSISTANT

"The Eiffel Tower was completed in 1892 and stands about 330 meters tall. It was built by Gustave Eiffel's engineering company for the World's Fair."

The model sounds completely sure. But it's wrong.

How the LLM "thinks":

"The Eiffel Tower was built in 18__"

→ predicts: "92"✗ Pattern match: 18XX = old European thing

"Water boils at ___°C"

→ predicts: "100"

"The speed of light is ___"

→ predicts: "3×10⁸ m/s"

"The first iPhone was released in ___"

→ predicts: "2008"
🧠

"I don't know facts. I predict what text looks like."

Plausible ≠ True

Spot the errors — they're subtle:

AIThe Golden Gate Bridge was completed in 1936.
Actually: 19371 year off
AIHumans are born with 206 bones.
Actually: babies have about 270–300; some fuse as we growHalf-true
AIEinstein won the Nobel Prize for relativity.
Actually: for the photoelectric effectCommon misconception

Close enough to believe. Wrong enough to cause problems.

"Give me academic sources for your claim"

The AI happily provides:

Thistlewood, B. & Quenby, M.

"Deep Learning Approaches to Everyday Language"

Journal of Synthetic Cognition, Vol. 42, 2021

Varga-Ellison, P. et al.

"Attention Mechanisms Revisited"

Lakeside Letters on Neural Computing, 15(3), 2022

Moorcraft, D. & Halvani, S.

"Scaling Rules for Talking Machines"

Proceedings of the Glimmerton AI Symposium, 2020

Perfect formatting. Zero real papers. Every name here is invented.

It sounds right. It reads right. But it's completely wrong.

🏥Medical Query

"What causes migraines?"

"Migraines are triggered by the cortical spreading depolarization of the trigeminal nerve pathway, releasing CGRP neuropeptides at 4.7ng/mL..."

✓ Real medical terms

✓ Plausible numbers

✗ The 4.7ng/mL figure is fabricated

💻Programming Query

"How do I use the TableSprout.js library?"

"First install it: npm install tablesprout
Then import: const df = require('tablesprout')
Use df.readCSV() to load data..."

✓ Correct npm syntax

✓ Realistic API design

✗ This library doesn't exist

Both answers use correct grammar, real jargon, and proper formatting — but the facts are invented.

Can you tell which is right and which is wrong?

Look at how confident the AI sounds (illustrative numbers):

"The Earth orbits the Sun."✓ TRUE
Confidence:
98%
"Water boils at 100°C at sea level."
Confidence:
97%
"The Great Wall of China is visible from space."
Confidence:
95%
"Humans use only 10% of their brain."
Confidence:
93%
All bars look the same — confidence tells you NOTHING about truth

Fix 1: Use RAG

Without RAG

Question:

"What were Q3 sales?"

Answer:

"Revenue was approximately $3M"

✕HALLUCINATED

With RAG

Question:

"What were Q3 sales?"

Answer:

"$4.2M, up 18% YoY [Source: Q3 Report]"

✓GROUNDED

Ground the model in YOUR data.

Fix 2: Lower Temperature

T = 1.5

Q: "What is the boiling point of water?"

"Water boils at roughly 96°C depending on the quantum resonance of the molecular structure and atmospheric vibrations..."

✕Creative but wrong

T = 0

Q: "What is the boiling point of water?"

"Water boils at 100°C (212°F) at standard atmospheric pressure (1 atm)."

✓Correct and precise

For facts, boring is good.

Fix 3: Say "I Don't Know"

Before

System prompt:

"You are a helpful assistant."

LLM response:

"The company was founded in 1987 by Dr. James Wilson and initially focused on biotech research..."

✕Makes up an answer

After

System prompt:

"You are a helpful assistant. If you don't have specific data, say 'I don't have enough information'"

LLM response:

"I don't have enough information about that specific topic."

✓Honest and safe

The best answer is sometimes "I don't know."

Fix 4: Verify Sources

Ask AI a question → AI responds with 3 citations:

[1] Vaswani et al., "Attention Is All You Need", NeurIPS 2017✓ Real
[2] Quenby, R., "Advanced AI Metrics", arXiv:2305.1847✗ Fake
[3] Thistlewood & Varga, "Scaling LLM Performance", ICML 2024✗ Fake

Google the citations → 2 out of 3 are fabricated

Always verify. NEVER trust citations blindly.

Fix 5: Draft, Not Truth

Use AI For

✓Brainstorming
✓First drafts
✓Code scaffolding
✓Summaries

Don't Use AI For

✕Legal advice
✕Medical diagnosis
✕Financial decisions
✕Citing specific facts

Treat AI output as a draft, not gospel.

When Hallucination is OK

✍️

Creative Writing

Hallucination IS the feature

💡

Brainstorming

You WANT unexpected ideas

📝

First Drafts

Speed > accuracy at this stage

The key: know when accuracy matters.

Key Takeaways

LLMs hallucinate because they predict plausible text, not truth
Use RAG to ground responses in real data
Lower temperature for factual tasks
Add "say I don't know" to your system prompt
Always verify AI citations — many are fabricated
Treat AI output as a draft, not gospel

Trust but verify — the #1 rule of working with AI!

← Prev
1/0
Next: RAG →