Module 3, Lesson 5
TL;DR: AI output fails in four predictable ways: hallucination (made-up facts), tone mismatch (wrong voice for the audience), missing constraints (left out what you asked for), and format errors (wrong structure or length). For each failure, you quote the specific text, name the category, and explain why it matters.
In Module 2, you saw the Q3 retention memo with three failures hiding in professional-looking text. Now you'll learn to catch these systematically instead of hoping you notice them.
1. Hallucination / Factual Inaccuracy
The AI generates content that sounds authoritative but is partially or entirely invented. This includes fake statistics, fabricated dates, nonexistent sources, invented procedures, and made-up details of any kind. Remember: hallucination is the mechanism (the model generates with confidence whether or not the claim is grounded in anything verifiable — it doesn't flag uncertainty, it just produces). Factual inaccuracy is the symptom (the text is wrong). The fix is verification — anything specific the AI gave you that you didn't put in your prompt has to be checked.
How to spot it: Any specific claim — a number, a name, a date, a citation — that you didn't provide in your prompt should be verified. If you can't verify it from your own knowledge or a quick check, treat it as suspect.
2. Tone Mismatch
The output uses the wrong voice for the audience. Formal language in a casual context, or casual language in a professional one. Technical jargon when the reader is non-technical. Sales-speak when the reader expects straight facts.
How to spot it: Read the output as if you're the recipient. Does it sound like YOU wrote it to THIS person? Or does it sound like a generic template?
3. Missing Constraint
The AI left out something you specifically asked for — or something that's obviously necessary even if you forgot to ask. You said "include the deadline" and there's no deadline. You asked for "three options" and got two.
How to spot it: Compare the output against your prompt's constraints, one by one. Then ask: "Is anything obviously missing that I should have asked for?"
4. Format Error
The output is the wrong length, wrong structure, or wrong layout. You asked for bullet points and got paragraphs. You asked for 150 words and got 400. You asked for an email and got an essay.
How to spot it: Check length, structure, and format against your prompt specifications. This is the easiest failure to catch and fix.
When you find a failure, document it with three pieces:
Here's an AI-generated draft of a client email and its diagnosis:
Dear Ms. Chen, Thank you for choosing Apex Solutions for your digital transformation needs. As per our discussion on March 3rd, I wanted to provide an update on the implementation timeline. Phase 1 is 95% complete, with the remaining 5% focused on data validation. According to Gartner's 2025 Digital Transformation Index, companies that complete Phase 1 ahead of schedule see 23% higher ROI in Year 1. We anticipate completing Phase 2 by April 15th. Best regards
Diagnosis:
| # | Quoted Text | Failure Category | Why It Matters | |
|---|---|---|---|---|
| 1 | "According to Gartner's 2025 Digital Transformation Index, companies that complete Phase 1 ahead of schedule see 23% higher ROI in Year 1" | Hallucination | This publication and statistic may not exist. If the client checks, your credibility is destroyed. Never cite AI-generated statistics in client communications without verifying them. | |
| 2 | "Thank you for choosing Apex Solutions for your digital transformation needs" | Tone mismatch | This is sales language, not project update language. The client already chose you — thanking them for "choosing" you sounds like a post-purchase email, not a progress report. | |
| 3 | (Missing) No mention of next steps or client action needed | Missing constraint | A project update without "what we need from you" or "what happens next" is incomplete. The client reads it and thinks "OK, so what?" |
You've seen the four categories and the diagnosis protocol. Now apply them to your own AI output from the previous lesson.