Why AI Makes Things Up

A user asks a chatbot for a book recommendation and receives a title, an author, a publisher, and a one-paragraph summary. It all sounds right. Then a search for the book turns up nothing, because it does not exist. In another case, a user asks about the legal rule on a topic and gets a confident answer with a case citation, and the case turns out to be a blend of three real ones with an invented name on top.

This is the problem commonly called AI hallucination. It is one of the most important things to understand about tools like ChatGPT, Claude, Gemini, and Perplexity, because it explains both why they are so useful and why they cannot be trusted blindly. This guide explains what is going on, where the risk is highest, and what a practical verification routine looks like.

The short answer: why does AI make things up?

AI chatbots make things up because they are built to produce plausible text, not to look up verified facts. A large language model predicts which words are likely to come next based on patterns learned from huge amounts of writing. When the pattern points toward a real fact, the answer is correct. When it does not, the model still produces fluent text that fits the pattern, and that text can be false.

The model has no built-in alarm that says "I do not actually know this." Unless it has been trained and equipped to stop and check, it keeps going, and the result sounds just as confident as a true answer.

What "hallucination" actually means

The word is borrowed from human psychology, and it is a little misleading. A person who hallucinates perceives something that is not there. A language model does not perceive anything. A better description is confabulation: filling a gap in knowledge with a convincing story, without any intent to deceive.

In practice, the term covers several different failures. Separating them helps, because they have different causes and different fixes.

Fabricated facts

The model states something specific that is simply false: a date, a statistic, a quote, a product feature, a historical event. The details are often plausible. A made-up statistic usually looks like a real one.

Fabricated sources

This is the most notorious type. The model invents a book, a journal article, a court case, a web page, or an expert, complete with a realistic title and author. Citations are especially prone to this because a citation is a pattern: author name, year, title in a certain style, journal name. A model can produce the pattern without any real source behind it.

Blended facts

The model mixes true details from two or more real things into a single false one. It might attribute one author's idea to another, combine two companies' histories, or merge two people with similar names. Each piece is real. The combination is not.

Faithfulness failures

A user gives the model a document and asks for a summary, and the summary includes claims that are not in the document. Researchers describe this as being "unfaithful" to the source. It is a different problem from inventing facts out of thin air, because the correct answer was right there in front of the model.

Reasoning errors

The model gets the facts right and the logic wrong. It botches arithmetic, loses track of a condition in a long problem, or reaches a conclusion that does not follow from its own steps. This is not really about memory at all, but it is often grouped with hallucination because the output is confidently wrong.

Tool and action errors

When a model is connected to tools, such as a search engine, a calculator, or a set of files, it can also misreport what the tool returned or claim to have run a step it never ran. This matters more as people use AI agents to do multi-step work.

Why it happens: the root causes

Understanding hallucination requires a rough picture of what a language model is. A related explainer covers the mechanics in how large language models work, but the key ideas are short enough to repeat here.

1. The model is trained to continue text, not to be true

A language model learns by reading enormous amounts of text and getting very good at predicting what comes next. Truth is a side effect. If most of the writing the model saw says that Paris is the capital of France, then "Paris" is the likely continuation, and the model is right. But the objective was never "say true things." It was "say likely things." Most of the time those overlap. At the edges, they do not.

Consider a student who has read thousands of essays and learned to write in the style of an expert, but who never learned to say "I am not sure." Asked a question the student half knows, the student writes a smooth, authoritative-sounding paragraph anyway.

2. Memory is compressed and lossy

A model does not store its training text like a library. It compresses patterns into billions of numerical settings called parameters. Common facts, repeated thousands of times across the internet, are stored robustly. Rare facts, such as the middle name of a minor historical figure, the exact page count of an obscure book, or the specifics of a small local business, are stored weakly or not at all.

When the model reaches for a weakly stored fact, it does not return an error. It reconstructs something that fits. That reconstruction is where many invented details come from. This is why models tend to be more reliable on famous topics and less reliable on niche ones.

3. Fluency hides uncertainty

Humans are inclined to trust smooth, confident language. A model's writing is almost always smooth. A wrong answer and a right answer arrive in the same polished tone, with the same formatting and the same helpful phrasing. There is no hesitation, no trailing off, no visible sign of doubt. The reader has to supply the skepticism.

4. Training rewards helpful-sounding answers

After the initial training, most chat models are tuned with human feedback so that they are more helpful, polite, and responsive. That tuning is valuable, but it creates a pull toward giving an answer rather than admitting a gap. If people rating responses prefer a complete, confident answer over "I do not know," the model learns that completeness pays off. Developers work to counter this, and newer systems are generally better at declining, but the pull does not disappear.

5. The question itself pushes toward invention

Some prompts practically invite fabrication:

• False premises. "Why did the author include the scene in chapter nine where the dragon dies?" If there is no such scene, a model may explain it anyway, because the question assumed it existed.
• Demands for specifics. "Give me five peer-reviewed studies that prove this." If five do not come to mind, the model may produce five that fit the shape.
• Leading questions. When a question signals the desired answer, many models lean toward agreeing. This tendency is often called sycophancy.
• Requests beyond the model's knowledge. Questions about events after the training period, private information, or very recent changes invite guessing.

6. Long contexts and long answers drift

The longer a response runs, the more chances for an early mistake to become the foundation for later ones. Each new sentence is generated partly based on what the model already wrote, including its own errors. A small invented detail in paragraph two becomes an established "fact" by paragraph six. Long documents and long conversations also strain the model's attention, so details can be dropped or distorted. A related explainer on prompting that works covers how to structure requests to limit this drift.

7. Gaps and conflicts in the training data

The internet contains errors, myths, outdated information, satire, and arguments. A model trained on it absorbs all of that. If a common misconception appears often, the model may repeat it. If sources disagree, the model may blend them. A model cannot be more reliable than the mix of material it learned from.

Where hallucinations are most likely

Knowing the danger zones lowers the risk considerably. Hallucinations cluster in predictable places.

The pattern is simple. The more specific, obscure, recent, or checkable the claim, the more care it deserves. The more the task is about language itself, the safer it tends to be.

Real-world examples of how it shows up

These are common kinds of failures, described generally so they are easy to recognize.

The invented reference. Someone asks for sources on a topic and receives a tidy list. A few exist. Others have real authors paired with titles they never wrote. A few do not exist at all. Pasting the list into a report without checking is risky.

The confident wrong feature. A user asks how to do something in a particular software product. The model describes a menu option or setting that sounds natural but does not exist in that product. The steps may work in a competing app, or they would work if the product were designed the way the model assumes.

The phantom function. A developer asks for code that uses a library. The model calls a function that has a perfectly reasonable name and signature but is not in the library. The code fails on the first run, which is at least a loud failure.

The summary that adds a flourish. A user pastes a long article and asks for a summary. The summary includes a conclusion the article never reached, because the conclusion seemed like a natural way to end.

The local business that is not quite real. A request for a list of businesses in a small town can return a mix of real names, plausible names, and details like hours or phone numbers that belong to somebody else. For this reason, facts about a business are best taken from the business itself.

The worldbuilding contradiction. In long fantasy projects, a model can forget its own earlier decisions and invent new ones that contradict them: a river that changes direction, a king who has two different fathers. That is a kind of hallucination relative to the project's own canon. A related explainer on AI worldbuilding covers managing it, and it is a major reason writers keep a separate reference sheet for any invented world.

How to spot a hallucination

Not everything can be caught by eye, but there are reliable warning signs.

Red flags in the content

• Suspiciously perfect specifics. An exact percentage, an exact date, or a quote with precise wording, all with no source attached.
• Citations that cannot be found. If a quick search for the exact title returns nothing, treat it as invented until proven otherwise.
• Details that are too convenient. A study that happens to say exactly what the reader hoped it would say.
• Unusual confidence about obscure topics. If a model seems to know an enormous amount about something poorly documented, caution is warranted.
• Generic filler dressed as detail. Vague claims phrased with authority often signal that nothing specific lies underneath.
• Internal inconsistency. Names, numbers, or timelines that shift within a single answer.
• Hedging in the wrong place. Sometimes a model is careful about easy facts and breathless about hard ones. Tone does not reliably track reliability.

Tests that take a minute

1. Ask the same question twice. Open a fresh conversation and ask again, or regenerate the response. If key facts change between runs, the model is guessing. Stable answers are not proof of truth, but unstable ones are a clear warning.
2. Ask for the source, then check it independently. A model's claim that a source exists is not evidence that it does. Search for the exact title or find the page.
3. Ask the model to flag its own uncertainty. A prompt like "Mark any claim you are not confident about" does not make the model perfect, but it often surfaces shaky items. The result is a hint, not a guarantee.
4. Ask a question with a known answer. Testing the model on a familiar area of expertise shows something about how it performs elsewhere.
5. Search one distinctive detail. Not everything needs verification. Pick the most specific, most important claim and check that one. If it fails, discount the rest.

How to verify: a practical routine

The goal is not to distrust everything, which would waste the benefit of using AI at all. The goal is to match effort to the stakes.

Step 1: Sort the claims by consequence

Before checking anything, decide how much a mistake would cost.

• Low stakes. A name suggestion, a rewrite of one's own paragraph, a list of brainstormed ideas. Skim and move on.
• Medium stakes. A fact that will be repeated publicly, a number in a client document, a how-to that will be followed. Verify the key claims.
• High stakes. Legal, medical, financial, safety, or anything that could harm someone or damage a reputation. Treat the AI as a starting point for questions to bring to a qualified professional and to primary sources, never as the final word.

Step 2: Go to the primary source

For any claim that matters, find where it originally came from.

• For statistics, find the original report or dataset, not another article that repeats the number.
• For laws and regulations, use the official government or court website.
• For product features, use the vendor's own documentation.
• For scientific claims, find the study itself and read the abstract at minimum.
• For quotes, find the original text or recording.
• For business details, use the business's own site, listing, or phone number.

Step 3: Use a second, independent source

If a claim shows up in two unrelated, reputable places, confidence goes up. Sources that all trace back to the same origin deserve caution. Ten articles quoting one press release are one source, not ten.

Step 4: Check the date

Many wrong answers were once right. It is worth asking when the information was published and whether it could have changed. Prices, features, rules, and rankings are the usual offenders. When a fact is time-sensitive, consult the live page.

Step 5: Run code and calculations

Code a model produces can be run. Math that matters can be redone with a calculator or spreadsheet. Executable output is the one area that offers fast, objective feedback. A model that can run code itself, through a built-in tool, is generally more reliable on arithmetic than one that is only generating text.

Step 6: Keep a verified-facts file

For recurring projects, a short document of confirmed facts can be pasted into the conversation as context. This is a useful habit for long projects, from articles with specific training claims to puzzle sites with fixed rules, because it replaces the model's memory with a checked record.

How to reduce hallucinations when prompting

Wording alone cannot eliminate fabrication, but it can shift the odds considerably.

Provide the source material

The most effective fix is to hand the model the facts. Pasting the document, the data, or the policy, and instructing the model to answer only from that material, works well. Models are much better at working with text in front of them than at recalling facts from training. An example prompt:

Answer using only the text below. If the answer is not in the text, say "Not stated in the provided text." Quote the sentence that supports each claim.

The quote requirement is useful. It forces the model to point to something checkable.

Give explicit permission to say "I do not know"

Models often behave as though refusing is failure. Stating that it is acceptable helps. An example prompt:

If you are not sure, say so. I would rather have a gap than a guess.

This does not fix everything, but it does move the balance.

Ask for reasoning, then check the reasoning

For logic and math, asking the model to work step by step often improves accuracy and provides something to inspect. Many current systems have a dedicated "thinking" or reasoning mode for harder problems. These modes tend to help with multi-step reasoning, though they do not make factual recall perfect. The live feature list of whichever tool is in use shows what is available.

Break big requests into smaller ones

Instead of asking for a twenty-section report in one go, a user can ask for an outline, review it, then build sections one at a time. Smaller steps mean fewer chances for drift and provide checkpoints for catching errors.

Avoid leading and loaded questions

"Why is X better than Y?" invites the model to build a case for X, even if X is not better. A neutral version works better: "Compare X and Y. What are the strengths and weaknesses of each, and when would someone choose one over the other?"

Ask for sources only when they will be verified

Asking for citations is fine as long as they are treated as leads to check. A fabricated citation that goes unchecked is worse than no citation.

Separate generating from fact-checking

One useful pattern is a first pass for drafting and a second, separate pass for review: "List every factual claim in this text and rate how easily it can be verified." The second pass often catches issues the first pass created. The review can even run in a different tool, since two systems rarely invent the same false detail.

Tools that help: search, retrieval, and grounding

Much of the progress against hallucination comes from changing the setup around the model rather than the model itself.

Web search and live retrieval

Many chat tools can search the web and base answers on what they find. This helps with recent events and niche facts, and it lets the tool show links. It is not a cure. The model can still misread a page, pull from a low-quality site, or summarize something the page did not say. The upside is that a reader can click through. Tools built around this idea, such as Perplexity, put citations at the center of the experience. Other major assistants, including ChatGPT, Claude, and Gemini, offer search features of their own. Availability varies by plan and over time, so the current settings in each app are the best guide.

Retrieval-augmented generation

Behind the scenes, many business tools use a pattern called retrieval-augmented generation, or RAG, which was introduced in an academic research paper by Lewis and colleagues. The system first searches a trusted collection of documents, such as a company handbook or a product catalog, and then asks the model to answer from the retrieved passages. When well built, RAG sharply reduces invented answers because the model is working from approved text. When badly built, it fails quietly: the search step pulls the wrong passage, and the model answers confidently from it.

Personal documents

Most major assistants allow file uploads or projects with reference material. Using these features turns open-ended recall into a reading task. For client work, that is almost always the sensible default.

Code execution and calculators

When a model can run code to do arithmetic, parse a spreadsheet, or check a result, it stops guessing at things a computer can compute exactly. Anyone working with numbers benefits from a tool with this capability, as long as it is switched on.

Citations as a feature, not a guarantee

A cited answer is easier to check than an uncited one, and that is a real advantage. But a citation only indicates where the tool says it found something. Opening the link shows whether the page actually supports the sentence it is attached to. Sometimes it supports part of it. Sometimes it supports nothing.

Will hallucinations ever go away?

Probably not entirely, and the reason is worth understanding.

Developers are making real progress. Newer systems tend to hallucinate less on common tasks, decline more often when they lack information, use search and tools more intelligently, and show their sources. Training methods increasingly reward calibrated answers over confident ones. These are meaningful gains, and the trend is toward improvement.

But the core mechanism, generating likely text, remains the foundation. As long as a system produces language by prediction, there will be some chance of a fluent, wrong statement. There is also a trade-off: a model tuned to never guess will refuse more often and be less useful for creative and exploratory work, where "making things up" is exactly the point.

That last point matters. Fabrication is a bug when facts are wanted and a feature when fiction is wanted. The same ability that invents a fake citation also invents a fantasy kingdom, a puzzle theme, or a product name. For worldbuilding, game design, and brainstorming, the model's imagination is the product. The risk lies in forgetting which mode is in use.

The practical conclusion is to build verification into the workflow permanently rather than waiting for a "fixed" version of AI. A reasonable mental model is a fast, well-read, occasionally overconfident collaborator who benefits from a fact-checker.

How different kinds of projects handle it

The right level of checking depends on the kind of work. Several common project types show how the rules differ.

• Browser games and puzzle sites. Game rules, scoring, and word lists have to be exactly right. AI can draft code and ideas, but everything needs testing by running it. A daily puzzle with an invalid answer is a bug players notice immediately, so generated word lists and puzzle solutions call for validation. A related explainer covers this in AI puzzles and brain games.
• Long-form health and fitness content. Training and nutrition claims need real sources. AI is useful for structure, clarity, and editing, but factual claims belong against primary research and human review before publishing.
• Tutoring resources. Educational material should not contain errors that a young learner will absorb as truth. Every fact and every worked example deserves checking. A related explainer on AI tutors for gifted kids looks at this from the parent side.
• Local business websites. Hours, services, service areas, licensing, and prices come from the owner. A model can polish the wording, but it should not supply the facts.
• Fantasy worldbuilding. A canon document works as the authority, and the model acts as a collaborator expected to follow it.

The common thread is simple: AI drafts, humans verify, and anything checkable gets checked.

Quick takeaways

• AI makes things up because it predicts plausible text; it does not look up verified facts unless it is connected to tools that do.
• Hallucination covers several failures: invented facts, invented sources, blended facts, unfaithful summaries, reasoning errors, and tool errors.
• Risk is highest for citations, exact numbers, obscure subjects, recent events, and legal or medical specifics.
• Fluent, confident tone is not evidence of accuracy.
• Asking "are you sure?" is not a real check. Verification means consulting outside sources.
• Supplying source material and permission to say "I do not know" improves results.
• Checking should scale with the stakes: skim low-risk work, verify medium-risk claims, and rely on qualified professionals for high-risk decisions.
• Search tools and citations help, but clicking through confirms whether the page supports the claim.
• Fabrication is a flaw for facts and a strength for fiction. Knowing which mode is in use matters.
• Verification belongs in the routine instead of waiting for a perfect model.

Frequently asked questions

Is hallucination the same as lying?

No. Lying requires intent to deceive, and a language model has no beliefs or intentions in that sense. It generates text that fits patterns. When that text is false, the cause is a mismatch between what is plausible and what is true, not a decision to mislead. The effect on the reader can be the same, though, which is why verification matters regardless of the cause.

Which AI hallucinates the least?

It depends on the task, the version, and the settings, and the answer changes over time, so any ranking would go stale. Tools that search the web and show citations generally make claims easier to check, and reasoning modes often help with logic. A sound approach is to test candidates on questions from a familiar field, where the answers are already known, and to compare several tools on the same prompts. Reviews of ChatGPT, Claude, Gemini, and Perplexity explain how each is built and how to test it.

Can an AI that provides sources be trusted?

It deserves more trust than one that does not, but verification still applies. A source link means the tool believes it found support, not that the page says what the tool claims. Opening the link and reading the relevant passage is the check. The quality of the source also matters: a confident answer built on a weak website is still weak.

Why does AI invent fake citations so often?

A citation is a highly structured pattern: author, year, title, journal, and sometimes page numbers. The model has seen millions of them and can reproduce the form easily. What it cannot do, without search or a document in front of it, is reliably recall which specific combinations are real. So it produces ones that look right. Anyone who needs real references should use a tool that searches the web or a database, and check every one.

Does a longer or more detailed prompt prevent hallucinations?

Not by itself. Length does not matter much, but content does. Prompts that supply source material, set clear constraints, allow "I do not know," and break the task into steps tend to improve reliability. Prompts that pile on demands for specifics without providing facts can make things worse. A related explainer on prompting that works describes patterns that help.

Are image and video AI tools affected too?

Yes, in their own way. Image generators can produce garbled text, extra fingers, impossible objects, or scenes that look real but depict events that never happened. Synthetic images and audio can also be used to mislead. The same principle applies: a generated image is not evidence of anything. Image tools are compared in AI image generators compared.

Is it safe to use AI for medical, legal, or financial questions?

It can help with understanding terms, preparing questions, and organizing thinking. It should not be the final source for decisions. The details in these fields depend on jurisdiction, individual circumstances, and current rules, and mistakes carry real costs. A sensible approach is to use AI to get oriented, then confirm with official sources and a qualified professional. It is also wise to be thoughtful about which personal details are shared, a topic covered in what not to tell a chatbot.

What should happen when an AI is caught making something up?

Arguing with it is not productive. A better approach is to correct the fact, paste in the right source, and ask the model to continue from that. If the error appeared early in a long conversation, starting a new chat with a cleaned-up summary can help, since the wrong detail can keep influencing later answers. Tools with a feedback button give users a way to report the error, which helps developers improve the system.