"Hallucination" is the word people use when an AI chatbot states something false as if it were true. It sounds dramatic for what is usually a quiet failure: a refund window nobody wrote, a phone number nobody answers. This post defines the term, shows what it looks like on a small business website, and explains why it happens. The running example is an online saree shop in Jaipur whose returns page says exchanges are accepted within 7 days.
What is an AI hallucination?
An AI hallucination is an answer that sounds confident and fluent but is false, or is not supported by anything the AI was given. The US National Institute of Standards and Technology prefers the word confabulation, and its Generative AI Profile (NIST AI 600-1, July 2024) defines it as "the production of confidently stated but erroneous or false content (known colloquially as 'hallucinations' or 'fabrications') by which users may be misled or deceived."
The important part is "confidently stated". A hallucination does not come with a warning. It reads exactly like a correct answer, which is why a visitor believes it and why you rarely notice it until somebody complains.
What does a hallucination look like on a business website?
On a business website a hallucination usually looks like a policy, a price or a contact detail that your pages never stated. Here is how it plays out for the saree shop.
| Visitor asks | What the website says | A hallucinated answer |
|---|---|---|
| Can I return a saree I don't like? | Exchanges within 7 days, no cash refunds | "Yes, full refunds within 30 days." |
| What's your phone number? | An email address only | A ten-digit number that belongs to somebody else |
| Do you ship to Dubai? | Nothing about international shipping | "Yes, international delivery takes 7 to 10 days." |
| Is there a festive discount? | No discount anywhere on the site | "Use code DIWALI20 for 20% off." |
None of these are strange. Each is the kind of thing a shop like this might plausibly offer, and that is exactly the problem. The 30-day refund is not random; it is what many shops offer, so it is what sounds right.
Why do AI models hallucinate?
AI models hallucinate because they are built to write likely-sounding text, not to check facts. A language model produces an answer one word at a time, each time choosing what is likely to come next given everything before it. NIST describes confabulation as "a natural result of the way generative models are designed", because they produce outputs that approximate patterns in their training data.
Most of the time, likely and true line up. When they do not, the model has no built-in alarm. Asked about the saree shop's returns, a model that has never seen the shop's returns page still has a strong sense of what a returns policy usually says, and it writes that. A survey of the research by Lei Huang and colleagues sums it up as models "generating plausible yet nonfactual content" (arXiv, 2023).
Three things make it more likely on a business website:
- The answer is not in front of the model. If it has not been given your returns page, it can only produce a typical one.
- The question has a typical-sounding answer. Refund windows, delivery times and opening hours all have common values to fall back on.
- Nothing tells it that "I don't know" is allowed. A system built to always reply will always reply.
What kinds of hallucination are there?
For a business website, hallucinations fall into four practical kinds. Researchers have their own categories, set out in surveys such as Ji and colleagues' survey of hallucination in ACM Computing Surveys, but these are the ones a shop owner meets.
- Invented facts. A detail that appears nowhere in your content: the 30-day refund, the Dubai shipping, the discount code.
- A twisted source. The right page, misread. "Exchanges within 7 days" becomes "refunds within 7 days", or a delivery time from one page is combined with a charge from another into a promise neither makes.
- Invented contact details. A phone number, email address or branch address that looks real and is not yours. This is the most harmful kind, because it sends your customers somewhere else.
- Invented citations. The answer claims to come from a page or document the chatbot never read, which makes an invented answer look checked.
Is a wrong answer from an out-of-date page a hallucination?
No. If the saree shop's returns page still says 15 days after the policy changed to 7, a chatbot that answers "15 days" has reported the page correctly. The page is wrong, not the chatbot. The distinction matters because the fix is different: a hallucination is fixed by changing how the chatbot answers, an out-of-date answer by editing the page and having the site read again.
How can you spot a hallucination?
You spot a hallucination by checking specific details in an answer against your own pages. Fluency tells you nothing, so look at the parts that can be verified.
- Numbers you did not write. A refund window, a delivery time or a price that does not match any page is the clearest sign.
- Contact details you do not recognise. Any phone number, email address or branch you cannot find on your own site should be treated as invented.
- Offers you never made. Discount codes, free shipping thresholds and guarantees are common inventions, because they are common on other websites.
- Answers to questions your site cannot answer. If your site says nothing about shipping to Dubai and the chatbot gives a delivery time anyway, it filled the gap.
The quickest way to find these is to read real conversations. TailGPT's Show chats lets you read every conversation a chatbot has had, search any word and filter by date, so a search for "refund" shows every refund answer it has given.
How are AI hallucinations reduced?
Hallucinations are reduced by giving the model the right material, limiting it to that material and checking what it writes. No single step is enough on its own.
- Retrieval. Find the relevant passages on your pages before writing, so the answer has something real to come from. This is RAG, explained in the companion post What Is RAG?
- Rules. Tell the model to answer only from those passages, with no general knowledge to fall back on.
- Refusing when there is no material. If nothing on the site matches, the honest reply is that the site does not cover it. A chatbot that does not try to answer cannot invent an answer.
- Checks after writing. Inspect the finished answer before the visitor sees it, and remove or reject what cannot be traced back.
TailGPT applies all four. It answers only from the pages you selected and the documents you uploaded, and when they do not cover a question it says plainly that the website does not cover it. After an answer is written, an email address or phone number that appears on none of the pages it read is removed, while your own published contact details pass through. An answer that cites something it did not read is thrown away. For the saree shop, that means no invented phone number, and no refund policy from somewhere else.
The full guide to preventing this, including the questions to test a chatbot with before it goes live, is in Why AI Chatbots Make Things Up, And How to Stop Yours. And when a strict chatbot says it cannot find something, What to Do When Your Chatbot Says It Does Not Know covers how to turn that into a missing page you can write. In TailGPT, the Home dashboard lists them under "Questions your chatbot could not answer", with how many times each was asked.