AI limits & hallucinations

Maybe you have experienced it yourself: you ask a chatbot for a source, you get a title, an author and a year, and none of it exists. Such invented details are called hallucinations. They are not a rare malfunction but a direct consequence of how language models work. Once you understand the mechanism, you also know when to be careful.

Why models hallucinate

A language model generates text by choosing the most plausible next word. Plausible means: fits the patterns of the training data. True means: matches the world. These are not the same thing. An invented study with a realistic title and a plausible author name is a good continuation as far as the model is concerned, because that is exactly what real citations look like. The model has no inner instance that checks whether it knows something or is seamlessly making it up. Newer systems hallucinate less often and can pull in web searches, but the problem has not disappeared.

Patterns instead of knowledge

At its core an AI model is made of numbers called weights that combine inputs into a decision. A single artificial neuron multiplies each input by its weight, adds the products into a weighted sum, and says yes as soon as that sum is above a set threshold. That alone is enough to separate apples from pears if the data forms a clear pattern. But that is exactly where the limit lies: the neuron knows neither apples nor pears, it just draws a line through numbers. When something arrives that does not fit the learned pattern, it still decides, and with full conviction. Large language models are billions of times more complex, but the principle stays the same: combining patterns, not checking knowledge.

Where AI is strong and where it is not

Language models are strong when you give them the material: summarising a text, rephrasing, translating, brainstorming, improving a draft, explaining code. There they work on your input and you spot mistakes quickly. It gets risky when the model is supposed to deliver facts from memory: exact numbers, quotes, legal clauses, names, current events or niche topics with little training data. The more specific the question and the more the exact answer matters, the higher the risk of hallucination. And beware: the model sounds just as confident when it invents an answer as when it is right.

Checking sources

Your best protection is you. Treat a chatbot's factual claims like a tip from a well read acquaintance: often right, never to be adopted unchecked when something depends on it. Concretely: look up names, numbers and quotes, actually open cited sources instead of trusting the title, and for important topics find a second independent source. By all means ask the model for its sources, but remember the citation itself can be hallucinated. Models with web search return real links, yet only you can see whether the link actually supports the claim by reading it.