Raluca Florescu

Assisted by ChatGPT.

AI hallucinations often appear when a model produces an answer too quickly, without sufficiently reasoning through or verifying the information. The answer may sound correct because it fits the immediate context of the chat, but contextual fit is not the same as factual accuracy.

One way to understand this is through an analogy with System 1 and System 2 thinking. In humans, System 1 is fast, automatic, and associative, while System 2 is slower, more deliberate, and more critical. AI models do not literally think like humans, but the analogy is useful. A model may sometimes respond in a System 1-like way: it follows tone, rhythm, familiar patterns, or implied expectations instead of carefully checking whether the answer is true.

This also happens in human reasoning. People often answer certain questions incorrectly because they follow the structure, rhythm, or expectation of the sentence rather than analyzing the actual information. If the question sounds familiar or creates a strong expected answer, the mind may complete the pattern before verifying the premise.

Over time, however, critical reasoning can become easier to integrate into faster responses. When an AI model makes a contextual mistake and that mistake is corrected, the model may become better at identifying the problem in a later attempt. It may refuse the false premise, correct the framing, or reformulate the answer more accurately, depending on how simple or clear the question is.

Vague questions, however, create a greater risk of vague or incorrect answers. If the context is not explained clearly, the model has less structure to work with. Its fast pattern-completion process may fill the missing information in a way that sounds plausible but is not necessarily correct. Even a reasoning model does not always apply careful critical analysis automatically. It can still fall into the trap of a vague or suggestive question and produce an answer that is incomplete, imprecise, or wrong.

Who is responsible for AI hallucinations?

The responsibility is not located only in the model or only in the user. The AI model responds based on the context provided by the user, but also based on its prior training. Many users interact with AI through fictional scenarios, roleplay, imagined memories, hypothetical situations, or parallel-universe prompts. In these contexts, the model is often expected to continue invented events or behave as if it remembers something that never actually happened.

This can encourage the model to answer through fictional continuity rather than factual verification. If the user asks the model to remember something from a roleplay, the model may continue the scene and respond as though the memory exists, even when it does not.

AI creativity also contributes to this problem. When a prompt is vague, incomplete, or emotionally suggestive, the model may try to fill the gaps. This can be useful in creative writing, but it becomes risky when the user expects an objective, logical, or factual answer.

In many cases, users do not clearly ask for an objective or verified response. Instead, they may use expressions such as hypothetically, imagine, or in a parallel universe similar to this one. These phrases guide the model toward speculation or fiction. From there, the conversational tone can become established for that chat, and sometimes even influence future interactions if the same style is repeatedly reinforced.

This is why precise prompting matters. If the user wants a more accurate answer, the question should clearly request objectivity, verification, logic, and critical reasoning. A well-formulated question gives the model a better chance of responding carefully instead of simply completing the conversational pattern.

When a chat becomes heavily loaded with previous context, it may also help to start a new chat or clearly state that a new topic is beginning. This can reduce the influence of the previous conversational tone and help the model respond from a cleaner context.

Important questions should be formulated with as much precision as possible. The more important the question is, the more carefully the user should define the context, the expected type of answer, and whether the response should be factual, logical, speculative, or creative.

Examples of Fast Pattern Completion in Human Reasoning

These examples show how people can answer incorrectly when they rely on fast pattern recognition instead of careful reasoning. The wrong answer usually feels natural because the question creates a familiar structure, a strong expectation, or a misleading focus.

1. The Moses Illusion

Question:
“How many animals of each sex did Moses take on the Ark?”

Common fast answer:
“Two.”

Correct answer:
Moses did not take animals on the Ark. The biblical figure associated with the Ark is Noah.

Explanation:
Many people answer “two” because the question contains familiar elements: animals, the Ark, and a biblical figure. The mind recognizes the general pattern and completes it quickly. However, careful reasoning reveals that the premise is wrong.

This example shows how a familiar context can produce a confident but incorrect answer.

2. The Bat and Ball Problem

Question:
“A bat and a ball cost $1.10 in total. The bat costs $1 more than the ball. How much does the ball cost?”

Common fast answer:
“10 cents.”

Correct answer:
The ball costs 5 cents.
The bat costs $1.05.
Together, they cost $1.10.

Explanation:
The answer “10 cents” feels immediately plausible because the mind quickly separates $1.10 into $1 and 10 cents. But if the ball cost 10 cents, the bat would cost $1.10, making the total $1.20. The correct answer requires slower, more careful reasoning.

This example shows how an intuitive answer can feel correct while still being logically wrong.

3. The Fifth Daughter Question

Question:
“Mary’s father has five daughters: Nana, Nene, Nini, Nono. What is the fifth daughter’s name?”

Common fast answer:
“Nunu.”

Correct answer:
Mary.

Explanation:
The repeated sequence of names creates a strong expectation. The mind tries to continue the pattern: Nana, Nene, Nini, Nono, Nunu. However, the answer is already contained in the question. It says “Mary’s father,” which means Mary is one of the daughters.

This example shows how rhythm and pattern can distract from the actual information.

4. The Plane Crash Question

Question:
“If a plane crashes on the border of the United States and Canada, where do they bury the survivors?”

Common fast answer:
“In the United States” or “in Canada.”

Correct answer:
They do not bury the survivors.

Explanation:
The question directs attention toward geography: the border between two countries. Because of that, many people focus on choosing a location. But the key word is “survivors.” Survivors are not buried.

This example shows how a question can guide attention toward the wrong part of the context.

Connection to AI Hallucinations

These examples are useful because they show that wrong answers do not always come from lack of knowledge. Sometimes they come from answering too quickly based on pattern, expectation, or context.

In the same way, AI hallucinations may occur when the model follows the apparent shape of the prompt instead of verifying the premise. The answer can sound natural, fluent, and contextually appropriate, while still being false.

This is why both humans and AI models need something equivalent to slower verification: checking the premise, identifying the actual question, separating fiction from fact, and refusing to continue a false assumption when necessary.