What Are Hallucinations of Large Models

Hallucinations of large models refer to the phenomenon in the field of artificial intelligence, especially in large language models, where the content generated by the model is inconsistent with real-world facts or the instructions input by the user. This kind of hallucination can be divided into factual hallucination and faithfulness hallucination: the former refers to generated content that does not match verifiable facts, while the latter refers to content that does not match user instructions or context. This phenomenon may be caused by data defects, insufficient training, or model architecture problems, leading the model to output inaccurate or unreliable information.

Hallucinations of large models refer to the phenomenon in which the content generated by a model is inconsistent with real-world facts or user input.

The Working Principles of Hallucinations of Large Models

Hallucinations in large language models originate from data compression and inconsistency. During training, the model needs to process and compress large amounts of data, and this compression leads to the loss of information, causing the model to possibly "fill in the blanks" when generating responses and produce content inconsistent with real-world facts. Problems with the quality of pretraining data can also lead to hallucinations. There may be outdated, inaccurate, or missing key information in the dataset, causing the model to learn incorrect information. During the training phase, the model uses real tokens as input, while during the inference phase, the model relies on its own generated tokens for subsequent predictions. This inconsistency may lead to hallucinations.

Large models predict the next token based on the previous token, only from left to right. This unidirectional modeling limits the ability to capture complex contextual dependencies and may increase the risk of hallucinations. The Softmax operation in the model's final output layer limits the expressive power of the output probability distribution, preventing the language model from outputting the desired distribution and leading to hallucination problems. During inference, randomness introduced through techniques such as temperature, top k, and top b may lead to hallucinations. When processing long texts, the model pays more attention to local information and lacks attention to global information, which may lead to situations such as forgetting instructions or failing to follow instructions, producing hallucinations. When generating responses, the model is uncertain about the meaning of its output. This uncertainty can be measured by predictive entropy; the higher the predictive entropy, the more uncertain the model is about the possible output. These factors work together to cause the hallucination phenomenon that may occur when large models generate content, namely generating descriptions that seem plausible but actually do not conform to known common sense.

Main Applications of Hallucinations of Large Models

Text summary generation: In text summary generation tasks, large models may produce summaries inconsistent with the content of the original document. They may incorrectly summarize the time when an event occurred or the people involved, causing the summary information to be distorted.
Dialogue generation: In dialogue systems, the hallucination problem of large models may lead to responses that contradict the dialogue history or external facts. They may introduce nonexistent people or events into the conversation, or provide incorrect information when answering questions.
Machine translation: In machine translation tasks, large models may produce translations inconsistent with the content of the original text. Information not present in the original text is added during translation, or important content is omitted.
Data-to-text generation: In data-to-text generation tasks, large models may produce text inconsistent with the input data. Information not present in the data is added when generating text, or key facts in the data are not accurately reflected.
Open-ended language generation: In open-ended language generation tasks, large models may produce content that does not align with real-world knowledge.

Challenges Faced by Large Model Hallucinations

Data quality issues: The text generated by the model may contain inaccurate or false information, such as producing content inconsistent with the original text in summarization generation. In dialogue systems, it may cause the model to provide incorrect suggestions or answers.
Challenges during training: The model may over-rely on certain patterns when generating text, such as positional proximity or co-occurrence statistics, resulting in output inconsistent with actual facts. In tasks requiring complex reasoning, the model may fail to provide accurate answers.
Randomness during inference: This causes the model output to deviate from the original context, such as producing a translation inconsistent with the source text in machine translation. In long-text generation tasks, it may lead to inconsistency between earlier and later information.
Legal and ethical risks: In applications in high-risk scenarios, such as judicial trials, medical diagnosis, etc., model hallucinations may lead to serious consequences. Users may lack vigilance toward the model's output, leading to misplaced trust in erroneous information.
Challenges in evaluating and mitigating hallucinations: Insufficient evaluation methods may lead to misjudgment of model performance, affecting model optimization and improvement. Insufficient mitigation strategies may cause the model to still produce hallucinations in practical applications, affecting user experience and the model's credibility.
Limited breadth of application: The hallucination problem of models limits their application in multiple domains, especially in domains requiring high accuracy. Domain specialization may cause models to produce more hallucinations when facing cross-domain tasks, affecting their breadth of application.
System performance issues: The performance issues of models may cause users to lose confidence in them, affecting their competitiveness in the market. Reduced credibility may limit the model's application in critical tasks, such as in financial analysis or policy making.

Development Prospects of Large Model Hallucinations

With the continuous development of deep learning technology, especially the optimization of pretrained models such as Transformer, large language models (LLM) have shown strong potential in comprehension and creativity. Research on the large model hallucination problem is not limited to natural language processing but has also expanded to multimodal fields such as image captioning and visual storytelling, showing broad application prospects. Researchers are exploring more effective methods to evaluate and mitigate hallucinations and improve the credibility and reliability of models. As large models are applied in high-risk fields such as healthcare and justice, the legal and ethical risks caused by their hallucination problems are also receiving increasing attention, which will promote the establishment of relevant regulations and ethical guidelines. Solving the large model hallucination problem requires cooperation across multiple fields such as natural language processing, knowledge graphs, and machine learning, and more interdisciplinary research and solutions are expected in the future. Solving the large model hallucination problem requires the joint efforts of the entire industry, including data providers, model developers, application developers, etc., to jointly promote the healthy development of artificial intelligence technology.