What Is CAR
CAR (Certainty-based Adaptive Reasoning) is an adaptive reasoning framework jointly launched by ByteDance and Fudan University. It can improve the reasoning efficiency and accuracy of large language models (LLMs) and multimodal large language models (MLLMs) across different tasks. The framework is based on dynamically switching between short answers and long-form reasoning, deciding whether detailed reasoning is needed based on the model's confidence in the answer (measured by perplexity, PPL). When the model's confidence in a short answer is low, CAR triggers long-form reasoning to improve the accuracy of the answer; when confidence is high, it directly outputs the short answer, saving computational resources and time. CAR performs outstandingly in tasks such as visual question answering (VQA) and key information extraction (KIE), and demonstrates good performance in complex reasoning tasks such as mathematics.
CAR's Main Features
- Dynamic Reasoning Switching: CAR can intelligently switch between short answers and long-form reasoning. For simple questions, it directly outputs short answers to improve efficiency; for complex questions, it triggers long-form reasoning to ensure accuracy.
- Improved Reasoning Efficiency: CAR significantly reduces the number of tokens generated by the model, lowers computational costs and reasoning time, and improves the model's efficiency in practical applications.
- Improved Reasoning Accuracy: In cases requiring detailed reasoning, it activates long-form reasoning, improving the model's performance on complex tasks and ensuring the accuracy of reasoning results.
- Adaptability to Multiple Tasks: CAR is applicable to tasks such as visual question answering (VQA) and key information extraction (KIE), and plays a role in complex tasks such as mathematical reasoning and commonsense reasoning, with broad applicability.
CAR's Technical Principles
- Model Training: Train the LLM or MLLM with training data containing short answers and long-form reasoning answers. During training, the model needs to learn how to generate corresponding short answers or long-form reasoning answers based on different prompts. The optimization objective is the cross-entropy loss, training the model by minimizing the difference between the probability distribution of predicted tokens and the distribution of true tokens.
- Obtaining the PPL of Short Answers: Perform short-answer reasoning on the training data and calculate the perplexity (PPL) of each short answer. PPL is a metric measuring the model's confidence in the answer; the lower the PPL, the higher the model's confidence in the answer.
- Gaussian Distribution Modeling: Assume that the PPL scores of correct and incorrect short answers follow Gaussian distributions. Estimate the distribution parameters based on the training data, calculate the PPL means and variances of correct and incorrect answers respectively, and obtain two Gaussian distribution models.
- Inference Process: For new input, first generate a short answer and calculate the PPL value. Based on the Gaussian distribution models, calculate the probability that the answer is correct under that PPL value.
CAR's Project Address
- arXiv Technical Paper：https://arxiv.org/pdf/2505.15154
CAR's Application Scenarios
- Visual Question Answering (VQA): CAR directly outputs short answers for simple questions to improve efficiency; for complex questions, it triggers long-form reasoning to ensure accuracy.
- Key Information Extraction (KIE): Dynamically select the reasoning method based on confidence, reduce token usage, and maintain high accuracy.
- Mathematical Reasoning: Directly output answers for simple math problems, and perform detailed reasoning for complex problems, improving accuracy and efficiency.
- Commonsense Reasoning: For simple common-sense questions, give the answer directly; for complex questions, perform long-form reasoning to reduce token consumption.
- Multimodal tasks: Combine text and image information, dynamically select the reasoning method, and improve the accuracy and efficiency of multimodal tasks.