Chain-of-Draft (CoD) is a new AI reasoning paradigm that improves reasoning efficiency through concise intermediate reasoning steps. It mimics the concise thinking of humans when solving problems, limiting the key information output at each step to no more than five words. Compared with traditionalChain-of-Thought(CoT), Chain-of-Draft greatly reduces Token usage, significantly lowering reasoning cost and latency while maintaining high accuracy. In a variety of reasoning tasks (such as arithmetic, commonsense, and symbolic reasoning), Chain-of-Draft performs excellently and is suitable for real-time AI applications, resource-constrained environments, and cost-sensitive scenarios.
What is Chain-of-Draft
Chain-of-Draft (CoD) is a new AI reasoning paradigm proposed by Zoom's research team, which improves reasoning efficiency and saves cost by imitating the concise thinking process of humans. It was inspired by humans relying on drafts or shorthand to capture key insights when solving problems. Compared with traditional Chain-of-Thought (CoT), Chain-of-Draft encourages the model to generate extremely concise intermediate reasoning steps, capturing only the key information for solving the problem.
How Chain-of-Draft Works
The working principle of Chain-of-Draft (CoD) is to imitate the concise way of thinking humans use when solving problems, allowing large language models (LLMs) to generate extremely concise but informative intermediate steps during reasoning. Specifically, the CoD strategy requires the model to limit the number of words used in each reasoning step, usually no more than five words. This method is not a mandatory restriction, but a guiding suggestion that promotes concise reasoning steps.
CoD encourages the model to generate minimal yet informative intermediate results at each reasoning step, focusing on the key calculation or transformation steps. By reducing verbose output, CoD significantly lowers Token usage and reduces the overall output length and latency. By reducing Token usage, CoD can still maintain accuracy comparable to traditional Chain-of-Thought (CoT) in a variety of reasoning tasks (such as arithmetic reasoning, commonsense reasoning, and symbolic reasoning).
Main Applications of Chain-of-Draft
- Real-time customer support: In the field of real-time customer support, response speed is crucial to user experience. Chain-of-Draft significantly reduces latency by reducing the number of Tokens in the reasoning process, enabling AI to provide solutions more quickly.
- Education and learning assistance: In the field of education, Chain-of-Draft can be used to provide quick answers to academic questions. For example, in mathematics education, CoD can quickly show the key steps to solving a problem, rather than a detailed reasoning process, helping students understand the core of the problem more quickly.
- Conversational AI systems: Conversational AI systems, such as chatbots, need to quickly and accurately understand user intent and respond. By generating concise intermediate reasoning steps, Chain-of-Draft enables AI systems to process user input and generate responses more quickly.
- Large-scale AI deployment: In scenarios requiring large-scale deployment of AI models, such as cloud computing services, cost control is an important consideration. By reducing Token usage, Chain-of-Draft significantly lowers the cost of reasoning tasks.
- Resource-constrained environments: In resource-constrained environments, such as mobile devices or edge computing devices, computing power and storage space may be very limited. By generating extremely concise but informative intermediate reasoning output, Chain-of-Draft enables AI models to run on these devices with lower resource consumption.
- Summarizing and extracting key information: In scenarios that require quickly extracting key information from large amounts of text, such as news summarization or rapid reading of research reports, Chain of Draft can effectively distill the core points. By limiting the number of words in each reasoning step, CoD helps users quickly grasp the main content of the text without deeply reading all the details.
- High-frequency financial trading: In the financial field, especially in high-frequency trading, decision speed is crucial to success. Chain of Draft reduces reasoning latency, enabling AI systems to analyze market data and make trading decisions faster.
- Autonomous driving decisions: Autonomous driving systems need to make complex decisions in an extremely short time. By providing fast and accurate reasoning capabilities, Chain of Draft helps autonomous driving systems respond quickly when facing complex traffic situations.
Challenges faced by Chain of Draft
- Performance degradation in zero-shot settings: In zero-shot settings where no few-shot examples are provided, the performance of Chain of Draft drops significantly. This may be because the training data of large language models lacks CoD-style reasoning patterns, making it difficult to generate concise and insightful “drafts” without sample guidance.
- Performance gap on small models: When CoD was tested on small language models with fewer than 3B parameters, although CoD can reduce the number of Tokens required for each response and improve accuracy, the performance gap compared with CoT is more obvious.
- Limitations in complex tasks: Chain of Draft may not be suitable for complex tasks that require extensive reflection, self-correction, or external knowledge retrieval. In these cases, the conciseness of CoD may limit the model's reasoning depth and accuracy.
- Balancing reasoning depth and conciseness: Although CoD reduces latency and computational cost by reducing redundancy and focusing on key insights, this conciseness may sacrifice reasoning depth. In some cases, detailed intermediate steps are essential for understanding and verifying the reasoning process. CoD's minimalist approach may make the reasoning process less transparent and make it difficult to trace and understand the model's thinking path.
- Trade-off between cost and performance: Although CoD performs excellently in reducing costs, in some application scenarios with high performance requirements, the cost advantage may come at the expense of accuracy. For example, in financial analysis or medical diagnosis, where extremely high accuracy is required, the performance of CoD may not be sufficient to meet the needs.
- Model adaptability: CoD requires the model to limit the number of words used in each reasoning step, and this restriction may not be applicable to all types of models. Different models may require different strategies to adapt to CoD's requirements, which may increase the complexity of model training and tuning.
- Challenges of real-time applications: Although CoD significantly reduces latency by reducing Token usage, in real-time applications such as autonomous driving or high-frequency trading, even a tiny delay may affect the quality of decisions. Therefore, CoD needs to ensure the accuracy and reliability of reasoning while maintaining low latency.
The development prospects of Chain of Draft
Chain-of-Draft (CoD), as an emerging AI reasoning paradigm, has very broad development prospects. By imitating the concise thinking process of humans, CoD decomposes complex reasoning tasks into concise and information-dense intermediate steps, significantly reducing token usage and inference latency. The efficiency and cost-effectiveness of CoD have significant application potential in multiple fields. For enterprises processing 1 million inference queries per month, using CoD can reduce costs from $3,800 to $760. This cost advantage is especially prominent in latency-sensitive scenarios such as real-time customer support, education, and financial services. CoD is simple to implement, requiring only a simple modification to the prompts of existing models to switch, and has high feasibility in large-scale deployment. Overall, CoD provides a more efficient and more economical solution for AI reasoning and is expected to be more widely promoted and applied in future AI applications. With the continuous advancement and optimization of technology, CoD is expected to overcome its current limitations and further improve its performance and applicability.