What is BitsAI-CR
BitsAI-CR is an automated code review tool based on large language models (LLM) launched by the ByteDance team. It achieves efficient code review through a two-stage processing pipeline: RuleChecker is responsible for detecting potential issues based on 219 rules, and ReviewFilter verifies the issues to improve precision. BitsAI-CR introduces the "Outdated Rate" metric to automatically evaluate developers' acceptance of review suggestions, and optimizes review rules through a data flywheel mechanism.
Main Features of BitsAI-CR
- High-Precision Code Review Comment Generation: BitsAI-CR achieves high-precision code review comment generation through a two-stage processing pipeline.
- Code Review Rule Classification: A comprehensive review rule classification system built on code style guides and practical review experience, providing foundational support for the system.
- Integration and Automation: BitsAI-CR can be easily integrated into existing code review processes, automatically identifying potential issues, confirming review categories, locating problematic code lines, and providing relevant comments. After developers modify code based on review suggestions, BitsAI-CR re-evaluates the code and marks the original comments as "outdated", while also providing "LGTM" (Looks Good To Me) approval.
Technical Principles of BitsAI-CR
- Two-Stage Review Pipeline：
- RuleChecker: Based on a comprehensive classification of 219 review rules, potential issues in code are detected through a fine-tuned LLM.
- ReviewFilter: After RuleChecker, another fine-tuned LLM is used to verify the detected issues, further improving review precision.
- Data Flywheel Mechanism: Large-scale industrial scenario optimization through real feedback.
- Outdated Rate Metric: Automatically measures the percentage of code lines modified after being flagged by BitsAI-CR, evaluating whether developers accepted and implemented the review suggestions.
- Dynamically Adjust Review Rules: Dynamically adjust review rules based on outdated rate and precision measurements, removing rules that generate low-value comments.
- Model Training and Optimization: Based on the LLM developed by ByteDance (Doubao-Pro-32K-0828), fine-tuned through low-rank adaptation (LoRA) technology to ensure a balance between model safety and performance. Trained on historical review data to optimize the model and improve review precision.
Project Address of BitsAI-CR
- arXiv Technical Paper：https://arxiv.org/pdf/2501.15134
Application Scenarios of BitsAI-CR
- Automated Code Review: BitsAI-CR automatically identifies potential issues in code through a two-stage review pipeline (RuleChecker and ReviewFilter), generates high-precision review comments, and locates problematic code lines.
- Continuous Optimization and Feedback Mechanism: BitsAI-CR introduces the "Outdated Rate" metric, dynamically adjusting review rules by automatically measuring developers' acceptance and modification of review suggestions.
- Multi-Language Support and Rule Classification: BitsAI-CR is based on a comprehensive review rule classification system and supports multiple programming languages (such as Go), with detailed review rules formulated for the characteristics and common issues of different languages.
- Integration and Automated Processes: BitsAI-CR can be seamlessly integrated into existing code review processes, automatically handling the entire process of code submission, review, modification, and approval.