What is Roubao
Roubao is an open-source AI phone automation assistant, serving as a free alternative to ByteDance's "Doubao Phone Assistant". Roubao is based on a vision-language model (VLM) and developed natively in Kotlin. It requires no computer or root access, and supports obtaining system-level control capabilities through Shizuku. Roubao's core uses a "screenshot-AI analysis-execution" loop architecture, supports two modes: Delegation (delegating to AI apps) and GUI automation, and can complete tasks such as ordering takeout, sending messages, and navigation.
Roubao's main features
- Intelligent AI Agent: Roubao can understand screen content based on a vision-language model, operate the phone through natural language instructions, and automatically plan and execute complex tasks.
- Refined UI design: Roubao adopts a carefully designed Material 3 interface, supports adaptive dark and light themes, and provides smooth animations and complete bilingual support in Chinese and English.
- Highly customizable: Roubao supports multiple vision-language models, including Alibaba Cloud Tongyi Qianwen, OpenAI GPT-4V, Claude, and more, and supports user-configured custom API endpoints and local models.
- Security protection mechanism: Roubao implements AES-256-GCM encrypted storage of API Keys, features an automatic detection and stop mechanism for sensitive pages, and makes the entire task execution process visible and interruptible manually at any time.
- Root mode extension: Roubao supports Root mode extension. When Shizuku runs with Root permissions, more system-level operation capabilities can be unlocked.
Roubao's technical principles
- Dual-layer Agent architecture: Roubao's core is a dual-layer Agent architecture, inspired by Claude Code and divided into a Tools layer and a Skills layer. The Tools layer encapsulates atomic capabilities such as app search, DeepLink jumping, clipboard operations, and other basic functions; the Skills layer is oriented toward user intent, mapping natural language to specific tasks and realizing the full chain from "ordering takeout" to executing operations.
- Loop-based automation process: Roubao adopts a screenshot-analysis-execution loop-based automation process, using Shizuku to call system-level commands to complete screen capture and simulated clicks. When receiving a user instruction, SkillManager first performs intent recognition. High-confidence tasks are directly delegated to professional AI apps through DeepLink, while low-confidence or complex tasks enter the standard Agent loop, with Manager planning, Executor deciding, and Reflector reflecting until completion.
- Native Android implementation: Roubao rewrites the MobileAgent framework in Kotlin, running entirely locally on the Android device without requiring a computer as an intermediary. Through Shizuku, it obtains ADB-level permissions to directly execute operations such as screenshots, clicks, and swipes, avoiding the cumbersome process of traditional Python solutions that require connecting to a computer.
Roubao's project address
- GitHub repository：https://github.com/Turbo1123/roubao
Roubao's application scenarios
- Daily task automation: Roubao can help users with daily operations such as ordering takeout, checking the weather, setting alarms, and sending messages. It only needs to hear the request to complete them automatically, without manually searching through apps.
- Cross-app information processing: Roubao can automatically take screenshots to recognize content, helping users send the last photo to Weibo, share a link to WeChat, or copy and paste information between different apps.
- Entertainment content acquisition: Roubao can automatically open a music app to play the daily recommendation, search for and play a specific video on Bilibili, or help users scroll short videos and watch popular content.
- Navigation and travel services: Roubao supports directly waking up Amap to search for routes, help users hail a taxi, or find location information for nearby restaurants and shops.