Lego-Edit - Xiaomi's Open-Source Image Editing Framework

Main Features of Lego-Edit

• Powerful image editing capability: Lego-Edit can complete various complex image editing tasks according to user instructions, including but not limited to local editing, global editing, and multi-step editing, meeting image processing needs in different scenarios.
• Flexible instruction understanding and execution: With the generalization capability of multimodal large language models (MLLM), Lego-Edit can understand and execute open-domain instructions. Even when faced with unseen instructions, it can process them through its powerful reasoning capability.
• Efficient utilization of model-level toolkit: The framework includes multiple model-level tools efficiently trained on limited data. These tools have various image operation functions, and the MLLM can call these tools to complete fine-grained editing operations, achieving efficient and precise image modification.
• Integrate new tools without retraining: Lego-Edit supports integrating newly introduced editing tools without additional fine-tuning, making its functionality easy to extend and adapt to evolving image editing needs.
• Mask input supports precise editing: Accepting mask input is a major highlight of Lego-Edit. Users can precisely specify the image regions to be edited through masks, achieving more precise local editing effects.
• Open source and ease of use: Lego-Edit's code is open-sourced under the Apache 2.0 license, and the models are open-sourced under the CC BY-NC 4.0 license, making them convenient for users to obtain and use. Through simple environment setup and pretrained model download, users can start image editing with the Gradio WebUI, lowering the barrier to use.

Technical Principles of Lego-Edit

• Model-level toolkit: Lego-Edit integrates multiple efficiently trained models, each responsible for a specific image operation function, such as color adjustment, object replacement, etc., forming a feature-rich toolkit that provides basic support for complex image editing tasks.
• Driven by multimodal large language models (MLLM): It leverages the strong generalization and reasoning capabilities of MLLM to understand user instructions and coordinate different models in the model-level toolkit, completing the conversion from instruction parsing to specific operations and achieving flexible image editing.
• Three-stage progressive reinforcement learning training strategy: First, supervised fine-tuning (SFT) is performed to let the model learn basic image editing knowledge; then reinforcement learning (RL) is used to train on specific editing tasks, building reasoning ability and tool-use knowledge; finally, additional RL training is conducted using a large amount of unlabeled instructions, with feedback provided by a large-scale critic model, further enhancing the ability to handle flexible instructions.
• Mask Input Mechanism: It supports users in specifying the image regions to be edited via masks, making editing operations more precise, enabling modifications to specific parts of the image without affecting other regions, and improving editing flexibility and accuracy.
• Tool Integration Without Retraining: Lego-Edit can directly integrate newly introduced editing tools without additional fine-tuning, quickly adapt to new editing needs, expand its functional scope, and maintain the system's efficiency and scalability.

Lego-Edit Project Address

• Project Website：https://xiaomi-research.github.io/lego-edit/
• Github Repository：https://github.com/xiaomi-research/lego-edit
• arXiv TechnicalPaper：https://arxiv.org/pdf/2509.12883

Lego-Edit Application Scenarios

• Creative Design: Designers can use Lego-Edit to quickly realize creative ideas, complete complex operations such as image composition and style transfer through simple instructions, improve design efficiency, and inspire more creative inspiration.
• Content Creation and Editing: In scenarios such as video production, advertising design, and social media content creation, Lego-Edit can help creators quickly modify image materials, such as adjusting colors, replacing backgrounds, and adding special effects, meeting diverse content creation needs.
• E-commerce and Product Display: E-commerce merchants can use Lego-Edit to optimize product images, such as removing blemishes, adjusting lighting, and adding virtual scenes, improving product display effects and enhancing users' willingness to purchase.
• Education and Training: In the field of education, Lego-Edit can serve as a teaching tool to help students learn image editing skills and cultivate creativity and aesthetic ability. At the same time, teachers can also use it to quickly produce image materials for teaching courseware.
• Personal Photo Beautification: Ordinary users can use Lego-Edit to beautify personal photos, such as removing backgrounds, adjusting skin tone, and adding decorative elements, easily producing satisfactory photos for social media sharing or personal collection.
• Virtual Reality and Game Development: In virtual reality and game development, Lego-Edit can be used to quickly generate and modify image resources in games, such as character appearances and scene elements, improving development efficiency and enriching the game's visual effects.