labelU is an open-source annotation platform for multimodal data, developed by OpenDataLab, mainly used to handle annotation tasks for image, video, and audio data, serving complex data analysis and model training scenarios.
-
Purpose labelU is designed to help users complete annotation work for multimodal data. It provides annotation tools and supporting workflows for three types of data—images, videos, and audio—and can be used for data preparation for tasks such as object detection, scene analysis, image recognition, machine translation, video retrieval, video summarization, action recognition, and audio analysis.
-
Main Capabilities
-
Image annotation: Provides tools such as 2D bounding boxes, semantic segmentation, polylines, and keypoints, which can be combined to handle various image processing tasks.
-
Video annotation: Supports video segmentation, video classification, and video information extraction, suitable for processing long-duration videos, extracting key information, and complex scene analysis.
-
Audio annotation: Supports audio segmentation, audio classification, and audio information extraction, simplifying the audio data processing workflow through visualized sound information.
-
AI-assisted annotation: Supports one-click loading of pre-annotation data and adjustment and optimization according to actual needs.
-
AI automatic annotation: Integrates AI model services. Clicking the "AI Annotate" button on the annotation page allows the model to automatically detect and segment image objects, supports batch annotation for the entire task, and tracks progress in real time. Three built-in reference model servers: Florence-2 (lightweight, CPU-friendly, about 4GB VRAM), GroundingDINO + SAM ViT-B (high-quality detection plus segmentation, about 4GB VRAM), SAM 3 (unified model, about 8GB VRAM, requires a high-end GPU). For model service configuration instructions, see model_server/README.md.
-
S3 data source import: Supports directly importing annotation data from S3-compatible object storage (AWS S3, MinIO, etc.). You can configure the data source connection in the task settings, browse and preview files, and then import selected files or all files under a path with one click.
-
Export formats: Supports exporting multiple data formats such as JSON, COCO, and MASK.
-
Usage Methods
-
Online trial: You can try the LabelU annotation toolkit, and you can also try the LabelU online version.
-
Local deployment:
- Install Miniconda, select the corresponding operating system type to download and install. Note: For MacOS systems with Intel chips, please install the intel x86_64 version of Miniconda.
- After installation is complete, run the relevant commands in the terminal (during the process, you can choose the default 'y' when prompted). Note: On the Windows platform, you can run the above commands in Anaconda Prompt.
- Activate the environment.
- Install LabelU.
- Install the test version: pip install labelu==<test revision>
- Install labelu with MySQL support.
- If you need to use the MySQL database after upgrading from version 1.x, run the corresponding command to migrate data from the built-in SQLite database to the MySQL database.
- Visit http://localhost:8000/ to use it.
-
Local development: Supports local development mode.
-
Related Projects and License labelU is built on LabelU-kit, a web front-end annotation toolkit. In the same ecosystem, there are also LabelLLM (an open-source LLM dialogue annotation platform) and Miner U (a one-stop open-source high-quality data extraction tool). This project is released under the Apache 2.0 license.
-
Limitations The information provided does not include details such as the platform's data scale limits, concurrency capability, detailed hardware requirements for deployment (except for the VRAM requirements of the AI automatic annotation model), whether a paid version or pricing is offered, or the specific annotation format details supported.
