Prodigy is a downloadable developer tool and library for creating training and evaluation data for machine learning models, suitable for LLM, NLP and computer vision tasks, such as named entity recognition, text classification, object detection, image segmentation and evaluation. It offers a new way to build custom AI systems: define classification schemes with real examples instead of relying only on prompts, and let powerful models assist with annotation, with no machine learning experience required.
Main capabilities:
- Supports a variety of annotation tasks, including named entity recognition, text classification, object detection, image segmentation and evaluation.
- Defines classification schemes through real examples, with model-assisted annotation.
- Built-in workflows and command-line commands cover common tasks, and components are provided for implementing custom workflow scripts.
- Fully custom data flows and interfaces can be defined to break tasks down and automate as much as possible, thereby improving annotation efficiency.
- Provides a convenient Python API and command-line interface, and all created data is accessible, making it easy to train custom models with standard libraries such as PyTorch or TensorFlow.
Usage and integration: Prodigy is a Python package and library that includes a web app and can be downloaded and installed like any other library. At its core are "recipes"—Python functions that describe workflows, enabling custom data processing and model training logic, integration with third-party or internal libraries and tools, and reusable workflows. It is tightly integrated with spaCy by default, and can also be used with other libraries and tools; the built-in train recipe provides first-class support for spaCy, and there are plugins for using and training Hugging Face models, and it integrates out of the box with major LLM API providers. Users can customize Prodigy with their own Python functions, mix and match front-end components, and even implement custom interactive interfaces with HTML, CSS and JavaScript. The web app is optimized for fast, intuitive and efficient annotation.
Operation and privacy: Prodigy runs entirely on the user's own machine and does not "phone home" or connect to official or any third-party servers. After installation, it can even run on a completely offline isolated machine. All data and models used and created are completely private and under the user's control. It can be deployed as a standard Python web server and application on any cloud provider, including fully local deployment.
Use cases and licensing: Suitable for individuals and teams that need to build custom AI systems, have strict requirements for data privacy, and want to run locally and avoid lock-in. It is designed as a developer tool and assumes that users have basic familiarity with the Python programming language and the command line, but once the annotation task is set up, the web app makes it easy for anyone to create annotations without programming experience. Prodigy uses a lifetime license, with a one-time payment for permanent use, offering flexible options for individuals and teams; researchers at academic institutions that grant degrees can apply for a temporary license for free use, and need to contact by email and provide detailed university information.
Limitations: Prodigy is a developer tool, and basic Python and command-line knowledge are required before use. Specific pricing, detailed platform support and version information are not currently provided.
