ART is an open-source reinforcement learning framework for agent training, with the full name Agent Reinforcement Trainer. Its core goal is to enable large language models to learn from experience, thereby improving the reliability of multi-step agents in real-world tasks. ART is based on the GRPO algorithm and provides an easy-to-use wrapper for introducing GRPO training into any Python application.
Main capabilities
What makes ART unique is that when introducing reinforcement learning training into existing applications, developers do not need to deal directly with the training infrastructure. The framework abstracts the training server into modular services, and the client connects to the developer's codebase through an OpenAI-compatible interface, allowing developers to continue using the familiar completion call style while letting the model continuously improve through iteration.
Training can be initiated from any client machine that can run Python: the ART client can run on a laptop and have the ART server start a temporary GPU environment, or it can run directly on a local GPU. ART integrates with hosted platforms such as W&B, Langfuse, and OpenPipe, providing flexible observability and easier debugging. ART provides optimized default training parameters and inference engine configurations, balancing training efficiency and stability, while also allowing customization for specific needs.
According to official information, ART supports models such as Qwen3.6, GPT-OSS, and Llama, and is theoretically compatible with most causal language models supported by vLLM/HuggingFace-transformers, or at least models supported by Unsloth; Gemma 3 currently appears not to be supported yet.
Usage and training loop
ART's functionality is divided into two parts: the client and the server. The OpenAI-compatible client is responsible for the interaction between ART and the codebase, and can pass messages and obtain completion results from the continuously improving LLM. The server runs independently on any machine with a GPU, abstracting away the complexity of inference and training in the RL loop while retaining a certain amount of space for custom configuration.
The training loop is roughly as follows:
- Inference: The code uses the ART client to execute the agent workflow (usually running multiple rollouts in parallel to collect data faster). Completion requests are routed to the ART server, which runs the model's latest LoRA in vLLM. During agent execution, every system, user, and assistant message is stored in the Trajectory. When a rollout ends, the code assigns a reward to its Trajectory, representing the LLM's performance.
- Training: After all rollouts are complete, the trajectories are grouped and sent to the server. Inference is blocked during training execution. The server uses GRPO to train the model, initializing from the latest checkpoint (using an empty LoRA on the first iteration). The server saves the newly trained LoRA to a local directory and loads it into vLLM. Inference is then unblocked, and the loop returns to step one. This training loop runs continuously until the specified number of inference and training iterations is completed.
To add ART to an existing project, the official instructions can be executed on the client machine running Python with the corresponding installation command (the specific content of the installation command is not given in the material).
Applicable Scenarios
ART is suitable for development scenarios that need to improve the performance of multi-step agents on real tasks and want to let models continuously learn from experience. For example, the official blog shows how to train a Qwen 2.5 14B email agent to surpass o3 on an email retrieval task in the ART·E Agent case. ART also supports integration with LangGraph for training agents with stronger multi-step reasoning and tool-use capabilities; MCP•RL can be used to automatically train models to master the tools of any MCP server; AutoRL targets zero-data training without labeled data; RULER is used for automatic generation of reinforcement learning rewards.
Limitations and Notes
ART is currently in active development; the official project welcomes contributions and provides CONTRIBUTING.md. Gemma 3 currently appears to be unsupported; if other models do not work, you can provide feedback on Discord or submit an issue on GitHub. The source code is released under the Apache-2.0 license. In addition, the material mentions that W&B Training (Serverless RL) is the first public service that can flexibly train models with reinforcement learning and can automatically manage training and inference infrastructure; its claimed benefits include 40% cost reduction, 28% faster training, no infrastructure management required, and each checkpoint can be deployed instantly through W&B Inference. This part belongs to the W&B Training service rather than the ART framework itself. Regarding specific prices, costs, and more usage steps, the material does not currently provide them.
