SuperAPI AI Directory
Partner with us中文AI newsFavorites
All toolsAI developer platformsAgentic-RAG-R1

Agentic-RAG-R1

Agentic-RAG-R1 (repository name Harness-RL) is a reinforcement learning-based agentic RAG framework, and the official implementation of the paper accepted at PCC 2026. It proposes training language model agents within a central-agent multi-agent execution framework (harness), without flattening the execution process into a single token sequence, but instead constructing black-box trajectories from interface calls and session prefix trees, and employing Conflict-Aware Policy Optimization (CAPO) to route the gradients of action tokens and argument tokens separately to their corresponding parameter subspaces. The framework supports central-only agent training and joint multi-agent training, integrates the production-grade agent framework StackPlanner and the slime distributed training stack, and targets Linux x86_64 + NVIDIA GPU environments.

Open website

About this tool

Agentic-RAG-R1 (repository name Harness-RL) is the official implementation of the paper "Black-Box Reinforcement Learning with Action-Args Decoupling for Central-Agent Multi-Agent Harnesses," and has been accepted at PCC 2026.

Purpose This framework is used to train language model agents within a central-agent multi-agent harness. Unlike approaches that flatten the execution process into a single token sequence, Harness-RL constructs black-box trajectories from interface calls and session prefix trees, and then, through Conflict-Aware Policy Optimization (CAPO), routes the gradients from action tokens and argument tokens to their respective corresponding parameter subspaces.

Main Capabilities

  • Supports central-only training and joint multi-agent training.
  • Employs CAPO to achieve action-argument decoupled gradient routing.
  • Integrates StackPlanner as a production-grade agent execution framework, providing standard interfaces such as web_search.
  • Integrates slime to provide distributed rollout, SGLang serving, Megatron training, checkpoint saving, and weight synchronization.
  • Implements retrieval configuration through the Search-R1 retriever.
  • Provides portable templates, starting in dry-run mode by default, making it easy to inspect the parsed commands before actually running them.
  • Supports resuming training from slime checkpoints.
  • Provides a pipeline for generating predictions and scoring them with an OpenAI-compatible judge.

Usage Environment requirements are Linux x86_64, NVIDIA GPU, a compatible CUDA driver, Git, Make, and Python 3.12. The default installation location is $HOME/.cache/harness-rl, which can be changed by setting SP_ENV_PREFIX, SP_SOURCE_ROOT, or SP_BUILD_CACHE.

The main steps include:

  1. Download and convertFixed ASearcher training split data, prepare the evaluation manifest; the generated dataset is written to the data/ directory and is not tracked by Git.
  2. Place the Hugging Face checkpoint and its Megatron torch-distributed conversion result; slime provides conversion tools; in the selected file under configs/, update hf_checkpoint, reference_checkpoint, tokenizer, distributed topology, and model architecture parameters.
  3. Configure retrieval: Harness-RL uses the Search-R1 retriever through the standard StackPlanner web_search interface; SP_SEARCH_R1_ROOT, SP_SEARCH_R1_CUDA_DEVICES, or SP_SERVICE_HOST can be set to match the deployment; the submitted configuration contains no credentials and uses the loopback service address by default. A central-agent-only run also requires a frozen OpenAI-compatible expert endpoint.
  4. Use the three portable templates provided; the templates use relative repository paths and start in dry-run mode by default; edit the model paths and hardware topology, check the parsed commands, then set dry_run to false.
  5. Resume training from the slime checkpoint.
  6. Build the CAPO routing artifacts: experiments B and C require routing metadata generated from successful rollout records; these files are runtime inputs and are not included in version control; joint multi-agent training uses --taxonomy all-agent-action-output, then point the capability field in the corresponding configuration to the generated directory.
  7. Use the production-grade StackPlanner graph to generate predictions, and score the generated predictions with an OpenAI-compatible judge.

Applicable scenarios Suitable for research or engineering scenarios that need to train language model agents within a multi-agent execution framework, especially reinforcement learning experiments that wish to avoid flattening execution into a single token sequence and require decoupled gradient optimization of actions and parameters. Supports both central-agent-only and joint multi-agent training modes.

Limitations The materials do not provide explanations regarding fees, pricing, platform support scope, effectiveness metrics, or specific performance data. The runtime environment is limited to Linux x86_64 and NVIDIA GPU, and requires a compatible CUDA driver, Git, Make, and Python 3.12. The retrieval configuration depends on the Search-R1 retriever and the StackPlanner web_search interface; a central-agent-only run also requires a frozen OpenAI-compatible expert endpoint. Some experiments require additionally generating CAPO routing metadata, and these files are not included in version control.

Related tools

Browse category