SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsverl-async-dapo

verl-async-dapo

General & other

View repository

Updated 10-16 05:00

About this skill

Verl single-async DAPO training

Interactive startup flow

Before starting the training service, the Agent mustask the user in the following order:

1. Container selection

检测到以下容器：
  [1] jins ( 运行中)
  [2] 创建新容器

请选择容器 (1/2):

2. Proxy configuration (e.g., creating a new container or network issues)

请提供代理配置（如不需要请回复"无"）：
  http_proxy: ?
  https_proxy: ?

3. Feature selection

是否有自定义特性需求？(回复"默认"使用默认配置)

性能特性（默认全开）:
  - flash_attn: Flash Attention 加速
  - dynamic_batch: 动态 Batch Size
  - remove_padding: Remove Padding 优化
  - gradient_checkpointing: 梯度检查点

显存特性（默认关闭，OOM 时自动开启）:
  - offload: 参数/优化器卸载
  - recompute: 重计算

可选特性:
  - prefix_cache: Prefix Cache
  - chunked_prefill: Chunked Prefill

4. SwanLab configuration (if not configured)

SwanLab 监控配置：
  Host: ?
  API Key: ?

Quick start

# 方式 1: 快速启动脚本 (推荐)
CONTAINER=jins TRAIN_STEPS=100 \
SWANLAB_HOST=http://10.143.2.129:8000 \
SWANLAB_API_KEY=your-key \
bash scripts/quick_start.sh

# 方式 2: 分步执行
# 2.1 训练前检查
bash scripts/preflight_check.sh

# 2.2 启动单异步 DAPO 训练 (使用默认特性进行训练)
export TRAIN_STEPS=4
bash scripts/run_dapo.sh

# 2.3 启动 One-Step-Off-Policy 训练
export TRAIN_STEPS=4
bash scripts/run_one_step_off_policy.sh

One-Step-Off-Policy training

One-Step-Off-Policy is a resource-isolated single-async training mode where Trainer and Rollout use independent GPU groups.

# 基本用法
export TRAIN_STEPS=100
export TRAINER_GPUS=4
export ROLLOUT_GPUS=4
bash scripts/run_one_step_off_policy.sh

# 启用 offload 特性
export ENABLE_OFFLOAD=True
bash scripts/run_one_step_off_policy.sh

# FSDP2 框架
export FRAMEWORK=fsdp
bash scripts/run_one_step_off_policy.sh

Configuration parameters:

Variable Default value Description
FRAMEWORK megatron Framework (megatron/fsdp)
TRAINER_GPUS 4 Number of training GPUs
ROLLOUT_GPUS 4 Number of Rollout GPUs
MAX_RESPONSE_LENGTH 8192 Maximum response length
ENABLE_OFFLOAD False Whether to enable offload

SwanLab configuration

SwanLab configuration is provided by swanlab-setup skill , and is automatically loaded atcommon.sh startup. See SKILL.md of the swanlab-setup skill for details.

Configuration is obtained automatically in priority order:Environment variables > configuration file > interactive input

# 环境变量（推荐）
export SWANLAB_HOST="http://your-swanlab-host:8000"
export SWANLAB_API_KEY="your-api-key"

Important: you must include trainer.logger in the "swanlab" parameter to enable monitoring:

# 正确的 logger 配置（启用 SwanLab 监控）
trainer.logger='["console", "swanlab"]'

# 仅控制台输出（不启用监控）
trainer.logger='["console"]'

SwanLab environment variables are automatically read by the verl tracking module:

  • SWANLAB_API_KEY: API key
  • SWANLAB_LOG_DIR: log directory (default: swanlog)
  • SWANLAB_MODE: mode (cloud/local, default: cloud)

Proxy configuration

export http_proxy="http://your-proxy:8080"
export https_proxy="http://your-proxy:8080"

Training monitoring strategy

Behavior after startup:

  1. Output the SwanLab link for the user to view themselves
  2. Run silently in the background, without pushing progress in real time
  3. Notify the user only on errors(OOM, crash, configuration error, etc.)

User monitors commands themselves:

# 查看当前进度
docker exec {container} grep "Training Progress" /verl/train.log | tail -1

# 实时跟踪日志
docker exec {container} tail -f /verl/train.log

# 检查是否出错
docker exec {container} grep -E "Error|error|OOM|Exception" /verl/train.log | tail -10

Feature management strategy

Feature classification

Type Feature Default status Description
Performance feature flash_attn ✅ On by default Flash Attention acceleration
Performance feature dynamic_batch ✅ On by default Dynamic Batch Size
Performance feature remove_padding ✅ On by default Remove Padding optimization
Performance feature gradient_checkpointing ✅ On by default Gradient checkpointing
Memory feature offload ❌ Off by default Parameter/optimizer offload (automatically appended on OOM)
Memory feature recompute ❌ Off by default Recomputation (automatically appended on OOM)
Optional feature prefix_cache ❌ Off by default Prefix Cache
Optional feature chunked_prefill ❌ Off by default Chunked Prefill

Feature priority

用户显式指定 > 默认性能特性 > OOM 自动追加显存特性

Important：

  • When the user does not specify → use the default performance features (all on)
  • On OOM → automatically append GPU memory features (even if the user specified other features)
  • Features explicitly specified by the user are never removed; they are only appended on OOM

OOM auto-recovery process

启动训练 (默认性能特性)
    ↓
OOM 错误?
    ├─ 是 → 追加 offload → 重试
    │   └─ 仍 OOM → 追加 recompute → 重试
    │       └─ 仍 OOM → 报告失败，建议调整参数
    └─ 否 → 训练成功（静默完成）

Configuration parameters

Environment variables

Variable Default value Description
FRAMEWORK megatron Framework (megatron/fsdp)
MODEL_PATH Qwen3-8B Model path
TRAIN_FILE dapo-math-17k.parquet Training data
VAL_FILE aime-2024.parquet Validation data
TRAIN_STEPS 100 Number of training steps
TRAINER_GPUS 4 Number of training GPUs
ROLLOUT_GPUS 4 Number of rollout GPUs
USER_FEATURES (empty) User-explicitly specified features, comma-separated
MAX_OOM_RETRIES 2 Maximum number of OOM retries
SWANLAB_HOST (obtained from configuration) SwanLab server address
SWANLAB_API_KEY (obtained from configuration) SwanLab API key

Detailed reference