SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsrl-msprobe

rl-msprobe

Data & analytics

View repository

Updated 10-15 17:19

About this skill

verl msprobe precision data collection

Automatically identify and execute three types of msprobe collection.First check the msprobe environment, then determine the collection mode.

Prerequisite: msprobe environment (mandatory)

All three modes (training / inference / consistency) depend on msprobe. Before modifying scripts, patching, or starting trainingbeforeexecute:

bash scripts/ensure-msprobe-env.sh
Item Description
PyPI package name mindstudio-probe
Python import from msprobe.pytorch import PrecisionDebugger
Environment and verl training/inference use the same Python(if inside a container, execute inside the container)
Auto-install The script executes by default pip install mindstudio-probe
Detect only bash scripts/ensure-msprobe-env.sh --no-install

If installation fails: stop the subsequent dump configuration, and prompt the user to install and verify it themselves:

pip install mindstudio-probe
python3 -c "from msprobe.pytorch import PrecisionDebugger; print('OK')"

Before passing the environment check, do not enter path A/B/C.

Default verl source code:VERL_ROOT=/home/zhouyang/skill/verl（The verl tree actually used by the user in training/inside the container shall prevail, may differ from the default path)

Mode Identifier Modify source code Backend differences
training Training collection global_profiler + stages Three stages No Shared by FSDP/Megatron
inference Inference rollout dump_config_path / msprobe_dump_config SGLang<0.5.11 requires modifying SGLang vLLM / SGLang
consistency Training-inference consistency Training engine patch + inference dump + PROMPTS_ONLY Yes (verl engine + optional vLLM dispatch) Training FSDP/Megatron + inference vLLM/SGLang

Shared hard constraints (inference-related)：actor_rollout_ref.rollout.enforce_eager=True(do not enable graph mode).

Shared config.json specification: all msprobe config.json Prefer "level": "mix"(L0 module-level + L1 API-level mixed collection). When the user does not specify level, default mix; only change it when it is explicitly requested to use only L0/L1 single granularity.

Step 0: Identify the collection mode

bash scripts/detect-dump-mode.sh "$TRAINING_SCRIPT" "$USER_HINT"
# 输出: training | inference | consistency | unknown
Signal Mode
"inference collection" "rollout dump" "generate dump" inference
"training-inference consistency" "prefill alignment" consistency
"training collection" "global_profiler" "precision_debugger" training
Mentions actor_compute_log_prob / ref_compute_log_prob / actor_update any stage training
Script contains global_profiler.tool=precision_debugger or precision_debugger.stages training
Script contains dump_config_path / msprobe_dump_config inference
Script contains DUMP_ON + PROMPTS_ONLY consistency

When it cannot be determined, ask the user to choose A training / B inference / C training-inference consistency。

Training collection (path A) obtains dump data separately according to the following three stages (select a subset or all via global_profiler.global_tool_config.precision_debugger.stages ):

stage Role Collection content
actor_compute_log_prob actor actor forward log prob
ref_compute_log_prob ref ref forward log prob
actor_update actor actor policy update (forward + backward)

By default all three stages are collected; the user may specify only a subset, but must simultaneously enable the corresponding role's profiler.enable。

Path A: training collection (training)

Detailed description:references/training-profiler-dump.md

Training-side data is collected at the stage granularity, and the core three stages are actor_compute_log_prob、ref_compute_log_prob、actor_update. Each stage is persisted to disk independently under {save_path}/step_{global_step}/{stage}/step0/rank0/dump.json .

Configuration key points

global_profiler.tool=precision_debugger \
global_profiler.steps='[1,2]' \
global_profiler.save_path=outputs/profile \
+global_profiler.global_tool_config.precision_debugger.config_path=/path/to/config.json \
+global_profiler.global_tool_config.precision_debugger.stages='[actor_compute_log_prob,ref_compute_log_prob,actor_update]' \
actor_rollout_ref.actor.profiler.enable=True \
actor_rollout_ref.ref.profiler.enable=True
  • stages determines which stages are collected; when omitted, verl defaults to no restriction (explicitly listing them is recommended)
  • When collecting only actor_compute_log_prob + actor_update , ref's profiler.enable can be disabled
  • Do not modify the verl source code;forbiddento set dump_path
  • config.json in "level": "mix"(default recommendation, see the shared specification above)
  • Validation:python3 scripts/check-training-profiler.py run_profiler_*.sh

Path B: inference collection (inference)

vLLM：references/inference-vllm-dump.md
SGLang：references/inference-sglang-dump.md

B0. Identify the rollout backend

bash scripts/detect-rollout-backend.sh "$TRAINING_SCRIPT"
# vllm | sglang | unknown

B1. Common prerequisites

export TORCHDYNAMO_DISABLE=1   # 推荐
actor_rollout_ref.rollout.enforce_eager=True   # 必须

B2. vLLM (two steps are enough, no need to modify verl source code)

① config.json(must include dump_path）：

{
  "task": "statistics",
  "dump_path": "/path/to/dump/generate",
  "rank": [],
  "step": [0],
  "level": "mix",
  "async_dump": false,
  "extra_info": true,
  "statistics": {
    "scope": [], "list": [], "tensor_list": [],
    "data_mode": ["all"], "summary_mode": "statistics"
  }
}

Template:assets/config_generate.json.example

② Script parameters：

actor_rollout_ref.rollout.name=vllm \
actor_rollout_ref.rollout.enforce_eager=True \
++actor_rollout_ref.rollout.engine_kwargs.vllm.additional_config.dump_config_path="/path/to/generate_config.json"

B3. SGLang (according to the official msprobe method)

SGLang version Approach
>= 0.5.11 verl parameter passing:++actor_rollout_ref.rollout.engine_kwargs.sglang.msprobe_dump_config=/path/to/config.json
< 0.5.11 According to sglang_eager_dump_instruct.md patch SGLang ModelRunner

The verl side still must:

actor_rollout_ref.rollout.name=sglang \
actor_rollout_ref.rollout.enforce_eager=True

PD disaggregation:export SGLANG_ENABLE_HEALTH_ENDPOINT_GENERATION=0

B4. Validation and artifacts

python3 scripts/check-inference-dump.py run_infer_dump_*.sh --backend vllm   # 或 sglang

Expected:{dump_path}/step_N/rank_M/dump.json(vLLM may include {pid}/ subdirectory)

Path C: training-inference consistency (consistency)

= Path B inference side + Training engine worker patch + alignment constraints

Main document：references/consistency-engine-worker.md(legacy megatron → engine worker migration)
Auxiliary patch：references/consistency-auxiliary-patches.md
Alignment constraints：references/shared-prerequisites.md

C1. Inference side

Configure the rollout dump according to B2/B3 (enforce_eager=True is mandatory).

Parallelism alignment (must be checked)：

  • FSDP：actor_rollout_ref.rollout.tensor_model_parallel_size = number of cards occupied by rollout (e.g., 4 cards → gen_tp=4); for the consistency stage it is recommended data_parallel_size=1
  • Megatron: actor/ref's TP/PP/EP must match rollout's tensor_model_parallel_size / data_parallel_size / expert_parallel_size item by item

See references/shared-prerequisites.md。

For vLLM training-inference consistency, vLLM-Ascend is recommended dispatch_logger patch（dispatch_log.jsonl associated prefill step), see the consistency-engine-worker document.

C2. Training side

Patch the engine worker + auxiliary files according to FSDP/Megatron:

export DUMP_ON=1 PROMPTS_ONLY=1 TORCHDYNAMO_DISABLE=1 DUMP_PHASE=log_prob
export MSPROBE_ACTOR_CONFIG=/path/to/config_actor.json
bash scripts/detect-train-backend.sh "$TRAINING_SCRIPT"
# engine worker 钩子（transformer_impl.py）
python3 scripts/check-engine-patch.py --verl-root "$VERL_ROOT" --backend "$TRAIN_BACKEND"
# 训推一致全量（PROMPTS_ONLY、request_id、可选 vLLM dispatch）
python3 scripts/check-consistency-patch.py \
  --verl-root "$VERL_ROOT" --backend "$TRAIN_BACKEND" \
  --vllm-ascend-root /path/to/vllm-ascend

Note: the patch must be applied to theactually runningverl source tree (may differ from VERL_ROOT , e.g., /verl）。

C3. Alignment constraints and artifacts

  • Batch：train/mini/micro=1，n=DP（DP=world_size/SP）；Priority actor_rollout_ref.actor.ulysses_sequence_parallel_size=1 and actor_rollout_ref.ref.ulysses_sequence_parallel_size=1(even if the original script sp_size>1 , it must still be overridden in the collection script)
  • Parallelism: FSDP inference tensor_model_parallel_size = number of rollout cards; Megatron training TP/PP/EP must match rollout item by item

See references/shared-prerequisites.md and references/dump-output-correlation.md。

Three-mode quick reference

Item training inference consistency
config dump_path Forbidden Mandatory Mandatory（generate）
enforce_eager — Must be True Must be True
vLLM entry point — dump_config_path Same as inference
SGLang entry point — msprobe_dump_config or patch Same as inference
Modify verl source code No No (old SGLang modified SGLang) Yes (engine)
Training-inference parallelism alignment — — FSDP: inference TP = number of cards; Megatron: training/inference partitioning consistent
step / stage control global_profiler.steps + stages three stages config step / msprobe step Controlled separately on both sides
Training stage actor_compute_log_prob / ref_compute_log_prob / actor_update — engine micro_batch

Prohibitions

  • Prohibited when unconfirmed msprobe can be imported, continue configuration or start dump
  • Prohibited under enforce_eager=False to perform inference dump
  • Prohibited to patch engine without determining the mode
  • Prohibited to directly modify the user's original training script
  • vLLM and SGLang collection methodsmust not be mixed
  • consistency prohibits under FSDP gen_tp < rollout 卡数(inference dump rank and training rank cannot be aligned by card)
  • consistency prohibits Megatron training partitioning and rollout partitioning being inconsistent(TP/PP/EP must be identical item by item)

Reference