SuperAPI AI Directory

pytrio

General & other

View repository

Updated 10-16 09:53

About this skill

PyTRIO SDK Skill

What is PyTRIO

PyTRIO is a remote LLM training and inference SDK. Local code only handles data preparation and scheduling; the actual forward pass, backward pass, optimizer steps, and inference sampling are all executed on a remote GPU cluster. No local GPU is required.

  • Install: pip install pytrio transformers torch(or uv add pytrio transformers torch)
  • Import convention: import pytrio as trio, all APIs are accessed via trio.xxx Access
  • Tokenizer: use train.get_tokenizer() or sampler.get_tokenizer()(requires installing modelscope)
  • For environment setup, see references/00-overview.md

Authentication (important)

Before using PyTRIO, authentication must be completed first; otherwise all API calls will fail.

Check whether already logged in: check whether ~/.pytrio/config.toml exists and contains the api_key field.

If not logged in, follow the process below:

  1. Prompt the user to provide an API Key (available from https://pytrio.cn/dashboard obtain)
  2. After the user provides the key, run the non-interactive login command:
    trio login -k <用户提供的API_KEY>
    
  3. After successful login, credentials are saved in ~/.pytrio/config.toml, and subsequent ServiceClient() do not need api_key passed

It can also be passed directly in code (suitable for temporary use):

client = ServiceClient(api_key="YOUR_API_KEY")

Note: the CLI command is trio login, not pytrio login。

PyTRIO ↔ PyTorch concept quick reference

PyTorch PyTRIO Description
model = AutoModel(...) client = ServiceClient(api_key=...) No local model object
model(input_ids) train.forward(data) Remote forward pass
loss.backward() train.forward_backward(data) Forward + backward combined
optimizer.step() train.optim_step(AdamParams(...)) Remote Adam
optimizer.zero_grad() Not required Automatically managed remotely
torch.save(...) train.save_state(name) Save to the cloud
model.generate(...) sampler.sample(prompt, ...) Remote inference
torch.Tensor TensorData / plain list Automatic mutual conversion

Core API quick reference

Entry point

import pytrio as trio

client = trio.ServiceClient(api_key="...")
models = client.get_supported_models()

Model selection rule: must use Qwen/Qwen3-4B-Instruct-2507, do not use Qwen/Qwen3-4B(unavailable on the Base model training side).

Training

import pytrio as trio

train = client.create_lora_training_client(
    base_model="Qwen/Qwen3-4B-Instruct-2507",
    rank=32,            # [4, 64]
    train_mlp=True,
    train_attn=True,
    train_unembed=False,
)
tokenizer = train.get_tokenizer()

# tokenizer 注意事项(官方示例的标准模式):
#   序列起始段(prompt)用 add_special_tokens=True   —— 保留 BOS
#   拼接延续段(completion)用 add_special_tokens=False —— 避免重复 BOS
#   推理时 encode 默认 add_special_tokens=True,保留 BOS 作为生成起始
#   apply_chat_template() 已自行管理 special tokens,返回文本再 encode 时也用 False

# 官方标准 SFT 数据构造模式:
prompt_tokens = tokenizer.encode(prompt_text, add_special_tokens=True)
prompt_weights = [0] * len(prompt_tokens)
completion_tokens = tokenizer.encode(completion_text, add_special_tokens=False)
completion_weights = [1] * len(completion_tokens)

tokens = prompt_tokens + completion_tokens
weights = prompt_weights + completion_weights

# 手动偏移对齐: model_input[i] 预测 target_tokens[i]
input_tokens = tokens[:-1]
target_tokens = tokens[1:]
weights = weights[1:]  # weights 必须同步偏移

sample = trio.Datum(
    model_input=trio.ModelInput.from_ints(input_tokens),  # 必须用 ModelInput.from_ints() 包装
    loss_fn_inputs={
        "target_tokens": target_tokens,  # 值可直接传 list,会自动转 TensorData
        "weights": weights,              # 可选,默认全 1;SFT 场景强烈建议传
    },
)

# 训练步骤
fb = train.forward_backward(data=[sample], loss_fn="cross_entropy")
result = fb.result()  # APIFuture -> ForwardBackwardOutput
# result.metrics 的 key 是 "loss:sum"(注意冒号),不是 "loss"
# per-token 归一化 loss(两种等价写法,需 import numpy as np):
#   方式 A: loss_sum / total_weight_sum
#          total_weight_sum = sum(sum(ex.loss_fn_inputs["weights"].tolist()) for ex in data)
#   方式 B: -np.dot(logprobs, weights) / weights.sum()
#          logprobs = np.concatenate([o["logprobs"].tolist() for o in result.loss_fn_outputs])
#          weights  = np.concatenate([ex.loss_fn_inputs["weights"].tolist() for ex in data])
loss_sum = result.metrics.get("loss:sum", 0.0)
train.optim_step(trio.AdamParams(learning_rate=1e-4)).result()

Loss function

loss_fn loss_fn_inputs required Purpose
"cross_entropy" target_tokens, optional weights SFT
"importance_sampling" target_tokens, logprobs, advantages,Strongly recommended weights GRPO / offline RL
"ppo" target_tokens, logprobs, advantages,Strongly recommended weights Online RL

For RL scenarios, the weights is used to mask the prompt part (0=ignore, 1=count toward loss); if not passed, the prompt will also participate in the policy gradient, which usually causes it to go off track. All official RL examples include weights。

ppo can be set via loss_fn_config customize the clipping threshold (default ε=0.2, i.e. ratio clipped to [0.8, 1.2]):

train.forward_backward(
    data=data, loss_fn="ppo",
    loss_fn_config={"clip_low_threshold": 0.9, "clip_high_threshold": 1.1},
)

Custom loss (requires local torch):

def my_loss(samples, logprobs_list):
    loss = ...  # torch.Tensor
    return loss, {"metric": loss.item()}
train.forward_backward_custom(data=samples, loss_fn=my_loss)

Save and restore

# 保存权重(仅用于推理)
save = train.save_weights_for_sampler("name").result()
# save.path -> 远端路径

# 保存完整 checkpoint(含优化器,用于断点续训)
state = train.save_state("name").result()

# 断点续训
train2 = client.create_training_client_from_state_with_optimizer(state.path)

# 仅加载权重续训(不含优化器动量)
train3 = client.create_training_client_from_state(state.path)

Inference

import pytrio as trio

# 从训练权重推理
sampler = train.create_sampling_client(model_path=save.path)
# 或独立推理
sampler = client.create_sampling_client(base_model="Qwen/Qwen3-4B-Instruct-2507")

tokenizer = sampler.get_tokenizer()
input_ids = tokenizer.encode("Hello")

result = sampler.sample(
    prompt=trio.ModelInput.from_ints(input_ids),  # 必须用 trio.ModelInput 包装
    num_samples=1,
    sampling_params=trio.SamplingParams(max_tokens=100, temperature=0.7),
).result()

text = result.sequences[0].text
logprobs = result.sequences[0].logprobs

Management

rest = client.create_rest_client()
rest.list_weights()            # 已保存权重
rest.list_training_runs()      # 训练记录
rest.list_checkpoints(run_id)  # 某次训练的 checkpoint
rest.list_user_checkpoints()   # 所有 checkpoint
rest.get_archive_url(id)       # 下载链接
rest.delete_checkpoint(run_id, cp_id)

AdamParams SDK default values

Note they differ from PyTorch defaults:beta2=0.95(PyTorch default 0.999),eps=1e-12(PyTorch default 1e-8). To match PyTorch behavior, explicitly pass AdamParams(beta2=0.999, eps=1e-8)。

Error handling

  • APIFuture.result(timeout=None) supports a timeout parameter (seconds); on timeout throws TimeoutError
  • Server-side exceptions throw pydantic.ValidationError or ValueError
  • WebSocket disconnections are not automatically reconnected; you need to rebuild ServiceClient

Pitfall checklist

  1. All methods return APIFuture — you must call .result() to get the actual value, otherwise you get a Future object
  2. model_input and target_tokens must have the same length — when auto_shift=False(default), the user is responsible for offset alignment
  3. importance_sampling/ppo requires four equal-length fields — model_input, target_tokens, logprobs, advantages must be equal length
  4. trainable_token_indices and train_unembed are mutually exclusive — cannot be enabled at the same time
  5. ignore_eos=True must be set when max_tokens — otherwise an exception is thrown
  6. the checkpoint path is a remote path — save_state()/save_weights_for_sampler() the returned path is a cloud path, not a local file
  7. forward_backward_custom requires local torch — the custom loss function computes gradients locally and sends them back to the remote
  8. Import convention import pytrio as trio — all APIs are accessed via trio.xxx (e.g. trio.ServiceClient、trio.Datum), do not use from pytrio import ...
  9. the login command is trio login — the CLI command is still trio, not pytrio
  10. all values of loss_fn_inputs are automatically converted to TensorData — you can directly pass a list, numpy array, or torch tensor
  11. model_input must be wrapped with trio.ModelInput.from_ints(list) Wrapping — you cannot directly pass list[int], otherwise Pydantic validation will report an error
  12. similarly, the prompt parameter must be wrapped with trio.ModelInput.from_ints(list) Wrapping — sampler.sample(prompt=trio.ModelInput.from_ints(ids), ...)
  13. tokenizer uses get_tokenizer() — train.get_tokenizer() or sampler.get_tokenizer()(requires installing modelscope)
  14. at runtime you need to bypass the local proxy — if there is a system proxy, you need to set the no_proxy=pytrio.cn https_proxy="" http_proxy="" environment variable
  15. you must use the Instruct model — Qwen/Qwen3-4B (Base) is unavailable on the training side, you must use Qwen/Qwen3-4B-Instruct-2507
  16. the naming differs from the HuggingFace API — PyTRIO's parameter names differ from HF transformers, do not mix them up:
    • the sampling parameter is max_tokens(not max_new_tokens)
    • the inference result is result.sequences[0].text(not result.samples)
    • forward propagation must use the keyword argument train.forward_backward(data=[...]) (not positional arguments)
    • the optimizer parameter is learning_rate(not lr)
    • the key of loss_fn_inputs is target_tokens(not labels)
    • use import pytrio as trio instead of from pytrio import ...
    • do not import internal modules (such as pytrio.types.forward.encoded_text_chunk)
  17. model_input and target_tokens must be offset by one — model_input=tokens[:-1], target_tokens=tokens[1:], you cannot pass the same tokens
  18. the loss key is "loss:sum"(with a colon) — fb_result.metrics.get("loss:sum"), not "loss"、"train_loss" or .loss attribute
  19. do not use -100 for token masking — HF uses -100 to mark ignored tokens, but PyTRIO does not support this mechanism. PyTRIO uses the weights field to control which tokens participate in loss computation (0.0=ignore, 1.0=compute), see best-practices/sft.md
  20. sampling_params must pass a trio.SamplingParams object — you cannot pass a dict,sampler.sample(sampling_params=trio.SamplingParams(max_tokens=100))
  21. add_special_tokens distinguished by segment — the prompt start segment uses True(preserve BOS), the completion concatenation segment uses False(avoid duplicate BOS). Do not make them all True (this will add extra BOS), and do not make them all False (the model loses its start marker and will train incorrectly)
  22. weights must be offset in sync with target_tokens Synchronous offset — the official standard pattern is to first construct full_weights, then weights = weights[1:], strictly aligned with target_tokens = tokens[1:] Strict alignment
  23. RL scenarios (IS/PPO) also need to pass weights — mask out the prompt part, otherwise prompt tokens will also participate in the policy gradient
  24. The asynchronous version ends with _async — for high concurrency / RL rollout, the asynchronous version is recommended, see — for high concurrency / RL rollout, the asynchronous version is recommended, see best-practices/async.md

Detailed documentation

The complete API documentation is in this Skill's references/ directory:

  • references/00-overview.md — Architecture overview and concept mapping
  • references/01-service-client.md — ServiceClient API
  • references/02-training-client.md — TrainingClient API
  • references/03-sampling-client.md — SamplingClient API
  • references/04-rest-client.md — RestClient API
  • references/05-data-types.md — Data type reference
  • references/06-debug.md — Debugging guide (triage process, error decoding, self-check checklist)

Example code

Runnable minimal closed-loop examples are in this Skill's examples/ directory:

  • examples/01_train_sft.py — SFT training
  • examples/02_inference.py — Inference
  • examples/03_checkpoint_resume.py — Training resumption from checkpoint
  • examples/04_model_management.py — Model management
  • examples/05_importance_sampling.py — GRPO-style training

Scenario best practices

Recommended code templates and notes for specific training scenarios are in the best-practices/ directory:

  • best-practices/sft.md — SFT supervised fine-tuning: prompt masking (weights usage), EOS appending, tokenizer options
  • best-practices/grpo.md — GRPO/PPO reinforcement learning: real rollout→reward→forward_backward closed loop
  • best-practices/async.md — Asynchronous training:_async API、asyncio.gather, concurrent scheduling

Deployment

  • references/07-openai-compat.md — After training, the model provides external services through an OpenAI-compatible API (chat.completions / completions)