What is PyTRIO
PyTRIO is a remote LLM training and inference SDK. Local code only handles data preparation and scheduling; the actual forward pass, backward pass, optimizer steps, and inference sampling are all executed on a remote GPU cluster. No local GPU is required.
- Install:
pip install pytrio transformers torch(oruv add pytrio transformers torch) - Import convention:
import pytrio as trio, all APIs are accessed viatrio.xxxAccess - Tokenizer: use
train.get_tokenizer()orsampler.get_tokenizer()(requires installing modelscope) - For environment setup, see
references/00-overview.md
Authentication (important)
Before using PyTRIO, authentication must be completed first; otherwise all API calls will fail.
Check whether already logged in: check whether ~/.pytrio/config.toml exists and contains the api_key field.
If not logged in, follow the process below:
- Prompt the user to provide an API Key (available from https://pytrio.cn/dashboard obtain)
- After the user provides the key, run the non-interactive login command:
trio login -k <用户提供的API_KEY> - After successful login, credentials are saved in
~/.pytrio/config.toml, and subsequentServiceClient()do not need api_key passed
It can also be passed directly in code (suitable for temporary use):
client = ServiceClient(api_key="YOUR_API_KEY")
Note: the CLI command is trio login, not pytrio login。
PyTRIO ↔ PyTorch concept quick reference
| PyTorch | PyTRIO | Description |
|---|---|---|
model = AutoModel(...) |
client = ServiceClient(api_key=...) |
No local model object |
model(input_ids) |
train.forward(data) |
Remote forward pass |
loss.backward() |
train.forward_backward(data) |
Forward + backward combined |
optimizer.step() |
train.optim_step(AdamParams(...)) |
Remote Adam |
optimizer.zero_grad() |
Not required | Automatically managed remotely |
torch.save(...) |
train.save_state(name) |
Save to the cloud |
model.generate(...) |
sampler.sample(prompt, ...) |
Remote inference |
torch.Tensor |
TensorData / plain list |
Automatic mutual conversion |
Core API quick reference
Entry point
import pytrio as trio
client = trio.ServiceClient(api_key="...")
models = client.get_supported_models()
Model selection rule: must use Qwen/Qwen3-4B-Instruct-2507, do not use Qwen/Qwen3-4B(unavailable on the Base model training side).
Training
import pytrio as trio
train = client.create_lora_training_client(
base_model="Qwen/Qwen3-4B-Instruct-2507",
rank=32, # [4, 64]
train_mlp=True,
train_attn=True,
train_unembed=False,
)
tokenizer = train.get_tokenizer()
# tokenizer 注意事项(官方示例的标准模式):
# 序列起始段(prompt)用 add_special_tokens=True —— 保留 BOS
# 拼接延续段(completion)用 add_special_tokens=False —— 避免重复 BOS
# 推理时 encode 默认 add_special_tokens=True,保留 BOS 作为生成起始
# apply_chat_template() 已自行管理 special tokens,返回文本再 encode 时也用 False
# 官方标准 SFT 数据构造模式:
prompt_tokens = tokenizer.encode(prompt_text, add_special_tokens=True)
prompt_weights = [0] * len(prompt_tokens)
completion_tokens = tokenizer.encode(completion_text, add_special_tokens=False)
completion_weights = [1] * len(completion_tokens)
tokens = prompt_tokens + completion_tokens
weights = prompt_weights + completion_weights
# 手动偏移对齐: model_input[i] 预测 target_tokens[i]
input_tokens = tokens[:-1]
target_tokens = tokens[1:]
weights = weights[1:] # weights 必须同步偏移
sample = trio.Datum(
model_input=trio.ModelInput.from_ints(input_tokens), # 必须用 ModelInput.from_ints() 包装
loss_fn_inputs={
"target_tokens": target_tokens, # 值可直接传 list,会自动转 TensorData
"weights": weights, # 可选,默认全 1;SFT 场景强烈建议传
},
)
# 训练步骤
fb = train.forward_backward(data=[sample], loss_fn="cross_entropy")
result = fb.result() # APIFuture -> ForwardBackwardOutput
# result.metrics 的 key 是 "loss:sum"(注意冒号),不是 "loss"
# per-token 归一化 loss(两种等价写法,需 import numpy as np):
# 方式 A: loss_sum / total_weight_sum
# total_weight_sum = sum(sum(ex.loss_fn_inputs["weights"].tolist()) for ex in data)
# 方式 B: -np.dot(logprobs, weights) / weights.sum()
# logprobs = np.concatenate([o["logprobs"].tolist() for o in result.loss_fn_outputs])
# weights = np.concatenate([ex.loss_fn_inputs["weights"].tolist() for ex in data])
loss_sum = result.metrics.get("loss:sum", 0.0)
train.optim_step(trio.AdamParams(learning_rate=1e-4)).result()
Loss function
| loss_fn | loss_fn_inputs required | Purpose |
|---|---|---|
"cross_entropy" |
target_tokens, optional weights |
SFT |
"importance_sampling" |
target_tokens, logprobs, advantages,Strongly recommended weights |
GRPO / offline RL |
"ppo" |
target_tokens, logprobs, advantages,Strongly recommended weights |
Online RL |
For RL scenarios, the weights is used to mask the prompt part (0=ignore, 1=count toward loss); if not passed, the prompt will also participate in the policy gradient, which usually causes it to go off track. All official RL examples include weights。
ppo can be set via loss_fn_config customize the clipping threshold (default ε=0.2, i.e. ratio clipped to [0.8, 1.2]):
train.forward_backward(
data=data, loss_fn="ppo",
loss_fn_config={"clip_low_threshold": 0.9, "clip_high_threshold": 1.1},
)
Custom loss (requires local torch):
def my_loss(samples, logprobs_list):
loss = ... # torch.Tensor
return loss, {"metric": loss.item()}
train.forward_backward_custom(data=samples, loss_fn=my_loss)
Save and restore
# 保存权重(仅用于推理)
save = train.save_weights_for_sampler("name").result()
# save.path -> 远端路径
# 保存完整 checkpoint(含优化器,用于断点续训)
state = train.save_state("name").result()
# 断点续训
train2 = client.create_training_client_from_state_with_optimizer(state.path)
# 仅加载权重续训(不含优化器动量)
train3 = client.create_training_client_from_state(state.path)
Inference
import pytrio as trio
# 从训练权重推理
sampler = train.create_sampling_client(model_path=save.path)
# 或独立推理
sampler = client.create_sampling_client(base_model="Qwen/Qwen3-4B-Instruct-2507")
tokenizer = sampler.get_tokenizer()
input_ids = tokenizer.encode("Hello")
result = sampler.sample(
prompt=trio.ModelInput.from_ints(input_ids), # 必须用 trio.ModelInput 包装
num_samples=1,
sampling_params=trio.SamplingParams(max_tokens=100, temperature=0.7),
).result()
text = result.sequences[0].text
logprobs = result.sequences[0].logprobs
Management
rest = client.create_rest_client()
rest.list_weights() # 已保存权重
rest.list_training_runs() # 训练记录
rest.list_checkpoints(run_id) # 某次训练的 checkpoint
rest.list_user_checkpoints() # 所有 checkpoint
rest.get_archive_url(id) # 下载链接
rest.delete_checkpoint(run_id, cp_id)
AdamParams SDK default values
Note they differ from PyTorch defaults:beta2=0.95(PyTorch default 0.999),eps=1e-12(PyTorch default 1e-8). To match PyTorch behavior, explicitly pass AdamParams(beta2=0.999, eps=1e-8)。
Error handling
APIFuture.result(timeout=None)supports a timeout parameter (seconds); on timeout throwsTimeoutError- Server-side exceptions throw
pydantic.ValidationErrororValueError - WebSocket disconnections are not automatically reconnected; you need to rebuild
ServiceClient
Pitfall checklist
- All methods return
APIFuture— you must call.result()to get the actual value, otherwise you get a Future object - model_input and target_tokens must have the same length — when
auto_shift=False(default), the user is responsible for offset alignment importance_sampling/pporequires four equal-length fields —model_input,target_tokens,logprobs,advantagesmust be equal lengthtrainable_token_indicesandtrain_unembedare mutually exclusive — cannot be enabled at the same timeignore_eos=Truemust be set whenmax_tokens— otherwise an exception is thrown- the checkpoint path is a remote path —
save_state()/save_weights_for_sampler()the returned path is a cloud path, not a local file forward_backward_customrequires local torch — the custom loss function computes gradients locally and sends them back to the remote- Import convention
import pytrio as trio— all APIs are accessed viatrio.xxx(e.g.trio.ServiceClient、trio.Datum), do not usefrom pytrio import ... - the login command is
trio login— the CLI command is stilltrio, notpytrio - all values of loss_fn_inputs are automatically converted to TensorData — you can directly pass a list, numpy array, or torch tensor
- model_input must be wrapped with
trio.ModelInput.from_ints(list)Wrapping — you cannot directly passlist[int], otherwise Pydantic validation will report an error - similarly, the prompt parameter must be wrapped with
trio.ModelInput.from_ints(list)Wrapping —sampler.sample(prompt=trio.ModelInput.from_ints(ids), ...) - tokenizer uses
get_tokenizer()—train.get_tokenizer()orsampler.get_tokenizer()(requires installing modelscope) - at runtime you need to bypass the local proxy — if there is a system proxy, you need to set the
no_proxy=pytrio.cn https_proxy="" http_proxy=""environment variable - you must use the Instruct model —
Qwen/Qwen3-4B(Base) is unavailable on the training side, you must useQwen/Qwen3-4B-Instruct-2507 - the naming differs from the HuggingFace API — PyTRIO's parameter names differ from HF transformers, do not mix them up:
- the sampling parameter is
max_tokens(notmax_new_tokens) - the inference result is
result.sequences[0].text(notresult.samples) - forward propagation must use the keyword argument
train.forward_backward(data=[...])(not positional arguments) - the optimizer parameter is
learning_rate(notlr) - the key of loss_fn_inputs is
target_tokens(notlabels) - use
import pytrio as trioinstead offrom pytrio import ... - do not import internal modules (such as
pytrio.types.forward.encoded_text_chunk)
- the sampling parameter is
- model_input and target_tokens must be offset by one —
model_input=tokens[:-1],target_tokens=tokens[1:], you cannot pass the same tokens - the loss key is
"loss:sum"(with a colon) —fb_result.metrics.get("loss:sum"), not"loss"、"train_loss"or.lossattribute - do not use -100 for token masking — HF uses
-100to mark ignored tokens, but PyTRIO does not support this mechanism. PyTRIO uses theweightsfield to control which tokens participate in loss computation (0.0=ignore, 1.0=compute), seebest-practices/sft.md sampling_paramsmust pass atrio.SamplingParamsobject — you cannot pass a dict,sampler.sample(sampling_params=trio.SamplingParams(max_tokens=100))add_special_tokensdistinguished by segment — the prompt start segment usesTrue(preserve BOS), the completion concatenation segment usesFalse(avoid duplicate BOS). Do not make them all True (this will add extra BOS), and do not make them all False (the model loses its start marker and will train incorrectly)weightsmust be offset in sync withtarget_tokensSynchronous offset — the official standard pattern is to first constructfull_weights, thenweights = weights[1:], strictly aligned withtarget_tokens = tokens[1:]Strict alignment- RL scenarios (IS/PPO) also need to pass
weights— mask out the prompt part, otherwise prompt tokens will also participate in the policy gradient - The asynchronous version ends with
_async— for high concurrency / RL rollout, the asynchronous version is recommended, see — for high concurrency / RL rollout, the asynchronous version is recommended, seebest-practices/async.md
Detailed documentation
The complete API documentation is in this Skill's references/ directory:
references/00-overview.md— Architecture overview and concept mappingreferences/01-service-client.md— ServiceClient APIreferences/02-training-client.md— TrainingClient APIreferences/03-sampling-client.md— SamplingClient APIreferences/04-rest-client.md— RestClient APIreferences/05-data-types.md— Data type referencereferences/06-debug.md— Debugging guide (triage process, error decoding, self-check checklist)
Example code
Runnable minimal closed-loop examples are in this Skill's examples/ directory:
examples/01_train_sft.py— SFT trainingexamples/02_inference.py— Inferenceexamples/03_checkpoint_resume.py— Training resumption from checkpointexamples/04_model_management.py— Model managementexamples/05_importance_sampling.py— GRPO-style training
Scenario best practices
Recommended code templates and notes for specific training scenarios are in the best-practices/ directory:
best-practices/sft.md— SFT supervised fine-tuning: prompt masking (weights usage), EOS appending, tokenizer optionsbest-practices/grpo.md— GRPO/PPO reinforcement learning: real rollout→reward→forward_backward closed loopbest-practices/async.md— Asynchronous training:_asyncAPI、asyncio.gather, concurrent scheduling
Deployment
references/07-openai-compat.md— After training, the model provides external services through an OpenAI-compatible API (chat.completions/completions)