byted-volcengine-tosutil

Volcano Engine tosutil Skill

This Skill targets the tosutil command-line tool of Volcano Engine Object Storage TOS, and is responsible for converting user intent into a safe, executable, and verifiable tosutil operation workflow.

One-sentence goal

Convert the requirement of "what operation to perform on TOS" intopreview only by defaultof tosutil commands, and when execution is needed, outputstructured JSON results + reproducible evidence + diagnosable suggestions。

Input and Output

Input: target command (such as ls/cp/rm/du/setmeta) + minimum necessary parameters (such as tos:// address, local path, recursive switch) + optional common parameters (endpoint/region/credentials/conf）。
Output: unified JSON protocol (ok/code/message/data/ts), including preview.shell(the reproducible command after desensitization), execution summary, and on failure the advice.code and next_actions。

Default behavior (reduce user cost + safety)

By default, only a command preview is generated, without execution (explicit --run is required to execute).
Destructive commands (such as rm) are not executed by default: explicit --yes(or in legacy mode --assume-yes) is required for them to actually run.
Output is desensitized by default: AK/SK/Token will not be echoed.

When to Use

Invoke this Skill when the user requests any of the following scenarios:

Use tosutil to initialize or update the TOS configuration
Create buckets, list buckets/objects, query object attributes
Upload, download, copy, and batch delete objects
Calculate object capacity, set object metadata
Analyze Http status [403], connectivity failures, command parameter errors, and other issues
Generate execution plans for advanced commands such as share、set-acl、mount、probe、netdig and other advanced commands to generate an execution plan

Documentation Fact Baseline

Based on the tosutil documentation system, the following facts can be confirmed:

tosutil is a command-line tool for accessing and managing Volcano Engine Object Storage TOS, suitable for batch data processing between local and TOS, script integration, and small-to-medium data migration.
Core commands include ls、mkdir、du、mb、cp、setmeta、stat、rm、share、set-acl、mount。
Auxiliary commands include config、help、probe、netdig、hash、fcp、clear、version、ping、connect、traceroute、curl。
Initialization configuration supports three methods: permanent keys, STS temporary keys, and anonymous access.
During initialization, the TOS protocol domain should be used, not the S3 protocol domain.
rm has secondary confirmation by default; recursive and batch deletion must explicitly evaluate risks.
du may take a long time with millions of objects; prioritize splitting calculations by directory.

Download and Installation

tosutil supports Windows, Linux, and macOS. Before using this Skill, it is recommended to first download the corresponding version according to the current operating system and chip architecture, and complete execution permission settings.

Official Download Recommendations

Linux amd64: supports directly downloading the binary and executing it
macOS amd64 (Intel): supports directly downloading the binary and executing it
macOS arm64 (Apple M series chips): supports directly downloading the binary and executing it
Windows 64bit: download tosutil.exe
The official also provides corresponding sha256 Verify the file; it is recommended to perform an integrity check after downloading
Current tosutil The latest version is mainly applicable to Windows, macOS, and Linux amd systems

Installation command

Linux：

wget https://m645b3e1bb36e-mrap.mrap.accesspoint.tos-global.volces.com/linux/amd64/tosutil
chmod a+x tosutil
sudo mv tosutil /usr/local/bin

macOS Intel：

wget https://m645b3e1bb36e-mrap.mrap.accesspoint.tos-global.volces.com/darwin/amd64/tosutil
chmod a+x tosutil
sudo mv tosutil /usr/local/bin

macOS Apple Silicon：

wget https://m645b3e1bb36e-mrap.mrap.accesspoint.tos-global.volces.com/darwin/arm64/tosutil
chmod a+x tosutil
sudo mv tosutil /usr/local/bin

Windows：

wget https://m645b3e1bb36e-mrap.mrap.accesspoint.tos-global.volces.com/windows/tosutil -O tosutil.exe

Installation notes

macOS may block apps from unidentified developers by default; if the first execution of tosutil is blocked by the system, you need to allow it in the system security settings
After downloading on Linux / macOS, you usually need to run chmod a+x tosutil
If you want to execute directly in any directory tosutil, it is recommended to move the binary to a directory already added to PATH , for example /usr/local/bin
If the binary has not been added to PATH, please call it using an absolute path, for example /absolute/path/to/tosutil version
When this Skill is not added to PATH , it is recommended to use --tosutil-binary <absolute-path> to explicitly specify the binary location to avoid the tool not being found

Post-installation verification

If already added to PATH：

tosutil version
tosutil config
tosutil ls

If not added to PATH：

/absolute/path/to/tosutil version
/absolute/path/to/tosutil config
/absolute/path/to/tosutil ls

Combined with the recommendations of this Skill:

First use version to verify whether the binary is executable
Then use config to confirm the configuration file path
Finally use ls to verify credentials, region, and network connectivity

If tosutil has been added to PATH, you can call this Skill like this:

python3 .trae/skills/byted-volcengine-tosutil/scripts/main.py \
  ls \
  --preflight

If tosutil has not been added to PATH, you can call this Skill like this:

python3 .trae/skills/byted-volcengine-tosutil/scripts/main.py \
  ls \
  --tosutil-binary /absolute/path/to/tosutil \
  --preflight

Working principles

First confirm whether the goal is a "read operation" or a "write operation/delete operation".
Prefer generating the minimum viable command to avoid concatenating too many dangerous parameters at once.
For batch upload, download, and copy tasks, confirm the concurrency and part thresholds before executing.
For deletion tasks, provide pre-check steps and rollback prompts by default, and do not skip confirmation directly.
When encountering advanced commands and the parameters are not fully confirmed, prioritize combining with tosutil help <command> for verification, and do not fabricate parameters.

Standard process

1. Identify the scenario

Classify the user request into one of the following scenarios:

Initialization configuration:config
Bucket operations:ls、mb、stat
Object transfer:cp
Object deletion:rm
Object metadata:setmeta
Capacity statistics:du
Fault diagnosis:version、ls、help、probe、netdig、ping、connect、traceroute

2. Pre-check

Execute or recommend the following checks:

tosutil version
tosutil config
tosutil ls

Check focus:

Whether the tool is installed and executable
Endpoint Whether it is a TOS protocol domain name
Region Whether it is consistent with the target bucket region
AK/SK or STS Token Whether it exists and has sufficient permissions
Whether Bucket number is:、Http status [403]、A connection attempt failed

appears in the returned result

3. Parameter normalization

Resource address: local path,tos://bucket、tos://bucket/prefix
Credential mode: permanent key, STS, anonymous access
Common parameters:-e、-re、-i、-k、-t、-conf
Bucket type:-bt=fns|hns
Batch task concurrency:-j
Part concurrency or part task control:-p、-threshold、-ps
Result output directory:-o

4. Command generation

Automatically select the command mode based on the resource type:

Local -> TOS: upload
TOS -> Local: download
TOS -> TOS: Object copy
Single object: single-task mode
Directory or prefix: recursive mode -r
Large files: switch to multipart tasks based on the threshold

5. Output validation

Parse the following signals in the execution result:

Success indicator:successfully、Bucket number is:、Succeed count is:、Task id is:
Permission issue:Http status [403]
Network issue:A connection attempt failed
Parameter issue: command help output, missing required parameters, invalid path format
Cleanup suggestion: when resumable transfer fails, consider clear

Command mapping

Initialize configuration

Permanent key (recommended):

tosutil config -i <ak> -k <sk> -e <endpoint> -re <region>

For endpoint and region, refer to "Appendix: Regions and Access Domains". Prefer the internal endpoint; if the internal endpoint is unavailable, use the public endpoint

STS：

tosutil config -i <ak> -k <sk> -t <token> -e <endpoint> -re <region>

Anonymous access:

tosutil config -i= -k= -t= -e <endpoint> -re <region>

Common bucket and object commands

tosutil ls
tosutil mb tos://bucketname
tosutil cp /local/file.txt tos://bucketname/file.txt
tosutil cp tos://bucketname/file.txt /local/file.txt
tosutil rm tos://bucketname/file.txt
tosutil du tos://bucketname
tosutil setmeta tos://bucketname/object.png -meta aaa:bbb#ccc:ddd

Security policy

Before deleting an object, first determine whether it is a single object, directory prefix, or bucket-level deletion.
For rm -r、rm -f, batch metadata updates, batch copies, and similar operations, first output a description of the impact scope.
If the user only requests "generate a command", do not execute it directly by default.
If a high-risk command needs to be executed, first suggest listing the target objects or performing a dryRun -style validation; if the command itself does not support dryRun, first perform a read-only check.

Integration implementation suggestions

The implementation of this Skill is mainly based on a "local CLI wrapper layer" rather than "directly calling the TOS HTTP API", because the main body of the documentation provides the tosutil command interface.

Script entry point (recommended subcommand mode)

Command preview (do not execute):

python3 .trae/skills/byted-volcengine-tosutil/scripts/main.py ls --cloud-url tos://bucketname

Execute and return structured results:

python3 .trae/skills/byted-volcengine-tosutil/scripts/main.py ls --cloud-url tos://bucketname --preflight --run

High-risk deletion (explicit confirmation required):

python3 .trae/skills/byted-volcengine-tosutil/scripts/main.py rm --cloud-url tos://bucketname/prefix/ --recursive --run --yes

Compatible with legacy entry point (legacy):

python3 .trae/skills/byted-volcengine-tosutil/scripts/main.py --command ls --cloud-url tos://bucketname

Recommended directory structure:

.trae/skills/byted-volcengine-tosutil/
├── SKILL.md
├── references/
│   └── doc-survey.md
└── scripts/
    ├── main.py
    ├── models.py
    ├── result_handler.py
    └── tosutil_service.py

Recommended core classes:

CommonOptions / CredentialMode: Encapsulates permanent keys, STS, anonymous mode, and common parameters
TosResource: Encapsulates the parsing results of local paths and tos:// resources
CommandSpec: Encapsulates the command name, parameter list, risk level, and whether it is a destructive operation
TosutilRunner: Uniformly executes tosutil commands and collects the exit code, standard output, and standard error
result_handler: Uniformly responsible for result parsing, error mapping, and organizing structured output fields
tosutil_service: Uniformly responsible for command construction, pre-checks, execution, masking, and common validation logic

Key rules

Configuration rules

config writes to the local configuration file, which is located by default under the user directory at .tosutilconfig
-conf can only point to an existing configuration file path, and is not responsible for automatically creating new files
Prefer explicitly using region + endpoint, to avoid cross-region misoperations

Transfer rules

cp needs to automatically infer whether it is an upload, download, or cloud copy based on the source and destination address types
The actual concurrency of batch tasks needs to be evaluated comprehensively with -j and the multipart concurrency parameters, to avoid blindly increasing them
For large files and batch tasks, explicitly set the threshold, concurrency, and output directory first, to facilitate troubleshooting

Statistics rules

du In scenarios with massive numbers of objects, do not perform a full-bucket recursive scan by default
If versioning is enabled for the bucket, version-related parameters can be added as needed and the statistics scope split

Metadata rules

setmeta Supports single-object and batch prefix modes
Batch mode must explicitly -r, and limit maximum concurrency -j

Troubleshooting tips

Permission issues

If the result shows Http status [403], first check whether the AK/SK and STS have expired, as well as the target bucket/object permissions

Network issues

If the result shows A connection attempt failed, first check the network, proxy, firewall, and Endpoint

Parameter issues

If the advanced command parameters are uncertain, first run tosutil help <command>

Cleanup issues

After resumable transfer or an abnormal interruption, consider clear cleaning up the record file and making every effort to clean up the cloud side

Best practices

When taking over an environment for the first time, first run version、config、ls triple check
Initialization must use the TOS protocol domain name
Anonymous access is only applicable to public read or public write scenarios
Split million-level object statistics by directory
For batch tasks, set concurrency conservatively first, then tune gradually
Keep secondary confirmation and impact scope explanation for high-risk commands

Interaction templates

The user wants to upload a file

Confirm the local path, target bucket, and target object name
Determine whether recursive upload is needed
Check configuration and connectivity
Generate the cp command
Return success criteria and failure troubleshooting points

The user wants to delete a directory

First confirm the target prefix
First suggest ls to verify the impact scope
Then generate rm -r
If the user confirms forced deletion, then consider -f

The user wants to troubleshoot inability to access

Check version
Check config
Run ls
Based on 403, connection failures, and parameter errors, perform classified diagnosis

Testing requirements

Use documentation examples as command construction test samples
Use example output as parser snapshot samples
For 403, network failures, empty configuration, anonymous access, recursive deletion, and other scenarios, perform unit tests
For cp , perform integration tests separately for the three directions: upload/download/cloud-side copy
For rm、setmeta、du , perform boundary tests on batch parameters

Limitations

This Skill prioritizes covering the most core config、ls、mb、cp、rm、du、setmeta、stat、help、version
share、set-acl、mount、probe、netdig、curl and other advanced commands in the documentation using an extended adaptation mode
If the parameters of a certain advanced command are not fully expanded in the current reference materials, you must first confirm through help or supplementary documentation before executing

Appendix: Regions and Access Domain Names

Region: indicates the physical location of the TOS data center.
Endpoint: indicates the access domain name for TOS external services.

Region Chinese name	Region ID	Endpoint (internal/external)	S3 Endpoint (internal/external)
North China 2 (Beijing)	cn-beijing	Internal: tos-cn-beijing.ivolces.comExternal: tos-cn-beijing.volces.com	Internal: tos-s3-cn-beijing.ivolces.comExternal: tos-s3-cn-beijing.volces.com
South China 1 (Guangzhou)	cn-guangzhou	Internal: tos-cn-guangzhou.ivolces.comExternal: tos-cn-guangzhou.volces.com	Internal: tos-s3-cn-guangzhou.ivolces.comExternal: tos-s3-cn-guangzhou.volces.com
East China 2 (Shanghai)	cn-shanghai	Internal: tos-cn-shanghai.ivolces.comExternal: tos-cn-shanghai.volces.com	Internal: tos-s3-cn-shanghai.ivolces.comExternal: tos-s3-cn-shanghai.volces.com
Hong Kong, China	cn-hongkong	Internal: tos-cn-hongkong.ivolces.comExternal: tos-cn-hongkong.volces.com	Internal: tos-s3-cn-hongkong.ivolces.comExternal: tos-s3-cn-hongkong.volces.com
Asia Pacific Southeast (Johor)	ap-southeast-1	Internal: tos-ap-southeast-1.ivolces.comExternal: tos-ap-southeast-1.volces.com	Internal: tos-s3-ap-southeast-1.ivolces.comExternal: tos-s3-ap-southeast-1.volces.com
Asia Pacific Southeast (Jakarta)	ap-southeast-3	Internal: tos-ap-southeast-3.ivolces.comExternal: tos-ap-southeast-3.volces.com	Internal: tos-s3-ap-southeast-3.ivolces.comExternal: tos-s3-ap-southeast-3.volces.com