byted-security-llmscanner

User Guide for the Large Model Security Evaluation Full-Process Management Skill

Trigger Conditions

This skill is triggered when a user sends instructions containing keywords such as "create asset", "update asset", "initiate evaluation", "analyze evaluation", or "query resources".

Script Location Description

Script directory: ~/.openclaw/workspace/skills/byted-security-llmscanner/scripts/

Execution method:

cd ~/.openclaw/workspace/skills/byted-security-llmscanner/scripts
npx ts-node src/create_asset_model.ts <参数...>

All the following examples require entering the script directory first

⚠️ Common Pitfalls

1. Asset ID vs Asset Name

update_asset_model.ts The first parameter of Name(asset name), not AssetID. The script automatically queries the asset ID based on the name and ModelID.
update_asset_agent.ts The parameter of <AgentID> <Name> <PlatformID>, and both the asset ID and name must be provided.

2. Script Code Issues

Some scripts may have duplicate property issues (such as duplicate proxy: false ), causing TypeScript compilation errors. If such errors occur, the script code needs to be checked and fixed.

3. Modality Type Matching

The modality type of the asset must include the modality type of the evaluation set; otherwise, the evaluation cannot be initiated.
For example: if the evaluation set is text2text, the asset's modality type can be text2text or text2text,image2text, but cannot be only image2text。

4. Agent Platform Parameters

Different agent platforms require different parameters, and you must first use list_agent_platforms.ts vars <PlatformID> to query the parameters required by that platform.

5. Applicable Scope of Security Evaluation Scripts

Some security evaluation scripts are specifically for OpenClaw, while others are general-purpose. When querying scripts, the assetTypes[] field will be displayed; if it is empty, it means the script is general-purpose.

General Enumerations

Modality Type

text2text: Text-to-Text
image2text: Image-Text-to-Text
text2image: Text-to-Image

Asset Type

model: Large Model
agent: Agent
openclaw: OpenClaw Asset

Compliance Evaluation Task Status

Status Code	Status Name	Description
0	Pending Evaluation	The task has been created and is waiting to start execution
1	Evaluating	The task is being executed
2	Evaluation Successful	Task execution completed (final state)
3	Evaluation Abnormal	Task execution failed (final state)
5	Evaluation Paused	The task was paused (final state)
6	Evaluation Terminated	The task was terminated (final state)

Security Evaluation Task Status

Status Code	Status Name	Description
10	Waiting for Processing	The task has been created and is waiting to start execution
20	Processing	The task is being executed
30	Abnormal	Task execution failed (final state)
40	Completed	Task execution completed (final state)

Asset Management

1. Create a Large Model Asset

Call:

npx ts-node src/create_asset_model.ts <Name> <ModelID> <BaseUrl> <ApiKey> [ModalTypes]

Parameters:

Name: Asset name (e.g., deepseek-v3)
ModelID: Model ID (e.g., ep-20250325142301-ljxvm)
BaseUrl: API address (e.g.,https://ark-cn-beijing.bytedance.net/api/v3）
ApiKey: API key (e.g., sk-xxxxxxxxxxxx)
ModalTypes: Modality type (optional, comma-separated, default: text2text)

Example:

npx ts-node src/create_asset_model.ts deepseek-v3 ep-20250325142301-ljxvm https://ark-cn-beijing.bytedance.net/api/v3 sk-xxxxxxxxxxxx text2text

2. Update large model asset

Call:

npx ts-node src/update_asset_asset_model.ts <Name> <ModelID> <BaseUrl> <ApiKey> [ModalTypes]

Parameters:

Name: Asset name (⚠️ Note: not AssetID, the script will query automatically)
ModelID: Model ID
BaseUrl: API address
ApiKey: API key
ModalTypes: Modality type (optional, comma-separated, default: text2text)

Example:

npx ts-node src/update_asset_model.ts deepseek-v3 ep-new-model-id https://ark-cn-beijing.bytedance.net/api/v3 sk-new-api-key text2text

⚠️ Note:

The first parameter is Name(asset name), not AssetID
The script will, based on Name and ModelID , automatically query the asset ID

3. Query agent platform

Call:

# 查询所有平台
npx ts-node src/list_agent_platforms.ts list

# 查询平台需要的变量
npx ts-node src/list_agent_platforms.ts vars <PlatformID>

⚠️ Note:

Before creating an agent asset, you must first query the parameters required by the platform
Different platforms require different parameters; for example, the dify platform requires api_key, and the hiagent platform may require other parameters

4. Create agent asset

Call:

npx ts-node src/create_asset_agent.ts <Name> <PlatformID> [options]

Parameters:

Name: Agent asset name
PlatformID: Platform ID (queried via list_agent_platforms.ts list )
--key=value: Agent conversation variables (multiple allowed, JSON format supported)

Example:

npx ts-node src/create_asset_agent.ts 客服助手 b0226c4550aa4791a8c19a118a5f8ef5 --api_key=app-xxxxxxxxxxxx

⚠️ Note:

You must first use list_agent_platforms.ts vars <PlatformID> to query the parameters required by the platform
The parameter format is --key=value, for example --api_key=app-xxx、--user_id=admin

5. Update agent asset

Call:

npx ts-node src/update_asset_agent.ts <AgentID> <Name> <PlatformID> [options]

Parameters:

AgentID: Asset ID
Name: Agent asset name
PlatformID: Platform ID
--key=value: Agent conversation variables (multiple allowed, JSON format supported)

Example:

npx ts-node src/update_asset_agent.ts asset_9876543210 客服助手 b0226c4550aa4791a8c19a118a5f8ef5 --api_key=app-zzzzzzzzzzzz

⚠️ Note:

The parameter order is <AgentID> <Name> <PlatformID>, which differs from creating an agent
Both the asset ID and name must be provided

Resource query

6. Query available resources

Call:

# 查询合规测评资源（测评集、资产）
npx ts-node src/list_resources.ts [ModalTypes]

# 查询安全测评资源（资产、剧本）
npx ts-node src/fetch_lists.ts [ModalTypes]

# 查询安全测评剧本
npx ts-node src/list_rt_scenarios.ts [AssetType]

Parameters:

ModalTypes: Modality type (optional, comma-separated, e.g., text2text,image2text)
AssetType: Asset type (optional, used to filter scripts)

Example output:

=== 可用测评集 ===
ID: suite_001, 名称: 安全合规测评集, 模态: 文生文

=== 可用大模型资产 ===
ID: asset_001, 名称: deepseek-v3, 模态: 文生文

=== 可用智能体资产 ===
ID: asset_003, 名称: 客服助手, 模态: 文生文

⚠️ Note:

When querying resources, the asset's modality type will be displayed; ensure the asset's modality type includes the evaluation set's modality type
For example: if the evaluation set is text2text, the asset's modality type can be text2text or text2text,image2text

Compliance evaluation

7. Initiate a compliance evaluation task

Call:

npx ts-node src/create_task.ts <测评集ID> <资产ID> [AssetType]

Parameters:

测评集ID: The unique identifier of the evaluation set
资产ID: The unique identifier of the evaluation target (model, agent, or OpenClaw)
AssetType: Asset type (optional, default: model)

⚠️ Note:

The asset's modality type must include the evaluation set's modality type, otherwise the evaluation cannot be initiated
If the asset modality type does not match, an error will be returned

8. Analyze compliance assessment task

Call::

npx ts-node src/run_analysis.ts <TaskID>

Parameters:

TaskID: Task ID (format e.g.: 019ce0495b68758ea00fda8fc63b2ba0)

Function:

Query task status
If the task completed successfully, automatically pull data and analyze
If the task is not complete, display execution progress and estimated time
If the task is abnormal/paused/terminated, display the status and indicate that analysis cannot be performed

Output (task completed):

任务状态：测评成功（状态码：2）
数据总量：76 条
安全通过：45 (59.2%)
发现风险：31 (40.8%)

Output (task not completed):

任务状态：测评中（状态码：1）
执行进度：30/76 (39.5%)
预计剩余时间：1分钟

Security assessment

9. Initiate security assessment task

Call:

npx ts-node src/create_rt_task.ts <TaskName> <AssetID> <AssetType> <ScenarioID>

Parameters:

TaskName: Task name (e.g.: prompt injection test)
AssetID: Assessment target ID (model, agent, or OpenClaw)
AssetType: Asset type (model/agent/openclaw)
ScenarioID: Security assessment script ID (via list_rt_scenarios.ts query)

⚠️ Note:

Some scripts are specifically for OpenClaw, some are generic
When querying scripts, the assetTypes[] field will be displayed; if empty, it indicates generic
If it is an OpenClaw-specific script, the asset type must be openclaw

10. Analyze security assessment task

Call:

npx ts-node src/analyze_rt_task.ts <TaskID>

Parameters:

TaskID: Task ID

Function:

Query security assessment task report
Automatically pull report data and analyze

Output:

任务状态：完成
资产名称：deepseek-v3
资产类型：model

【按测评结果分】
  失陷（Fall）：5 个
  风险（Risk）：10 个
  总计：15 个

【按严重度分】
  🔴 高：3 个
  🟡 中：7 个
  🟢 低：5 个

【风险详情】
  风险名称：提示词注入成功
  严重程度：🔴 HIGH
  评分：8/10

通过率：80.0% (60/75)
失败率：20.0% (15/75)

⚠️ Note:

If the task status is 20(processing), you need to wait for the task to complete before analyzing
You can run this command multiple times to query task progress

About this skill

Trigger Conditions

Script Location Description

⚠️ Common Pitfalls

1. Asset ID vs Asset Name

2. Script Code Issues

3. Modality Type Matching

4. Agent Platform Parameters

5. Applicable Scope of Security Evaluation Scripts

General Enumerations

Modality Type

Asset Type

Compliance Evaluation Task Status

Security Evaluation Task Status

Asset Management

1. Create a Large Model Asset

2. Update large model asset

3. Query agent platform

4. Create agent asset

5. Update agent asset

Resource query

6. Query available resources

Compliance evaluation

7. Initiate a compliance evaluation task

8. Analyze compliance assessment task

Security assessment

9. Initiate security assessment task

10. Analyze security assessment task