CRITICAL — Before starting, MUST first use the Read tool to read ../arkcli-shared/SKILL.md, which contains the authentication gate, API Key error recovery, configuration troubleshooting, and command selection order.
CRITICAL — Before executing +understand , MUST first use the Read tool to read references/arkcli-understand.md(commands/flags/return values/errors) and references/sub-skills.md(the purpose and expected output form of the 12 sub-skills). Do not assemble commands from memory.
Core Concepts
+understandNot 12 implementations, but 1 engine + a layer of semantics: the underlying engine is the data-plane Responses API used by+chat; each sub-skill is just onerecipe{模态, fallback 模型, 内置 system prompt}. When the Platform Profile omits--model, the current Profile'sResources.Text.DefaultEndpoint must be used; only Plan-type Profiles use the recipe fallback model.- Command form:
arkcli +understand <sub-skill> --input @file [prompt]。args[0]Hits the registry(one of the 12) → treated as an explicit sub-skill, and the remaining positional arguments are treated as a prompt layered on top of the built-in prompt.args[0]No hit → the entire positional argument string is treated as the prompt, and the server automatically routes by the file modality of thefirst--inputautomatic file modality routingto that modality's default sub-skill.
- Must have
--input: a sub-skill is "understanding a certain file"; without--inputit will directlymissing_inputerror out. A pure prompt cannot derive a recipe. - Return values are completely identical to
+chat: arkcli flat schema{id, model, content, reasoning_content, usage}，is not Responses nativeoutput[].content[].textnesting. See the "Return Values" section ofreferences/arkcli-understand.md. - Multimodal upload: image/video/doc are automatically routed through the Files API by the SDK
file://preprocessor;audio is a special case—inlined as a base64 data URL,with a 25MB limit. See reference for details. - When the user temporarily provides
--api-key/--base-url/ Endpoint, MUST read../arkcli-shared/references/execution-context.md. Do not guess the data plane/workflow based on the Key text or Endpoint name. --dry-runOnly resolves recipes locally, the default Endpoint already persisted in the current Profile, explicit models, and input references, and outputs a unifiedpreview.v1; it does not read online Endpoint/model metadata, does not upload files, does not call the Responses API, does not generate model usage, and does not create a response id or persist responses. Values that can only be filled in online must be marked asunresolved, and the preview must not be treated as server-side validation.
Quick decision (understand vs chat vs gen)
| User intent | Which one to use |
|---|---|
| Hasexplicit output formmultimodal understanding tasks (transcription / translation / subtitles / box localization / GUI operation / field extraction / chapter-by-chapter summarization / multi-speaker / meeting minutes) | +understand(hits a certain sub-skill) |
Open-ended dialogue with images/videos, follow-up questions, reasoning, requiring multi-turn continuation (--store/--previous-response-id), requiring tools (web_search/function), or --text-format json_schema strict JSON |
../arkcli-chat/SKILL.md |
| Generateimages / videos (not understanding existing assets) | ../arkcli-gen/SKILL.md |
One-sentence criterion:If the task maps to one of the sub-skill names below → use +understand; otherwise use +chatfor open-ended dialogue, and use +gen。
Agent quick execution order
- Determine which sub-skill the user's understanding task hits (see the table below). If it hits, explicitly include the sub-skill name; do not let it go through automatic routing to guess.
- The user only provides
ep-...→arkcli resources resolve <ep-id> --format json; only when the user's task is a clear understanding output andsupported_workflowscontainsunderstand, use this skill. - The user provided a temporary Key/Base URL/Endpoint → decide the parameters according to the shared execution-context combination rules; do not switch profiles first.
- When there is no complete stateless context, pass the authentication gate: if the login state is uncertain, first
arkcli auth status; for authentication failure, handle it according to../arkcli-auth/references/auth-modes.md's "Error recovery in API Key mode",do not retry in place。 - Prepare the input:
--input @<file>(can be multiple times). For local files use the@prefix; for remote usehttps:///tos://URL。 - By default do not pass
--model: the Platform Profile will use theResources.Text.Defaultalready deployed inep-xxx, consistent with+chat; only Plan-type Profiles use the sub-skill's recipe fallback model. Pass the full versioned ID orep-xxx。 - For segment-by-segment output add
--stream; to fine-tune the task instruction use--system-prompt-append "...", or--system-prompt-override "..."。 - After running, return to the user's original goal; do not stop at intermediate artifacts.
sub-skill quick reference table (12 / 4 modalities)
The table below shows the recipe fallback models used by Plan-type Profiles. Platform Profiles do not use these bare model names, but instead use the current Profile's
Resources.Text.DefaultEndpoint. If an override is truly needed, see the execution order above.
| sub-skill | Modality | Plan recipe fallback model | One-sentence purpose |
|---|---|---|---|
image-caption |
image | doubao-seed-1-6 |
Single-image/multi-image description, OCR (preserve the original text, do not paraphrase) |
image-grounding |
image | doubao-seed-1-6 |
Visual grounding: output the target bbox (x1,y1,x2,y2) + confidence |
image-gui |
image | doubao-seed-1-6 |
GUI screenshot → executable action sequence (JSON array of 12 action types) |
doc-extract |
file | doubao-seed-1-6 |
Extract structured JSON from PDFs/documents according to a schema (fill missing with null, preserve page) |
video-summary |
video | doubao-seed-1-6 |
Video summary: overall / segment / chapter + key timestamps |
video-qa |
video | doubao-seed-1-6 |
Video question answering: answer precisely by combining visuals + audio + timeline |
vau |
video | doubao-seed-2-0-lite |
Joint audio-video understanding: analyze the sound elements in the video and produce an audio analysis report |
asr |
audio | doubao-seed-2-0-lite |
Speech transcription: output plain text only, with no prefix/suffix/formatting whatsoever |
asr-align |
audio | doubao-seed-2-0-lite |
Subtitle timing: default SRT format (index + start/end time + text) |
asr-speakers |
audio | doubao-seed-2-0-lite |
Multi-speaker transcription: label [spk0] / [spk1] ... |
ast |
audio | doubao-seed-2-0-lite |
Speech translation: translate the speech in the audio into text |
meeting-minutes |
audio | doubao-seed-2-0-lite |
Meeting minutes: structured markdown (topic/participants/key points/decisions/follow-ups) |
For the expected output form and example commands for each item, see references/sub-skills.md。
Automatic routing when the sub-skill is omitted
args[0] When it does not hit the registry, followthe first --input file extensioninfer the modality and fall back to that modality's default sub-skill:
| Inferred modality | Fallback sub-skill |
|---|---|
image（.jpg/.png/.webp/...） |
image-caption |
video（.mp4/.mov/...） |
video-qa |
audio（.mp3/.wav/.m4a/...） |
asr |
| file (other extensions) | doc-extract |
- Without
--input, automatic routing is impossible(a recipe cannot be inferred from a pure prompt) → reportmissing_sub_skill。 - Currentlyfallback is based only on file modality, with no prompt keyword-level routing(for example, saying "box out" will not automatically select
image-grounding). If the Agent wants a non-default sub-skill (such as grounding / gui / summary / subtitles / translation), it shouldexplicitly write the sub-skill name, and not rely on automatic routing.
Command overview
| Command | Description |
|---|---|
arkcli +understand image-caption --input @photo.jpg "描述这张图" |
Image description |
arkcli +understand image-caption --input @doc.png "OCR，原样保留文字" |
OCR |
arkcli +understand image-grounding --input @scene.jpg "框出所有红色car" |
Visual grounding bbox |
arkcli +understand doc-extract --input @invoice.pdf "抽取 发票号/金额/日期" |
Document field extraction |
arkcli +understand video-summary --input @clip.mp4 "按 chapter 总结" |
Video chapter-by-chapter summary |
arkcli +understand video-qa --input @clip.mp4 "视频里出现了几辆车？" |
Video question answering |
arkcli +understand asr --input @speech.mp3 |
Speech transcription (no prompt needed) |
arkcli +understand asr-align --input @speech.mp3 |
Generate SRT subtitles |
arkcli +understand asr-speakers --input @meeting.wav |
Multi-speaker transcription |
arkcli +understand ast --input @speech.m4a "译成英文" |
Speech translation |
arkcli +understand meeting-minutes --input @meeting.m4a |
Meeting minutes |
arkcli +understand "<prompt>" --input @photo.jpg |
Omit sub-skill → automatically route by modality (here → image-caption) |
arkcli +understand asr --input @speech.mp3 --stream |
Streaming segment-by-segment output |
arkcli +understand image-caption --input @x.jpg --system-prompt-append "用英文回答" |
Append instructions after the built-in prompt |
arkcli +understand image-caption --input @x.jpg "描述图片" --dry-run --format json |
Return the recipe / model / request summary preview.v1; zero network, no upload, no inference |
Detailed documentation
- All flags,
@file/audio upload mechanism, return value flat schema, error codes, raw API fallback, and the relationship with+chat→references/arkcli-understand.md - The item-by-item purpose of the 12 sub-skills,expected output form, example commands →
references/sub-skills.md - Minimal evaluation/regression test cases →
references/evals.md