SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsgenerate-sls-sop

generate-sls-sop

Data & analytics

View repository

Updated 10-15 19:55

About this skill

Generate SLS SOP

Path variables：<project_dir> project-level directory (e.g. .input/my-project/） · <logstore_dir> = <project_dir>/<logstore_name>/

Two input methods are supported:

  1. SLS Project name: automatically via aliyun CLI to fetch index/dashboard data, then generate the SOP
  2. Local directory(input directory): generate the SOP directly from an existing data directory

Input directory structure(either output from the fetch script or manually prepared):

<logstore_dir>/
├── index.json              # 必需：logstore 索引配置
├── dashboards/             # 资源目录：关联的仪表盘 JSON
│   └── *.json
├── alerts/                 # 资源目录：关联的告警规则 JSON
│   └── *.json
├── scheduled_sqls/         # 资源目录：关联的定时 SQL JSON
│   └── *.json
└── saved_searches/         # 资源目录：关联的快速查询 JSON
    └── *.json

A valid directory must contain index.json(generated by fetch_sls_data.py ).

Execution conventions

⚠️ mustbe executed in the workspace root directory; cd-ing into the skill directory is prohibited. Violating this convention will cause relative path resolution to fail.

Prerequisites

  • When the input is an SLS project name, aliyun CLI >= v3.0.308 must be installed and authentication configured (thealiyun sls subcommand was introduced in v3.0.308)
    • Installation reference:https://github.com/aliyun/aliyun-cli
    • Version verification: run aliyun version, confirm the version number is >= 3.0.308
    • Authentication verification: run aliyun configure list, confirm a profile with a configured AccessKey exists; if there is no profile, prompt the user to first run aliyun configure to configure the AccessKey ID/Secret and Region
  • When executing fetch or validate_queries, it must be run in anon-sandbox environment(the sandbox restricts access to the system certificate chain, causing TLS failures)

Workflow

Checklist(check off during execution):

  • Step 0: Recovery detection (--resume-check -> route by action)

Phase A(project-level preparation, executed once globally):

Steps 1-3 are executed once for the entire project, handling all logstores in one pass.

  • Step 1: Locate the data source (SLS fetch / local / keyword search)
  • Step 2：prepare_project.py batch preprocessing
  • Step 3: Batch filtering + confirm output path <- global checkpoint; do not enter Phase B before completion

Phase B(per-logstore generation):

For each logstore (<logstore_dir>) execute:

  • update_status.py <project_dir> --mark-in-progress <logstore>
  • Step 4: Reference document extraction (conditionally executed)
  • Step 5: Generate field descriptions
  • Step 6: Curate queries
  • Step 7: Normalize templates
  • Step 8: Validate queries (optional)
  • Step 9: Clean up + annotate
  • Step 10: Render fragments + report
  • Step 11: Assemble output
  • update_status.py <project_dir> --mark-completed <logstore>

Phase C(global index update):

  • Step 12: Global index update

Phase D(audit, optional):

  • Step 13: Audit

Data file reference

<project_dir>/project_summary.json: Step 2 output -> Step 3 input. <project_dir>/selected_logstores.json: Step 3 script output -> Phase B / Step 12 input.

<logstore_dir>/ work files under:

<logstore_dir>/
  ├── skill_options.json                 # Step 1 写入，多步骤读取
  ├── parsed/                            # 中间数据
  │   ├── fields.json                    # Step 2：原始字段（保留）
  │   ├── queries.json                   # Step 2：已去重（不可变，id: q0, q1...）
  │   ├── prepare_summary.json           # Step 2：统计
  │   ├── reference_queries.json         # Step 4（条件，id: r0, r1...）
  │   ├── field_annotations.json         # Step 5 LLM
  │   ├── query_selection.json           # Step 6 LLM
  │   ├── query_pipeline.json            # Step 6 脚本 → Step 7 归一化(+pre_cleaned_query) → Step 8c 更新
  │   ├── query_validation.json           # Step 8a 脚本 → Step 8b 写回 pass/error
  │   ├── query_validation_LLM.json      # Step 8a 脚本（条件） → LLM 处理 → Step 8b 写回
  │   ├── query_annotations.json         # Step 9 LLM
  │   └── query_report.md                # Step 10 脚本
  └── fragments/                         # 组装片段
      ├── datasource.md                  # Step 2（不可变）
      ├── fields_table.md                # Step 5 脚本渲染（render_fields.py）
      ├── queries_selected.md            # Step 10（selected 或 extra 非空时）
      ├── queries_extra.md               # Step 10（条件）
      └── common_values.md               # Step 10 脚本（合并 reference 查询值，空时不生成）

queries.json each query contains the source_type field ("dashboard"、"alert"、"scheduled_sql"、"saved_search") and the logstore field, for reference during curation, classification, and Step 3 inference.

Step 0: Recovery detection

LLM: Infer from user input <project_dir>(SLS project name → .input/<project>/; local path → use directly or take parent directory; keyword → search workspace). If it cannot be determined or the directory does not exist, skip directly to Step 1.

Script: Run recovery detection:

python3 scripts/update_status.py <project_dir> --resume-check

Output example:

{
  "action": "resume_phase_b",
  "summary": {
    "total": 100,
    "pending": 68,
    "in_progress": 2,
    "completed": 27,
    "failed": 3
  },
  "pending_logstores": ["logstore_c", "logstore_d", "..."],
  "in_progress_logstores": ["logstore_a", "logstore_b"],
  "failed_logstores": [
    {"name": "logstore_x", "failed_step": 6}
  ]
}

Based on the action field, jump to the corresponding phase;action meaning:

  • "first_run" -- no selected_logstores.json -> execute Step 1 normally
  • "resume_phase_b" -- has pending/in_progress -> jump to Phase B
  • "all_completed" -- all completed -> jump to Phase C (Step 12)

LLM: Report the progress in summary to the user, and route to the corresponding step according to action.

Step 1: Locate the data source

The user input has four forms, matched by priority:

  1. SLS Project name: the user provides a project (optionally with a logstore name) → continue this step
  2. Local path (project directory): verify that logstore subdirectories exist under the directory → jump to Step 2
  3. Local path (logstore directory): verify the directory structure (contains index.json and at least one resource subdirectory) → infer project_dir = parent directory, logstore = directory name → use at Step 2 --logstores=<logstore_name>
  4. Keyword: search the workspace for a matching directory (containing index.json) → jump to Step 2 after finding it

The following is executed only for SLS Project input.

Confirm that aliyun CLI is available:

  1. Execute aliyun version, confirm the version is >= 3.0.308; if the version is too low or it is not installed, prompt the user to upgrade/install
  2. Execute aliyun configure list, confirm that a profile with configured AccessKey exists; if there is no profile, prompt the user to first run aliyun configure to configure AccessKey ID/Secret and Region

Determine parameters：

  • project only: do not pass --logstores; fetch automatically handles all logstores with associated resources (internal logstores are already automatically filtered)
  • project + logstore(s): pass --logstores=<ls1,ls2,...>, fetch only the user-specified logstore

Execute fetch(must run in a non-sandbox environment):

python3 scripts/fetch_sls_data.py <project> .input/<project>/ \
  [--logstores=<ls1,ls2,...>] \
  > /tmp/sls_fetch_summary_<project>.json \
  2>/tmp/sls_fetch_err_<project>.log

Parameters and output format are detailed in the docstring at the top of fetch_sls_data.py .

Result check: read /tmp/sls_fetch_summary_<project>.json, check whether errors is empty. If not empty, check the specific error (common: API authentication failure, dashboard does not exist).

Data summary: display the data_summary.md content in stdout, and report the data filtering flow to the user.

Post-fetch processing: based on logstores_processed in summary, display the number of associated resources for each logstore. Check whether there is a "No index config" prompt in warnings — these logstores will be automatically skipped at Step 2 (no index.json）。

User option persistence(parse the user instruction, call the script to batch-write skill_options.json）：

Parse the optional parameters in the user instruction:

  • output_format: the user mentions "generate SKILL", "SKILL format", "skill document", etc. → add --output-format SKILL; otherwise default to --output-format SOP
  • validate_queries: the user mentions "validate query", "check query", etc. → add --validate-queries
  • reference_source：
    • User specifies reference directory: add --reference-dir <path>(Script automatically matches by filename)
    • User explicitly specifies logstore + document: add --reference <logstore>=<file>
python3 scripts/save_options.py <project_dir> \
  [--output-format SOP|SKILL] \
  [--validate-queries] \
  [--reference-dir <path>] \
  [--reference <logstore>=<file> ...]

The script writes for each logstore skill_options.json, ensuring subsequent steps can independently read user options.

Note：save_options.py does not require the --logstores parameter—it automatically identifies valid logstores (i.e., logstores already processed in the Step 1 fetch stage) by checking index.json .

Step 2: Batch preprocessing

Process all valid logstores under the entire project in a single call:

python3 scripts/prepare_project.py <project_dir> \
  > /tmp/sls_prepare_summary_<project>.json 2>/tmp/sls_prepare_err_<project>.log

Automatically process all valid logstores under <project_dir> (pure local parsing, executes very quickly). For parameters and output format, see the docstring at the top of prepare_project.py .

Result check: read <project_dir>/project_summary.json, check whether errors is empty.

Step 3: Name simplification + confirm output paths

Global checkpoint (Phase A wrap-up); do not enter Phase B until processing is complete.

Perform name simplification and deduplication for all logstores, and confirm output paths. Read naming_rules.md, and execute strictly according to the rules.

Input:<project_dir>/project_summary.json(Step 2 output) Output:<project_dir>/selected_logstores.json + each confirmed logstore's skill_options.json appended with output_path

Phase B begins

Run python3 scripts/update_status.py <project_dir> --resume-check to get the current status:

  • pending_logstores: not started → start a new task
  • in_progress_logstores: in progress (possibly another agent is executing, or it was interrupted last time) → do not start again
  • failed_logstores: failed → manual intervention required

Execution mode(choose according to platform capabilities):

  • Parallel(platform supports multiple tasks/sub-agents): start an independent task for each logstore, each executing Steps 4-11
    • Rate-limit protection: the number of concurrently running agents must not exceed 8
  • Serial(single logstore or platform does not support parallelism): execute Steps 4-11 logstore by logstore

Prohibitedalternating execution across logstores (e.g., first doing all Step 5 and then all Step 6), to avoid context contamination.

Progress monitoring: periodically run --resume-check to refresh the status.

End condition: when both pending_logstores and in_progress_logstores are empty, Phase B is complete, and enter Phase C.

Steps 4-11 below are executed in the context of the current logstore (<logstore_dir>).

Step-level checkpoint: mark progress after each step is completed, supporting resumption from the last completed position after an interruption.

Detect recovery point：python3 scripts/update_status.py <project_dir> --step-resume-check <logstore>

Output example:{"resume_from": 8}

Jump to the corresponding step according to resume_from ; if it is 4, start from the beginning; if it is 12, it is already complete.

Mark in progress：python3 scripts/update_status.py <project_dir> --mark-in-progress <logstore>

Step 4: Reference document extraction

Conditional step: executed only when skill_options.json contains the reference_source field; otherwise skipped.

Read reference_extract.md, and execute strictly according to the rules.

Input:skill_options.json the reference document specified by reference_source Output:parsed/reference_queries.json(same schema as queries.json , id prefix r）

Step 5: Generate field descriptions

Generate a short Chinese description for each field. Read field_desc.md, and execute strictly according to the rules.

Input:parsed/fields.json(field list generated in Step 2) Output:fragments/fields_table.md

Step 6: Curate queries

The LLM is responsible for semantic selection, and the script is responsible for data assembly.

Read query_select.md, and execute strictly according to the rules.

Input:parsed/queries.json（Step 2） + parsed/reference_queries.json(Step 4, if present) Output:parsed/query_pipeline.json(complete pipeline, the data source for subsequent steps)

Step 7: Normalize templates

Run the normalization script to unify all template syntax into the <var;default> / <var> format, and generate pre_cleaned_query(strip ;default）：

python3 scripts/normalize_templates.py <logstore_dir>

Pure script execution, no LLM needed.

Input:parsed/query_pipeline.json(Step 6 output) Output:parsed/query_pipeline.json(update: add to each query normalized_query and pre_cleaned_query）

pre_cleaned_query = normalized_query in <var;default> are automatically stripped to <var>, for review by the Step 9 LLM.

Record step success：python3 scripts/update_status.py <project_dir> --mark-step <logstore> --step 7

Step 8: Validate queries (optional)

Execute only when skill_options.json in validate_queries is true, otherwise skip.

Perform syntax validation on the queries in the pipeline, remove failed items and fill in replacements. Read query_verify.md, and execute strictly according to the rules.

Input:parsed/query_pipeline.json(Step 7 output, including normalized_query) Output:parsed/query_pipeline.json(update: remove failed items, fill in replacements, add validation statistics)

Step 9: Cleanup + annotation

Must be handled directly by the LLM; writing a script as a substitute is not allowed.

Read query_format.md, and execute strictly according to the rules.

Input:parsed/query_pipeline.json(Step 7/8 output) Output:parsed/query_annotations.json(standalone annotation file, containing only id/title/category/cleaned_query)

Step 10: Render snippets + report

Run the rendering script:

python3 scripts/render_queries.py <logstore_dir>

The script reads parsed/query_pipeline.json + parsed/query_annotations.jsonand, after merging, generates:

  • fragments/queries_selected.md(when selected or extra is non-empty)
  • fragments/queries_extra.md(when extra is non-empty)
  • parsed/query_report.md(required)

The script also reads parsed/prepare_summary.json、parsed/reference_queries.json(if present) and parsed/query_validation.json + parsed/query_validation_LLM.json(if present) for the report. This step is pure script execution, no LLM needed.

Record step success：python3 scripts/update_status.py <project_dir> --mark-step <logstore> --step 10

Step 11: Assemble output

Refer to <logstore_dir>/fragments/queries_selected.md(query classification) and fragments/fields_table.md(field composition), and generate:

  • name: a short readable name (semantic translation preferred, e.g. xxx_audit_log → "Audit Log (xxx_audit_log)"; if it cannot be translated, use the original name)
  • description: a one-sentence functional positioning, inferring the system and purpose from the query classification and fields; directly listing classification names or applying generic templates is prohibited

Run the assembly script:

python3 scripts/assemble_overview.py <logstore_dir> \
  --name "..." --description "..."

The assembly result is written to the path specified by skill_options.json in output_path .

Record step success：python3 scripts/update_status.py <project_dir> --mark-step <logstore> --step 11

Record processing complete：python3 scripts/update_status.py <project_dir> --mark-completed <logstore>

Phase B ends: return to process the next pending logstore, or proceed to Phase C.

Step 12: Globally update index

Global one-time execution (Phase C).

Update or create index files level by level from the deepest directory upward. Read index_rules.md, and execute strictly according to the rules.

Input:<project_dir>/selected_logstores.json(including the output_path and output_root of all confirmed logstores) Output: index files for directories at each level (update if they already exist; create new ones according to the template if they do not exist)

Phase C complete: Prompt the user that SOP generation is complete, and ask whether a quality audit is needed (Phase D, optional, recommended to be executed in a new session).

Phase D starts(optional, recommended in a new session): execute only when the user explicitly requests an audit or when the user confirms after Phase C completion that a quality review is needed. It is strongly recommended that the audit be executed in a separate session to obtain an objective perspective and avoid context bias from the generation phase.

Step 13: Audit

Optional step, used to audit the semantic quality of LLM-generated content.

Read audit_rules.md, strictly follow the rules.

Input:<project_dir>/selected_logstores.json(list of completed logstores) Output:<project_dir>/_audit/(audit plan, per-logstore results, summary report)

Troubleshooting

See troubleshooting.md。