byted-volcengine-topology-builder

Volcengine Foundational Asset Data and Topology Skill

Capability Positioning

The responsibility of this Skill is to build a reusable layer of foundational data, rather than serving only a single upper-layer analysis scenario.

• Collection layer: pull as much of the currently connected Volcengine resource assets as possible
• Modeling layer: organize the raw snapshot into structured asset data and a unified relationship expression
• Output layer: persist to disk as json/md/dot/svg/png and other reusable artifacts
• Reuse layer: for continued consumption by scenarios such as topology analysis, impact assessment, automated operations, troubleshooting, asset inventory, and dependency organization

Default Artifacts

By default, under the business_topologies/<business_key>/ of the current workspace, the following will be generated:

business_topologies/<business_key>/
  account_assets_snapshot.json
  topology.json
  topology.md
  topology.dot
  topology.svg   # 本地装有 Graphviz 或自动安装成功时生成
  topology.png   # 本地装有 Graphviz 或自动安装成功时生成

Notes:

• account_assets_snapshot.json is the raw asset snapshot, the foundational input for all subsequent analysis and reconstruction
• topology.json is the structured relationship model, suitable for continued consumption by scripts, agents, and analysis tools
• topology.md is a text view for people to browse quickly
• topology.dot will always be generated, making it convenient for subsequent conversion with Graphviz or other tools
• topology.svg/png is mainly used for "viewing diagrams/generating diagrams/displaying to users"

One-Click Pipeline

1. Prepare the .env：

VOLCENGINE_AK=...
VOLCENGINE_SK=...

1. Run the general entry script:

python3 <byted-volcengine-topology-builder-skill>/scripts/run_topology_pipeline.py \
  --workspace-root "$(pwd)" \
  --env-path ./.env \
  --region cn-shanghai \
  --business default-topology

This pipeline will by default execute in sequence:

• Collect asset snapshot
• Build topology from the snapshot
• Save topology.json/topology.md
• Generate topology.dot
• If Graphvizis available locally, continue to generate topology.svg/topology.png

If the dot command is not available locally, the script will first try to automatically install Graphviz; if installation fails, it will degrade to outputting only topology.dot, without blocking the entire foundational data persistence process.

Common Parameters

The general entry script run_topology_pipeline.py supports:

• --business: business or asset view identifier
• --region: region
• --include: resource types to collect, can be passed repeatedly or separated by commas
• --entry: entry resource types to prioritize when building the graph, can be passed repeatedly or separated by commas
• --skip-render-graph: generate only structured artifacts, without additionally producing diagrams

For example:

python3 <byted-volcengine-topology-builder-skill>/scripts/run_topology_pipeline.py \
  --workspace-root "$(pwd)" \
  --env-path ./.env \
  --region cn-shanghai \
  --business payment-core \
  --include ecs,eip,clb,alb,natgateway,rds_mysql,redis \
  --entry eip --entry clb

Layered Execution

1. Collect Asset Snapshot

python3 <byted-volcengine-topology-builder-skill>/scripts/dump_account_assets.py \
  --region cn-shanghai \
  --env-path ./.env \
  --include ecs,eip,clb,alb,natgateway,rds_mysql,redis \
  --output-file ./business_topologies/payment-core/account_assets_snapshot.json

The goal of this layer is to consolidate stable, reusable, and reconstructable foundational asset data as much as possible.

2. Build Graph from Snapshot

python3 <byted-volcengine-topology-builder-skill>/scripts/build_topology_from_account_assets.py \
  --assets-file ./business_topologies/payment-core/account_assets_snapshot.json \
  --region cn-shanghai \
  --entry eip --entry clb --entry alb --entry natgateway \
  --output-file ./business_topologies/payment-core/topology.json

This layer is responsible for organizing discrete assets into a unified relationship model, facilitating subsequent analysis and visualization.

3. Save and Render Artifacts

python3 <byted-volcengine-topology-builder-skill>/scripts/save_topology.py \
  --business payment-core \
  --root ./business_topologies \
  --topology-file ./business_topologies/payment-core/topology.json

If you only want to separately draw a diagram for an existing topology.json :

python3 <byted-volcengine-topology-builder-skill>/scripts/render_topology_graph.py \
  --topology-file ./business_topologies/payment-core/topology.json \
  --output-dir ./business_topologies/payment-core

Relationship Expression

Currently topology.json uses the nodes + chains structure:

• path[].relation: primary link relationships
• contexts.<node_id>: contextual resources of path nodes

Current relationship semantics:

• attached_to: A is bound to B, for example EIP -> CLB、ECS -> EBS
• has: A owns B, for example CLB -> 后端服务器组
• contains: A contains B, for example 后端服务器组 -> ECS、VPC -> 子网
• belongs_to: A belongs to B, for example ECS -> VPC/子网/安全组

The display layer defaults to using the resource id as the primary identifier, avoiding ambiguity caused by duplicate instance names;name is retained in the metadata as supplementary information.

Current coverage

When permissions allow and snapshot fields are complete, the following basic relationships are currently prioritized for collection:

• EIP -> CLB -> 后端服务器组 -> ECS
• EIP -> ECS
• ECS -> VPC/子网/安全组/EBS
• VPC -> 子网

This means it can already provide basic input for many upper-layer scenarios, but it remains an extensible foundation and should not be understood as "already fully covering all cloud resource relationships."

Common questions

• AccessDenied (403): the account does not have read-only permissions for the corresponding product; the script will degrade as much as possible and retain the assets that were successfully pulled
• rds_mysql/redis The corresponding SDK method or field is unstable: this indicates that the current collection adaptation still needs further expansion, but it does not affect the existing basic artifacts being written to disk
• There is no local dot command: it will not block the generation of topology.json/topology.md , and at least topology.dot
• If you do not want the rendering script to automatically install Graphviz, you can add --skip-auto-install-graphviz