pipeline-model-architecture

BK-CI Pipeline Model Architecture

Applicable Scenarios

• Understanding Model -> Stage -> Container -> Element hierarchical relationships
• Modifying pipeline model fields, structures, or validation logic
• Adding new container types, plugin types, or model auxiliary structures
• Troubleshooting model serialization, version compatibility, or YAML conversion issues
• Determining whether a requirement should land in the model definition, runtime state, or persistence layer

Inapplicable Scenarios

• Only resource scheduling and allocation, build machine execution, or plugin runtime issues
• Only ordinary backend API development, not involving the pipeline model itself
• Only database script design, not involving model structures or model storage

Quick Guide

1. First classify the problem into one of the following 5 categories, then continue reading the corresponding reference document:
   • Structure definition: see reference/1-core-structure.md
   • Persistence and versions: see reference/2-persistence-versioning.md
   • Runtime, YAML, validation, extension: see reference/3-runtime-conversion-extension.md
2. Model is the core carrier of pipeline configuration, but it is not the authoritative source for all runtime state. For fields such as running state, duration, and retries, distinguish between "configuration state" and "runtime state".
3. The trigger container is the model entry point, by default located in the first Stage of the first Container. Many parameters, triggers, and startup logic enter from here.
4. Container and Element are both polymorphic structures. When adding a new type, you cannot change only one class definition; you usually also need to check together:
   • Jackson polymorphic serialization configuration
   • Frontend orchestration model
   • Backend conversion and validation
   • Runtime engine or Worker recognition logic
5. Version compatibility and YAML conversion should be considered together. Once model fields enter existing data, you cannot look at them only from the perspective of "currently creating a new process".
6. If what you are modifying is a capability such as process control, execution conditions, Finally, or Matrix, first confirm whether it affects the Stage, Job, or Element layer.

High-Signal Rules

• Model cares about "how the pipeline is defined", whileProcess cares more about "how the pipeline is orchestrated and executed"
• Stage expresses stage boundaries and process control,Container is closer to a Job, andElement is closer to a specific task or plugin
• id、stageIdForUser, plugin code, and runtime state fields are not concepts at the same level and cannot be mixed up
• As long as new model fields are involved, you must simultaneously check serialization, compatibility, default values, and historical version reading

Key Pitfalls

• Treating runtime fields directly as persistent configuration fields to modify, causing anomalies in old build details or historical snapshots
• Only changing the Element/Container class definition, without adding polymorphic declarations, validation, or frontend-side mappings
• Ignoring special structures such as Finally, Matrix, and execution conditions, and handling them as ordinary Stage/Job
• Only verifying the UI new-process flow, without verifying historical Model replay, retries, version switching, and YAML conversion
• Seeing that the problem appears in a model field, directly modifying the model, when in fact the problem may be in Process execution chain or conversion layer

Further Reading

• Core structures:reference/1-core-structure.md
• Persistence and versioning:reference/2-persistence-versioning.md
• Runtime, transformation, and extension:reference/3-runtime-conversion-extension.md
• When execution path ownership is involved: first look at process-module-architecture
• When YAML/PAC is involved: then look at yaml-pipeline-transfer