SuperAPI AI Directory
Partner中文AI newsFavorites
Skillstestany-pipeline

testany-pipeline

Testing & quality

View repository

Updated 10-16 06:14

About this skill

Testany Pipeline

This skill manages pipelines on the Testany platformthrough Testany MCP tools. Platform object operations are performed via remote API; it is permitted to read local handoff, generate, and statically check YAML for this orchestration. This does not authorize executing tests or returning the commands.

Key Prerequisites：

  • A pipeline is Testany'sexecution and orchestration unit
  • Conventional scenario execution uses pipelines; case dry-run is another verification action that requires execution authorization and cannot be treated as a read-only check or automatically replace a pipeline
  • A trigger is merely the execution entry point of a pipeline, not the orchestration layer

User input: $ARGUMENTS

Host Capability Adaptation

  • Follow Overall Objective and Handoff: For a complete objective, registration, orchestration, a single execution, and waiting continue as needed on demand; do not provide only the next-step suggestion; a configuration-only objective is not expanded into running.
  • Prefer using the host-provided structured question tool (such as AskUserQuestion) to collect missing information in one go.
  • If the host does not support that tool, ask the same questions together in one ordinary message; for low-risk fields, default value suggestions may be provided.
  • If the host supports slash commands, you may recommend the command entry point of the relevant workflow; otherwise continue the corresponding workflow directly in the current thread.

First Unify the Mental Model

Before using this skill, first understand the boundaries according to automation-model.md :

  • What the upstream provides is usually traditional test scenario
  • testany-case-writing is responsible for breaking it down into platform cases
  • This skill is responsible for orchestrating these platform cases into pipeline
  • testany-trigger is responsible for configuring the pipeline Plan / Manual Trigger / Gatekeeper

Important Conclusion：

  • The primary input of this skill should not be "let me guess the business process from the case description"
  • The primary input of this skill should be the explicit upstream-provided automation design / decomposition

Upstream Input Priority

Select the input mode according to the following priority:

  1. Primary：automation design / decomposition

    • From testany-case-writing
    • preferably generated based on the approved Test Spec's Testany Automation Handoff Generate
    • Case inventory, dependency relationships, relay map, and whether there are branches are already explicit

  2. Secondary: case keys + dependency description explicitly provided by the user

    • For example, "use A1B2C3D4 to log in first, then use E5F6A7B8 to query"

  3. Fallback: infer from existing case metadata

    • Use only when neither of the first two is available
    • The result must be echoed back to the user for confirmation
    • A "guessed process" cannot be taken as the primary path

Operations Quick Reference

User Intent Operation Type Tool
Create new pipeline Create testany_create_pipeline
View pipeline details Read testany_get_pipeline
View pipeline YAML Read testany_get_pipeline_yaml
Search/list pipelines (by workspace) Read testany_list_pipelines
List my pipelines (by workspace) Read testany_list_my_pipelines
Modify pipeline configuration Update testany_update_pipeline
Delete pipeline Delete testany_get_pipeline_used_by → testany_delete_pipeline
Validate YAML syntax Validate testany_verify_pipeline
Check referenced status Query testany_get_pipeline_used_by

Create

Local YAML checks, remote validation, and execution are separate; follow Delivery Verification. After creation, read back the configuration and the required YAML using the real key, and verify the name, workspace, case references, and this orchestration; accepted does not mean the configuration is consistent, let alone that execution succeeded.

There is a data dependency between creation and calls that depend on the new key: first obtain and check the creation response, then pass the actual pipeline_key to verification, read-back, or execution. Do not fill example/guessed keys into the same batch of calls before creation;&& only guarantees that the previous command succeeded and will not automatically pass the new key to the next command. Scripted orchestration can parse the response and continue after checking that the key is non-empty; stop the related calls when it is missing or uncertain.

Phase 0: First determine the input mode

Primary: an automation design / decomposition already exists

If the upstream has already provided the following, orchestrate directly according to it:

  • platform case inventory
  • The responsibility of each case
  • source_case_ids / scenario_group(if from a Test Spec handoff)
  • dependencies
  • relay map
  • Whether there is whenFailed / expect: fail

Secondary: the user has explicitly given the case keys and order

If the user directly provides:

  • case keys
  • Execution order
  • relay relationships

then go directly to YAML construction.

Fallback: can only be inferred from existing cases

Use only when there is no upstream design:

  • testany_list_cases / testany_get_case
  • In combination with case_labels、description、environment_variables[].description
  • give candidate orchestration plans
  • must be confirmed by the user

Phase 1: Prepare data

Fetch in parallel:

  • testany_get_my_workspaces
  • testany_list_cases or testany_list_my_cases

If the user has not yet registered the platform cases with the Testany platform:

  • Do not use non-existent case keys to create a pipeline
  • When the complete goal includes registration and it has been authorized, read testany-case to register first, then return with the real key; when only orchestrating existing cases, explain that the object is missing and do not expand into registration

Phase 2: Build the pipeline design

Based on the input mode, determine:

  • pipeline name
  • the workspace it belongs to
  • which case keys are included
  • order and prerequisite dependencies
  • relay variable relationships
  • whether there is a failure branch
  • whether there is expect: fail

When to use case_keys

Only when the following conditions are met is it allowed to directly use case_keys to automatically generate a simple sequential pipeline:

  • No conditional branches
  • No relay
  • No expect: fail
  • The user only wants the simplest sequential execution

When YAML must be handwritten

When any of the following occurs, YAML must be explicitly generated:

  • There is relay
  • There is whenPassed / whenFailed
  • There is expect: fail
  • Need to express branches, prerequisites, cleanup, or failure paths

Phase 3: Verify relay and dependencies

If there is relay, you must:

  1. testany_get_case Check whether the source case has type='output' variables
  2. testany_get_case Check whether the target case has type='env' variables
  3. Ensure the source case is positioned before the target case in rules
  4. Ensure relay is not combined with whenFailed combination
  5. It is forbidden to use type='secrets' rows as relay sources or targets: secret values come from the workspace Credential Safe and are not passed through relay; the target side also cannot be overwritten by relay. If the relay source/target hits a secrets row, stop and explain to the user, and request switching to env or output rows

Phase 4: Create pipeline

Call testany_create_pipeline：

Parameter Required Description
name Yes pipeline name
workspace Yes Owning workspace key
description No Description
definition No Pipeline YAML configuration
case_keys No Case keys array (only for simple sequential scenarios)

Phase 5: Verify

Call testany_verify_pipeline：

  • Check kind whether it is within the supported range (rule/v1.2 or rule/v1.3）
  • Check rules whether the structure is valid
  • Check relay and dependency constraints

Fallback: Infer from existing cases (last resort only)

Only when neither of the first two input modes exists is inferring from existing cases allowed.

Information that can be used for judgment

Field Purpose Reliability
case_labels Filter by User Story number or functional module High
description Understand the action, preconditions, and verification target Medium
environment_variables[].description Understand the semantics of input/output variables Medium
name Auxiliary judgment Low

Do not guess

If it still cannot be determined:

  • Which cases should be included
  • How the order should be arranged
  • How relay should be configured

You must confirm with the user instead of guessing.

Read (query)

Scenario Tool Description
Get pipeline details testany_get_pipeline Pass in the pipeline key
Get YAML content testany_get_pipeline_yaml Pass in the pipeline key
Search/list pipelines (by workspace) testany_list_pipelines workspace Required
List only my pipelines (by workspace) testany_list_my_pipelines workspace Required

Update

Updatable fields

Parameter Description
name pipeline name
description Description
definition YAML definition
case_keys Case keys for simple sequential execution
environments Environment tag list
owned_by Owner email
pipeline_labels Pipeline tag list

Update process

  1. testany_get_pipeline Read the current configuration
  2. testany_get_pipeline_yaml Read the current YAML (if orchestration needs to be modified)
  3. Prefer updating according to the existing automation design rather than re-guessing on the spot
  4. If the YAML is modified, re-verify relay and dependencies
  5. testany_update_pipeline Submit the update
  6. testany_get_pipeline Read back the authorized fields; if orchestration was changed, read the YAML again to verify. If the readback is still the old value or unreadable, report unverified/inconsistent; do not automatically retry, execute, or delete

Delete

Before deletion, references must be checked:

  1. testany_get_pipeline_used_by
  2. If referenced by Plan / Manual Trigger / Gatekeeper , first prompt the user to remove the reference
  3. Delete only after there are no references

Warning: This operation is irreversible.

Common issue handling

Scenario Handling approach
The user only has a scenario, with no registered case First go to testany-case-writing + testany-case
The user has only one case and wants to execute it directly A single-case pipeline still needs to be created
There is a relay but the source/target is not clearly specified Go back upstream to confirm the decomposition
There is only an existing case library, with no design Fallback reverse inference is possible, but the user must confirm
The user wants to configure when it runs Switch to testany-trigger
The user wants to execute the pipeline immediately Switch to testany-trigger
The user wants to view execution progress or history Switch to testany-execution

Return format

After the task is completed, report to the user:

  • Pipeline Key
  • Pipeline name
  • Workspace it belongs to
  • Source input mode:automation design / explicit case keys / fallback inference
  • Number and order of included cases
  • Relay configuration summary
  • Whether the overall objective still requires testany-trigger / testany-execution: if authorized, actually continue; if not included, do not extend

Reference documentation