SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillstestany-execution

testany-execution

General & other

View repository

Updated 10-16 08:11

About this skill

Testany Execution Operations

Manage executionon the Testany platform, including progress observation, history queries, status refresh, cancellation, and failure handover.

User input: $ARGUMENTS

First establish a unified mental model

Follow Overall objectives and handover, use the real execution key from the previous step and the wait budget that has not been reset. Only check the current status and do not wait automatically; do not on your own expand a failed terminal state into retry, cancellation, or troubleshooting.

Before starting, first follow automation-model.md to understand the boundaries:

  • pipeline is the execution and orchestration unit
  • trigger is responsible for "how to initiate execution", including long-term entry points and immediate runs
  • This skill is responsible for "how to manage an execution after it has been initiated"
  • Root cause analysis of a failed execution belongs to testany-debug

Important conclusions：

  • This skill is not responsible for creating Plan / Manual Trigger / Gatekeeper
  • This skill is also not responsible for immediately initiating an execution; that belongs to testany-trigger
  • This skill focuses on the execution lifecycle: view, query, refresh, stop, handover; query, wait, and cancel do not authorize each other

Scope of responsibility

  • View execution details and case-level results
  • List/search execution history
  • Refresh execution status
  • Poll and wait for an execution to reach a terminal state
  • Cancel an execution that has not yet started
  • Summarize execution results and decide whether to hand over to testany-debug
  • When needed, obtain execution case details and the log signature entry point

Quick operation reference

User intent Operation type MCP tool
List execution records Read testany_list_executions
View execution details Read testany_get_execution
View details of a case within an execution Read testany_get_execution_case
Refresh execution status Update testany_refresh_execution
Cancel an execution that has not started Update testany_cancel_execution
Get workspace execution status summary Read testany_get_workspace_execution_status
Get filterable status values Read testany_filter_execution_status
Get pipelines with execution records Read testany_filter_execution_pipelines
Get log signature Read testany_log_sign

Core knowledge

Execution status

Status Value Meaning Whether terminal state
NOT_STARTED -1 Not started/queued (waiting for a concurrency slot) No
RUNNING 0 Running No
SUCCESS 1 All passed Yes
FAILURE 2 Has failures Yes
SKIPPED 3 Skipped (Case only) Yes
FAIL_AS_EXPECTED 4 Expected failure (Case only) Yes
CANCELLED 5 Canceled Yes
ERROR 99 System error Yes

Workspace queue status

Testany uses workspace-level concurrency slots (Redis semaphore) to control execution parallelism. Through testany_get_workspace_execution_status you can obtain:

Field Meaning
limit Workspace concurrency limit (Community=2, Paid=4, Enterprise=8, adjustable)
claimed List of execution keys currently running (already occupying slots)
pending List of execution keys in the queue (waiting for a slot)

When a user's execution status is NOT_STARTED and remains unchanged for a long time, you should proactively query the queue status:

  • claimed count = limit → slots are full, this execution is queued in pending queued
  • claimed has a long-running old execution → the old execution is occupying the slot

When a user asks "why isn't my execution running", this is the first direction to check, before checking YAML and relay.

Standard handling chain of this skill

trigger 已返回 execution_key
  -> 按用户目标查询当前状态，或有界等待终态
  -> 如失败，按需 testany_get_execution_case / testany_log_sign
  -> 必要时交给 testany-debug

Typical workflows

1. View a single execution's progress

Applicable input:

  • execution_key
  • "See how far this execution has gotten"

Handling:

  1. testany_get_execution
  2. Only when the user requests a refresh, or there is specific stale evidence and refreshing is permitted,testany_refresh_execution; RUNNING itself is not stale evidence
  3. Report the current status, start time, and pass/fail counts

Checking the status only completes here; do not automatically long-poll, cancel, retry, or start a new execution just because it is still running.

2. Poll and wait for an execution to complete

Applicable input:

  • "Watch this execution for me until it finishes"
  • Known execution_key

Handling: Follow the execution permission and waiting contract：

  1. Before the first query, save the start time and deadline from the actual clock, then call testany_get_execution; a verbally promised budget does not replace the timing values.
  2. Only when the known state is a non-terminal -1/0 and there is still budget, wait and testany_refresh_execution; back off the interval from 5 seconds up to a maximum of 30 seconds, checking the deadline before each call.
  3. Default total wait is 10 minutes; a shorter user budget takes precedence, tool time counts toward the budget; do not extend it on your own.
  4. Stop making new calls on a terminal state, unknown state, tool error, exhausted budget, or user interruption; report the last status and limitations. A wait timeout is not a remote failure/cancellation; do not automatically call cancel, retry, or a new execution. If get or refresh has already given a terminal state, close the case directly and do not append further queries for the same object; use sleep for waiting intervals, do not busy-wait in an empty loop.

3. View historical executions

Applicable input:

  • "Look at the most recent failed execution"
  • "List the recent executions of a certain pipeline"
  • "Look at the execution situation of a certain workspace"

Handling:

  1. Call as appropriate testany_filter_execution_status / testany_filter_execution_pipelines
  2. testany_list_executions
  3. If necessary, then testany_get_execution

Common filter dimensions:

  • workspace
  • pipeline_key
  • status
  • triggered_by
  • env
  • exe_start_time_from / exe_start_time_to

4. Cancel an execution

Applicable input:

  • "Cancel this execution that hasn't started yet"

Handling:

  1. testany_get_execution
  2. Only when the user has authorized canceling that object and the status is still not started / pending, call testany_cancel_execution
  3. If it has already started running, clearly state that it cannot be canceled
  4. Read back the cancellation status; if the request has been accepted but there is no terminal cancellation state, you cannot say it has been canceled. If permissions are insufficient or a race condition fails, report it truthfully

5. View queue status / diagnose execution queuing

Applicable input:

  • "Why isn't my execution running"
  • "The YAML is parallel but the executions are running serially"
  • "Take a look at the queue status"
  • "The execution stays NOT_STARTED"

Handling:

  1. testany_get_workspace_execution_status → get limit / claimed / pending
  2. Judge and report:
    • Whether claimed count = limit (slots full)
    • Whether the execution the user is concerned about is in the pending list
    • Whether there is a long-running old execution in claimed occupying a slot
  3. If the slots are not full but the case is still executed serially:
    • Prompt the user to check the pipeline YAML version (rule/v1.2 does not support case-level parallelism)
    • Suggest checking the case's start_time/finish_time to confirm whether it was scheduled serially
    • If in-depth analysis is needed, switch to testany-debug(scheduling/queue diagnostic branch)

6. Hand failed executions to debug

Applicable input:

  • "Why did this execution fail"
  • "Help me locate the failed case"

Handling method:

  1. testany_get_execution
  2. Find the failed case
  3. If only locating is needed, first testany_get_execution_case
  4. If root cause analysis and log interpretation are needed, switch to testany-debug

Boundary clarification

When to go to testany-trigger

The following scenarios do not belong to this skill:

  • Execute a pipeline immediately right now
  • Configure a Plan for a pipeline
  • Create a Manual Trigger
  • Create a Gatekeeper

All of these should switch to testany-trigger。

When to go to testany-debug

The following scenarios should be forwarded to testany-debug：

  • The user wants to see the root cause of the failure
  • The user wants to read and analyze logs
  • The user wants to determine whether it is an assertion failure, timeout, dependency error, or infrastructure issue

Return format

After the task is completed, report to the user:

  • Execution Key (e.g. Y2K-0601-0000B）
  • Owning Pipeline
  • Current status or final-state result
  • Number passed/failed/skipped
  • List of failed cases (if any)
  • Whether cancellation was actually requested and whether there is evidence of cancellation; do not treat waiting for completion as cancellation
  • If the final state was not obtained, explain the last observed status, budget/tool limitations, and unfinished items
  • Next-step suggestions:
    • Need to continue triggering execution → go to testany-trigger
    • Need to analyze the failure reason → go to testany-debug

Reference documentation