orchestrator

Testany Test Orchestration

Configure Testany test orchestration and automation.

User input: $ARGUMENTS

Scope of Responsibilities

• Create/Query/Update/Delete Gatekeeper (Webhook Trigger)
• Create/Query/Update/Delete Plan (Scheduled Plan)
• Establish binding between Gatekeeper and Pipelines (Pipeline Group)
• Provide copyable Webhook / CI integration examples (Note: Webhook URL may need to be obtained by the user from the UI)

---

Operation Quick Reference

User Intent	Operation Type	MCP Tool
List Gatekeepers	Read	testany_list_gatekeepers
View Gatekeeper Details	Read	testany_get_gatekeeper
Create Gatekeeper	Create	testany_create_gatekeeper
Update Gatekeeper Fields	Update	testany_update_gatekeeper
Delete Gatekeeper	Delete	testany_delete_gatekeeper
List Plans	Read	testany_list_plans
View Plan Details	Read	testany_get_plan
Create Plan	Create	testany_create_plan
Update Plan (Note the risk of overwrite-style updates)	Update	testany_update_plan
Delete Plan	Delete	testany_delete_plan
Transfer Plan Owner	Update	testany_assign_plan

Common helpers:

• testany_get_my_workspaces: Select workspace
• testany_list_pipelines: Select pipelines

Core Knowledge

Gatekeeper (Webhook Trigger)

The core capability of Gatekeeper is:Trigger the execution of a "Pipeline Group" via Webhook(that is, trigger a group of pipelines).

Important clarification (to avoid misunderstanding):

• Gatekeeper itself is not a "pass rate threshold" configuration center, nor does it provide a "pass rate threshold" field on the backend
• "Whether to allow deployment" is logic on the CI/CD platform: after calling the Webhook, you can then decide whether to continue based on the Testany Execution result

Create Gatekeeper (Recommended Process)

Prerequisite: You need to know the list of pipelines to trigger (pipeline keys).

Creating and binding are two separate things：

1. Create Gatekeeper (generate Gatekeeper Key and webhook token)
2. Bind pipelines to Gatekeeper (otherwise the webhook will report an error "no pipelines in gatekeeper")

Operation Steps：

1. testany_get_my_workspaces → 选择 workspace
2. testany_list_pipelines → 找到要触发的 pipelines
3. testany_create_gatekeeper → 创建 gatekeeper，得到 gatekeeper key（形如 G-<WS>-<HEX>）
4. 绑定 pipelines 到 gatekeeper（Pipeline Group 绑定）
   - 优先使用 MCP 的 pipeline-group 绑定工具（如果已提供）
   - 如果 MCP 暂不支持绑定：提示用户去 Testany UI 的 Gatekeeper 详情页里选择 pipelines 完成绑定
5. testany_update_gatekeeper → 配置 trigger_method / trigger_name / trigger_condition / watchers / notify_ignore_success / owned_by 等
6. Webhook 集成：使用 Gatekeeper webhook URL 在外部系统触发

How to Obtain the Webhook URL (Important Limitation)

The Gatekeeper webhook URL is usually in the form: https://<host>/api/v2/gatekeeper/webhook/<hook_token>

Note: The current backend internal API (used by MCP) may not return this when fetching Gatekeeper details: hook_url. If the Gatekeeper details returned by MCP do not contain hook_url, please ask the user to copy the Webhook URL from the Gatekeeper details page in the Testany UI, and then use it for CI/CD integration.

Plan (Scheduled Plan)

Scheduled plans are used to automate test execution.

Cron Expression Format: Standard 5-field (UNIX)分 时 日 月 周

Time zone：

• timezone When empty, the backend defaults to Asia/Shanghai
• The interpretation of Cron is based on the plan's timezone

Scenario	Cron Expression	Description
Every day at 2 AM	0 2 * * *	Daily regression
Every hour	0 * * * *	Continuous monitoring
9 AM on weekdays	0 9 * * 1-5	Pre-work check
Early morning every Monday	0 0 * * 1	Weekly report generation

Create a Plan (recommended process)

Once the Plan is created successfully, it will be triggered automatically according to the schedule. There is no extra step of "enable after creation".

Steps：

1. testany_get_my_workspaces → 选择 workspace
2. testany_list_pipelines → 选择要定时执行的 pipelines
3. testany_create_plan → 创建计划（需要 schedule_expr，建议同时填写 timezone / schedule_str / watchers）

Update a Plan (high risk: avoid "partial updates")

The backend's implementation of update plan is closer to an "overwrite-style update", and missing fields such as pipelines / schedule_expr may cause errors or overwrite fields with empty values.

Safe update steps：

1. testany_get_plan Read the existing configuration
2. Construct a "complete update payload" based on the existing configuration (including at least pipelines, schedule_expr, timezone, notify_ignore_success, etc.)
3. testany_update_plan Submit the update

CI/CD integration example

GitHub Actions

- name: Trigger Testany Gatekeeper
  run: |
    curl -X POST "${{ secrets.TESTANY_GATEKEEPER_WEBHOOK_URL }}" \
      -H "Content-Type: application/json" \
      -d '{"source":"github-actions"}'

Jenkins

stage('Quality Gate') {
    steps {
        sh '''
          curl -X POST "${TESTANY_GATEKEEPER_WEBHOOK_URL}" \
            -H "Content-Type: application/json" \
            -d '{"source":"jenkins"}'
        '''
    }
}

GitLab CI

quality_gate:
  stage: test
  script:
    - |
      curl -X POST "$TESTANY_GATEKEEPER_WEBHOOK_URL" \
        -H "Content-Type: application/json" \
        -d '{"source":"gitlab-ci"}'

Return format

After the task is completed, report to the user:

• Created resources (Gatekeeper/Plan)
• Key configurations (Cron expression, timezone, watchers, notify_ignore_success, trigger_method/condition/name, etc.)
• List of pipelines bound to the Gatekeeper (whether they are bound)
• Webhook URL (if obtainable; otherwise prompt the user to obtain it from the UI)
• Integration code example

Reference documentation

For detailed concepts, please refer to:

• Core concepts