SuperAPI AI Directory

e2e-run

Testing & quality

View repository

Updated 10-16 10:20

About this skill

E2E Run

Run the e2e assets that already exist in the current project, and consolidate execution evidence, problem classification, and next-step recommendations into stable artifacts. This skill is not used to generate new test cases; if the repository has no e2e scripts or specs, it should clearly recommend using /e2e-find. This skill is only intended for explicit user triggering and is not used for chained invocation by other skills.

Goals

  1. Discover existing e2e assets first, then decide the execution path; do not blindly run commands.
  2. Prefer reusing the project's native test framework; only fall back to spec-driven validation when scripts are missing.
  3. Save a traceable snapshot for every execution, including a human-readable report and a machine-readable summary.
  4. Problem analysis must be actionable, distinguishing environment, data, script failure, spec drift, and unknown problems.

Additional Resources

Evaluation Assets

  • evals/evals.json Provides trigger and behavior examples, used to verify whether assets are discovered first, whether routing to script execution or spec validation is correct, and whether it falls back to /e2e-find
  • agents/openai.yaml when cases are missing. Provides the UI metadata for this skill, which should be consistent with the positioning and trigger words in this document

Workflow

Step 1: Gather project context

Prioritize reading the following sources of truth:

  • In the root directory and subpackages, package.json
  • pnpm-workspace.yaml、turbo.json、nx.json
  • playwright.config.*、cypress.config.*
  • Makefile、docker-compose*.yml
  • README*、docs/

If the user passes in $ARGUMENTS, treat it as a scope hint, for example:

  • Route or page:/login
  • Feature:checkout
  • Case path or directory:tests/e2e/auth

Step 2: Discover e2e assets

According to references/discovery-and-routing.md Discover and classify assets:

  • script: Playwright/Cypress/project-native e2e scripts that can be run directly
  • spec: Markdown specs, usually from specs/e2e/、docs/e2e/、tests/e2e/specs/

Record the following facts:

  • Which runners, script commands, test directories, and spec files were discovered
  • The default run scope is smoke、critical、main-path or the scope explicitly specified by the user
  • Which candidate assets were excluded, and the reasons for exclusion

If there is neither script nor spec, stop immediately, and clearly recommend that the user first use /e2e-find to generate or complete test cases.

Step 3: Infer how to run the service

Based on references/discovery-and-routing.md , infer the startup command and entry address of the service under test.

Prefer:

  1. The command closest to the project's recommended development method
  2. A target application that can support the current $ARGUMENTS scope validation
  3. The command with the lowest startup cost and the easiest way to obtain stable evidence

Do not blindly run multiple candidate commands at the same time without explanation. If the inferred correct startup or execution command exceeds the current allowlist, you may return needs-input, and clearly state what was discovered and why this skill should not blindly run it.

Step 4: Select execution mode

Route in the following order:

  1. If executable scripts are discovered, primarily use script mode
  2. If there are no scripts but Markdown specs exist, enter spec mode
  3. If both types exist, specs may be used to help narrow scope, name artifacts, and supplement problem analysis, but do not let specs override the real script results

If the user has not specified a scope, by default only run smoke、critical、main-path Small set; if the project contains none of these markers, choose the group of cases with the clearest main path. If safe convergence is not possible, stop and explain why.

Step 5: Execute and collect evidence

Script mode

  • Start the service
  • Record the PID or equivalent identifier of the process started in this run; only processes started by this run may be cleaned up
  • Run the project's native e2e command or the minimal necessary runner command
  • Write stdout, stderr, and key failure snippets to the snapshot directory
  • Preserve the runner's built-in reports, traces, screenshots, videos, or junit/xml files as much as possible

Spec mode

  • Start the service
  • Record the PID or equivalent identifier of the process started in this run; only processes started by this run may be cleaned up
  • Read the main flow, preconditions, final-state assertions, and code anchors from the target specification
  • Prefer to use agent-browser to perform real interaction verification
  • If unavailable or unstable, downgrade to minimal evidence such as HTTP reachability checks with retries, entry HTML, key URL redirects, key API responses, etc.
  • Do not disguise a specification as an already-automated script; mark steps that cannot be reliably verified as blocked

Step 6: Save outputs

Strictly follow references/output-policy.md:

  • Choose a stable in-project results directory for this run
  • Create an independent snapshot directory for a single execution
  • Write at least report.md、summary.json、service.log、runner.log
  • If there are screenshots, traces, videos, or HAR files, put them all in evidence/
  • During the wrap-up phase, stop the service process started by this skill; if the service was not started by this run, only record the current status and do not actively terminate it

report.md and summary.json fields must respectively follow:

Step 7: Analyze issues

Attribute each failure, blockage, or suspicious result; categories are limited to:

  • environment
  • data/auth
  • test failure
  • spec drift
  • unknown

The analysis must include:

  • The use case or specification that triggered the issue
  • Source of evidence
  • Why it was classified into that category
  • Minimal next-step recommendation

Step 8: Report results to the user

When reporting, at least explain:

  • This time used script or spec mode
  • Which startup command and entry address were chosen
  • Which use cases or specifications were executed
  • How many passed, failed, and were blocked
  • The results directory and the locations of key evidence files
  • If there are no use cases, why it is recommended to switch to /e2e-find

Guardrails

  • Do not start the service first and then guess the use cases.
  • Do not run the full e2e suite by default unless the user explicitly requests it.
  • Do not overwrite the previous results; each execution must preserve an independent snapshot.
  • Do not dump logs directly into the final reply; write logs to files, and give conclusions and paths in the reply.
  • Do not describe specification-driven verification as "all tests have passed automatically."
  • Do not omit failure evidence and next-step recommendations.

Examples

Example 1: Execute existing Playwright use cases

The user says:运行这个项目的登录 e2e,并把失败原因记下来

You should:

  1. First find the Playwright configuration, test directories, and login-related tags or files
  2. Determine the service startup command and login entry point
  3. Run the minimal necessary login use case
  4. Save the results as a snapshot and output failure attribution

Example 2: Only a specification, no script

The user says:根据现有 e2e 规格验证 checkout 主流程,并保存结果

You should:

  1. Find the checkout specification file
  2. Start the target service
  3. Prefer to use agent-browser Verify the main flow, otherwise fall back to HTTP and entry-point evidence
  4. Mark the result as spec mode, and explain which steps still require real automation scripts

Example 3: No e2e assets at all

The user says:帮我跑一下这个项目的 e2e

You should:

  1. Discover that the repository has neither e2e scripts nor e2e specs
  2. Clearly provide the directories and configurations that were checked
  3. Suggest that the user switch to /e2e-find to first generate or fill in test cases

Troubleshooting

Unable to determine the startup command

List the files checked, candidate commands, and missing information. Do not blindly run multiple services.

Requires an account, verification code, or external dependency

Allow continuing to collect minimal evidence, but mark the affected test cases as blocked or needs-input, and explain the gaps in the report.

The runner's built-in report directory is unstable

Prefer to keep the original runner artifact paths, while creating an index or copying necessary evidence within this snapshot directory. Do not forcibly rewrite the project's existing configuration.

The service may leave behind background processes

Only allow terminating processes that were explicitly started by this skill and whose PIDs were recorded. If the port is already occupied by an existing service, only record it and reuse it or return needs-input, and do not kill processes of unknown origin.