SuperAPI AI Directory

test

Testing & quality

View repository

Updated 10-23 23:08

About this skill

You are executing the Test phase (4/6) of CocoBrew.

Before proceeding, verify that .cocoplus/ exists. If not: output "CocoPlus not initialized in this directory. Run $pod init to begin." Then stop.

Read .cocoplus/lifecycle/meta.json. Verify phases_completed contains "build". If not: output "The Build phase must be completed before testing. Run $build first." Then stop.

Generate Test Cases

Read .cocoplus/lifecycle/spec.md. For each Success Criterion and Deliverable:

  1. Generate a test case that validates that criterion or deliverable exists
  2. Each test case has: ID, description, test approach (SQL query / file existence / quality check), expected result

Execute Tests

For each test case:

  1. Run the test (SQL validation via SnowflakeSqlExecute if applicable, file existence check via Bash, quality check via quality-advisor)
  2. Record: PASS or FAIL with actual result
  3. If a test initially fails because of environment setup, permissions, missing metadata, or transient service behavior, document the recovery step explicitly before re-running. Do not hide the failed attempt.

Recovered tests count as PASS only when the final assertion actually passes. The original failure must still appear under ## Recoveries with the failed command/query, error, recovery action, and final outcome.

Write Test Results

Write .cocoplus/lifecycle/test.md:

# Test Results

**Date:** [ISO 8601 timestamp]
**Phase:** Test (4/6)
**Phase ID:** test-YYYYMMDD-001

## Summary
- Tests Run: [count]
- Passed: [count]
- Failed: [count]

## Test Cases

### [TEST-001]: [description]
**Approach:** [SQL / file check / quality]
**Expected:** [expected result]
**Actual:** [actual result]
**Status:** PASS / FAIL

[repeat for each test case]

## Failures
[Detail any failures with remediation suggestions]

## Recoveries
[For each recovered test: original command/query, error observed, recovery action taken, final PASS/FAIL]

Update State

Update lifecycle/meta.json — add test to phases_completed. Append to AGENTS.md (≤200 lines): Phase: Test (4/6) — [PASS/FAIL summary]

Git Commit

git add .cocoplus/lifecycle/test.md .cocoplus/lifecycle/meta.json .cocoplus/AGENTS.md
git commit -m "test: test execution and validation"

Output: "Test phase complete. [N] tests passed, [M] failed. Check .cocoplus/lifecycle/test.md for details. Proceed to $review."

Anti-Rationalization

Shortcut / Temptation Why It Fails
Skip tests for deliverables that "obviously exist" The spec mandates verifiable outcomes — assumption-based passing is not evidence
Mark phase complete even if some tests failed Failing tests are open issues; committing and moving on hides defects from the Review phase
Generate test cases without reading spec.md success criteria Test cases not anchored to success criteria will miss spec-mandated outcomes entirely
Collapse recovered failures into clean passes Recovery behavior is release evidence; hiding it removes the signal needed to fix flaky or environment-sensitive tests

Exit Criteria

  • .cocoplus/lifecycle/test.md exists with a ## Summary section showing Tests Run, Passed, and Failed counts
  • .cocoplus/lifecycle/test.md contains ## Recoveries, even if it says "No recovered failures"
  • Every Success Criterion from spec.md has at least one corresponding test case in test.md
  • .cocoplus/lifecycle/meta.json phases_completed array contains "test"
  • Git commit with message test: test execution and validation exists in log