api-testing

API Testing (api-testing)

Interface-level testing—another execution path beyond E2E.

Input: API documentation (OpenAPI/Swagger), automatable interface test cases in the use case Schema, execution_model , information about the environment under test (base URL, account/Token); the performance axis (type matrix axis 1) is handed over to package 专项移交_性能_*.yaml
Output (written to disk): API test scripts (pytest + requests, or the project's established tech stack) + execution sub-report 测试报告_api_{日期}.md(naming follows ../core/report-template.md header note, entries aligned with its §3; qa breakpoint criteria are checked and accepted by this file name); when performance is undertaken, additionally produce k6 load test scripts and load test report (references/k6-conventions.md）
Boundary: Web UI flow → automated-e2e-testing; manual interface test case design → test-case-writing

When to Use

Given OpenAPI/Swagger documentation, need to produce and run interface automation tests
Filter interface-level automatable test cases from the use case Schema, convert them into API scripts for execution
Need to cover interface-layer special topics such as authentication/privilege escalation, idempotency, concurrent writes, error responses
Given a performance handover package or interface list, need to generate and run load test scripts (k6)—according to references/k6-conventions.md

When NOT to Use

Web UI interaction flow (clicks / page state) → automated-e2e-testing
Writing manual test cases for interfaces → test-case-writing
End-to-end pipeline → qa orchestration
Standalone load test environment setup and capacity assurance → Ops/special-topic collaboration (agent-side load test undertaking see references/k6-conventions.md; external execution scenarios follow the handover package of test-strategy )
Mock Server setup → development collaboration matter, outside the scope of this skill

Scaffolding (default pytest + requests, replaceable with the project's established stack)

api-tests/
├── conftest.py            # fixture：base_url、会话/Token、环境配置（环境变量注入，可由 .env 加载，不硬编码）
├── common/
│   └── client.py          # 统一请求封装：日志、超时、鉴权头、断言辅助
├── test_{模块}_{接口}.py   # 一个接口一个文件，test 名沿用 TC 编号
└── requirements.txt

# common/client.py —— 统一请求封装（requests.Session 不支持 base_url，必须显式拼接）
import requests

class Client:
    def __init__(self, base_url: str, token: str):
        self.base_url = base_url.rstrip("/")
        self.s = requests.Session()
        self.s.headers.update({"Authorization": f"Bearer {token}"})

    def request(self, method: str, path: str, **kwargs):
        url = f"{self.base_url}/{path.lstrip('/')}"
        kwargs.setdefault("timeout", 10)
        return self.s.request(method, url, **kwargs)

    def get(self, path, **kw):  return self.request("GET", path, **kw)
    def post(self, path, **kw): return self.request("POST", path, **kw)
    # put / delete / patch 同理扩展

def login(user: str, password: str) -> str:
    """按项目实际登录接口实现（如 POST /login 换 token）——占位，勿直接照抄"""
    raise NotImplementedError("按项目登录接口实现")

# conftest.py 关键 fixture
import os
import pytest

from common.client import Client, login

@pytest.fixture(scope="session")
def client():
    base_url = os.environ["API_BASE_URL"]          # 环境与账号不硬编码，走环境变量
    token = login(os.environ["API_USER"], os.environ["API_PASSWORD"])
    return Client(base_url, token)

Sensitive information (account/Token/environment address) is always injected via environment variables, never into the code repository (consistent with the constants convention of automated-e2e-testing ).

Workflow

1. Input parsing and scope confirmation

Extract from OpenAPI documentation: interface list, parameter table (required/type/range/default value), error codes, authentication method
Filter from use case Schema:automation.framework: api test cases, and execution_model: dev-collab test cases where framework is not manual → conversion targets (test names follow TC numbers:test_TC_05_01_写入字段读回一致; dev-collab test cases with framework: manual remain manual collaboration, not converted)
Environment unknown → ask the user (base URL, account, whether a production bypass environment is writable),ask when uncertain, do not guess interface behavior(question format and decision recording uniformly follow ../core/clarify-pattern.md, scenario uses "execution confirmation")

2. Test case design (at this point load `../core/testing-principles.md`, method details `../core/methods/data-driven.md`）

One parameter matrix per interface (analysis process tool, construct in mind or as a draft, not written to disk as an intermediate file—conclusions go directly into test cases, consistent with test-case-writing's no-intermediate-file approach), parameter by parameter × attribute by attribute; focus on covering:

Category	Must-test points
Parameters	Required missing / type error / boundary values (empty/max/min/oversized, see `../core/methods/boundary.md`）
Authentication	No Token / expired Token / wrong Token / privilege escalation (other people's resource id)—privilege escalation method baseline `../core/methods/permission.md`(Role×Action×Resource matrix + vertical/horizontal privilege escalation, i.e., the execution layer of type matrix axis 2)
Idempotency	Repeated submission of the same business key → no duplicate creation; retry-safe
Concurrency	Concurrent writes to the same resource → no mutual overwriting, no intermediate state
Error responses	Trigger conditions for each error code + response body structure and wording
Data consistency	Read-after-write consistency; associated data remains consistent after cascading operations

The combination surface of multi-parameter interfaces follows ../core/methods/data-driven.md Section 2explicit downgradeexecution (full combination → pairwise combination → risk-based selection): do full coverage for each attribute of a single parameter; for cross-parameter interactions (cross-field rules / parameter dependencies), select a coverage tier,with pairwise combination as the default tier; downgrades and excluded combination surfaces are written into the rationale, not silently shrunk.

3. Script writing

One test file per interface; one test tests only one point, named following the TC numbering
Output budget discipline (to prevent empty truncation): for similar boundary/parameter validations use @pytest.mark.parametrize to merge into a single parameterized test; do not expand value by value into repeated test functions or excessively long repeated assertions — a single file exceeding ~250 lines should be converged via parameterization (the generation channel has an output limit; exceeding it will truncate and produce uncompilable code)
Assertion triad: status code + business code + key fields of the response body (do not write weak assertions that "only assert 200")
Test data is self-created and self-cleaned (setup creates / teardown deletes), does not depend on execution order; for data templates (unique names, etc.), first verify the material field constraints (maxLength/enum/format); the total template length (prefix + random segment) ≤ constraint upper limit −2 — hitting the limit exactly is a data self-creation defect (measured self-inflicted case: a unique name template of 22 characters hit the contract maxLength of 20; the two measured cases and test-case-writing §4 each keep a copy from the same source, and revisions must be synchronized in both places);Recheck cross-field business rules cell by cell in the parameter matrix(e.g., "the usage threshold cannot be lower than the face value") — combinations that violate the rules should be changed to legal values or split into explicit negative test cases; do not make implicit illegal combinations (measured self-inflicted case: an amount at the limit of 1000 paired with a threshold lower than the face value → 400 THRESHOLD_INVALID while the test case expected 201)
Test cases that depend on prior state explicitly create data in the preconditions; do not assume there is data in the database (data creation patterns:../core/methods/data-factory.md— makeX constructor / choose one of three data creation channels / prefix isolation)

4. Running and results

The report entries and statistical criteria use ../core/report-template.md as the sole source (including the machine-readable summary fragment, loaded at wrap-up), ensuring it can be directly assembled into qa the final report at wrap-up:

pytest api-tests/ -v --tb=short          # 全量
pytest api-tests/test_coupon_create.py   # 单文件

For failed test cases, first distinguish: system under test bug / environment issue / test case error —do not assume on your own, list environment issues and expectation ambiguities to ask the user (question format same as above,../core/clarify-pattern.md）
when failures ≥3, escalate tobatch triage: first classify by ../core/triage.md the four categories (A true defect / B asset issue [B1 product expectation change · B2 test case error] / C environment and dependency failure / D flaky), only category A enters the Bug recording process below, replacing single-case-by-single-case discrimination
Pipeline runs: when entering PR smoke / nightly tasks in headless mode, checkpoints are downgraded to pending items, artifacts are written to disk per the specification, and exit codes separate infrastructure failures from true defects — see the three conventions in ../core/pipeline-integration.md(loaded at this point)
Structural coverage supplementary evidence (optional): When the service under test has code and the test environment can be instrumented (Python service coverage run startup; JVM service JaCoCo agent), after interface test cases finish running, takethe line/branch coverage of the service under testas supplementary E3 evidence—used only to discoverinterfaces and branches with zero coverage/extremely low coverage(signals of missed testing, switch to adding test cases or raise the strategy level), not as a vanity metric to chase higher numbers; if there is no instrumentation condition, skip directly and do not block delivery
Bugs found: evidence (raw request/response text, timestamp) recorded per ../core/report-template.md §3 entries, root cause analysis handed over to bug-analysis

5. Delivery

Script path + run statistics (§2 execution statistics: P0/P1/P2 × pass/fail/blocked/not executed, by ../core/report-template.md) + Bug entries + remaining issues list + (when instrumented) structural coverage summary: list of zero-coverage/low-coverage interfaces.

Contract and schema consistency (type matrix axis 10 execution layer)

When axis 10 decision is include, execute according to this section (decisions and levels come from test-strategy's type_scope; this section only handles execution form); report collection follows report-template §7 table, fill in the executor column truthfully.

Two-layer form (division of labor does not substitute for each other)

Layer	Object	Method
schema consistency	Response body vs OpenAPI definition field by field (type/required/enum)	Schemathesis one-command integration:`schemathesis run openapi.yaml --base-url $API_BASE_URL`; or hand-write jsonschema assertions and merge them into existing test files
structural-level negative fuzzing	structural robustness (type mismatch / overly long / format distortion)	Schemathesis automatically generates invalid inputs. Division of labor with §2 parameter matrix:the matrix handles business semantic combinations, fuzzing handles structural robustness

Concept boundaries

schema validation ≠ consumer contract: schema only verifies "the response conforms to the definition", contract verifies "the set of interactions allowed between consumer and provider"
Criteria for enabling Pact-style consumer-driven contracts: multi-consumer microservices with frequent field evolution—for single-system projects, schema consistency is sufficient; do not adopt a contract framework
Contract execution artifacts enter the execution strategy decision according to type_scope (script-type consumption method); if mock/sandbox dependencies are unavailable, record blocked per matrix R5

Execution discipline

5xx / undefined error codes found by fuzzing are regarded as candidate defects, attributed item by item (same process as §4 failure triage), and are not automatically recorded as Bugs
All schema validations passing ≠ no contract problems: field semantic changes (structure unchanged) are invisible at the schema layer; semantic regression relies on business test cases

Common Mistakes

Error	Consequence	Correct practice
Only assert status code, not business code and response body	Bugs missed (200 but business failure)	Status code + business code + key fields trio
Hard-code environment addresses and accounts	Cannot run across environments, leaks sensitive information	Environment variable injection
Shared mutable state between tests	Order dependency, flaky failures	Self-created data + self-cleanup, each test independent
Repeated submission without testing idempotency	Duplicate-creation-type Bugs go live	Duplicate requests with the same business key must be tested
All failures are recorded as bugs	False positives pollute the report	First attribute the cause (system/environment/test case); when ambiguous, ask the user
For no-permission/privilege escalation, only test frontend behavior	Missed detection of privilege escalation not blocked by the backend	Call the API directly to test authentication (no Token/expired/another user's id)
For multi-parameter APIs, either expand every value fully or sample arbitrarily	Combinatorial explosion truncation / silent missed testing of parameter interaction defects	Explicitly select a level per the downgrade strategy (full combination → pairwise → risk-based selection), and keep a record of the downgrade
Schema test cases with placeholders/fictitious entry points are still directly converted into scripts	Hallucinated scripts: they run successfully but do not test the real API	Before conversion, pass the `../core/executability.md` red-line gate; those that cannot be completed are temporarily deferred to the legacy list
If all schema validations pass, the contract is considered fine	Missed detection of field semantic changes (structure unchanged)	Schema only verifies structure; semantic regression relies on business test cases (see the "Contract and schema consistency" section)