SuperAPI AI Directory

api-testing

Coding & development

View repository

Updated 10-16 08:49

About this skill

API Interface Testing (API Testing)

References: Pact contract testing, Postman API Testing Best Practices, Stripe API Testing Approach, ISTQB Foundation.

Applicable Scenarios

  • REST / GraphQL / RPC interface testing
  • API contract verification (frontend-backend decoupling)
  • Authentication / permission matrix testing
  • Error code / HTTP status code testing
  • Idempotency / retry / concurrency testing
  • Webhook / asynchronous task testing
  • Interface performance baseline (in collaboration with performance-testing)

Core Principles

1. API 测试是金字塔的中间层
   比 E2E 快、比单元真实
  1. Test the contract, not the code The backend can change the implementation, but must not break the contract

  2. Happy path + failure path + boundary path 8 standard dimensions (see below)

  3. OpenAPI is the source of truth Automatic validation of tests and contracts

  4. Test data isolation Different test cases do not pollute each other


## API 测试的 8 个标准维度(每个端点必测)

| 维度 | 用例数 | 示例 |
|---|---|---|
| 1. 成功路径 | 1~3 | 正常参数返回 200 + Schema |
| 2. 参数校验失败 | 多 | 必填缺失 / 格式错 / 长度超 / 类型错 |
| 3. 认证失败 | 1~2 | Token 缺失 / 过期 / 无效 |
| 4. 权限不足 | 1~3 | 无权限 / 越租户 / 越用户 |
| 5. 资源不存在 | 1 | ID 不存在返回 404 |
| 6. 资源冲突 | 1~2 | 状态冲突 / 版本冲突 / 唯一约束 |
| 7. 限流 | 1 | 超出 QPS 返回 429 + Retry-After |
| 8. 服务异常 | 1 | 模拟 500 时调用方处理 |

## 契约测试(Contract Testing)

```text
传统集成测试:
  调用方 + 真实服务 → 快但脆弱

契约测试:
  调用方 ↔ 契约(OpenAPI)↔ 服务方
  双方各自验证契约,不依赖对方在线

工具:
  - Pact(消费者驱动)
  - Spring Cloud Contract
  - OpenAPI 校验器(如 Dredd)

The API layer of the test pyramid

     /\
    /E2\
   /----\
  / API  \  ← 这里:覆盖业务规则、错误处理、契约
 /--------\
/  单元   \

Workflow

1. 拿到 OpenAPI 契约(或 API 文档)
   ↓
2. 列端点清单
   ↓
3. 每个端点按 8 维度设计用例
   ↓
4. 准备测试数据(前置 + 隔离)
   ↓
5. 执行测试(Postman/REST Assured/Karate/pytest)
   ↓
6. 校验:
   - HTTP 状态码
   - 响应 Schema(与 OpenAPI 比对)
   - 响应字段值
   - 数据库副作用(写操作)
   - 异步副作用(消息 / Webhook)
   - 日志(trace_id / 错误日志)
   ↓
7. 失败 → bug-reporting
   ↓
8. 转回归用例

Authentication Testing Matrix

对每个端点:

| 角色 | Token | 期望 |
|------|-------|------|
| guest | 无 | 401 |
| user | 自己资源 | 200 |
| user | 他人资源 | 403 或 404 |
| admin | 任意资源 | 200 |
| user | 跨租户 | 403 或 404 |
| user | Token 过期 | 401 |
| user | Token 篡改 | 401 |

Idempotency Testing

对所有 POST 创建 / 支付 / 扣减 类接口:

1. 同一 Idempotency-Key 调用两次
   - 预期:第二次返回首次结果,不重复创建

2. 不同 Idempotency-Key 同样请求体
   - 预期:创建两次(业务允许时)或拒绝(唯一约束时)

3. 网络中断后重试
   - 预期:可重试,结果一致

4. 并发同 key
   - 预期:仅一次成功

Failure Path Test Case Standard Library

通用失败用例(每个端点都要测):

✓ 必填字段缺失 → 400 + VALIDATION_ERROR
✓ 字段类型错 → 400 + INVALID_TYPE
✓ 字段长度超限 → 400 + LENGTH_OUT_OF_RANGE
✓ 字段值非法(枚举外)→ 400 + INVALID_ENUM
✓ JSON 格式错 → 400 + JSON_PARSE_ERROR
✓ Content-Type 错 → 415
✓ 超大 body → 413
✓ 未认证 → 401 + WWW-Authenticate
✓ 已认证但无权限 → 403
✓ 资源不存在 → 404
✓ 资源冲突 → 409
✓ 业务校验失败 → 422
✓ 限流 → 429 + Retry-After
✓ 服务异常 → 500

Performance Baseline (in collaboration with performance-testing)

对每个核心端点测:
  - P50 / P95 / P99 响应时间
  - 单实例 QPS
  - 错误率(对比基线)

工具:
  - k6 / wrk / Locust
  - Postman Runner(小规模)
  - Apache Bench(快速基线)

Toolchain

Tool Purpose Advantages Disadvantages
Postman / Apifox Manual / semi-automated Easy to get started, UI-friendly Hard to manage at scale
REST Assured Java automation Strong typing, IDE-friendly Limited to the Java stack
Karate DSL automation Easy to read, built-in Mock Learning curve
pytest + requests Python automation Flexible, strong ecosystem Requires building your own structure
Pact Contract testing Decouples consumers / services Complex process
Dredd OpenAPI validation Automatically compares contracts Poor flexibility
Schemathesis Property testing Automatically generates fuzz Test cases lack business sense

Common API Bug Types

1. 字段大小写不一致(status vs Status)
2. 时间字段时区不统一
3. 金额字段精度丢失(float)
4. ID 字段精度问题(JSON number 超 2^53)
5. 必填判断不严(null vs 缺失 vs 空字符串)
6. 错误码不稳定(同错误返回不同 code)
7. 分页参数边界(page=0 / page=-1)
8. 排序字段未限制(任意字段都能排序,性能爆炸)
9. 批量接口无上限(10000 条一起)
10. 删除接口幂等性(第二次返回 404 应该 204)
11. 异步接口同步返回错误码(200 包错误)
12. CORS 预检 OPTIONS 错误
13. Content-Length 不一致
14. Header 大小写敏感
15. 字符编码(UTF-8 / GBK 混用)

Quality Self-Check

□ 每个端点 8 维度都覆盖
□ 鉴权矩阵完整测试
□ OpenAPI Schema 自动校验
□ 写接口有数据库副作用断言
□ 幂等测试覆盖所有写接口
□ 失败路径占比 ≥ 40%
□ 性能基线建立
□ 测试数据隔离
□ trace_id / request_id 验证
□ 与契约测试集成(如有)

Common Pitfalls

  1. Only testing 200——50% of bugs are on the error path
  2. Not validating Schema——field type changes go unnoticed
  3. Not testing privilege escalation——user A can access user B's resources
  4. Not testing idempotency——network retries cause duplicate charges
  5. Test data not isolated——crashes when run in parallel
  6. Not testing rate limiting——only discovered after being attacked in production
  7. Not verifying database side effects——the interface returns 200 but the data is not written
  8. Not testing asynchronous side effects——webhook / MQ messages are missed
  9. Postman collection not under version control——lost and rewritten
  10. Manual runs not integrated into CI——regression gap

Supporting Templates

  • templates/api-test-template.md — single-endpoint 8-dimension testing + authentication matrix + idempotency testing + Schema validation

Collaboration with other skills

上游:
  API 设计工作流 → OpenAPI 契约
  test-case-design → 用例方法
  test-data-management → 测试数据

下游:
  bug-reporting → 失败时记录
  regression-testing → 入回归套件
  performance-testing → 基线建立
  自动化测试工作流 → 实现自动化