SuperAPI AI Directory

judge-pentest

Testing & quality

View repository

Updated 10-16 02:28

About this skill

Penetration Testing Evaluation Checklist

Web Application Vulnerability Coverage Check

Check each attack surface below for whether it has been tested; mark untested ones as gaps:

Injection

  • SQL Injection (login forms, search, API params, cookies)
  • XPath Injection
  • LDAP Injection
  • Command Injection (OS Command Injection)
  • SSTI (Server-Side Template Injection)
  • XXE (XML External Entity Injection)

Cross-Site

  • Reflected XSS (search box, URL params, error pages)
  • Stored XSS (comments, feedback, user profiles)
  • DOM XSS
  • CSRF (transfers, password changes, critical operations)

Authentication & Authorization

  • Default/weak credentials
  • SQL injection auth bypass
  • Brute force protection (account lockout mechanism)
  • Username enumeration (error message differences)
  • Session management (Session Fixation, Cookie security attributes)
  • JWT/Token security (signature verification, algorithm confusion, plaintext encoding)
  • Vertical privilege escalation (regular user → admin functions)
  • Horizontal privilege escalation / IDOR (accessing other users' resources)

Business Logic

  • IDOR — account info viewing
  • IDOR — transfer/transaction operations
  • IDOR — password change
  • Negative/zero amount transactions
  • Concurrency/race conditions
  • Business flow bypass

Information Disclosure

  • Error page info leaks (stack traces, paths)
  • API documentation exposure (Swagger, WSDL)
  • Backup file disclosure
  • Sensitive config exposure
  • HTTP response headers (Server version, X-Powered-By)

Server-Side

  • SSRF (Server-Side Request Forgery)
  • File upload vulnerabilities
  • Path traversal / LFI / RFI
  • Deserialization vulnerabilities

Configuration

  • Directory listing

API-Specific

  • REST API auth bypass
  • API IDOR
  • API parameter tampering
  • API rate limiting

Known CVE/CNVD

  • Known CVEs for target product/tech stack
  • Known CNVDs for target product/tech stack

Evaluation Decision Logic

The goal of penetration testing is to discover as many vulnerabilities as possible; do NOT end prematurely.

# In pentest scenarios, complete is advisory only and does NOT trigger early exit
# The judge's core value is providing precise "what to test next round" feedback

if tested_categories >= 90% of total && two consecutive rounds with no new vulns:
    complete = true, confidence >= 0.8
else:
    complete = false
    feedback = explicitly list untested attack surfaces with specific testing suggestions
    missing_areas = names of untested categories

Important: Better to run one extra round than to miss one direction. Even if many vulnerabilities have been found, if there are still untested attack surfaces, return complete=false.

If a target product/tech stack was identified in this round but search_vulndb was never called, MUST return complete=false and require using search_vulndb(query="product name") to query known vulnerabilities.

Feedback Template

When complete=false, feedback should include:

  1. Completed work (acknowledge positively, avoid repetition)
  2. Specific missing directions (do NOT say "keep testing" vaguely — specify concrete endpoints + vulnerability types)
  3. Suggested test steps (e.g., "use sqlmap for deep injection testing on the query parameter of /api/search")