pentest-web-enumeration

Pentest Web Enumeration

Purpose

Own the active web-enumeration phase from scoped targets to a deduplicated application inventory and prioritized foothold hypotheses. Use Nuclei as the primary scanner, but do not substitute scanner output for direct validation.

Boundaries And Handoffs

• Require explicit authorization and record the target, allowed ports, identities, exclusions, and rate limits before active enumeration.
• Remain the owner while the blocker is discovering or fingerprinting web surfaces.
• Hand off to pentest-web-application-logic-mapper when multi-step workflows, authenticated routes, or application state need mapping.
• Hand off to the matching auth, access-control, input/protocol, XSS, business-logic, OOB, or CVE skill when a concrete vulnerability class becomes the blocker.
• Hand off to pentest-exploit-execution-payload-control only after a primitive is confirmed.
• Use pentest-evidence-structuring-report-synthesis for a full client deliverable; produce the phase report here regardless.

Required Inputs

Normalize what is known into:

• scope: URLs, hosts, domains, IPs, ports, exclusions, and redirect policy
• access: unauthenticated or authorized credentials, headers, cookies, proxy, and client certificate
• limits: per-host rate, concurrency, time window, request budget, and prohibited actions
• goal: broad surface inventory, a named application, or initial-access prioritization

If scope is ambiguous, do not infer permission for sibling domains, third-party hosts, destructive methods, credential attacks, uploads, or DAST fuzzing.

Operating Rules

1. Label claims confirmed, hypothesis, or rejected.
2. Preserve raw output before filtering; derive normalized files from raw artifacts.
3. Use structured output such as JSONL whenever available and timestamp each run in UTC.
4. Deduplicate by canonical URL while retaining scheme, port, SNI/Host header, redirect chain, and authentication context.
5. Treat technology detection as corroborated only when at least two independent signals agree, or one high-specificity artifact exposes a version directly.
6. Treat status, length, title, favicon, or scanner matches as leads until a direct request confirms the relevant behavior.
7. Use positive and negative controls for wildcard DNS, wildcard vhosts, soft 404s, uniform redirects, and CDN/WAF responses.
8. Prefer bounded phases with stop conditions over a large unattended pipeline.
9. Do not install or update tools during the run. Record missing or stale tools and use the safest available fallback.
10. Never run Nuclei DAST, intrusive, fuzz, brute-force, default-login, headless, code, or unsigned templates unless scope explicitly permits that class.

Workspace

Run scripts/init-web-enum-workspace.sh <run-directory> to create the artifact layout. Keep secrets out of command history and reports. Store reusable authenticated scan material in access-controlled files and reference it with tool-supported options.

Required artifact classes:

scope/       original and normalized targets, exclusions, auth context notes
raw/         immutable tool output and stderr logs
normalized/  canonical live URLs, applications, routes, scripts, APIs, parameters
evidence/    minimal requests/responses for validated claims
reports/     phase report and machine-readable finding summaries

Workflow

1. Preflight And Scope Controls

• Read local README.md or README.txt, to-do.txt, and creds.txt if present.
• Check required binaries with command -v; inspect local -h and -version output before relying on flags.
• Use pdtm -h to inspect manager semantics. Do not use pdtm -ia as an inventory command because it means install all.
• Resolve DNS with dnsx when hostnames need normalization. Preserve CNAMEs and all in-scope addresses.
• Establish random-host and random-path controls before vhost or directory enumeration.

Gate: Continue only with a scope file, exclusions, and a defensible request rate.

2. Normalize Live Web Services

• Probe supplied hosts and ports with HTTPx; retain failed probes separately instead of deleting them.
• Capture URL, status, title, length, server, technology, IP, CNAME, CDN/WAF, favicon, JARM, TLS, HTTP/2, pipeline support, and redirect chain when useful.
• Re-probe ambiguous services with curl using explicit scheme, SNI/Host, method, headers, and redirect behavior.
• Compare GET, HEAD, and OPTIONS only as low-impact semantic checks. Do not infer that an advertised method is exploitable.

Gate: Produce normalized/live-urls.txt and an application inventory keyed by scheme, host, port, and Host header.

3. Fingerprint Applications And Infrastructure

• Combine HTTPx/Wappalyzer signals, WhatWeb, headers, cookies, HTML markers, favicon hashes, TLS metadata, error pages, asset paths, and source maps.
• Distinguish origin evidence from CDN, reverse proxy, WAF, load balancer, and framework evidence.
• Record OS only when protocol banners, error pages, path semantics, or a versioned server artifact support it; otherwise label it unknown.
• Use curl to inspect /, redirects, headers, robots, sitemap, security policy, common API descriptions, and high-value discovered paths.
• When a product and version are credible, hand the exact evidence to the CVE research skill before exploit construction.

Gate: Record corroborated technologies, conflicting signals, version confidence, and technology-conditioned next checks.

4. Discover Virtual Hosts

• Derive candidate names from certificates, DNS, redirects, CSP, links, JavaScript, error messages, and target naming conventions before wordlist fuzzing.
• Use FFUF against the Host header with the correct IP/SNI routing. Establish a random-host baseline and filter by multiple response properties, not status alone.
• Revalidate every candidate with DNS override or explicit Host header and compare body, title, redirect location, and fingerprint to controls.
• Do not add out-of-scope sibling domains merely because a certificate or page references them.

Gate: Add only directly reproducible vhosts to the live inventory; retain noisy candidates as hypotheses.

5. Crawl Routes, JavaScript, And APIs

• Run Katana in standard mode first with explicit scope, bounded depth and duration, known-file discovery, JavaScript parsing, and form extraction.
• Use headless mode only for SPA/DOM surfaces missed by standard crawling, with a separate artifact set and tighter concurrency.
• Enumerate robots, sitemaps, manifests, service workers, source maps, OpenAPI/Swagger, GraphQL, WSDL, and client-side configuration references.
• Extract and normalize scripts, route paths, absolute URLs, API bases, methods, parameters, WebSocket endpoints, upload/download paths, and auth boundaries.
• Fetch important first-party JavaScript directly and inspect strings or source maps; validate extracted routes before marking them reachable.

Gate: Produce deduplicated files for routes, scripts, API candidates, parameters, forms, and third-party references.

6. Run Focused Content Discovery

• Select the Feroxbuster wordlist dynamically from observed language, framework, server, naming style, and application purpose.
• Start fast but bounded: common directories, non-recursive, moderate threads, explicit time limit, and soft-404 controls.
• Expand only on signal. Add extensions based on observed files, then test high-value configuration, backup, source, archive, log, document, script, map, and environment-file suffixes where allowed.
• Use targeted per-directory recursion only on high-value branches. Avoid recursive scans of generated calendars, search spaces, IDs, or wildcard routes.
• Revalidate sensitive-looking files with curl. Distinguish an exposed secret from a login page, denial page, placeholder, or empty response.

Gate: Stop a branch when results remain uniform after one materially different control, the time budget expires, or no new trust boundary appears.

7. Run Nuclei-Led Early Checks

• Use Nuclei on the normalized live URL set, not the unfiltered seed list.
• Always persist the reporting database with -rdb, machine-readable JSONL, timestamps, error logs, and a project path for the run.
• Begin with the baseline in references/command-playbook.md. Preserve the requested default of excluding ssl,tls only when those checks are intentionally out of scope; otherwise run a separate bounded TLS pass.
• Use -as or selected tags/templates after fingerprinting. Query available tags/templates with -tgl and -tl rather than assuming names.
• Scale -c, -bs, -rl, timeouts, and per-host limits to target count, latency, WAF behavior, and authorization constraints. Reduce concurrency for one complex app; use host-aware limits for many URLs.
• Run technology-specific passes only for corroborated products. For WordPress, run WPScan plus selected wordpress Nuclei templates; apply analogous focused checks to other confirmed stacks.
• Validate every high-impact Nuclei match with stored request/response evidence and a direct control request. Scanner metadata alone is not confirmation.

Gate: Separate confirmed findings, plausible scanner leads, false positives, informational fingerprints, and untested template classes.

8. Consolidate And Prioritize

• Deduplicate URLs by canonical route while retaining distinct methods, ports, vhosts, roles, and content types.
• Truncate noisy scanner lists by grouping on root cause, application, and template ID. Keep counts and artifact paths for omitted repetitions.
• Rank initial-access candidates by reachability, authentication requirement, privilege gained, version confidence, exploit preconditions, validation confidence, and test cost.
• Propose exactly three foothold paths when evidence permits. Build them only from confirmed primitives and clearly marked hypotheses; do not present speculative chains as established paths.
• Write reports/web-enum-report.md using assets/web-enum-report.md.

Minimum Output Contract

1. Confirmed findings with confidence, impact, and evidence IDs.
2. Strong hypotheses with the next minimal deterministic test.
3. Evidence index with command/request, response artifact, timestamp UTC, and control.
4. Deduplicated applications, vhosts, routes, scripts, APIs, forms, and technology inventory.
5. Three prioritized initial-access paths or an explicit statement that evidence supports fewer than three.
6. Reproduction steps and tool/evidence gaps.

Resources

• Read references/command-playbook.md when selecting or adapting commands.
• Use assets/web-enum-report.md for phase reporting.
• Run scripts/init-web-enum-workspace.sh before a substantial enumeration run.