httpx-probe

httpx HTTP Liveness Probing and Fingerprinting Methodology

httpx is a multi-purpose HTTP probing tool from ProjectDiscovery. Core advantages:High-speed concurrency(default 50 threads) + Rich probes(status codes/titles/tech stacks/CDNs/screenshots, etc.) + Pipeline core(the bridge connecting port scanning and vulnerability scanning).

Project address:https://github.com/projectdiscovery/httpx

Phase 1: Basic Liveness Detection

# 单个目标
httpx -u http://target.com

# 批量目标（从文件）
httpx -l urls.txt

# 从 stdin 管道
cat urls.txt | httpx

Silent output (only show live URLs)

cat urls.txt | httpx -silent

## Phase 2: 信息探针（核心能力）

```bash
# 显示状态码 + 标题 + 技术栈（最常用组合）
httpx -l urls.txt -sc -title -tech-detect

# 显示状态码 + 内容长度 + Web 服务器
httpx -l urls.txt -sc -cl -server

# 显示 IP + CNAME + ASN
httpx -l urls.txt -ip -cname -asn

# CDN/WAF 检测
httpx -l urls.txt -cdn

# 响应时间
httpx -l urls.txt -rt

# 完整信息输出
httpx -l urls.txt -sc -title -tech-detect -server -ip -cdn -cl

Phase 3: Technology Stack Identification (Fingerprinting)

httpx has a built-in Wappalyzer dataset and can identify web frameworks, CMSs, programming languages, etc.:

# 技术栈检测
httpx -u http://target.com -tech-detect

# 输出示例：
# http://target.com [200] [WordPress] [PHP,MySQL,Apache]

# 批量识别 + JSON 输出
httpx -l urls.txt -tech-detect -json -o fingerprints.json

# 自定义指纹文件
httpx -l urls.txt -tech-detect -custom-fingerprint-file /path/to/fingerprints.json

Phase 4: Filtering and Matching

# 只显示 200 OK 的目标
httpx -l urls.txt -mc 200

# 排除 404 和 403
httpx -l urls.txt -fc 404,403

# 匹配包含特定字符串的响应
httpx -l urls.txt -ms "admin"

# 正则匹配
httpx -l urls.txt -mr "login|admin|dashboard"

# 按响应长度过滤
httpx -l urls.txt -fl 0

# 匹配特定 CDN
httpx -l urls.txt -match-cdn cloudflare

# 过滤错误页面
httpx -l urls.txt -filter-page-type error,parked

Phase 5: Screenshots (Headless)

# 页面截图
httpx -l urls.txt -screenshot

# 使用系统 Chrome
httpx -l urls.txt -screenshot -system-chrome

# 截图 + 技术栈 + 状态码
httpx -l urls.txt -screenshot -tech-detect -sc -title

Phase 6: Pipeline Integration

httpx is the ProjectDiscovery toolchain'score hub：

# 子域名 → HTTP 存活
subfinder -d target.com -silent | httpx -silent

# 端口扫描 → HTTP 存活 → 指纹
naabu -host target.com -silent | httpx -tech-detect -title -sc

# 子域名 → 存活 → 漏洞扫描
subfinder -d target.com -silent | httpx -silent | nuclei -severity critical,high

# 子域名 → 存活 → 爬虫 → 漏洞扫描（完整链）
subfinder -d target.com -silent | httpx -silent | katana -jc -silent | nuclei

# 网段扫描 → HTTP 存活 → 指纹 → 高价值目标筛选
naabu -host 10.0.0.0/24 -p 80,443,8080 -silent | httpx -tech-detect -title -sc -silent

Phase 7: Advanced Usage

# 探测多端口（单个主机的多端口 HTTP 检测）
echo target.com | httpx -ports 80,443,8080,8443

# 自定义请求方法
httpx -l urls.txt -x GET,POST

# 自定义 Header
httpx -l urls.txt -H "Cookie: session=abc"

# 跟随重定向
httpx -l urls.txt -follow-redirects

# TLS 信息抓取
httpx -l urls.txt -tls-grab

# 提取正则匹配内容（如提取邮箱）
httpx -l urls.txt -extract-preset mail,ipv4

# 自定义正则提取
httpx -l urls.txt -extract-regex "api[_-]?key['\"]?\s*[:=]\s*['\"]?([a-zA-Z0-9]+)"

# JSON 输出（包含完整信息）
httpx -l urls.txt -json -o results.json

# 控制并发和速率
httpx -l urls.txt -threads 100 -rate-limit 50

# 去重（过滤相似页面）
httpx -l urls.txt -filter-duplicates

Common Scenario Quick Reference

Scenario	Command
Quick liveness detection	cat urls.txt | httpx -silent
Technology stack identification	httpx -l urls.txt -tech-detect -sc -title
Find admin panels	httpx -l urls.txt -mr "admin|login|dashboard" -sc -title
CDN identification	httpx -l urls.txt -cdn -ip
Batch screenshots	httpx -l urls.txt -screenshot -system-chrome
Extract subdomains	httpx -l urls.txt -extract-fqdn -json
Connect naabu	naabu -host target -silent | httpx -tech-detect