SuperAPI AI Directory
Partner中文AI newsFavorites
Skillsvisa-vulnerability-agentic-harness

visa-vulnerability-agentic-harness

Agents & models

View repository

Updated 10-16 07:27

About this skill

Visa Vulnerability Agentic Harness (VVAH) Skill

Skill by ara.so — AI Agent Skills collection.

VVAH is Visa's open-source harness for autonomous vulnerability discovery using frontier AI models. It implements a 9-stage pipeline: attack surface mapping → threat modeling → multi-lens research → adversarial verification → exploit chaining → SARIF reporting. Designed to reduce Mean Time to Adapt (MTTA) from AI-discovered weakness to validated fix.

Installation

Prerequisites:

  • Python ≥ 3.10
  • Claude Code session (claude login) OR API key (Anthropic/OpenAI)

Install into virtual environment (recommended):

python3 -m venv .venv
source .venv/bin/activate  # On Windows: .\.venv\Scripts\Activate.ps1
pip install .

Or install globally with pipx:

pipx install .

This installs the vvaharness CLI command.

Configuration

Initial setup:

# Copy environment template
cp .env.example .env

# Edit .env with your credentials
# For Claude Code (default): set CLAUDE_CODE_OAUTH_TOKEN or run `claude login`
# For Anthropic SDK: set ANTHROPIC_SDK_API_KEY
# For OpenAI: set OPENAI_API_KEY and OPENAI_BASE_URL

Backend selection (via profiles):

The default profile uses via: cli (Claude Code) for all stages. To customize:

# Copy a profile template
cp vvaharness/config/profiles/full.yaml ./config.yaml

# Edit config.yaml to select backends per stage

Profile structure:

roles:
  surface_explorer:
    via: cli              # Options: cli, sdk, openai
    model: claude-sonnet-4-6
    computer_use: false
    
  threat_modeler:
    via: sdk
    model: claude-opus-4
    temperature: 1.0
    
  vulnerability_researcher:
    via: openai
    model: gpt-4
    base_url: ${OPENAI_BASE_URL}

Environment variables:

# Claude Code (cli backend)
CLAUDE_CODE_OAUTH_TOKEN=your_token_here
CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1  # For enterprise gateways

# Anthropic SDK (sdk backend)
ANTHROPIC_SDK_API_KEY=sk-ant-xxxxx
ANTHROPIC_SDK_BASE_URL=https://your-gateway.com  # Optional
ANTHROPIC_SDK_CA_CERT=/path/to/ca.pem            # For mTLS
ANTHROPIC_SDK_CLIENT_CERT=/path/to/client.pem    # For mTLS

# OpenAI (openai backend)
OPENAI_API_KEY=sk-xxxxx
OPENAI_BASE_URL=https://api.openai.com/v1        # Or compatible endpoint

# Enterprise gateway
NODE_EXTRA_CA_CERTS=/path/to/ca-bundle.pem

Key Commands

Setup and Validation

# Verify configuration and credentials
vvaharness doctor

# Check what's needed for your setup
vvaharness setup

# Install agent instructions (Claude/Copilot/Gemini)
vvaharness setup --install-agents

Cost Estimation

# Estimate scan cost without spending
vvaharness estimate --repo /path/to/target

# Estimate for remote repository
vvaharness estimate --repo https://github.com/org/repo --application-id 12345

Single Repository Scan

# Basic scan
vvaharness scan --repo /path/to/target --application-id 12345

# Scan with custom config
vvaharness scan \
  --repo /path/to/target \
  --application-id 12345 \
  --config ./custom-config.yaml

# Scan specific module
vvaharness scan \
  --repo /path/to/target \
  --application-id 12345 \
  --module backend

# Dry run (no LLM calls)
vvaharness scan --repo /path/to/target --dry-run

Batch Scanning

# Scan multiple repos from CSV
vvaharness scan \
  --repo-file repos.csv \
  --workspace ./scans \
  --group-by-app \
  --keep-clones

# repos.csv format:
# repo_url,application_id,module_name
# https://github.com/org/app1,101,frontend
# https://github.com/org/app2,102,backend

Output Locations

Scan outputs are written to <target>/security-scan/:

<target>/
└── security-scan/
    ├── <module>_<timestamp>_report.md        # Human-readable findings
    ├── <module>_<timestamp>_report.sarif     # SARIF 2.1.0 format
    ├── <module>_<timestamp>_errors.jsonl     # Non-fatal errors
    └── run_manifest.json                     # Metadata (version, models, git SHA)

Pipeline Stages

VVAH runs a 9-stage pipeline across three phases:

Discovery & Modeling (S1-S3):

  • S1: Attack surface mapping (code, CMDB, CVE, controls)
  • S2: Threat modeling (STRIDE, OWASP, trust boundaries)
  • S3: Vulnerability research strategy (taint, API boundaries)

Deep Dive & Verification (S4-S6):

  • S4: Multi-lens research (Language, Crypto, Logic, Access Control, IaC)
  • S5: Policy gates
  • S6: Adversarial verification (exploit chains, trust boundaries)

Synthesis & Reporting (S7-S9):

  • S7: Deduplication
  • S8: Exploit chain construction (CWE, attack paths)
  • S9: SARIF emission

Configuration Examples

Budget Controls

step4:
  max_budget_usd: 5.0          # Per-stage spending cap
  max_budget_usd_per_finding: 0.50  # Per-finding cap

step6:
  max_budget_usd: 3.0
  verification_passes: 3       # Majority-vote rounds (sdk/openai only)

Language-Specific Research

lenses:
  - name: language_specific
    languages: [python, java, javascript]
    focus_areas:
      - injection vulnerabilities
      - unsafe deserialization
      - path traversal
      
  - name: crypto_specific
    focus_areas:
      - weak algorithms
      - hardcoded keys
      - IV reuse
      
  - name: access_control
    focus_areas:
      - broken authorization
      - privilege escalation
      - IDOR

Model Selection Strategy

# High-reasoning stages use Opus
roles:
  threat_modeler:
    via: sdk
    model: claude-opus-4
    
  adversarial_reviewer:
    via: sdk
    model: claude-opus-4

# Volume stages use Sonnet    
  vulnerability_researcher:
    via: cli
    model: claude-sonnet-4-6
    
  exploit_strategist:
    via: sdk
    model: claude-sonnet-4-6

Common Patterns

Enterprise Gateway Setup

# .env configuration for enterprise
ANTHROPIC_SDK_BASE_URL=https://ai-gateway.corp.com
ANTHROPIC_SDK_CA_CERT=/etc/ssl/certs/corp-ca.pem
ANTHROPIC_SDK_CLIENT_CERT=/etc/ssl/certs/client.pem
ANTHROPIC_SDK_CLIENT_KEY=/etc/ssl/private/client-key.pem
NODE_EXTRA_CA_CERTS=/etc/ssl/certs/ca-bundle.pem
CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1

CI/CD Integration

# .github/workflows/security-scan.yml
name: VVAH Security Scan

on:
  pull_request:
    branches: [main]
  schedule:
    - cron: '0 2 * * 1'  # Weekly Monday 2am

jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.10'
          
      - name: Install VVAH
        run: |
          python -m pip install --upgrade pip
          pip install .
          
      - name: Run scan
        env:
          ANTHROPIC_SDK_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
        run: |
          vvaharness scan \
            --repo . \
            --application-id ${{ github.repository_id }} \
            --config .vvah-config.yaml
            
      - name: Upload SARIF
        uses: github/codeql-action/upload-sarif@v2
        with:
          sarif_file: security-scan/*_report.sarif

Programmatic Integration

import subprocess
import json
from pathlib import Path

def run_vvah_scan(repo_path: str, app_id: str) -> dict:
    """Run VVAH scan and return manifest."""
    result = subprocess.run(
        [
            "vvaharness", "scan",
            "--repo", repo_path,
            "--application-id", app_id,
            "--config", "config.yaml"
        ],
        capture_output=True,
        text=True,
        check=True
    )
    
    # Load manifest
    manifest_path = Path(repo_path) / "security-scan" / "run_manifest.json"
    with open(manifest_path) as f:
        return json.load(f)

def estimate_scan_cost(repo_path: str) -> float:
    """Get cost estimate before running."""
    result = subprocess.run(
        ["vvaharness", "estimate", "--repo", repo_path],
        capture_output=True,
        text=True,
        check=True
    )
    # Parse estimate from stdout
    # Format: "Estimated cost: $X.XX"
    for line in result.stdout.splitlines():
        if "Estimated cost:" in line:
            return float(line.split("$")[1])
    return 0.0

Multi-Model Strategy

# config.yaml - Mix backends for cost/quality tradeoff
roles:
  # Fast exploration with CLI (no API cost if using Claude Code)
  surface_explorer:
    via: cli
    model: claude-sonnet-4-6
    
  # Deep reasoning with SDK Opus
  threat_modeler:
    via: sdk
    model: claude-opus-4
    temperature: 1.0
    
  # Bulk research with OpenAI (if cheaper in your region)
  vulnerability_researcher:
    via: openai
    model: gpt-4-turbo
    
  # Final verification with SDK Opus
  adversarial_reviewer:
    via: sdk
    model: claude-opus-4

Troubleshooting

Authentication Issues

# Check credential status
vvaharness doctor

# For Claude Code
claude login
# Or
claude setup-token  # Get token for CLAUDE_CODE_OAUTH_TOKEN

# For SDK
echo $ANTHROPIC_SDK_API_KEY  # Should start with sk-ant-

# For OpenAI
echo $OPENAI_API_KEY  # Should start with sk-

Budget Exceeded

# Review per-stage budgets in config
cat config.yaml | grep max_budget_usd

# Reduce budgets
step4:
  max_budget_usd: 2.0          # Down from 5.0
  max_budget_usd_per_finding: 0.25  # Down from 0.50

Gateway/Proxy Issues

# Test connectivity
curl -H "x-api-key: $ANTHROPIC_SDK_API_KEY" \
  $ANTHROPIC_SDK_BASE_URL/v1/messages \
  -d '{"model":"claude-sonnet-4","max_tokens":10,"messages":[{"role":"user","content":"hi"}]}'

# Check CA cert
openssl verify -CAfile $NODE_EXTRA_CA_CERTS $ANTHROPIC_SDK_CA_CERT

# Disable betas for enterprise gateways
export CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1

No Findings Generated

# Check errors log
cat <repo>/security-scan/*_errors.jsonl

# Increase verbosity (if supported in future version)
vvaharness scan --repo . --application-id 123 --verbose

# Verify lenses are enabled
grep -A5 "lenses:" config.yaml

SARIF Validation

# Validate SARIF output
npm install -g @microsoft/sarif-multitool
sarif-multitool validate security-scan/*_report.sarif

# View in GitHub Code Scanning format
gh api repos/:owner/:repo/code-scanning/sarifs \
  -F sarif=@security-scan/*_report.sarif

Performance Optimization

# Limit to specific file patterns (custom config)
discovery:
  include_patterns:
    - "**/*.py"
    - "**/*.js"
  exclude_patterns:
    - "**/node_modules/**"
    - "**/venv/**"
    - "**/*.test.js"
    
# Reduce verification passes
step6:
  verification_passes: 1  # Single-pass (faster, less FP filtering)

Best Practices

  1. Always estimate first: Run vvaharness estimate before full scans
  2. Use profiles: Copy and customize profiles/full.yaml rather than editing defaults
  3. Stage budgets: Set max_budget_usd per stage to prevent runaway costs
  4. Batch scans: Use --repo-file + --group-by-app for multi-repo workflows
  5. Human review required: VVAH findings are triage candidates, not confirmed CVEs
  6. Keep manifests: run_manifest.json tracks model versions for reproducibility
  7. Authorized use only: Scan only code you own or have permission to test

Output Format

Markdown Report Structure:

# Security Scan Report

## Executive Summary
- Total findings: X
- Critical: X | High: X | Medium: X | Low: X

## Findings

### F-001: SQL Injection in User Authentication
**Severity:** Critical
**CWE:** CWE-89
**Location:** src/auth/login.py:45-52

**Description:** User-controlled input flows to SQL query...

**Exploit Path:**
1. Attacker provides `' OR '1'='1` as username
2. Query becomes: `SELECT * FROM users WHERE username='' OR '1'='1'`
3. Authentication bypassed

**Remediation:**
- Use parameterized queries
- Apply input validation

## Dropped Findings
[Findings rejected during adversarial review]

SARIF 2.1.0 Structure:

{
  "version": "2.1.0",
  "runs": [{
    "tool": {
      "driver": {
        "name": "VVAH",
        "version": "1.0.0",
        "rules": [...]
      }
    },
    "results": [{
      "ruleId": "CWE-89",
      "level": "error",
      "message": {"text": "SQL Injection in User Authentication"},
      "locations": [{
        "physicalLocation": {
          "artifactLocation": {"uri": "src/auth/login.py"},
          "region": {"startLine": 45, "endLine": 52}
        }
      }]
    }]
  }]
}