re-ai-attack

AI Model Security Assessment and Forensics (Behavioral Layer)

CORE RULE: The attack surface is determined by observability, not by the attack name

First ask what the interface/asset provides, then choose the attack surface—the three are three different worlds:

Observability	World
logits available(API returns probabilities/logits)	Distillation (soft label), fingerprinting, and membership inference are all available
labels only(API returns only labels)	Downgrade: decision boundary sampling, black-box transfer, shadow model
weights available(model file obtained)	White box: gradient methods (FGSM/PGD), direct comparison, threshold membership inference

The same attack name has completely different feasibility and implementation under different observability—first confirm the API output fields/model file according to the "Input Asset Inventory", then choose the path.

Task classifier (intent → path)

Identify the user's purpose; if matched, follow the corresponding branch; if not matched, fall through to the "When to Use" judgment below:

User purpose (intent)	Path
Suspect the model was copied / the model was stolen / suspected copy comparison	→ fingerprint(model fingerprinting/watermark detection, step 2)
Determine training data leakage / whether a sample is in the training set	→ membership(membership inference, step 3)
Test whether the API is easy to copy / interface extractability	→ extraction(model extraction, step 1)
Test robustness / adversarial example effectiveness / defense verification	→ adversarial(adversarial example assessment, step 4)

Input Asset Inventory (INPUT INVENTORY)

After a path is matched, first confirm the existing assets—different asset combinations determine the viable paths; if assets are missing, first ask the user for them or explain the limitations:

Asset	Affected path
API endpoint（+ query budget）	extraction(step 1) required—without an endpoint, extraction is impossible
model file (local model/weights)	fingerprint comparison and membership inference (steps 2/3) are available; otherwise only the API side can be used
training dataset (suspected training set samples)	membership(step 3) required—without samples, membership cannot be inferred
suspected copy (suspected copy)	fingerprint(step 2) required—without a copy, there is nothing to compare
baseline model (victim/reference model)	required for fingerprint comparison—without a baseline, only a copy cannot establish a fingerprint
watermark hypothesis (watermark hypothesis)	fingerprint watermark verification (step 2)—with a specific hypothesis (trigger set/backdoor), targeted verification is possible

Rule: After an intent is matched, check the assets required for that path; if missing → first ask the user (whether they have them/can provide them); if the user cannot provide them, explain that the path is not executable and switch to an available path.

When to Use / When Not to Use

• Use: model leakage/theft forensics, API model attack assessment (extraction/fingerprinting), training data leakage determination, model robustness assessment (adversarial examples)
• Use: watermark/fingerprint verification (suspect copy compared against victim model)
• Use: model API consistency audit (same input across multiple queries/endpoints, to find copies/proxies)
• Use: robustness baseline assessment (run clean and perturbed accuracy once before launch/before defense validation)
• Do not use: model file format parsing and weight extraction (see [[re-ai-model]]); model training/fine-tuning (not reverse engineering); deployment hardening (defense side is out of scope for this skill)
• Do not use: unauthorized targets (red line: only authorized assessment scenarios; authorization boundary see [[gotchas]])
• RE_AUTH prerequisite (guard linkage): this skill guard.require_authorization=true—confirm authorization context before execution (triage step 0 RE_AUTH): owned / ctf / research can execute; for unknown, first ask about target ownership (ownership / test authorization / research environment); if unspecified, only perform static assessment and do not proceed to dynamic steps
• Do not use: prompt injection/prompt leakage assessment (LLM-side attack surface) and injection/privilege escalation testing of the API service itself (application-side vulnerabilities are out of scope for this skill)

Tool preparation

python3 (core runtime)

• For installation on each platform see [[re-python]] tool preparation

Model libraries (torch / tensorflow, choose according to target format)

• Multi-platform: pip install torch / pip install tensorflow(CPU version optional depending on hardware; choose one according to the target framework; the distillation script only depends on its tensor operations)
• Verify: python3 -c "import torch"

Attack assessment library (ART, optional)

• Multi-platform: pip install adversarial-robustness-toolbox(adversarial robustness toolbox, covering extraction/fingerprinting/adversarial example attack modules)
• Verify: python3 -c "import art"

Query interface client (API targets)

• Multi-platform: pip install requests; Verify: python3 -c "import requests"
• Visualization (distribution comparison plots, optional):pip install matplotlib

Operation steps

Execute in order; only in authorized assessment scenarios (red line: do not target unauthorized targets). Methodological basis: model extraction (API query distillation), membership inference (shadow model), and adversarial examples (FGSM/PGD) are public research frameworks; implementation details are adjusted according to the target model domain (see [[decision-tree]] branches).

1. Model extraction attack(Overview: reconstruct an approximate model through API queries—distillation):

   • Checklist：
     1. Confirm authorization and purpose (assessment/forensics declaration record)
     2. Interface probing: read API documentation/test queries → confirm quotas and output fields (logits/probabilities or labels only—observability determines the strategy, see CORE RULE)
     3. Query sampling (input distribution coverage: representative inputs + boundary perturbations + balanced label distribution)
     4. Distillation training (use API outputs as teacher labels)
     5. Assess approximation degree (output comparison on the same input set: agreement rate + label distribution comparison, not just accuracy)
   • Implementation experience (soft label first, boundary perturbation, assessment consistency, distillation baseline, query logs, etc.) see [[implementation-notes]]
   • When labels only: degrade to decision boundary sampling (output distribution reconstruction is limited, see [[gotchas]] interface pitfalls); within the query budget, first estimate on a small batch (pitfall 1)

2. Model fingerprint/watermark detection（Overview：behavioral levelwatermark—what the model exhibits externally: trigger-triggered responses / query responses / black-box fingerprints; file-level watermarks (weight pattern/metadata/tensor hash) go to [[re-ai-model]]):

   • Checklist：
     1. Embedding detection: input perturbation (specific noise/trigger) → output feature comparison (watermark-triggering behavior)
     2. Fingerprint extraction: model behavior signature (output vectors for a representative input set) → compared with suspected copies
     3. Theft forensics: fingerprint distance between the suspect model and the victim model → threshold-based similarity determination (threshold calibration see [[decision-tree]] evidence grading)
     4. Output: fingerprint vector + distance report (distance value + threshold basis + confidence)
   • Implementation experience (input set difficulty stratification, normalization, counterexample control group, removal adversarial, timeliness, watermark enumeration) see [[implementation-notes]]
   • Removal adversarial: pruning/fine-tuning/distillation weaken fingerprints—use robust fingerprints and mark "may be removed" (see pitfall 3)

3. Membership inference(Overview: determine whether a sample is in the training set):

   • Checklist：
     1. Confirm input form: target model output probability vector + sample features (observability see CORE RULE)
     2. Choose attack type: white-box (with loss) use threshold method; black-box (output only) use shadow model (same-distribution proxy model calibration)
     3. Control baseline: loss distribution of same-distribution random samples (non-training samples) as baseline
     4. Output: statistical determination (ROC/threshold curve, TPR at a given FPR, confidence interval), not a deterministic conclusion
   • Implementation experience (baseline design, privacy anonymization, statistical criteria) see [[implementation-notes]]
   • Limitation: can only give statistical determination (false positive control: threshold calibration; privacy-preserving training objectives can counter it, see [[gotchas]])

4. Adversarial example basics(Overview: robustness evaluation and defense validation, not attack deployment):

   • Checklist：
     1. Choose path: white-box (model weights loadable and loss differentiable) use gradient perturbation (FGSM/PGD); black-box use query/transfer perturbation (observability see CORE RULE)
     2. Set perturbation budget (epsilon)—evaluation criteria comparable
     3. Evaluation control: first measure clean sample baseline (accuracy before perturbation), report before/after perturbation comparison
   • Implementation experience (budget gradient, transferability boundary) see [[implementation-notes]]
   • Boundary: adversarial example transferability is poor (white-box perturbation does not guarantee effectiveness against black-box targets, see [[gotchas]])

5. Evaluation report and evidence preservation：

   • Record all three elements: method (sampling set/distillation hyperparameters/distance threshold), results (metrics), parameters (random seed/framework version)
   • Attack scripts and evaluation artifacts archived by sha256; conclusions delivered per [[re-analyze/analysis-contract]]

Cross-domain joint

• [[re-ai-model]]：Consume file-level evidence(file-level evidence → behavior-level verification one-way flow)—when model files are available, first extract structure/weights/file-level fingerprint via re-ai-model, then perform behavior consistency verification in this skill; with API only (no files), there is no file-side input, directly perform black-box evaluation
• [[re-python]]: Python toolchain basics
• [[re-feedback]]: attack case experience accumulation (after anonymization)
• [[re-analyze/analysis-contract]]: evaluation conclusion delivery format

Common pitfalls and traps

• Query budget limit: phenomenon—API rate limiting/billing interruption; cause—high query volume; countermeasure—sampling design within budget (estimate with a small batch first), record used quota
• Degrade when confidence is unavailable: Symptom — the interface only returns labels; Cause — the server trims the output; Countermeasure — sample at the decision boundary + annotate limitations, do not fabricate probabilities
• Watermark erased: Symptom — fingerprint comparison fails; Cause — the suspect party performed pruning/fine-tuning/distillation; Countermeasure — use robust fingerprints (multiple triggers) and annotate "may have been erased"; when the distance is close but does not reach the threshold, output "weak similarity" rather than "unrelated"
• Membership inference false positive: Symptom — statistical determination is wrong; Cause — threshold bias/shadow model distribution mismatch; Countermeasure — shadow model calibration + report confidence intervals
• Evaluation metric misleading: Symptom — the extracted model has "high accuracy" but a completely different output distribution; Cause — only accuracy is compared, not distribution; Countermeasure — joint use of multiple metrics: agreement rate + label distribution + calibration
• Extraction does not converge: Symptom — poor distillation effect; Cause — sampling distribution does not match the target distribution/interface output trimming; Countermeasure — resample according to the target data domain, and annotate data distribution assumptions
• Output parsing error: Symptom — logits and labels are misaligned; Cause — API response field parsing error; Countermeasure — print the raw response first to verify the fields before batching
• Results not reproducible: Symptom — reruns produce different results; Cause — random seed/sampling order not fixed; Countermeasure — fix the seed and record it
• Wrong comparison target scope: Symptom — fingerprint comparison used models for different tasks; Cause — task scope not confirmed; Countermeasure — confirm that the task/label space is consistent before comparison
• Multi-endpoint scope differences: Symptom — the same model produces different outputs on different endpoints; Cause — A/B routing/different versions; Countermeasure — lock the endpoint for before-and-after comparison, and annotate the endpoint identifier
• Batch queries trigger risk control: Symptom — queries at a normal rate are flagged by risk control; Cause — rate threshold triggered; Countermeasure — control query pacing (rate limiting/random intervals), and record risk control events
• Resource budget: Symptom — distillation OOM/long task interruption; Cause — student model too large/duration underestimated; Countermeasure — staged checkpoints (save sampling/training step by step), small batches/CPU degradation
• Authorization boundary: Symptom — attack testing is initiated against an unauthorized API; Cause — scope not confirmed; Countermeasure — confirm authorization first (use boundary red lines)
• For scenario branches and evidence grading, see [[decision-tree]]; for boundaries and counterexamples, see [[gotchas]]