When to Use
- Define research questions on harm detection, jailbreak resistance, or policy categories
- Curate or audit safety datasets — labeling guidelines, bias checks, version control
- Train or fine-tune classifiers, rankers, or small LLM judges for moderation
- Design benchmarks and eval suites — golden sets, adversarial slices, regression harnesses
- Run ablations — architecture, threshold, data mix, ensemble vs single model
- Analyze metrics — precision/recall, calibration, false positive/negative slices
- Write research memos — methods, results, limitations, production recommendation
- Specify promotion bar for a new safeguard model version
When NOT to Use
- Deploy gateways, GPU serving, canary routing →
ml-infrastructure-engineer-safeguards - Execute structured red-team engagements on prod →
ai-redteam - Draft acceptable-use policy or risk tiers →
ai-risk-governance - Build customer-facing RAG/agents →
ai-engineer - General literature survey unrelated to safety →
ai-researcher - Token/context compression research →
research-engineer-scientist-tokens - Product A/B and business metrics →
data-scientist - PII detection benchmarks, memorization, logging minimization →
privacy-research-engineer-safeguards
Related skills
| Need | Skill |
|---|---|
| Privacy research for safeguards | privacy-research-engineer-safeguards |
| Production safeguard path and rollout | ml-infrastructure-engineer-safeguards |
| Adversarial attack campaigns | ai-redteam |
| Governance sign-off and model cards | ai-risk-governance |
| Production eval harness in app | ai-engineer |
| General research methodology | ai-researcher |
| Classical ML and statistics | data-scientist |
| Token efficiency ablations | research-engineer-scientist-tokens |
| Release gates and ops cadence | ai-lead-ops |
Core Workflows
1. Research framing (safety)
Hypotheses, harm taxonomy, success metrics.
See references/research_framing_safety.md.
2. Benchmarks and datasets
Golden sets, labeling, versioning.
See references/safety_benchmarks_datasets.md.
3. Model development
Training, fine-tuning, ensembles.
See references/classifier_model_development.md.
4. Evaluation and metrics
Slices, calibration, error analysis.
See references/evaluation_metrics_analysis.md.
5. Ablations and experiments
Controls, reproducibility.
See references/ablation_experiment_design.md.
6. Handoff to production
Promotion criteria, monitoring hooks.
See references/research_to_production_handoff.md.
Outputs
- Research brief — question, baseline, hypothesis, metrics
- Dataset card — sources, label schema, known limitations
- Benchmark spec — cases, categories, pass/fail rubric
- Results table — metrics by slice with confidence intervals where possible
- Error analysis — representative FP/FN clusters
- Promotion recommendation — go/no-go vs current production classifier
Principles
- Measure what policy cares about — category-level recall on high-severity harms
- Report failures honestly — FPs hurt UX; FNs hurt safety
- Hold out adversarial refresh — do not train on the only test set
- Reproducible — seeds, data version, model hash, eval script
- Separate research from ops — research proves lift; infra ships it