Skill for configuring promptfoo prompt evaluation and testing.
Overview
promptfoo enables:
- Prompt regression testing
- Multi-provider comparison
- Assertion-based validation
- HTML report generation
Use When
This skill is automatically invoked when:
- Setting up prompt testing
- Comparing LLM providers
- Creating evaluation pipelines
- Defining test assertions
Available Scripts
| Script | Description |
|---|---|
scripts/init-promptfoo.sh |
Initialize promptfoo project |
scripts/run-eval.sh |
Run evaluation with output |
Available Templates
| Template | Description |
|---|---|
templates/promptfooconfig.yaml |
Basic configuration |
templates/multi-provider.yaml |
Provider comparison |
templates/assertions.yaml |
Assertion examples |
Best Practices
- Use YAML anchors for repeated config
- Store prompts in separate files
- Use variables for dynamic tests
- Set appropriate thresholds