Helps choose and apply AI image / T2I evaluation benchmarks and metrics. Full benchmark details (metrics, datasets, links) are in reference.md.
Assist in selecting and applying AI image / text-to-image evaluation benchmarks and metrics. For each benchmark's metrics, datasets, and resource links, see reference.md。
When to use | When to use
- Need to evaluate text-to-image/generated image quality, image-text alignment, human realism, compositional ability, etc.
- Choose a benchmark (e.g., GenAI-Bench, PartiPrompt, ImageEval, T2I-CompBench++)
- Choose metrics (VQAScore, FID, CLIPScore, human evaluation, heatmap/interpretable feedback)
- Check dataset scale and evaluation method
Quick reference | Quick reference
| Requirement | Recommendation |
|---|---|
| General image-text alignment | GenAI-Bench + VQAScore |
| Composition/reasoning | T2I-CompBench++、HRSBench、DALL-EVAL |
| Chinese + fine-grained | ImageEval |
| Human realism/distortion | BodyMetric、Human Distortion Benchmark |
| Interpretability and improvement | RAHF (heatmap + score) |
| Instruction editing | Reason-Edit |
Single-image evaluation (unified report)
Given one image and one prompt, run multi-metric evaluation and get one HTML report with:
| Dimension | Metric | Description |
|---|---|---|
| Image-text relevance | CLIPScore | 0–1 |
| Image-text relevance | VQAScore | GenAI-Bench style, BLIP-2, optional --no-vqa |
| Image-text relevance | ImageReward | Human preference reward model, more consistent with human judgment, optional --no-imagereward |
| Generation quality | Sharpness Laplacian | No-reference clarity 0–1 |
| Aesthetics/attractiveness | CLIP aesthetics proxy | 0–1 |
| Human realism/distortion/RAHF | — | Requires a dedicated model, see reference.md |
cd image-evaluation
python3 -m venv .venv && .venv/bin/pip install -r requirements-eval.txt
.venv/bin/python evaluate_image.py --image /path/to/image.png "生图 prompt" --output-dir reports
# 可选：--no-vqa 跳过 VQAScore；--no-imagereward 跳过 ImageReward
Report output to reports/report_YYYY-MM-DD_HH-MM-SS.html(unique each time). Layout: sticky summary at top + images on the left and table on the right, reducing vertical scrolling.
References
- Full benchmarks & metrics → reference.md(overview table + descriptions of each benchmark + usage recommendations)