SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsmodel-card

model-card

Writing & research

View repository

Updated 10-15 16:51

About this skill

Model-Card Skill

Purpose

This skill produces the documentation an engineer-built medical-imaging model must carry: a Model Card (intended use, out-of-scope use, training data, per-subgroup performance, caveats), a Datasheet for its dataset (provenance, composition, collection, labelling, consent), and a METRIC-informed data-quality pass. It fills the templates from facts the user supplies — it never invents a number, a provenance detail, a consent status, or a licence — and ships a deterministic gate that no required section is missing or left as an unfilled [NEEDS INPUT] placeholder.

It is the reporting seam of the model-engineering lane: after /model-assessment audits the design and produces the numbers, this skill records them in a portable, auditable card that /write-paper and /check-reporting consume. It mirrors /version-dataset structurally (generate + deterministic verify).

When to use

  • A trained model needs a Model Card / Datasheet for a repo, Hugging Face card, or manuscript supplement.

When NOT to use

  • Auditing the validation design / metrics → /model-assessment.
  • Versioning the dataset bytes → /version-dataset; tabular variable docs → /generate-codebook.
  • Item-by-item reporting-guideline compliance of the manuscript → /check-reporting.
  • Building / training the model → /model-scaffold.

Workflow

Phase 1 — Collect the facts

Gather, from the user / the model's developers: task + architecture + provenance + licence; intended use and out-of-scope use; training and evaluation cohorts; the reference standard and inter-reader agreement; overall and per-subgroup performance; data collection, consent, and de-identification. Anything not supplied stays [NEEDS INPUT] — never guess.

Phase 2 — Fill the Model Card

Copy ${CLAUDE_SKILL_DIR}/references/model_card_template.md to MODEL_CARD.md and fill each section from the facts. Keep the headings. Numbers come only from /model-assessment / executed results.

Phase 3 — Fill the Datasheet

Copy ${CLAUDE_SKILL_DIR}/references/datasheet_template.md to DATASHEET.md and fill the seven question groups (Motivation, Composition, Collection, Preprocessing/Labeling, Uses, Distribution, Maintenance).

Phase 4 — METRIC data-quality pass

Walk ${CLAUDE_SKILL_DIR}/references/metric_dimensions.md (completeness, correctness, consistency, representativeness, timeliness, provenance, label provenance, fairness/coverage, leakage safety) and record each finding in the Datasheet. Anything that affects the headline metric's validity is also a /model-assessment finding — cross-check there.

Phase 5 — Verify completeness (deterministic gate)

python3 ${CLAUDE_SKILL_DIR}/scripts/check_model_card_complete.py \
  --card MODEL_CARD.md --datasheet DATASHEET.md --strict

MISSING_SECTION / EMPTY_REQUIRED_SECTION / UNFILLED_FIELD must be zero before the card ships. UNFILLED_FIELD names each field of a required section still left as [NEEDS INPUT] / [VERIFY] (e.g. License or subgroup performance), even when a sibling field is filled. An explicit N/A / None counts as an answer only as a field's whole value, and never in Intended Use, Training Data, Evaluation Data, Metrics or Quantitative Analyses. Only the template tokens as written ([NEEDS INPUT ...], [VERIFY], [VERIFY: ...], upper case) count; markdown link text such as [verify the protocol](https://...) or [Verify][ref], and anything inside an HTML comment or a fenced code block, is not a placeholder. Known limits: a field left as a hand-written TODO / TBD / <...> / XXXX is caught only when the whole section is unfilled; keep the template's bracketed [NEEDS INPUT] markers for open fields.

Phase 6 — Hand off

Carry the card into /write-paper (the Methods / supplement reference it), /check-reporting (CLAIM 2024 / TRIPOD+AI item audit of the manuscript), and /self-review.

Anti-Hallucination

  • Never invent evaluation numbers, subgroup results, or dataset provenance. Every figure comes from /model-assessment or the user's executed results; every provenance / consent / licence statement is user-confirmed. Unknown → [NEEDS INPUT], which the gate flags.
  • Never mark a section complete without user-supplied content, and never auto-fill a placeholder to pass the gate.
  • Never assert a licence or consent status the user did not confirm.
  • The gate checks presence, not truth — a complete card can still contain a wrong number; validity is /model-assessment and the human's responsibility.

Deterministic gate

scripts/check_model_card_complete.py — verifies every required Model Card / Datasheet section is present and non-empty (stdlib, network-free). Reproducible challenge: bash ${CLAUDE_SKILL_DIR}/scripts/check_model_card_complete_challenge/verify.sh.

Note on classification

Model Cards (Mitchell et al. 2019) and Datasheets (Gebru et al. 2021) are documentation standards, not clinical reporting guidelines, so they live here as references/ templates (uncounted), not in /check-reporting's counted checklist set — the same way appraisal_tools/METRICS.md is kept separate. /check-reporting still owns the manuscript-level CLAIM 2024 / TRIPOD+AI item audit.

Boundaries

model-assessment (audit design + metrics)
  └─ model-card (this skill: Model Card + Datasheet + METRIC pass, completeness-gated)
       └─ write-paper + check-reporting (manuscript) ; version-dataset (dataset bytes)