Most product metrics are designed to be moved, not to be believed. This skill is about the second kind — a number an outsider with no reason to trust you can be asked to accept, and the inference errors that quietly turn a measurement into a story.
Start with the instrument. Then check what you are allowed to conclude from it.
Design the instrument
- [[references/benchmark-on-an-instrument-you-did-not-build]] — an internal completion metric proves users moved through your product; an external scale proves the product did something. Includes funding the field that will adjudicate your claim.
- [[references/ship-a-subscore-only-if-it-adds-information]] — the three gates a component score must pass before it earns a place on the dashboard, and why the noisiest component is often the most valuable one.
- [[references/measure-willingness-alongside-capability]] — for anything users decline rather than fail at, nerve is the binding constraint and it moves independently of skill.
Report it honestly
- [[references/publish-the-component-where-you-lose]] — naming your worst dimension is what makes the good numbers legible as measurement rather than marketing, and what it costs you.
- [[references/clean-cohort-buys-attribution-costs-generality]] — a filtered cohort is the only way to isolate what you caused, and the same filter stops the result generalising. Also covers disclosing the stages of a funded study you were walled off from.
- [[references/audit-bias-at-intersections]] — bias can live at a combination of attributes and vanish when either is checked alone, so a one-row-per-attribute fairness table proves almost nothing.
Don't let the instrument lie
- [[references/classifier-categories-encode-their-training-population]] — a group the training data never contained gets filed under its nearest surface match, and the misfiling comes back as a finding.
- [[references/resemblance-is-not-lineage]] — contact manufactures shared surface material exactly as descent does; the discriminating evidence is in the generative machinery, not the borrowed inventory.
- [[references/sample-cases-that-could-not-have-influenced-each-other]] — agreement among connected cases is one observation counted many times, and the population predisposed to disagree is the highest-information test.
- [[references/triangulate-weak-independent-sources]] — three signals that fail in different ways beat one that fails silently; disagreement between them localises the error.
Boundary
This skill stops at the number. Turning a validated result into press coverage, or commissioning research to remove a specific buyer objection, belongs to duo-timely-data-publishing. Segmenting users by declared motive in order to decide what to build belongs to duo-motivation-segmentation. Visual craft — how a result is charted, typeset or animated — routes out to the external design-engineering skill; this pack does not own pixels.
Sibling skills
[[../duo-experimentation/SKILL]] · [[../duo-inclusive-access/SKILL]] · [[../duo-difficulty-calibration/SKILL]] · [[../duo-product/SKILL]]
Sources
Distilled from 21 extracted claims across blog.duolingo.com — subscore validation, efficacy studies, fairness auditing, research grants and the linguistics explainers used here as inference-trap case studies. Every node carries its own dated citation; slugs are checked against scripts/sources.json.