Overview
Validates that users can successfully complete core tasks through systematic observation and expert evaluation. Covers moderated and unmoderated testing, heuristic evaluation, accessibility checks, and issue severity scoring. Not a substitute for analytics or A/B testing -- those measure what happens, usability testing reveals why.
When to use: Testing user flows, validating designs, identifying friction points, running heuristic evaluations, ensuring users can complete core tasks, planning and executing usability test sessions.
When NOT to use: Analytics or A/B test setup, visual design critique without task-based evaluation, automated UI testing (use a testing framework), performance benchmarking.
Quick Reference
| Method | Best For | Participants | When to Use |
|---|---|---|---|
| Moderated testing | Deep insights, complex flows | 5-8 per persona | Design and prototyping stage |
| Unmoderated testing | Scale, quantitative data | 20-50+ | Pre-launch and post-launch |
| Guerrilla testing | Quick validation, early concepts | 5-10 random | Early concept stage |
| First-click testing | Navigation, information architecture | 20-50 | Any stage, especially IA redesigns |
| Heuristic evaluation | Expert review against principles | 3-5 evaluators | Before user testing, design audits |
| Cognitive walkthrough | Task flow analysis | 2-3 evaluators | Early design, new feature review |
| Accessibility audit | Inclusive design validation | 3-5 users with disabilities | Pre-launch, compliance reviews |
| Synthetic user testing | Scalable task validation with AI agents | N/A (automated) | Continuous, regression testing |
| AI-moderated sessions | Async moderated testing at scale | 10-50+ | When moderator availability is limited |
Core Metrics
| Metric | Target | What It Measures |
|---|---|---|
| Task success rate | 80% or higher for core tasks | Can users complete the task |
| Time on task | Simple under 30s, medium 1-2m, complex 3-5m | Efficiency |
| Error rate | Fewer than 2 per task | Learnability and clarity |
| Post-task satisfaction | 4.0 or higher on 5-point scale | Subjective ease |
| SUS score | 68+ (industry average), 80+ (excellent) | Overall usability |
Issue Severity
Severity equals Impact (1-3) multiplied by Frequency (1-3). Critical (8-9): fix before release. High (6-7): fix before release. Medium (4-5): next release. Low (1-3): backlog.
Common Mistakes
| Mistake | Correct Pattern |
|---|---|
| Writing task scenarios with product jargon or step-by-step instructions | Use natural language with realistic context and clear goals to test discoverability |
| Helping users or explaining the UI during test sessions | Observe silently and let users struggle; confusion reveals real issues |
| Testing only the happy path with ideal inputs | Include error states, edge cases, and recovery scenarios in test plans |
| Using fewer than 5 participants per persona | Recruit 5-8 users per persona; after 5, diminishing returns per Nielsen research |
| Collecting findings without severity scoring or follow-up actions | Score every issue by impact times frequency and assign fix priority |
| Skipping heuristic evaluation before user testing | Run expert review first to catch obvious issues cheaply before spending on recruitment |
| Ignoring accessibility in test planning | Include participants with disabilities and test with assistive technologies |
| Running only one round of testing | Test in waves: 5 users, fix issues, test 5 more until success rate meets targets |
Delegation
- Test plan and scenario creation: Use
Taskagent to draft objectives, research questions, and realistic task scenarios for each persona - Heuristic evaluation: Use
Taskagent to walk through interfaces against Nielsen's 10 heuristics and document violations with severity ratings - Metrics analysis and severity scoring: Use
Taskagent to calculate SUS scores, task success rates, and severity ratings from raw test data - Test report generation: Use
Taskagent to compile findings into structured reports with prioritized recommendations - Test infrastructure planning: Use
Planagent to select testing methods, define recruitment criteria, and schedule test waves
If the
quality-auditorskill is available, delegate full codebase quality audits and scoring to it. Otherwise, recommend:pnpm dlx skills add oakoss/agent-skills -s quality-auditor -a claude-code -y
References
- Test Planning -- objectives, research questions, recruitment, task scenario templates, screening criteria
- Conducting Tests -- think-aloud protocol, facilitation rules, post-task questions, session structure
- Testing Methods -- unmoderated, guerrilla, first-click, cognitive walkthrough, method selection by lifecycle stage
- Heuristic Evaluation -- Nielsen's 10 heuristics, evaluation process, severity rating, combining with user testing
- Metrics and Severity -- success rate, time on task, errors, satisfaction, SUS scoring, severity formula
- Reporting -- test report template, key insights format, stakeholder presentation, recommended actions
- Accessibility Testing -- inclusive recruitment, assistive technology testing, WCAG alignment, accessibility heuristics
- Remote and Tools -- remote vs in-person comparison, testing tools, test frequency, checklists