Planning
- 5 participants per round (catches ~85% of issues)
- 4-6 tasks per session (1 hour max)
- Mix task types: specific ('Find X') and open ('Explore the settings')
Task Writing
- Scenario-based: 'Your friend recommended a restaurant. Find it and make a reservation.'
- No clues in the task: don't use the same words as the UI labels
- One clear success criteria per task
During the Session
- 'Think aloud' protocol: ask users to verbalize their thinking
- Don't help, guide, or react to mistakes
- Take notes on: what they did, what they said, what they seemed to feel
- Record with permission (screen + face)
Analysis
- Severity rating per issue: Critical (blocks task), Major (significant difficulty), Minor (minor friction)
- Rainbow spreadsheet: tasks × participants, color-coded by success/failure
- Top 5 findings with evidence and recommendations
Reporting
- Executive summary: 3-5 key findings
- Per-finding: observation, impact, recommendation, evidence (video clip)
- Prioritized fix list