Testing with Users
An inspection can identify a likely problem. Watching someone attempt meaningful work helps establish whether it occurs, what causes it and how much it matters. Use both: Usability Harmonics gives you questions to investigate, rather than a substitute for observing the work.
Choose Participants and Tasks​
Recruit for domain experience and application fluency separately, as described in Understanding the Work. Include relevant access needs, equipment and working conditions. Report who took part so that readers can judge how far the findings apply.
Use a brief with a recognisable purpose and an assessable result. “Make this title readable over the supplied footage, then deliver an editable version for another animator” tests more than “find the shadow control”. Supply realistic assets and constraints without teaching the intended route through the interface.
Agree what counts as success before the session. A useful task record includes:
| Measure | What to record |
|---|---|
| Result quality | Did the work meet the brief? Use an agreed rubric or domain reviewer. |
| Independence | Was it completed unaided, after a hint, or after a demonstration? |
| Control | Could the participant predict, inspect and revise the result? |
| Effort | Time, avoidable repetition, recovery and requests for help. |
| Transfer | Could they apply the same idea to a different asset or constraint? |
Record assistance explicitly. Finishing after the researcher supplies the missing concept is not independent success. In open-ended creative tasks, several different results may meet the brief.
Include a Change of Plan​
After the first result, introduce a plausible revision: a different duration, an extra language, a changed aspect ratio or a replacement source asset. This exposes premature commitment and rigidity that a first-run task may miss.
Where relevant, test resuming after an interruption, recovering a mistake, handing work to another person and taking it through another application. Do not cram every condition into one session; choose those that address the present design question.
Ask About the Experience Without Leading​
Ask “What did you expect to happen?” and “What would you do next?” before explaining the design. If continuous narration interferes with skilled work, discuss a recording afterwards. Distinguish an observed failure from a participant's preference and from your own proposed remedy.
Test Learning Beyond the First Session​
A short session can reveal immediate confusion, but says little about retention or the cost of adopting a new method. When those are the question, arrange a later task using different material. Record the gap, help available and intervening practice. Check whether participants can explain and use the concept, rather than merely repeat a demonstrated sequence.
For established users, ask them to sketch how their experience has changed and explain the turning points. The UX Curve callout describes this retrospective method and its limits. Use it alongside observation and usage data.
Write Findings That Can Be Acted Upon​
For each finding, record the task, participant context, observed behaviour, consequence, likely explanation and unresolved alternatives. Include a short example or recording reference, then propose a change and a way to check it. Avoid reporting a small sample as a population percentage.
Use a finding record that keeps the evidence beside the proposed remedy:
| Field | What to record |
|---|---|
| User and task | Role, domain experience, application familiarity and intended outcome. |
| Observation | What happened, where, and with what consequence; include assistance. |
| Explanation | Evidence for the suspected cause and plausible alternatives. |
| Design decision | Proposed change and the trade-offs it introduces. |
| Verification | Task, outcome measure and a result that would challenge the explanation. |
| Confidence | Observed with users, inferred from telemetry or predicted by inspection. |
For example, “the participant changed both colour variants unintentionally, then restored the earlier version” is an observation. “The shared property was not apparent” is an explanation to check. “Add an affected-variants preview” is a proposed remedy to test, rather than a finding in itself.
Practical check: retest the difficult case and one nearby task after the change. A quicker completion is insufficient if it removes useful control, damages the output or makes recovery harder.