Understanding Your Users
Understanding your users requires evidence of who they are, what they can do and how their use of the software is developing. Personas built from invented details cannot provide this. The analytics in this section examine capability, experience and changes in usage, providing a basis for interpreting the other metrics you collect.
User Sophistication Scoring
A score can help describe the range of capabilities a person uses. It cannot, by itself, tell you how accomplished they are. An experienced motion designer may produce excellent work with five layers and a small, familiar toolset. Another user may try every feature without understanding any of them particularly well.
Distinguish domain expertise, interface fluency and repertoire. A professional animator learning your application has considerable expertise and little interface fluency. A frequent user may have the reverse. The design support each needs will be different.
What It Measures
A composite description of recorded usage, combining feature diversity, advanced capabilities, customisation and imported asset types. “Sophistication” here names an instrumentation model, not a validated measure of professional skill.
How to Implement
The following 0–50 point model is an illustrative starting point. Choose categories relevant to the roles and tasks your software supports, then check the score against observed work and users' own accounts.
| Category | Example points | What to record |
|---|---|---|
| Feature diversity | 0–20 | Distinct feature types used meaningfully, excluding routine starter actions |
| Extended capabilities | 0–20 | Use of scripting, constraints, rendering options or custom automation where the task calls for them |
| Customisation | 0–5 | Saved layouts, shortcuts or presets |
| Asset diversity | 0–5 | Relevant types of imported or integrated material |
For reporting, divide the score into bands, such as under 10, 10–24, 25–39 and 40–50. Keep the numbers attached to their definition. Calling the lowest band “beginners” would turn an observation about usage into a claim about the person.
Record the user's role, intended task and opportunity to use each capability. A colourist's lack of scripting is only a problem if scripting would help them do their work. Assess capability through the quality and correctness of an outcome, the user's explanation and their ability to adapt it when the brief changes.1
What to Watch For
- A stable score: the user may have a sufficient repertoire, an efficient reusable workflow, or a barrier to further learning. Observe a task that would benefit from another method before choosing an explanation.
- A falling score: check for changed responsibilities, project mix, missing telemetry or increased automation, as well as loss of access or fluency.
- A rising score without better outcomes: broader use may reflect exploration, but it may also reflect increasingly elaborate workarounds.
A useful dashboard shows repertoire alongside successful task outcomes and users' accounts of control and effort. The Testing with Users chapter explains how to obtain that evidence.
User Maturity Cohorts
What It Measures
Cumulative exposure to the application. Session count is a rough proxy: a short visit and a day of production work each count as one session unless you define the measure differently.
| Example cohort | Sessions | Interpretation |
|---|---|---|
| New | Under 10 | Limited recorded exposure |
| Learning | 10–50 | More opportunities to encounter the interface |
| Regular | 50–100 | Repeated use |
| Veteran | 100+ | Extensive recorded exposure |
These boundaries are illustrative. They do not establish expertise, and the cohort called “Learning” need not contain only learners. Consider active time, task frequency and gaps in use alongside session count.
How to Implement
Track session number and report repertoire bands within each cohort, segmented by role and comparable tasks. Keep the cohort sizes visible, so a change in the population is not mistaken for a change in the same people's behaviour.
A user with 200 sessions and a low repertoire score is a candidate for investigation, rather than proof of a plateau. The Paradox of the Active User becomes relevant when you establish that they need a more effective method but remain with a familiar one.
What to Watch For
Check where users attempting comparable work stop progressing or require repeated help. Compare those observations with feature discovery and method adoption. A group completing its work successfully with a narrow repertoire may need no intervention at all.
Engagement Trend Analysis
Provides overall product health context for all other metrics. Segment engagement trends by sophistication level to detect dimension-specific problems — changes may suggest questions about Rigidity or Conceptual Burden, but task mix and project cycles can explain them too.
What It Measures
Classify each active user by their rolling activity pattern over the last four weeks:
| Trend | Definition | Signal |
|---|---|---|
| Growing | Consistent week-over-week increase | Deepening engagement |
| Stable | Active, no clear trend | Established routine |
| Declining | Consistent week-over-week decrease | At risk of churning |
| New | Appeared in the last two weeks | Too early to classify |
| Inactive | No activity in last two weeks but prior activity | May have churned |
Why It Matters
Usage can decline before a user leaves, but project cycles can produce the same pattern. Engagement trends identify changes worth investigating; they do not establish churn or its cause.
How to Implement
For each user, compare weekly session counts over a four-week rolling window (w1 = most recent, w4 = oldest):
-
Growing: w1 > w2 > w3, and w1 > w4 by a significant margin
-
Declining: w1 < w2 < w3, and w1 < w4 by a significant margin
-
Stable: Active in most weeks, no consistent directional trend
-
New: First appeared in w1 or w2
-
Inactive: Zero sessions in w1 and w2, but sessions in w3 or w4
What to Watch For
The ratio of Growing to Declining users describes the balance of changing activity. Compare it with your own baseline, project cycles and successful outcomes; greater activity is not inherently better. Investigate shifts:
-
After a new release: Did the update improve or hurt engagement?
-
Seasonally: Professional software often follows project-driven cycles.
-
By sophistication segment: Do changes differ by recorded usage band or independently established expertise?
If the Declining segment suddenly grows, investigate what has changed. Check for breaking changes in a recent update, a competitor launch, or an external industry shift.
Growth Trends
Provides essential context for interpreting all other metrics. Segment growth trends by sophistication level to identify whether growth is among beginners or power users. Divergent trends reveal where design investment is needed.
What It Measures
Week-over-week (WoW) and month-over-month (MoM) percentage changes for active users, total sessions, and average session duration.
Why It Matters
Growth trends are closer to traditional product analytics than the other metrics here, but they provide essential context for interpreting those metrics. A decline in session duration means something very different if total users are growing (new, shorter-session users diluting the average) versus shrinking (a changing mix of remaining users). For complex software, segment growth trends by sophistication level. Does growth differ by usage band, task or independently established expertise? The answer determines how you interpret the trend.
How to Implement
Calculate WoW and MoM percentage changes for three core metrics:
| Metric | This Week | Last Week | WoW Change |
|---|---|---|---|
| Unique active users | 342 | 318 | +7.5% |
| Total sessions | 1,204 | 1,150 | +4.7% |
| Median session duration | 28 min | 31 min | -9.7% |
A compact summary table with trend indicators (arrows or +/- percentages) works well as a dashboard header. Include new user counts per period and segment all metrics by sophistication level where possible.
What to Watch For
-
Growing users but shrinking duration: Check whether newcomers have shorter tasks, whether workflows have become faster, or whether people are leaving before completing their work.
-
Growing duration but shrinking users: Check changes in project demands, task completion and avoidable effort. Longer sessions do not necessarily indicate greater commitment.
-
Divergent trends by repertoire: Investigate whether different roles or project types explain the split before attributing it to lost depth or a rising entry barrier.
-
New user count trending down while total users hold steady: Fewer new users are arriving, but retention metrics will not reveal this for weeks. The new user count provides an earlier warning.