Understanding Your Users
You can't improve what you don't understand. And "understanding your users" doesn't mean conjuring personas with stock-photo faces and fictional hobbies — it means looking at real data that reveals who your users are, what they can do, and where they're headed. These three analytics types segment your user base by capability, experience, and trajectory, giving you a foundation for interpreting every other metric you'll collect.
User Sophistication Scoring
Helps detect: Conceptual Burden (are users learning the software's concepts?), Visibility (are features being discovered and used?). A low average sophistication score across your user base suggests either high conceptual burden or poor feature visibility.
What It Measures
A composite score reflecting how deeply a user engages with the software's capabilities across multiple dimensions. This isn't a simple feature count; sophistication scoring weights different types of engagement to distinguish genuine depth from surface-level clicking around.
Why It Matters
In complex software, "active user" is a nearly useless metric without knowing how someone uses the tool. A user who only drags basic shapes around is a fundamentally different creature from one writing scripts, building custom constraints, and pushing advanced rendering — even if both log identical session counts and durations. Sophistication scoring reveals who your power users actually are, and how many users are operating well below their potential.
How to Implement
Build a multi-dimensional score (e.g., 0–50 points) from several categories:
| Category | Points | How to Score |
|---|---|---|
| Feature Diversity | 0–20 | Count unique feature types used across sessions. Subtract a baseline for "starter" features everyone uses. Cap at a ceiling. |
| Advanced Feature Flags | 0–20 (4 pts each) | Boolean checks for specific advanced capabilities: scripting, advanced rendering, plugin/extension usage, complex data operations, custom automation. |
| Customisation Signals | 0–5 | Custom keyboard shortcuts configured, workspace layouts saved, presets created. |
| Asset Diversity | 0–5 | Range of external asset types imported or integrated. |
Segment users by score:
| Segment | Score | Typical Profile |
|---|---|---|
| Beginner | Under 10 | Uses basic features only, default workspace |
| Intermediate | 10–24 | Explores beyond basics, some customisation |
| Advanced | 25–39 | Uses sophisticated features, customised workflow |
| Power User | 40+ | Full feature utilisation, scripting, extensions |
What to Watch For
Track the distribution across segments over time. A healthy product shows users gradually migrating upward. Warning signs:
-
Beginner segment grows but Advanced doesn't — users aren't progressing through the learning curve.
-
Power Users churn — something is broken at the expert end (performance issues, missing features, breaking changes).
-
Large Intermediate plateau — users get past the basics but hit a ceiling before reaching advanced capability.
This connects to research by Scarr et al. (2011) showing that expertise exists on a continuum, and that "strategic knowledge" — knowing efficient combinations of features — is a distinct layer beyond knowing individual features.1
User Maturity Cohorts
Helps detect: Conceptual Burden (does experience translate to capability?). Cross-referencing maturity with sophistication reveals the Paradox of the Active User — veteran users stuck at beginner sophistication levels indicate the conceptual design has hard ceilings.
What It Measures
Segment users by cumulative session count into maturity bands reflecting their exposure to the software.
| Cohort | Sessions | Interpretation |
|---|---|---|
| New | Under 10 | Still in first contact, learning basics |
| Learning | 10–50 | Past initial friction, building capability |
| Regular | 50–100 | Established user, developing expertise |
| Veteran | 100+ | Long-term user, deep familiarity |
Why It Matters
Session count is a rough proxy for exposure — not expertise. The real power of maturity cohorts comes from cross-referencing them with sophistication scores, which reveals whether experience is actually translating into capability.
A user with 200 sessions but a Beginner sophistication score has plateaued. This is the Paradox of the Active User (see Introduction) made visible in your data — someone who shows up every day but never progresses.
How to Implement
Track session_number per user. Report sophistication score distributions within each maturity band. The key visualisation is a maturity × sophistication matrix:
Beginner Intermediate Advanced Power User
New (under 10) ████████ ██ ░ ░
Learning (10-50) ████ ██████████ ████ ░
Regular (50-100) ██ ████ ████████ ████
Veteran (100+) ░ ██ ██████ ██████ ██
In a healthy product, the distribution shifts rightward as maturity increases. Cells in the bottom-left — Veteran + Beginner — are the Paradox of the Active User, alive and blinking in your dashboard.
What to Watch For
The Learning → Regular transition is where most complex software loses users. If the drop-off between these bands is steep, your intermediate learning curve is too harsh. Cross-reference with Feature Adoption data (covered in the next section) to pinpoint which specific features or concepts create the barrier.
Engagement Trend Analysis
Provides overall product health context for all other metrics. Segment engagement trends by sophistication level to detect dimension-specific problems — declining experts may indicate Rigidity issues, while declining beginners suggest Conceptual Burden.
What It Measures
Classify each active user by their rolling activity pattern over the last four weeks:
| Trend | Definition | Signal |
|---|---|---|
| Growing | Consistent week-over-week increase | Deepening engagement |
| Stable | Active, no clear trend | Established routine |
| Declining | Consistent week-over-week decrease | At risk of churning |
| New | Appeared in the last two weeks | Too early to classify |
| Inactive | No activity in last two weeks but prior activity | May have churned |
Why It Matters
In complex professional software, churn has a long lead time. Users don't quit overnight — they gradually disengage over weeks. Engagement trend analysis identifies at-risk users before they churn, giving the product team time to investigate and intervene rather than just count the bodies.
How to Implement
For each user, compare weekly session counts over a four-week rolling window (w1 = most recent, w4 = oldest):
-
Growing: w1 > w2 > w3, and w1 > w4 by a significant margin
-
Declining: w1 < w2 < w3, and w1 < w4 by a significant margin
-
Stable: Active in most weeks, no consistent directional trend
-
New: First appeared in w1 or w2
-
Inactive: Zero sessions in w1 and w2, but sessions in w3 or w4
What to Watch For
The ratio of Growing to Declining users is a leading indicator of product health — more sensitive and more actionable than aggregate DAU/MAU. Track this ratio over time and investigate when it shifts:
-
After a new release: Did the update improve or hurt engagement?
-
Seasonally: Professional software often follows project-driven cycles.
-
By sophistication segment: Are beginners declining while experts grow? (Or vice versa?)
If the At-Risk segment suddenly grows, something has changed. Check for breaking changes in a recent update, a competitor launch, or an external industry shift.
Growth Trends
Provides essential context for interpreting all other metrics. Segment growth trends by sophistication level to identify whether you're growing beginners or power users — divergent trends reveal where design investment is needed.
What It Measures
Week-over-week (WoW) and month-over-month (MoM) percentage changes for active users, total sessions, and average session duration — the vital signs of the product.
Why It Matters
This is closer to traditional product analytics than the other metrics here, but growth trends provide the essential context without which everything else is uninterpretable. A decline in session duration means something very different if total users are growing (new, shorter-session users diluting the average) versus shrinking (remaining users disengaging). The key insight for complex software: segment growth trends by sophistication level. Are you growing beginners, intermediates, or power users? The answer changes everything.
How to Implement
Calculate WoW and MoM percentage changes for three core metrics:
| Metric | This Week | Last Week | WoW Change |
|---|---|---|---|
| Unique active users | 342 | 318 | +7.5% |
| Total sessions | 1,204 | 1,150 | +4.7% |
| Median session duration | 28 min | 31 min | -9.7% |
A compact summary table with trend indicators (arrows or +/- percentages) works well as a dashboard header. Include new user counts per period and segment all metrics by sophistication level where possible.
What to Watch For
-
Growing users but shrinking duration: You're attracting users who don't stick. Check first-session quality and onboarding.
-
Growing duration but shrinking users: The remaining users are more committed, but you're losing reach. The product may be becoming more niche.
-
Divergent trends by sophistication: Growth in beginners but decline in power users suggests the product is losing depth. Growth in power users but decline in beginners suggests the entry barrier is rising.
-
New user count trending down while total users hold steady: Your funnel is drying up, but you won't notice in retention metrics for weeks. This is your early warning.