Skip to main content

Hard Mental Operations

Adapted from the Cognitive Dimensions of Notations framework (Green & Petre, 1996; Blackwell & Green, 2003). See About the Dimensions for framework references.

Hard Mental Operations are places where the system demands difficult cognitive processing — where users need to mentally track invisible state, perform calculations in their head, juggle multiple pieces of information simultaneously, or use external aids like pencil and paper to keep up with the interface.

In complex professional software, some cognitive difficulty is inherent to the domain — understanding 3D spatial relationships, reasoning about temporal dependencies in animation, or interpreting statistical patterns in data. This intrinsic cognitive load is the work itself and should not be eliminated. What should be eliminated is extraneous cognitive load: difficulty imposed by the interface rather than the domain. When a user reaches for a notepad because the software does not show them what they need to see, the interface has failed.

The Quantitative-to-Qualitative Gap​

Barbara Mirel identifies one of the most pervasive hard mental operations in professional software: the quantitative-to-qualitative transformation.1 Software displays numbers — values, percentages, metrics, coordinates — but users need meaning. "Is this value good or bad? Is this trend significant? How does this compare to what I saw earlier?" The gap between raw data and professional judgment is a hard mental operation that the system could bridge with visual cues, thresholds, contextual comparisons, and trend indicators, but often does not.

Invisible Processing Chains​

When the system performs calculations or transformations but does not show intermediate results, users must mentally reconstruct the processing chain to understand the output. Mirel calls this the need for "signature of algorithms"1 — the ability to see not just what the system produced, but how it got there. In node-based compositing, if a chain of 12 nodes produces an unexpected result, the user needs to be able to inspect the output at each intermediate step. In data analysis, if a series of filters and transformations produces a surprising dataset, the user needs to trace which records were included and excluded at each stage.

Deeply Nested State​

Complex software often involves deeply nested hierarchies — scene graphs, layer stacks, class hierarchies, nested compositions. Tracking parent-child relationships six levels deep, or understanding which override is actually controlling a property when multiple levels of inheritance are involved, is a hard mental operation. The system could show the full inheritance chain, highlight the active override, or provide a flattened view — but when it doesn't, users must mentally reconstruct the hierarchy themselves.

Temporal Reasoning​

In animation, video editing, and simulation, temporal reasoning is a persistent source of hard mental operations. "If I move this keyframe, how will it affect the timing of everything downstream?" "If I change the playback speed of this clip, what happens to the audio sync?" These are questions the software could answer visually — through ghost frames, timeline previews, or ripple indicators — but often leaves to the user's mental arithmetic.

Cognitive Load Theory2​

A useful framework for evaluating hard mental operations distinguishes three types of cognitive load:

  • Intrinsic load: The inherent complexity of the domain task (cannot and should not be reduced)
  • Extraneous load: Complexity imposed by the interface design (should be minimised)
  • Germane load: Effort devoted to building understanding and mental models (should be supported)

The goal is to reallocate cognitive budget from extraneous load (fighting the interface) to intrinsic and germane load (doing the work and learning from it).

Testing​

  1. Where do users reach for pencil and paper, a calculator, or a separate document to keep track of what the interface should be showing them?
  2. Are there operations that require mental tracking of invisible state? For example, remembering which filters are active, which overrides are applied, or what was excluded from a view.
  3. Could the system show intermediate results or processing chains that are currently hidden? Test whether making these visible reduces errors and speeds up decision-making.
  4. Check for temporal reasoning demands: When users modify timing, sequencing, or dependencies, does the interface show the downstream consequences, or must users calculate them mentally?
  5. Evaluate nested hierarchies: Can users easily understand which level they are operating at, and which parent/ancestor is affecting the current element?

Conclusion​

The system should do the mental work that it can do faster and more reliably than the user. Wherever users are performing calculations, tracking state, or reasoning about invisible dependencies in their heads, there is an opportunity for the interface to take on that burden — freeing cognitive resources for the domain-level thinking that only the user can do.

Check After Improvement​

  • Visibility
  • Real-Time Review
  • Secondary Notation

Footnotes​

  1. Mirel, B. (2004). Interaction Design for Complex Problem Solving: Developing Useful and Usable Software. Morgan Kaufmann. ↩ ↩2

  2. Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285. ↩