Context

Many of the frustrations of today’s software — cryptic error messages, tedious procedures, and brittle behaviour — are often due to the program taking actions that may be right given the software’s assumptions, but wrong for the user’s actual context. The only way out is to have the software know more about, and be more sensitive to, context. H. Lieberman and T. Selker

Context shapes what an action means and how the software should respond. The same command may need different behaviour depending on the selected object, the current frame, the user’s goal or the conditions in which they are working. There are two broad types of context: macro context, concerning the environment and circumstances of use, and micro context, concerning the work itself.¹

This chapter examines how software can respond appropriately to those conditions. Wayfinding covers how users understand their position, progress and available routes through the work. A contextual control may offer the right action for a selection; wayfinding support makes that action discoverable and its consequences visible.

Environmental Context

Depending on the software you're designing, there are some obvious macro contextual considerations you should make:

• Place of use (office, train, car, home, outdoors, indoors etc.).
  • Lighting, noise and interruptions can critically affect user interactions.
  • Online vs offline or both.
• Time of use (day, night, regular intervals, random intervals, time of year).
• User's personal information (non technical/ technical, young/old, personal views).
• Accessibility - situational/ temporary/ permanent disability.
• Device of use - desktop, mobile, tablet, wearable, etc.
• Culture - Different cultures may have different interpretations and expectations. It's essential to consider cultural nuances and if any relating to your ideas are found, design accordingly.

This is the type of information that should have formed part of your user research, covered in Understanding the Work — so it's worth revisiting this work to see if this provides some ideas of where you can improve your user experience depending on, for example, time of use. The most obvious example of this is of course, supporting dark mode. Another example would be things like arriving at a location triggering a reminder, or supporting usage offline if your product will ever be used on a commute (where signal is often a problem).

Working Context

Within a workflow, context includes prior actions, current selection, editing mode, time and the purpose of the task. Contextual behaviour uses this information to reduce unnecessary work, preserve the user’s intent and offer relevant actions.

Adding contextual behaviour introduces complexity. Users need to understand when behaviour changes, and developers need to manage the states involved. The aim is to make the work easier without making the software unpredictable. Users also differ in how readily they can hold a mental model of the system; visible context can help them reason about an action before taking it.

Reification can help make contextual behaviour concrete: for example, exposing an editable handle for a selected object turns an operation into something the user can act on directly.²

In Context Editing

In many kinds of apps, creative apps, spreadsheet apps and so on, in context editing means editing values where you see the result. So in a spreadsheet, this is double clicking on a cell to enter the new value into the cell, as opposed to clicking a cell once, and then entering the value in the Formula Bar. In a creative app, this means being able to change the size of a circle inside the content area, rather than having to go to a properties panel to do this.

Generally speaking the more you can do in context the better, there are some very good papers on this topic,³ and they all recommend this with one very glaring caveat. (For the fuller theory of why closing this distance matters — semantic and articulatory distance, the gulfs of execution and evaluation — see Direct Manipulation.) In context editing should never get in the way. Controls should be transient, only appear when relevant, and should be very lightweight. An argument can be made that this heavy in context panel in Adobe Illustrator is very much in the way.

Interaction style can also influence how users approach a problem. An interface that makes experimentation easy may encourage trial and error, while another arrangement may encourage users to gather information before acting. That difference in strategy does not by itself establish a difference in lasting understanding.

StudyINTERACT 1997

Balloons, Boats and Ponies: interaction shapes strategy

Holst, Churchill and Gilmore compared two interfaces for a constraint-based planning task with 22 undergraduates. One group used direct drag-and-place interaction; the other selected elements through an indirect interface. Both then attempted a related transfer task using the indirect interface. During training, the indirect group gathered more information before its first move, while the direct group was more inclined to try actions before discovering the main constraint.⁴

The interfaces encouraged different information-search and solution strategies. However, the level of task success showed no significant group effect or group-by-task interaction, and the authors concluded that this measure did not sufficiently support lasting learning effects. For DCC design, this is a reason to examine how an interaction directs attention and planning, rather than assume that making editing less direct will produce better understanding.

PDF

Preview Changes Across Time

In motion graphics, context includes time. A useful preview should show how an edit behaves across the relevant sequence, including neighbouring scenes, rather than stopping at the current frame.

StudyCHI 2024

Piet: colour across time

Piet is a research prototype for recolouring animated videos, combining colour controls with video playback. Its design followed interviews with six domain experts, who described the difficulty of judging colour changes when artwork was edited in Illustrator and animation in After Effects.⁵

The separation made it difficult to assess how a colour change affected visual continuity across scenes. Piet brought colour editing and playback together so designers could judge changes in the animated work.

PDF

Remembering and Reusing Prior Work

Software can use knowledge of prior actions to reduce repetition:

• Store form information for reuse.
• Learn an irregular spelling that the user intentionally uses.
• Offer recently used items and files.
• Remember preferences and settings that the user has deliberately chosen.
• Suggest content or order search results using previous activity.

Suggestions based on past behaviour deserve care: they can reinforce biases or make unfamiliar material harder to find. Remembering a preference should help the user, while leaving them able to choose something different.

A visible record of the work serves a different purpose: it helps users review, revisit and branch from earlier decisions. Investigation histories, bookmarks and saved workspaces are covered in Wayfinding.

Responding to User Intent

Contextual behaviour should take account of what the user’s latest action means. If someone corrects an automatic spelling correction, applying the same correction again disregards that intent. The system should recognise that the user has deliberately restored the spelling.

Other examples include:

• Dragging an image into an editing application’s content area to add it to the composition.
• Holding dragged files over a folder to open it and continue into a nested location. This behaviour, known as spring loading, removes steps from the process of placing files.

These behaviours depend on interpreting an action in its working context. The feedback that confirms what happened is covered in Wayfinding.

Respecting the Working State

The state that affects an action depends on the domain. In accountancy software, the accounting period may determine where an entry belongs. In animation software, the current frame, selection and editing mode determine what a change affects. In a word processor, the current font and style are part of the working state; pasting content can cause frustration when it unexpectedly changes them.

The software should respect the state in which the user is working, and make any consequential change clear. Orientation cues make the relevant state visible so users can judge the scope of an action.

Be Honest About State

The Three Mile Island nuclear disaster was caused by the system state not being accurately reflected in the user interface⁶:

• An operator activated a valve switch.
• A light went on.
• The operator reasonably assumed this meant a valve had opened. It had not, the light went on regardless of the valve working or not, it was effectively a light switch.
• A partial meltdown occurred. A much milder modern version of this would be a system for submitting jobs for processing on a server, here's what not to do:
• User submits a job.
• A status update immediately changes to "In Progress" in order to give some instant feedback.
• A few seconds later, the servers return an error of some sort as there was a fault with the submission, the status changes to "Error". In this situation the message should never have been changed to "In Progress", because the job never got that far. Instead "Submitting" or some other preparatory piece of copy would be better as it would communicate to the user that something has happened - but that the job is yet to start.

Offering Relevant Actions

Context-Sensitive Controls

Show controls when the working state makes them relevant. In Cavalry, selecting a gradient layer reveals controls in the viewport that let the user manipulate the gradient directly. The selection establishes the context, and the software provides the appropriate tools without requiring a separate search for them.

Previewing Consequences

Feed-forward shows what an operation would do before the user commits to it. Previewing an image filter lets the user judge its effect. When dragging a file into a text editor, a drop preview can indicate whether it will open in a new tab or a split view.

Where an action is incompatible with the current selection, disable it and make the reason available. The behaviour should reflect what is possible in the current context; Wayfinding covers how those possibilities are presented.

Naming Controls by Purpose

Context can also determine the most useful name for a control. A day-of-the-week selector might set a creation day in one place and a due day in another. Calling both ‘Day of the Week’ describes the control, but calling the second ‘Due Day’ explains its purpose. Clarity may justify different labels for otherwise similar controls; see Consistency vs Clarity.

A different representation of the same data can also reveal a useful action. The choice and coordination of those views is covered in Multiple Representations.

Deeper Context Modelling: The ICV Framework

For a more rigorous approach to understanding context in complex software, Barbara Mirel proposes the ICV model⁷ — Interactivity, Contexts, and Variability. This framework helps design teams model the conditions that shape how users actually work:

• Interactivity: Conditions across multiple contexts that dynamically open or close possible actions. What the user can do next depends on what they have done, what the data shows, and what external pressures exist.
• Contexts: Four overlapping contexts shape every user's work — the problem context, the technology/data context, the work domain context, and the subjective context (expertise, preferences, time pressure).⁷
• Variability: Every professional workflow has stable elements (professional conventions, recurring patterns) and variable elements (situational factors, emerging insights, shifting priorities). Designs must support both.⁷

The model gives you a basis for deciding which contextual behaviours are appropriate. Its application to the organisation of the workspace is discussed in Information Architecture.

Returning to Work

The conditions of use can change between sessions. A returning user may have a different goal, less time, or less familiarity with the software than before. Returning Users examines how skill decay affects re-engagement and the support people may need.

Restoring their place in an investigation is covered in Wayfinding Across Sessions, including workspace preservation, annotations and bookmarks.

Check the Contextual Behaviour

Check	Practical question or example
Design decision	Which part of the working state justifies this change in behaviour?
Failure to look for	A selection change silently changes the meaning of the same command.
Verification task	Repeat an operation with a different selection and mode; ask the user to predict the result before acting.

Footnotes

1. Lieberman, H., & Selker, T. (2000). Out of context: computer systems that adapt to, and learn from, context. IBM Systems Journal, 39(3–4), 617–632. ↩

2. Beaudouin-Lafon, M., & Mackay, W. E. (2000). Reification, polymorphism and reuse: Three principles for designing visual interfaces. In Proceedings of the Working Conference on Advanced Visual Interfaces (AVI 2000) (pp. 102–109). ACM. ↩

3. Hutchins, E. L., Hollan, J. D., & Norman, D. A. (1985). Direct manipulation interfaces. Human–Computer Interaction, 1(4), 311–338. ↩

4. Holst, S. J., Churchill, E. F., & Gilmore, D. J. (1997). Balloons, boats and ponies: Interface manipulation style and learning in a constraint-based planning task. Human–Computer Interaction: INTERACT'97, 180–187. Sections 2.3–3. Published paper. ↩

5. Shi, X., Wang, Y., Wang, Y., & Zhao, J. (2024). Piet: Facilitating Color Authoring for Motion Graphics Video. Proceedings of CHI 2024. Sections 3–6. Published paper. ↩

6. President's Commission on the Accident at Three Mile Island (J. G. Kemeny, Chairman). (1979). The Need for Change: The Legacy of TMI. Washington, DC: U.S. Government Printing Office. https://www.osti.gov/servlets/purl/5082863 ↩

7. Mirel, B. (2004). Interaction Design for Complex Problem Solving: Developing Useful and Usable Software. Morgan Kaufmann. ↩ ↩² ↩³