Skip to main content

Cognitive Dimensions and Usability

Developing a Methodology to Analyze Cognitive Dimension-Based Feedback

Introduction​

This paper investigates a methodology for analysing usability feedback using the Cognitive Dimensions Framework (CDF), which captures the cognitive factors affecting system usability. The authors conduct a systematic review of 70 studies that employed CDF questionnaires and propose a set of guidelines for analysing qualitative feedback, particularly for security APIs (Application Programming Interfaces).

Target​

This research is primarily relevant for software developers and usability researchers analysing qualitative feedback in usability evaluations, especially in API usability.

Key Insights​

  • Gap in CDF Analysis Methodology: Although CDF-based usability questionnaires are common, there’s a lack of structured guidelines for analysing the feedback they yield.
  • Five Data Analysis Methods Identified:
    • Thematic Analysis: Categorises responses into thematic groups.
    • Positive/Negative Classification: Differentiates positive, negative, and neutral feedback.
    • Grounded Theory: Develops themes from the data without predefined categories.
    • Specific Criticisms: Focuses on pinpointing specific usability problems noted by respondents.
    • Issue-Focused Analysis: Identifies comments directly pointing out usability issues.
  • Proposed Guidelines for Developers:
    • The authors offer a framework that simplifies the process of extracting usability issues from CDF feedback, aiming to help developers unfamiliar with complex qualitative analysis methods.

Supporting Data​

  • Empirical Study: The authors conducted an experiment with software developers using the proposed guidelines to analyse API usability feedback, finding that developers using the guidelines identified more usability issues with higher validity than those without.

Other Insights​

  • Specificity in Feedback: A notable finding was that developers often missed critical context or specifics in usability issues, emphasising the importance of detailed guidelines.
  • Non-Native English Speakers: Non-native English-speaking developers showed slightly lower accuracy, highlighting potential language barriers in interpreting usability feedback.

Practical Applications​

  • Guidelines for Usability Analysis: Developers analysing CDF-based feedback can use these structured guidelines to identify usability issues more effectively.
  • Enhanced Examples for Contextual Understanding: Include examples that specify common usability challenges in APIs, improving clarity for developers.

Reference​

Wijayarathna, C., Grobler, M., & Arachchilage, N. A. G. 2019. Cognitive Dimensions and Usability: Developing a Methodology to Analyze Cognitive Dimension-Based Feedback. Behaviour and Information Technology, 38(12), 1251–1263.