Cognitive
Walk throughs
Time
⏲
1hr/User
Difficulty
🕹
Moderate
Materials
📦
UI
Personas
Forms
Cards
People
🕴
2+ Reserachers
2+ Users
Overview
Cognitive walkthrough – a usability evaluation method in which one or more evaluators work through a series of tasks and ask a set of questions from the perspective of the user.
The focus of the cognitive walkthrough is on understanding the system’s learnability for new or infrequent users.
What
Cognitive walkthrough – a usability evaluation method in which one or more evaluators work through a series of tasks and ask a set of questions from the perspective of the user.
The focus of the cognitive walkthrough is on understanding the system’s learnability for new or infrequent users. The cognitive walkthrough was originally designed as a tool to evaluate walk-up-and-use systems like postal kiosks, automated teller machines (ATMs), and interactive exhibits in museums where users would have little or no training. However, the cognitive walkthrough has been employed successfully with more complex systems like CAD software and software development tools to understand the first experience of new users
- Start with a simple task and move to more complex tasks.
- Consider how many tasks you can complete in a single walkthrough session. A common theme in the research and case study literature is that only a few tasks can be examined in any cognitive walkthrough session. A recommendation is to consider evaluating 1- 4 tasks in any given session depending on complexity.
- Choose realistic tasks that include core features of the product. Core features are ones that are fundamental to the product and used across different tasks.
- Consider tasks that involve multiple core features so you can get input on transitions among the core features.
Why
Advantages
- May be done without first hand access to users.
- Unlike some usability inspection methods, takes explicit account of the user’s task.
- Provides suggestions on how to improves learnability of the system
- Can be applied during any phase of development.
- Is quick and inexpensive to apply if done in a streamlined form.
Disadvantages
- The value of the data is limited by the skills of the evaluators.
- Tends to yield a relatively superficial and narrow analysis that focuses on the words and graphics used on the screen.
- The method does not provide an estimate on the frequency or severity of identified problems.
When
Commonly used at the earliest design stages, but can be used at other stages
Steps
Step 1 Make a research plan
Materials needed: A representation of the user interface, a persona, a task list that includes all the tasks that you will use in the walkthrough and an action sequence that details the specific task flow from beginning to end, a problem reporting form, cards for listing design ideas for later use, incentives for participants
Step 2 Ready your participants
Participants:
- Facilitator: The facilitator is generally the organizer and is responsible for making sure that the walkthrough team is prepared for the session and follows the ground rules for the walkthrough.
- Evaluators: Representatives from the product team. These representatives could be usability practitioners, requirements engineers, business analysts, developers, writers, and trainers.
- Notetaker: The notetaker records the output of the cognitive walkthrough.
- Product expert: Since the cognitive walkthrough can be conducted early in the design stage (after requirements and a functional specification for example), a product expert is desired to answer questions that members of the walkthrough team may have about the systems features or feedback.
- Domain experts: A domain expert is often, but not always a product expert. For example, if you were evaluating a complex engineering tool, you might include a domain expert in addition to product experts.
Step 3 Log and Process
- Define the users of the product and conduct a context of use analysis.
- Determine what tasks and task variants are most appropriate for the walkthrough.
- Assemble a group of evaluators (you can also perform an individual cognitive walkthrough).
- Develop the ground rules for the walkthrough. Some groundrules you might consider are:
- No discussions about ways to redesign the interface during the walkthrough.
- Designers and developers will not defend their designs.
- Participants are not to engage in Twittering, checking emails, or other behaviors that would distract from the evaluation.
- The facilitator will remind everyone of the groundrules and note infractions during the walkthrough.
- Conduct the actual walkthrough
- Provide a representation of the interface to the evaluators.
- Walk through the action sequences for each task from the perspective of the “typical” users of the product. For each step in the sequence, see if you can tell a credible story based on the following questions
- Will the user try to achieve the right effect?
- Will the user notice that the correct action is available?
- Will the user associate the correct action with the effect that the user is trying to achieve?
- If the correct action is performed, will the user see that progress is being made toward the solution of the task?
- Record success stories, failure stories, design suggestions, and problems that were not the direct output of the walkthrough, assumptions about users, comments about the tasks, and other information that may be useful in design. Use a standard form for this process.
- Bring all the analysts together to develop a shared understanding of the identified strengths and weaknesses.
- Brainstorm on potential solutions to any problems identified.
Step 4 Follow-up and Learn More
- Schedule follow-up interviews with your most engaged participants
- Ask for participant feedback
- Turn your qualitative data into quantitative data with follow-up surveys, usability tests, A/B tests, etc.
Step 5 Analyze and share
- Synthesize the data
- Create a summary of findings
- Tag like-items to find patterns (i.e. “distracting” or “hard” were common terms used)
- Share insights with stakeholders
Tools
None