Module 7: Usability Testing  All about going out and getting random people on the

street or in a café to try your website, app, or product.

7.1: Usability Testing  It has the lowest barrier of entry of any testing
method, but it also yields the roughest and lowest
Usability testing is a research method that helps you
quality data. Because you select the people at
understand how well users interact with a product in order to
random, there’s no guarantee they align with your
improve the overall experience.  
ideal customers.
What is the purpose of usability testing?  A test like this is more useful during the pre-launch
stages of a product.
1. Gain insight into the relative value of your product.  To
understand and validate your solution’s unique value 4. PROBLEM DISCOVERY
proposition.   You test if users can complete certain tasks without
2. Understand the most important features and why they running into any issues
are important.  This understanding will help you clarify how  First, you give a set of test users a sheet with critical
much time and effort is required to further develop a particular tasks to fulfill on your website/app. Then, your
aspect of your product. Understanding the tasks, features, and facilitators will observe each test user and identify
user flows that derive the most value will guide you in any problem that arises.
creating more efficient, user-friendly designs.  With unmoderated remote tests, testers will self-
3.  Identify pain points in the user experience. Pain points report issues, and the team can analyze recorded test
are points of friction within the user experience that make a sessions instead.
user’s interaction with your product inefficient or unpleasant.  Because it’s about letting test users freely explore
how your product/website structures a task, problem
4. Explore what people are doing with your solution and discovery is a great initial test for existing
why. products/websites and working prototypes.
It’s also important to follow up your observations with  The feedback can help you quickly develop new
probing questions that reveal the hidden motivations behind versions for further testing.
user behaviors.
5. BENCHMARK
7.2: Usability Test Types  You directly compare the usability of two different
versions of your app/site. Typically, the goal is to
1. MODERATED IN-PERSON find the best way to complete a specific task.
 Usually happen in a laboratory setting, with third-
party professionals overseeing the trial in person and  6. LEARNABILITY
analyzing the results.  You aren’t looking for specific usability problems,
 Since the tests are happening right in front of the but rather testing how easy or difficult it is to learn to
facilitators, it’s easy to oversee every step of the complete tasks effectively over time.
process and uncover unforeseen issues or user  By measuring the differences in task completion
behavior times, you can visualize the learning curve with a
 This method is most suited for established companies graph.
with a large existing user base, where there is a
considerable upside to every small usability win. It 7. EYE TRACKING
can also be suitable for startups with big budgets and
 A standard approach for testing design solutions for
aggressive growth goals.
websites and applications.
 There’s a high barrier to entry since tests of this kind
 It monitors where a test user’s attention starts, and in
require a significant investment of money and
what order they focus on every element that follows.
working hours.
7.3: Check List for Usability Testing
2. UNMODERATED REMOTE
 Use a web-based service to monitor the test and 1. DETERMINE THE FEATURES TO TEST
administer survey questions. Since the software
 What insights do you want to learn from your users?
controls the test and records user actions, there is no
 Know if you’re building a new product or want
need for test facilitators.
feedback on a specific feature of an existing product.
 With the right questions and session recordings, you
 Not sure which feature to test first? Red
can gain comparable insights to the much more pricey
Routes and MOSCOW might be the answer.
usability lab tests
 Using online tools, it’s a great place to start for any
company with an app, digital product, or website. RED ROUTES: A tool used to help identify which
features of an existing or new product need to be
prioritized.
3. GUERILLA TESTING
 MOSCOW: Short for MUST HAVE/SHOULD  Note taking: Pen and paper? Miro? Google Sheets?
HAVE/COULD HAVE/ WOULD   HAVE. This tool is Recommend against Nation as it doesn’t let you do
used to analyze the ranking of your product’s features. affinity mapping
 Who will be joining the test? Will anyone be
2.  SET SUCCESS METRICS observing?
 Effectiveness – 80 % of participants finish applying for  Ensure your participant has been well briefed to be
enrollment online in a quiet room. We usually onboard participants at
 Efficiency – participants can complete their online least a day ahead.
enrollment within 5 minutes
 Satisfaction – 85 % of the participants rated the online 7.4: How To Write Effective Usability Testing Questions
enrollment experience above 6.
Usability testing is a research method that helps you
3. DETERMINE WHO TO TEST ON understand how well users interact with a product in order to
 Which users will provide the most valuable feedback? improve the overall experience. Traditionally, usability testing
 Test with a wide range of people is used to achieve the following goals:
 5 people per persona (one at a time) is all you need
1. Gain an insight into the relative value of your
4. CREATE TASK SCENARIOS product/service. In a competitive market, it’s important to
understand what makes your solution better than the other
 Give instructions to the user on what we want them to
solutions people use. One of the goals of user testing,
do (not how)
therefore, is to understand and validate your solution’s unique
 Creating a task scenario helps make the usability test
value proposition. In other words, how does your product
feel real, putting the user in the right mindset of the
solve problems better than existing solutions people
context.
commonly use?
 A context-less task such as “Buy a product” lacks
meaning and would not be likely to simulate a real- 2. Understand the most important features and why they
world situation. are important. It is important to dig deeper to identify
 Context-less: Buy some food. features of your design that are the main drivers of value. This
 Make it realistic and actionable: You’ve just finished a understanding will help you clarify how much time and effort
long day at the office which involved back-to-back is required to further develop a particular aspect of your
meetings, zoom fatigue, and a difficult conversation product. Furthermore, understanding the tasks, features, and
with the client. You’ve just turned off your laptop and user flows that derive the most value will guide you in
are hungry and want to order food for yourself. creating more efficient, user-friendly designs.
Navigate to the site and try to order something to 3.  Identify pain points in the user experience. Pain points
satisfy your craving. are points of friction within the user experience that make a
 Avoid giving clues or describing steps: Click on view user’s interaction with your product inefficient or unpleasant.
product, then click on add to cart then purchase the Pain points are usually related to:
product.  User’s misunderstanding of your messaging or
 Avoid tasks that don’t have definite end points: Explore content (for websites and mobile apps);
the credit card section.  A misalignment between what the user expects and
what happens, or;
5. WRITE A TEST SCRIPT  Experiences that keep the user from completing their
 A script you follow when conducting your tests. It goal in using your product. 
helps ensure the tests are standardized- you need to
make sure your participants answer the same questions 4. Explore what people are doing with your solution and
and do the same tasks. why. Usability testing will help you understand users’ actual
 It will help align your team and describe the tasks you behaviors. It’s true that people often say one thing and do
have designed for the usability test. another. Therefore, it’s up to the user researcher or designer to
become an unbiased observer of the actual interactions that
 6. PREPARE YOUR ENVIRONMENT take place between a user and a product. It’s also important to
follow up your observations with probing questions that reveal
 Identify what tools you’ll be using for the usability test:
the hidden motivations behind user behaviors.
o How will you and the participant see each other?
o How will you give access to the prototype?
o Where/how will you take notes? USABILITY TESTING QUESTIONS
 Video: Any introductory video conferencing app can
work through a note that not all have record Getting the information, you need to design a product begins
functionality on the free version so you may have to with asking the right questions. Usability testing sessions can
screen record separately. Ask for consent first. be divided into three main phases of questioning: pre-testing,
 Prototype: Figma? Marvel? Adobe XD? Something testing, and post-testing.
else?
PRE-TESTING QUESTIONS
In this phase, the goal is to collect background information on 1. Observing User Behaviors. Defining specific tasks for
the users engaging in the testing session. Pre-testing questions users to achieve while using the product is critical to
should enable you to: testing. In addition, it’s important to observe the user’s
 Uncover user demographics; actual interactions and how these interactions relate to a
 Understand the existing approach to solving the pleasant or unpleasant user experience. Example
problem your product aims to solve, and; questions include:
 Explore the user’s prior knowledge with respect to your  What are you thinking as you view [specific page or
product and the domain in which your solution exists. feature]?
 If you were looking for [information], where would
 1. USER DEMOGRAPHICS: Understanding the user’s you expect to find it?
background and demographics is important to narrow down  How was the experience of using the product to
your target market and identify factors that may affect the use complete this task?
of your product. Example questions include:         [Researcher:] What are your thoughts on the
 To get started, can you tell me briefly about yourself? language used?
         [Researcher:] What is your current occupation?         [Researcher:] How easy or difficult was it to
         [Researcher:] What is your household navigate?
composition?         [Researcher:] What are your thoughts on the
 On a scale of 1 to 5 (1=not at all confident, 5=very design and layout?
confident), how would you rate your level of  [Internal] How long did it take the user to complete
confidence in using your mobile phone for [one or this task?
several tech savviness-related activities]?
2. User Motivations for Behaviors
2. EXISTING APPROACHES: It’s also important to ask After observing user behaviors, it’s important to
questions that give you a clear indication of what the ideal understand the “why” behind user interactions. This
experience would be like. This type of questioning feedback provides greater insight into what
provides insight into the relative benefits of your product. improvements should be made in order to align your
Example questions include: product with user expectations. Example questions
o How often do you [description of tasks related to include:
your product]? o Why did you navigate to [Page A] instead of [Page
         [Researcher:] When was the last time you B]?
engaged in this task? o What motivated you to click [a specific interaction]?
         [Researcher:] What tools do you use, if any, to
help with this task? POST-TESTING QUESTIONS
         [Researcher:] Please describe your experience
with this tool. In this phase, the goal is to gather feedback on the overall user
experience. Post-testing questions should enable you to
3. PRIOR KNOWLEDGE AS A BASELINE: Lastly, it’s comprehensively evaluate the performance of each task,
helpful to evaluate users’ awareness of the problem your connected together to form a seamless and cohesive
product is solving. Prior understanding can play a experience.
significant role in product usage. An example question
includes: 1. End-to-End User Experience. An evaluation of the end-
o Which of the following indicates how much you to-end user experience provides insight into any changes
know about [problem your product is solving]? that need to be made to the workflow within your product
          [Option 1:] I don’t know anything about that. experience, such as a feature within an application that
          [Option 2:] I know a little, but I could learn feels disconnected from another. Example questions
more include:
          [Option 3:] I am an expert.  How would you describe your overall experience
with the product?
 What did you like the most about using this
TESTING QUESTIONS
product?
In this phase, the goal is to allow the user to interact with your  What did you like the least?
product, while uncovering pain points within the experience.  What, if anything, surprised you about the
Testing questions should enable you to: experience?
 Identify why users engage or disengage with the  What, if anything, caused you frustration?
product;
 Understand how can you increase the product’s ease 2. Overall User Impression. Post-testing questions should
of use, and; also provide insight into the user’s impression of the
 Explore how to increase users’ overall satisfaction product. Example questions include:
with your product. o On a scale from 1 to 5 (1=not at all likely, 5=very
likely), how likely are you to recommend this
product to a friend?
o How frequently would you use this product?
      [Option 1:] Never
      [Option 2:] Very Rarely (once per month)
      [Option 3:] Rarely (2-3 times/month)
      [Option 4:] Occasionally (2-3 times/week)
      [Option 5:] Frequently (1-2 times/day)
      [Option 6:] Very Frequently (3+ times/day)