CS-582

Advance Human
Computer Interaction

Muhammad Adnan Kaim Khani

Department of Computer Sciences,
ILMA University, Karachi

1
Contents

1. Usability heuristics
2. Principles of Usability Testing
3. Metrics and Measures for Evaluation
Reference No.1 Topic: Usability heuristics

Usability heuristics are general principles for designing user interfaces that make systems more intuitive
and user-friendly.
These heuristics, most notably defined by Jakob Nielsen, serve as guidelines to assess a system's
usability. Here are ten key usability heuristics:
1.Visibility of System Status: The system should keep users informed about what is happening,
providing timely and clear feedback.
2.Match Between System and the Real World: The system should speak the user's language, using
familiar concepts and presenting information logically and naturally.
3.User Control and Freedom: Users should have the ability to undo and redo actions. They should feel in
control and not trapped by unintended actions.
4.Consistency and Standards: The interface should be consistent within the system and with platform
conventions, reducing the cognitive load for users.
Reference No.1 Topic: Usability heuristics

5. Error Prevention: Design the system to prevent errors from happening in the first place by providing
clear guidance or constraints.
6. Recognition Rather Than Recall: Minimize the need for users to remember information between
different parts of the interface by providing cues and visible options.
7. Flexibility and Efficiency of Use: Allow users to customize and optimize their interactions.
Experienced users should be able to perform tasks more efficiently with shortcuts.
8. Aesthetic and Minimalist Design: Only show necessary information at any given time to avoid
cluttering the user interface.
9. Help Users Recognize, Diagnose, and Recover from Errors: Error messages should be clear,
constructive, and help users resolve the issue quickly.
10. Help and Documentation: Provide easily accessible help content and documentation, especially for
more complex interactions, to assist users when needed.
Reference No.1 Topic: Usability heuristics

Principles of Usability Testing

Usability testing focuses on evaluating a product by testing it on real users.
The goal is to uncover usability issues, assess ease of use, and improve the overall user experience.
Below are the core principles of usability testing:
1. Involving Real Users: The system should be tested by real users who represent the target audience.
This ensures the testing environment is relevant to actual use cases.
2. Tasks Based on Real Scenarios: Users should perform tasks that reflect real-life interactions with the system.
These tasks should be relevant to common or critical use cases.
3. Observation of User Behavior: Usability testing requires carefully observing how users interact with the system,
capturing struggles, frustrations, or ease of use, without interfering too much.
4. Iterative Testing: Usability testing should be done in cycles, with improvements made after each round. This allows
the design to evolve based on user feedback.
Reference No.1 Topic: Usability heuristics

5. Qualitative and Quantitative Data: Both data types should be gathered. Qualitative data can offer insights into
user emotions and behaviors, while quantitative metrics (e.g., task completion time) offer measurable feedback.
6. Usability Metrics:
1. Efficiency: How fast users can complete a task once they understand how to perform it.
2. Effectiveness: The number of errors made and the ability of users to complete the tasks successfully.
3. Satisfaction: How enjoyable and stress-free the interaction is for users.
7. Early and Frequent Testing: Testing should start early in the development process and continue throughout, helping
to identify and address issues before they become too costly to fix.
8. Prioritizing User Feedback: It is important to act on the feedback from usability tests and prioritize fixes that have
the biggest impact on user experience.
9. Creating a Realistic Environment: The testing environment should closely simulate how the product will be used in
the real world to get more accurate insights.
10. Ethical Testing: Ensure participants know they are testing the product, not being tested themselves. Avoid putting
undue pressure on users and maintain confidentiality.
Reference No.1 Topic: Metrics and Measures for Evaluation

Metrics to Measure and Quantify Usability

The usability of a product is contextual and depends on the different roles of users, environments, and tasks they need
to do.
It’s important to measure effectiveness relative to these and to make sure we have quantitative and qualitative data to
help us collect information and compare against benchmarks.
Reference No.1 Topic: Metrics and Measures for Evaluation

Metrics to Measure and Quantify Usability

What is Usability?
Usability is the intersection between effectiveness, efficiency, and satisfaction in a context of use.
• Effectiveness: The degree to which a goal or objective is achieved. It measures how well a task or process
accomplishes its intended outcome, focusing on whether the desired result is met.
• Efficiency: The ability to achieve a goal using the least amount of resources, such as time, energy, or money. It
emphasizes optimizing the process to minimize waste and maximize productivity.
• Satisfaction: The feeling of contentment or fulfillment that comes from meeting expectations or needs. It often
reflects the subjective experience of users or stakeholders regarding how well their desires or requirements were
addressed.
In other words, it’s all about getting things done and how happy a user is when trying to get things done.
If they are getting their task done, it’s not taking too long, they are not making too many mistakes or errors and they
think highly of the application after they use it, then it’s going to be a usable product.
Reference No.1 Topic: Metrics and Measures for Evaluation

How to Measure Usability

We can measure usability by testing a product (or a competitor’s product).
An initial test gives us a “before state”, which is just as valuable as the “after state”.
This enables you to compare your designs or how you stack up against your competitors.
Once you have an idea of the effectiveness, efficiency, and how satisfied your customers are with the experience you
can set out to create a new design to improve such measures.
Reference No.1 Topic: Metrics and Measures for Evaluation

 Measure Effectiveness

The “success rate” (or completion rate) refers to the percentage of participants who correctly achieve each goal.
So ideally, before you undertake testing, you will have identified several scenarios to test.
The test should also be unassisted by a test moderator.
There are several other measures you can take to measure effectiveness.
In my experience a completion rate of 100% is great, but anything above 78% is acceptable!
Reference No.1 Topic: Metrics and Measures for Evaluation

How to Resolve Effectiveness Issues

If users are struggling with completing the task, either through frustration or inability to find the next step, you may
consider the following:

• Make sure that buttons look like buttons

• Make click through options obvious (e.g. clicking on a tile might seem obvious to you but to users it may not be
intuitive)
• Simplify your workflow
• Reduce the learning curve of using the system by including onboarding if necessary
Reference No.1 Topic: Metrics and Measures for Evaluation

 Measure Efficiency

The efficiency metric refers to the average time it takes to complete each task. Alongside this you can also calculate the
range and standard deviation. This is the main metric you will typically look at, but there a numerous other metrics
you can gather:
• Time taken on the first attempt
• Time to perform task compared to an expert
• Time correcting errors
Reference No.1 Topic: Metrics and Measures for Evaluation

This is not a comprehensive list and you should choose tasks that make sense to you. For example, if you’re testing a
very short flow it might not be worth calculating something like time correcting errors.

User errors are common, these may include actions, slip ups or mistakes. I normally assign a short description, severity
rating and classify each under their respective section. Based on industry benchmarks I aim for no more than 0.7 errors
per task.

Efficiency metrics can also be good when comparing types of users. For example, beginners vs. experts.
Reference No.1 Topic: Metrics and Measures for Evaluation

How to Resolve Efficiency Issues

If it’s taking users a long time to perform tasks you might consider the following:

• See if there is a mismatch between hyperlinks and the title of the page the link leads to.

• Make sure search results include a description of the link, in addition to the title of the page.

• Design your sitemap in a logical manner.

• Provide an alphabetical index which includes as many categories, content areas, departments and keywords as
possible.
Reference No.1 Topic: Metrics and Measures for Evaluation

 Measure Satisfaction

Satisfaction can be measured and calculated using the “System Usability Scale”. The standard scale has ten questions
which measure the user’s overall impression of the usability of the software. You can also add more questions to your
questionnaire, however, in my experience it’s best to use the SUS scale because it is so popular and there are therefore
industry benchmarks which you can measure against. For example, if you do the test with four people and, overall,
you get a score of less than 78%, it’s probably a good indicator that you need to keep working on redesigning the
workflow or interface to improve users’ level of satisfaction.
Reference No.1 Topic: Metrics and Measures for Evaluation

How to Resolve Satisfaction Issues

In my experience, the best way to resolve poor satisfaction ratings is to take on board the qualitative feedback and work
with the business to make changes to the product or service offering.
Reference No.1 Topic: Metrics and Measures for Evaluation

Conclusion

If you’re new to measuring usability, start with the basics. Just measure the success rate: the number of people who can
complete the task without assistance. If you want to go beyond this, then collect “time on task” data and satisfaction data
(with a survey at the end of the study).
These measures are nearly always sufficient. If you want to dig deeper make sure that any other additional metrics are
relevant and a good use of your time.
References / Resources

• Designing Interactive Systems: A Comprehensive Guide to HCI, UX and Interaction

Design, Benyon, D

• https://www.booksfree.org/designing-interactive-systems-by-david-benyon-pdf/

• https://webdesign.tutsplus.com/tutorials/3-metrics-for-quantifying-usability--cms-
29150