Lecture 3

HCI Fundamentals: Human

Sensory System & Cognitive
Processes

Dr. Narin Akrawi

User- Centered Design
• Know the user’s
• Cognitive abilities
• Memory
• Perception
• Physical abilities

• Keep users involved throughout the system-

building process

2
The Multidisciplinary of HCI
• Computer Science: Technical backbone for interactive system design.
• Human Factors Engineering (HFE): Ergonomics to ensure safety,
efficiency, and comfort.
• Cognitive Science: Insight into human thought processes for
interface design.
• Psychology: Emotional and psychological understanding of user
interactions.
• Information Science: Effective information access and processing
within interfaces.
• Design & User Experience (UX): Aesthetics and usability to create
intuitive interfaces.
• Anthropology & Sociology: Cultural and social dynamics in
technology use.
 General Principles of HCI Design

Compatibility: Align with user expectations and existing systems.

Ease of Learning: Intuitive for new users to adopt quickly.
Memorability: Simple to remember how to use after time away.
17
Predictability: Behaves in a way users anticipate.
Simplicity: Free from unnecessary complexity.
Flexibility: Adapts to various user styles and requirements.
Responsiveness: Reacts promptly to user input.
Protection: Guards against errors and data loss.
Invisible Technology: Works seamlessly without user awareness of the backend processes.
Control: Users feel they have command over the interaction.
WYSIWYG (What You See Is What You Get): The final result is directly reflected in the editing
process. .
 The Human

• Info Input/Output:
• Visual: Sight-based perception.
• Auditory: Hearing-based reception.
• Haptic: Information through touch.
• Movement: Gestures as inputs/outputs.
• Memory Systems:
• Sensory: Immediate, brief information hold.
• Short-term: Active memory for current thoughts.
• Long-term: Extensive knowledge/experience storage.
• Thinking and Processing:
• Reasoning: Logic to interpret and decide.
• Problem Solving: Tackling and overcoming tasks.
• Error: Interaction mishaps.
• Emotional Impact: Influence of emotions on cognition and interaction.
• Individuality: Design for diverse abilities, states, and age-related changes.
 The Stroop Test

The Stroop Test is a psychological assessment that illustrates:

- the conflict between automatic and controlled processing by requiring

participants to name the color of the ink used to print words, which
themselves are names of different colors.

It demonstrates how our ingrained habit of reading can interfere with our
ability to quickly process and respond to visual color cues.
 The Stroop Test

Read out loud the color of these words, NOT the word itself
The Stroop Test

What did you notice about your response time

and accuracy?

What might this tell us about how our brains

process conflicting sensory information?

What are the implications of this understanding

on the HCI design?
 The Stroop Test

• Go with the Flow: Make things work the way people expect, so
they don't have to think too hard.
• Keep it Simple: Don’t confuse people with too many choices or
clashing instructions.
• Stop Mistakes Before They Happen: Place things where
people won't click the wrong button by accident.
• Highlight What's Important: Make the most important stuff
catch the eye first.
• Stay the Course: Use familiar signs and buttons so people don't
have to learn everything new each time.
 Vision

Two stages in vision

1.Physical Reception of Stimulus: This is where it all begins — our eyes capturing light. This
step is all about the initial contact between light and our eyes, which is crucial because it
determines how well the visual information is received before it's even processed.

2.Processing and Interpretation of Stimulus: After our eyes do their job, it's up to our brains
to make sense of what we're seeing. This is the part where design plays a massive role
because how information is presented can make the difference between a user-friendly
interface and one that's confusing.

Both stages are essential for creating an effective HCI design. Good design caters to the
physical aspects of seeing, like ensuring there's enough light and contrast, and also to the
cognitive aspects, like organizing information in a way that's easy for the brain to interpret.
 The Eye - physical
reception

- Eye as biological camera: captures light, and converts to electrical signals.

- Light reflects from objects, giving color via specific wavelengths.
- Images focused upside down on the retina.
- Retina has rods for low light and cones for color vision.
- Bipolar cells transmit signals from photoreceptors to ganglion cells.
- Ganglion cells send visual information to the brain, which is sensitive to patterns and
movement.

- HCI design influenced by eye physiology:

- Use high-contrast colors for readability in various lighting.
- Employ precise animations for attention, utilizing ganglion cells’ movement detection.
 Brain Interpreting the signal

- Visual angle and acuity: Our brain uses them to measure object size and depth.
• Visual Angle: This is the angle that an object appears to occupy in your field of view, which varies with
both the actual size of the object and its distance from you. The closer an object is, or the larger it is, the
larger the visual angle.
• Visual Acuity: This refers to the sharpness or clarity of vision, which is the ability of the eye to see fine
details. The higher the visual acuity, the smaller the visual angle can be while still perceiving details
accurately.
- Overlapping cues: If one object covers another, it seems closer.
- Relative size: Smaller objects are perceived as further away.
- Perspective: Converging lines (like a road's edges) indicate depth.
These cues are used in UI design to:
-Big Elements: Important features like buttons are made large so users can easily find and interact with them.
-Clear View: Designs are crafted for maximum clarity, allowing users to notice every detail effortlessly.
-Top Layer: The most crucial information or elements are placed at the forefront, similar to the top sheet in a stack of
papers.
-Tiny Equals Less Important: Smaller items on the screen are typically less critical, signaling to users that they are
secondary.
 Interpreting the signal (cont)

Brightness: subjective reaction to levels of light: It is a subjective reaction because each

individual perceives brightness differently. What seems bright to one person may be dim to
another.
• This perception is affected by the luminance of an object, which is a measure of the amount
of light that an object reflects.
Color
• made up of hue, intensity, and saturation
• cones sensitive to color wavelengths
• Blue acuity is lowest: Blue acuity is the lowest, which means that blue light is not as
sharply perceived as other colors. This has implications for how we use blue in designs,
especially small text or fine details.
• 8% of males and 1% of females colour blind

HCI Design Implications: When designing interfaces, it is essential to take into account these
aspects of visual perception. For instance:
• We must ensure that there is sufficient contrast between text and background, regardless of
the user's screen brightness.
• We should be cautious when using blue for small, detailed elements.
 Reading

Reading is multi-stage: The cognitive process of reading is not merely about recognizing letters and words but
involves a complex interaction of visual perception and linguistic understanding.

- - There is the quick simultaneous movement of both eyes called saccades (eye jumps) between two or more phases
of pausing and taking in information fixations (eye pauses).

- Perception primarily during fixations.

- Word shape impacts recognition.

- Negative contrast may aid reading on screens. (light text on dark background)

HCI design: text layout, font choice, contrast, and line length are key for readability.

Example: In a digital library interface, books are displayed with titles in a large, sans-serif font for clear recognition,
while descriptions use a smaller size to differentiate importance. Quick eye movements (saccades) are anticipated
with well-spaced line items, facilitating a seamless browsing experience. Fixations are optimized with high-contrast
text against a dark background to reduce tension and improve focus during selection.
Hearing

• Provides information about environment:

distances, directions, objects etc.

• Physical apparatus:
• outer ear – protects inner and amplifies sound
• middle ear – transmits sound waves as
vibrations to inner ear
• inner ear – chemical transmitters are released
and cause impulses in auditory nerve
• Sound

• pitch – sound frequency

• loudness – amplitude
• timbre – type or quality

In HCI, a navigation app uses sound to guide users: high pitch for a right turn, low
pitch for the left, increasing volume as you approach a turn, and distinct tones for
different alerts like reminders or warnings.
Touch

• Provides important feedback about the environment.

• May be key sense for someone visually impaired.
• It is mediated through various receptors in the skin:
• thermoreceptors – heat and cold
• nociceptors – pain
• mechanoreceptors – pressure
(some instant, some continuous)
• Some areas are more sensitive than others e.g. fingers.
• Kinesthesis - awareness of body position
• affects comfort and performance.

In HCI, a smartphone might vibrate gently to indicate a new notification (using

mechanoreceptors), or use varying vibration patterns for different touch actions
(enhancing sensitivity, especially for visually impaired users). This tactile feedback
helps users understand and navigate the device with a sense of touch, impacting
user comfort and interaction efficiency.
 Movement
• Time taken to respond to stimulus:
reaction time + movement time

• Movement time dependent on age, fitness etc.

• Reaction time - dependent on stimulus type:

• visual ~ 200ms
• auditory ~ 150 ms
• pain ~ 700ms

• Increasing reaction time decreases accuracy in the unskilled

operator but not in the skilled operator.
 Movement (cont)
• Fitts' Law describes the time taken to hit a screen
target:

Mt = a + b log2(D/S + 1)
Mt is movement time
a and b are empirically determined constants
a is the time it takes to initiate the movement
b is the rate at which movement time
increases with the index of difficulty log2(D/S
+ 1)
D is Distance to the target
S is the Size of the target

 targets as large as possible

distances as small as possible
sensory memory
• Buffers for stimuli received through senses
• iconic memory: visual stimuli
• echoic memory: aural stimuli
• haptic memory: tactile stimuli

In HCI, sensory buffers like iconic, echoic, and haptic memory are essential for:

- Iconic memory: Designing visuals that are instantly understandable and leave a lasting
impression, aiding in navigation and task completion.

- Echoic memory: Creating auditory cues that users can recognize and respond to even
after a short delay, essential for alerts and instructions.

- Haptic memory: Informing the design of tactile interactions, ensuring users can
associate specific physical feedback with their actions, crucial for touch-based devices.

These memories contribute to intuitive, efficient, and satisfying user experiences by

leveraging the natural processing capabilities of the human brain.
Memory
There are three types of memory functions:

Sensory memories
The immediate initial recording of info

Short-term memory or working memory

The temporary storage of what we are
thinking about at this moment

Long-term memory
Our permanent storage where information
is stored and encoded
Short-term memory (STM)
STM functions like a temporary holding place or "scratch-pad" for information that
you are actively using or processing at the moment. It's not meant for long-term
storage but rather for handling information that is currently in use.

• rapid access ~ 70ms

• rapid decay ~ 200ms

• limited capacity - 7± 2 chunks

Examples

212348278493202

0121 414 2626

HEC ATR ANU PTH ETR EET

Long-term memory (LTM)

Repository for all our knowledge

• slow access ~ 1/10 second

• slow decay, if any
• huge or unlimited capacity

Two types

• episodic – serial memory of events

• semantic – structured memory of facts,concepts, skills
• semantic LTM derived from episodic LTM

• In HCI, long-term memory (LTM) guides the creation of consistent, intuitive

interfaces that help users remember how to use a system without relearning,
making interactions more memorable and engaging.
Long-term memory (cont.)

• Semantic memory structure

• provides access to information
• represents relationships between bits of information
• supports inference

• Model: semantic network

• inheritance – child nodes inherit properties of parent nodes
• relationships between bits of information explicit
• supports inference through inheritance
LTM - semantic network
Models of LTM - Frames

Information organized in data structures

Slots in structure instantiated with values for instance of data
Type–subtype relationships

DOG COLLIE

Fixed Fixed
legs: 4 breed of: DOG
type: sheepdog
Default
diet: carniverous Default
sound: bark size: 65 cm

Variable Variable
size: colour
colour
LTM - Storage of information

• rehearsal
• information moves from STM to LTM

• total time hypothesis

• amount retained proportional to rehearsal
time

• distribution of practice effect

• optimized by spreading learning over time

• structure, meaning and familiarity

• information easier to remember
LTM - Forgetting
decay
• information is lost gradually but very slowly

interference
• new information replaces old: retroactive
interference
• old may interfere with new: proactive inhibition

so may not forget at all memory is selective …

… affected by emotion – can subconsciously `choose' to

forget
LTM - retrieval
recall
• information reproduced from memory
can be assisted by cues, e.g. categories,
imagery

recognition
• information gives knowledge that it has
been seen before
• less complex than recall - information is
cue
 Thinking
Reasoning:
This is the process of forming conclusions, judgments, or inferences
from facts or premises. lists three types of reasoning:
Deduction: This is reasoning from one or more statements
(premises) to reach a logically certain conclusion. It is often
summarized as "if A, then B".
Induction: This is reasoning from detailed facts to general
principles. Unlike deduction, the conclusions reached are not
necessarily certain, but probable.
Abduction: This is a form of reasoning that starts with an
observation and then seeks the simplest and most likely
explanation. This reasoning is often used in hypothesis
formation.

Problem Solving:
This refers to the process of finding solutions to difficult or
complex issues. It's a mental process that involves discovering,
analyzing, and solving problems. The ultimate goal of problem-
solving is to overcome obstacles and find a solution that best
resolves the issue.
Deductive Reasoning

• Deduction:
• derive logically necessary conclusion from given premises.
e.g. If it is Friday then she will go to work
It is Friday
Therefore she will go to work.

• Logical conclusion not necessarily true:

e.g. If it is raining then the ground is dry
It is raining
Therefore the ground is dry
Inductive Reasoning

• Induction:
• generalize from cases seen to cases unseen
e.g. all elephants we have seen have trunks
therefore all elephants have trunks.

• Unreliable:
• can only prove false not true

… but useful!

• Humans not good at using negative evidence

e.g. Wason's cards.
Wason's cards

7 E 4 K
If a card has a vowel on one side it has an even number on the other

Is this true?
How many cards do you need to turn over to find out?
…. and which cards?
Abductive reasoning

• reasoning from event to cause

e.g. Sam drives fast when drunk.
If I see Sam driving fast, assume drunk.

• Unreliable:
• can lead to false explanations
 Problem solving
Process of finding solution to unfamiliar task using knowledge.

Problem-solving strategies often include:

1. Trial and Error: Trying multiple solutions until finding one that works.
2. Algorithmic Approach: Following a step-by-step, guaranteed solution path.
3. Heuristics: Applying a practical approach or rule of thumb to reduce complex
problem spaces.
4. Divide and Conquer: Breaking down a large problem into smaller, more
manageable parts.
5. Backtracking: Reversing steps to find a new solution path when encountering a
dead-end.
6. Brainstorming: Generating a wide range of ideas and solutions before refining
the selection.
7. Hypothesis Testing: Formulating hypotheses and testing them systematically to
see if they solve the problem.
8. Analogical Thinking: Using similarities with previously solved problems to solve
new ones.
 Problem solving
In HCI, several strategies can be considered to facilitate human
problem-solving:

1. Consistency: Keeping interface elements consistent to reduce

learning time and confusion.
2. Feedback: Providing immediate feedback for actions to help
users understand the results of their interactions.
3. Error Prevention and Recovery: Designing systems to prevent
errors and making it easy to recover from them when they do
occur.
4. Affordances: Using design elements that suggest how they
should be used, like buttons that look pushable.
5. Mapping: Designing controls to correspond spatially with the
elements they affect to support intuitive use.
6. Recognition over Recall: Making information visible so users
don't have to remember information from one part of the interface
to another.
7. Cognitive Load Management: Avoiding overloading users with
too much information or too many choices at once.
 Errors and mental
models
Types of error

• slips
• right intention, but failed to do it right
• causes: poor physical skill, inattention, etc.
• change to an aspect of skilled behavior can cause slip
Example: You mean to hit the 'like' button on a social media post
but accidentally press 'share' because the buttons are close together.
• mistakes
• wrong intention
• cause: incorrect understanding
humans create mental models to explain behaviour.
if wrong (different from the actual system) errors can occur

Example: You delete an important email thinking it's spam

because you don't understand the filtering system that marks
emails with a red flag.
 Emotion
- Definition: Emotions are complex psychological states that
involve a subjective experience, physiological response, and
behavioral or expressive response.

- Types:

- Positive Emotions: Joy, Trust, and Anticipation can increase

user engagement and satisfaction.

- Negative Emotions: Sadness, Disgust, and Anger can lead to

user frustration and product abandonment.

- Effects on Design:

- Positive: Encourage continued use, improve learning, and

foster trust.

- Negative: Can inform areas needing redesign to reduce user

errors and enhance usability.

Design that resonates emotionally can turn a boring interaction

into a memorable experience.
Emotion (cont.)
• The biological response to physical stimuli is called affect

• Affect influences how we respond to situations

• positive → creative problem solving
• negative → narrow thinking

“Negative affect can make it harder to do even easy tasks; positive affect
can make it easier to do difficult tasks”

(Donald Norman)
Emotion (cont.)
• Implications for interface design

• stress will increase the difficulty of problem-solving

• relaxed users will be more forgiving of shortcomings in design
• aesthetically pleasing and rewarding interfaces will increase positive
affect
Individual
differences
• long term
– sex, physical, and intellectual abilities
• short term
– effect of stress or fatigue
• changing
– age

Ask yourself:
will the design decision exclude a section of
user population?