‫الفصل الثالث‬

‫اللقاء التاسع‬
‫من مادة تفاعل اإلنسان والحاسوب‬
‫د‪ .‬عبدالرافع محمد الزاملي‬

‫أستاذ هندسة نظم المعلومات والبرمجيات المساعد‬

‫كلية الحاسبات وتكنولوجيا المعلومات‬
‫جامعة األقصى‬
 Chapter 3 –
CONCEPTUALIZING INTERACTION
 Learning Objectives
The main goals of this chapter are to accomplish
the following:
 Explain how to conceptualize interaction.
 Describe what a conceptual model is and how
to begin to formulate one.
 Discuss the use of interface metaphors as part
of a conceptual model.
 Outline the core interaction types for
informing the development of a conceptual
model.
 Introduce paradigms, visions, theories, models,
and frameworks informing interaction design.
 Introduction
When coming up with new ideas as part of a design project,
it is important to conceptualize them in terms of what the
proposed product will do. Sometimes, this is referred to as
creating a proof of concept. In relation to the double
diamond framework, it can be viewed as an initial pass to
help define the area and also when exploring solutions. One
reason for needing to do this is as a reality check where
fuzzy ideas and assumptions about the benefits of the
proposed product are scrutinized in terms of their
feasibility: How realistic is it to develop what they have
suggested, and how desirable and useful will it actually be?
Another reason is to enable designers to begin articulating
what the basic building blocks will be when developing the
product. From a user experience (UX) perspective, it can
lead to better clarity, forcing designers to explain how users
will understand, learn about, and interact with the product.
 For example
For example, consider the bright idea that a designer has of
creating a voice-assisted mobile robot that can help waiters
in a restaurant take orders and deliver meals to customers
(see Figure 3.1). The first question to ask is: why? What
problem would this address? The designer might say that the
robot could help take orders and entertain customers by
having a conversation with them at the table. They could
also make recommendations that can be customized to
different customers, such as restless children or fussy
eaters. However, none of these addresses an actual
problem. Rather, they are couched in terms of the putative
benefits of the new solution. In contrast, an actual problem
identified might be the following: “It is difficult to recruit
good wait staff who provide the level of customer service to
which we have become accustomed.”
For example cont.
Conceptualizing Interaction
When beginning a design project, it is important to be clear about
the underlying assumptions and claims. By an assumption, we
mean taking something for granted that requires further
investigation; for example, people now want an entertainment and
navigation system in their cars. By a claim, we mean stating
something to be true when it is still open to question. For instance,
a multimodal style of interaction for controlling this system—one
that involves speaking or gesturing while driving—is perfectly safe.
 Assumptions and Claims
 Explaining people’s assumptions and claims about why they think something
might be a good idea (or not) enables the design team as a whole to view
multiple perspectives on the problem space and, in so doing, reveals
conflicting and problematic ones. The following framework is intended to
provide a set of core questions to aid design teams in this process:
 Are there problems with an existing product or user experience? If so, what
are they?
 Why do you think there are problems?
 What evidence do you have to support the existence of these problems?
 How do you think your proposed design ideas might overcome these
problems?
 Conceptual Models
A model is a simplified description of a system or process that helps describe
how it works. In this section, we look at a particular kind of model used in
interaction design intended to articulate the problem and design space—the
conceptual model. In a later section, we describe more generally how models
have been developed to explain phenomena in human computer interaction.
In a The core components are as follows:
 Metaphors and analogies that convey to people how to understand what a product
is used for and how to use it for an activity (for example browsing and
bookmarking).
 The concepts to which people are exposed through the product, including the task-
domain objects they create and manipulate, their attributes, and the operations
that can be performed on them (such as saving, revisiting, and organizing).
 The relationships between those concepts (for instance, whether one object
contains another).
 The mappings between the concepts and the user experience the product is
designed to support or invoke (for example, one can revisit a page through looking
at a list of visited sites, most-frequently visited, or saved websites).
 Interface Metaphors
Metaphors are considered to be a central component of
a conceptual model. They provide a structure that is
similar in some way to aspects of a familiar entity (or
entities), but they also have their own behaviors and
properties. More specifically, an interface metaphor is
one that is instantiated in some way as part of the user
interface, such as the desktop metaphor. Another well-
known one is the search engine, originally coined in the
early 1990s to refer to a software tool that indexed and
retrieved files remotely from the Internet using various
algorithms to match terms selected by the user.
 Interface Metaphors cont.
An interface metaphor that has become popular in the last few years is
the card. Many of the social media apps, such as Facebook, Twitter, and
Pinterest, present their content on cards. Cards have a familiar form,
having been around for a long time. Just think of how many kinds there
are: playing cards, business cards, birthday cards, credit cards, and
postcards to name a few. They have strong associations, providing an
intuitive way of organizing limited content that is “card sized.” They can
easily be flicked through, sorted, and themed. They structure content
into meaningful chunks, similar to how paragraphs are used to chunk a
set of related sentences into distinct sections. In the context of the
smartphone interface, the Google Now card provides short snippets of
useful information. This appears on and moves across the screen in the
way people would expect a real card to do—in a lightweight, paper-based
sort of way. The elements are also structured to appear as if they were
on a card of a fixed size, rather than, say, in a scrolling web page (see
Figure 3.5).
 ‫الفصل الثالث‬
‫اللقاء العاشر‬
‫من مادة تفاعل اإلنسان والحاسوب‬
‫د‪ .‬عبدالرافع محمد الزاملي‬

‫أستاذ هندسة نظم المعلومات والبرمجيات المساعد‬

‫كلية الحاسبات وتكنولوجيا المعلومات‬
‫جامعة األقصى‬
 Types of interaction

Here, we describe in more detail each of the five types of interaction.

It should be noted that they are not meant to be mutually exclusive
(for example, someone can interact with a system based on different
kinds of activities); nor are they meant to be definitive. Also, the label
used for each type refers to the user’s action even though the system
may be the active partner in initiating the interaction.
• Instructing: Where users issue instructions to a system.
This can be done in a number of ways, including typing in commands,
selecting options from menus in a windows environment or on a
multitouch screen, speaking aloud commands, gesturing, pressing
buttons, or using a combination of function keys.
 Types of interaction CON.
 Conversing: Where users have a dialog with a system. Users can speak via an
interface or type in questions to which the system replies via text or speech
output.
 Manipulating: Where users interact with objects in a virtual or physical space by
manipulating them (for instance, opening, holding, closing, and placing). Users
can hone their familiar knowledge of how to interact with objects.
 Exploring: Where users move through a virtual environment or a physical space.
Virtual environments include 3D worlds and augmented and virtual reality
systems. They enable users to hone their familiar knowledge by physically moving
around. Physical spaces that use sensor-based technologies include smart rooms
and ambient environments, also enabling people to capitalize on familiarity.
 Responding: Where the system initiates the interaction and the user chooses
whether to respond. For example, proactive mobile location-based technology can
alert people to points of interest. They can choose to look at the information
popping up on their phone or ignore it. An example is the Google Now Card, shown
in Figure 3.5, which pops up a restaurant recommendation for the user to
contemplate when they are walking nearby.
 Instructing
This type of interaction describes how users carry out their tasks by telling the
system what to do. Examples include giving instructions to a system to perform
operations such as tell the time, print a file, and remind the user of an
appointment. A diverse range of products has been designed based on this model,
including home entertainment systems, consumer electronics, and computers. The
way in which the user issues instructions can vary from pressing buttons to typing
in strings of characters. Many activities are readily supported by giving
instructions. In Windows and other graphical user interfaces (GUIs), control keys or
the selection of menu options via a mouse, touch pad, or touch screen are used.
 Conversing
This form of interaction is based on the idea of a person having a
conversation with a system, where the system acts as a dialogue partner. In
particular, the system is designed to respond in a way that another human
being might when having a conversation. It differs from the activity of
instructing insofar as it encompasses a two-way communication process,
with the system acting like a partner rather than a machine that obeys
orders. It has been most commonly used for applications where the user
needs to find out specific kinds of information or wants to discuss issues.
Examples include advisory systems, help facilities, chatbots, and robots.
 Manipulating
This form of interaction involves manipulating objects, and it
capitalizes on users’ knowledge of how they do so in the physical
world. For example, digital objects can be manipulated by moving,
selecting, opening, and closing. Extensions to these actions include
zooming in and out, stretching, and shrinking—actions that are not
possible with objects in the real world. Human actions can be imitated
through the use of physical controllers (for example, the Wii) or
gestures made in the air, such as the gesture control technology now
used in some cars. Physical toys and robots have also been embedded
with technology that enable them to act and react in ways depending
on whether they are squeezed, touched, or moved. Tagged physical
objects (such as balls, bricks, or blocks) that are manipulated in a
physical world (for example, placed on a surface) can result in other
physical and digital events occurring, such as a lever moving or a sound
or animation being played.
 Manipulating

 In so doing, direct manipulation interfaces are assumed to enable users to feel that they
are directly controlling the digital objects represented by the computer. The three core
principles are as follows:
 Continuous representation of the objects and actions of interest
 Rapid reversible incremental actions with immediate feedback about the object of
interest
 Physical actions and button pressing instead of issuing commands with complex syntax
According to these principles, an object on the screen remains visible while a user
performs physical actions on it, and any actions performed on it are immediately visible.
For example, a user can move a file by dragging an icon that represents it from one part
of the desktop to another. The benefits of direct manipulation include the following:
 Helping beginners learn basic functionality rapidly
 Enabling experienced users to work rapidly on a wide range of tasks
 Allowing infrequent users to remember how to carry out operations over time •
Preventing the need for error messages, except rarely
 Showing users immediately how their actions are furthering their goals
 Reducing users’ experiences of anxiety
 Helping users gain confidence and mastery and feel in control.
Exploring
This mode of interaction involves users moving through virtual
or physical environments. For example, users can explore
aspects of a virtual 3D environment, such as the interior of a
building. Physical environments can also be embedded with
sensing technologies that, when they detect the presence of
someone or certain body movements, respond by triggering
certain digital or physical events. The basic idea is to enable
people to explore and interact with an environment, be it
physical or digital, by exploiting their knowledge of how they
move and navigate through existing spaces.
Paradigms, Visions, Theories, Models,
and Frameworks
 A paradigm refers to a general approach that has been adopted by a
community of researchers and designers for carrying out their work in terms
of shared assumptions, concepts, values, and practices.
 A vision is a future scenario that frames research and development in
interaction design— often depicted in the form of a film or a narrative.
 A theory is a well-substantiated explanation of some aspect of a
phenomenon; for example, the theory of information processing that
explains how the mind, or some aspect of it, is assumed to work.
 A model is a simplification of some aspect of human-computer interaction
intended to make it easier for designers to predict and evaluate alternative
designs.
 A framework is a set of interrelated concepts and/or Figure 3.9 Google Lens
in action, providing pop-up information about Pembroke Welsh Corgi having
recognized the image as one Source: a set of specific questions that are
intended to inform a particular domain area (for example, collaborative
learning), or an analytic method (for instance, ethnographic studies).
 Paradigms

Following a particular paradigm means adopting a set of practices upon

which a community has agreed.
These include the following:
 The questions to be asked and how they should be framed
 The phenomena to be observed
 The way in which findings from studies are to be analyzed and
interpreted
 Visions
Visions of the future, like Mark Weiser’s vision of ubiquitous technology,
provide a powerful driving force that can lead to a paradigm shift in
terms of what research and development is carried out in companies and
universities. A number of tech companies have produced videos about
the future of technology and society, inviting audiences to imagine what
life will be like in 10, 15, or 20 years’ time. One of the earliest was
Apple’s 1987 Knowledge Navigator, which presented a scenario of a
professor using a touchscreen tablet with a speech-based intelligent
assistant reminding him of what he needed to do that day while
answering the phone and helping him prepare his lectures. It was 25
years ahead of its time—set in 2011— the actual year that Apple
launched its speech system, Siri. It was much viewed and discussed,
inspiring widespread research into and development of future interfaces.
 Theories
Over the past 30 years, numerous theories have been imported into
human-computer interaction, providing a means of analyzing and
predicting the performance of users carrying out tasks for specific
types of computer interfaces and systems (Rogers, 2012). These have
been primarily cognitive, social, affective, and organizational in
origin.

For example, cognitive theories about human memory were used in

the 1980s to determine the best ways of representing operations,
given people’s memory limitations. One of the main benefits of
applying such theories in interaction design is to help identify factors
(cognitive, social, and affective) relevant to the design and evaluation
of interactive products. Some of the most influential theories in HCI,
including distributed cognition, will be covered in the next chapter.
 Models

We discussed earlier why a conceptual model is

important and how to generate one when designing a
new product. The term model has also been used
more generally in interaction design to describe, in a
simplified way, some aspect of human behavior or
human-computer interaction. Typically, it depicts
how the core features and processes underlying a
phenomenon are structured and related to one
another. It is usually abstracted from a theory coming
from a contributing discipline, like psychology.
 For example
For example, Don Norman (1988) developed a number of
models of user interaction based on theories of cognitive
processing, arising out of cognitive science, which were
intended to explain the way users interacted with
interactive technologies. These include the seven stages of
action model that describes how users move from their
plans to executing physical actions that they need to
perform to achieve them to evaluating the outcome of
their actions with respect to their goals. More recent
models developed in interaction design are user models,
which predict what information users want in their
interactions and models that characterize core
components of the user experience, such as Marc
Hassenzahl’s (2010) model of experience design.
 Frameworks
Numerous frameworks have been introduced in interaction
design to help designers constrain and scope the user
experience for which they are designing. In contrast to a
model, a framework offers advice to designers as to what to
design or look for. This can come in a variety of forms,
including steps, questions, concepts, challenges, principles,
tactics, and dimensions. Frameworks, like models, have
traditionally been based on theories of human behavior, but
they are increasingly being developed from the experiences
of actual design practice and the findings arising from user
studies. Many frameworks have been published in the
HCI/interaction design literature, covering different aspects
of the user experience and a diversity of application areas.
 For example,

For example, there are frameworks for helping designers think about
how to conceptualize learning, working, socializing, fun, emotion, and
so on, and others that focus on how to design particular kinds of
technologies to evoke certain responses, for example, persuasive
technologies (see Chapter 6, “Emotional Interaction”). There are others
that have been specifically developed to help researchers analyze the
qualitative data they collect in a user study, such as Distributed
Cognition (Rogers, 2012). One framework, called DiCoT (Furniss and
Blandford, 2006), was developed to analyze qualitative data at the
system level, allowing researchers to understand how technologies are
used by teams of people in work or home settings. (Chapter 9, “Data
Analysis,” describes DiCoT in more detail.)