UNIT-IV HCI (CS-348)

Empirical research

Definition:-Empirical research is defined as any research where conclusions of the study is strictly drawn
from concretely empirical evidence, and therefore “verifiable” evidence.
For example: A research is being conducted to find out if listening to happy music in the
workplace while working may promote creativity? An experiment is conducted by using a music website
survey on a set of audience who are exposed to happy music and another set who are not listening to
music at all, and the subjects are then observed. The results derived from such a research will give
empirical evidence if it does promote creativity or not.
In today’s world, the word empirical refers to collection of data using evidence that is collected through
observation or experience or by using calibrated scientific instruments. All of the above origins have one
thing in common which is dependence of observation and experiments to collect data and test them to
come up with conclusions.
Types and methodologies of empirical research

Empirical research can be conducted and analyzed using qualitative or quantitative methods.

Quantitative research methods aid in analyzing the empirical evidence gathered. By using these a
researcher can find out if his hypothesis is supported or not.

 Survey research: Survey research generally involves a large audience to collect a large amount
of data. This is a quantitative method having a predetermined set of closed questions which are
pretty easy to answer. Because of the simplicity of such a method, high responses are achieved.
It is one of the most commonly used methods for all kinds of research in today’s world.

 Experimental research: In experimental research, an experiment is set up and a hypothesis is

tested by creating a situation in which one of the variable is manipulated. This is also used to
check cause and effect. For example: A particular product company is trying to find what is the
reason for them to not be able to capture the market. So the organization makes changes in
each one of the processes like manufacturing, marketing, sales and operations. Through the
experiment they understand that sales training directly impacts the market coverage for their
product. If the person is trained well, then the product will have better coverage.

 Correlational research: Correlational research is used to find relation between two set
of variables. Regression analysis is generally used to predict outcomes of such a method. It can
be positive, negative or neutral correlation.
For example: Higher educated individuals will get higher paying jobs. This means higher education
enables the individual to high paying job and less education will lead to lower paying jobs.

 Longitudinal study: Longitudinal study is used to understand the traits or behavior of a subject
under observation after repeatedly testing the subject over a period of time. Data collected
from such a method can be qualitative or quantitative in nature.
For example: A research to find out benefits of exercise. The target is asked to exercise everyday for a
particular period of time and the results show higher endurance, stamina, and muscle growth. This
supports the fact that exercise benefits an individual body.
  Cross sectional: Cross sectional study is an observational type of method, in which a set of
audience is observed at a given point in time. In this type, the set of people are chosen in a
fashion which depicts similarity in all the variables except the one which is being researched..
For example: A medical study to find the prevalence of under-nutrition disorders in kids of a given
population. This will involve looking at a wide range of parameters like age, ethnicity, location, incomes
and social backgrounds..

 Causal-Comparative research: This method is based on comparison. It is mainly used to find out
cause-effect relationship between two variables or even multiple variables.
For example: A researcher measured the productivity of employees in a company which gave breaks to
the employees during work and compared that to the employees of the company which did not give
breaks at all.
Qualitative research methods
Some research questions need to be analyzed qualitatively, as quantitative methods are not applicable
there. In many cases, in-depth information is needed or a researcher may need to observe a target
audience behavior, hence the results needed are in a descriptive analysis form. Qualitative research results
will be descriptive rather than predictive. It enables the researcher to build or support theories for future
potential quantitative research. In such a situation qualitative research methods are used to derive a
conclusion to support the theory or hypothesis being studied.

 Case study: Case study method is used to find more information through carefully analyzing
existing cases. It is very often used for business research or to gather empirical evidence for
investigation purpose. It is a method to investigate a problem within its real life context
through existing cases.
For example: A report mentioning the solution provided by a company to its client. The challenges they
faced during initiation and deployment, the findings of the case and solutions they offered for the
problems. Such case studies are used by most companies as it forms an empirical evidence for the
company to promote in order to get more business.

 Observational method: Observational method is a process to observe and gather data from its
target. Since it is a qualitative method it is time consuming and very personal. It can be said
that observational research method is a part of ethnographic research which is also used to
gather empirical evidence. This is usually a qualitative form of research, however in some
cases it can be quantitative as well depending on what is being studied.
For example: setting up a research to observe a particular animal in the rain-forests of amazon. Such a
research usually take a lot of time as observation has to be done for a set amount of time to study patterns
or behavior of the subject. Another example used widely nowadays is to observe people shopping in a
mall to figure out buying behavior of consumers.

 One-on-one interview: Such a method is purely qualitative and one of the most widely used. The
reason being it enables a researcher get precise meaningful data if the right questions are
asked. It is a conversational method where in-depth data can be gathered depending on where
the conversation leads.
For example: A one-on-one interview with the finance minister to gather data on financial policies of the
country and its implications on the public.

 Focus groups: Focus groups are used when a researcher wants to find answers to why, what and
how questions. A small group is generally chosen for such a method and it is not necessary to
interact with the group in person. A moderator is generally needed in case the group is being
 addressed in person. This is widely used by product companies to collect data about their
brands and the product.
For example: A mobile phone manufacturer wanting to have a feedback on the dimensions of one of their
models which is yet to be launched. Such studies help the company meet the demand of the customer and
position their model appropriately in the market.

 Text analysis: Text analysis method is a little new compared to the other types. Such a method is
used to analyse social life by going through images or words used by the individual. In today’s
world, with social media playing a major part of everyone’s life, such a method enables the
research to follow the pattern that relates to his study.
For example: A lot of companies ask for feedback from the customer in detail mentioning how satisfied
are they with their customer support team. Such data enables the researcher to take appropriate decisions
to make their support team better.
Sometimes a combination of the methods is also needed for some questions that cannot be answered
using only one type of method especially when a researcher needs to gain a complete understanding of
complex subject matter.
Steps for conducting empirical research

Step #1: Define the purpose of the research

This is the step where the researcher has to answer questions like what exactly do I want to find out?
What is the problem statement? Are there any issues in terms of the availability of knowledge, data, time
or resources. Will this research be more beneficial than what it will cost.
Step #2 : Supporting theories and relevant literature
The researcher needs to find out if there are theories which can be linked to his research problem. He has
to figure out if any theory can help him support his findings. All kind of relevant literature will help the
researcher to find if there are others who have researched this before, or what are the problems faced
during this research.
Step #3: Creation of Hypothesis and measurement
Before beginning the actual research he needs to provide himself a working hypothesis or guess what will
be the probable result. Researcher has to set up variables, decide the environment for the research and
find out how can he relate between the variables.
Step #4: Methodology, research design and data collection
In this step, the researcher has to define a strategy for conducting his research. He has to set up
experiments to collect data which will enable him to propose the hypothesis. The researcher will decide
whether he will need experimental or non experimental method for conducting the research.
Step #5: Data Analysis and result
Data analysis can be done in two ways, qualitatively and quantitatively. Researcher will need to find out
what qualitative method or quantitative method will be needed or will he need a combination of both.
Depending on the unit of analysis of his data, he will know if his hypothesis is supported or rejected.
Analyzing this data is the most important part to support his hypothesis.
Step #6: Conclusion
A report will need to be made with the findings of the research. The researcher can give the theories and
literature that support his research. He can make suggestions or recommendations for further research on
his topic.
 Empirical research methodology cycle

A.D. de Groot, a famous dutch psychologist and a chess expert conducted some of the most notable
experiments using chess in the 1940’s. During his study, he came up with a cycle which is consistent and
now widely used to conduct empirical research. It consists of 5 phases with each phase being as important
as the next one. Following is the empirical cycle.

 Observation: At this phase an idea is sparked for proposing a hypothesis. During this phase
empirical data is gathered using observation. For example: a particular species of flower bloom
in a different color only during a specific season.
 Induction: Inductive reasoning is then carried out to form a general conclusion from the data
gathered through observation. For example: As stated above it is observed that the species of
flower blooms in a different color during a specific season. A researcher may ask a question
“does the temperature in the season cause the color change in the flower?” He can assume that
is the case, however it is a mere conjecture and hence an experiment needs to be set up to
support this hypothesis. So he tags a few set of flowers kept at a different temperature and
observes if they still change the color?
 Deduction: This phase helps the researcher to deduce a conclusion out of his experiment. This
has to be based on logic and rationality to come up with specific unbiased results.For example:
In the experiment, if the tagged flowers in a different temperature environment do not change
the color then it can be concluded that temperature plays a role in changing the color of the
bloom.
 Testing: This phase involves the researcher to return to empirical methods to put his hypothesis
to the test. The researcher now needs to make sense of his data and hence needs to
use statistical analysis plans to determine the temperature and bloom color relationship. If the
researcher finds out that most flowers bloom a different color when exposed to the certain
temperature and the others do not when the temperature is different, he has found support to
his hypothesis. Please note this not proof but just a support to his hypothesis.
 Evaluation: This phase is generally forgotten by most but is an important one to keep gaining
knowledge. During this phase the researcher puts forth the data he has collected, the support
argument and his conclusion. The researcher also states the limitations for the experiment and
his hypothesis and suggests tips for others to pick it up and continue a more in-depth research
for others in the future.
 Advantages of Empirical research
There is a reason why empirical research is one of the most widely used method. There are a few
advantages associated with it. Following are a few of them.

 It is used to authenticate traditional research through various experiments and observations.

 This research methodology makes the research being conducted more competent and authentic.
 It enables a researcher understand the dynamic changes that can happen and change his strategy
accordingly.
 The level of control in such a research is high so the researcher can control multiple variables.
 It plays a vital role in increasing internal validity.
Disadvantages of Empirical research
Even though empirical research makes the research more competent and authentic, it does have a few
disadvantages. Following are a few of them.

 Such a research needs patience as it can be very time consuming. The researcher has to collect
data from multiple sources and the parameters involved are quite a few, which will lead to a
time consuming research.
 Most of the time, a researcher will need to conduct research at different locations or in different
environments, this can lead to an expensive affair.
 There are a few rules in which experiments can be performed and hence permissions are needed.
Many a times, it is very difficult to get certain permissions to carry out different methods of
this research.
 Collection of data can be a problem sometimes, as it has to be collected from a variety of sources
through different methods.

Empirical research questions

In HCI involves identifying specific phenomena or issues within the domain of human-computer
interaction that warrant investigation. These research questions guide the design, execution, and
interpretation of empirical studies, helping researchers to address gaps in knowledge, test hypotheses, and
contribute new insights to the field. Here's a systematic approach to formulating empirical research
questions in HCI:

Identify a Research Area: Begin by identifying a broad area of interest within HCI. This could be
usability, user experience, interface design, interaction techniques, accessibility, collaborative systems, or
any other aspect relevant to human-computer interaction.

Review Existing Literature: Conduct a comprehensive review of existing research literature in the chosen
area to understand the current state of knowledge, identify gaps or unanswered questions, and identify
areas where empirical investigation is needed. Pay attention to recent studies, theoretical frameworks,
methodologies used, and emerging trends in HCI research.

Identify Specific Topics of Interest: Based on the literature review, identify specific topics or subdomains
within the chosen area that you find particularly interesting or promising for empirical investigation.
These topics should be relevant, feasible to study, and have the potential to contribute new knowledge or
insights to the field.
Define Variables and Concepts: Clearly define the key variables, concepts, or constructs that will be
investigated in the research. These could include user behaviors, system characteristics, interface designs,
task contexts, or user characteristics relevant to the research area. Consider how these variables relate to
each other and what hypotheses or research questions they suggest.

Formulate Research Questions:

Based on the identified topics and variables, formulate research questions that articulate the specific aims
or objectives of the empirical study. Research questions in HCI should be clear, focused, and concise,
addressing specific aspects of human-computer interaction that you intend to investigate. Consider the
following guidelines when formulating research questions:

Clarity: Ensure that the research questions are clearly stated and free from ambiguity. Avoid vague or
overly broad questions that are difficult to answer.

Relevance: Ensure that the research questions address important issues or phenomena within the domain
of HCI. The questions should be aligned with the research area and contribute to advancing knowledge in
the field.

Specificity: Formulate research questions that are specific and focused on particular aspects of human-
computer interaction. Avoid overly general questions that cover too broad a scope.

Feasibility: Consider the practical feasibility of addressing the research questions given available
resources, time constraints, and methodological considerations. Ensure that the research questions are
realistic and can be investigated using appropriate methods and techniques.

Consideration of Research Methodology: While formulating research questions, consider the appropriate
research methodology or methodologies that can be used to address them. Different types of research
questions may require different methods, such as experiments, surveys, field studies, or qualitative
interviews. Ensure that the research questions are compatible with the chosen methodology and can be
effectively investigated using the selected approach.

Iterative Refinement: Refine and iterate on the research questions based on feedback from peers, advisors,
or stakeholders in the research project. Consider revising the questions to address any gaps or limitations
identified during the formulation process, ensuring that they accurately reflect the goals and objectives of
the empirical study.

Example Research Questions in HCI:

How does the design of touch-screen interfaces impact the efficiency and usability of smartphone
applications for older adults?
What are the effects of incorporating gamification elements into educational software on student
engagement and learning outcomes?
How do different notification strategies affect users' attention and task performance in multitasking
scenarios?
What are the barriers and facilitators to the adoption of voice-based virtual assistants among users with
visual impairments?
How does the use of augmented reality interfaces influence user perceptions of spatial understanding and
navigation in indoor environments?
Experiment Design

in Human-Computer Interaction (HCI) involves careful consideration of several key elements to ensure
the study effectively addresses research questions and provides meaningful insights. Below is a structured
approach to designing an empirical research experiment in HCI:

1. Identify Research Goals and Questions:

 Clearly define the research goals and objectives. What specific aspect of HCI are you
investigating?
 Formulate research questions that align with the goals of the study. These questions
should be specific, measurable, and relevant to the field of HCI.
2. Select Participants:
 Determine the target population for the study. Who are the users or potential users of the
technology being studied?
 Define inclusion and exclusion criteria for participant selection based on relevant
demographic factors (e.g., age, gender, expertise).
3. Choose Experimental Variables:
 Identify independent variables (factors manipulated by the researcher) and dependent
variables (outcomes measured to assess the effects of the independent variables).
 Independent variables in HCI experiments could include interface design features, task
complexity, input modalities, or system performance parameters.
 Dependent variables may include user performance metrics (e.g., task completion time,
accuracy), subjective evaluations (e.g., usability ratings, user satisfaction), or
physiological responses (e.g., heart rate, eye movements).
4. Design Experimental Conditions:
 Define the experimental conditions or treatment groups that participants will be exposed
to.
 Ensure that experimental conditions are clearly defined, controlled, and systematically
varied to isolate the effects of the independent variables.
 Consider factors such as counterbalancing and randomization to minimize bias and
confounding effects.
5. Develop Experimental Materials:
 Create the stimuli, tasks, or prototypes necessary for the experiment.
 Ensure that experimental materials are standardized, validated (if applicable), and
representative of real-world usage scenarios.
 Pilot test the materials with a small sample of participants to identify and address any
potential issues or ambiguities.
6. Choose Experimental Design:
 Select an appropriate experimental design based on the research questions and objectives.
 Common experimental designs in HCI include:
 Between-subjects Design: Different groups of participants are exposed to
different experimental conditions.
 Within-subjects Design: Each participant experiences all experimental
conditions, with the order of conditions counterbalanced.
 Mixed-design: Combines elements of both between-subjects and within-subjects
designs to capitalize on their respective strengths.
7. Plan Data Collection Methods:
  Determine the data collection methods and instruments needed to measure the dependent
variables.
 Consider using a combination of quantitative (e.g., surveys, performance metrics) and
qualitative (e.g., interviews, observational notes) data collection techniques to obtain
comprehensive insights.
8. Address Ethical Considerations:
 Ensure that the experiment complies with ethical guidelines and regulations governing
research involving human participants.
 Obtain informed consent from participants, maintain confidentiality of participant data,
and minimize potential risks or discomfort.
 Obtain approval from institutional review boards (IRBs) or ethics committees before
conducting the experiment.
9. Implement the Experiment:
 Conduct the experiment according to the planned procedures, ensuring consistency and
rigor in data collection.
 Monitor participant behavior and address any unexpected issues or challenges that arise
during the experiment.
10. Analyze and Interpret Results:
 Analyze the collected data using appropriate statistical or qualitative analysis techniques.
 Interpret the results in relation to the research questions and objectives, considering both
the statistical significance and practical significance of findings.
 Discuss the implications of the results for theory, design practice, or future research in
HCI.
11. Report Findings:
 Write up the results of the experiment in a clear and concise manner, following the
conventions of academic writing.
 Include detailed descriptions of the experimental design, methods, results, and
interpretations.
 Convey the significance of the findings and their implications for the broader field of
HCI.

Data Analysis
In HCI research, data analysis involves examining collected data to derive insights, identify patterns, and
draw conclusions that address research questions or hypotheses. Let's consider an example scenario where
an HCI researcher conducts a study to investigate the impact of interface design on user satisfaction and
performance in a mobile shopping application.

1. Data Collection:
 Participants are recruited to interact with two versions of a mobile shopping application:
one with a traditional interface design and another with a minimalist interface design.
 Data is collected through task performance metrics (e.g., time taken to complete a
purchase, number of errors), subjective ratings of usability and satisfaction, and
demographic information (e.g., age, gender).
2. Data Preprocessing:
 Before analysis, the collected data may require preprocessing steps such as cleaning,
organizing, and formatting.
 Missing or erroneous data points are addressed, outliers are identified and either
corrected or removed if necessary, and variables are properly labeled and coded.
 3. Descriptive Analysis:
 Descriptive statistics are calculated to summarize the characteristics of the data.
 Measures such as means, standard deviations, frequencies, and percentages are computed
for quantitative variables (e.g., task completion time, user ratings), providing an overview
of central tendency and variability.
4. Inferential Analysis:
 Inferential statistical tests are conducted to examine relationships and differences
between variables.
 For example, a paired samples t-test may be used to compare mean satisfaction ratings
between the two interface conditions, while a repeated-measures ANOVA could be
employed to analyze differences in task completion times across multiple interface
designs.
5. Correlation Analysis:
 Correlation analysis is performed to explore relationships between variables.
 For instance, Pearson correlation coefficients might be calculated to assess the strength
and direction of associations between user demographics (e.g., age, experience) and
satisfaction ratings.
6. Qualitative Analysis:
 Qualitative data, such as open-ended responses from user surveys or observations of user
behavior, are analyzed to extract themes and insights.
 Techniques like thematic analysis or content analysis may be employed to identify
recurring patterns and interpret qualitative findings.
7. Integration of Data:
 Quantitative and qualitative data are integrated to provide a comprehensive understanding
of the research findings.
 Triangulation techniques may be used to corroborate results from different data sources
and methodologies, enhancing the validity and reliability of the analysis.
8. Interpretation of Results:
 The results of the data analysis are interpreted in relation to the research questions and
objectives.
 Key findings are discussed, implications for theory and practice are considered, and any
limitations or areas for future research are identified.
9. Reporting of Results:
 The findings of the data analysis are documented in a research report or manuscript.
 The report includes detailed descriptions of the analysis methods, results, interpretations,
and any visual representations (e.g., tables, charts) of the data.

Statistical significance test

Statistical significance tests are essential in empirical research within Human-Computer Interaction (HCI)
to determine whether observed differences or relationships in data are likely to be due to chance or
random variability. These tests help researchers assess the strength and reliability of their findings. Here's
an example of how statistical significance tests can be applied in HCI research:

Example: Let's consider a study that investigates the effect of two different interface designs (A and B)
on users' task completion time in a software application. The researcher recruited 30 participants and
randomly assigned them to either interface A or interface B condition. Each participant performed a set of
tasks using the assigned interface, and their task completion times were recorded.
Research Hypothesis:

 Null Hypothesis (H0): There is no difference in task completion time between interface A and
interface B.
 Alternative Hypothesis (H1): There is a difference in task completion time between interface A
and interface B.

Statistical Analysis: To test the research hypothesis, the researcher can use a statistical significance test
such as the independent samples t-test, assuming that the task completion times are normally distributed
and the variances of the two groups are approximately equal.

1. Data Preparation:
 Calculate the mean task completion time for each interface condition (A and B).
 Compute the standard deviation of task completion times for each group.
 Check assumptions of normality and homogeneity of variances.
2. Selecting the Significance Level (α):
 Choose a significance level (α), commonly set at 0.05, which represents the probability of
rejecting the null hypothesis when it is actually true.
3. Performing the t-test:
 Conduct an independent samples t-test to compare the mean task completion times
between interface A and interface B.
 The t-test will yield a test statistic (t-value) and a p-value.
4. Interpreting the Results:
 If the p-value is less than the chosen significance level (α), typically 0.05, then the null
hypothesis is rejected. This indicates that there is a statistically significant difference in
task completion time between the two interface conditions.
 If the p-value is greater than α, the null hypothesis cannot be rejected, suggesting that
there is no statistically significant difference in task completion time between the two
interface conditions.
5. Reporting the Findings:
 Report the results of the t-test, including the test statistic (t-value), degrees of freedom,
and p-value.
 Provide interpretation of the findings in the context of the research question and
hypothesis.
 Discuss the practical implications of the results for interface design and user experience.

Multi Key press Hindi Test Input Method on a Mobile Phone

Introduction:

 Brief overview of the importance of text input methods on mobile devices.

 Introduction to the challenges of typing in Hindi on mobile phones, especially with limited
keyboards or touchscreens.
 Introduction to the proposed solution: a multi-keypress Hindi input method designed to improve
text input efficiency.

Objective:
  To design and evaluate a multi-keypress Hindi input method for mobile phones.
 To assess the effectiveness, efficiency, and user satisfaction of the proposed input method
compared to existing methods.

Methodology:

1. Design of the Multi-Keypress Input Method:

 Description of the design process, including considerations for key layout, predictive text
algorithms, and user interface design.
 Incorporation of feedback from language experts and potential users during the design
phase.
2. Prototype Development:
 Development of a functional prototype implementing the multi-keypress Hindi input
method on a mobile phone platform (e.g., Android).
 Iterative refinement based on usability testing and feedback from pilot users.
3. Evaluation Procedure:
 Recruitment of participants: Target users proficient in Hindi language and regular users
of mobile text input methods.
 Experimental tasks: Participants will be asked to perform text entry tasks using the multi-
keypress input method and compare it with existing methods (e.g., QWERTY keyboard,
traditional Hindi keyboard).
 Metrics: Efficiency metrics (e.g., text entry speed, error rates), effectiveness (e.g.,
accuracy), and user satisfaction (e.g., Likert scale ratings, qualitative feedback).
 Data collection methods: Observations, task completion times, error logs, and post-task
questionnaires.

Results:

 Quantitative analysis of efficiency metrics comparing the multi-keypress input method with
existing methods.
 Analysis of effectiveness measures and error rates.
 Summary of user satisfaction ratings and qualitative feedback.

Discussion:

 Interpretation of results: Assessing the performance of the multi-keypress input method in

improving text input efficiency and accuracy.
 Comparison with existing methods: Highlighting the advantages and limitations of the proposed
method compared to traditional input methods.
 Implications for design: Insights for optimizing the design of mobile text input methods for Hindi
and other languages.

Conclusion:
 Summary of findings: The multi-keypress Hindi input method demonstrates improved efficiency
and user satisfaction compared to existing methods.
 Future directions: Suggestions for further research and development in mobile text input methods,
including potential enhancements and broader usability studies.
 Overall significance: The study contributes to the field of HCI by addressing the specific needs of
Hindi-speaking users and advancing the design of mobile text input technologies.