Mixed Methods Research:

Design and Procedures

by John W. Creswell, Ph.D.
Department of Educational Psychology,
University of Nebraska-Lincoln, and Founding Co-Editor,
Journal of Mixed Methods Research

University of Pretoria, October 21, 2008

Please do not duplicate or use these slides without the express permission of the
author. 1
Topics

Introductions
Positioning myself
A definition of mixed methods research
Steps in the process of designing a mixed
methods study

2
Positioning myself

View research as set of interactive

components; not always linear
Focus on rigorous data collection and
analysis
Work as an applied research methodogist
Trained in quantitative, self-trained in
qualitative, lst generation mixed methods
writer
Serve as a consultant on mixed methods
on projects
A few more thoughts about myself

Published by Sage Publications, Pearson Education (Merrill Educ.)

A Definition of Mixed Methods
Research
How would you combine two types of data?

Qualitative
Text Data

This is a sample of a text file of

words that might be collected
on interview transcripts,
observation fieldnotes, or
optically-scanned documents. Quantitative Numeric Data

2 3 4 2 5 2 3 1 1 2 3 4 2 3 3 2 1 1
153412314455412143
351423155221535131
532251324431242241
554215
Framework for viewing
perspectives on mixed methods

Quantitative Data Qualitative Data

Mixed Methods

Method
Methodology
Paradigm
Perspective Use of mixed methods
in other designs
7
A mixed methods researcher

Collects both quantitative and qualitative data

Mixes them
Mixes them at the same time (concurrently) or one
after the other (sequentially)
Emphasizes both equally or unequally
A definition
Mixed methods research is both a
method and methodology for conducting
research that involves collecting,
analyzing, and integrating quantitative
and qualitative research in a single study
or a longitudinal program of inquiry.
The purpose of this form of research is
that both qualitative and quantitative
research, in combination, provide a better
understanding of a research problem or
issue than either research approach alone.
9
Collecting both quantitative
and qualitative data
Quantitative data
Instruments
Checklists
Records

Qualitative data
Interviews
Observations
Documents
Audio-visual materials
10
Quantitative and qualitative
data analysis

Qualitative analysis
Use text and
Quantitative analysis
images,
Use statistical
For coding
analysis,
For theme
For description
development
For comparing
For relating themes
groups
For relating variables

11
Mixing or linking the data

Converge
data:
Qual Results Quan

Connect data:

Qual Quan Results

Embed the data:

Quan data
Qual data
12
Typical situations in which mixed
methods is used

To compare results from quantitative and

qualitative research
To use qualitative research to help explain
quantitative findings
To explore using qualitative research and
then to generalize findings to a large
population using quantitative research
To develop an instrument because none are
available or useful
To augment an experiment with qualitative
data
 What is the reason for using mixed
methods?

The insufficient argument either quantitative or qualitative may be

insufficient by itself
Multiple angles argument quantitative and qualitative approaches
provide different pictures
The more-evidence-the-better argument combined quantitative and
qualitative provides more evidence
Community of practice argument mixed methods may be the
preferred approach within a scholarly community
Eager-to-learn argument it is the latest methodology
Its intuitive argument it mirrors real life
Designing a Mixed Methods Study

Preliminary considerations
Creating a title
Posing a general question
Listing the types of data collection and
analysis
Making explicit your worldview
Identifying your research design
Drawing a figure of your design
Writing a purpose statement
Writing research questions
Completing a research plan
Preliminary considerations (before
you begin to design)

Research problem
Content any topics
Fit the problem to mixed methods (arguments)

Access to both qualitative and quantitative data

Background and resources

Receptive audience

16
Create a working title

Writing the title

Short
Topic
Participants
Include the words Mixed methods
Neutral neither quan or qual

17
Pose the general question to be
answered

Write it as a question
Look to see how it is phrased
Make sure that it is specific enough and
focused (an answerable question)
Ask yourself, when I end the study, what
question would like to have answered?

18
List your types of data
collection (a review)
Quantitative data
(closed-ended)
Instruments
Behavioral
checklists
Records
Qualitative data (open-ended)
Interviews
Observations
Documents
Audio-visual materials
19
Activity List your sources of data

Quantitative Sources
of Data

Qualitative Sources of
Data
20
List your approach to data
analysis (a review)

Qualitative analysis
Quantitative analysis
Use text and
Use statistical
images,
analysis,
For coding
For description
For theme
For comparing
development
groups
For relating themes
For relating variables
Design-type
Design-type

21
Determine your worldview
Postpositivism Constructivism
Determination Understanding
Reductionism Multiple participant meanings
Empirical observation and Social and historical
measurement construction
Theory verification Theory generation

Advocacy/Participatory Pragmatism
Political Consequences of actions
Empowerment issue-oriented Problem-centered
Collaborative Pluralistic
Change-oriented Real-world practice oriented

22
Several stances on philosophy in
mixed methods

One paradigm (pragmatism, transformative)

(Tashakkori & Teddlie, 2003; Mertens, 2003)
Multiple paradigms (dialectic perspective)
(Greene, 2007)
Linking paradigms to design features) (Creswell &
Plano Clark, 2007)
Epistemological stance (ontology, epistemology,
axiology, methodology) (Guba & Lincoln, 2005)
Shared beliefs in a research field (Morgan, 2007)

What it is
How it informs your study
23
Make explicit your interpretive lens
(theory)

Advocacy lens (feminist, racial, ethnic,

disability, sexual orientation) (Mertens, 2003)
Social science lens (social science theory)

Components:
What is it
Who has used it in your field
How it will shape your study (rephrase your
guiding research question, if a lens applies)

24
State your procedures (methods)

Procedures for handling your qualitative and

quantitative data

Sequence concurrent or sequential or both

Emphasis emphasis on qualitative or
quantitative
Sometimes both concurrent and sequential
phases are used
Designs may include more than two phases
Think about using a simple, elegant design

25
monious designs (Creswell & Plano Clark,
Concurrent Mixed Methods Designs
Triangulation Design

QUAN
QUAN QUAL
QUAL
Data
Data&& Data
Data&&
Interpretation
Results
Results
Interpretation Results
Results

Embedded Design

QUAN Intervention QUAN

QUAN QUAN
Pre-test
Pre-test Post-test
Post-test Interpretation
Interpretation
Data
Data&& Data
Data&&
qual
qual
Results
Results Results
Results
Process
Process

26
 Sequential
designs
Explanatory Design
QUAN
QUAN qual
qual
Data
Data&& Data
Data&&
Interpretation
Results
Results Following up
Results
Results Interpretation

Exploratory Design
QUAL
QUAL quan
quan
Data
Data&& Data
Data&& Interpretation
Results
Results Building to
Results
Results Interpretation

Sequential Embedded Design

Before-
QUAN
QUAN After-
Before- Intervention After-
intervention Intervention
intervention Trial intervention
Trial intervention
qual
qual qual Interpretation
Interpretation
qual

27
 Design options

Design Name Equal priority QUAN emphasis QUAL emphasis

Concurrent, triangulation QUAL+QUAN QUAN+qual QUAL+quan

Concurrent, embedded n/a QUAN(qual) QUAL(quan)

Explanatory, sequential, quan first QUANQUAL QUANqual quanQUAL

Exploratory sequential, qual first QUALQUAN qualQUAN QUALquan

Sequential, embedded n/a (qual) QUAN (quan) QUAL

QUAN (qual) QUAL (quan)

28
Explanatory sequential design
Quantitative Case Selection Qualitative Interpretation
Qua itative
Data Data Analysis Data Analysis
nt Collection
(quan) (QUAL) based on quan
ad QUAL
results
+
Qualitative
Data Collection

Quantitative Data* Quantitative Analysis

Case Selection Qualitative Analysis Interpretation
Number of cigarettes Graphic plot of CES
- Selected 5 cases Descri
ption of each Why did changes in
CES-D6 D6 scores over time maximally varying case smoking occur?
for each participant Identified critical Identification of life
Qualitative Data* Graphic plot of months in which events occurring
Semi
structured
- cigarettes/day values smoking varied during critical
over time for each months where
interviews, audio
participant smoking increased or
recorded and
decreased
transcribed
Thematic analysis of
life events for each
* Data collected 10 times
case
over the course of a
calendar year for 40 Cross
case
- thematic
participants analysis

Source: Creswell, Plano Clark, Shope, McVea. (in progress)

 Phase I Qualitative Research - Year 1

Qualitative Data Collection Unstructured Interviews -

50 participants
8 observations at the site
16 documents

Qualitative Data Analysis Text Analysis: Using QSR N6

Development of codes and themes

Qualitative Findings for each site

Phase II Quantitative Research - Year 2

Create approximately a 80-item
Quantitative Instrument Development instrument plus demographics

Administer survey to 500 individuals

Quantitative Test of the Instrument
Determine factor structure of items and
conduct reliability analysis for scales

Quantitative Results
Determine how groups differ
using ANOVA test

xploratory sequential design

Embedded research design

Experiment
Intervention
QUAN QUAN
Data collection Data collection
Pre-test Post-test

Process collection
and analysis of qualitative
data
(before, during, after trial)
32
Mixing the quan and qual data

Type of Mixing Type of Why Mixing Where Mixing

Design Occurs Occurs in
Research
Process
Connecting Sequential One phase Between data
builds on the analysis (Phase
other 1) and data
collection
(Phase 2)
Merging Concurrent Bring results After analysis of
together both quan and
qual typically
in discussion
Embedding Sequential or Either building Either between
Concurrent or bringing phases or in
results together discussion after
analysis 33
Methodological issues

Concurrent Designs Sequential designs

Use strategies to In Explanatory Design,
select qual sub-sample
explore contradictory
from quan sample
findings
In Explanatory Design,
Use parallel questions
consider alternatives for
Select sub-sample of followup qual sampling
quantitative for In Exploratory Design,
qualitative samples can differ
Be sensitive to bias In Exploratory Instrument
from one data Design, consider qual
collection to the other data analysis approaches
for developing instrument
34
Write a purpose statement for a
triangulation design
This mixed methods study will address _________________
(overall content-aim of the study). A triangulation mixed
methods design will be used, and it is a type of design in
which different but complementary data will be collected
on the same topic. In this study, _________________
(quantitative instruments) will be used to test the theory
of _____________ (the theory) that predicts that
__________________ (independent variables) will influence
________________ (positively, negatively) the __________
(dependent variables or outcomes) for ____________
(participants) at __________ (the research site). Concurrent
with this data collection, qualitative _______________ (type of
qualitative data, such as interviews) will explore ____________
(the central phenomenon) for _________________ (participants)
at _____________(site). The reason for collecting both
quantitative and qualitative data are to bring together the
strengths of both forms of research to ___________________
(e.g., compare results, validate results, corroborate
35
results).
Write a purpose statement for
a embedded design
This mixed methods study will address _________________
(overall content aim of the study). An embedded mixed
method design will be used, and it is a design in which
one data set provides a supportive, secondary role in a
study based primarily on the other data set. The primary
purpose of this study will use ________________ (quantitative
instruments) to test the theory of _____________ (the
theory) that predicts that __________________ (independent
variables) will influence ________________ (positively,
negatively) the __________ (dependent variables or
outcomes) for ____________ (participants) at __________ (the
research site). A secondary purpose will be to gather
qualitative data _______________ (type of qualitative data,
such as interviews) that will explore ____________ (the
central phenomenon) for _________________ (participants) at
_____________(site). The reason for collecting the secondary
database is ________________ (e.g., to address different
question, to provide support for the primary purpose).

36
Write a purpose statement for
an explanatory design
This study will address _______________ (content-aim of the
study). An explanatory mixed methods design will be used,
and it will involve collecting qualitative data after a
quantitative phase in order to explain or follow up on the
quantitative data in more depth. In the first quantitative
phase of the study, ______________ instrument data to be
collected from _______________ (participants) at ___________
(research site) to test _______ (the theory) that explains
why ______________ independent variables) relate to the
____________ (dependent variables). The second qualitative
phase will be conducted because ______________ (intent of
the qualitative phase). In this exploratory follow-up, the
______________ (central phenomenon) will be tentatively
explored with ___________ (participants) at _____________ (the
research site). The reason for the exploratory follow-up is
to _______________ (e.g., to help explain or build upon initial
quantitative results).
37
Write a purpose statement for
an exploratory design
This study addresses _________________ (content-area of the
study). The purpose of this exploratory sequential design will
be to __________________ (e.g., develop an test an instrument,
generate a taxonomy). The first phase of the study will be a
qualitative exploration of ________________ (the central
phenomenon) by collecting ___________________ (types of data)
from _________________ (participants) at _____________ (the
research site). The second quantitative phase will follow up
on the qualitative phase for the purpose of __________(intent of
this followup). In the quantitative phase, ___________
(instrument data) will be collected from ____________
(participants) at ______________ (research site). Quantitative
research questions/hypotheses will be formulated after the
completion of the initial qualitative phase. The reason for
collecting qualitative data initially is that _____________________
(e.g., instruments are not available, variables are not known,
there is little guiding theory).
38
Qualitative research questions

Qualitative central question

Begin with what or how
Focus on single phenomenon
Use exploratory verbs (discover, understand,
explore)
Non-directional language
A general question (allowing participants
perspectives to emerge)

39
Quantitative research questions

Can be hypotheses or questions

State variables independent, dependent,
mediating, covariates
Develop from theory
Use distinct measures for independent and
dependent variables
Order variables from independent to
dependent

40
Writing research
questions/hypotheses in mixed
methods research

Write qualitative research questions and write

quantitative research questions/hypotheses

Also write a mixed methods research question

Write these questions separately

41
A new type of research question: A mixed
methods question
Three ways to write this question:

Methodologically-focused:

To what extent do the qualitative results confirm the quantitative

results?

Content-focused:

How do the interviews with adolescent boys support the

quantitative results that their self-esteem changes during the
middle school years?

Hybrid of quantitative and qualitative elements:

What results emerge from comparing the exploratory qualitative

Order the topics for your plan

Title
Introduction
(Problem)
Worldview/theoretical lens
Audience
Purpose Statement
Research Questions
(Literature Review)
Methods
Type of Mixed Methods Design (also add definition of mixed methods)
Types of Data Collection
Types of Data Analysis
Sequence/Emphasis/ Mixing Procedures
Figure of procedures
Anticipated methodological issues
Ethical issues anticipated
Validity issues
Researcher resources and skills
References, Appendices

43
Additional resources
Books:
Creswell, J. W., & Plano Clark, V. L. (2007). Designing and
conducting mixed methods research. Thousand Oaks, CA:
Sage.
Creswell, J. W. (2008). Research design: Qualitative,
quantitative, and mixed methods approaches (3rd ed.).
Thousand Oaks, CA: Sage Publications.
Greene, J. C. (2007). Mixed methods in social inquiry. San
Francisco: Jossey-Bass.
Mertens, D. M. (2005). Research methods in education and
psychology: Integrating diversity with quantitative and
qualitative approaches (2nd ed.). Thousand Oaks, CA: Sage
Publications.
Plano Clark, V. L., & Creswell, J. W. (2008). The mixed
methods reader. Thousand Oaks, CA: Sage.
Tashakkori, A. & Teddlie, C. (1998). Mixed methodology:
Combining qualitative and quantitative approaches.
Thousand Oaks, CA: Sage Publications.
Tashakkori, A. & Teddlie, C. (Eds.) (2003). Handbook of
mixed methods in social and behavioral research. Thousand
Oaks, CA: Sage Publications. 44
Additional resources
Articles and Chapters:

Caracelli, V. J., & Greene, J. C. (1993). Data analysis strategies for mixed-
method evaluation designs. Educational Evaluation and Policy Analysis, 15 (2),
195-207.
Creswell, J. W., Plano Clark, V. L., Gutmann, M., & Hanson, W. (2003).
Advanced mixed methods research designs. In: A. Tashakkori & C. Teddlie
(Eds.), Handbook of mixed methods in social and behavioral research (pp. 209-
240). Thousand Oaks, CA: Sage.
Creswell, J. W., Plano Clark, V. L., & Garrett, A. L. (2008). Methodological
issues in conducting mixed methods research. In M.M. Bergman (Ed.),
Advances in mixed methods research. London: Sage.
Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual
framework for mixed-method evaluation designs. Educational Evaluation and
Policy Analysis, 11 (3), 255-274.
Ivankova, N. V., Creswell, J. W., & Stick, S. (2006). Using mixed methods
sequential explanatory design: From theory to practice. Field Methods, 18(1),
3-20.
Morgan, D. L. (2007). Paradigms lost and pragmatism regained. Journal of
Mixed Methods Research, 1, 48-76.
Morse, J. M. (1991). Approaches to qualitative-quantitative methodological
triangulation. Nursing Research, 40, 120-123.

45
 Mixed Methods Research:
Design and Procedures

by John W. Creswell, Ph.D.

Department of Educational Psychology,
University of Nebraska-Lincoln, and Founding Co-Editor,
Journal of Mixed Methods Research

University of Pretoria, October 21, 2008

Please do not duplicate or use these slides without the express permission of the
author. 46