Overview
Mixed Methods Research, Defined
Quantitative Research Qualitative Research When to use Mixed Methods Research Types of Mixed Methods Research Designs Key Characteristics Steps in Conducting a Mixed Methods study Evaluating a Mixed Methods study
Mixed Methods Research, Defined
A mixed methods research design is a procedure for
collecting, analyzing, and mixing both quantitative and qualitative research and methods in a single study to understand a research problem.
To utilize this design effectively, you must understand
both quantitative and qualitative research.
Philosophical Approaches
Creswell , J. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (4thed.). Upper Saddle River, NJ: Pearson Education.
Quantitative Research
A type of educational research in which the research
decides what to study; asks specific, narrow questions, collects quantifiable data from participants (a large number of participants); analyzes these numbers using statistics; and conducts the inquiry in an unbiased, objective manner. Postpositivism singular reality; objective; deductive
Quantitative Research (contd)
Generally attempts to quantify variables of interest;
questions must be measureable. Example:
What is the relationship between graduate students
level of interaction, measured by the number of hits in the course, and students grades in an online research methods course?
Quantitative Methodology
Generally involves collecting numerical data that can
be subjected to statistical analysis Examples of data collection methodologies
Performance Tests
Personality Measures Questionnaires (with closed-ended questions or open-
ended but transferred to quan data) Content Analysis
The data is generally referred to as hard data
Qualitative Research
A type of educational research in which the researcher
relies on the views of participants; asks broad, general questions; collects data consisting largely of words (or text) from participants; describes and analyzes these words for themes; and conducts the inquiry in a subjective, biased manner.
Constructivism multiple realities; biased; inductive
Qualitative Research (contd)
There are times we wish to know not how many or
how well, but simply how (Shulman, 1988, p. 7). Example:
What are the factors that influence a graduate students
experience in an online research methods course?
Qualitative Methodology
Generally involves listening to the participants voice
and subjecting the data to analytic induction (e.g., finding common themes) More Exploratory in nature Examples of data collection methods
Interviews Open-ended questionnaires Observations Content analysis Focus Groups
Steps for Conducting a Mixed Methods Study
Develop quantitative, qualitative, and mixed methods questions
Step 4
Collect quantitative and qualitative data
Step 5
Analyze data separately or concurrently
Identify the data collection strategy and type of design Step 3
Priority Sequence Visualization
Step 6
Identify a rationale for a mixed methods study Step 2
Determine if a mixed methods study is feasible Step 1
Write the report as a one- or two-phase study Step 7
When to Use Mixed Methods Designs
When both quantitative and qualitative data, together,
provide a better understanding of your research problem than either type by itself. When one type of research (qualitative or quantitative) is not enough to address the research problem or answer the research questions. Pragmatism practicality; multiple view points; biased and unbiased; subjective and objective
Creswell , J. (2008). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (3rd ed.). Upper Saddle River, NJ: Pearson Education.
When to Use Mixed Methods (contd)
To incorporate a qualitative component into an
otherwise quantitative study To build from one phase of a study to another
Explore qualitatively then develop an instrument Follow-up a quantitative study qualitatively to obtain
more detailed information
Creswell , J. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (4thed.). Upper Saddle River, NJ: Pearson Education.
Mixed Methods Research Methodology
Utilizes both quantitative and qualitative data
collection methodologies. Examples
Interviews and Questionnaires Performance Tests and Observation Questionnaires and follow up Focus groups Document analysis, Performance Tests, Questionnaire,
and Interviews
What Is Mixed Methods Research?
A mixed methods research design is a procedure for collecting, analyzing, and mixing both quantitative and qualitative research and methods in a single study to understand a research problem.
Key Characteristics of Mixed Methods Designs: Rationale
Rationale Test findings of first phase Explain results of first phase in more detail Provide a more complete understanding than either quantitative or qualitative alone Collecting both quantitative and qualitative data Numeric data Text data
Mixed Methods Legend Notation QUAN QUAL quan Defined Quantitatively driven study. Qualitatively driven study. Quantitative data is secondary to qualitative data.
qual
Qualitative data is secondary to quantitative data.
Indicates that quantitative and qualitative data are collected concurrently. Indicates that quantitative and qualitative data are collected sequentially.
Key Characteristics of Mixed Methods Designs: Priority and Sequence
Priority Equal weight QUAN carries more weight than qual QUAL carries more weight than quan Sequence Collect both quantitative and qualitative data at the same time Collect quantitative data first, followed by qualitative data Collect qualitative data first, followed by quantitative data
Key Decisions In Choosing A Mixed Methods Study
The level of interaction between the quantitative and
qualitative strands The priority of the strands The timing of the strands Where and how to mix the strands
18
The Convergent Parallel Design
Quantitative Data Collection and Analysis
Compare or relate
Qualitative Data Collection and Analysis
Interpretation
20
The Convergent Design
The researcher: Collects quantitative and qualitative data concurrently Analyzes the two data sets separately Mixes the two databases by merging the results during interpretation (and sometimes during data analysis)
21
Purposes for the Convergent Design
Obtain a more complete understanding from two
databases Corroborate results from different methods Compare multiple levels within a system
22
When to Use the Convergent Design
Choose this design if: Need to collect both types of data in one visit to the field Both types of data have equal value for understanding the research problem Have quantitative and qualitative research skills Can manage extensive data collection activities individually or with a team
23
Convergent Design
Philosophical assumptions: Best suited to an "umbrella" paradigm such as pragmatism
Common variants: Parallel-databases variant Data-transformation variant Data-validation variant
24
Strengths: Convergent Design
Intuitive
Efficient Lends itself to teams
25
Challenges: Convergent Design
Requires substantial effort and expertise
Issues related to the samples and sample sizes Difficult to converge two sets of different data How to resolve discrepant results
26
The Explanatory Sequential Design
Quantitative Data Collection and Analysis
Follow up with
Qualitative Data Collection and Analysis
Interpretation
27
The Explanatory Design
The researcher: Starts by collecting and analyzing quantitative data Collects and analyzes qualitative data in a second phase as a follow-up to the quantitative results Connects the phases by using the quantitative results to shape the qualitative research questions, sampling, and data collection
28
Purposes for the Explanatory Design
To use qualitative data to help explain quantitative
results that need further exploration To use quantitative results to purposefully select best participants for qualitative study
29
When to Use the Explanatory Design
Choose this design if: Researcher and research problem are quantitatively oriented Know important variables and instruments are available Participants available for second data collection Have time to conduct two phases Have limited resources and need to collect and analyze one data type at a time New questions emerge from quantitative results
30
Explanatory Design
Philosophical assumptions: Begin from postpositivism for the quantitative phase Shift to constructivism for the qualitative phase
Common variants: Follow-up explanations variant Participant-selection variant
31
Strengths: Explanatory Design
Appealing to quantitative researchers Straightforward to implement two phases Final report can be written in two phases Lends itself to emergent approaches
32
Challenges: Explanatory Design
Two phases require lengthy time to implement
Difficult to secure IRB approval when second phase cannot
be specified before first phase complete Need to decide what results to follow up Must decide criteria for selecting participants Need to contact participants for a second round of data collection
33
The Exploratory Sequential Design
Quantitative Data Collection and Analysis
Follow up with
Qualitative Data Collection and Analysis
Interpretation
34
The Exploratory Design
The researcher: Collects and analyzes qualitative data first followed by quantitative data Analyzes the qualitative data and uses results to build to the subsequent quantitative phase Connects the phases by using the qualitative results to shape the quantitative phase by specifying research questions and variables, developing an instrument, and/or generating a typology
35
Purposes for the Exploratory Design
To first explore because variables, theories, hypotheses
not known To develop an instrument or typology that is not available To assess whether qualitative themes generalize to a population
36
When to Use the Exploratory Design
Choose this design if: Researcher and research problem are qualitatively oriented Important variables not known and instruments not available Have time to conduct two phases Have limited resources and need to collect and analyze one data type at a time New questions have emerged from qualitative results
37
Exploratory Design
Philosophical assumptions: Begin from constructivism for the qualitative phase Shift to postpositivism for the quantitative phase
Common variants: Theory-development variant Instrument-development variant
38
Strengths: Exploratory Design
Straightforward to design, implement, and report
Quantitative component can make the qualitative
approach more acceptable to quantitative-biased audiences Researcher produces a product, such as an instrument Lends itself to emergent approaches
39
Challenges: Exploratory Design
Two phases require lengthy time to implement Difficult to specify quantitative procedures when
applying for initial IRB approval; may have to apply twice Deciding the qualitative findings to use for quantitative phase Procedures for developing a valid and reliable instrument
40
The Embedded Design
Quantitative (or Qualitative) Design Quantitative (or Qualitative) Data Collection and Analysis Qualitative (or Quantitative) Data Collection and Analysis (before, during, or after)
Interpretation
41
Embedded Approach
collect qual
Experimental Design (can use quasiexperimental design)
Correlational QUAN designs to be used within an Embedded Approach Explanatory design
Experimental Between-subjects -pre- and posttest design Within-subjects -cross-over design -factorial design
Predictive design
collect qual
Correlational Design
The Embedded Design
The researcher: Collects and analyzes quantitative and qualitative data within a quantitative research design, qualitative research design, or research procedure Collection and analysis of secondary data set occurs before, during, and/or after the primary methods
43
Purposes for the Embedded Design
To address different questions that call for different
methods To enhance an experiment such as by
improving recruitment procedures examining the intervention process explaining reactions to participation
44
When to Use the Embedded Design
Choose this design if: Have expertise with the primary design Are comfortable with the primary orientation Have little prior experience with the supplemental method Resources limit placing equal priority on both methods Need for a secondary data set emerges
45
Embedded Design
Philosophical assumptions: Worldview may reflect the primary approach, use pragmatism for a concurrent approach, or shift in a sequential approach Common variants: Embedded experiment Embedded correlational Embedded instrument development and validation Mixed methods case study Mixed methods narrative research Mixed methods ethnography
46
Strengths: Embedded Design
May require less time and fewer resources
Improve the larger design with supplemental data Fits team approach well May be able to publish results separately Appealing to those accustomed to traditional designs
47
Challenges: Embedded Design
Need expertise in primary design and mixed methods Must specify purpose for collecting the supplemental data Must decide when to collect supplemental data Results are difficult to integrate Must consider treatment bias if qualitative data collected during experiment
48
Light, G. et al. (2009). Assessing the impact of a year-long faculty development program on faculty approaches to teaching. International Journal of Teaching and Learning in Higher Education, 20(2), 168-181.
Objective: Determine if the Faculty Development
Program (FDP) impacts teaching style of pre-tenure faculty. Research Questions
Quantitative What effect did the FDP have on
teachers approaches to teaching? Qualitative (Central) How did the teachers teaching strategies change in response to the FDP?
Sub - What steps did the teachers take to implement the change?
Mixed Method How do the qualitative (qual) findings
explain (expand on) the quantitative (QUAN) results?
Design. The mixed method design employed was an
explanatory approach with an quasi-experimental design. The quantitative method was quasiexperimental between-subjects approach utilizing a pre- and posttest control group design. Qualitative data was collected at two time points post collect qual intervention.
Assignment
N = 81
n = 52 n = 29
Pretest
ATI ATI
Treatment
Posttest
ATI ATI
qual
Critical Reports Critical Reports
qual (n = 25)
NR NR
FDP -
Interviews
NR = non-random ATI = Approach to Teaching Inventory FDP = Faculty Development Program
Chapter 1: Introduction
Statement of the Problem
The topic. The research problem. Background and justification (philosophical view points). Deficiencies in the evidence. Audience.
Definition of Terms
Chapter 2: Literature Review
Lit Review should include relevant headings and
subheadings. Following the lit review the purpose statement then research questions should be presented. Purpose Statement -(a) the overall content aim, (b) the type of
mixed method design, (c) the forms of data collection that will be used (very general), (d) the data collection site(s), and (e) the reason for collecting both forms of data (see Creswell, 2007).
Research Questions
Quantitative
Qualitative
Mixed Methods
Chapter 3: Methodology
Participants Quantitative. Qualitative. Instruments Procedures Design
Quantitative data. Qualitative data.
Data analysis.
Limitations
Mixed Methods Template
Go to the ARC website for the Mixed Methods specific
template: http://www.fischlerschool.nova.edu/appliedresearch/procedures_and_resources Thank you for your Attention!
