Practical Research: Planning and Design

Twelfth Edition, Global Edition

Chapter 4
Planning Your Research
Project
Research Design
• When we talk about research design, we are referring to a
general strategy for addressing a research problem or question.
This includes:
– The procedures the researcher follows
– The data the researcher collect
– The data analyses the researcher conducts
• Simply put, research design is planning!
Planning a General Approach (1 of 2)
• Think broadly about the problem as arising out of a particular
area
• Are you studying
– People
– Things
– Records
– Thoughts & ideas
– Dynamics & energy
▪ Each of these is a potential unit of analysis
Planning a General Approach (2 of 2)
• Think about the kinds of data you need to address your
problem
– Do you need/can you find participants
– Do you have the right equipment and skills
– Do you know how to interpret the data and draw
conclusions from them
Research Planning: Selecting a Particular
Research Methodology
• Planning
– Determining the general approach to a study
– May be similar across disciplines
• Methodology
– The techniques one uses to collect and analyze data
– May be specific to a particular academic discipline
The Nature and Role of Data in Research (1 of
2)

• Data are pieces of information that help form a bigger picture

• Data are transient — what is true at any point in time may not
be true at another point in time
The Nature and Role of Data in Research (2 of
2)
• Data may be primary or secondary
– Primary data are closest to the truth (the source)
– Secondary data are derived from primary data
▪ Distorted by interpretations and communication
 Planning for Data Collection

1. What data are needed?

2. Where are the data located?
3. How will the data be obtained?
4. What limits will be placed on the nature of acceptable data?
5. How will the data be interpreted?
Linking Data and Methodology
• Quantitative methods
– Involve collecting numerical data
• Qualitative methods
– Involve collecting textual or image-based data
• Mixed methods
– Use both quantitative and qualitative methods in the
same study
To Determine an Approach, First Ask
Yourself These Questions
• What is my purpose?
• What is the nature of the process?
• What are the data like/how are they collected?
• How are data analyzed?
• How are the findings communicated?
Also Consider These Issues (1 of 2)
• Your comfort with the assumptions of the qualitative tradition
• The audience for your study
• The nature of the research question
• The extensiveness of the related literature
• The depth of what you want to discover
• The amount of time you have available for conducting the
study
Also Consider These Issues (2 of 2)
• The extent to which you are willing to interact with the people
in your study
• The extent to which you feel comfortable working without
much structure
• Your ability to organize and draw inferences from a large body
of information
• Your writing skills
Quantitative V s. Qualitative (1 of 3)
ersu

• Purpose
– Quant : Pursuit of explanations and predictions that, in most cases, will
itative

generalize to other persons or places

– Qual : Seek to better understand complex situations and is often exploratory in
itative

nature
• Process
– Quant : Methods allow the research to objectively measure the variable(s) of
itative

interest
– Qual : Holistic and emergent, with specific focus, design, data-collection
itative

techniques, and interpretations developing and possibly changing along the

way
Quantitative V s. Qualitative (2 of 3)
ersu

• Data Collection
– Quant : Identify a few variables to study and then collect data specifically
itative

related to those variables.

– Qual : Qualitative research operate under the assumption that reality isn’t easily
itative

divided into discrete, measureable variables.

• Data Analysis
– Quant : Rely on deductive reasoning, beginning with certain premises and then
itative

drawing logical conclusions from them.

– Qual : Make considerable use of inductive reasoning.
itative
Quantitative V s. Qualitative (3 of 3)
ersu

• Reporting Findings
– Quant : Typically reduce their data to summarizing
itative

statistics.
– Qual : Often construct interpretive narratives from their
itative

data and try to capture the complexity of a particular

phenomenon.
Deciding between Quant and Qual (1 of 2)
1. Consider your own comfort level with the assumptions of the qualitative tradition.

2. Consider the nature of your research question.

3. Consider the extensiveness of the related literature.

4. Consider the depth of what you wish to discover.

5. Consider the amount of time you have available for conducting the study.

6. If applicable, consider the extent to which you are willing to interact with the
people in your study.
Deciding between Quant and Qual (2 of 2)
7. Consider the extent to which you feel comfortable working
without much structure.
8. Consider your ability to organize and draw inferences from a
large body of information.
9. Consider your writing skills.
Select a Research Methodology
• Action research • Ex post facto research
• Case study • Grounded theory research
• Content analysis • Historical research
• Correlational research • Observation study
• Design-based research • Phenomenological research
• Developmental research • Quasi-experimental research
• Ethnography • Survey research
• Experimental research
Enhancing the Credibility of Your Findings (1 of 2)
• A research study has credibility to the extent that other people:
– agree that its design and methods are being appropriate for
the research problem or question
– judge its results as being reasonably accurate and
trustworthy
– find the researcher’s interpretations of the dat to be
plausible.
Enhancing the Credibility of Your Findings (2 of 2)
• Quantitative researchers often use the term internal validity
when evaluating credibility.
• Qualitative researchers use a variety of terms such as:
– trustworthiness
– confirmability
– interpretative rigor
Strategies to Enhance Confirmability (1 of 2)
• Triangulation
– Multiple sources of data are collected with the hope that they will all converge to support
a particular assertion, hypothesis, theory, or conclusion

• A controlled laboratory study

• Extensive time in the field

• Unobtrusive measures
– Gathering data about people’s behaviors when those people don’t know their actions are
being recorded

• Thick description
Strategies to Enhance Confirmability (2 of 2)
• Respondent validation
– Confirming conclusions with participants
• Analysis of contradictory data, such as discrepant perspectives
or quantitative outliers
• Follow-up studies specifically designed to eliminate alternative
explanations for findings
Enhancing the Generalizability of Your
Findings
• Generalizability is the extent to which results obtained and conclusions drawn can
be applied to other people, situations, or contexts
• Quantitative researchers usually use the term external validity

• Qualitative researchers use terms such as:

– transferabilty
– particularizability
– replicability
– confirmability
Strategies for Enhancing Generalizability
• A naturalistic, real-world setting
• Obtaining a representative sample
• Replication in a different context
Choosing Appropriate Assessment
Strategies
• Many methodologies require one or more assessment
strategies.
– Researchers must in some way capture and evaluate the
nature of important characteristics, behaviors, or other
variables under investigation.
• Assessment often involves measurement, in other words, it
imposes a range of numbers on an entity being assessed.
 Measurement (1 of 3)

• Limiting the data of any phenomenon—substantial or

intangible— so that those data may be interpreted and,
ultimately, compared to a particular qualitative or quantitative
standard
 Measurement (2 of 3)

• Limiting the data of any phenomenon—substantial or

intangible—so that those data may be interpreted and
ultimately, compared to a particular qualitative or quantitative
standard
– Substantial = have physical substance.
– Intangible = exist only as concepts, ideas, opinions,
feelings, or other intangible entities.
 Measurement (3 of 3)
• Limiting the data of any phenomenon—substantial or
intangible—so that those data may be interpreted and,
ultimately, compared to a particular qualitative or
quantitative standard
– transformed into new discoveries, revelations, and
enlightenments.
Assessing Intangible Phenomena: An
Example
Measuring interpersonal dynamics in a small group
• Ask each person: Who do you like most, who do you like least,
and who evokes neutral feelings
• Allow the researcher to identify patterns and draw conclusions
• Create a sociogram, or a chart of interpersonal reactions (see
Figure 4.3 in the text)
Scales of Measurement
• A scale specifies the categories of measurement
• Scales ultimately dictate the statistical procedures (if any) that
can be used in processing numerical data
 Nominal Scale

• Measures data by assigning names or dividing into discrete

categories
– Boys, girls
– North of Main Street, South of Main Street
• Statistical procedures
– Mode
– Percentage
– Chi-square test
 Ordinal Scale
• Rank-order data as more/higher or less/lower
• Think in terms of greater or less than
• Elementary, high school, college, or graduate education
• Unskilled, semiskilled, or skilled labor
• Statistical procedures = median, percentile rank, Spearman’s
rank-order correlation
Interval Scale
• Equal units of measurement
• Zero point established arbitrarily
• Fahrenheit (F) and Celsius (C) scales
• Rating scales, such as surveys, assumed to fall on interval
scales
• Statistical procedures = means, standard deviations, Pearson
product moment correlations
Ratio Scale
• Equal measurement units (similar to interval scale)
• Absolute zero point (0 = total absence of the quality being
measured)
• Distance
• Ratio = can express values in terms of multiples and fractional
parts
Summary & Comparison
• Nominal scale: One object is different from another
• Ordinal scale: One object is bigger or better or more of
anything than another
• Interval scale: One object is so many units (e.g., degrees,
inches) more than another
• Ratio scale: One object is so many times as big or bright or tall
or heavy as another
Validity & Reliability of Measurement
• Validity
– the extent to which a measurement instrument measures
what it is intended to measure
• Reliability
– the consistency with which a measurement instrument
yields a certain result when the entity being measured
hasn’t changed
Validity of Measurement Instruments (1 of 2)
• Face Validity
– Is extent to which an instrument looks like it measures a
characteristic
– Relies on subjective judgment
• Content Validity
– Is extent to which a measurement instrument is a
representative sample of the content area being measured
Validity of Measurement Instruments (2 of 2)
• Criterion Validity
– The extent to which the results of an assessment correlate
with another, related measure
• Construct Validity
– The extent to which an instrument measures a
characteristic that cannot be directly observed but is
assumed to exist (such as intelligence)
 Determining Validity (1 of 2)
• Table of specifications
– The researcher constructs a two-dimensional grid listing
the specific topics and behaviors that reflect achievement
in the domain.
• Multitrait-multimethod approach
– Two or more different characteristics are each measured
using two or more different approaches. The two measures
of the same characteristic should be highly related.
• Strive for consistency with a particular conceptual framework
 Determining Validity (2 of 2)

• Conduct one or more pilot tests of your assessment strategy

• Judgment by a panel of experts
– Several experts in a particular area are asked to scrutinize
an instrument to ascertain its validity for measuring the
characteristic in question
 Reliability

• Reliability is the consistency with which a measuring

instrument yields a certain result when the entity being
measured hasn’t changed.
• Instruments designed to measure social and psychological
characteristics (insubstantial phenomena) tend to be even less
reliable than those designed to measure physical (substantial)
phenomena.
Determining the Reliability of a Measurement
Instrument (1 of 2)
• Interrater reliability
– the extent to which two or more individuals evaluating the
same product or performance give identical judgments
• Test-retest reliability
– the extent to which a single instrument yields the same
results for the same people on two different occasions
Determining the Reliability of a Measurement
Instrument (2 of 2)

• Equivalent forms reliability

– The extent to which two different versions of the same
instrument yield similar results
• Internal consistency reliability
– The extent to which all of the items within a single
instrument yield similar results
 Enhancing Reliability
• Identify specific, concrete criteria to be used in categorizing or
rating a characteristic, behavior, or other variable.
• When an assessment involves a structured paper-and-pencil or
performance-based instrument, use the instrument in a
consistent manner for all participants or other entities under
investigation.
– Quantitative researchers call this strategy standardization.
• Explicitly train raters to apply criteria in a similar manner.
 Ethical Issues (1 of 6)

• Participants must be protected from harm

– Benefits to participants must outweigh risks
– Participants should be debriefed
 Ethical Issues (2 of 6)

• Participation must be voluntary and informed

– Individuals know what they are being asked to do
– Individuals can decline without penalty
– Individuals know they can withdraw at any time without
penalty
 Ethical Issues (3 of 6)
• Participants have a right to privacy
– Data and information about participants are confidential
▪ Identifiable data should not be shared (even in class)
without written consent
– Names should be coded to ensure anonymity
 Ethical Issues (4 of 6)

• Researchers must be honest

– Data should be trustworthy
– Reports should be complete and accurate
– Contributors should be credited
 Ethical Issues (5 of 6)

• Research must be reviewed before data collection begins

– Institutions maintain an IRB (review board) and sometimes
I ACUC
▪ Scholars and researchers across disciplines
▪ Review proposals to assess risks and ensure that
participants’ rights are honored
 Ethical Issues (6 of 6)

• Researchers are expected to adhere to professional code of

conduct within their field
• Visit the homepage of your own professional organization to
learn more