Critical Thinking & Research Methods

Todays topics
Critical Thinking
Research Methods
Statistical Reasoning

Chapter 1: Thinking Critically With Psychological Science

Limits of Intuition and Common Sense

We cant rely on intuition and
common sense.
Latan & Darley (1970)
Helping behavior (woman falling
off ladder)
Intuition: she should receive help
almost all of the time.
Wrong!

Chapter 1: The Need for Psychological Science

Hindsight Bias
Hindsight bias the tendency to believe, after
learning an outcome, that we would have
foreseen it.
Results seem obvious after youve seen the
outcome!

Chapter 1: The Need for Psychological Science / Did We Know It All Along? Hindsight Bias

Hindsight Bias Example: Baratz (1983)

Gave subjects pairs of supposed psychological findings
- Group 1: told that researchers found a particular psychological finding
- Group 2: told that researchers found the OPPOSITE finding!

EXAMPLE:
- People who go to church regularly tend to have more children than
people who go to church infrequently
OR
- People who go to church infrequently tend to have more children than
people who go to church regularly

- FINDINGS:
Most students rated whichever finding they were given as
something they would have predicted.
RESULTS SEEM OBVIOUS OR PREDICTABLE AFTER YOU KNOW THE RESULT!
Chapter 1: The Need for Psychological Science / Did We Know It All Along? Hindsight Bias

Overconfidence
Intellectual arrogance - We think
we know more than we really do!
Koriat, et al (1980)
- Subjects answered two-choice questions
and indicated their confidence in their answer
- Subjects consistently overconfident by about 15%

People also overconfident about how well they drive,

how well theyve done on a test, and how quickly they
can get tasks done (Dunning et al, 2004)
Chapter 1: The Need for Psychological Science / Overconfidence

The Scientific Attitude

Curious have a passion to investigate and
understand
Skeptical challenge claims, ask for evidence
Humility accept when your own ideas are
wrong
Critical thinking thinking that doesnt blindly
accept arguments and conclusions
Chapter 1: The Need for Psychological Science / The Scientific Attitude

The Scientific Method

Scientific method an orderly, systematic process for

asking questions and observing the answers

Basic Process:
1.
Observe and describe some phenomenon
2.
Form theory to explain phenomenon and
predict results
3.
Generate a testable hypothesis
4.
Conduct research to test hypothesis
5.
Refine theory

Chapter 1: How Do Scientists Ask and Answer Questions? / The Scientific Method

Theories & Hypotheses

Scientific theories:
General principle that explains some phenomenon generally
arises from previous observations or research
produce hypotheses (testable predictions), which tell us what
research results would support our theory and what results would
disconfirm it.

Example: Meditation
Theory: Meditation reduces anxiety.
Prediction if you meditate, your level of anxiety
will decline.

Hypothesis: People who meditate at least

four hours per week should score lower on
the Taylor Manifest Anxiety Scale than similar
people who do not meditate.
Chapter 1: How Do Psychologists Ask and Answer Questions? / The Scientific Method

Theory Revision
What if we run the experiment and there seems to be no
effect of meditation on anxiety?
Change hypothesis
Maybe 8 hours of meditation is needed to see an effect

Change theory
Maybe meditation only works for people starting at a high level
of anxiety

Chapter 1: How Do Psychologists Ask and Answer Questions? / The Scientific Method

Experiment Replication
If other researchers can run our same experiment with
different subjects and get similar results, we will be more
confident in the reliability our original findings.
To make experiments more replicable, we use:
Operational Definitions statements of the procedures used to
define our research variables.

Chapter 1: How Do Psychologists Ask and Answer Questions? / The Scientific Method

Research Methods
Descriptive (observational) Methods
case studies, surveys, naturalistic
observation, correlations
Data gathered, but no variables manipulated

Experimental Methods
variables manipulated
probe cause and effect relationships
Chapter 1: How Do Psychologists Ask and Answer Questions? / The Scientific Method

The Case Study

Psychologist studies one individuals history and
behavior in great depth, hoping to reveal things about all
of us.
Pros: Large amount of info is gathered on a single
person. Suggests areas for experiments.
Cons: No guarantee that this data will relate to the
general population. Doesnt tell us WHY anything is
occurring.

Chapter 1: How Do Psychologists Ask and Answer Questions? / Description / The Case Study

The Survey
Ask large numbers of people about their attitudes,
behaviors, or opinions.
Looks at many cases in much less depth than case
studies.
Example: Public opinion Polls
Must be careful though!
Beware of wording effects
Random Sampling who are you collecting data from?

Chapter 1: How Do Psychologists Ask and Answer Questions? / Description / The Survey

Surveys: Wording Effects

Even subtle changes in wording of question can have a
major effect on findings.
Example: Censorship in media
Only 27% approve of government censorship
66% approve of more restrictions on what is shown on T.V.

When looking at results of surveys, take into account

how the questions were phrased.

Chapter 1: How Do Psychologists Ask and Answer Questions? / Description / The Survey / Wording Effects

Surveys: Representative Sample

You must have a representative sample!
You cant just ask the people around you they share
our attitudes and habits. Theyre biased.
The sample must represent the population we are
concerned with!

Chapter 1: How Do Psychologists Ask and Answer Questions? / Description / The Survey / Random Sampling

Surveys: Population
Population all cases in a group, from which samples
may be drawn for a study.
Human beings?
Americans?
UCI students?

Chapter 1: How Do Psychologists Ask and Answer Questions? / Description / The Survey / Random Sampling

Surveys: Random Sampling

To obtain a representative sample from our population,
we need a random sample.
Random sample everyone in the population has an
equal chance at inclusion in the study.
How could we randomly sample UCI students?

Chapter 1: How Do Psychologists Ask and Answer Questions? / Description / The Survey / Random Sampling

Naturalistic Observation
Observe and record the behavior of organisms in their
natural environment.
Naturalistic behavior is allowed to unfold in the setting
in which it would normally occur.
Example Main & George (1985)
Interested in the effects of physical abuse on childrens reactions
to another distressed child
Research question: Are young children who have been battered
less likely to show concern for a distressed child?
2 groups of 1-3 year old children.
All children similarly disadvantaged, but one group had been
battered.
Chapter 1: How Do Psychologists Ask and Answer Questions? / Description / Naturalistic Observation

Main & George (1985)

Trained raters, each observing an individual child,
carefully recorded the childs social behaviors.
After making many observations, researchers could
determine how children responded to other childrens
distress (crying or other signs of fright or panic).
Results: Abused children often responded to another
childs distress with fear and aggression; non-abused
children rarely responded that way.

Chapter 1: How Do Psychologists Ask and Answer Questions? / Description / Naturalistic Observation

Naturalistic Observations (Pros/Cons)

Pros:
Behavior of subjects likely to reflect their true behavior, as it takes
place in a natural setting where subjects dont realize they are
being watched.
Good source of ideas for theories.
Only way certain behaviors can be studied

Cons:
Researcher has no control over the setting. May not be able to
observe the behavior of interest.
Can only look at overt behavior, not attitudes, thoughts, or
feelings.
Chapter 1: How Do Psychologists Ask and Answer Questions? / Description / Naturalistic Observation

Correlational Studies

Allow us to compare two variables

Looks for relationships between variables
Correlation
- the extent to which two factors vary together
- Correlations occur when two variables vary together in some systematic way

Simple Example: Childrens ages and heights

These are correlated because there is a systematic relationship (low values for age
go along with low values for height, and vice versa)

positive correlation - variables tend to move in same direction (when

one variable increases, so does the other and when one decreases, so does
the other)

negative correlation - variables tend to move in opposite directions

(when one variable increases, the other decreases)

Chapter 1: How Do Psychologists Ask and Answer Questions? / Description / Correlation

Correlation Example: Drink Sales

Example: Running drink concession at
football stadium

When temperature is high, you tend to sell a

lot of soft drinks and when temperature is low,
you tend to sell relatively few soft drinks
This is a positive correlation as temperature
increases, so do soft drink sales (and vice-versa)
When temperature is low, you tend to sell a lot of coffee and when
temperature is high, you tend to sell relatively little coffee
This is a negative correlation as temperature increases, coffee sales
decrease, and when temperature decreases, coffee sales increase

Chapter 1: How Do Psychologists Ask and Answer Questions? / Description / Correlation

Scatterplots: Drink Sale Example

positive correlation

negative correlation

Chapter 1: How Do Psychologists Ask and Answer Questions? / Description / Correlation

Correlation Coefficient
Correlation coefficient
describes strength and direction
of correlation
positive correlations have positive
correlation coefficients
negative correlations have negative
correlation coefficients
variables that are completely
uncorrelated have a correlation
coefficient of 0
strongest positive correlation coefficient is +1.00 (perfect positive correlation)
strongest negative correlation coefficient is -1.00 (perfect negative correlation)
weakest possible correlation has correlation coefficient of 0
perfect correlations give perfect predictability if we know one variable we
absolutely know the other
Chapter 1: How Do Psychologists Ask and Answer Questions? / Description / Correlation

Correlation Coefficient (Examples)

Chapter 1: How Do Psychologists Ask and Answer Questions? / Description / Correlation

Correlation and Causation

Correlations help us predict values, but they dont tell us
about cause and effect.
Example: Battered children
Battered children tend to respond to other childrens
distress in an aggressive, non-supportive way.
Does that mean that battering CAUSES children to
respond aggressively to others distress?
NO!
Chapter 1: How Do Psychologists Ask and Answer Questions? / Description / Correlation / Correlation and Causation

Experimentation
Manipulate the variable of interest while
holding other factors constant
Goal see how manipulating one variable
affects some other variable.

Chapter 1: How Do Psychologists Ask and Answer Questions? / Experimentation

Example: Visual Imagery in Word Recall

Example: Does visual imagery aid memory?
Task listen to a list of words and memorize.
Subjects randomly assigned to experimental or
control group.
Experimental group uses imagery to remember word
list. Otherwise the groups are the same.
Test recall. We can tell if imagery CAUSES an
increase in recall.

Chapter 1: How Do Psychologists Ask and Answer Questions? / Experimentation

Independent and Dependent Variables

Independent variable the variable that
is manipulated in an experiment
Dependent variable the variable being
measured in an experiment (subjects
response)

Chapter 1: How Do Psychologists Ask and Answer Questions? / Experimentation / Independent and Dependent Variables

Evaluating Therapies

When evaluating therapies, factors internal to the

subject can affect results.

Peoples symptoms can improve simply from the

belief they are being treated, or by their enthusiasm
for the treatment.

Placebo effect experimental results are caused by expectations

alone.

Placebo a chemically inert substance (typically a sugar pill) that

the patient believes will help him/her.

Chapter 1: How Do Psychologists Ask and Answer Questions? / Experimentation / Random Assignment

Blind & Double-Blind Procedures

Blind procedure subjects dont know
what, if any, treatment they are receiving.
Double-blind procedure Neither
subject nor researcher knows whether
subject is in experimental or control group.

Chapter 1: How Do Psychologists Ask and Answer Questions? / Experimentation / Random Assignment

Statistical Reasoning
We use statistics to organize, summarize, and draw
inferences from the data weve gathered.
Descriptive statistics statistics used to describe or
summarize a collection of data

Chapter 1: Statistical Reasoning in Everyday Life

Organize and Describe Data

Raw data

Frequency Distribution Histogram

Ordered data

Chapter 1: Statistical Reasoning in Everyday Life / Describing Data

Measures of Central Tendency

Summarize data
Measure of central tendency is a single number
that represents a whole set of scores.
The goal is to find the single score that is most
typical or representative of the entire group.
3 measures of central tendency: mean, median,
and mode.

Chapter 1: Statistical Reasoning in Everyday Life / Describing Data / Measures of Central Tendency

Mean
LA Angels Stolen Bases 2012
(200 or more at bats)
Player
SB
Trout
49
Aybar
20
Izturis
17
Kendrick
14
Hunter
9
Pujols
8
Trumbo
4
Callaspo
4
Wells
3
Ianetta
1
Morales
0

The mean is the average of all scores

=

(total of all scores)

(number of scores)

Sum = 129
Number of scores = 11
Mean = 129/11 = 11.7

Chapter 1: Statistical Reasoning in Everyday Life / Describing Data / Measures of Central Tendency

Median
LA Angels Stolen Bases 2012
(200 or more at bats)
Player
SB
Trout
49
Aybar
20
Izturis
17
Kendrick
14
Hunter
9
Pujols
8
Trumbo
4
Callaspo
4
Wells
3
Ianetta
1
Morales
0

The median is the middle score

Median = 8

Chapter 1: Statistical Reasoning in Everyday Life / Describing Data / Measures of Central Tendency

Mode
LA Angels Stolen Bases 2012
(200 or more at bats)
Player
SB
Trout
49
Aybar
20
Izturis
17
Kendrick
14
Hunter
9
Pujols
8
Trumbo
4
Callaspo
4
Wells
3
Ianetta
1
Morales
0

The mode is the score with the greatest

frequency (occurs the most times).

Mode = 4

Chapter 1: Statistical Reasoning in Everyday Life / Describing Data / Measures of Central Tendency

Which Measure of Central Tendency is Best?

LA Angels Stolen Bases 2012
(200 or more at bats)
Player
SB
Trout
49
Aybar
20
Izturis
17
Kendrick
14
Hunter
9
Pujols
8
Trumbo
4
Callaspo
4
Wells
3
Ianetta
1
Morales
0

The mean is good unless there are extreme

scores.
The median is better when there extreme
scores in the distribution.
The mode can be included in addition to
either the mean or median or both.
Mean = 11.7
Median = 8
Mode = 4

Chapter 1: Statistical Reasoning in Everyday Life / Describing Data / Measures of Central Tendency

Measure of Variation
Tells us how similar or diverse the scores are.
Range difference between highest and lowest
score in the data set
Only a simple estimate, since extreme scores
can give a deceptively large range.
Standard Deviation uses information from
each score. A better estimate for the variation of
the whole distribution.
Chapter 1: Statistical Reasoning in Everyday Life / Describing Data / Measures of Variation

Key Terms and Concepts

hindsight bias & overconfidence
scientific attitude
scientific method
theories & hypotheses
experiment replication
operational definition
case studies
surveys
population
random sampling
naturalistic observation
correlational studies
scatterplot
positive correlation
negative correlation
correlation coefficient
experiment
random assignment

independent & dependent variables

experimental & control groups
placebo effect
blind & double-blind procedures
descriptive statistics
frequency distribution histogram
measures of central tendency (mean,
median, mode)
measures of variation (range, standard
deviation)

do not worry about:

Illusory Correlations
Perceiving Order in Random Events
Making Inferences
Frequently Asked Questions About Psychology

Chapter 1: Thinking Critically With Psychological Science / Key Terms and Concepts