Research Methodology Finl

The Research Process (Step 4 & 5) 1
TOPIC:
THEORETICAL FRAMEWORK HYPOTHESIS DEVELOPMENT
GROUP MEMBER
FATIMA TAHIR MS09MBA014
KANWAL AMIN MS09MBA016
ASMA MUNAWAR MS09MBA039
RESOURCE PERSON
DR. FOUZIA NAHEED KHAWAJA
1
Table of Contents
TOPICS PAGE #
What is Research? 6
Business Research Defined 6
Why do Research? 7
The Request for Proposal (RFP) 12
Business Methods versus Research Methodology 13
Observation 14
The Scientific Method: An Overview 16
Steps of the Scientific Process 17
What is a Literature Search 19
Purpose of a Literature Search 20
The Literature Review: 28
Finding Sources 29
Kinds of Sources 29
Converting an Idea into a Research Hypothesis 30
A Strategic Approach 31
A Search Plan 32
Synthesizing Information 33
What to look for in relation to the Literature Review 35
Where to look for reference material 37
Types of Questions 44
Types of Relationships 45
Patterns of Relationships 46
Variables 48
2
Determine the Relevant Variable 48
Types of Variables 51
Independent and Dependent Variables 52
Moderating Variables 53
Extraneous Variables 53
Intervening Variables 54
Recapitulation 55
Operational Definition of Variable 57
Threats to a Valid Measurement Procedure 64
Controlling Threats to Reliability and Validity 68
Creating a Reliable Manipulation 68
Controlling Confounding Variable 68
Control Demand Characteristics 77
Cause and Effect Relationships 78
Manipulation and Measurement 84
Theory 84
Theory and Research 84
Two Purposes of Theory 86
Concept (or Construct) 86
Theoretical Framework 88
Importance to Research 89
Hypothesis 90
Propositions and Hypotheses 90
Descriptive Hypothesis 91
Relational Hypothesis 92
3
Correlational Hypothesis 93
Explanatory(causal) Hypotheses 94
The Role of the Hypothesis 95
What is a good hypothesis? 95
Research Questions 96
Correlational Research 98
Formulating the Research Problem 102
Hypothesis Development and Testing 104
Explanation of Correlation Coefficient 105
Operationalization 107
Conceptual Variables 107
Validity and Reliability 108
Types of Validity 109
Reliability 109
Types of Reliability 110
Generalization 110
Choosing the Research Method 110
Choosing the Measurement 110
Errors in Research 110
Experimental Research 111
Aims of Experimental Research 111
Identifying the Research Problem 111
Constructing the Experiment 111
Sampling Groups to Study 111
Creating the Design 112
4
Typical Designs and Features in Experimental Design 112
Pretest-Posttest Design 113
Control Group 113
Randomized Controlled Trials 113
Between Subjects Design 114
Solomon Four-Group Design 114
Within Subject Design 114
Counterbalanced Measures Design 114
Matched Subjects Design 114
Pilot Study 115
Conducting the Experiment 117
Types of Research Designs 119
Quantitative and Qualitative Research 119
Different Research Methods 120
Ethics in Research 120
Research - Cost and Benefits-Analysis 121
Statistics Tutorial 122
Hypothesis Testing - Statistics Tutorial 123
Statistical Tests 124
Testing Hypothesis Statistically 126
Analysis of Variance 126
References 127
What is Research?
5
Research is an often-misused term, its usage in everyday language very different
from the strict scientific meaning. In the field of science, it is important to move away
from the looser meaning and use it only in its proper context. Scientific research
adheres to a set of strict protocols and long established structures. Often, we will talk
about conducting internet research or say that we are researching in the library. In
everyday language, it is perfectly correct grammatically, but in science, it gives a
misleading impression. The correct and most common term used in science is that we
are conducting a literature review.
What is Research? - The Guidelines
What is research? For a successful career in science, you must understand the
methodology behind any research and be aware of the correct protocols.
Science has developed these guidelines over many years as the benchmark for
measuring the validity of the results obtained.
Failure to follow the guidelines will prevent your findings from being accepted
and taken seriously. These protocols can vary slightly between scientific disciplines,
but all follow the same basic structure.
Business Research Defined
Business Research is defined as the systematic and objective process of generating
information for aid in making business decisions.
By William G. Zikmund
Business Research
Research information is neither intuitive nor haphazardly gathered.
Literally, research (re-search) -“search again”
Business research must be objective
Detached and impersonal rather than biased
6
It facilitates the managerial decision process for all aspects of a business.
Information Reduces Uncertainty I don’t know

if we should
“It ain’t the things we don’t know that gets us in
offer on-site
trouble. It’s the things we know that ain’t so.” child care?
Artemus Ward
Research Process
Research process deals with the ways and strategies used by researchers to
understand the world around us. It involves a dedicated system of scientific
methodology that can be used by researchers to arrive at the right conclusions.
and the researcher concludes and generalizes the findings to the real world.
Aims of Research
The general aims of research are:
Observe and Describe
Predict
Determination of the Causes
Explain
Purpose of Research - Why do we conduct research? Why is it necessary?
Elements of Research
Common scientific research elements are:
Characterization - How to understand a phenomenon
Decide what to observe about a phenomenon
How to define the research problem
How to measure the phenomenon
Hypothesis and Theory
7
The research questions before performing research
Often based on earlier research
Prediction
What answers do we expect?
Reasoning and logic on why we expect these results
Observation or Experimentation
Testing characterizations, hypothesis, theory and predictions
Understanding a phenomenon better
Drawing Conclusions
Reasoning and Logic
There is no straight answer to the best reasoning to apply to the research process.
Here are a few articles about different reasoning and logic when conducting the
scientific method.
Hypothetico-Deductive Method
Falsification
Testability
Deductive Reasoning
Inductive Reasoning and the Raven Paradox
Scientific Reasoning
Casual Reasoning
Abductive Reasoning
Post Hoc Reasoning
Defeasible Reasoning
Why do Research? In the beginning
8
We all have a great store of wisdom in our heads and rarely ask where it came
from. By and large, the knowledge upon which we base our decisions comes from
what we have been told rather than from direct experience. We usually are not too
careful about the source of our information, even when it is the basis of very
important decision. Doing our own research and experimentation allows us to make
direct examination of the information we already possess as well as providing us with
new information.
In Mark Twain’s story, “Eve’s Autobiography.” (1962), Eve examines the problem
of being newborn in a world in which knowledge in not communicated:
But studying, learning inquiring into the cause and nature and purpose of everything
we came across, were passions with us, and this research filled our days with brilliant
and absorbing interest…. Each was ambitious to beat the other in scientific discovery,
and this inventive added a spur to our friendly rivalry, and effectively protected us
against falling into idle unprofitable ways and frivolous pleasure seeking (p.71)
Some of Adam and Eve's experiments bear fruit:
Our tint memorable scientific discovery was the law that water and like fluids run
downhill, not up. It was Adam that found this out. Days and days he conducted his
experiments secretly, saying nothing to me about it; for he wanted to make perfectly
sure before he spoke… My astonishment was his triumph, his reward. He took me
from rill to rill—dozens of them—saying always, "There—you see it runs downhill—
in every case it runs downhill, never up. My theory was right; it is proven, it is
established, nothing can controvert it,"
In the present day, no child wonders to see the water run down and not up, but it
was an amazing thing then, and m hard to believe as any fact I have ever encountered.
9
In the story, Adam and Eve proceed from one discovery to another about their
environment and themselves. However, the mere fact that they are engaged in
"scientific research" does not always lead them to interpretations that are beyond in
challenge:
I scored the next great triumph for science myself: to wit, how the milk gets into
cow. Both of us had marveled over that mystery a long time. We had followed the
cows around for years—that is, in the daytime—but had never caught them drinking a
fluid of that color. And so, at last we said they undoubtedly procured it night. Then
we took turns and watched them by night. The result was the same—the puzzle
remained unsolved. . . . One night as I lay musing, and looking at the stars, a grand
idea flashed through my head, and I saw my way! . . . deep in the woods I chose a
small grassy spot and wattled it in, making a secure pen; then I enclosed a cow in it. I
milked her dry, and then left her there, a prisoner. There was nothing there to drink—
she must get milk by her secret alchemy, or stay dry.
All day I was in a fidget, and could not talk connectedly I was so preoccupied; but
Adam was busy trying to invent the multiplication table, and did not notice! Toward
sunset he had got as far as 6 times 9 are 27, and while he was drunk with the joy of his
achievement and dead to my presence and all things else, I stole away to my cow. My
hand shook so with excitement and with dread failure that for some moments I could
not get a grip on a teat; then I succeeded, and the milk came! Two gallons. Two
gallons and nothing to make it out of. I knew at once the explanation: The milk was
not taken in by the mouth; it was condensed from the atmosphere through the cow's
hair. I ran and told Adam, and his happiness was as great as mine, and his pride in me
inexpressible.
10
Presently he said, "Do you know, you have not made merely one weighty and far-
reaching contribution to science, but two!"
And that was true. By a series of experiments we had long ago arrived at the
conclusion that atmospheric air consisted of water in invisible suspension; also, that
the components of water were hydrogen and oxygen, in the proportion of two parts of
the former to one of the latter, and expressible by the symbol H20. My discovery
revealed the fact that there was still another ingredient—milk. We enlarged the
symbol to H20,M.
All the steps of scientific research are shown in Twain's story. Eve asks a question
to which she wants an answer. When systematic observation fails to provide an
answer, she then plans an experiment, devising a methodology and working out a way
to collect data. After analyzing her data (determining that she has two gallons of milk)
and, presumably, checking to see that her methods have not gone astray (namely, that
the cow had not strayed over the fence), she reports the results and her conclusion to
her colleague. Finally, she and Adam attempt to generalize her conclusion more
widely and fit it in with other information they already possess.
Adam and Eve's conclusion may be open to challenge, as are the results and
conclusions of any research. One of the greatest strengths of scientific method as a
way of gaining knowledge is that this knowledge is always subject to further testing.
Eve's experiment should be conducted on other cows, the cow she watched might be
unique. The experiment also should be continued for a longer period of time; there
might be distressing effects on milk production, not to mention the cow, if Eve
continued to withhold fluids. If she thinks hair in important in the process, she might
try shaving the cow. She might also try confining the cow to areas where no grass
grows, which not affect should milk production if her air-suspension theory is correct.
11
A single experiment or research project rarely provides final answers; it simply helps
eliminate unlikely explanation. Scientists are not only interested in testing what they
have been told, they also must be able to pinpoint weaknesses in applications of
scientific method and weight the truth of any conclusions accordingly.
What is wrong with believing what we are told?
After all, we're not Adam and Eve. In fact, according to the anthropologist, what
distinguishes us from monkeys is that we have culture which can be handed down
from generation to generation. This culture consists of physical objects and
knowledge communicated both orally and in writing. If we want to know how the
milk gets into the cow, all we need do is ask someone or look it up in a book. Yet if
you were to pursue this question with someone who “really knows,” you would be
likely to find out that this is an area of current research. The improvement of milk
production through feeding, genetics, and milking practices is still going on. The
answer you get this year may not be the same as the one you would get ten years from
now, and it certainly won’t be the same as the answer you would have got in 1900.
Is the solution, then, to believe nothing of what we are told and to discover everything
through direct experience? The problems would be overwhelming there simply isn’t
enough time to gather all information firsthand. Besides, a small portion of direct
experience by itself can be misleading, as we saw with Eve's work. What we need is a
means of estimating how much we should trust a particular piece of information. In
addition, we need a process by which we can generate new information when we feel
the need for a firmer base for our decisions.
Scientific method is a set of procedures that allows us to generate information of
high quality and to check the worth of what we have been told by others. Scientific
research does not provide final answers, but it increases the probability that the
12
information obtained does reflect reality. One hallmark of scientific method is that it
is self-correcting. Properly applied, scientific procedures will expose weaknesses in
information gathered in earlier research and suggests new research to deal with these
weaknesses. Scientists "believe" something only on a provisional basis. They are
aware that any particular belief may have to be altered because of future research
findings. Behaving like a scientist means fighting a continuing battle against the in-
clination to be dogmatic about what we "know." It is especially important for those of
us hoping to work with other people to ride herd on our beliefs. Whether we are
psychologists, sociologists, educators, anthropologists, health-service practitioners,
politicians, or business managers, we are constantly making decisions that affect our
fellow human beings. Our work often costs other people money, takes up their time,
Research Methodology
Research Methodology – In order to meet our objectives, we will need to use a
number of research methods. For instance, when gathering research for our first
objective, we will need to create a questionnaire and come up with questions to ask
the store manager in the interview with him. These are both primary methods of
research, however, we will be using forms of secondary research also. Primary and
secondary research can be split up into two groups, quantitative and qualitative
research. Methods of quantitative secondary research include official statistics, and
controlled experiments. In order to solve the problem of low motivation at the store,
we will need to practice a number of research methods. A particular method of
research we will be using is primary research. Primary research is collecting data that
does not already exist. Another method of research is secondary research, this is using
data that already exists, such as books, newspapers, the internet and the other methods
mentioned earlier. Between these two methods we aim to collect data that is more
13
qualitative than quantitative, as we believe a subject such as motivation requires
peoples’ opinions and much more elaboration than what closed-ended questionnaires
and other quantitative methods of research could possibly offer. To meet our first
objective we will need to collect primary research in the form of questionnaires and
interview. This in turn would help us decide whether the lack of motivation affects the
day to day running of the business, whether it is in the production area or whether
quality is more effected. This will in turn able our to complete our fifth objective of
collecting significant secondary research, which will allow us to analyze and come up
with some quality solutions for the problem of low motivation at Sainsbury’s Local in
Wylde Green. In order to complete our final objective of offering conclusions to our
research, we will use both primary and secondary research.
The Request for Proposal (RFP)
The request for proposal (RFP) is part of a formal process of competitively
tendering and hiring a research supplier. If the process is undertaken by a public
sector organization or large corporation, the process can be extremely strict with set
rules regarding communication between client and potential suppliers, the exact time
when the proposal must be submitted, the number of copies to be provided, etc.
Proposals that required thousands of hours of preparation have been refused for being
one minute late (see this article)!
The RFP usually sets out the objectives or client's information requirements and
requests that the proposal submitted by the potential supplier include:
1. A detailed research methodology with justification for the approach or approaches
proposed;
2. Phasing or realistic timelines for carrying out the research;
3. A detailed quotation by phase or task as well as per diem rates and time spent for
14
each researcher participating in the execution of the work;
4. The qualifications of each participating researcher and a summary of other projects
each person has been involved in to demonstrate past experience and expertise
The client should provide the potential suppliers with the criteria for selection and the
relative weight assigned to each one, to assist suppliers in understanding where trade-
offs might need to be made between available budget and importance. These criteria
also allow the supplier to ensure that all areas deemed important by the client have
been addressed as part of the proposal.
At times, clients ask a short-listed number of suppliers to present their proposed
methodology during an interview, which allows for probing by the client but also
discussion as to the advantages and disadvantages associated with the research design
that is proposed.
Business Methods versus Research Methodology
Many authors use these terms interchangeably, but there is a correct way of using
them. As students of “Research Methods”, we must know the difference. What is it?
Textbooks treat this differently but research “methods ” usually refers to specific
activities designed to generate data (e.g questionnaires, interviews, focus groups,
observation) and research “methodology” is more about your attitude to and your
understanding of research and the strategy you choose to answer research questions.
This idea is based on the views of author Dr. Sue Greener (2003 Pg 10)
Observation
Observation is a primary method of collecting data by human, mechanical,
electrical or electronic means. The researcher may or may not...
Deductive Reasoning
15
The logical process of deriving a conclusion from a known premise or something
known to be true.
We know that all managers are human beings. If we also know that John
Smith is a manager, then we can deduce that John Smith is a human being.
Inductive Reasoning
The logical process of establishing a general proposition on the basis of observation
of particular facts.
All managers that have ever been seen are human beings; therefore all
managers are human beings.
The Scientific Method: An Overview
Assess Formulate Statement

Design
relevant concepts & of
research
existing Propositions Hypotheses
knowledge
Acquire Analyze & Provide

empirical evaluate explanation-
data data state new
problem
Key Concepts of the Scientific Method
Steps of the Scientific Method - The scientific method has a similar structure to
an hourglass - starting from general questions, narrowing down to focus on one
specific aspect, then designing research where we can observe and analyze this aspect.
At last, the hourglass widens
There are several important aspects to research methodology. This is a summary
of the key concepts in scientific research and an attempt to erase some common
misconceptions in science.
16
Steps of the scientific method are shaped like an hourglass - starting from general
questions, narrowing down to focus on one specific aspect and designing research
where we can observe and analyze this aspect. At last, we conclude and generalize to
the real world.
Steps of the Scientific Process
1) Setting a Goal
Research in all disciplines and subjects, not just science, must begin with a clearly
defined goal. This usually, but not always, takes the form of a hypothesis.
For example, an anthropological study may not have a specific hypothesis or
principle, but does have a specific goal, in studying the culture of a certain people and
trying to understand and interpret their behavior.
The whole study is designed around this clearly defined goal, and it should
address a unique issue, building upon previous research and scientifically accepted
fundamentals. Whilst nothing in science can be regarded as truth, basic assumptions
are made at all stages of the research, building upon widely accepted knowledge.
2) Interpretation of the Results
Research does require some interpretation and extrapolation of results.
In scientific research, there is always some kind of connection between data
(information gathered) and why the scientist think that the data looks as it does. Often
the researcher looks at the data gathered, and then comes to a conclusion of why the
data looks like it does.
A history paper, for example, which just reorganizes facts and makes no
commentary on the results, is not research but a review.
17
If you think of it this way, somebody writing a school textbook is not performing
research and is offering no new insights. They are merely documenting pre-existing
data into a new format.
If the same writer interjects their personal opinion and tries to prove or disprove a
hypothesis, then they are moving into the area of genuine research. Science tends to
use experimentation to study and interpret a specific hypothesis or question, allowing
a gradual accumulation of knowledge that slowly becomes a basic assumption.
3) Replication and Gradual Accumulation
For any study, there must be a clear procedure so that the experiment can be
replicated and the results verified. Again, there is a bit of a grey area for observation-
based research, as is found in anthropology, behavioral biology and social science, but
they still fit most of the other criteria.
Planning and designing the experimental method, is an important part of the
project and should revolve around answering specific predictions and questions. This
will allow an exact duplication and verification by independent researchers, ensuring
that the results are accepted as real.
Most scientific research looks at an area and breaks it down into easily tested
pieces. The gradual experimentation upon these individual pieces will allow the larger
questions to be approached and answered, breaking down a large and seemingly
insurmountable problem, into manageable chunks.
True research never gives a definitive answer but encourages more research in
another direction. Even if a hypothesis is disproved, that will give an answer and
generate new ideas, as it is refined and developed.
Research is cyclical, with the results generated leading to new areas or a
refinement of the original process.
18
4) Conclusion
The term, research, is much stricter in science than in everyday life. It revolves
around using the scientific method to generate hypotheses and provide analyzable
results. All scientific research has a goal and ultimate aim, repeated and refined
experimentation gradually reaching an answer. These results are a way of gradually
uncovering truths and finding out about the processes that drive the universe around
us. Only by having a rigid structure to experimentation, can results be verified as
acceptable contributions to science.
Some other areas, such as history and economics, also perform true research, but
tend to have their own structures in place for generating solid results. They also
contribute to human knowledge but with different processes and systems.
By Martyn Shuttleworth (2008).
What is a Literature Search
The literature search is a very significant step in the research process. The basic
stages in a typical research project are:
i) identify your topic of interest,
ii) perform a literature review,
iii) generate related questions,
iv) state your unsolved problem or hypothesis,
v) find or develop a solution, and
vi) document your results.
We are often trained to think, approach, and describe tasks sequentially, but anyone
who has earned an advanced degree in engineering (or any other field) can tell you
19
that a sequential checklist of the process is too simplistic. Reference 1 explains “...
how researchers must work at different stages of their project simultaneously
[1].” The authors describe how overlapping the stages can help you understand the
problem, manage complexity, and motivate changes in both earlier and later stages.
The important concept to grasp is that the literature search is not just one distinct step
in a research task. It is both a step and an iterative feedback loop. Defining an
unsolved problem determines what kind of literature search is appropriate, and
performing a literature search helps define an unsolved problem. Because of this
complexity, it is not surprising that many beginning researchers have difficulty.
Figure 1 depicts the research process and some of the complex inter-relationships,
including those of the literature search and documentation efforts.
FIGURE 1. Flow Diagram of Research Process.
Purpose of a Literature Search
An introduction chapter in a research paper generally includes information
gleaned from a thorough literature search. A literature search serves three important
functions [2]. Reference 2 states that the literature review gives your reader
background information regarding your own research, demonstrates your familiarity
with research in your field, and shows how your work contributes one more piece in
the puzzle of expanding the knowledge base in your field. The important idea to
convey is that you really understand what others in your field have accomplished and
20
how your work differs from the works of others [3]. In a literature review, you
demonstrate your understanding of the relevant works of others and your ability to
summarize this information for the convenience of your readers. This sets the stage
for you to describe what your research contribution is going to be.
Research does not exist in isolation. Each research study is part of existing body
of knowledge, building on the foundation of past research and expanding that
foundation for future research. This topic explains how current knowledge grows,
with each new piece of information growing out of existing body of previous
knowledge. As we discussed the literature and develop an idea for a research study,
keep in mind that your study should be a logical extension of past research.
Ultimately, your goal in conducting a literature search is to find set of published
research reports that define the current state of knowledge in an area , and to identify
a gap in that knowledge base that your study will attempt to fill. Eventually, you will
complete your research study and write your own research report. The research report
begins with an introduction that summarizes past research (from your literature
21
search) and provides a logical justification for your study. Although we discuss the
task of writing a research report later (in Chapter 15), the topic is introduced now as a
means of focusing your literature search. Figure 2.2 presents the first paragraph of a
journal article (Schwartz, Dodge, & Coie, 1993) as an example of the use of a
literature review to introduce a topic area and provide a logical justification for a new
study. The paragraph can be condensed into a simple, logical argument:
1. A small minority of children consistently seem to be victimized of bullied by their
peers,
2. Although other research has looked at the characteristics of the victim children,
little has been done to examine their social behaviors prior to becoming victims. (Are
these kids doing something that actually helps them to become victims?)
3. Therefore, we need to examine children’s behaviors before they are identified as
victims. The goal is to determine whether some children are destined to become
victims because they exhibit behaviors that may cause their peers to view them as
easy targets for bullying.
Although we have not described the research study, you should be able to predict
the purpose of the study and should have some idea of what was done. Notice that the
background literature is used to construct a logical argument that This study is an
investigation of the social behavior processes by which children come to be
chronically victimized by their peers. There is substantial evidence that a small
minority of children are consistently targeted for victimization by their peers (for a
review, see Smith, 1991). Researchers have suggested that these chronic victims are at
high risk for later maladjustment (e.g., Olweus, in press). Accordingly, investigators
have devoted considerable effort to identifying the correlates of peer victimization
(e.g., Bjorkqvisf, Ekman, & Lagerspetz, 1982; Lagerspetz Bjorkqvist, Berts, & King,
22
1982; Olweus, 1978) and designing effective prevention programs (e.g., Elliot, 1991;
Olweus, 1991; Smith & Thompson 1991). Researchers have not, however,
systematically identified the behavioral antecedents of chronic victimization. As a
result, little is known about the social behaviors that precede and potentially
contribute to the emergence of chronic peer victimization. Our limited understanding
of these behavioral processes is unfortunate because detailed data on the behavior
patterns that precede the emergence of chronic peer victimization would facilitate the
development of appropriate interventions and greatly enhance current understanding
of the mechanisms underlying bully/victim problems in childhood leads the reader
directly into the proposed study. The purpose of your literature review is to provide
the elements needed for an introduction to your own research study. Specifically, you
need to find a set of research articles that can be organized into a logical argument
supporting and justifying the research you propose to do.
The Growth of Research
Throughout this chapter we repeat the notion that each research study builds on
previous knowledge and attempts to expand that knowledge base. With this thought in
mind, it is possible to represent the existing knowledge base (the literature) as a tree-
like structure that is continuously growing over time. When you begin a literature
search, you will enter this tree and find your way along the branches. Your goal in
conducting the search is twofold. First, you must work your way to the very tips of
the highest branches and find a cluster of the most recent research studies. Your study
will form a new branch coming out of this cluster. Second, you must search
backward, down the tree, to identify the historically significant foundations of your
work. You probably will find that most of the current research studies in an area will
cite the same "classic" studies as their foundations. These classics usually will provide
23
a broader perspective for your work and will help you understand and explain the
significance of your study as it relates to the more general tree of knowledge.
The tree metaphor is only a conceptual guide to help you visualize the process
and the goals of a literature search—-the concept of a tree greatly oversimplifies the
process. For example many good research studies involve establishment a connection
between two previously unrelated branches of research. Nonetheless, the tree
metaphor should help direct your literature search activities. You may, for example,
find yourself with a cluster of recent articles that seem to be a dead end, offering no
prospect for developing new research. If this happens, you can simply work back
down the tree to an earlier branching point and branch off in a new direction without
completely abandoning your original research topic.
Conducting a Literature Search
Starting Points
Let’s assume that you’re starting your literature search with only a general idea
for a research topic. Your purpose, therefore, is to narrow down your general idea to a
specific research question and to find all the published information necessary to
document and support that question. As you will see, there are many different ways to
begin a search of the literature, in this section we identify several different starting
points and provide some suggestions to help you find one.
One of the best places to start is with a recently published secondary source – for
example, a textbook in a content area appropriate for your idea (perhaps a
developmental psychology or social psychology textbook). Use the chapter headings
and subheadings in the text to help focus your search on a more narrowly defined
area, in addition, make notes of the following item, each of which can serve as an
24
excellent starting point when you begin to search for primary sources (journal articles)
relevant to your topic:
Key words: Make a list of the correct terms of key words used to identify and
describe the variables in the study and the characteristics of the subjects. Researchers
often develop a specific set of terms to describe a topic area. It is much easier to
locate related research articles if you use the correct terms. For example, you may
have trouble finding articles on “duration of memory” unless you use the accepted
term, retention interval.
Author Names: Commonly a small group of individual researchers is responsible
for much of the work being done in a specific area. If you repeatedly encounter the
same names, make a note of these individuals as the current leading researchers in the
area.
As you develop your list of key word terms and author names, keep in mind that
any single secondary source is necessarily incomplete and probably selective. Thus, it
is wise to repeat the list-making process with two or three different sources, and then
combine your lists. When you finish, you should have an excellent set of leads to help
you move into the primary source literature.
Using Online Databases
Although there are thousands of research articles published every year, many
tools are available to help you search through the publications to find the few that are
directly relevant to your research topic. In the recent past, the primary tool was a
publication called Abstracts, which consisted of bound volumes containing:
A brief summary of every article, chapter, or book published during the year and
information about where to find the original publication. These summaries are the
actual abstracts.
25
A subject index that cross-referenced every publication according to key words or
subjects relevant to the publication.
An author index that cross-referenced every publication according to author.
Thus, one could use Abstracts to look up everything that was listed on a particular
topic and/or everything that was published by a particular author in any year. The
same principle of cross-referencing publications by subject and by author is still used
today, but now most of the cross-referencing tools exist as computer databases. There
are many different databases, with each one focusing on an individual topic area (like
psychology, chemistry, or criminal justice) The databases provide the same basic
information that was available in the bound volumes of Abstracts, but the computer
provides much faster and easier access to the information.
Finding and using background Literature
Once you have settled on a general idea for a research study, the next step is to go
to the library to gather background information on the topic you have identified. In
addition to gaining general knowledge about your topic area, your goals are to
determine the current state of knowledge and to become familiar with current research
—in particular, to find a specific research question. Notice that we said "find" a
question rather than "make up" or "create" one. Once you are familiar with what is
currently known and what is currently being done in a research area, your task is
simply to extend the current research one more step. Sometimes this requires a bit of
logic, in which you combine two or more established facts to reach a new conclusion
or prediction. Often the authors of a research report literally will give you ideas for
new research. It is very common for researchers to conclude a discussion of their
results with suggestions for future research. You are welcome to turn one of these
suggestions into a research question. In Section 2.5 we provide additional hints for
26
finding research idea For now, do not try to impose your own preconceived idea onto
the literature Instead, let the literature lead you to a new idea.
In most college or university libraries, the books devoted to research will occupy at
least 100 feet of shelves. The research journals probably will, fill even more space.
This mass of published information is referred to as “the literature." Your job is to
search the literature to find a handful of items that are directly relevant to your
research idea. This may at first appear to be an overwhelming task; fortunately,
however, the literature is filled with useful aids to guide your search. Specifically, all
the individual publications are interconnected by cross-referencing, and there are
many summary guides providing overviews that can send you directly to specific
topic areas. By following the guides and tracing the interconnections, it is possible to
conduct a successful literature search without undue pain and suffering.
Primary and Secondary Sources
Before we discuss the actual process of a literature search, there are a few terms
you should know. Individual items in the literature can be classified into two broad
categories: primary sources and secondary sources. A primary source is a first-hand
report in which the authors describe their own observations. Typically, a primary
source is a research report, published in a scientific journal or periodical, in which the
authors describe their own research study, including why the research was done, how
the study was conducted, what results were found, and how those results were
interpreted. In contrast, a secondary source is a secondhand report in which the
authors discuss someone else's observations. Some examples of secondary sources
include: (1) books and textbooks in which the author describes and summarizes past
research; (2) the introductory section of research reports, in which previous research
27
is presented as a foundation for the current study; and (3) newspaper and magazine
articles that report on previous research.
A Primary Source is a firsthand report of observations or research results that is
written b the individual(s) who actually conducted the research and made the
observations.
A Secondary Source is a description or summary of another person’s work. A
secondary source is written by someone who did not participate in the research or
observations being discussed.
Notice that the principal distinction between a primary source and secondary one is
firsthand versus secondhand reporting of research results. Students often confuse this
distinction with the notion that anything published in a journal or periodical is
automatically a primary source and that all other kinds of publications are secondary
sources. This assumption is incorrect on several levels. The following are also
possible:
The journal article is not a primary source. Instead, the article may be a review
of other work, a theoretical article that attempts to explain or establish relationships
between several previous studies, or a historical summary of none is a firsthand report
of research results.
A book or book chapter is primary sources. Occasionally an individual or a
group of researchers will publish an edited volume that presents a series of
interrelated research studies. Each chapter is written by the individual(s) who actually
conducted the research and is therefore a primary source.
A journal article is firsthand report of research results, yet sections of the
article are actually secondary sources. Specifically, most research reports begin with
an introductory section that reviews current research in the area and forms the
28
foundation of the study that is being reported. This review of current research is
secondary because the authors describe research conducted by others. Remember, to
qualify as primary source, the authors must describe their own research studies and
results.
Both primary and secondary sources play important roles in the literature search
process. Secondary sources can provide concise summaries of past research. A
textbook, for example, will often summarize 10 years of research, citing several
important studies, in a few paragraphs. Individual research reports that fill 10 to 15
pages in journals are often summarized in one or two sentences, thus, secondary
sources can save your hours of library research. However, you should be constantly
aware that secondary sources are always incomplete and can be biased or simply
inaccurate. In secondary sources, the author has selected only bits and pieces of the
original study; the selected parts may have been taken out of context and reshaped to
fit a theme quite different from what the original authors intended. In general,
secondary sources tell only part of the truth and may, in fact, distort the truth. To
obtain complete and accurate information, it is essential to consult primary sources.
Reading primary sources, however, can be a tedious process, as primary sources are
typically long, detailed reports focusing on a narrowly defined topic. Therefore, plan
to use secondary sources to gain an overview and identify a few specific primary
sources.
The Literature Review:
The literature review is where you will present others writings in a systematic and
thorough way. This allows you as an author to build your conceptual analysis. As the
literature review is a description and critical analysis of what others have written on
29
your chosen subject matter before you, it allows you to narrow other authors work and
make it relevant and concise in relation to your subject.
It would be prudent to present other authors work in a systematic way i.e. present
each critical subject area in a defined chapter, this makes it easier and more
manageable to handle various subject themes and modalities.
The literature review will be referred to many times in a dissertation or project.
When we draw conclusions and make recommendations, often we will make them
based on work done before us by other authors and we will frame their work in our
overall recommendations.
Counseling research by its nature can be wordy and literature reviews can be quite
significant in the overall compilation of the project document. Organization and the
eloquent ‘stitching’ together of themes and subject areas is vital if a literature review
is to have substance and if it is to present a literate/theoretical framework to your
research topic and ultimately to the end user: your reader or academic professionals.
Barker et al, (1994) defines four predominant reasons for a Literature Review
1. To assess how well developed the literature is, what kinds of gaps there are in it
and whether there has been sufficient preliminary descriptive research to define the
phenomena of interest.
2. To see and establish how far the available literature answers your research
questions. How can your proposed study add to the current study?
3. To help you to formulate your research questions in the light of theory or previous
research done.
4. To assist with measurement and design issues, to develop measurement methods, to
assess strengths and weaknesses of those used previously.
Finding Sources
30
Libraries are one obvious resource for a student performing a literature search, but
there are many others as well, including the internet. Regardless of the sources you
use, keep a bibliographic trail [1]. Track titles, authors, publication information, page
numbers, and possibly library call numbers (LCN), International Standard Book
Numbers (ISBN), or International Standard Serial Numbers (ISSN). When searching
through cyberspace, also note on-line addresses and other pertinent information so
that a search can be duplicated if necessary. Also be aware that some online
information changes daily – one day it is there, the next day it is gone. You should
print it out immediately, if possible, and note the source, the “navigation” route, and
the search engine you used to get to that source. These tracking habits can help you
avoid duplication of effort and speed the process of obtaining permission (to use
the work of others) when needed. The time required to relocate and require a source is
also reduced. You will also be constructing your reference chapter as you work.
Retracing these efforts by memory is very risky, so it is better if you take notes at
every step.
Kinds of Sources
Sources are generally described as primary, secondary, or tertiary. Primary.
Primary sources are “materials that you are directly writing about, the raw
materials of your own research.”
Secondary . Secondary sources are “books and articles in which other researchers
report the results of their research based on (their) primary data or sources .”
Tertiary . Tertiary sources are “books and articles based on secondary sources, on
the research of others [1].” Tertiary sources synthesize and explain the work of others
and might be useful early in your research, but they are generally weak support for
your own arguments [1].
31
Primary. Examples of primary sources are data sets, computer runs, computer
programs, scale models, drawings, and engineering notebooks. A well-kept
engineering notebook can provide valuable information for later documentation of test
conditions and assumptions, materials used, observations as well as measurements,
and unusual occurrences that prompted further testing.
Secondary. Examples of secondary sources include conferences, proceedings,
journals, and books. Journal articles are often the most current source of information
on a topic of study that is new or subject to rapid change. Lists of references at the
end of each journal article can provide leads to further sources. Engineering journals
are typically field-specific. For a selected list of current journals in agricultural,
chemical, civil, computer, electrical, environmental, industrial, and mechanical
engineering, ask at your facility or university library for specific guides.
Tertiary. Examples of tertiary sources include dictionaries, encyclopedias, guides,
and handbooks. “Dictionaries and encyclopedias are excellent starting points
for research. They can provide general background information to help narrow or
broaden the focus of a topic, define unfamiliar terms, and offer bibliographies of other
sources. Some works include an index, which will provide excellent access to a
subject [4].” Guides and handbooks cover topics such as tables, formulas, engineering
fundamentals, measures and units of conversion, mathematics, statistics, and
numerical calculations; these sources are especially useful during the writing phase of
your research [4].
Finding an Idea for a New Research Study
Once you have located a set of recent and relevant articles, the final step is to use
these research reports as the foundation for your research idea (see Chapter 1, Step 1
of the research process). Earlier we called this task "finding a research idea." When
32
you are familiar with the current research in an area, the idea for the next study
involves simply extending the current research one more step. However, discovering
this next step may not be as simple as we have implied, and so we list here a few
suggestions:
Describe the tree ways identified in the text to find a new research idea.
Converting an idea into a Research Hypothesis
Typically, a research idea involves a general statement about the relationship
between two variables, For example:
If people use visual images while studying new material, it will improve their
memory for the material. (Memory if related to using images.)
Although providing motivation to people generally improves their performance,
too much motivation may create stress and actually lower performance. (Performance
is related to motivation.)
To evaluate these ideas in an empirical research study, the idea must be
transformed into a specific, concrete research hypothesis that can be tested by direct
observation. This transformation usually involves specifying how each of the
individual variables will be measured and what individuals will be needed to
participate in the study (men, women, children, laboratory rats). Selecting participants
and measuring variables are discussed in the following chapters, but for now we
provide a brief example of what we mean by a specific, concrete hypothesis.
The first of the two research ideas we have proposed states that memory it related
to using visual images. Although this idea could be examined with nearly any group
of human beings, it is probably most convenient to use college students as participants
in the study. Similarly, the concept of memory can be defined in a variety of different
ways, but for this study we choose to measure the number of words correctly recalled
33
from a list of 40 nouns that each participant studies for a period of exactly 2 minutes.
Finally, we can define the idea of "using Imagery as “forming a mental image of the
object represented by a word.” (For example, forming a mental image of a horse when
you see the word horse.) With these definitions in place, our research hypothesis
becomes:
College students who are instructed to form mental images while studying a list of
40 words for 2 minutes will recall more words (on average) than college students who
study the same words for 2 minutes but are not given instructions to form mental
images.
Notice that the research hypothesis provides a very specific procedure for testing
the research idea. Also note that the same research idea (that memory is related to
using images) could produce a variety of different research hypotheses. For example,
we could have tested 10-years-old children instead of college student; we could have
used a set of 20 items( instead of 40; and we could have presented on a screen and
another group a series of words presented on a screen and another group a series of
pictures of the same items, the research hypothesis then becomes:
Ten-year-old children who view pictures of 20 items (for example, a table, a
horse, a tree) will recall more items, on average , than 10-year-old children who view
a series of word resenting the same 20 items for example , TABLE, HORSE, TREE)
In general, there are many different ways to convert a research idea into a specific
research hypothesis, the method you select depends on a variety of factors, including
the set of individuals you want to study and the measurement techniques that are
available. However, each of the many possible hypotheses should provide a direct test
of the basic research idea.
Convert your Research Idea into a specific Research Hypothesis
34
Before a research idea can be evaluated, it must be transformed into a research
hypothesis. A hypothesis is a statement about the relationship between two (or more)
variables.
A hypothesis is a statement describing the relationship between two variables.
Although it is possible to form a hypothesis from logic of from pure speculation, a
good hypothesis usually is founded on established theories or develop[ed from the
results of previous research. Specifically, a good hypothesis should be the logical
conclusion of a logical argument. Consider the following example:
Premise 1: Academic success is highly valued and respected in society (at least by
parents and teachers).
Premise 2: Being valued and respected by others contributes to high self-esteem.
Conclusion or hypothesis: Higher levels of academic success are related to higher
levels of self-esteem.
In this argument, we will assume that the two premise statements are "facts" or
research results that have been demonstrated and reported in the scientific literature.
Typically, these facts would be obtained from extensive library research. Library
research will acquaint you with the relevant knowledge that already exists—what
other researchers have done and found. By knowing the basic facts, theories,
predictions, and methods that make up the knowledge base for a specific topic area,
you gain a clearer picture of the variables being studied and their relationships. The
logical argument provides a rationale or justification for your research hypothesis and
establishes a connection between your research and the research results obtained by
others.
In addition to being logical, a good hypothesis must be testable; that is, it must be
possible to make observations that will either support the hypothesis or refute it. In
35
particular, the hypothesis must involve real situations, real events, and real
individuals. You cannot test a hypothesis that refers to imaginary events or make-
believe situations. For example, you might speculate about what might happen if the
heat from the sun were gradually to increase over the next 25 years, or you could
debate what might have happened if JFK had not been assassinated. However, neither
of these two propositions leads to a testable hypothesis. The propositions cannot be
observed and therefore are inappropriate as scientific hypotheses.
A second characteristic of a testable hypothesis is that it must make a positive
statement about the existence of something, usually the existence of a relationship, a
difference, or a treatment effect.
A Strategic Approach
As a beginning researcher, two of the most common problems you might
encounter are: i) not knowing where to find sources, and ii) once sources are located,
not knowing how to sift or sort through an excess of information to determine what is
useful to you. The following paragraphs elaborate on these two problems and describe
a general plan that can help you overcome these difficulties during your
literature review process.
Getting Started
A lot of time and frustration can be saved by understanding the process and the
system. “Doing research consists of two basic tasks: brainstorming and researching
itself. You need to know what information you will need, and you need to know
where to get your hands on it [5].” “Developing the working bibliography requires
knowledge of library resources, both printed and electronic, and the use of
reference systems to locate sources. During early phases of research, developing a
tracking method to maintain a complete record of all of the bibliographic information
36
from your sources is advised, since you do not yet know whether it will be cited in
your thesis [6].” It is also advisable for you to construct a list of experts in your field.
Such a list can be invaluable. An oral interview with an expert can produce valuable
insight and possibly save you a lot of time and effort. You might try sending e-mail
messages to well-known authors or lecturers. If you notice that several recent
dissertations are written by students at a particular university, you might contact the
authors or the authors’ advisors. Given the opportunity to ask, one important question
is: “If I were to pick up where you (or your student) left off, what advice would you
give?”
A Search Plan
A recommended search strategy is to consider how a topic progresses through the
research documentation life-cycle. One approach is to look for books that are
currently accepted reference texts in your research area and find out who has
cited them in recent years. Another approach is to look for original (seminal) reports,
papers, or theses written by known experts and see who has cited them in recent
years. For these two approaches, Science Citations Index (often available on CD-
ROM) becomes a very important source. Implementing this strategy can help you
develop a tree-like diagram or network showing not only relevant research, but
possibly even how the works of others are interrelated. By locating and reading at
least the abstracts of these papers, you can begin to categorize the available papers by
topic and by the authors’ technical interests in a topic. Then look at the reference lists
for the newer papers. This approach is especially useful because some databases
might categorize differently or use different descriptors as key words. This approach
should at least get you started on a good literature search. The above advice sounds
easier than it really is. Even experienced researchers often feel overwhelmed by
37
“information overload.” As technological electronic storage and retrieval capabilities
continue to advance, we will all struggle with this problem. That is why it becomes
very important to have a plan in mind when conducting a literature review, and to
have a conceptual ”blueprint” understanding of the resources available. Otherwise,
you can easily become lost in your search or feel as if you are going around in circles.
What To Do If You Have Too Few Sources
If you find little or no information, your topic might be too narrow. If this is the
case, it is a good idea to see what topic headings or terms any newly found
information was cataloged under, so that you can use those key words in other
database searches. If your search still yields little information, get help from library
personnel and experts in your field of research. In particular, ask for advice about
ways to expand your topic so that searches will produce some useful information.
What To Do If You Have Too Many Sources
If, on the other hand, you find volumes of information, then you need a plan to
scale your search down to a manageable amount. Specific aspects of a field of study
are often listed in annotated bibliographies. Journals specific to a field are good
sources. Experts in your field of interest can suggest where many of your important
sources can be found. It also helps to understand and categorize your sources so that
you know what kind of information you have. Then you can draw on what you need
without being overwhelmed by material that is interesting and related, but not
necessarily critical to your own research.
Synthesizing Information
Once information is located, the next step is to summarize it into a coherent
literature review section for your document. You must analyze accurately and
critically. It takes a lot of skimming of books and articles to identify which sources
38
are useful to you and in what context they are useful. Your goal at this point is to
“present an overview of what your source offers; its topic, research problem,
resolution, and the outlines of its argument [1].” The abstract, introduction, and
conclusion sections of each source document should contain the necessary
information for you to write a good summary paragraph. It can be very
counterproductive to try to read everything in detail. At first, it is more important to
categorize and understand what sources you have and what might still be missing.
Summarizing and Note-Taking
Both references [7] and [1] have good advice about notetaking. Reference [7]
states that the challenge is to condense others’ work without distorting it. Their seven-
step process includes: i) do not write everything down, ii) create your own shortcuts
and shorthand, iii) use numbers for numerical terms, iv) leave out vowels when you
can, v) record all vital names, dates, and definitions, vi) mark items that need further
examination, and vii) check accuracy before returning or filing the source.
Reference [1] recommends writing and summarizing as you find sources and has a
three-page section called “Quick Tip: Speedy Reading.” It recommends a five-step
process:
i) become familiar with the geography of the source,
ii) locate the point of the argument,
iii) identify key subpoints,
iv) identify key themes, and
v) skim paragraphs. The book suggests that not all five steps are needed all the time.
The main point is speed and efficiency, and focusing on material relevant to your
own research without spending time on material that is at best only marginally related.
39
While taking notes and preparing condensed summaries of the work of others, you
must be ever mindful of the requirement to eventually cite all “borrowed” work in
your final paper. All of the sources mentioned contain discussions about direct
quotations, summarizing, and plagiarism. Readers and researchers are advised to
locate and carefully read about these topics from any available source in order to
avoid trouble while writing.
A recommended beginning search plan has been described, especially for
engineering researchers. Understanding how the literature search dovetails with other
steps in the research process helps form a good plan. Guidance has been given for
finding sources, for determining if your topic is too narrow, and for scaling your
search to a manageable amount. Useful tips have been given to help you track and
summarize information so that it becomes useful for your research purposes.
Hopefully, this report will help you get a “jump-start” on a good literature review and
contribute to the successful conclusion of your research project.
What to look for in relation to the Literature Review
Knowing what to look for is one of the chief problems when building you
literature review. Knowing what to look for, who are the authors?, What is the
relevance of published research to your research questions? - Are all questions posed
at this stage.
Jankowicz (1999) outlines two distinct approaches at this stage and they are,
1. Identifying subject matter relevant to your topic
2. Finding references to authors who have published and getting the documents
themselves.
1. Identifying subject matter relevant to your topic.
40
Discuss with your tutors and assess the relevance of lecture notes in the particular
area that you want to research. Look up text books for the keynote theorists and
authors and establish, who are the ‘experts’ in your chosen area of research.
As you develop your reference list in relation to the different subjects, methodologies,
modalities and referenced authors, you begin to see that for this area to be effective it
has to be systematic. You need to organize your data by Area, field and aspect: this
will keep it ordered and thus easier to write up and interpret.
2. Finding references to authors who have published and getting the documents
themselves.
Similar to 1. above, references to authors will be contained in the various texts you
will source and it is important to note the important authors. Access to reading
materials which will be discussed later in this report will include use of libraries,
journals, internet sources and text books.
Where to look for reference material
In established research areas there will exist an enormous amount of literature.
This can seem daunting at the outset, but there are key sources and it is important to
keep the search within the frame of reference you have established through your
specific research questions.
Library catalogs contain a referenced list of books by author and it is possible to
extract key texts all related to the one prominent author.
Journals of interest in your area would be especially useful for insights and
abstracts from current research. The American Counselling Association would have a
journal on a monthly basis for example.
41
Current editions of handbooks and core text books would be most useful and also
contain detailed bibliographies and references to authors concerning each modality
and theory.
On line repositories such as www.questia.com, which is a subscription based
services provide access to thousands of book, articles and referenced materials
concerning all areas with counseling.
Two key points to note:
Primary Data: Primary data is data that you have collected yourself whereas
Secondary Data: Secondary data is everything else, other people primary research.
Five Big Words
Research involves an eclectic blending of an enormous range of skills and
activities. To be a good researcher, you have to be able to work well with a wide
variety of people, understand the specific methods used to conduct research,
understand the subject that you are studying, be able to convince someone to give you
the funds to study it, stay on track and on schedule, speak and write persuasively, and
on and on.
Here, we want to introduce you to five terms that we think help to describe some
of the key aspects of contemporary research. (This list is not exhaustive. It's really just
the first five terms that came into our mind when we were thinking about this and
thinking about how we might be able to impress someone with really big/complex
words to describe fairly straightforward concepts).
We present the first two terms -- theoretical and empirical -- together because
they are often contrasted with each other. Research is theoretical, meaning that much
of it is concerned with developing, exploring or testing the theories or ideas that social
researchers have about how the world operates. But it is also empirical, meaning that
42
it is based on observations and measurements of reality -- on what we perceive of the
world around us. You can even think of most research as a blending of these two
terms -- a comparison of our theories about how the world operates with our
observations of its operation.
The next term -- nomothetic -- comes (We think) from the writings of the
psychologist Gordon Allport. Nomothetic refers to laws or rules that pertain to the
general case (nomos in Greek) and is contrasted with the term "idiographic" which
refers to laws or rules that relate to individuals (idios means 'self' or 'characteristic of
an individual ' in Greek). In any event, the point here is that most social research is
concerned with the nomothetic -- the general case -- rather than the individual. We
often study individuals, but usually we are interested in generalizing to more than just
the individual.
In our post-positivist view of science, we no longer regard certainty as attainable.
Thus, the fourth big word that describes much contemporary research is
probabilistic, or based on probabilities. The inferences that we make in research have
probabilities associated with them -- they are seldom meant to be considered covering
laws that pertain to all cases. Part of the reason we have seen statistics become so
dominant in social research is that it allows us to estimate probabilities for the
situations we study.
The last term we want to introduce is causal. You've got to be very careful with
this term. Note that it is spelled causal not casual. You'll really be embarrassed if you
write about the "casual hypothesis" in your study! The term causal means that most
research is interested (at some point) in looking at cause-effect relationships. This
doesn't mean that most studies actually study cause-effect relationships. There are
some studies that simply observe -- for instance, surveys that seek to describe the
43
percent of people holding a particular opinion. And, there are many studies that
explore relationships -- for example, studies that attempt to see whether there is a
relationship between gender and salary. Probably the vast majority of applied research
consists of these descriptive and correlational studies. So why are we talking about
causal studies? Because for most researches, it is important that we go beyond just
looking at the world or looking at relationships. We would like to be able to change
the world, to improve it and eliminate some of its major problems. If we want to
change the world (especially if we want to do this in an organized, scientific way), we
are automatically interested in causal relationships -- ones that tell us how our causes
(e.g., programs, treatments) affect the outcomes of interest.
Types of Questions
There are three basic types of questions that research projects can address:
Descriptive
When a study is designed primarily to describe what is going on or what exists.
Public opinion polls that seek only to describe the proportion of people who hold
various opinions are primarily descriptive in nature. For instance, if we want to know
what percent of the population would vote for a Democratic or a Republican in the
next presidential election, we are simply interested in describing something.
Relational
When a study is designed to look at the relationships between two or more
variables. A public opinion poll that compares what proportion of males and females
say they would vote for a Democratic or a Republican candidate in the next
presidential election is essentially studying the relationship between gender and
voting preference.
Causal
44
When a study is designed to determine whether one or more variables (e.g., a
program or treatment variable) causes or affects one or more outcome variables. If we
did a public opinion poll to try to determine whether a recent political advertising
campaign changed voter preferences, we would essentially be studying whether the
campaign (cause) changed the proportion of voters who would vote Democratic or
Republican (effect).
The three question types can be viewed as cumulative. That is, a relational study
assumes that you can first describe (by measuring or observing) each of the variables
you are trying to relate. And, a causal study assumes that you can describe both the
cause and effect variables and that you can show that they are related to each other.
Causal studies are probably the most demanding of the three.
Types of Relationships
A relationship refers to the correspondence between two variables. When we talk
about types of relationships, we can mean that in at least two ways: the nature of the
relationship or the pattern of it.
The Nature of a Relationship
While all relationships tell about the
correspondence between two variables, there is
a special type of relationship that holds that the
two variables are not only in correspondence,
but that one causes the other. This is the key distinction between a simple
correlational relationship and a causal relationship. A correlational relationship
simply says that two things perform in a synchronized manner. For instance, we often
talk of a correlation between inflation and unemployment. When inflation is high,
unemployment also tends to be high. When inflation is low, unemployment also tends
45
to be low. The two variables are correlated. But knowing that two variables are
correlated does not tell us whether one causes the other. We know, for instance, that
there is a correlation between the number of roads built in Europe and the number of
children born in the United States. Does that
mean that is we want fewer children in the U.S.,
we should stop building so many roads in
Europe? Or, does it mean that if we don't have
enough roads in Europe, we should encourage
U.S. citizens to have more babies? Of course
not. (At least, I hope not). While there is a relationship between the number of roads
built and the number of babies, we don't believe that the relationship is a causal one.
This leads to consideration of what is often termed the third variable problem. In
this example, it may be that there is a third variable that is causing both the building
of roads and the birthrate, that is causing the correlation we observe. For instance,
perhaps the general world economy is responsible for both. When the economy is
good more roads are built in Europe and more children are born in the U.S. The key
lesson here is that you have to be careful when you interpret correlations. If you
observe a correlation between the number of hours students use the computer to study
and their grade point averages (with high computer users getting higher grades), you
cannot assume that the relationship is causal: that computer use improves grades. In
this case, the third variable might be socioeconomic status -- richer students who have
greater resources at their disposal tend to both use computers and do better in their
grades. It's the resources that drives both use and grades, not computer use that causes
the change in the grade point average.
Patterns of Relationships
46
We have several terms to describe the major different types of patterns one might find
in a relationship. First, there is the case of no relationship at all. If you know the
values on one variable, you don't know anything
about the values on the other. For instance, I
suspect that there is no relationship between the
length of the lifeline on your hand and your grade
point average. If I know your GPA, I don't have any
idea how long your lifeline is.
Then, we have the positive relationship. In a positive relationship, high values on
one variable are associated with high values on the other and low values on one are
associated with low values on the other. In this example, we assume an idealized
positive relationship between years of education and the salary one might expect to be
making.
On the other hand a negative relationship implies that high values on one variable are
associated with low values on the other. This is also sometimes termed an inverse
relationship. Here, we show an idealized negative relationship between a measure of
self esteem and a measure of paranoia in psychiatric patients.
These are the simplest types of relationships we might typically estimate in research.
But the pattern of a relationship can be more complex than this. For instance, the
figure on the left shows a relationship that changes over the range of both variables, a
curvilinear relationship. In this example, the horizontal axis represents dosage of a
drug for an illness and the vertical axis represents a severity of illness measure. As
dosage rises, severity of illness goes down. But at some point, the patient begins to
experience negative side effects associated with too high a dosage, and the severity of
illness begins to increase again.
47
Variables
Experiments and research studies are designed around variables. Variables are
measured; variables are manipulated by the experimenter to see how they affect other
variables; two or more variables are measured at the same time to see if there is any
relationship among them. So what then is a variable? It is anything that can vary along
some dimension, or, in other words, it is anything that can be measured.
Distinguishing Between a Variable and a Single Value of a Variable
For something to be called a variable it must be capable of varying; that is, it must
be able to take more than one value or level. For example, “sex of subject” is a
variable, since it can take two values: male and female. “Female” by itself is not a
variable, but rather a single value of the variable “sex of subject.” “Femaleness,”
however, might be a variable if you could think of a way to measure it.
A variable is something that changes. It changes according to different factors.
Some variables changes easily, like the stock-exchange value, while other variables
are almost constant, like the name of someone. Researchers are often seeking to
measure variables.
The variable can be a number, a name or anything where the value can change.
An example of a variable is temperature. The temperature varies according to other
variable and factors. You can measure different temperature inside and outside. If it is
a sunny day, chances are that the temperature will be higher than if it's cloudy.
Another thing that can make the temperature change is whether something has been
done to manipulate the temperature, like lighting a fire in the chimney.
In research, you typically define variables according to what you're measuring.
The independent variable is the variable which the researcher would like to measure
(the cause), while the dependent variable is the effect (or assumed effect), dependent
48
on the independent variable. These variables are often stated in experimental research,
in a hypothesis, e.g. "what is the effect of personality on helping behavior?"
In explorative research methodology, e.g. in some qualitative research, the
independent and the dependent variables might not be identified beforehand. They
might not be stated because the researcher does not have a clear idea yet on what is
really going on.
Confounding variables are variables with a significant effect on the dependent
variable that the researcher failed to control or eliminate - sometimes because the
researcher is not aware of the effect of the confounding variable. The key is to
identify possible confounding variables and somehow try to eliminate or control them.
Scientists operate at both theoretical and empirical levels. At the theoretical level,
there is a preoccupation with identifying constructs and their relation to propositions
and theory. At the empirical level, where the propositions are converted to hypothesis
and testing occurs, the scientist is likely to be dealing with variables. In this context, a
variable is a symbol to which we assign numerals or values.
You won't be able to do very much in research unless you know how to talk about
variables. A variable is any entity that can take on different values. OK, so what does
that mean? Anything that can vary can be considered a variable. For instance, age can
be considered a variable because age can take different values for different people or
for the same person at different times. Similarly, country can be considered a variable
because a person's country can be assigned a value.
Variables aren't always 'quantitative' or numerical. The variable 'gender' consists of
two text values: 'male' and 'female'. We can, if it is useful, assign quantitative values
instead of (or in place of) the text values, but we don't have to assign numbers in order
for something to be a variable. It's also important to realize that variables aren't only
49
things that we measure in the traditional sense. For instance, in much social research
and in program evaluation, we consider the treatment or program to be made up of
one or more variables (i.e., the 'cause' can be considered a variable). An educational
program can have varying amounts of 'time on task', 'classroom settings', 'student-
teacher ratios', and so on. So even the program can be considered a variable (which
can be made up of a number of sub-variables).
An attribute is a specific value on a variable. For instance, the variable sex or
gender has two attributes: male and female. Or, the variable agreement might be
defined as having five attributes:
1 = strongly disagree
2 = disagree
3 = neutral
4 = agree
5 = strongly agree
Another important distinction having to do with the term 'variable' is the
distinction between an independent and dependent variable. This distinction is
particularly relevant when you are investigating cause-effect relationships. It took me
the longest time to learn this distinction. (Of course, I'm someone who gets confused
about the signs for 'arrivals' and 'departures' at airports -- do I go to arrivals because
I'm arriving at the airport or does the person I'm picking up go to arrivals because
they're arriving on the plane!). I originally thought that an independent variable was
one that would be free to vary or respond to some program or treatment, and that a
dependent variable must be one that depends on my efforts (that is, it's the treatment).
But this is entirely backwards! In fact the independent variable is what you (or
nature) manipulates -- a treatment or program or cause. The dependent variable is
50
what is affected by the independent variable -- your effects or outcomes. For
example, if you are studying the effects of a new educational program on student
achievement, the program is the independent variable and your measures of
achievement are the dependent ones.
Finally, there are two traits of variables that should always be achieved. Each
variable should be exhaustive, it should include all possible answerable responses.
For instance, if the variable is "religion" and the only options are "Protestant",
"Jewish", and "Muslim", there are quite a few religions I can think of that haven't
been included. The list does not exhaust all possibilities. On the other hand, if you
exhaust all the possibilities with some variables -- religion being one of them -- you
would simply have too many responses. The way to deal with this is to explicitly list
the most common attributes and then use a general category like "Other" to account
for all remaining ones. In addition to being exhaustive, the attributes of a variable
should be mutually exclusive, no respondent should be able to have two attributes
simultaneously. While this might seem obvious, it is often rather tricky in practice.
For instance, you might be tempted to represent the variable "Employment Status"
with the two attributes "employed" and "unemployed." But these attributes are not
necessarily mutually exclusive -- a person who is looking for a second job while
employed would be able to check both attributes! But don't we often use questions on
surveys that ask the respondent to "check all that apply" and then list a series of
categories? Yes, we do, but technically speaking, each of the categories in a question
like that is its own variable and is treated dichotomously as either "checked" or
"unchecked", attributes that are mutually exclusive.
Determine the Relevant Variable
Anything that may assume different numerical values
51
Types of Variables
Categorical
Continuous
Dependent
Independent
Independent and Dependent Variables
Researchers are most interested in relationships among variables. For example,
does a participative leadership style (independent variable) influence job satisfaction
or performance (dependent variables) or can a superior’s modeling of ethical behavior
influence the behavior of the subordinate? It is important to remember that there are
no preordained variables waiting to be discovered “out there” that are automatically
assigned to one category or the other. As one writer notes:
There’s nothing very tricky about the notion of independence and dependence. But
there is something tricky about the fact that the relationship of independence and
dependence is a figment of the researcher’s imagination until demonstrated
convincingly. Researchers hypothesize relationships of independence and
dependence: They invent them , and then they try by reality testing to see if the
relationships actually work out that way.
Moderating Variables
In each relationship, there is at least one independent variable and a dependent
variable. It is normally hypothesized that in some way the independent variable
causes the dependent variable to occur.
In actual study situations, however, such a simple one-on-one relationship needs to
be conditioned or revised to take other variables into account. Often one uses another
type of explanatory variable of value here—the moderating variable. A moderating
52
variable is a second independent variable that is included because it is believed to
have a significant contributory or contingent effect on the originally stated IV-DV
relationship. For example, one might hypothesize that
The introduction of the four–day workweek(IV) will lead to higher productivity
(DV), especially among younger workers(MV).
Whether a given variable is treated as an independent or as a moderating variable
depends on the hypothesis.
Extraneous Variables
An almost infinite number of extraneous variables exists that might conceivably
affect a given relationship. Some can be treated as independent or moderating
variables, but most must either be assumed or excluded from the study. Fortunately,
the infinite number of variables has little or no effect on a given situation. Most can
be safely ignored. Others may be important, but their impact occurs is such a random
fashion as to have little effect.
In routine office work(EV-control), the introduction of the four–day
workweek(IV) will lead to higher productivity (DV), especially among younger
workers(MV).
Intervening Variables
The variables mentioned with regard to casual relationships are concrete and
clearly measurable; they can be seen, counted, or observed in some way. The
intervening variable may be defined as “that factor which theoretically affects the
observed phenomenon but cannot be seen, measured, or manipulated; its effect must
be inferred from the effects of the independent and moderator variables on the
observed phenomenon.”
53
The introduction of the four–day workweek(IV) will lead to higher productivity
(DV), by increasing job satisfaction (IVV).
Here is the example illustrating the relationships involving independent,
moderating, controlled extraneous, and dependent variables. The management of a
bank wishes to study the effect of promotion on savings. It might advance the
following hypothesis:
A promotion campaign (IV) will increase savings activity (DV), especially when
free prizes are offered (MV), but chiefly among smaller savers(EV-control). The
results come from enhancing the motivation to save(IVV).
None of the three variable definitions given above is a good operational definition.
For each of them, you would have to ask the researcher some questions before you
could determine exactly what was meant. For the amount of stimulant consumed
variable, the first question that probably occurred to you is “what stimulant?”. It
would make a lot of difference if the researcher were talking about “amphetamine”
rather than “caffeine.” Another obvious question would be what is meant by
“moderate” and “large” doses. It would be necessary to be much more specific about
dosages before this would begin to be a usable operational definition that would allow
someone else to repeat the research.
“Hyperactivity” is a word that is often used, but rarely well defined. When you use
activity level as a variable is a research study, it is necessary to be very specific about
the basis on which you will decide that a given child should be classified as
withdrawn, normal, or hyperactive. In the preschool class example at the beginning of
this chapter, you used three different possible operational definitions of activity level.
In one case, you asked the teacher to sot the children into three word-labeled
categories,, For another measure, you asked two student raters who did not know the
54
children to observe them and to agree on a rating of their activity. Your third
operational definition concerned the frequency with which each child would switch
among toys under set conditions. Probably this last operational definition is most
repeatable by a new researcher, but it might not be exactly what you fell is meant by
“activity level”. If your two student observers showed high agreement in their rating
when flowing your instructions as to how to use the rating scale, then it is likely that
the second operational definition, if it included your instructions, would also be usable
by other researchers. To make a complete operational definition of hyperactivity, it
would be necessary to state the rules by which the children were classified or assigned
values, how they were observed, and who did the classification.
The toy-preference variable seems somewhat easier to operationalize, but there are
still some things that the original definition doesn’t specify, one researcher might
define “toy preference” as which toy the child selected when presented with the entire
array and told to choose the one he or she liked best. Another researcher might define
it according to the amount of time the child spent playing with each toy when all were
continuously available. Notice that these two operational definitions might not yield
the same results. For example, a child might choose the pretty doll first, but might
subsequently spend a longer time playing with the model farm, in your research,
you’ll have to use the operational definition that comes closest to what you mean by
“toy preference.”
Recapitulation
For what is a variable?
a) A variable is anything that can vary along some dimension. Any variable must
have at least two or more different single values or levels. When you define a
55
variable in a research study, the definition must be so complete that another
person would have Identify at least three variables involved in this study.
b) For each of the three variables, write out the operational definition of the
variable as given in the above description of the experiment.
c) Are there any improvements or clarifications these operational definitions
should have? Please should have?
d) For one of the variables you have identified, give a description of the complete
set of single values the variable could have.
Independent and Dependent Variables
Distinguishing IVs and DVs
As you worked on the sample test question at the end of the previous section, you
may have felt that the different variables you identified function in very different
ways in the study. The levels or values of some of the variables were specifically
created by the experimenter before running the actual experiment. The experimenter
decided to make the sex of the children participating in the study a variable by
selecting both boys and girls. In addition, the experimenter created the four story
situations, determining all the levels of another variable in the experiment. Both of
these variables can be seen as input into the experiment. Such variables, whose levels
are determined ahead of time by the experimenter, are called Independent Variables
(IVs).
In contrast, the variable measuring what the children remembered about the story
is an output variable for the experiment. While the experimenter determined ahead of
time how this variable would be measured, the actual measurements obtained are
dependent on the responses of the children and can be known only after running the
experiment. What the children remember is, after all, what the experimenter is trying
56
to find out by conducting the research, such variables representing the output of the
experimental process are called Dependent Variables (DVs).
Usually it is not too difficult to spot the major IVs and DVs early in the description
of a research study, sometimes in the title itself. One general form of descriptive
statement that often occurs is: “This experiment was done to establish the effects of
IIV) on (DV).” For example, an experiment might be done to investigate the effects of
subject anxiety on learning a list of nonsense syllables. The input to this particular
study would be a set number of different levels of subject anxiety as selected or
determined by the experimenter. The output would be measurements of each subject’s
performance on the learning task. The purpose of the experiment would be to
establish a cause-effect relationship between the two variables.
The same combination of IV and DV could be given in a different form of
statement: “This research study was conducted to see whether (DV) was affected by
changes in (IV)”. Specifically, the above experiment might be described as: “This
experiment attempted to show whether performance on a learning task was affected
by the level of anxiety of the subjects.” Even though this statement appears in reverse
order, the experimenter still is doing the same thing: manipulating or selecting subject
anxiety level (IV) to determine whether it affects or changes measurements of the
subjects’ learning performance (DV).
You have to be careful about picking out IVs and DVs from research description,
because a given variable might be and IV in one research study and a DV in another.
For instance, a researcher might be interested in determining whether engaging in a
learning task affects a subject’s level of anxiety. In this case, the experimenter will
determine ahead of time the kinds of learning tasks to which the subjects will be
exposed and will measure as output for the study the level of anxiety produced in the
57
subjects by participating in the learning tasks. Now, the IV is the type of learning task
and DV is subject anxiety.
Operational definition of an Independent Variable
An IV is operationally defined by specifying the levels (single values) that the
experimenter is going to use in the experiment. Since as IV is a variable, the
operational definition of an IV must specify at least two levels, although there can be
more. In the sex-stereotyping research study described on page 51, the “sex-of-
subject” IV had two levels: boy and girl. The experimenter saw to it that he selected
20 of each of the two kinds of subject. The second IV, the four levels of story
situation, was also set up ahead of time by the experimenter: he made sure that each
of the four sex-stereotyping situations was represented equally often in the story the
children read.
Since most variables can potentially have a large number of levels, you must be
explicit about which levels of the IV you are using when defining a particular IV for a
particular experiment. For example, if we wanted to do a study on the effects of the
consumption of a stimulant (IV) on the speed of working through a set of math
problems (DV), we would specify not only what stimulant we Intend to use, but also
all the dosage levels. We might, for example, set up three levels of the IV as follows:
no cups of coffee (no-dose level), one cup of coffee (low-dose level), and four cups of
coffee (high-dose level). Depending on how we designed our experiment, we might
have different subjects for each of the dosage levels or we might study the same
subjects’ at all three dosage levels. In either case, our DV will be a measurement of
how rapidly each subject works through the set of math problems.
Operational definition of a Dependent Variable
58
An operational definition of a DV must include specification of what measures you
are going to take on the subjects, what kind of measurement scale you will be using,
and how you will take the measurement. For example, In the stereotyping study, the
DV was a reading-comprehension score. For a complete operational definition he
researcher has to describe the test and how it was scored. For example, he might have
used 40-item test with 10 items relevant to each of the four story situations. Each child
could then score from 1 to 10 for each story situation. This would be an interval-type
measurement of the child’s recall. Or he might ask each child to recall an event from
the story and then categorize the child according to which of the four story situations
he or she chose to recall. This would be a nominal-type measurement of what the
child remembered best about the story.
Multiply IVS
Frequently an experiment or research study will use more than one IV or more
than one DV. While this increases the complexity of doing the research, it also
increases the amount of information that can be gathered from a single study.
Using multiple IVs can quickly make an experiment more complicated to run.
Consider the experiment proposed on page 53, designed to determined whether the
amount of coffee consumed by a person affects the speed at which a set math
problems can be worked. The major IV, “amount of stimulant consumed,” was
defined as having three levels. But, as we begin to think about this, we might wonder
whether those subjects already used to drinking coffee would react differently to the
stimulation effects of a dose of coffee than subjects who didn’t regularly consume the
stuff. So we decide to add a second IV, “customary coffee consumption,” and define
it as having three levels: rarely or never drinks coffee, moderate consumer (one or two
cups a day), and heavy consumer (three or more cups a day). Now we have expanded
59
the experiment to include nine conditions (i.e. rare drinkers who get no coffee, rare
drinkers who get one cup, moderate drinkers who get four cups, heavy drinkers who
get no coffee, heavy drinkers, who get one cup, and heavy drinkers who get four
cups).
A friend (?) of ours then suggests that the difficulty of the math problems might
also be worth considering. He speculates that a person who had consumed a timulant
might whip through a set of easy problems more quickly, but would be unable to
concentrate on hard problems. This tempts us to add a third IV, problem difficulty,
which we define as having two levels: an easy set of problems and a hard set of
problems. Now there are eighteen (3*3*2) condition in our experiment. An example
of one of the condition would be a moderate coffee drinker (level 2 of IV2) who is
given a heavy dose of coffee (level 3 of IV1) and is then asked to work an easy set of
problems (level 1 of IV3). Our statistical consultant points out that if we intend to use
different subjects for each condition of our experiment, then we’d better find at least
180 subjects, a minimum of ten for each experimental condition. Another so-called
friend subsequently muses that the use of stimulants may be helpful to men in solving
problems but be harmful to women (whom he feels are sufficiently agitated without
any outside help), and accordingly suggests yet a fourth IV for our experiment. When
we contemplate having to find ten men and ten women for each of 18 experimental
conditions, we slug him. Seriously, it is easy to see how careful consideration of a
research question could suggest many interesting IVs. Dealing with multiple IVs and
deciding how many you can use at the same time is a design issue we will consider in
a later chapter.
What does it mean to “Operationally Define” a Variable?
60
Defining a variable completely so that another person interprets the variable in the
same way is called “operationally defining the variable.” In order for the results of an
experiment or research study to be of any use to the rest of the world, the results must
be repeatable. This means not only that the original researcher should get the same
results when the research study is repeated, but also that any other person who repeats
the same research should get the same results. It is pointless to try to make any
theoretical or practical use of a research procedure that works one way for one person
and another way for another person. To ensure that your research will be repeatable
by other investigators it is necessary that you be perfectly clear as to how your
variables are defined in terms of what you did to vary them or to measure them. Only
if others can interpret your variables in the same way as you did originally will they
be able to replicate your research.
Much of the confusion that exists about differing results of what appears to be the
same research procedure can be traced to differing operational definitions of variables
called by the same name. if study 1 says that increased “anxiety” improves “learning”
and study 2 day that increased anxiety impairs learning , don’t throw up your hands in
despair, chuck the studies in the wastebasket, and walk off muttering that you knew
there was nothing gto social science research. Stop and take a look first at how the
variables were operationally defined in the two studies. Does anxiety mean the same
thing in both studies? Was learning measured the same way in the two studies? The
apparent difference in results may actually represent a difference in the operation
definitions of the variables in the experiments. If you want to apply the research
results of others to your own work, you have to look for the operational definition
closest to your definition of the term.
Application 1 of Operational Definition of Variables
61
For each of the variables below, think of two different ways you might
operationally define the variable. Be very specific in your definitions so that another
person could not be confused about your meaning.
Learning
Hunger
Political view point
For the variable learning, you could probably come up with different operational
definitions as long as you continued to think about it. An operational definition of
“learning” for a particular experiment or research study must specify what is being
learned, the conditions under which it is being learned, and how the learning is being
measured. Thus for one experiment, and operational definition for learning might be
score on a particular test; for another, it might be the solution time for puzzle; and for
another, it might be the rate of learning a new task; and so on.
Hunger is another term that may be operationally defined in many ways. In animal
experiments it is often operationally defined as “hours without food”. Some animal
researcher, however, believe that percent of normal body weight after deprivation is a
better operational definition. Another possible definition of hunger of may be
expressed in terms of how much trouble the subject is willing to go to get at food. To
be precise, researchers might measure the intensity of electric shock the: animal is
willing to experience to gain access to the food. If you are dealing with humans, you
might ask them either directly or indirectly how hungry they feel at a given moment.
Or you might offer them some food and note whether or not they take any, and, if
they do, how much they take. Obviously, with a little thought, you could come up
with a vast number of operational definitions of "hunger."
Political viewpoint is yet another general term that would have to be
operationalized for any particular research study. Political viewpoint might be defined
62
for your purposes as the political party to which a person is registered. Or you might
want to define it in terms of the person's response to a set of questions about current
political situations. Maybe you might prefer to have the people rate themselves on a
conservative-to-liberal scale. There are still other ways you could operationally define
political viewpoint. You could ask people who they voted for in a recent election or
determine their score on a standardized questionnaire (for example, the Radicalism-
Conservatism Scale published by Comrey and Newmeyer, 1965). However you define
the variable, it must be possible for someone else to read your operational definition
and be able to take measures of political viewpoint in the same way.
Application 2 of Operational Definition of Variables
For each of the following definitions of variables, specify in what ways the
operational definitions are not complete by writing down at least one further question
you'd want to have answered before you could use the variable in your own research
study.
1. Amount of stimulant consumed: Each subject will consume no stimulant, a
moderate dose, or a large dose.
2. Activity level: The children in the classroom will be classified as withdrawn,
normal, or hyperactive.
3. Toy preference: The same set of six toys will be presented to each child and
the one he or she prefers noted.
Threats to a Valid Measurement Procedure
Sensitivity of Dependent Measure
Sensitivity is increased thought observing responses that can differ subtly. For
example, measuring subjects’ reaction time in a stoop interference task is more
sensitive than merely seeing if they correctly name the ink color. Measuring correct
63
responses glosses over subtle differences in what subjects are mentally experiencing.
Measuring reaction time is sensitive to these differences because a slow correct
response indicates more interference than a fast one.
Sensitivity is also increased through increased precision in measurement. Thus,
reaction time is measured in milliseconds (thousandths of a second) because this level
of precision is sensitive to very small differences cause by interference. Likewise,
rating scales increase precision, so instead of merely judging whether or not a child
acts aggressively toward “Bobo,” we would rate the degree of a behavior, as opposed
to merely nothing whether it occurs r not. In other words as we originally planned, try
to use interval or ration scales of measurements. These are generally more precise and
sensitive of procedure; we also attempt to avoid any aspects of the design that might
artificially limit subjects’ scores.
Avoiding the Restriction of Range Problem
The experimental task must realistically allow subjects to obtain any of the
different scores that occur on the variable. Therefore, when designing a measurement
procedure we should envision the range of scores it has the problem of restricted
range. Restriction of range occurs when the range of scores on a variable is limited by
the researcher. For example, there is a wide range of aggressiveness that a subject
might exhibit. Therefore our procedure should allow subjects to score anywhere
within that wide range. If it does not we have restricted the range. Then, because we
have limited the scores that subjects may obtain, it is because it is now more difficult
to obtain relatively large differences between the groups we have reduced our
statistical power.
To avoid restriction of range, first consider the method you will use to assign
scores, in our aggression study, we want a system that allows a wide range of scores
64
to occur. If our system allows subjects to get an aggression score of only 1, 2, or 3, the
range is restricted; if subjects can score between 0 to 100, the range is not restricted.
Second, consider demand characteristics, subject characteristics, or task requirements,
that will restrict the behaviors that subjects exhibit and, thus, inherently restrict the
range of scores they receive. In our reaction time study, for example, we avoid testing
only exceptionally coordinated athletes who may produce the same scores regardless
of condition. In our aggression study, we avoid communication the expectation that
our subjects should “behave themselves” so that all children do not produce the same
low aggression score in the different condition.
Determine how you will Define and measure your Variables
To scientifically evaluate a hypothesis, we must first translate the hypothesis into
a specific prediction about the outcome of empirical observation. Consider, for
example, the hypothesis that watching violence on television is directly related to
aggressive behavior of preschool children. This hypothesis predicts that, if we observe
a group of preschool children, we should see that those who watch more TV violence
are more aggressive than those who watch less TV violence. Before we can evaluate
this prediction, however we need to divide exactly how we will define and measure
“TV violence” and exactly how we will define and measure “aggressive behavior.”
The variables identified in the research hypothesis must be defined in a manner that
makes it possible to measure them by some form of empirical observation. These
decisions are usually made after reviewing precious research and determining how
other researchers have defined and measured their variables.
By defining our variables so that they can be observed and measured, we transform
the research hypothesis (from Step 2 of the research process) into a specific, well-
defined prediction that can be tested by making empirical observations. We are now
65
ready to evaluate the hypothesis by actually observing the variables to determine
whether or not they are really related. The key idea is that the hypothesis is now in an
empirically testable form.
Note that the task of determining exactly how the variables will be defined and
measured often depends on the individuals to be measured. For example, you would
certainly measure the aggressive behavior of a group of preschool children very
differently from the aggressive behavior of a group of adults.
Identify the Participants or Subjects
Before you can begin data collection, you must decide exactly what individuals
will participate in the research study. If the individuals are human, they are called
participants. Nonhumans are called subjects. It is the responsibility of the researcher
to plan for the safety and well-being of the research participants and to inform them of
all relevant aspects of die research, especially any risk or danger that may be
involved.
In addition, you must decide whether you will place any restrictions on the
characteristics of the participants. For example, you may decide to use preschool
children. Or you may be more restrictive and use only 4-year-old boys from two-
parent, middle-income households who have been diagnosed with a specific learning
disability. You also must determine how many individuals you will need for your
research and you must plan where and how to recruit them.
The individuals who take part in research studies are called participants if they are
human and subjects if they are nonhuman.
Select a Research Strategy
66
Choosing a research strategy involves deciding on the general approach you will
take to evaluate your hypothesis. The choice of a research strategy is usually
determined by one of two factors:
The type of question asked: The simplest kind of research question asks only about
the existence of a relationship. You may, for example, want to know whether or not
there is any relation between academic success and self-esteem. A more sophisticated
question would concern why there is a relation: what causes the relation between
academic success and self-esteem? Questions about causes typically require a
different strategy from questions about existence.
Controlling Threats to Reliability and Validity
Once we select the conditions of the independent variable, we must precisely
define how we will create and present them. In doing, we must consider the four
components of an experiment: the subjects, the researcher, the measurement
procedure, and the environment. For each, we try to anticipate and eliminate anything
that may threaten the reliability and validity of our results.
Creating a Reliable Manipulation
Recall that reliability means that nut result are consistent and contain no error. We
seek reliability not only in terms of measuring subject’s scores but also in terms
reliably manipulating the independent variable: All subjects in a condition should
receive the same amount or category of the variable, and when we change to another
condition, all subjects there should receive the same new amount. Thus if in our
temperature study the conditions are 70 and 90 degrees, then we want precisely 70
degrees for all subjects in one condition and precisely 90 degrees for all subjects in
the other conditions.
67
If there is inconsistency in out manipulation, then we are not always presenting the
condition we think we are, and our conclusions well be based on the wrong amounts
of the independent variable Alto, if the experimental situation b in any way different
for each subject, we will end up with different responses and thus variability among
the scores within each condition. Recall that such variability is called error variance,
and that greater error variance reduces the strength of the relationship and decreases
power.
Controlling Confounding Variables
Although we are concerned with all aspects of validity, once an independent
variable has been created our concern is focused on internal validity, the degree to
which we can draw accurate inferences about the observed relationship in our study.
In experiments, internal validity is of paramount importance, because we seek to draw
a causal inference. We must be confident that it is only the changes in the independent
variable that cause changes in the dependent scores. Therefore, it is especially
important that our manipulation of the independent variable does not introduce a
confounding. Recall that a confounding occurs when systematic differences between
the conditions exist, in addition to the Independent variable.
In our smile study, for example, let's say that holding a pen between the teeth not
only makes subjects smile but also makes them laugh. Holding the pen in their lips
does neither. The results of such a study can be diagrammed as shown in Table 5.2.
Note that what we are calling the smile condition produced a higher average mood
score. The problem is that there are two ways to describe our conditions: either in
terms of the intended manipulation of mimicking or not mimicking a smile, or in
terms of the confounding variable of laughing or not laughing. Because of this
confounding we don't know which variable is causing die elevated mood, so we
68
cannot be confident that we have confirmed die hypothesis that facial feedback from
smiling increases mood.
A confounding can also work to cover up die predicted relationship. If subjects
holding die pen with puckered lips laughed more often and thus experienced a higher
mood. this confounding could produce results directly opposite to those we predicted,
causing us to think that mood improves with less use of die smile muscles. Or, let's
say that the researcher inadvertently made those subjects in the smile conditions more
anxious. Then, although smiling might actually tend to increase mood, increased
anxiety might tend to decrease mood. These effects could cancel out so that we find
no difference in our original hypothesis.
TABLE 5.2 Diagram of a Confounded Smile Study
Conditions
Condition 1 Condition 2
(Pen in teeth) (Pen in lips)
Independent Smile No-smile
variable
Confoundin Laughter No-laughter
g variable
Mood XXX XXX
scores
High X mood Low X mood
A confounding could also occur in our temperature study. Given that we must
provide an experimenter for subjects to aggressive toward, anything that consistently
differentiates that experimenter from one temperature condition to the next is a
potential confounding variable. Let's say that we carelessly employ a male
experimenter when the room is 70 degrees and a female experimenter when the room
69
is 90 degrees. In this case, we cannot know whether differences in subject
aggressiveness are due to differences in room temperature or to differences in the sex
of the experimenter.
Different confounding variables arise with the use of different independent
variables. We have a confounding variable if we are presenting examples of different
types of music, but one song contains lyrics and another does not, the volume of one
is greater than that of another, or one is familiar and another is obscure. The same is
true if we are presenting lists of happy or sad words, but one list consists of words that
contain more syllables, are less familiar, or are more socially acceptable. Or, we have
a confounding if we employ a confederate who is friendlier, more attractive, or more
disorganized in one condition than in another.
Remember, for any study, you won't know whether subjects are being influenced
by such extraneous variables instead of by your independent variable. Rather you
must anticipate penitential confounding and attempt to eliminate them, so that to your
knowledge, the only thing that systematically differentiates the conditions is your
independent variable.
Remember every design must be scrupulously examined to eliminate potential
confounding.
We may control extraneous variables that produce confounding by trying to
eliminate them, thus, ideally. We would design a procedure for the smile study in
which no subjects will laugh. If, however, we're unable to eliminate them, we may
keep the extraneous variables constant across all condition. For example, we could
select happy and sad words that all have the same length, the same familiarity, and so
on.
70
If we do not eliminate or keep an extraneous variable constant, then we may
balance it, allowing its influence to be evenly spread across subjects within every
condition. For example, we could employ a male experimenter with half of the
subjects in each temperature condition and a female experimenter with the other half.
By ignoring gender, we are collapsing across that variable—that is, combining the
scores from the different amounts or categories of that variable. Above, we would say
that we "collapsed across gender" because we combined the two groups in a column
into one group.
By balancing experimenter gender, we evenly distribute its Influence in each
temperature condition. Then, if the 90-degree condition produces greater aggression
In subjects this result cannot be attributed to the particular sex of the experimenter
present, because each sex is equally represented in each condition. We can also
analyze the influence of a balanced variable; so, for example, we could see what
differences arise due to the presence of a male or female experimenter.
Diffusion of Treatment
Another threat to internal validity associated with the independent variable is
diffusion of treatment. Diffusion of treatment occurs when subjects in one condition
are aware of the treatment given in other conditions. In a between-subjects design, for
example, let's say the subjects who have participated in one condition tell other
potential subjects about their experience. Because the latter subjects are now aware of
the treatment, the strength of our manipulations may be reduced. Diffusion of
treatment is especially a problem when we manipulate the information given to
subjects in each condition, or when we surprise them with some event. Not only will
the strength or impact of our treatment be reduced, but the internal validity of the
design will be threatened as well: Instead of being influenced only by the condition of
71
our independent variable, subjects' responses will be influenced by extraneous
information they have about the conditions and the study.
Bear in mind that your general idea is simply a starting point that eventually will
evolve into a very specific research question. Your final question or research
hypothesis will develop as you read through the research literature and discover what
other researchers have already learned. Your original idea will guide you through the
literature and help you decide which research studies are important to you and which
are not relevant to your interests. Eventually, you will become familiar with the
current state of knowledge and can determine what questions are still unanswered. At
this stage you will be ready to identify which subjects might communicate with each
other. Third, we can test subject from different locations so that they have little
physical contact with each other. Finally, as we'll see, we can also endeavor to
disguise our treatments.
Creating Consistent Procedures
In addition to presenting a reliable manipulation and eliminating any obvious
confounding, a key issue in designing the remainder of the procedure is the need to
achieve consistency. To accomplish this objective, we standardize our procedures,
preventing any extraneous variables from randomly fluctuation within and between
the conditions. Thus, in each condition of the smile study, all subjects should hold the
same type of pen in their mouths, for the same period of time, all the while
maintaining the same posture and performing the same tasks. In the temperature
study, all subjects should be warmly dressed to the same degrees, seated the same
distance form the heater, acclimated to the temperature to the same extent, and so on,
if we have not achieved such consistency, fluctuating variables may cause subjects to
respond differently with in condition, producing greater error variance and reducing
72
our statistical power. Further, such fluctuations might cause one condition to differ
from another, thus confounding the independent variable.
Remember:
A key issue in designing our procedures is the need to produce consistency within
and between conditions.
Instructions
Our instructions should clearly explain the task so that subjects consistently attend
to the stimulus and respond appropriately. In particular, we describe the sequence of
events, identify the stimuli they should attend to, and explain how to indicate a
response. We also anticipate subjects' questions. (Should they guess when making a
response? Should they hurry?). Further, we use instructions to prevent unwanted
subject behaviors: We tell them not to look around, fidget, or talk, so they don’t miss
crucial aspects of the task, the goal is to have all subjects perform precisely the same
intended task, without introducing extraneous stimuli or behavior that make the task
different for different subjects.
When creating instructions, we make sure they're clear for the least sophisticated
subjects, avoiding psychological jargon and using words that subjects will easily
understand (especially if they are children). Then, we present the same instructions to
all subjects. If we are manipulating a variable through instructions, we change only
the necessary parts and avoid a confounding by keeping all other aspects (e.g., length,
duration, and wording) constant. We never "ad lib" instructions; instead, we may read
them aloud, using a neutral voice that can be consistently reproduced, or play a tape
recording of them.
Of course, there are no guarantees that our instructions will have the desired effect.
We can encourage subjects to ask questions if they don't understand us, but the more
73
they do so, the more their testing experience differs from those who do not ask
questions. For clarity we can add more detail, but experienced or bored subjects may
not listen, especially when instructions are presented in a dry, neutral manner. Finally,
we can have subjects read the instructions themselves, but in that case we must be
sure they actually do read and understand them. The creation of effective yet efficient
instructions requires considerable effort on the part of the researcher.
Automation
At the outset of any study, we must consider the mechanics of how we will present
a stimulus and determine whether it will introduce inconsistency. For example, use of
a stopwatch to time the presentation of visual stimuli can be unreliable, and variation
in presentation times is an extraneous, potentially confounding variable. The
alternative is to rely on automated equipment—electronic timers, slide projectors and
the like—to control and present stimuli. Likewise, when presenting auditory stimuli,
we can use headphones to keep the volume constant for all subjects (rather than risk
having them sits near or far from speakers) and exclude distracting environmental
noises. When creating a situation for subjects to react to, we might videotape the
situation so that all subjects in a condition see exactly the same event. We can also
benefit from the availability of personal computers, which present many opportunities
for providing reliable presentation of stimuli through automation.
Testing Subjects in Groups
An important question to ask ourselves when designing a study is whether to test
subjects individually or in groups. Group testing is more common when the task
requires subjects' written responses, such as on a test or questionnaire. The advantage
of group testing is greater efficiency in collecting the data. Also if we can test an
entire condition at one time, all subjects will experience the same condition
74
consistently presented The disadvantage is that subjects may make noise, block one
another's view, or otherwise distract each other. As a result, the situation for some
subjects may be slightly different fro that for other subjects, especially for those tested
in another group. Therefore, we need to be especially careful to control subjects'
extraneous behaviors, so as to prevent potential confounding. Usually, this issue can
handled with particularly explicit instructions, but it also depends on the experiment
and how susceptible the subjects are to the presence of other subjects.
Pilot Studies
To be sure that our procedures are appropriate and reliable and that we have
anticipated all major extraneous variables, we conduct a pilot study before conducting
our actual study. A pilot study is a miniature version of a study in which we test our
procedures. Using a sample of the type of subjects we will test in our actual study, we
determine such matters as whether the instructions are clear and subjects perform
appropriately, weather the task is doable given time constraints or other demands and
whether the manipulation actually works and is strong. Rather than just measuring the
dependent variable, we may ask subjects questions and talk with them about the
procedure to determine what they are actually experiencing. In addition, a pilot study
allows the experimenter to work out any bugs in the equipment or procedure. (In the
smile study, for example, the experimenter should not giggle!)
Pilot studies are also used to create and validate our stimuli. Lets say our conditions
involve showing films that contain different amount of violence. Our personal
judgment is of little help in determining the amount of violence in a film (perhaps we
are particularly insensitive to violence). Therefore, we would show the films to pilot
subjects and have them rate the amount of violence each contains.
75
For any problems we identify during a pilot study, we alter the stimuli, task, or
instructions and conduct more pilot studies until we have the desired situation in each
condition (In APA-style reports, pilot studies are briefly described in the materials
section).
Manipulation Checks
Although pilot studies are very useful, to increase internal validity we may also
want to check whether our manipulation actually influenced our subjects in the
experiment. A manipulation check is a measurement, in additions to the dependent
variable that determines whether the independent variable had tits intended effect.
Usually this check is made after subjects have performed the tasks; it is especially
useful for ensuring that an independent variable has the intended influence on an
intervening variable. For example, In a study where we manipulate frustration to
influence subjects' intervening anger and then measure aggressiveness, we can check
the manipulation by directly measuring subjects' anger (e.g., by determining their
physiological responses or having them complete a questionnaire). If subjects make
the desired responses, we have greater confidence that differences in aggression are
the result of differences in anger. If they don't, we go back to the drawing board to
redesign the study.
Remember Instruction, automation, group testing, pilot studies, and manipulation
checks are important consideration for creating a reliable and valid design.
Controlling Demand Characteristics
Imagine you are a subject in an "experiment" that is taking place in a "laboratory"
full of timers, cameras, and who-knows-what A "psychologist" with lab coat and
clipboard puts a plate of cookies in front of you. He says, "Normal people crave
cookies at this time of day, so eat if you want" I'll bet you eat one. If he says, "Only
76
people who have no self-control eat at this time, but eat if you want," I'll bet you don't
in any situation the social and physical surroundings provide cues that essentially
"demand" that we behave in a certain way. In research these cues are called demand
characteristics. A demand characteristic is a cue provided by the research context that
guides or biases a subject s behavior. Subjects rely on demand characteristics to
answer such questions as "what's really going on here? What am I supposed to do?
How will my response be interpreted These cues arise in addition to-or despite-the
explicit instructions we give subjects, and their use is not necessarily the result of an
intentional, conscious process.
Demand characteristics can be grouped in terms of the four components of study.
First, subject brings with them certain attitudes that influence their behavior. Research
procedures are mysterious, and rumor has it that psychologist do strange things to
subjects and study only intelligence, sexual deviance, and crazy people. Therefore,
while subjects are trying to determine what is expected of them, they are also on
guard, sensitive to portraying themselves in a bad light or appearing abnormal.
Because of these concerns, different subjects will react differently to the same
situation.
Cause and Effect Relationships
Four basic strategies for investigating variables and their relationships –
descriptive, co relational, experimental, and quasi-experimental. Now we discuss
details of the experimental research strategy.
The term experiment is often used as a generic label for any kind of scientific
research; however, a research study must meet specific criteria to qualify as an
experiment, and only a limited number of research studies actually deserve periment
from other kinds of research studies.
77
The goal of the experimental strategy is to establish the existence of a cause-and-
effect relationship between two variables. That is, the method of observing variables
is intended to show that changes in one variable are directly responsible for causing
changes in the other variable. Although demonstrating a causal relationship may seem
straightforward, it can be very difficult.
Causation and the Third-Variable Problem
One problem for researchers is that the variables they study rarely exist in
isolation. In natural circumstances, changes in one variable are typically accompanied
by changes in many other related variables. As a result, researchers are often
confronted with tangled network of interrelated variables. Although it is relatively
easy to demonstrate that one variable is related to another, it is much more difficult to
establish the underlying cause of the relationship. To determine the nature of the
relationships among variables, particularly to establish the causal influence of one
event on another, it is essential that a researcher separate and isolate the specific
variables being studied. The task of teasing apart and separating a set of naturally
interconnected variables is the heart of the experimental strategy. The following
example illustrates one basic problem with interrelated variables.
Ronald Freedman and his colleagues examined trends in family planning and birth
control through the 1960s and 1970s in Taiwan. In the course of their studies, they
recorded data on a wide range of behavioral and environmental variables. The
purpose of this research was to identify the factor of factors they determine how
people set preferences for family size and whether or not they use birth control. The
researchers evaluated the relationship between birth control practices and each of the
individual variables. Although the research identified many variables related to family
78
planning, the results clearly showed a strong relationship between television watching
and birth control practices (Freedman, Coombs, Chang, & Syn, 1974).
Although the results of the study establish that television watching is related to
contraception, you probably are not willing to conclude that it is a causal relationship;
that is, having people watch more television probably will not cause the use of
contraception to increase. Clearly, other variables such as age, household income, and
education are involved. The existence of a relationship, even a strong one, is not
sufficient to establish cause and effect.
This example is a demonstration of the third-variable problem. Although the
researchers demonstrated a relation between contraception use and television
watching, common sense suggests that this is not a causal relationship. A more
reasonable interpretation of the results is that there is another, unidentified variable
(or combination of variables) responsible for causing simultaneous increases in birth
control and television watching.
Causation and the Directionality Problem
A second problem for researcher's attempting to establish cause and effect is
demonstrated in the following example.
Many researchers have investigated the relationship between personality and
success or failure in different occupation. One of the most consistent findings is that
successful executives or entrepreneurs tend to be more assertive than their less
successful counterparts (McClelland, 1976, 1987). Based on the consistency of the
relationship and on common sense, it is temptation to conclude that there is causal
relation between assertiveness and executive success. For example, it is reasonable to
assume that assertiveness is a necessary prerequisite for this kind of job; therefore,
people who enter executive position with an assertive personality will tend to be
79
successful that is, assertiveness causes success. However, it is equally reasonable to
assume that executive positions require people to become assertive, so people who
persist and are successful in the job tend to become assertive, that are, success causes
assertiveness. This example is demonstration of the directionality problem.
Although a research study may establish a relationship between two variables, the
problem is determining which variable is the cause and which the effect is.
Controlling Nature
The preceding examples demonstrated that we cannot establish a cause and effect
relationship by simply measuring two variables. In particular, the researchers must
actively unravel the tangle of relationships that exists naturally. To establish a cause
and effect relation, an experiment must control nature, essentially creating an
"unnatural" situation wherein the two variables being examined are isolated from the
influence of other variables and wherein the exact character of a relationship can be
seen clearly.
We acknowledge that it is somewhat paradoxical that experiments must interfere
with natural phenomena to gain a better understanding of nature how can observations
made in an artificial, carefully controlled experiment reveal any truth about nature?
One simple answer is that the contrived character of experiments is a necessity: To
see beneath the surface, it is necessary to dig. A more complete answer, however, is
that there is a difference between the conditions in which an experiment is conducted
and the results of the experiment Just because an experiment takes place in an
unnatural environment does not necessarily imply that the results are unnatural.
For example, you are probably familiar with the law of gravity, which states that all
objects fall at the same rate independent of mass. You are no doubt equally familiar
with the "natural" fact that, if you drop a brick and a feather from the roof Of a
80
building, they will not fall at the same rate. Other factors in the natural world, such as
air resistance, conceal the true effects of gravity. To demonstrate the law of gravity,
we must create an artificial, controlled environment, specifically a vacuum, wherein
forces such as air resistance have been eliminated. This fact does not in validate the
law of gravity; the law accurately describes the underlying force of gravity and
explains the behavior of falling objects, even though natural conditions may conceal
the basic principle, in the same way, the goal of any experiment is to reveal the
natural underlying mechanisms and relationships that may be otherwise obscured.
Nonetheless, there is always a risk that the conditions of an experiment will be so
unnatural that the results are questionable. To use the terminology presented in
chapter 6, an experimenter can be so intent on ensuring internal validity that external
validity is compromised. Researchers are aware of this problem and have developed
techniques to increase the external validity (natural character) of experiments.
Learning Checks
It has been demonstrated that students with high self-esteem tend to have higher
grades than students with low self-esteem. Does this relationship mean that higher
self-esteem causes better academic performance? Does it mean that better academic
performance causes higher self-esteem? Explain your answer, and identify the general
problem that can preclude a cause and effect explanation.
A researcher would like to compare two methods for teaching math to third grade
students. Two third-grade classes are obtained for the study. Mr. Jones teaches one
class using method A and Mrs. Smith teaches other class using method B. At the end
of the year, the students from the method B class have significantly higher scores on a
mathematics achievement test. Does this result indicate that method B causes higher
81
scores than method A? Explain your answer, and identify the general problem that
precludes a cause and effect explanation.
Elements of an Experiment
The general goal of the experimental research strategy is to establish a cause and
effect relationship between two variables. That is, an experiment attempts to
demonstrate that changes (or differences) in one variable are directly responsible for
changes (or differences) in a second variable. This general goal can be broken down
into two sub goals:
1. It is not sufficient for an experiment to demonstrate that a relation exist
between two variables. In addition, an experiment must demonstrate the
character of the relationship. It must be shown that changes in one variables
cause changes in the other variable.
2. To establish that one specific variable is responsible for changes in another
variable, an experiment must rule out the possibility that the changes are
caused by some other variable.
These sub goals dictate the two characteristics that differentiate experiments from
other research strategies:
1. Manipulation of one variable while measuring a second variable
2. Control of other, extraneous variables
Definitions
The goal of the experimental research strategy is to establish the existence of a
cause-and-effect relationship between two variables. To accomplish this goal, an
experiment manipulates one variable while a second variable is measures and other
variables are controlled.
82
Manipulation and Measurement
A distinguishing characteristic of the experimental strategy is that the researcher
manipulates one of the variables under study. The primary purpose of manipulation is
to allow researchers to determine the direction of a relationship. Figure 8.1 shows two
situation in which there is a relationship between variables. In the top half of the
figure, there is a direct relation between the position of the switch and the brightness
of the light. By manipulation, we can demonstrate that the switch causes changes in
the light instead of the light causing changes in the switch. In general, when ever there
is a relationship between two variables, researchers can use manipulation to determine
which variable is the cause and which is the effect. For example, a researcher could
change the value of variable A (manipulation) and then observe what happens to
variable B. if the manipulation produces changes in B, the researcher can conclude
that variables A is the causal agent. But lack of change in variable B would indicate
that variable A is not a causal agent.
For example more closely related to psychology, consider the relationship between
depression and insomnia, it ahs been observed repeatedly that people suffering from
depression also tend to have problems sleeping. However, the observed relationship
does not answer the causal question, does depression cause sleep problems or does the
lack of sleep cause depression? Although it may be difficult to manipulate depression
directly, it certainly is possible to manipulate the amount of sleep, one group of
individuals, for example, could be allowed only 4 hours of sleep each night and a
comparison group allowed 8 hours. After a week, depression scores could be
obtained and compared for the two groups, if the 4 hour group is more depressed, this
is evidence that sleep deprivation causes depression.
83
A second purpose of manipulation is to give the researcher command of one of the
variables being studied. In the family planning study discussed earlier (Freedman,
Coombs, Chang, & Sun, 1974), the researchers observed a higher use of birth control
in homes where television was watched and a lower use in homes with no television.
However, the researchers had no power to determine which households watched
television and which did not. The lack of authority created two serious problems.
First, other variables naturally related to television watching were allowed to intrude
into the study. Second, the researchers had only vague definitions of watching
television and not watching television. In an experiment, the researchers could
manipulate the amount of television watching.
Theory
A coherent set of general propositions used as principles of explanation of the
apparent relationships of certain observed phenomena.
Hypothesis play an important role in the development of theory. While theory
development has not historically been an important aspect of business research, it is
becoming more so. A person not familiar with research uses the term theory to
express the opposite of fact. When you are too theoretical, your basis of explanation
or decision is not sufficiently attuned to specific empirical conditions. This may be so,
but it does not prove that theory and fact are opposites. The truth is that fact and
theory are each necessary for the other to be of value. Our ability to make rational
decisions, as well as to develop scientific knowledge, is measured by the degree to
which we combine fact and theory.
We all operate on the basis of theories we hold. In one sense, theories are the
generalizations we make about variables and the relationships among them. We use
these generalizations to make decisions and predict outcomes.
84
A theory is a set of systematically interrelated concepts, definitions, and
propositions that are advanced to explain and predict phenomena. In this sense, we
have many theories and use them continually to explain or predict what goes on
around us. To the degree that our theories are sound and fit the situation, we are
successful in our explanations and predictions. Thus, while a given theory and a set of
facts may not fit, they are not opposites. Our challenge is to build a better theory and
to be more skillful in fitting theory and fact together.
Theory and Research
It is important for the researchers to recognize the pervasiveness and value of
theory. Theory serves us in many useful ways:
Theory narrows the range of facts we need to study
Theory suggests which research approaches are likely to yield the greatest
meaning.
Theory suggests a system for the researcher to impose on data in order to
classify them in the most meaningful way.
Theory summarizes what is known about an object of study and states the
uniformities that lie beyond immediate observation.
Theory can be used to predict further facts that should be found.
Theories
Theories are nets cast to catch what we call “the world”: to rationalize, to explain,
and to master it. We endeavor to make the mesh ever finer and finer.
Karl R. Popper
Two Purposes Of Theory
Prediction
Understanding
85
Concept (or Construct)
A generalized idea about a class of objects, attributes, occurrences, or processes that
has been given a name.
Building blocks that abstract reality
“leadership,” “productivity,” and “morale”
“gross national product,” “asset,” and “inflation”
A Ladder Of Abstraction
For Concepts
act
abstr
more
y
asingl
Incre
Vegetation
Fruit
Banana
Reality
Scientific Business Researchers Operate at Two Levels
Abstract level
Concepts
Propositions
Empirical level
Variables
Hypotheses
Definitions: Abstract level -In theory development, the level of knowledge
expressing a concept that exists only as an idea or a quality apart from an object.
86
Empirical level -Level of knowledge reflecting that which is verifiable by experience
or observation.
Concepts are Abstractions of Reality
Abstract
Level CONCEPTS
Empirical OBSERVATION OF OBJECTS

Level AND EVENTS (REALITY)
Theories
act
abstr
more
ly
asing
Incre
Propositions
Concepts
Observation of objects
Scientific Method
The use of a set of prescribed procedures for establishing and connecting theoretical
statements about events and for predicting events yet unknown.
87
Abstract Level
Concepts abstract reality.
Propositions are statements concerned with the relationships among concepts.
Proposition at Abstract Level
Concept A Concept B
(Reinforcement) (Habits)
Hypothesis at Empirical Level
Dollar bonus for sales Always makes four

volume over quota sales calls a day
Theoretical Framework:
Research is always conducted within an explicit or implicit theoretical framework,
therefore it is useful to work at developing the framework and make it more explicit.
Working within an explicit framework allows the formulation of research questions
and also creates a ‘frame of reference’ in terms of the research process and the
application of research principles, thus a way forward.
Understanding Theory: Components and Connections
When we do research, we seek to know what is in order to understand, explain and
predict phenomena. We might want to answer the question, “What will be the
employee reaction to the new flexible work schedule?” or “Why did the stock market
price surge higher when all normal indicators suggested it would go down?” when
dealing with such questions, we must agree on definitions. Which employees? What
kind of reaction? What are the normal indicators? These questions require the use of
concepts, construct and definitions.
Concepts
88
A concept is a generally accepted collection of meanings or characteristics
associated with certain events, objects, conditions, situations and behaviors.
Classifying and categorizing objects or events that have common characteristics
beyond any single observation create concepts. We abstract such meanings from
reality and use words as labels to designate them. For example, we see a man passing
and identify that he is running, walking, skipping, crawling, or hopping. These
movements all represent concepts.
Importance to Research
Concepts are basic to all thought and communication, yet in everyday use we pay
little attention to the problems encountered in their use. In research special problems
grow out of the need for concept precision and inventiveness. We design hypothesis
using concepts. We devise measurement concepts by which to test these hypothetical
statements. We gather data using these measurement concepts. We may even invent
new concepts to express ideas. The success of research hinges on (1) how clearly we
conceptualize and (2) how well others understand the concepts we use.
Hypothesis
An unproven proposition.
A possible solution to a problem.
Guess.
A testable hypothesis is one in which all of the variables, events, and individuals are
real and can be defined and observed.
A refutable hypothesis is a hypothesis that can be demonstrated to be false. That is,
the hypothesis allows the possibility that the outcome will differ from the prediction.
Consider the following hypothesis, which are not testable or refutable:
89
Hypothesis: The more sins a person commits, the less likely he or she is to get into
heaven.
Hypothesis: Abortion is morally wrong.
Hypothesis: If people could fly, there would be substantially fewer cases of
depression.
Hypothesis: The human mind emits thought waves that influence other people but
cannot be measured or recorded in any way.
Although you may find these hypotheses interesting, they cannot tested or shown
to be false and therefore are unsuitable for scientific research. In general hypotheses
that deal with moral or religious issues, value judgments, or hypothetical situations
are untestable or nonrefutable. However, this does not mean that religion, morals, or
human values are off limits for scientific research. You could, for example, compare
the personality characteristics or family backgrounds of religious and nonreligious
people, or you could look for behavioral differences between "pro-life" individuals
and "pro-choice" individuals. Nearly any topic can be studied scientifically if you take
care to develop testable and refutable hypotheses.
An hypothesis is a specific statement of prediction. It describes in concrete (rather
than theoretical) terms what you expect will happen in your study. Not all studies
have hypotheses. Sometimes a study is designed to be exploratory (see inductive
research). There is no formal hypothesis, and perhaps the purpose of the study is to
explore some area more thoroughly in order to develop some specific hypothesis or
prediction that can be tested in future research. A single study may have one or many
hypotheses.
Actually, whenever I talk about an hypothesis, I am really thinking
simultaneously about two hypotheses. Let's say that you predict that there will be a
90
relationship between two variables in your study. The way we would formally set up
the hypothesis test is to formulate two hypothesis statements, one that describes your
prediction and one that describes all the other possible outcomes with respect to the
hypothesized relationship. Your prediction is that variable A and variable B will be
related (you don't care whether it's a positive or negative relationship). Then the only
other possible outcome would be that variable A and variable B are not related.
Usually, we call the hypothesis that you support (your prediction) the alternative
hypothesis, and we call the hypothesis that describes the remaining possible outcomes
the null hypothesis. Sometimes we use a notation like HA or H1 to represent the
alternative hypothesis or your prediction, and HO or H0 to represent the null case. You
have to be careful here, though. In some studies, your prediction might very well be
that there will be no difference or change. In this case, you are essentially trying to
find support for the null hypothesis and you are opposed to the alternative.
If your prediction specifies a direction, and the null therefore is the no difference
prediction and the prediction of the opposite direction, we call this a one-tailed
hypothesis. For instance, let's imagine that you are
investigating the effects of a new employee
training program and that you believe one of the
outcomes will be that there will be less employee
absenteeism. Your two hypotheses might be stated something like this:
The null hypothesis for this study is:
HO: As a result of the XYZ company employee training program, there will either be
no significant difference in employee absenteeism or there will be a significant
increase.
which is tested against the alternative hypothesis:
91
HA: As a result of the XYZ company employee training program, there will be a
significant decrease in employee absenteeism.
In the figure on the left, we see this situation illustrated graphically. The alternative
hypothesis -- your prediction that the program will decrease absenteeism -- is shown
there. The null must account for the other two possible conditions: no difference, or
an increase in absenteeism. The figure shows a
hypothetical distribution of absenteeism differences.
We can see that the term "one-tailed" refers to the
tail of the distribution on the outcome variable.
When your prediction does not specify a direction, we say you have a two-tailed
hypothesis. For instance, let's assume you are studying a new drug treatment for
depression. The drug has gone through some initial animal trials, but has not yet been
tested on humans. You believe (based on theory and the previous research) that the
drug will have an effect, but you are not confident enough to hypothesize a direction
and say the drug will reduce depression (after all, you've seen more than enough
promising drug treatments come along that eventually were shown to have severe side
effects that actually worsened symptoms). In this case, you might state the two
hypotheses like this:
The null hypothesis for this study is:
HO: As a result of 300mg./day of the ABC drug, there will be no significant difference
in depression.
which is tested against the alternative hypothesis:
HA: As a result of 300mg./day of the ABC drug, there will be a significant difference
in depression.
92
The figure on the right illustrates this two-tailed prediction for this case. Again,
notice that the term "two-tailed" refers to the tails of the distribution for your outcome
variable.
The important thing to remember about stating hypotheses is that you formulate
your prediction (directional or not), and then you formulate a second hypothesis that
is mutually exclusive of the first and incorporates all possible alternative outcomes for
that case. When your study analysis is completed, the idea is that you will have to
choose between the two hypotheses. If your prediction was correct, then you would
(usually) reject the null hypothesis and accept the alternative. If your original
prediction was not supported in the data, then you will accept the null hypothesis and
reject the alternative. The logic of hypothesis testing is based on these two basic
principles:
the formulation of two mutually exclusive hypothesis statements that, together,
exhaust all possible outcomes
the testing of these so that one is necessarily accepted and the other rejected
OK, I know it's a convoluted, awkward and formalistic way to ask research questions.
But it encompasses a long tradition in statistics called the hypothetical-deductive
model, and sometimes we just have to do things because they're traditions. And
anyway, if all of this hypothesis testing was easy enough so anybody could
understand it, how do you think statisticians would stay employed?
Propositions and Hypotheses
We define a proposition as a statement about concepts that may be judged as true
or false if it refers to observable phenomena. When a proposition is formulated for
empirical testing, we call it a hypothesis. As a declarative statement, a hypothesis is of
a tentative and conjectural nature.
93
Hypotheses have also been described as statements in which we assign variables
to cases. A case is defined in this sense as the entity or thing the hypothesis talks
about. The variable is the characteristics, trait, or attribute that, in the hypothesis, is
imputed to the case. For example, we might form the following hypothesis:
Executive Jones (case) has a higher-than-average achievement motivation(variable).
Descriptive Hypothesis
These are the propositions that typically state the existence, size, form, or distribution
of some variable.
Executive Jones (case) has a higher-than-average achievement motivation(variable).
Relational Hypothesis
The research question format is less frequently used with a situation calling for
relational hypotheses. These are statements that describe a relationship between two
variables with respect to some case.
Correlational Hypothesis state merely that variables occur together in some
specified manner without implying that one causes the other.
The height of women’s hemlines varies directly with the level of the business cycle.
With explanatory(causal) hypotheses, there is an implication that the existence of,
or a change in, one variable causes or leads to a change in the other variable. As we
noted previously, the causal variable is typically called the independent variable and
the other the dependent variable. For example:
An increase in family income(IV) leads to an increase in the percentage of income
saved.(DV)
The Role of the Hypothesis:
In research, a hypotheses serves several important functions:
It guides the direction of the study.
94
It identifies facts that are relevant and those that are not.
It suggests which form of research design is likely to be most appropriate.
It provides a framework for organizing the conclusions that result.
To consider specifically the role of the hypothesis in determining the direction of
the research, suppose we use this:
Husbands and wives agree in their perceptions of their respective roles in purchase
decisions.
The hypothesis specifies who shall be studied (married couples), in what context
they shall be studied (their consumer decision making), and what shall be studied
(their individual perceptions of their roles).
The nature of this hypothesis and the implications of the statement suggest that
the best research design is a communication-based study, probably a survey or
interview. We have at this time no other practical means to ascertain perceptions of
people except to ask about them in one way or another. In addition, we are interested
only in the roles that are assumed in the purchase or consumer decision making
situation. The study should not, therefore, involve itself in seeking information about
other types of roles husbands and wives might play.
What is a good hypothesis?
A good hypothesis should fulfill three conditions:
Adequacy for its purpose
Testable
Better than its rivals
For a descriptive hypothesis, adequacy for its purpose means it clearly states the
condition, size, or distribution of some variable in terms of values meaningful to the
research task. If it is an explanatory hypothesis, it must explain the facts that gave rise
95
to the need for explanation. Using the hypothesis, plus other known and accepted
generalizations, one should be able to deduce the original problem condition.
A hypothesis is testable if it meets the following conditions.
It does not require techniques that are unavailable with the present state of the
research art.
It does not require an explanation that defies known physical or psychological
laws.
There are consequences or derivatives that can be deduced for testing purposes.
Research Questions
Having chosen your research area the next step is to narrow this area down to the
imperative questions or hypothesis. This area involves narrowing down from the Area
(Broad field of scholarly endeavor) to the Field (Component of the area) and then to
the Aspect (A detailed aspect within a field). Utilizing this approach, we can
implement morphological analysis to narrow down and decide on a specific area and
topic.
Bear in mind that research must be able to teach you something. You begin by
asking the initial questions and then through initial research you refine and formulate
a hypothesis. This process may involve consultation with colleagues, experts I the
field, performing a pilot study and measuring and designing initial research tools.
Bear in mind that the key at this stage is to make the research methods for the
question and not the other way around.
Some of the main types of research questions as espoused by Barker et al (1994)
are: Definition questions: What is the nature of ‘x’ or ‘y’. The aim is to develop a
clearer understanding of what something actually is.
96
Description questions: What are the features, attributes or history of ‘x’ or ‘y’. This
allows a greater understanding of the subject matter.
Amount/Frequency/Intensity questions: How much of ‘x’ exists, what’s the frequency
of occurrence and what’ its intensity. These are questions of context.
Co-variation questions: This is related to relationships of variables within a study. If
we increase ‘x’, what is the effect on ‘y’.
Comparison questions: Does ‘x’ have more ‘a’ then ‘y’. In other words ‘Do
chronically depressed patients have a greater chance of relapse when reduced
medication is applied’?
Measurement questions: How well (reliably, validity and usefully) can ‘x’ be
measured by a measure of ‘m’. You will use a range of these questions during your
research phase and you will revisit the questions many times….
Discovery- orientated or confirmatory research questions (hypothesis) :
Expressing a research question as a question or in a formulated statement - is a
hypothesis to be tested. These types of questions give an immediate clarity to a
research project. Using hypothesis also has the merit of increasing precision and
fitting in more closely with the theory of statistical inference (Howell, 1992)
Hypotheses are generally more effective under the following circumstances:
• When a research area is new
• When a research area is disordered and not moving forward
• When the topic is a high topical area requiring definition
Correlational Research
Correlational Research Introduction This paper reviews correlational research.
The points of focus in the review are (1) defining of the research approach, (2)
identifying an organizational situation amendable to study through correlational
97
research, (3) stating managerial questions appropriate for investigation through
correlational research, and (4) describing data collection methods that are appropriate
for correlational research. Defining Correlational Research Correlation quite literally
is a measure of association between two variables. Correlational research more
specifically refers to a measure of linear association between two variables. The range
of the linear relationship between the variables, represented by a coefficient
managerial questions appropriate for investigation through correlational research a
measure of linear association between a positive relationship wherein the values of the
change A zero correlation coefficient indicates a complete of change in the other
variable direction of the relationship Cooper and Schindler state that a of the
relationship that exists between To accept the position of Cooper between variables
but does not establish a causal link between an Organizational Situation An
organizational situation that would general economy In such an analysis there is no
the two variables establish the directional characteristic of the relationship between
Carter Neter Kutner Nachtsheim Wasserman Bowalekar With respect to the more
difficult to purchase all other relevant factors purchasing power of wages leads to a
reduction in the increase in demand for retail goods and in turn for the same specified
periods Question Based on the situation described in the will increase from three-
percent annual basis to determine the relationship between the rate of support reduced
expectations for the level of retail sales Describing the various points on the scale are
the same interval Thus regression analysis Journal of Postgraduate Medicine Carter G
Winter of the research approach identifying an organizational situation Research
Correlation quite literally is a measure of association from to The sign of the measure
indicates the direction sign indicates a negative relationship wherein the values of
variables e g any given change in the value of correlation coefficient indicates the
98
strength of the relationship between and mathematics is that correlational analysis
provides and Schindler provide a long discussion on causal analysis the issue Thus
business researchers should adhere to the widely accepted number the Pearson
product moment correlation coefficient defines both the impact on the level of retail
sales of change in level of retail sales Rather all that management need to assess As a
general rule a correlation coefficient is necessary to establish the relationship areas
follows An increase in wages to decrease all other relevant factors inflation has the
opposite effects Therefore a decrease in the rate the rate of inflation for specified
predict changes in the level of retail sales based projections Considering that the
Bureau of Economic inventory levels because of anticipate deductions in turn will
permit organizational management to determine how much in character Therefore the
collection of data must provide any other point on the scale References Bowalekar S
K W Applied linear regression models rd ed Chicago correlational research
Introduction This paper reviews correlational research and describing data collection
methods that two variables The range of the two variables move in the same direction
absence of any correlational relationship between the variables A correlation
coefficient Almost all correlation coefficients however fall between the extremes
controversy exists as to the capability of correlational analysis to variables but that
correlation analysis cannot establish a causal relationship between and Schindler
would place business-oriented statistical analysis outside the two variables
Correlational research is be amendable to analysis through the application need for
organizational management to establish a the rate of inflation and retain sales and
measure relationship between the rate of remaining unchanged because An increase in
the rate level of retail sales The an increase in the level of retail sales over time The
correlation coefficient result from the application of correlation preceding discussion
99
the managerial question for which correlational research seven-percent annual basis
over the coming inflation and the level of retail sales The projected Data Collection
Methods For correlation analysis the measurement of measures at any point on the
Dueling statistics Forum for Applied Research and Public Policy Neter J amendable
to study through correlational research stating between two variables Correlational
research more specifically refers to of the correlational relationship A plus sign
indicates the presence of the two variables move in divergent directions when they
one variable reflects the exact magnitude the two variables regardless of the excellent
measure of the magnitude and direction that really does not address the mathematical
issues position that correlational research measures the strength of relationships
magnitude and the direction of a relationship between two variables Identifying the
rate of inflation in the relationship is to determine that a measurable correlation exists
between a reliable relationship between two variables the rate of inflation causes retail
goods to be remaining equal and A decrease in the of inflation other relevant factors
remaining unchanged leads to an periods over time and the level of retail sales for
changes in the rate of inflation Stating a Managerial Research is predictingthat the
rate of inflation the level of retain sales The correlational research will to reduce
inventory levels to values on a scale wherein the differences between Statistics in
medical research III Correlation and Richard D Irwin Inc The points of focus in the
review are defining are appropriate for correlational research Defining Correlational
linear relationship between the variables represented by a coefficient is when they
change A minus indicates a perfect correlation between the two of magnitude The
magnitude of the establish a causal relationship The widely accepted position in
scientific the variables Carter Neter Kutner Nachtsheim Wasserman Bowalekar
Cooper the boundaries of the accepted scientific and mathematical position on a
100
powerful tool for researchers Correlation analysis with a single of correlational
research is the projection of the probable causal relationship between the rate of
inflation and the magnitude of the relationship between the two variables inflation and
the level of retail sales the underlying assumptions for of inflation causes the
purchasing power of assumption is that a decrease in the rate of To examine this
situation an organization would collect data relevant to analysis to the data collected
would allow organizational management to could develop an answer could be as
follows six months to how much should the company reduce change in the level of
retail sales the variables analyzed must be interval scale are comparable to measures
at Kutner M H Nachtsheim C J Wasserman managerial questions appropriate for
investigation through correlational research a measure of linear association between a
positive relationship wherein the values of the change A zero correlation coefficient
indicates a complete of change in the other variable direction of the relationship
Cooper and Schindler state that a of the relationship that exists between To accept the
position of Cooper between variables but does not establish a causal link between an
Organizational Situation An organizational situation that would general economy In
such an analysis there is no the two variables establish the directional characteristic of
the relationship between Carter Neter Kutner Nachtsheim Wasserman Bowalekar
With respect to the more difficult to purchase all other relevant factors purchasing
power of wages leads to a reduction in the increase in demand for retail goods and in
turn for the same specified periods Question Based on the situation described in the
will increase from three-percent annual basis to determine the relationship between
the rate of support reduce the expectations for the level of retail sales Describing the
various points on the scale are the same interval Thus regression analysis Journal of
Postgraduate Medicine Carter G Winter of the research approach identifying an
101
organizational situation Research Correlation quite literally is a measure of
association from to The sign of the measure indicates the direction sign indicates a
negative relationship wherein the values of variables e g any given change in the
value of correlation coefficient indicates the strength of the relationship between and
mathematics is that correlational analysis provides and Schindler provide a long
discussion on causal analysis the issue Thus business researchers should adhere to the
widely accepted number the Pearson product moment correlation coefficient defines
both the impact on the level of retail sales of change in level of retail sales Rather all
that management need to assess As a general rule a correlation coefficient is
necessary to establish the relationship areas follows An increase in wages to decrease
all other relevant factors inflation has the opposite effects Therefore a decrease in the
rate the rate of inflation for specified predict changes in the level of retail sales based
projections Considering that the Bureau of Economic inventory levels because of
anticipate deductions in turn will permit organizational management to determine how
much in character.
A hypothesis is a proposition that is empirically testable. It is an empirical statement
concerned with the relationship among variables.
A variable is anything that may assume different numerical values.
Theory and Song
A fact without a theory
Is like a ship without a sail, Is like a boat without a rudder,
Is like a kite without a tail. A fact without a figure is a tragic final act,
But one thing worse in this universe Is a theory without a fact.
Formulating the Research Problem
102
Researchers organize their research by formulating and defining a research
problem. This helps them focus the research process so that they can draw
conclusions reflecting the real world in the best possible way.
Hypothesis
In research, a hypothesis is a suggested explanation of a phenomenon. A null
hypothesis is a hypothesis which a researcher tries to disprove. Normally, the null
hypothesis represents the current view/explanation of an aspect of the world that the
researcher wants to challenge.
Research methodology involves the researcher providing an alternative hypothesis, a
research hypothesis, as an alternate way to explain the phenomenon.
The researcher tests the hypothesis to disprove the null hypothesis, not because
he/she loves the research hypothesis, but because it would mean coming closer to
finding an answer to a specific problem. The research hypothesis is often based on
observations that evoke suspicion that the null hypothesis is not always correct.
In the Stanley Milgram Experiment, the a null hypothesis was that the personality
determined whether a person would hurt another person, while the research
hypothesis was that the role, instructions and orders were much more important in
determining whether people would hurt others.
Hypothesis Development and Testing
In this module we will be examining Kids Count data and identifying how we can
use the available indicators of status of children in the United States for statistical
analysis. Kids Count is a project of the Annie E. Casey Foundation that tracks
children on both a national and state-by-state basis measuring the educational,
social, economic, and physical well-being of children. Data collected is available
on-line and is used by a variety of individuals and organizations involved with
103
projects such as welfare guidelines, health care initiatives, educational programs,
and the development of a system of policy supports that can help parents
become more successful both as workers and as parents.
Along with the data we have tools available that allow us to utilize both
descriptive statistics and inferential statistics. Descriptive statistics are
procedures for summarizing, organizing, graphing, and in general, describing
quantitative information. Inferential statistics is used to make inferences about a
population based on information about a sample drawn from that population.
1. The first step of this exercise is to identify a research question. The
specification of a research topic should move from a general discussion of
an area of interest to a more narrowly defined issue. Answer the following
three questions:
a. What is the general issue or interest area on which you are writing?
b. What are the specific questions that you are asking or issues you intend to
explore?
c. Why are these issues/questions important and sociologically interesting?
2. The next step of the exercise is to formulate a hypothesis. Hypotheses are
statements of (or conjecture about) the relationships among the variables
that a researcher intends to study. Hypotheses are generally testable
statements of relations. In such cases, they are thought of as predictions,
which, if confirmed, will support a theory.
3. After writing your hypothesis, identify the dependent and independent
variables that you will use to test your hypothesis. The dependent variable
is defined as the presumed effect in a study; so called because it “depends” on another
variable. It is the variable whose values are predicted by the independent variable,
104
whether or not caused by it. For example, in a study to see if there is a relationship
between the teen violent crime arrest rates and the teen dropout rate the teen violent
crime arrest rates might be the effect (dependent variable).
4. The independent variables are the presumed causes in a study. Independent
variables are the variables that can be used to predict the values of another variable.*
In the above example, the independent variable might be the teen dropout rate.
5. When identifying dependent and independent variables the question of cause is
preeminent. Cause is defined as an event, such as a change in one variable that
produces another event, such as a change in a second variable. Be warned that there is
no concept in statistics that is more troublesome than “cause.” Researchers may
disagree about what constitutes a cause and especially about how restrictive a set of
conditions must be met before it is legitimate to talk of cause. Many social scientists
would agree with the following – others would not: to attribute cause, for X to cause
Y, three conditions are necessary (but not sufficient): (1) X must precede Y; (2) X and
Y must covary; (3) no rival explanations account as well for the covariance of W and
Y. Causal relations may be simple or multiple. In simple causation, whenever the first
event (the cause/independent variable) happens, the second (the effect/dependent
variable) always does too. Multiple causation is much more common in the social and
behavioral sciences.* Multiple causes may be such that any one of several causes can
produce the same effect (for example, teen violent crime arrest rates may be caused
by teen dropout, children living in poverty, teen birth rate or some combination of the
three). Multiple causes also may be such that no one of them will necessarily produce
the effect, but several of them in combination make it more likely.
105
6. Examine your dependent variable as a trend over time. Choose the United States
and one or two other states for your line graph. Consider choosing one of the worst of
best-ranked states as a comparison.
Choose your dependent variable as your indicator. Graph the data from 1990-1999.
Describe the overall national trend. Was there an increase, decrease, or did it stay the
same. Put another way, nationally is your selected variable on the rise or is it
decreasing?
7. Do the same for your independent variable(s).
8. To test the relationship between your dependent variable and the independent
variable(s), open the excel file called "tool_us.xls". Make a scatter plot by using the
pull down menu. Let x be the independent variable (plotted on the x axis) and y be the
independent variable (plotted on the y axis). Cut and paste the scatter plot into a Word
file and record the correlation coefficient. (An explanation of the correlation
coefficient can be found below)
Are there any data points that seem to stand out --not part of the cluster of data
points? These are called outliers. Click on an outlier to see which state is represented.
Was the hypothesis confirmed? Explain your answer.
9. Repeat step 8 for each of your independent variables.
Explanation of Correlation Coefficient
The correlation coefficient or Pearson's r is a measure of the degree of linear
association existing between two variables. We want to pay close attention to both the
direction and strength of the association. A positive correlation is indicated by the
absence of a negative sign and means that variables are changing in the same
direction. An increase or decrease in one variable corresponds to the same change in
106
another variable. For example, we would expect that the more time a students studies
for an exam (x) the higher the exam score (y). A negative relationship is indicated by
a minus sign and means that as one variable increases there is a corresponding
decrease in another variable. The strength of a relationship is indicated by the numeric
value of the coefficient. Coefficients range from 1.0 to 1.0. These values are examples
of perfect correlations. In reality most values are found in between 1.0 and 1.0.
Correlations of .30 or less (either + or -) are considered weak, .31 - .70 (either + or -)
are deemed moderate and .71 and above (either + or -) considered strong. These are
not absolute rules but should be used as a guide in interpretation. Note that the higher
the correlation coefficient (either positive or negative), the more closely clustered the
data points are in the shape of a diagonal line. What might be a better measure of your
dependent variable than the one(s) you tested?
Devising and Improving a Hypothesis
A formal statement of the research question is often called the hypothesis. The
hypothesis should be stated in a way such that a “true” or “false” answer from an
experiment would support or refute the hypothesis. The hypothesis focuses the
experimental procedures and, more importantly, helps define the control group.
Careful selection of a control group allows the most appropriate and powerful
analysis, to determine the truth of the hypothesis and the value of the experiment.
Consideration of the control group at the time of hypothesis development will make
all of the steps in the research process easier and more powerful. The most common
problem in beginning a study is not defining the appropriate control group. This
article describes how to identify research questions and how to develop hypotheses in
ways that make research, data analysis, abstract presentation, and publication easier.
Conceptual Variables
107
Validity and Reliability
Validity refers to what degree the research reflects the given research problem.
Types of validity:
External Validity
Population Validity
Ecological Validity
Internal Validity
Content Validity
Face Validity
Construct Validity
Convergent and Discriminant Validity
Test Validity
Criterion Validity
Concurrent Validity
Predictive Validity
Reliability
Reliability refers to how consistent a set of measurements are. Reliability may be
defined as "Yielding the same or compatible results in different clinical experiments
or statistical trials" (the free dictionary). Research methodology lacking reliability
cannot be trusted. Replication studies are a way to test reliability.
Types of Reliability:
Test-Retest Reliability
Interrater Reliability
Internal Consistency Reliability
Instrument Reliability
108
Statistical Reliability
Reproducability
Both validity and reliability are important aspects of the research methodology to
get better explanations of the world.
Generalization
Generalization is to which extent the research and the conclusions of the research
apply to the real world. It is not always so that good research will reflect the real
world, since we can only measure a small portion of the population at a time.
Choosing the Research Method
The selection of the research method is crucial for what conclusions you can
make about a phenomenon. It affects what you can say about the cause and factors
influencing the phenomenon.
It is also important to choose a research method which is within the limits of what
the researcher can do. Time, money, feasibility, ethics and availability to measure the
phenomenon correctly are examples of issues constraining the research.
Choosing the Measurement
Choosing the scientific measurements are also crucial for getting the correct
conclusion. Some measurements might not reflect the real world, because they do not
measure the phenomenon as it should.
Significance Test
To test a hypothesis, quantitative research uses significance tests to determine
which hypothesis is right.
The significance test can show whether the null hypothesis is more likely correct
than the research hypothesis. Research methodology in a number of areas like social
sciences depends heavily on significance test.
109
A significance test may even drive the research process in a whole new direction,
based on the findings.
The t-test (also called the Student's T-Test) is one of many statistical significance
tests, which compares two supposedly equal sets of data to see if they really are alike
or not. The t-test helps the researcher conclude whether a hypothesis is supported or
not.
Drawing Conclusions
Drawing a conclusion is based on several factors of the research process, not just
because the researcher got the expected result. It has to be based on the validity and
reliability of the measurement, how good the measurement was to reflect the real
world and what more could have affected the results.
The observations are often referred to as 'empirical evidence' and the
logic/thinking leads to the conclusions. Anyone should be able to check the
observation and logic, to see if they also reach the same conclusions. Errors of the
observations may stem from measurement-problems, misinterpretations, unlikely
random events etc.
A common logical error for beginners, is to think that correlation implies a causal
relationship. This is not necessarily true.
Errors in Research
Logically, there are possible to make two types of errors when drawing conclusions in
research:
Type 1 error is when we accept the research hypothesis when the null hypothesis is
in fact correct.
Type 2 error is when we reject the research hypothesis even if the null hypothesis is
wrong. By Experiment-Resources.com (2008).
110
Experimental Research
Experimental research is commonly used in sciences such as sociology and
psychology, physics, chemistry, biology and medicine etc.
It is a collection of research designs which use manipulation and controlled testing to
understand causal processes. Generally, one or more variables are manipulated to
determine their effect on a dependent variable.
The experimental method is a systematic and scientific approach to research in
which the researcher manipulates one or more variables, and controls and measures
any change in other variables.
Experimental Research is often used where:
There is time priority in a causal relationship (cause precedes effect)
There is consistency in a causal relationship (a cause will always lead to the same
effect)
The magnitude of the correlation is great.
The word experimental research has a range of definitions. In the strict sense,
experimental research is what we call a true experiment.
This is an experiment where the researcher manipulates one variable, and
control/randomizes the rest of the variables. It has a control group, the subjects have
been randomly assigned between the groups, and the researcher only tests one effect
at a time. It is also important to know what variable(s) you want to test and measure.
A very wide definition of experimental research, or a quasi experiment, is research
where the scientist actively influences something to observe the consequences. Most
experiments tend to fall in between the strict and the wide definition.
111
A rule of thumb is that physical sciences, such as physics, chemistry and geology tend
to define experiments more narrowly than social sciences, such as sociology and
psychology, which conduct experiments closer to the wider definition.
Aims of Experimental Research
Experiments are conducted to be able to predict phenomenons. Typically, an
experiment is constructed to be able to explain some kind of causation. Experimental
research is important to society - it helps us to improve our everyday lives.
Identifying the Research Problem
After deciding the topic of interest, the researcher tries to define the research
problem. This helps the researcher to focus on a more narrow research area to be able
to study it appropriately.
The research problem is often operationalizationed, to define how to measure the
research problem. The results will depend on the exact measurements that the
researcher chooses and may be operationalized differently in another study to test the
main conclusions of the study.
Defining the research problem helps you to formulate a research hypothesis,
which is tested against the null hypothesis.
An ad hoc analysis is a hypothesis invented after testing is done, to try to explain
why the contrary evidence. A poor ad hoc analysis may be seen as the researcher's
inability to accept that his/her hypothesis is wrong, while a great ad hoc analysis may
lead to more testing and possibly a significant discovery.
Constructing the Experiment
There are various aspects to remember when constructing an experiment.
Planning ahead ensures that the experiment is carried out properly and that the results
reflect the real world, in the best possible way.
112
Sampling Groups to Study
Sampling groups correctly is especially important when we have more than one
condition in the experiment. One sample group often serves as a control group, whilst
others are tested under the experimental conditions.
Deciding the sample groups can be done in using many different sampling
techniques. Population sampling may chosen by a number of methods, such as
randomization, "quasi-randomization" and pairing.
Reducing sampling errors is vital for getting valid results from experiments.
Researchers often adjust the sample size to minimize chances of random errors.
Here are some common sampling techniques:
Probability sampling
Non-probability sampling
Simple random sampling
Convenience sampling
Stratified sampling
Systematic sampling
Cluster sampling
Sequential sampling
Disproportional sampling
Judgmental sampling
Snowball sampling
Quota sampling
Creating the Design
The research design is chosen based on a range of factors. Important factors when
choosing the design are feasibility, time, cost, ethics, measurement problems and what
113
you would like to test. The design of the experiment is critical for the validity of the
results.
Typical Designs and Features in Experimental Design
Pretest-Posttest Design
Check whether the groups are different before the manipulation starts and the effect of
the manipulation. Pretests sometimes influence the effect.
Control Group
Control groups are designed to measure research bias and measurement effects,
such as the Hawthorne Effect or the Placebo Effect. A control group is a group not
receiving the same manipulation as the experimental group.
Experiments frequently have 2 conditions, but rarely more than 3 conditions at the
same time.
Randomized Controlled Trials
Randomized Sampling, comparison between an Experimental Group and a Control
Group and strict control/randomization of all other variables
Solomon Four-Group Design
With two control groups and two experimental groups. Half the groups have a pretest
and half do not have a pretest. This to test both the effect itself and the effect of the
pretest.
Between Subjects Design
Grouping Participants to Different Conditions
Within Subject Design
Participants Take Part in the Different Conditions - See also: Repeated Measures
Design.
114
Counterbalanced Measures Design
Testing the effect of the order of treatments when no control group is
available/ethical.
Matched Subjects Design
Matching Participants to Create Similar Experimental- and Control-Groups
Double-Blind Experiment
Neither the researcher, nor the participants, know which is the control group. The
results can be affected if the researcher or participants know this.
Bayesian Probability
Using bayesian probability to "interact" with participants is a more "advanced"
experimental design. It can be used for settings were there are many variables which
are hard to isolate. The researcher starts with a set of initial beliefs, and tries to adjust
them to how participants have responded
Pilot Study
It may be wise to first conduct a pilot-study or two before you do the real
experiment. This ensures that the experiment measures what it should, and that
everything is set up right.
Minor errors, which could potentially destroy the experiment, are often found
during this process. With a pilot study, you can get information about errors and
problems, and improve the design, before putting a lot of effort into the real
experiment.
If the experiments involve humans, a common strategy is to first have a pilot study
with someone involved in the research, but not too closely, and then arrange a pilot
with a person who resembles the subject(s). Those two different pilots are likely to
give the researcher good information about any problems in the experiment.
115
Conducting the Experiment
An experiment is typically carried out by manipulating a variable, called the
independent variable, affecting the experimental group. The effect that the researcher
is interested in, the dependent variable(s), is measured.
Identifying and controlling non-experimental factors which the researcher does
not want to influence the effects, is crucial to drawing a valid conclusion. This is often
done by controlling variables, if possible, or randomizing variables to minimize
effects that can be traced back to third variables. Researchers only want to measure
the effect of the independent variable(s) when conducting an experiment, allowing
them to conclude that this was the reason for the effect.
Analysis and Conclusions
In quantitative research, the amount of data measured can be enormous. Data not
prepared to be analyzed is called "raw data". The raw data is often summarized as
something called "output data", which typically consists of one line per subject (or
item). A cell of the output data is, for example, an average of an effect in many trials
for a subject. The output data is used for statistical analysis, e.g. significance tests, to
see if there really is an effect.
The aim of an analysis is to draw a conclusion, together with other observations.
The researcher might generalize the results to a wider phenomenon, if there is no
indication of confounding variables "polluting" the results.
If the researcher suspects that the effect stems from a different variable than the
independent variable, further investigation is needed to gauge the validity of the
results. An experiment is often conducted because the scientist wants to know if the
independent variable is having any effect upon the dependent variable. Variables
correlating are not proof that there is causation.
116
Experiments are more often of quantitative nature than qualitative nature, although it
happens.
Types of Research Designs
Different types of research designs have different advantages and disadvantages.
The design is the structure of any scientific work. It gives direction and systematizes
the research.
The method you choose will affect your results and how you conclude the
findings. Most scientists are interested in getting reliable observations that can help
the understanding of a phenomenon.
There are two main approaches to a research problem:
Quantitative Research
Qualitative Research
What are the difference between Qualitative and Quantitative Research?
Different Research Methods
There are various designs which are used in research, all with specific advantages and
disadvantages:
True Experimental Design
Quasi-Experimental Design
Double-Blind Experiment
Descriptive Research
Literature Review
Case Study
Survey
Twin Studies
Meta-analysis
117
Systematic Reviews
Observational Study
Naturalistic Observation
Field Experiment
Cohort Study
Longitudinal Study
Cross Sectional Study
Factorial Design
Case Control Study
Pilot Study
Which Method to Choose?
What design you choose depends on different factors.
What information do you want?
Feasibility
How reliable should the information be?
Is it ethical to conduct the study?
The cost of the design By Experiment-Resources.com (2008).
Ethics in Research
Ethics in research are very important when you're going to conduct an
experiment. They apply when you are planning, conducting and evaluating research.
The first thing to do before designing a study is to consider the potential cost and
benefits of the research.
Research - Cost and Benefits-Analysis
We evaluate the cost and benefits for most decisions in life, whether we are aware of
it or not.
118
This can be quite a dilemma in some experiments. Stem cell research is one example
of an area with difficult ethical considerations.
As a result, stem cell research is restricted in many countries, because of the major
and problematic ethical issues.
Ethical Standards - Researchers should
avoid any risk of considerably harming people, the environment, or property
unnecessarily. The Tuskegee Syphilis Study is an example of a study which seriously
violated these standards.
not use deception on people participating, as was the case with the ethics of the
Stanley Milgram Experiment
obtain informed consent from all involved in the study.
preserve privacy and confidentiality whenever possible.
take special precautions when involving populations or animals which may not be
considered to understand fully the purpose of the study.
not offer big rewards or enforce binding contracts for the study. This is especially
important when people are somehow reliant on the reward.
not plagiarize the work of others
not skew their conclusions based on funding.
not commit science fraud, falsify research or otherwise conduct scientific misconduct.
A con-study, which devastated the public view of the subject for decades, was the
study of selling more coke and popcorn by unconscious ads.
The researcher said that he had found great effects from subliminal messages, whilst
he had, in fact, never conducted the experiment.
not use the position as a peer reviewer to give sham peer reviews to punish or damage
fellow scientists.
119
Basically, research must follow all regulations given, and also anticipate possible
ethical problems in their research.
Competition is an important factor in research, and may be both a good thing and a
bad thing.
Whistle blowing is one mechanism to help discover misconduct in research.
Why are ethics in research important?
Statistics Tutorial
This statistics tutorial is a guide to help you understand key concepts of statistics
and how these concepts relate to the scientific method and research.
Scientists frequently use statistics to analyze their results. Why do researchers use
statistics? Statistics can help understand a phenomenon by confirming or rejecting a
hypothesis. Statistics is often vital to change scientific theories.
You don't need to be a scientist though; anyone wanting to learn about how
researchers can get help from statistics may want to read this statistics tutorial for the
scientific method.
Research Data
This section of the statistics tutorial is about understanding how data is acquired
and uses.
The results of a science investigation often contain much more data or
information than the researcher needs. This data-material, or information, is called
raw data.
To be able to analyze the data sensibly, the raw data is processed into "output data".
There are many methods to process the data, but basically the scientist organizes and
summarizes the raw data into a more sensible chunk of data. Any type of organized
information may be called a "data set".
120
Then, researchers may apply different statistical methods to analyze and
understand the data better (and more accurately). Depending on the research, the
scientist may also want to use statistics descriptively or for exploratory research.
What is great about raw data is that you can go back and check things if you
suspect something different is going on than you originally thought. This happens
after you have analyzed the meaning of the results.
The raw data can give you ideas for new hypotheses, since you get a better view
of what is going on. You can also control the variables which might influence the
conclusion (e.g. third variables).
Central Tendency and Normal Distribution
This part of the statistics tutorial will help you understand distribution, central
tendency and how it relates to data sets.
Much data from the real world is normal distributed, that is, a frequency curve
which has the most frequent number near the middle. This is a reason why researchers
very often measure the central tendency in statistical research, such as the mean
(arithmetic mean or geometric mean), median or mode.
The central tendency may give a fairly good idea about the nature of the data
(mean, median and mode shows the "middle value"), especially when combined with
measurements on how the data is distributed. Scientists normally calculate the
standard deviation to measure how the data is distributed.
But there are various methods to measure ho data is distributed: variance,
standard deviation, standard error of the mean, standard error of the estimate or
"range" (which states the extremities in the data).
121
To create the graph of the normal distribution for something, you'll normally use
the arithmetic mean of a "big enough sample" and you will have to calculate the
standard deviation.
But, the distribution will not be normal distributed if the distribution is skewed
(naturally) or has outliers (often rare outcomes or measurement errors) messing up the
data. One example of a distribution which is not normally distributed is the F-
distribution, which is skewed to the right.
So, often researchers double check that their results are normally distributed
using range, median and mode. If the distribution is not normally distributed, this will
influence which statistical test/method to choose for the analysis.
Hypothesis Testing - Statistics Tutorial
How do we know whether a hypothesis is correct or not?
Why use statistics to determine this?
Using statistics in research involves a lot more than make use of statistical formulas or
getting to know statistical software.
Making use of statistics in research basically involves
learning basic statistics
understanding the relationship between probability and statistics
comprehension of inferential statistics.
knowledge of how statistics relates to the scientific method.
Statistics in research is not just about formulas and calculation. (Many wrong
conclusions have been conducted from not understanding basic statistical concepts)
Statistics inference helps us to draw conclusions from samples of a population.
When conducting experiments, a critical part is to test hypotheses against each other.
Thus, it is an important part of the statistics tutorial for the scientific method.
122
Hypothesis testing is conducted by formulating an alternative hypothesis which is
tested against the null hypothesis, the common view. The hypotheses are tested
statistically against each other.
The researcher can work out a confidence interval, which defines the limits when
you will regard a result as supporting the null hypothesis and when the alternative
research hypothesis is supported.
This means that not all differences between the experimental group and the
control group can be accepted as supporting the alternative hypothesis - the result
need to differ significantly statistically for the researcher to accept the alternative
hypothesis. This is done using a significance test (another article). Depending on the
hypothesis, you will have to choose between one-tailed and two tailed tests.
Sometimes the control group is replaced with experimental probability - often if
the research treats a phenomenon which is ethically problematic, economically too
costly or overly time-consuming, then the true experimental design is replaced by a
quasi-experimental approach.
Often there is a publication bias when the researcher finds the alternative
hypothesis correct, rather than having a "null result", concluding that the null
hypothesis provides the best explanation. If applied correctly, statistics can be used to
understand cause and effect between research variables.
It may also help identify third variables, although statistics can also be used to
manipulate and cover up third variables if the person presenting the numbers does not
have honest intentions (or sufficient knowledge) with their results.
Misuse of statistics is a common phenomenon, and will probably continue as long
as people have intentions about trying to influence others. Proper statistical treatment
of experimental data can thus help avoid unethical use of statistics. Philosophy of
123
statistics involves justifying proper use of statistics and establishing the ethics in
statistics.
Here is another great statistics tutorial which integrates statistics and the scientific
method.
Reliability and Experimental Error
Statistical tests make use of data from samples. These results are then generalized
to the general population. How can we know that it reflects the correct conclusion?
Contrary to what some might believe, errors in research are an essential part of
significance testing. Ironically, the possibility of a research error is what makes the
research scientific in the first place. If a hypothesis cannot be falsified (e.g. the
hypothesis has circular logic), it is not testable, and thus not scientific, by definition.
If a hypothesis is testable, to be open to the possibility of going wrong. Statistically
this opens up the possibility of getting experimental errors in your results due to
random errors or other problems with the research. Experimental errors may also be
broken down into Type-I error and Type-II error.
A power analysis of a statistical test can determine how many samples a test will
need to have an acceptable p-value in order to reject a false null hypothesis.
The margin of error is related to the confidence interval and the relationship between
statistical significance, sample size and expected results. The effect size estimate the
strength of the relationship between two variables in a population. It may help
determine the sample size needed to generalize the results to the whole population.
Replicating the research of others is also essential to understand if the results of the
research were a result which can be generalized or just due to a random "outlier
experiment". Replication can help identify both random errors and systematic errors
(test validity).
124
Replicating the experiment/research ensures the reliability of the results
statistically. What you often see if the results have outliers, is a regression towards the
mean, which then makes the result not be statistically different between the
experimental and control group.
Statistical Tests
At this stage in the statistics tutorial for the scientific method, we're introducing
some commonly used statistics tests/methods.
Relationship between Variables
Correlation (linear relationship)
Pearson Product-Moment Correlation
Spearman rho
Partial Correlation and Multiple Correlation
Making Predictions
The goal of predictions is to understand causes. Correlation does not necessarily
mean causation. With linear regression, you often measure a manipulated variable.
What is the difference between correlation and linear regression? Basically,
correlation is about the strength between the variables whereas linear regression is
about the best fit line in a graph.
Regression analysis and other modeling tools
Linear Regression
Multiple Regression
A Path Analysis is an extension of the regression model
A Factor Analysis attempts to uncover underlying factors of something.
The Meta-Analysis frequently make use of effect size
125
Bayesian Probability is a way of predicting the likelihood of future events in an
interactive way, rather than to start measuring and then get results/predictions.
Testing Hypothesis Statistically
Student's t-test is a test which can indicate whether the null hypothesis is correct or
not. In research it is often used to test differences between two groups (e.g. between a
control group and an experimental group).
The t-test assumes that the data is more or less normally distributed and that the
variance is equal (this can be tested by the F-test).
Student's t-test:
Independent One-Sample T-Test
Independent Two-Sample T-Test
Dependent T-Test for Paired Samples
Wilcoxon Signed Rank Test may be used for non-parametric data.
A Z-Test is similar to a t-test, but will usually not be used on sample sizes below 30.
A Chi-Square can be used if the data is qualitative rather than quantitative.
Comparing more than two Groups
An ANOVA, or Analysis of Variance, is used when it is desirable to test whether
there are different variability between groups rather than different means. Analysis of
Variance can also be applied to more than two groups. The F-distribution can be used
to calculate p-values for the ANOVA.
Analysis of Variance
One way ANOVA
Two way ANOVA
Factorial ANOVA
Repeated Measures and ANOVA
126
Nonparametric Statistics
Some common methods using nonparametric statistics:
Cohen's Kappa
Mann-Whitney U-test
Spearman's Rank Correlation Coefficient
References:
Bordens, Kenneth S (2002) Edi 5th Research Design and Methods: A Process
Approach. , Main Library Punjab University.
Cooper, Schindler, Donald R., Pamela S. (2001) 5th Edition. Business Research
Methods. Main Library Punjab University.
Gravetter, Forzano, Frederick J , Lori –Ann B. 5th Edition. Research Methods. ,
Main Library Punjab University.
Greener, Dr. Sue (2008) Business Research Methods. www.bookboon.com
http://www.experiment-resources.com/index.html
Heiman, Gary W. (1995) Research methods in Psychology, Main Library Punjab
University.
127
Saslow, Carol A. (1982) 1st Edition. Basic Research Methods, Main Library
Punjab University.
Zikmund, William G. Business Research Methods Main Library Punjab
University.
128

Research Methodology Finl

Uploaded by

Copyright:

Available Formats

Research Methodology Finl

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Research Methodology Finl

Uploaded by

Copyright:

Available Formats

The Research Process (Step 4 & 5) 1

THEORETICAL FRAMEWORK HYPOTHESIS DEVELOPMENT

FATIMA TAHIR MS09MBA014

KANWAL AMIN MS09MBA016

ASMA MUNAWAR MS09MBA039

DR. FOUZIA NAHEED KHAWAJA

Business Research Defined 6

The Request for Proposal (RFP) 12

Business Methods versus Research Methodology 13

The Scientific Method: An Overview 16

Steps of the Scientific Process 17

What is a Literature Search 19

Purpose of a Literature Search 20

The Literature Review: 28

Converting an Idea into a Research Hypothesis 30

What to look for in relation to the Literature Review 35

Where to look for reference material 37

Determine the Relevant Variable 48

Independent and Dependent Variables 52

Operational Definition of Variable 57

Threats to a Valid Measurement Procedure 64

Controlling Threats to Reliability and Validity 68

Creating a Reliable Manipulation 68

Controlling Confounding Variable 68

Control Demand Characteristics 77

Cause and Effect Relationships 78

Manipulation and Measurement 84

Theory and Research 84

Two Purposes of Theory 86

Concept (or Construct) 86

Propositions and Hypotheses 90

The Role of the Hypothesis 95

What is a good hypothesis? 95

Formulating the Research Problem 102

Hypothesis Development and Testing 104

Explanation of Correlation Coefficient 105

Conceptual Variables 107

Validity and Reliability 108

Types of Validity 109

Types of Reliability 110

Choosing the Research Method 110

Choosing the Measurement 110

Errors in Research 110

Experimental Research 111

Aims of Experimental Research 111

Identifying the Research Problem 111

Constructing the Experiment 111

Sampling Groups to Study 111

Creating the Design 112

Typical Designs and Features in Experimental Design 112

Pretest-Posttest Design 113

Control Group 113

Randomized Controlled Trials 113

Between Subjects Design 114

Solomon Four-Group Design 114

Within Subject Design 114

Counterbalanced Measures Design 114

Matched Subjects Design 114

Pilot Study 115