research method

A Dictionary of Research Methodology and Statistics in Applied Linguistics Hossein

Tavakoli RAHNAMA PRESS 2012 Tehran
a systematic and rigorous way of collecting and analyzing information.
QUANTITATIVE RESEARCHers use a wide variety of instruments to gather
data, including TEST, QUESTIONNAIRE, and RATING SCALE. In QUALITATIVE
RESEARCH this includes, for example, PARTICIPANT OBSERVATION,
INTERVIEW, VERBAL REPORT, DIARY STUDY, and DISCOURSE ANALYSIS.(548)

research methodology
a theory of how inquiry should occur. Research methodology defines the
kinds of problems that are worth investigating and frames them, determines
what research approaches and research methods to use, and also
how to understand what constitutes a legitimate and warranted explanation.
Research methodology involves such general activities as identifying
problems, review of the literature, formulating hypotheses, procedure
for testing hypotheses, measurement, data collection analysis of data, interpreting
results, and drawing conclusions. Researchers need to understand
the assumptions underlying various techniques and they need to

qualitative research(508)
a RESEARCH METHODOLOGY that places primary importance on studying
small samples of purposely chosen individuals; not attempting to control
contextual factors, but rather seeking, through a variety of methods, to
understand things from the informants’ points of view; and creating a
rich and in-depth picture of the phenomena under investigation. There is
less of an emphasis on statistics (and concomitant attempts to generalize
the results to wider populations) and more of an interest in the individual
and his/her immediate context. By definition, qualitative research is synthetic
or holistic (i.e., views the separate parts as a coherent whole), heuristic
(i.e., discovers or describes the patterns or relationships), with little
or no control and manipulation of the research context, and uses data collection
procedures with low explicitness.
Qualitative research has roots in a number of different disciplines, principally
ANTHROPOLOGY, sociology, and philosophy, and is now used in
almost all fields of social science inquiry, including applied linguistics.
Qualitative research is the primary example of HYPOTHESIS-GENERATING
RESEARCH. That is, once all the data are collected, hypothesis may be derived
from those data. The ultimate goal of qualitative research is to discover
phenomena such as patterns of behavior not previously described
and to understand them from the perspective of participants in the activifollowing
characteristics:
1) Rich description: The aims of qualitative researchers often involve
the provision of careful and detailed descriptions as opposed to the
quantification of data through measurements, frequencies, scores, and
ratings;
2) Natural and holistic representation: Qualitative researches aim to
study individuals and events in their natural settings, i.e., rather than
attempting to control and manipulate contextual factors through the
use of laboratories or other artificial environments, qualitative researchers
tend to be more interested in presenting a natural and holistic
picture of the phenomena being studied. In order to capture a sufficient
level of detail about the natural context, such investigations are
usually conducted through an intense and prolonged contact with, or
immersion in, the research setting;
3) Few participants: Qualitative researchers tend to work more intensively
with fewer participants, and are less concerned about issues of
GENERALIZABILITY. Qualitative research focuses on describing, understanding,
and clarifying a human experience and therefore qualitative
studies are directed at describing the aspects that make up an idiosyncratic
experiences rather than determining the most likely, or
mean experience, within a group. Accordingly, at least in theory,
qualitative inquiry is not concerned with how representative the respondent
sample is or how the experience is distributed in the population.
Instead, the main goal of sampling is to find individuals who can
provide rich and varied insights into the phenomenon under investigation.
This goal is best achieved by means of some sort of PURPOSIVE
SAMPLING;
4) Emic perspective (or participant or insider point of view): Qualitative
researchers aim to interpret phenomena in terms of the meanings people
attach to them, i.e., to adopt an emic perspective, or the use of
categories that are meaningful to members of the speech community
under study. An emic perspective requires one to recognize and accept
the idea of multiple realities. Documenting multiple perspectives
of reality in a given study is crucial to an understanding of why people
think and act in the different ways they do. Emic perspectives can
be distinguished from the use of etic perspective (or researcher or
outsider point of view), which is an outsider’s understanding of a culture
or group that is not their own. Etic perspectives are more common
in QUANTITATIVE RESEARCH;
5) Cyclical and open-ended processes: Qualitative research is often process-
oriented or open ended, with categories that emerge. The research often follows an inductive
path that beings with few perceived
notions, followed by a gradual fine tuning and narrowing of focus.
Ideally, qualitative researchers enter the research process with a completely
open mind and without setting out to test preconceived hypotheses.
This means that the research focus is narrowed down only
gradually and the analytic categories and concepts are defined during,
rather than prior to, the process of the research. Thus, qualitative researchers
tend to approach the research context with the purpose of
observing whatever may be present there, and letting further questions
emerge from the context;
6) Possible ideological orientations: Whereas most quantitative researchers
consider impartiality to be a goal of their research, some
qualitative researchers may consciously take ideological positions.
This sort of research is sometimes described as critical, meaning that
the researcher may have particular social or political goals, e.g., CRITICAL
DISCOURSE ANALYSIS, a form of qualitative research, is a program
of social analysis that critically analyzes discourse, (i.e., language
in use), as a means of addressing social change;
7) Interpretive analysis: Qualitative research is fundamentally interpretive,
which means that the research outcome is ultimately the product
of the researcher’s subjective interpretation of the data. Several alternative
interpretations are possible for each data set, and because qualitative
studies utilize relatively limited standardized instrumentation
or analytical procedures, in the end it is the researcher who will
choose from them. The researcher is essentially the main measurement
device in the study. Accordingly, in qualitative research, the researcher’s
own values, personal history, and position on characteristics
such as gender, culture, class, and age become integral part of the
inquiry; and
8) The nature of qualitative data: Qualitative research works with a wide
range of data including recorded INTERVIEWs, various types of texts
(e.g., FIELDNOTES, JOURNAL and diary entries (see DIARY STUDY),
DOCUMENTs, and images (photos or videos). During data processing
most data are transformed into a textual form (e.g., interview recordings
are transcribed) because most qualitative data analysis is done
with words.
While the description of qualitative research stands in contrast with that
presented for quantitative research (with its emphasis on RANDOMIZATION,
STATISTICS, and generalizability), it should be understood that
quantitative and qualitative approaches are not polar opposites (as the
traditional labels of positivistic (see POSITIVISM) and interpretivist (see
INTERPRETIVE PARADIGM) for quantitative and qualitative research, respectively, sometimes
imply. It should also be kept in mind that, as it is
not the case that certain methods (e.g., QUESTIONNAIREs, interviews,
TESTs) are inherently either qualitative or quantitative. Questionnaire results,
for example, can be analyzed quantitatively by determining what
percentage of respondents answered in a particular manner, or qualitatively,
by examining in detail the exact responses individuals provided
and using them to triangulate (see TRIANGULATION) other data from
those same participants. It is the researcher’s approach to the data collection
and analysis task that may be considered qualitative or quantitative—
not the methods themselves.
A plethora of research designs has been developed within qualitative research,
including NATURALISTIC INQUIRY, NARRATIVE INQUIRY, CASE
STUDY, ETHNOGRAPHY, ACTION RESEARCH, PHENOMENOLOGY, CONVERSATION
ANALYSIS, LIFE HISTORY RESEARCH, and GROUNDED THEORY.
These approaches use a wide variety of data collection methods,
such as OBSERVATION, interview, open-response questionnaire items,
VERBAL REPORT, diary study, and DISCOURSE ANALYSIS. And within
each of these research approaches and methods, a number of research
techniques and strategies have been developed to help qualitative researchers
do their day-to-day work—conceptualizing the research project,
collecting and analyzing data, and writing up findings.
quantitative research(509)
a RESEARCH METHODOLOGY that stresses the importance of large groups
of randomly selected participants, manipulating VARIABLEs within the
participants’ immediate environment, and determining whether there is a
relationship between the manipulated (independent) variable and some
characteristic or behavior of the participants (the DEPENDENT VARIABLE).
Statistical procedures are used to determine whether the relationship
is significant—and when it is significant, the results are typically
generalized to a larger population beyond the immediate group of participants.
At best, the quantitative research is systematic, rigorous, focused,
and tightly controlled, involving precise measurement and producing reliable
and replicable data that is generalizable to other contexts. Quantitative
research is the primary example of hypothesis-testing research (see
HYPOTHESIS TESTING), which begins with a question or hypothesis to be
investigated through data quantification and numerical analyses.
Main characteristics of quantitative research are listed as follows:
1) Using numbers: the single most important feature of quantitative research
is, naturally, that it is centered around numbers. This both
opens up a range of possibilities and sets some limitations for researchers.
Numbers are powerful. Yet numbers are also rather powerless
in themselves because in research context they do not mean anything
without contextual backing, i.e., they are faceless and meaningless
unless we specify exactly the category that we use the specific
number for, and also the different values within the variable. Thus,
for numbers to work, the researcher need precise definitions of the
content and the boundaries of the variables s/he uses and exact descriptors
for the range of values that are allowed within the variable;
2) A priori categorization: because the use of numbers already dominates
the data collection phase, the work requires specifying the categories
and values needed to be done prior to the actual study. If, e.g.,
respondents are asked to encircle figures in a questionnaire item, they
have to know exactly what those figures represent, and in order to
make sure that each respondent gives their numerical answer based on
the same understanding, the definitions and value descriptors need to
be unambiguous; 3) Variables rather than cases: quantitative researchers are less interested
in individuals than in the common features of groups of people.
Therefore, it is centered around the study of variables that capture
these common features and which are quantified by counting, scaling,
or by assigning values to CATEGORICAL DATA. All the various quantitative
methods are aimed at identifying the relationships between variables
by measuring them and often also manipulating them. Therefore,
specifying the relationships amongst variables as the defining
feature of quantitative social research;
4) Statistics and the language of statistics: this is undoubtedly the most
salient quantitative feature statistical analyses can range from calculating
the average (the MEAN) of several figures on a pocket calculator
to running complex multivariate analyses on a computer. Because
of the close link of quantitative research and statistics, much of the
statistical terminology has become part of the quantitative vocabulary,
and the resulting unique quantitative language adds further power
to the quantitative paradigm;
5) Standardized procedures to assess objective reality: the general quantitative
aspiration is to eliminate any individual-based subjectivity
from the various phases of the research process by developing systematic
canons and rules for every facet of data collection and analysis.
Quantitative methodology has indeed gone a long way towards
standardizing research procedures to ensure that they remain stable
across investigators and subjects. This independence of idiosyncratic
human variability and bias has been equated with OBJECTIVITY by
quantitative researchers;
6) Quest for GENERALIZABILITY and universal laws: numbers, variables,
standardized procedures, statistics, and scientific reasoning are all
part of the ultimate quantitative quest for facts that are generalizable
beyond the particular and add up to wide-ranging, ideally universal,
laws. However, QUALITATIVE RESEARCHers often view quantitative
research as overly simplistic, decontextualized, reductionist in terms
of its generalizations, and failing to capture the meanings that actors
attach to their lives and circumstances.
The difference between quantitative and qualitative research is often seen
as quite fundamental, leading people to talk about paradigm wars in
which quantitative and qualitative research are seen as belligerent and incompatible
factions. Many researchers define themselves as either quantitative
or qualitative. This idea is linked to what are seen as the different
underlying philosophies and worldviews of researchers in the two paradigms.
According to this view, two fundamentally different worldviews
underlie quantitative and qualitative research. The quantitative view isdescribed as being
realist (see REALISM) or sometimes positivist (see POSITIVISM),
while the worldview underlying qualitative research is viewed
as being subjectivist (see INTERPRETIVE PARADIGM).
The term qualitative and quantitative were originally introduced to denote
in antagonistic standpoint and this initial conflicting stance was given
substance by the contrasting patterns of the two research paradigms
in: (a) categorizing the world (quantitative: predetermined numerical category
system; qualitative: emergent, flexible verbal coding); (b) perceiving
individual diversity (quantitative: using large samples to iron out any
individual idiosyncrasies; qualitative: focusing on the unique meaning
carried by individual organisms); and (c) analytical data (quantitative: relying
on the formalized system of statistics; qualitative: relying on the researcher’s
individual sensitivity.
In short, quantitative research was seen to offer a structured and highly
regulated way of achieving a macro-perspective of the overarching
trends in the world, whereas qualitative research was perceived to represent
a flexible and highly context-sensitive micro-perspective of the everyday
realities of the world. Although the two paradigms represent two
different approaches to EMPIRICAL RESEARCH, they are not necessarily
exclusive. They are not extremes but rather form a continuum that has
led to an emerging third research approach, i.e., MIXED METHODS RESEARCH.
Quantitative research can be classified into one of the two broad research
categories: EXPERIMENTAL RESEARCH, and NONEXPERIMENTAL RESEARCH.

Qualitative research in applied linguistics (5) Qualitative Research in Applied

LinguisticsJuanita Heigham
Sugiyama Jogakuen University
and
Robert A. Croker
Nanzan University2009 by
PALGRAVE MACMILLAN UK
Applied linguistics is a broad and exciting interdisciplinary field of study.
It focuses on language in use, connecting our knowledge about languages
with an understanding of how they are used in the real world. Applied linguists
work in diverse research areas including second-language acquisition
(SLA), teaching English as a second or other language (TESOL), workplace
communication, language planning and policy, and language identity and
gender – to name just a few. Many applied linguists also work in related
fields such as education, psychology, sociology, and anthropology.
One important area of applied linguistics research is language analysis. SLA
researchers, for example, look at what language errors learners commonly
make at different stages in their language development, or TESOL researchers
consider how a writing textbook helps students develop their composition
skills. A second important area in applied linguistics is investigating
the contexts and experiences of language use. For instance, researchers specializing
in workplace communication could examine how immigrant women
with differing degrees of language proficiency use the target language to
communicate with co-workers, or TESOL researchers might investigate how
the classroom milieu affects students’ attitudes toward language learning.
Similarly, language identity researchers might consider how sexual minorities
structure their identity through language.
How do researchers approach such issues? In essence, they have three
choices: to use quantitative research, qualitative research, or to use both in
what is termed mixed methods research. In very broad terms, quantitative
research involves collecting primarily numerical data and analyzing it using statistical
methods, whereas qualitative research entails collecting
primarily textual data and examining it using interpretive analysis. Mixed
methods research employs both quantitative and qualitative research
according to the aims and context of the individual project and the nature
of the research questions.
In the following chapters, we mainly focus on qualitative research, first
exploring what it is, then illustrating how it is used to investigate the manifold
contexts and experiences of language in use; however, there is also a
chapter that provides a thorough introduction to mixed methods research What is
qualitative research?
An umbrella term
The term ‘qualitative research’ is an umbrella term used to refer to a complex
and evolving research methodology. It has roots in a number of different
disciplines, principally anthropology, sociology, and philosophy, and
is now used in almost all fields of social science inquiry, including applied
linguistics. A plethora of research approaches has been developed within
qualitative research, including narrative inquiry, case study, ethnography,
action research, phenomenology, and grounded theory. These approaches
use a wide variety of data collection methods, such as observation, interviews,
open-response questionnaire items, verbal reports, diaries, and discourse
analysis; all of these approaches and methods will be discussed in
this book. And within each of these research approaches and methods,
a number of research techniques and strategies have been developed to
help qualitative researchers do their day-to-day work – conceptualizing the
research project, collecting and analyzing data, and writing up findings.
The number of terms and concepts used to define these approaches and
methods and their associated strategies and techniques has grown to the
point where there is now even a Dictionary of Qualitative Inquiry (Schwandt,
2007).

The research process

In a qualitative study, researchers often use multiple data collection methods,
including observations, interviews, open-response questionnaires, and diaries.
Each of these ‘makes the world visible in a different way’ (Denzin &
Lincoln, 2005, p. 4), so a fuller, richer picture of the participants’ perspective
can be explored and represented. All of these data collection methods createdata that is
primarily textual not numerical: researchers doing observations
create written notes, called field notes; researchers using interviews generate
written transcripts or summaries; and the other data collection methods
such as questionnaires and diaries use text that the participants themselves
have written. That is not to say that numerical data is not used, but that its
purpose is supplementary not central. A vast amount of textual data is created
in a qualitative research study, and managing it is often challenging.
The textual data that researchers create in their field notes and interview
summaries should be richly detailed and descriptive of the participants and
the research setting – capturing what researchers have seen, heard, smelled,
and touched. As they create this data, and later as they think about them,
researchers add their own thoughts and reflections. Taken together, this creates
a thick description of the participants and setting. Qualitative researchers
then use interpretive analysis to sift through their data and group similar
ideas together, to discover patterns of behavior and thinking.
The data that researchers collect permits them to paint a richly descriptive
picture of their participants’ worlds – the participants themselves, the
setting, and the major and minor events that happen there. A well- written
qualitative research study will carefully use the participants’ own words
to augment the researcher’s vivid description and clear interpretation. It
should give readers a sense of entering the participants’ worlds and sharing
the experience of being there with them. The process is, in a sense, like filmmaking
– the researcher assembles data into montages by blending images,
sounds, and understandings together to create a compelling composite creation
(see Denzin & Lincoln, 2005, for a fuller explanation).
The nature of qualitative research
When little is known about a phenomenon or existing research is limited,
qualitative research is a very useful research methodology because it is
exploratory – its purpose is to discover new ideas and insights, or even generate
new theories. This research is not necessarily done to predict what may
happen in the future or in another setting – what is learned about the phenomenon,
participants, or events in the setting can be an end in itself. That
is, qualitative research mostly focuses on understanding the particular and
the distinctive, and does not necessarily seek or claim to generalize findings
to other contexts. Some qualitative researchers do consider the extent to
which their findings may be generalizable, but many leave it up to the readers
to decide to what degree the features of the research setting are relevant
to their own context. The richer the description the researcher provides in
the study’s report, the easier it is for readers to envisage the research setting
and thus make a judgment about the relevance of the research for them.
As qualitative research is often exploratory, most researchers do not define
specific research questions at the outset of the study, as doing so would likely
impose their own framework on the research context. Rather, they usuallybegin the study
with only a research purpose and conceptual framework, and
a sense of the initial focus of interest. They then prefer to enter the research setting
and become familiar with the context and the participants, and ascertain
what participants think the main issues and problems are, before determining
their specific research questions. These questions are modified and refined,
and the research design developed, as their understandings of the research setting,
participants, and research focus mature. This reflects what is called the
emergent nature of the qualitative research process – understanding emerges
as the research proceeds. Donald Freeman explores this in Chapter 2.
The quantitative research cycle is usually characterized as being linear,
each stage being carried out one after the other: research questions are
formulated, data is collected then statistically analyzed, and findings written
up. By contrast, qualitative research is more simultaneous, nonlinear,
and iterative. That is, collecting, analyzing, and interpreting data are done
largely at the same time, with researchers constantly moving back and forth
between all three until new information does not add to their understanding
of a topic, a point called data saturation. In fact, data analysis will
often steer data collection, as ongoing analysis indicates what avenues of
research to pursue – who to observe or interview next, what questions to
ask, and what documents to request – so the emergent nature of qualitative
research is also evident throughout the research cycle.
Although emergent, qualitative research is systematic and rigorous. As
Gretchen Rossman and Sharon Rallis illustrate in Chapter 13, for readers
and other researchers to trust your research, there must be a strong conceptual
framework to guide your study, and congruence between the research
approach that structures your study and the data collection methods that
you employ. Moreover, you need to demonstrate that your research practices
are sound and that you have used clear logic, provide strong evidence to
substantiate the claims that you make, and diligently document the process
of gathering, analyzing, and interpreting your data.
While qualitative research is systematic, it is not formulaic, so there is no
requirement that researchers follow a set of prescribed research steps. This
point is stressed by Rallis and Rossman (2003):
... inquiry proceeds through a complex, nonlinear process of induction,
deduction, reflection, inspiration, and just plain old hard thinking. This
can be characterized as researcher praxis (Jones, 2002), that is, an iteration
between theoretical ideas, data, and the researcher’s reflection on both.
A ... feature of qualitative research ... is a reliance on sophisticated reasoning
that is multifaceted and iterative, moving back and forth between the
parts and the whole. (p. 11).
Qualitative research also requires you to be intuitive, to see links and patterns
in the data, and to build these into themes that simultaneously fulfillyour research
purposes and also express both the particular and the essential
nature of the setting and its participants. Qualitative research is a discipline
that calls for a balance between order and insight.
Mixed methods (17)(Chapter 7) combines both qualitative and quantitative
research methods in a single study. For example, language proficiency test
scores are used along with student interviews to create a more multidimensional
view of a language learning process, or a teacher questionnaire
is combined with classroom observations and teacher diaries to generate a
fuller understanding of one aspect of language teaching. In their chapter,
Nataliya Ivankova and John Creswell illustrate the procedures for collecting,
analyzing, and mixing qualitative and quantitative data at different
stages in the research process. A mixed methods study could emphasize
qualitative and quantitative data equally, or give one type greater emphasis.
It is an emerging field of study and is becoming more commonly used
in research in applied linguistics.

Qualitative data collection methods

In most qualitative studies, researchers use a variety of research methods to
collect data, in order to obtain as many perspectives as possible on the phenomenon
being researched. In this book, six data collection methods most
commonly used in qualitative research in applied linguistics are addressed
in Part III:
● Observation (Chapter 8) occurs when researchers carefully watch participants

in the research setting with the aim of understanding their

experience of being there, as Neil Cowie explains in his chapter. It is used
to collect information about participants’ external behavior, which can
be further explored casually in conversation or more formally in interviews,
with questions about participants’ inner ideas, beliefs, and values.
Researchers can choose to be ‘complete observers’ and not take part in
the learning or teaching phenomenon being studied, or they can choose
to take part as ‘participant observers’. Data is created in the form of fieldnotes, which
include explanations of what researchers observed as well as
their reflections.
● Interviews (Chapter 9) offer a way to explore people’s experiences and

worldviews and the meanings they bring to them, as Keith Richards

illustrates in his chapter. Interviews can be carefully structured by predetermined
questions to elicit specific information, or be more open to
allow for generating richer insights. The greatest challenge in interviewing
is getting the interaction with the participant right, by recognizing
that interviews are jointly constructed encounters.
● Open-response items on questionnaires (Chapter 10) are questions on

a survey that do not require respondents to select their answers from a

limited list or selection; rather, participants answer in their own words.
They are commonly used when researchers would like to quickly and efficiently
collect textual data from a relatively large number of participants.
James Dean Brown in his chapter provides guidelines for writing good
questions, and for how to administer and analyze data.
● Verbal reports (Chapter 11) are oral records of a participant’s thought

processes, provided by individuals when they are thinking aloud either

during or immediately after completing a language learning or teaching
task. Diaries (Chapter 11) are another way of accessing participants’ inner
worlds; they are an account of a language experience as recorded in a firstperson
journal. These accounts may be analyzed and published by the
diarists themselves or by an independent researcher. Both verbal reports
and diaries are particularly important in applied linguistics, and their use
is discussed by Sandra Lee McKay in her chapter.
● Discourse analysis (Chapter 12) looks at how language is used in spoken

and written communication. It uses authentic language that has been

produced spontaneously in naturally occurring events, that were not elicited
experimentally specifically for the sake of research. The researcher
should analyze this data with few or no preconceived notions, but allow
the patterns of language use to emerge. Anne Lazaraton explains how to
collect and analyze such spoken data in her chapter.
These six data collection methods can be conceptualized by placing them
along two intersecting continuums. The first of these continuums expresses
the amount of control researchers have over the research setting as they
collect their data. In most qualitative research, researchers do not control
the research setting at all, as they are interested in authentic behavior in
natural settings. However, researchers using verbal reports do control what
participants do during the research process, as you will learn when you read
Chapter 11. By comparison, with research done outside qualitative research,
most language analysis also involves researchers carefully controlling the
research environment: researchers try to control the language that participants
use when they complete a language task, and they also collect data in settings specified
for the purposes of collecting data, like a researcher’s
office or a language laboratory, rather than in natural settings.

The qualitative methods (8) Introduction to The

Encyclopedia
of Applied Linguistics
CAROL A. CHAPELLE
topic area designed by Linda Harklau and Meryl Siegal demonstrates
the ways in which qualitative principles and practices shared with others in the
social sciences are used in applied linguistics. Entries show how researchers use codes,
labels, categorization systems, and narratives, for example, in studying problems in
applied
linguistics in addition to illustrating the types of problems that are investigated. Qualitative
approaches, for example, have helped to document the detail associated with individual
language learning and use in a manner that is not revealed in quantitative research.
The topic area of quantitative and mixed methods research designed by Joan Jamieson
and Kim McDonough demonstrates how quantitative methods from the social sciences
are used to address a range of applied linguistics problems in areas such as the study of
the effectiveness of particular language teaching practices or the language needs of
prospective
employees in a business. In addition, it includes entries that explain basic concepts
on quantitative research as it is used in applied linguistics. Quantitative approaches have
been an important source of new knowledge in applied linguistics as it has been used for
describing relevant situations of language use in addition to testing hypotheses about the
nature of language use and learnin