Content Analysis In

Qualitative Research
By Dr. Aftab Hussain
Objectives
1.Define content analysis.
2.Describe when it is appropriate to use content analysis.
3.List the steps involved in content analysis.
4.Describe the importance of categorization in content analysis.
5.Describe the two major coding categories.
6.Identify how reliability and validity is established in content analysis
research.
7.Describe how data is analyzed in content analysis research.
8.Identify two major advantages and disadvantages of content analysis.
9.Examine a journal article written about research that uses content analysis.
Overview

• Content analysis is a research tool used to determine the presence of certain

words, themes, or concepts within some given qualitative data (i.e. text).
• Using content analysis, researchers can quantify and analyze the presence,
meanings, and relationships of such certain words, themes, or concepts.
• As an example, researchers can evaluate language used within a news article to
search for bias or partiality.
• Researchers can then make inferences about the messages within the texts, the
writer(s), the audience, and even the culture and time of surrounding the text.
Define content analysis
• Analysis of the contents of communication.
• Technique that allows us to study human behavior indirectly through analyzing communications.
• Textbooks, Newspapers, Essays, Magazines, Articles, Cookbooks, Songs, Political Speeches, Novels, Advertisements, and
Pictures.

• According to Mayring (2000), qualitative content analysis is “an approach of empirical, methodological controlled analysis of
texts within their context of communication, following content analytic rules and step-by-step models, without rash
quantification” (p. 23).
 Content analysis (introduction)
• Content analysis is a technique for examining the content or information and
symbols contained in written documents or other communication media
(e.g., photographs, movies, song lyrics, advertisements).
• To conduct a content analysis, we identify a body of material to analyze (e.g.,
school textbooks, television programs, newspaper articles) . then create a
system for recording specific aspects of its content.
• Content analysis is a nonreactive method because the creators of the content
didnt know whether anyone would analyze it.
• Content analysis lets us discover and document specific features in the
content of a large amount of material that might otherwise go unnoticed.
• We most frequently use content analysis for descriptive purposes, but
exploratory or explanatory studies are also possible .
When to use QCA
• When you are dealing with rich data that requires interpretation
• On verbal data
• On visual data
• On data that you have sampled from other sources (documents,
internet, etc.)
• On data that you have collected yourself (interviews, focus groups,
etc.)
 Key features of qualitative research
 Interpretive
 Naturalistic
 Situational
 Reflexive
 Has emergent flexibility
 Inductive
 Case-oriented
 Puts emphasis on validity
Qualitative and quantitative content
analysis
QCA, compared against quantitative content analysis, is often referred to as “latent level analysis,
because it concerns a second-level, interpretative analysis of the underlying deeper meaning of the
data” (Dörnyei, 2007, p. 246); while the latter is usually described as “manifest level analysis”,
providing an objective and descriptive overview of the “surface meaning of the data.”

The techniques of data sampling are different, as the quantitative approach requires
random sampling or other techniques of probability to ensure validity, while qualitative
analysis uses intentionally chosen texts.

There are different products of the two approaches; while quantitative analysis caters for
statistical methods and numerical results, the qualitative approach brings descriptions.
Mixing of both

Mixing qualitative and Despite of these

quantitative methods is differences, it has been
known as one of the ways highlighted by numerous
of using triangulation, scholars that, in research
which, according to Flick practice, the two
(2010, p. 405), is “used as approaches are often
a strategy of improving applied in combination
the quality of qualitative (Dörnyei, 2007; Flick, 2007;
research …”. Zhang & Wildemuth, 2009).
Why to use content analysis?
1. Researchers use QCA is to illustrate the range of meanings of
phenomena, describe the characteristics of message content, and
identify themes or categories within a body of text.
2. Bryman (2008) maintained that QCA comprises a searching out of
underlying themes in the texts being analyzed by researchers.
3. Researchers, if they intend to better explain the characteristics of
message content, or understand phenomena, must possess an
encyclopedic knowledge of QCA.
4. Determine psychological or emotional state of persons or groups
5. Identify the intentions, focus or communication trends of an individual, group or institution
 The goals of content analysis

• To examine trends and relationships in the text and multimedia

produced or used in the fields context to provide an insight into it.
• To identify the intentions, focus or communication trends of an
individual, group or institution.
• To describe attitudinal and behavioral responses to communications
• To determine psychological or emotional state of persons or groups.
 Summary
The goals

• To reduce large amounts of unstructured content.

• To Identify and present important aspects of the content.
• To focus on groups or individual intention .
• To support some argument by understanding the characteristic of the content .
• To determine the behavioral and psychology
Three Approaches To Content
Analysis
 1.CONVENTIONAL CONTENT ANALYSIS
• Conventional content analysis is generally used with a study design whose aim is to describe a
phenomenon.
• This type of design is usually appropriate when existing theory or research literature on a phenomenon is
limited. Researchers avoid using preconceived categories (Kondracki & Wellman, 2002), instead allowing the
categories and names for categories to flow from the data.

• Researchers immerse themselves in the data to allow new insights to emerge (Kondracki & Wellman,
2002).
• With a conventional approach to content analysis, relevant theories or other research findings are addressed
in the discussion section of the study. The discussion would include a summary of how the findings from her
study contribute to knowledge in the area of interest and suggestions for practice, teaching, and future
research.
Cont….
• The advantage of the conventional approach to content analysis is gaining direct information from
study without imposing preconceived categories.

• One challenge of this type of analysis is failing to develop a complete understanding of the context, thus
failing to identify key categories. This can result in findings that do not accurately represent the data.

• Note: Many qualitative methods share this initial approach to study design and analysis.
 2.DIRECT CONTENT ANALYSIS

• The goal of a directed approach to content analysis is to validate or extend conceptually a

theoretical framework or theory. Existing theory or research can help focus the research
question. It can provide predictions about the variables of interest or about the relationships
among variables, thus helping to determine the initial coding scheme or relationships
between codes.

• Using existing theory or prior research, researchers begin by identifying key concepts or
variables as initial coding categories (Potter & Levine- Donnerstein, 1999). Operational
definitions for each category are determined using the theory.

• The second strategy that can be used in directed content analysis is to begin coding
immediately with the predetermined codes.
Cont….
• The main strength of a directed approach to content analysis is that existing theory can be supported and
extended.
• Disadvantages
• Researchers might be more likely to find evidence that is supportive rather than non-supportive of a theory.
• Second, in answering the probe questions, some participants might get cues to answer in a certain way or
agree with the questions to please researchers.
• Third, an overemphasis on the theory can blind researchers to contextual aspects of the phenomenon.
 3.SUMMATIVE CONTENT ANALYSIS
• A study using a summative approach to qualitative content analysis starts with identifying and
quantifying certain words or content in text with the purpose of understanding the contextual use of the
words or content.

• A summative approach to qualitative content analysis goes beyond mere word counts to include latent
content analysis. Latent content analysis refers to the process of interpretation of content (Holsti, 1969).

• In this analysis, the focus is on discovering underlying meanings of the words or the content (Babbie,
1992;). Researchers report using content analysis from this approach in studies that analyze manuscript
types in a particular journal or specific content in textbooks.
 Cont….
• In a summative approach to qualitative content analysis, data analysis begins with searches for occurrences of the
identified words by hand or by computer. Word frequency counts for each identified term are calculated, with
source or speaker also identified.

• It allows for interpretation of the context associated with the use of the word or phrase. Researchers try to explore
word usage or discover the range of meanings that a word can have in normal use.

• ADVANTAGES: It is an unobtrusive(not easily noticed) and nonreactive way to study the phenomenon of
interest (Babbie, 1992). It can provide basic insights into how words are actually used.

• DISADVANTAGES: The findings from this approach are limited by their inattention to the broader meanings present
in the data. this type of study relies on credibility.
 Methodology

• All approaches to qualitative content analysis require a similar analytical process of seven
classic steps, including formulating the research questions to be answered, selecting the
sample to be analyzed, defining the categories to be applied, outlining the coding process
and the coder training, implementing the coding process, determining trustworthiness,
and analyzing the results of the coding process (Kaid,1989).

• Different research purposes require different research designs and analysis techniques
(Knafl & Howard, 1984).
• The question of whether a study needs to use a conventional, directed, or summative
approach to content analysis can be answered by matching the specific research purpose
and the state of science in the area of interest with the appropriate analysis technique.
Data Analysis
• Frequencies
• Proportions of an specific occurrence to a total occurrences.
• Themes
• Computer Analysis
 PROCESS OF CONTENT ANALYSIS
 Content analysis is a research technique for systematically analyzing written or
recording communication. It has been used to study books, essays. news articles,
speeches, pamphlets and other written material.

Despite its name, content analysis is more of a data reduction technique than an
analytical one because it breaks down lengthy text material into more manageable units
of data.

One can use different modes such as transcripts of interviews/discourses, protocols of

observation, video tapes and written documents for communication.

The method of analysis comprises following 8 steps

General steps for conducting a conceptual content
analysis:

1. Decide the level of analysis: word, word sense, phrase, sentence, themes

2. Decide how many concepts to code for: develop a pre-defined or interactive set of
categories or concepts. Decide either: A. to allow flexibility to add categories through
the coding process, or B. to stick with the pre-defined set of categories.

• Option A allows for the introduction and analysis of new and important material that
could have significant implications to one’s research question.
• Option B allows the researcher to stay focused and examine the data for specific
concepts.
Cont….
3. Decide whether to code for existence or frequency of a concept.
The decision changes the coding process.

• When coding for the existence of a concept, the researcher would

count a concept only once if it appeared at least once in the data and
no matter how many times it appeared.

• When coding for the frequency of a concept, the researcher would

count the number of times a concept appears in a text.
 Cont….
4. Decide on how you will distinguish among concepts:
• Should text be coded exactly as they appear or coded as the same when they appear in different
forms? For example, “dangerous” vs. “dangerousness”.
• The point here is to create coding rules so that these word segments are transparently categorized
in a logical fashion.
• The rules could make all of these word segments fall into the same category, or perhaps the rules
can be formulated so that the researcher can distinguish these word segments into separate codes.
• What level of implication is to be allowed? Words that imply the concept or words that explicitly
state the concept? For example, “dangerous” vs. “the person is scary” vs. “that person could cause
harm to me”.
• These word segments may not merit separate categories, due the implicit meaning of “dangerous”.
Cont….
• 5. Develop rules for coding your texts. After decisions of steps 1-4 are complete, a
researcher can begin developing rules for translation of text into codes. This will keep
the coding process organized and consistent.

• The researcher can code for exactly what he/she wants to code. Validity of the coding
process is ensured when the researcher is consistent and coherent in their codes,
meaning that they follow their translation rules. In content analysis, obeying by the
translation rules is equivalent to validity.

• 6. Decide what to do with irrelevant information: should this be ignored (e.g.

common English words like “the” and “and”), or used to reexamine the coding
scheme in the case that it would add to the outcome of coding?
Cont…..
7. Code the text: This can be done by hand or by using software. By using software, researchers
can input categories and have coding done automatically, quickly and efficiently, by the software
program.
When coding is done by hand, a researcher can recognize errors far more easily (e.g. typos,
misspelling). If using computer coding, text could be cleaned of errors to include all available
data.
This decision of hand vs. computer coding is most relevant for implicit information where
category preparation is essential for accurate coding.

8. Analyze your results: Draw conclusions and generalizations where possible. Determine what to
do with irrelevant, unwanted, or unused text: reexamine, ignore, or reassess the coding scheme.
Interpret results carefully as conceptual content analysis can only quantify the information.
Typically, general trends and patterns can be identified.
 TYPE OF CONTENT ANALYSIS

 Conceptual analysis

 Procedural analysis

 Relational analysis

 Emotional analysis
 1.Conceptual Analysis
What concept are presented in text or set of text.
Also known as thematic analysis
the focus here is on looking at the occurrence of selected terms within a text or texts
• Explicit
• Implicit
Methods
•1.Dictionaries (Harvard and Lass well dictionaries)
•2.Translation rules.
Strength
• Totally automated and applied to vast amount of data
Weakness
• Words with Same frequency and very different meaning
• Less effective at micro level
 2.Procedural analysis

What procedure or actions are presented in the text?

Types
1.Decision based
• Single actor in a task such as chess or mathematics to focus what a writer is thinking and doing in task
• To locate implicit or explicit rules that writer used to perform a particular task or error they make

•2.Plot based
• This is typically uses article, books, stories, and focuses on the story or plot.
3.Relational analysis
It focuses on both on what concepts are present in the text and on the
relation between those concept.
• Examining the data among concept in the texts.
• Theoretical Approaches
1. linguistic
• Focus on text on the level of linguistic unit (single)
• Grammatically coded a text
2. Cognitive
• Decision map
• Mental model
3.Relational Analysis
• Relational analysis begins like conceptual analysis, where a concept is chosen for
examination.
• However, the analysis involves exploring the relationships between concepts. Individual
concepts are viewed as having no inherent meaning and rather the meaning is a product of
the relationships among concepts.
• To begin a relational content analysis, first identify a research question and choose a sample
or samples for analysis.
• The research question must be focused so the concept types are not open to interpretation
and can be summarized.
• Next, select text for analysis. by balancing having enough information for a thorough
analysis so results are not limited with having information that is too extensive so that the
coding process becomes too arduous and heavy to supply meaningful and worthwhile
results.
Cont….
• There are three subcategories of relational analysis to choose from prior to going on to
the general steps.
• 1.Affect extraction: an emotional evaluation of concepts explicit in a text. A challenge to
this method is that emotions can vary across time, populations, and space. However, it
could be effective at capturing the emotional and psychological state of the speaker or
writer of the text.
• 2.Proximity analysis: an evaluation of the co-occurrence of explicit concepts in the text.
Text is defined as a string of words called a “window” that is scanned for the co-
occurrence of concepts. The result is the creation of a “concept matrix”, or a group of
interrelated co-occurring concepts that would suggest an overall meaning.
• 3.Cognitive mapping: a visualization technique for either affect extraction or proximity
analysis. Cognitive mapping attempts to create a model of the overall meaning of the
text such as a graphic map that represents the relationships between concepts.
 Cont…
• General steps for conducting a relational content analysis:
1. Determine the type of analysis: after selecting sample, the researcher
needs to determine what types of relationships to examine and the level of
analysis: word, word sense, phrase, sentence, themes.
2. Reduce the text to categories and code for words or patterns. A
researcher can code for existence of meanings or words.
3. Explore the relationship between concepts: once the words are coded,
the text can be analyzed for the following:
 Strength of relationship: degree to which two or more concepts are related.
 Sign of relationship: are concepts positively or negatively related to each other?
 Direction of relationship: the types of relationship that categories exhibit. For example, “X implies Y” or “X occurs before Y” or
“if X then Y” or if X is the primary motivator of Y.
Cont….
4. Code the relationships: a difference between conceptual and
relational analysis is that the statements or relationships between
concepts are coded.

5. Perform statistical analyses: explore differences or look for

relationships among the identified variables during coding.

6. Map out representations: such as decision mapping and mental

models.
4.Emotional analysis
Many content have affective or emotional content which if ignored may reduce the validity of
the analysis but which if attended to may increase the researchers ability to interpret the text.
However , a number of difficulties in attending to the affective content of the text many of
which stem from the fact that a large no of emotions are not semantically distinct.
Example
 john loves Mary, john was broken hearted when Mary eloped Thomas
Emotion reveal underlying goal,
emotions signal the occurrence of expectation failure.
Emotion indicate the status of interpersonal relation.
2 goals. Becoming un-hungry and achieving goal
5 emotion love, hate, fear, anger, oblivion(state of being unaware or unconscious)
Steps And Considerations In Conducting And
Reporting Content Analysis
1. Identify Data Sources Based On Researchable Question(s) or sub-
question(s) to be answered

• What is the source of the information?

• Will you use a sample or content analyze the population?
• How will you obtain the information, and what kind of format will it
be in? What is your unit of analysis? (For example, you
• might choose items from a survey, or comment fields from an agency
database.) Once you’ve decided on your unit of analysis, decide on
selection of items to be analyzed. Will you do a content analysis of a
sample or the entire population?
2. Develop and refine categories
• This step often takes a while, and is arguably the most important step in
content analysis. It requires a similar amount of effort no matter which
content analysis approach or tools you are using. Category development
requires iterations and multiple people are generally involved in identifying
the categories.
Inter-rater reliability is important; you need to make sure the categories are
the right ones, that is, useful given the researchable questions, and that more
than one person would agree on the categories and their meanings. After a trial
phase on a sample of the data to develop the coding categories (and before full
coding begins), a work paper should be developed which contains explicit
definitions of codes and any preliminary disagreement between coders.
Cont….
• An approach for doing this would be to:
• Test a few items as a team (either independently or as a group) to identify
items that may be ambiguous or still need further clarification.
• Code a subset of items using two independent analysts.
• Calculate preliminary inter-rater reliability.
• Refine categories, defining specifications about what should be included as
well what should not be included when there may be uncertainty to ensure
reliability of coding Pay attention to validity and reliability concerns both in
• creating/refining the categories and in placing the items into categories.
This ensures the content analysis will be defensible and accurate,
3. Code the data

• Each coder reads each item and places it into one or more category,
depending on the coding scheme that is being used.
• The decision on how many analysts will code data should be made
with respect to time and resources available to the engagement team,
risk level of the job, and if the analysis will provide sole support or
corroborative support findings, conclusions, or recommendations.
4. Independently code the same data
• In coding the data, a decision must be made as to whether two
analysts will independently code the same data, or if a less rigorous
method will be performed where one analyst will code the data and
another analyst will verify each decision made by coder as to whether
or not they agree with the code selected.

• Independent coding tests whether judgment is consistent between

coders and ensures objectivity, and therefore requires limited
additional corroboration. A less rigorous method likely needs to rely
on more corroborating evidence.
5. Assess Reliability

• The fourth step is to conduct reliability checks to ensure objective and accurate categorization process
• Although attention to reliability was given during the first step as categories were developed and refined,
overall reliability should be assessed to see how often the coders agreed (e.g. 80%). Because of the
attention in the beginning, the agreement between coders (which we refer to as inter-rater reliability)
should be fairly high.
• The subsequent assessment of inter-rater reliability will determine the extent that the coders agreed on
item placement into the categories, and will be used to identify disagreements that need to be resolved.
For items in which there was not agreement, the two analysts should meet to discuss their thinking about
the decision-making process and why they each selected the category that they did.
• Often, this discussion leads to an agreement between the two analysts about which category is most
appropriate. For instances in which an agreement was not reached, a third person who did not participate
in the initial coding should review and arbitrate.
• It is recommended that the third person conduct a blind review – in other words, they will see which codes
were assigned but they will not know which analyst assigned the corresponding codes. Blind reviews help to
minimize subjectivity or biases the adjudicator may have regarding one of both of the analysts.
6. Summarize/analyze results
• Based on your coding scheme, there are a various ways to summarize the results of a
content analysis effort into information that will be useful in a GAO product. Although
content analysis is much more than simply counting, you may find it useful to not the
number and/or percentage of items in each category as you discuss themes that
emerged.

• For example, noting number or percentage of items that have correlated responses
(for example, do people who report being satisfied with their jobs also report higher
raises?)

• If there are a lot of categories, you may want to “roll up” the categories into broader
categories, each capturing several
Cont…

• subcategories – this is defensible, as long as you keep validity and

reliability in mind, for example, by ensuring that more than one person
would agree on the “roll up,” and making sure the items still fit into the
larger categories. The level at which you report your findings should be
consistent with your researchable questions..
• All steps need to be documented for the workpapers, including how items
were selected, how categories were developed and finalized, what steps
were taken to ensure validity and reliability of both the categories
themselves and the placement of items in them, and how the results were
summarized.
RELIABILITY AND VALIDITY
 Reliability
• Reliability here refers to replicability or consistency in the coding or interpretation
of content or portions of content. Reliability issues associated in content analysis
are with the ambiguity of word meanings or coding rules.
• Three types of reliability are relevant to content analysis which are:
• Stability refers to the extent which content classification in invariant over time.
Stability can be ascertained when the same content is coded more than once by
the same coder.it is relatively weak form of reliability.
• Reproducibility(inter-coder reliability) refers to the extent to which content
classification produces the same results when the same text is coded by more than
one coder. High reproducibility is the minimum standard of for content analysis.
• Accuracy the strongest form of reliability refers to the extent to which the
classification of text corresponds to the a particular standard or norm.
 Validity Of A Content Analysis

1.Closeness of categories: this can be achieved by utilizing multiple classifiers to arrive at an agreed upon definition of each
specific category.
 Using multiple classifiers, a concept category that may be an explicit variable can be broadened to include synonyms or
implicit variables.
2.Conclusions: What level of implication is allowable? Do conclusions correctly follow the data? Are results explainable by
other phenomena? This becomes especially problematic when using computer software for analysis and distinguishing
between synonyms.
For example, the word “mine,” variously denotes a personal pronoun, an explosive device, and a deep hole in the ground
from which ore is extracted.
Software can obtain an accurate count of that word’s occurrence and frequency, but not be able to produce an accurate
accounting of the meaning inherent in each particular usage. This problem could throw off one’s results and make any
conclusion invalid.
3.Generalizability of the results to a theory: dependent on the clear definitions of concept categories, how they are
determined and how reliable they are at measuring the idea one is seeking to measure.
Generalizability parallels reliability as much of it depends on the three criteria for reliability.
Strength of content analysis
looks directly at communication via texts or transcripts, and hence gets at the
central aspect of social interaction
can allow for both quantitative and qualitative operations
can provides valuable historical/cultural insights over time through analysis of texts
allows a closeness to text which can alternate between specific categories and
relationships and also statistically analyzes the coded form of the text
can be used to interpret texts for purposes such as the development of expert
systems (since knowledge and rules can both be coded in terms of explicit
statements about the relationships among concepts) is an unobtrusive means of
analyzing interactions provides insight into complex models of human thought
and language use
 Advantages of Content Analysis
Directly examines communication using text
Allows for both qualitative and quantitative analysis
Provides valuable historical and cultural insights over time
Allows a closeness to data
Coded form of the text can be statistically analyzed
Unobtrusive(not easily noticed) means of analyzing interactions
Provides insight into complex models of human thought and language use
When done well, is considered a relatively “exact” research method
Content analysis is a readily-understood and an inexpensive research method
• A more powerful tool when combined with other research methods such as interviews,
observation, and use of archival records. It is very useful for analyzing historical material,
especially for documenting trends over time.
 Disadvantages of Content Analysis
Can be extremely time consuming
Is subject to increased error, particularly when relational analysis is used to attain a
higher level of interpretation
Is often devoid of theoretical base, or attempts too liberally to draw meaningful
inferences about the relationships and impacts implied in a study
Is inherently reductive, particularly when dealing with complex texts
Tends too often to simply consist of word counts
Often disregards the context that produced the text, as well as the state of things
after the text is produced
Can be difficult to automate or computerize
Readings and references

1. Fang Hseih, Hsiu., Shannon, Sarah. E. (2005). Three approaches to qualitative content
analysis, Qualitative Health Research- Sage Publications, Vol. 15, No. 2, pg. 1277-1285.
2. Sandorova, Zuzuna. (2014). Content analysis as a research method in investigating the
cultural components in foreign language textbooks, Journal of language and culture
education, pg. 95-123.
3. http://www.utsc.utoronto.ca/~kmacd/IDSC10/Readings/Readings/text%20analysis/CA.pdf
4. http://www.zoltandornyei.co.uk/uploads/2012-dornyei-csizer-rmsla.pdf
5. http://www.paxamerica.org/2012/09/01/qualitative-content-analysis-in-social-research-an-
epigrammatic-summation-of-presidential-state-of-the-union-addresses
/
6. http://www.utsc.utoronto.ca/~kmacd/IDSC10/Readings/Readings/text%20analysis/CA.pdf
7. http://www.utsc.utoronto.ca/~kmacd/IDSC10/Readings/Readings/text%20analysis/CA.pdf
References
• Frankel, J. R. & Wallen, N. E. (2006). How to design and evaluate
research in education, sixth edition. Boston: McGraw Hill.
• Journal of Agricultural Education
• Journal Article
• Implications of Performance Measures and Standards for Evaluation and Assessment in Agricultural Education
• Prolific Authors in the Journal of Agricultural Education: A Review of the Eighties
• An Examination of Selected Preservice Agricultural Teacher Education Programs in the United States