See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/258508505

Can Online Courses Deliver In-class Results? A Comparison of

Student Performance and Satisfaction in an Online versus a
Face-to-face Introductory Sociology Course

Article in Teaching Sociology · October 2012

DOI: 10.1177/0092055X12446624

CITATIONS READS
298 3,765

5 authors, including:

Adam Driscoll Karl A Jicha

University of Wisconsin - La Crosse North Carolina State University
8 PUBLICATIONS 1,105 CITATIONS 9 PUBLICATIONS 464 CITATIONS

SEE PROFILE SEE PROFILE

Andrea N. Hunt Lisa Tichavsky

University of North Alabama Northern Arizona University
26 PUBLICATIONS 1,332 CITATIONS 8 PUBLICATIONS 670 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Karl A Jicha on 28 May 2014.

The user has requested enhancement of the downloaded file.

446624
urnal of Management EducationDriscoll et al.
2012
TSOXXX10.1177/0092055X12446624Jo

Article
Teaching Sociology

Can Online Courses Deliver 40(4) 312­–331

© American Sociological Association 2012
DOI: 10.1177/0092055X12446624
In-class Results? http://ts.sagepub.com

A Comparison of Student
Performance and Satisfaction in
an Online versus a Face-to-face
Introductory Sociology Course

Adam Driscoll1, Karl Jicha1, Andrea N. Hunt1,

Lisa Tichavsky1, and Gretchen Thompson1

Abstract
This study uses a quasi-experimental design to assess differences in student performance and satisfaction
across online and face-to-face (F2F) classroom settings. Data were collected from 368 students enrolled
in three online and three F2F sections of an introductory-level sociology course. The instructor, course
materials, and assessments were consistent between the two delivery formats. The investigators compare
student satisfaction and student performance on midterm exams and an integrating data analysis
assignment. Ordinary least squares regression is used to evaluate the effect of the different course settings,
independent of a number of demographic and control variables. Results indicate that differences in student
performance between the two settings may be accounted for by the presence of a selection effect and that
student satisfaction does not significantly differ across the two settings. These findings are interpreted to
mean that when online courses are designed using pedagogically sound practices, they may provide equally
effective learning environments.

Keywords
online education, online pedagogy, distance learning, distance education

The prevalence of online education in institutions of Despite this intensifying relevancy, online educa-
higher learning is increasing rapidly (Means tion has been relatively unmentioned in the pages of
et al. 2009). As colleges and universities attempt to Teaching Sociology, outside of Clark-Ibáñez and
reconcile budget reductions and rising student Scott’s (2008) thorough article on best practices for
enrollment, online courses offer a cost-effective online teaching and a few other studies that examine
alternative to the traditional classroom. Currently, particular online techniques and strategies (Jaffee
nearly 30 percent of higher education students take 1997; Little, Titarenko, and Bergelson 2005; Pear-
at least one online course during their academic son 2010). What is especially lacking is empirical
career, and online enrollment is increasing at a sub-
1
stantially faster rate than that of overall higher edu- North Carolina State University, Raleigh, USA
cation (Allen and Seaman 2010). Consequentially,
Corresponding Author:
understanding the relative effectiveness of online
Adam Driscoll, North Carolina State University, 334
learning environments is an issue of increasing 1911 Building, Campus Box 8107, Raleigh, NC, USA
importance and the subject of growing debate. Email: ardrisco@ncsu.edu

Downloaded from tso.sagepub.com at ASA - American Sociological Association on October 1, 2012

Driscoll et al. 313

work that examines the effectiveness of online factors and selection effects including demograph-
courses relative to traditional face-to-face (F2F) ics, academic background, experience and comfort
classroom settings in the field of sociology. with the online environment, and student learning
The broader literature on the efficacy of online preferences. We also use a more nuanced statistical
courses is expansive and divided. A large number analysis, as multivariate regression allows for
of empirical comparisons between online and F2F greater elaboration of causal relationships than the
courses find that online students perform as well as analysis of variance and t-test analyses that domi-
or better than F2F students (Russell 1999; Tucker nate most comparisons of online and F2F courses.
2001). Additionally, studies have shown that stu- Finally, we are unaware of any previous study that
dent satisfaction does not significantly differ has compared online to F2F instructional methods
across the two instructional mediums (Allen et al. within a sociology course.1 As course content may
2002; York 2008), providing further support that produce different results due to different pedagogi-
online classes can be equally effective learning cal approaches and subject matter, it is important
environments. However, many of the studies to explore the difference between the two formats,
within this body of literature suffer from a range of specifically pertaining to teaching sociology
methodological weaknesses, such as relying on courses, rather than rely on comparative studies
small, nonrandom samples; failing to replicate done within other fields.
findings; lacking demographic controls; and com- In the section that follows we examine the
paring courses with substantial differences in con- debate surrounding the efficacy of online learning
tent, materials, instructors, and methods of and then present a brief review of the literature on
evaluating student performance (Bernard et al. best practices within online education to provide a
2004; Jahng, Krug, and Zhang 2007; Means et al. background for the rationale used in designing the
2009; Urtel 2008). Recent work that uses compari- course being studied. Following this we briefly
sons of larger samples and replicated courses describe the course and present our research
demonstrates that F2F students score higher grades method. After presenting our findings, we con-
on identical assessments (Urtel 2008). clude with a discussion of the potential of online
The current study addresses a number of the instruction and possibilities for making it a more
methodological shortcomings outlined above by effective teaching medium.
using a quasi-experimental design that compares
student performance and satisfaction between
online and F2F sections of a course that was taught
Literature Review
by one instructor over multiple terms with very Debates on the Efficacy of Online
little change in course materials and assessments. Education
When the course was designed, deliberate effort
was made to keep the two types of classes as simi- The online education literature is characterized by
lar as possible. By keeping the instructor, course a debate on the effectiveness of online versus F2F
material, and assessments relatively constant classrooms. Empirical studies have both supported
between the two sections, we eliminate a substan- (McFarland and Hamilton 2005; Parkhurst et al.
tial amount of the variation that has weakened 2008; Russell 1999; Summers, Waigandt, and
previous studies. Additionally, this study obtains a Whittaker 2005; Tucker 2001) and refuted (Logan,
higher degree of external validity than the majority Augustyniak, and Rees 2002; Urtel 2008) the abil-
of prior studies by incorporating data from multi- ity of online courses to provide a learning experi-
ple sections of the same course offered over multi- ence that is equal to that of an F2F classroom.2
ple terms. Most comparisons of online to F2F Meta-analyses performed on the literature as a
classes have relied upon smaller classes, typically whole tend to find a lack of significant difference
only taught once, which leaves those findings of between the two types of classes, but this is often
questionable generalizability. Furthermore, by col- a reflection of an even divide in the literature
lecting survey data from students, we are able to between studies that find F2F classes outperform-
control for a wide range of potentially mitigating ing online courses and those that find the opposite

Downloaded from tso.sagepub.com at ASA - American Sociological Association on October 1, 2012

314 Teaching Sociology 40(4)

(Allen et al. 2004; Bernard et al. 2004; Jahng et al. learning is certainly possible and does occur in
2007; Sitzmann et al. 2006). The arguments sur- well-designed F2F courses, the remote nature of
rounding this debate can largely be grouped into online education creates a structural impetus for
two broad categories. On one hand, some argue this style of learning that is not automatically pre-
that the online environment can be an equal or sent in F2F classrooms.
superior medium for education due to its flexibility The online environment can also provide a
and student-centered approach (Logan et al. 2002; more comfortable venue for participation among
Russell 1999; Summers et al. 2005; York 2008). students who are shy or lack confidence and who
On the other hand, others argue that online class- may be intimidated by the public setting of an F2F
rooms have yet to be proven an effective teaching classroom (Clark-Ibáñez and Scott 2008). Addi-
medium and that there is substantial basis for tionally, online courses provide great flexibility
doubt due to online education’s “McDonaldized” and access to multiple instructional methods that
(Ritzer 2004) nature and the distance that it may not be available in the traditional classroom
imposes between students and their instructors as (Sitzmann et al. 2006). Within the online environ-
well as students and their peers (Parkhurst et al. ment, particularly within courses that are struc-
2008; Urtel 2008). tured around an asynchronous design, students
One of the earliest and most significant publi- have the ability to learn at their own pace, go back
cations that falls into the first category is Russell’s and reread or review portions of lectures they
(1999) The No Significant Difference Phenome- found challenging, take breaks when they are tired,
non. Although looking at a broader range of non- and work at times most conducive to their own
traditional classroom settings than just online learning (York 2008).
courses, this seminal study found that when tech- Within the second category of arguments,
nology is appropriately and practically applied, it numerous doubts and questions have been raised
does not necessarily denigrate instruction. Rus- as to the efficacy of the online environment as an
sell’s (1999) argument is that the amount of learn- instructional medium. Some have argued that the
ing that occurs in a course is independent of the increased prevalence of online learning in higher
instructional medium or the technology involved education represents a “McDonaldization” of the
and instead depends on the pedagogical practices learning process, where standardized courses built
used. Effective teaching can occur in any class- around generic content and multiple choice exams
room setting where both students and teachers are are replacing distinctive classes taught by special-
invested in the learning process. ized faculty (Ritzer 2004:155). Many of these
The online environment does offer distinct doubts center on concerns that online classrooms
opportunities for flexible, student-centered learn- cannot fully replicate the interaction that occurs
ing. Within online courses, students are required to within the F2F classroom, which is vital to the
take responsibility for their own education and are learning process (Rovai and Barnum 2003). Stu-
forced to be proactive in the learning process dents learn far more in courses than direct content,
(Logan et al. 2002). Without an instructor present and in an online environment they do not have the
to provide pace, order, and focus, students must same opportunities for spontaneous, open discus-
self-regulate their work and assume greater sion with their instructors and peers (Bok 2003:89).
accountability for the learning process. Addition- Although online courses use a host of electronic
ally, the lack of a professor who is immediately forms of interaction (announcements, discussion
available to respond to questions about the mate- boards, e-mail, etc.), some scholars argue that
rial can prompt students to discover answers on these are not comparable to a real-time, in-person
their own, a process that usually reinforces knowl- discussion (Summers et al. 2005:246).
edge more strongly (Atkinson and Hunt 2008). If online courses are inherently lacking in their
Essentially, students must participate directly in ability to provide interaction, then they are at a
the construction of knowledge and cannot rely on serious disadvantage in creating an effective learn-
passive, “instructivist” pedagogy (Summers et al. ing environment. This lack of F2F interaction can
2005:236). Although such student-centered, active also create impressions of isolation and alienation

Downloaded from tso.sagepub.com at ASA - American Sociological Association on October 1, 2012

Driscoll et al. 315

among students (Gallager and McCormick 1999; satisfaction can be an indicator of timely and sub-
McIsaac et al. 1999). Additionally, the content of stantive interaction between students and their
discussions can differ between online and F2F instructor and it may also indicate that the teaching
classrooms, as some studies have shown that stu- methods being used strongly reflect learning goals
dents in F2F classes tend to ask far more technical and student expectations (Moore 2005). Addition-
and logistical questions during discussions, whereas ally, student satisfaction can influence student
online students voice more content-based questions motivation and attitude, both of which may
(Logan et al. 2002). These different discussions strongly influence how well students learn in a
may be granting students in the F2F classes a course (McFarland and Hamilton 2005). As with
clearer understanding of requirements and instruc- student performance, the link between student
tions. The differences in understanding may satisfaction and successful learning is debatable, as
account for observations of lower student satisfac- students may report higher satisfaction with
tion in online courses, as students can feel that the courses that they perceive as easy, fun, or less
instructor’s explanations, concern toward students, demanding, none of which are necessarily linked
and interest in student learning all suffer in the to successful learning. Still, sufficient reason exists
online environment (Summers et al. 2005). Finally, to expect that student satisfaction may be higher in
online courses tend to rely more exclusively upon classrooms where successful learning is taking
self-regulated learning, and not all students are suf- place.
ficiently proficient in the skills necessary to suc-
ceed in such an environment. Students who are not
comfortable with learning-centered experiences
Best Practices in Online Education
may be at a disadvantage in an online classroom. Although online education is still a relatively
Studies on both sides of the argument about the recent and growing form of instruction, there is an
efficacy of the online classroom generally use emerging field of literature that describes (and
similar strategies to operationalize successful learn- debates) what teaching practices, strategies, and
ing. The majority of such research relies on student methods work best to promote student learning in
performance on standard evaluations (Logan et al. online settings. Although the fundamental princi-
2002; Olson 2002; Summers et al. 2005) and final ples of quality pedagogy are constant across both
course grades (McFarland and Hamilton 2005; online and F2F mediums (good teaching is always
Urtel 2008; York 2008). Obviously, a wide range of good teaching), translating those elements into the
variance exists within the different types of evalua- online environment presents a unique challenge.
tions, exams, questions, and grading rubrics used Although we discuss the following principles in
by the various instructors in the different courses. terms of their importance to online education, it is
Additionally, the efficacy of exams and course worth noting that they are also applicable to hybrid
grades in measuring successful student learning is courses that use online elements in conjuncture
far from certain, as different evaluations vary in with traditional teaching methods.
their ability to capture authentic learning (Lom- The most ubiquitous principle that underlies
bardi 2008). However, they are the most universal discussions of online teaching practices is that
and easily accessible indicators and often provide quality online courses must incorporate a substan-
reliable information on how well students are tial amount of interaction, both among the students
achieving course objectives (Suskie 2004). and between the students and the instructor (Clark-
Student satisfaction is another widely used Ibáñez and Scott 2008; Sumner 2000). Interaction
indicator of successful learning (McFarland and in the online environment consists of a wide range
Hamilton 2005; Parkhurst et al. 2008; Summers et of text-based exchanges which, although limited in
al. 2005; York 2008). This operationalization relies comparison to in-person conversations, allow for a
upon the argument that when students report their broad interchange of ideas, questions, and opin-
satisfaction with a course, they are assessing the ions. The level of interaction has been shown to be
quality of their learning experience (Piccoli, a predictor of perceived learning in online courses
Ahmad, and Ives 2001). A high level of student (Rovai and Barnum 2003). Interaction is essential

Downloaded from tso.sagepub.com at ASA - American Sociological Association on October 1, 2012

316 Teaching Sociology 40(4)

to the learning process, and incorporating it into The effectiveness of the learning environment
the online classroom is one of the primary chal- created within the context of an online course is
lenges in designing an effective course (Brooks also highly influenced by course structure. A clear
1997). Interaction and discussion among peers course design and layout are essential elements of
promote critical thinking and force students to an effective online learning environment (Clark-
engage with course material at higher levels of Ibáñez and Scott 2008). As online learning is new
learning. Through interaction among students, to many students, understanding expectations,
learning occurs in the social realm where students requirements, and how to access course materials
can benefit from one another’s insights and essen- can often be a challenge. A clear and simple course
tially teach each other, learning both as the instruc- organization can help students overcome that chal-
tor and as the instructed. Additionally, interaction lenge (Ko and Rossen 2008). Students within
between students and the instructor is essential for online courses have indicated a preference for
providing feedback and encouragement as well as well-organized content and coursework (Tucker
clarifying instructions, due dates, and expecta- 2001). It can also be very helpful to give students
tions, which is most essential for students who some level of “frontloading” in the form of an
have experienced most of their education in tradi- introductory letter or e-mail that communicates to
tional classroom settings (Jaffee 1997). Interaction students some of the differences between online
in an online classroom can be fostered through a and F2F courses and strategies for being successful
number of devices, including discussion boards, in the online environment (Clark-Ibáñez and Scott
chat rooms, course announcements, online blogs, 2008).
and standard e-mail. Finally, successful online courses should be
One learning device that has been shown to designed around strong pedagogical standards
promote interaction in online courses is the use of rather than new, complicated modes of delivery.
asynchronous learning networks (ALNs), a term Students have repeatedly stated a greater concern
that denotes online classrooms through which stu- for solid teaching over “bells and whistles”
dents interact with one another and their instruc- (Clark-Ibáñez and Scott 2008:37). When educa-
tors but at times of their own choosing (Jaffee tors adopt curriculum to fit the technology, rather
1997; Moore 2005). ALNs create opportunities for than choose the technology that fits the curricu-
interaction while preserving the flexibility of time lum, the instructional pedagogy suffers (Bennett
and place that is such an essential component of and Green 2001). Although the judicious use of
online education. ALNs are a marked improve- technology can certainly enhance the learning
ment over correspondence-course styles of online process, abuse of multimedia elements can dis-
instruction, where information is presented in a tract and detract from actual content and learning.
static form and students are expected entirely to Students in online courses have reported that they
teach themselves. Through the use of computer- did not enjoy listening to lectures, tapes, and
mediated activities such as threaded discussions, speeches (Tucker 2001). Additionally, the inclu-
progressive writing assignments, or group wikis, sion of more media in online courses does not
students can take part in active, collaborative appear to affect the amount that students learn
learning that promotes engagement with the course (Means et al. 2009) and actually lowers student
material at higher levels of learning (Jaffee 1997). satisfaction levels (Allen et al. 2002). Rather than
The meta-analysis conducted by Bernard et al. design an online course around the teaching tools
(2004) found that when online courses relied on that are available, designers should make course
synchronous networks (live chats, streamed lec- goals and student learning outcomes the foci of a
tures, etc.) in which students had to adhere to a successful online course. Student expectations for
rigid schedule, F2F classes performed better. How- learning in online courses should match those of
ever, online courses that used an asynchronous F2F courses in both coverage and level of under-
design outperformed the F2F classes. Thus, the use standing. Although technology is a key element of
of ALNs is a vital component to a successful a successful online course, its use should always
online course design. be content driven.

Downloaded from tso.sagepub.com at ASA - American Sociological Association on October 1, 2012

Driscoll et al. 317

In summary, the literature reports that a well- As an introductory level course, it was open to all
designed online course is one that stresses interac- students, possessed no prerequisites, and fulfilled
tion and active participation, preferably through one of the university’s general education require-
the use of ALNs to facilitate higher-level learning ments. The course drew material from both anthro-
while maintaining student flexibility. Such a course pology and sociology and was generally taken by
should be structured in a clear and consistent man- a diverse range of majors. It was a popular course
ner, and the content and expectations of the course and was regularly filled to capacity. Course subject
should be equal to those of an F2F class. The tech- matter dealt with the role of technology in societies
nology and media that are used should be content throughout history and in processes of societal
driven and not included for the sake of providing change. The course objectives included students
“flash” to the course. Through these practices, an demonstrating the extent to which they increased
online course can use an active and engaged peda- their understanding and mastery of: (1) processes
gogy that promotes learning in any medium. of social and cultural change, (2) basic sociologi-
cal and anthropological concepts, (3) the social
impacts of technological innovation, (4) the role of
Hypotheses technology in producing and/or addressing social
This study tests whether student performance and problems, (5) the unequal distribution of the ben-
satisfaction intrinsically differ across online and efits of technology, and (6) how innovations in
F2F classroom settings, independent of student communications technology have affected social
characteristics. Drawing from the arguments above interaction.
that distance education can be an equally effective Student success in achieving these goals was
teaching format (Logan et al. 2002; Russell 1999; primarily assessed through three unit exams and an
Summers et al. 2005; York 2008), we hypothesize integrating data analysis (IDA) assignment (see
that when an online section of a course is designed Hilal and Redlin 2004 for a discussion). The exams
around best practices and various measures of stu- consisted of a range of multiple choice, short
dent aptitude and background are controlled for, answer, and essay questions that were designed to
the following will hold true: measure how well students were mastering the
content of that particular unit and meeting the
H1: There will be no significant difference overall learning goals for the course. By using
between online and F2F students’ perfor- questions that required students to apply course
mance on course exams. concepts, we were able to measure directly our
H2: There will be no significant difference second course goal. We also included a number of
between online and F2F students’ perfor- questions asking about specific historical exam-
mance on the integrating data analysis ples of technological innovation or requiring that
assignment. students provide their own, which enabled us to
H3: There will be no significant difference assess how well students were meeting the first
between online and F2F students’ satis- and third course goals. The essay questions were
faction levels. particularly designed to evaluate students’ higher-
level learning. By structuring them around differ-
ent social problems, historical and contemporary
Methods examples, and different course theories, we were
Course Design able to analyze how well students were meeting all
of the course goals. The IDA assignment was a
This quasi-experiment was conducted within an part of the second unit and required students
undergraduate, dual-listed anthropology/sociology to find various national-level indicators of devel-
course3 offered at a large urban university in North opment, well-being, and technological advance-
Carolina. The study qualifies as a quasi-experiment ment and analyze those statistics using course
due to our inability to randomly assign or match content. Students had to discuss both the social
students to the different study groups (i.e., classes). impacts of technological innovation and the

Downloaded from tso.sagepub.com at ASA - American Sociological Association on October 1, 2012

318 Teaching Sociology 40(4)

unequal distribution of the benefits of technology. and completed identical IDA assignments, all of
Thus, this assignment directly measured how well which were evaluated using the same grading
students were meeting the third and fifth learning rubrics. Over the three terms that this course was
goals. These evaluations are certainly limited offered, the instructor made several minor altera-
instruments for capturing how well students were tions to the material covered in the first and third
meeting all of the learning objectives for the unit exams but essentially kept the second unit of
course, but as a whole they provided a solid indica- the course intact. As the exams for the second unit
tion of how successfully students were learning. and the IDA assignment that was a part of that unit
The data used in this study were drawn from six were mostly consistent across all three terms, they
sections of the course that were taught over three provide the basis for evaluating student perfor-
consecutive terms (spring, summer, and fall) in mance in this study.5
2010. The course was offered as an online section The two types of courses primarily differed in
for all three terms and as an F2F section once in the the mode, timing, and location of course lectures
spring term and twice in the fall term. Both types and discussions. The F2F sections used standard
of sections were capped at 80 students, and class in-class lectures and unstructured discussions two
sizes ranged from 68 to 79. Within the three F2F to three times a week in a university classroom.
sections there were a total of 231 students, whereas Although the F2F sections could be qualified as
the three online sections contained 212. All sec- “hybrid” sections due to their use of some online
tions were taught by the same instructor (one of the teaching elements (e-mail correspondence, online
co-authors) and largely covered the same course assessments, and distribution of readings through
material. Given that the majority of the principles the LMS), we characterize them as F2F due to their
outlined above for best practices within online reliance upon traditional teaching methods as the
education (interaction, clear organization and primary mode of delivering information. For the
structure, and a focus on content over delivery online sections, the content and discussions were
method) are reflective of best practice in education entirely housed on the LMS. The material was
as a whole, the online and F2F versions of this clearly organized into weekly lessons that mirrored
course were purposely designed to be as similar as the F2F schedule. The presentation of material in
possible. Some minor refinements were made from the online sections relied on weekly PowerPoint
term to term, but the books, supplemental read- lecture slideshows and asynchronous, threaded
ings, and section topics were largely consistent discussion boards structured around specific
throughout. During the terms in which online and prompts. These discussion boards were a forum for
F2F sections were being taught simultaneously, interaction among students (students were required
both types of classes were built around identical to comment on each other’s posts) as well as inter-
learning goals and outcomes, covered the exact action with the instructor, as the instructor partici-
same content, used the same readings, and were pated in the discussions, provided feedback on
held to identical standards in terms of level of students’ posts, posed further questions, and graded
understanding. Both types of courses were taught the content of students’ posts. The discussion
using the same learning management system boards constituted an ALN, in that students were
(LMS).4 Evaluations for both types of classes were processing course material with one another but
administered online through the LMS, allowing all within a flexible schedule that allowed them to
students to take their exams at a location of their work at their own pace and at the times that were
own choosing. Students could access their exams convenient for them. The online sections also gen-
at any point within an approximate four-day win- erated interaction between the instructor and stu-
dow. Therefore, both the F2F and online sections dents through the use of regular announcements
took their exams and completed their IDA assign- that were both posted on the LMS and e-mailed to
ments under identical conditions. students. These announcements served to clarify
Again, during the terms in which online and instructions and remind students of upcoming
F2F sections were being taught simultaneously, deadlines as well as provide broad feedback on the
both types of classes took the exact same exams classes’ performance and grasp of course concepts.

Downloaded from tso.sagepub.com at ASA - American Sociological Association on October 1, 2012

Driscoll et al. 319

By constructing the online sections in this fashion, academic aptitude, we asked the participants to
the instructor was able to make the online and F2F self-report their estimated student GPA on a 7-point
sections as similar as possible while designing the scale ranging from 1.00-1.49 to 4.00 and above. To
online sections according to pedagogically based measure the amount of time students had available
teaching practices. to devote toward this course, we had them self-
report the number of credit hours taken during that
term as well as the hours worked per week at any
Data and General Procedure paying job. To control for basic university experi-
This study was approved by the university’s insti- ence, we had participants report their year in
tutional review board (IRB# 1371-10). The data school, which we collapsed into a dummy variable
used were obtained from two sources: the instruc- for senior versus nonsenior, as preliminary analy-
tor’s records of students’ grades on course assess- sis revealed that this was the only distinction that
ments and student responses to a semi-anonymous affected student performance. Finally, in order to
online survey. At the end of each term, all sections control for students’ experiences with online learn-
of the course were sent an e-mail inviting them to ing, we had participants indicate the number of
participate in the study. The e-mail contained a link online courses taken prior to this semester.
to an online survey that was constructed using In addition to controlling for potential selection
CALS Survey Builder, a Web-based tool for creat- effects among students, we measured student pref-
ing online surveys. The survey required approxi- erences for interaction with their peers and instruc-
mately ten minutes to complete and consisted of tor. As the online version of this course was
questions about students’ living statuses, learning structured so heavily on providing interaction, stu-
styles, experience with online education, access to dent inclination toward this style of learning could
the Internet, satisfaction with the instructor and have greatly influenced student performance and
course, and general demographics (see the appen- satisfaction. Therefore, we asked participants to
dix for select elements of the survey). To stimulate indicate on a 5-point Likert scale (5 indicating
response, numerous reminder messages were sent strongly agree and 1 indicating strongly disagree)
to students stressing the importance of the study, how much they enjoyed working with classmates
and the survey was left open for an entire month. (like working with others) and how much they felt
The response rates for the survey were 82.7 per- that success in a classroom requires frequent inter-
cent (overall), 79 percent (online), and 86 percent action with the instructor (instructor interaction).
(F2F).6 This gave us a total sample size of 368 Although these measures only capture student pref-
subjects (170 online, 198 F2F). Students provided erence for interaction, and not the amount of inter-
informed consent on the survey and reported the action provided, examining the degree to which
last four digits of their student ID numbers. Those students with a high preference for interaction were
ID codes were used to link the survey results satisfied with the course (one of our dependent
to students’ grade records, after which all identify- variables) does give us an indirect indication of how
ing information was deleted to preserve subject successful the two types of sections were at provid-
anonymity. ing an interactive learning environment.
For our dependent variables, we used students’
scores (on a standard 0-100 percentile scale) on the
Measures second exam and IDA assignment, as these were
We drew a number of predictor variables from the the two evaluations that were virtually constant
survey, including the basic demographic indicators across all terms. The second unit exam consisted of
of age and gender. While exploring the difference a range of multiple choice, short answer, and essay
in student performance and satisfaction between questions that were designed to measure how well
online and F2F settings, we controlled for a num- students were mastering the various learning
ber of confounding factors that could potentially objectives for that unit. The IDA assignment
create differences between the two classes through required students to research specific quantitative
a selection effect. To represent students’ general indicators of national development and technology

Downloaded from tso.sagepub.com at ASA - American Sociological Association on October 1, 2012

320 Teaching Sociology 40(4)

use and analyze those data using course concepts. reveal substantial differences between the popula-
For our third dependent variable we measured tions of the two types of classes. Students in the
student satisfaction with the instructor and course, F2F sections of the course generally had higher
as it is a widely used parameter that may indicate GPAs and were enrolled in more credit hours than
the effectiveness of learning environments (Piccoli students in the online sections. Students in the
et al. 2001). We constructed an additive index online sections tended to be older, to have taken
measure of student satisfaction from a number of more online courses, and to work more hours dur-
questions in our survey that required participants ing the week. Mean comparisons of the dependent
to evaluate both the course and the instructor. variables show that students in the F2F sections of
Exploratory factor analysis was used to test how the course performed better on both assessments
well the separate questions reflected a common than students in the online sections and that student
underlying dimension of student satisfaction. Prin- satisfaction did not differ significantly between the
ciple axis factoring revealed that the six items different types of courses. These initial univariate
characterize a single factor with a Cronbach’s comparisons replicate the basic findings of Urtel
alpha of .865, which strongly supports the validity (2008) and Logan et al. (2002), where in a simple
of the index. The factor loadings for the individual comparison without any controls, F2F students
variables ranged from .620 to a .836, which is suf- appear to perform better than online students on
ficiently high to justify merging them in a single course assessments. This serves as our starting
index (Hair et al. 1998). point for further analysis.
Table 2 contains the results for our first hypoth-
esis, that there will be no significant difference
Analysis between online and F2F students’ performances on
The analysis of our data was performed using ordi- course exams. Overall, the regression findings sup-
nary least squares (OLS) regression. As we were port the tested hypothesis and reveal a potential
analyzing the effect of the type of classroom (e.g., selection effect that may account for the observed
F2F vs. online) on student performance and satis- difference in student performance between the two
faction, net of the control variables described types of classes. In Model 1 there is a negative
above, OLS regression provided a parsimonious zero-order effect, as was shown in the univariate
tool that allowed us to fully elaborate the causal statistics, where online students perform signifi-
relationships of interest. Our basic experimental cantly worse on the second exam than F2F stu-
design for all of our dependent variables was to dents. However, once our control variable for
begin with a zero-order model that simply shows student GPA is introduced, this effect is eliminated,
the difference between the F2F and the online stu- indicating a potential selection effect among stu-
dents (course type) and then sequentially introduce dents taking the two different types of courses. As
our control variables in subsequent models to shown in Table 1, students taking the online ver-
examine how this alters the influence of the type of sions of the course tend to have a significantly
course. Our fifth model was fully saturated, con- lower GPA than students taking the F2F course.
taining all of our variables. This experimental Once included as a predictor variable, student GPA
design allowed us to see how controlling for vari- has a positive, significant influence on student
ous factors may “explain away” any initially per- performance across all models. This suggests a
ceived difference between F2F and online students selection effect, where academically stronger stu-
and expose potential selection effects that may be dents are gravitating toward the F2F sections of a
at work (such as stronger students being more course and performing better on the exam, creating
prone to take F2F courses than online versions). a corresponding difference in student performance
between the two types of classes.
The additional control variables in our analysis
Results of student performance on the midterm are largely
Table 1 presents the univariate statistics for the insignificant. Year in school, online courses taken,
variables tested. These basic descriptive statistics like working with others, gender, credit hours taken,

Downloaded from tso.sagepub.com at ASA - American Sociological Association on October 1, 2012

Driscoll et al. 321

Table 1. Descriptive Statistics for Variables in Analysis by Type of Course

Face-to-Face Online Classes

Classes (n = 198) (n = 170)
Difference in
Range M SD M SD Means
Independent variables
Student GPAa 5 4.75 1.044 4.37 1.084 0.386***
Year in school (senior) 1.00 0.500 0.501 0.577 0.496 –0.077
Online courses taken 10 0.98 1.400 2.22 2.355 –1.238***
Like working with others 4 3.12 1.081 3.11 1.110 0.010
Instructor interaction 4 3.61 1.044 3.72 0.966 –0.105
Gender (male) 1 0.56 0.498 0.55 0.499 0.003
Age 22/16 21.38 3.005 22.63 3.903 –1.251***
Credit hours taken 22/16 14.95 2.282 10.59 4.771 4.365***
 Hours worked per week 3.00 1.52 0.804 2.14 1.098 –0.620***
Dependent variables
Second exam 40/63 81.88 7.565 78.94 10.246 2.940**
IDA 46.5/62.5 87.53 7.615 84.99 10.179 2.540**
Student satisfaction 23/12 25.30 3.652 25.24 3.230 0.060

Note: A t-test for equality of means was used to determine the significance of the difference in means across course
type for the independent variables. The Mann-Whitney U test (nonparametric test of differences in mean scores) was
used for the dependent variables.
a
Student GPA is self-reported on a 7-item scale, not the standard 4.0 scale (see Methods for details).
*p < .05. **p < .01. ***p < .001, using a two-tailed t-test.

and hours worked per week all show no significant low R-square values are not of concern. What is
influence on student performance on the second revealing, however, is the pattern of R-square val-
exam.7 Additionally, the inclusion of these variables ues that we see among the different models. In
does not affect the predictive power of student GPA Model 1, when we only examine course type, we
or the absence of a significant effect for course type, see that although it is a significant predictor of
indicating that they are not a part of any selection student performance, it explains very little of the
effect. The instructor interaction variable does variation within that outcome (2.4 percent). When
show a significant negative effect on student perfor- student GPA is included in subsequent models, the
mance on the midterm in the saturated model, indi- amount of variation explained jumps up to 21 to 22
cating that the more important students think that percent. This lends further support to the presence
interacting with their instructor is, the worse they of a selection effect, where student aptitude is the
tend to do on the exam. Instructor interaction does most important predictor of student performance,
not differ between the two types of courses (see and it is only the increased presence of stronger
Table 1) and its inclusion does not affect the impact students in the F2F sections of the course that cre-
of course type or student GPA, indicating that this ates the appearance of the online classroom being
effect holds true for both types of students and that a less effective learning environment.
this variable does not play a role in the potential Table 3 contains the results for our second
selection effect. hypothesis, that there will be no significant differ-
In examining the adjusted R-square values for ence between online and F2F students’ perfor-
our models, we are not overly concerned with the mance on the IDA assignment. Once again, the
total amount of variance explained, as the focus of results largely support our hypothesis and point to
this paper is not on explaining student performance a potential selection effect at work. In Model 1,
but rather is on understanding the impact of course course type shows a significant negative zero-order
type on that performance. Therefore, the relatively effect on student performance, again replicating the

Downloaded from tso.sagepub.com at ASA - American Sociological Association on October 1, 2012

322 Teaching Sociology 40(4)

Table 2. Ordinary Least Squares (OLS) Regression Models Predicting Second Exam

Independent and
Control Variables Model 1 Model 2 Model 3 Model 4 Model 5
Course type (online) –2.944** –1.446 –1.105 –1.009 –1.549 (0.995)
(0.931) (0.848) (0.881) (0.875)
Student GPA 3.741*** 3.669*** 3.547*** 3.532***
(0.391) (0.394) (0.393) (0.395)
Year in school 0.634 0.789 0.727 0.694
(senior) (0.834) (0.840) (0.835) (0.845)
Online courses taken –0.309 –0.369 –0.405
(0.222) (0.221) (0.223)
Like working with –0.635 –0.579
others (0.398) (0.400)
Instructor interaction –0.794 –0.890*
(0.428) (0.433)
Gender (male) –0.950
(0.839)
Credit hours taken –0.104
(0.122)
Hours worked per 0.227
week (0.471)
Intercept 1.884*** 63.788*** 64.356*** 69.870*** 71.902***
(0.633) (1.984) (2.022) (2.768) (3.633)
Adjusted R2 .024 .216 .217 .232 .231

*p < .05. **p < .01. ***p < .001, using a two-tailed t-test. Unstandardized coefficients reported with standard errors in
parentheses.

initial difference shown in Table 1. Once student Table 4 contains the results for our third hypoth-
GPA is introduced, however, that effect is elimi- esis, that there will be no significant difference
nated, lending support to the possibility of a selec- between online and F2F students’ satisfaction with
tion effect. As the impact of student GPA is both their course. The results support our hypothesis, as
positive and significant, students with higher GPAs there is no observed influence of course type on
tend to perform better on the IDA assignment, and student satisfaction in any of the models. There is
the increased prevalence of those stronger students also no evidence of any selection effect at work, as
in the F2F sections may account for those sections the insignificant effects of course type stay con-
performing better on the assignment. None of the stant across all of the models. Of interest in this
other control variables introduced in this analysis table are the results of our two variables that meas-
yield a significant effect on student performance or ure student learning preferences, like working with
substantially alter the effects of course type and others and instructor interaction. These variables
student GPA, indicating that those factors are not measure students’ preferences for interacting with
part of any selection effect. We see a similar pattern their classmates and their instructor, respectively.
among the adjusted R-square values for this analy- Both show a significant, positive impact on student
sis as we did with the second unit exam, which satisfaction, indicating that students who prefer an
provides further support to the presence of a selec- interactive mode of learning are more likely to be
tion effect. With just course type included, only 1.7 satisfied with the course. As neither student prefer-
percent of the variation in student performance is ences for interaction nor student satisfaction sig-
explained. Once student GPA is included, the per- nificantly differ across the two types of classes
centage increases to approximately 13 percent for (see Table 1), it appears that both types of courses
all subsequent models. were successful in adequately meeting students’

Downloaded from tso.sagepub.com at ASA - American Sociological Association on October 1, 2012

Driscoll et al. 323

Table 3. Ordinary Least Squares (OLS) Regression Models Predicting Integrated Data Analysis Assign-
ment

Independent and
Control Variables Model 1 Model 2 Model 3 Model 4 Model 5
Course type (online) –2.546** –1.397 –.1.140 –1.137 –1.750
(0.930) (0.891) (0.928) (0.931) (1.056)
Student GPA 2.973*** 2.919*** 2.893*** 2.953***
(0.411) (0.415) (0.418) (0.419)
Year in school (senior) –0.064 0.053 0.067 –0.145
(0.877) (0.884) (0.889) (0.896)
Online courses taken –0.233 –0.250 0.225
(0.234) (0.236) (0.237)
Like working with –0.261 –0.307
others (0.423) (0.424)
Instructor interaction 0.013 –0.003
(0.456) (0.460)
Gender (male) 1.383
(0.891)
Credit hours taken –0.193
(0.130)
Hours worked per –0.344
week (0.500)
Intercept 87.534*** 73.437*** 73.864*** 74.765*** 77.394***
(0.632) (2.085) (2.129) (2.946) (3.856)
Adjusted R2 .017 .136 .136 .132 .137

*p < .05. **p < .01. ***p < .001, using a two-tailed t-test. Unstandardized coefficients reported with standard errors in
parentheses.

wishes for an interactive learning environment. controlling for a number of other relevant variables
None of the other control variables included in this representing student aptitude, learning preferences,
analysis demonstrate a significant effect on student and background.
satisfaction or substantially alter the effect of any The results of this study support arguments that
other variables. online education can be an equally effective teach-
ing format when the online course is designed using
appropriate pedagogy. There was no significant dif-
Discussion and ference in student satisfaction between the two dif-
Conclusions ferent courses. The fact that students in both types
This study examines the degree to which student of classes were equally satisfied supports the idea
attainment and satisfaction are influenced by that the online sections of the course were success-
course type (F2F versus online). It represents a ful in promoting effective student learning. Further-
significant improvement over prior studies by more, students who enjoy working with others and
including data from one course that was offered in view interaction with their instructor as important to
both settings over multiple terms. This course was learning tended to be more satisfied with the course,
always taught by the same instructor, and subject independent of the type of classroom they are in.
material and evaluations were largely held con- Although our study did not directly measure the
stant over the different terms and between the two amount of interaction provided, these findings are
types of sections. Through the use of multivariate consistent with both types of courses providing suf-
regression techniques, we were able to explore the ficient interaction for students who prefer this mode
impact of the different classroom settings while of learning. These results support the proposition

Downloaded from tso.sagepub.com at ASA - American Sociological Association on October 1, 2012

324 Teaching Sociology 40(4)

Table 4. Ordinary Least Squares (OLS) Regression Models Predicting Student Satisfaction

Independent and
Control Variables Model 1 Model 2 Model 3 Model 4 Model 5
Course type (online) –0.053 –0.129 –0.223 –0.305 –0.629
(0.362) (0.370) (0.385) (0.371) (0.421)
Student GPA –0.156 –0.136 –0.040 –0.051
(0.171) (0.172) (0.167) (0.167)
Year in school (senior) –0.155 –0.197 –0.139 –0.154
(0.364) (0.367) (0.354) (0.357)
Online courses taken 0.085 0.131 0.110
(0.097) (0.094) (0.094)
Like working with 0.456** 0.490**
others (0.169) (0.169)
Instructor interaction 0.696*** 0.639***
(0.182) (0.183)
Gender (male) –0.597
(0.355)
Credit hours taken –0.066
(0.052)
Hours worked per 0.103
week (0.199)
Intercept 25.296*** 26.115*** 25.959*** 21.492*** 22.836***
(0.246) (0.865) (0.884) (1.175) (1.536)
Adjusted R2 –.003 –.005 –.006 .067 .074

*p < .05. **p < .01. ***p < .001, using a two-tailed t-test. Unstandardized coefficients reported with standard errors in
parentheses.

that students equally desire interaction in both ence in academic aptitude is unknown, which may
online and F2F settings and that a well-designed limit the generalizability of these findings to uni-
online course is capable of providing a sufficiently versities and colleges with different rules. The
interactive learning environment. increased presence of lower performing students in
In terms of student performance on course online sections may also be partially explained by
assessments, initial differences in student perfor- student perceptions that online courses will be
mance between the two types of class settings are easier than F2F courses. This explanation is sup-
potentially the result of a selection effect, where ported by studies that have found that students who
academically stronger students are more likely to prefer an online section tend to perform worse in
be enrolled in the F2F sections of the course than the class, regardless of whether they take it in an
the online sections. This effect may be partially online or F2F setting (Olson 2002). The perception
due to a university policy that allows students on that online courses are easier could have the dual
academic suspension, who are not permitted to effect of both attracting stronger and more com-
take F2F courses, to enroll in online courses as a mitted students to F2F sections, where they assume
way of demonstrating that they are fit to be rein- they will find a more enriching learning environ-
stated as full-time students. By definition, these ment, while simultaneously driving weaker, less
students have lower prior academic performance dedicated students to the online sections, where
than nonsuspended students and the policy that they think they will encounter a reduced work load
only allows them to enroll in online courses consti- and lower expectations.
tutes a fixed selection effect. The full extent to It is important to note that both student satisfac-
which this policy accounts for the observed differ- tion and student performance are only proximate

Downloaded from tso.sagepub.com at ASA - American Sociological Association on October 1, 2012

Driscoll et al. 325

measures of how successfully students are learning The findings in this study also reinforce the
and may inadequately capture key differences in utility of the best practices that were used while
the quality of the learning experience between the designing this course. Deliberate effort was made
two different types of classrooms. Future work that while designing the online version of the course to
operationalizes learning in a more precise fashion incorporate a high degree of interaction both
should be able to examine this possibility. How- between the students and the instructor and among
ever, the manner in which student learning was the students themselves. Through the use of ALNs,
evaluated is fairly consistent with standard assess- primarily in the form of threaded discussion
ment techniques (Suskie 2004), which gives our groups, students in the online sections had the
quasi-experiment a greater generalizability (exter- opportunity to critically engage with course mate-
nal validity) than we would have had if we had rial, much as students would do in a discussion in
used a more direct indicator of student success a traditional classroom. The online sections of the
in achieving a particular learning goal. Most course were also designed around the exact same
important for the arguments of this paper is not content as the F2F sections, meaning the teaching
the accuracy of our measures of successful student tools were adapted to the content and not vice
learning but the consistency in those outcomes versa. Finally, great effort was made to keep the
between the online and F2F sections of the online courses’ structure clear and consistent,
course. with simple and effective instructions. The fact
Overall, these findings support arguments that that students in the online sections were able
there is no inherent deficiency in the effectiveness to perform as well as those in the F2F sections
of the online classroom. They furthermore cast sig- (once GPA was controlled for) supports the use of
nificant doubt on a number of prior studies that, pedagogically based teaching practices for online
through failing to control for previous academic courses. Furthermore, it supports the argument that
performance, have found the opposite to be true. quality pedagogy leads to better learning out-
Our results suggest that the conclusions of any com- comes, regardless of the medium through which a
parison between online and F2F courses that fails to course is being taught. Many of the best practices
control for student aptitude should be called into that we used in designing the online version of this
question. Our results also highlight the need for course would be suitable for use in creating hybrid
further examination of which students are electing classes that rely upon both traditional and online
to take online classes and the reasons behind that methods of instruction.
decision. The students in our study who selected an An interesting and unexpected finding in this
online section of the course tended to be older, study is the negative impact of instructor interac-
tended to have lower GPAs, tended to have greater tion on student performance on the second unit
experience with online courses, were more likely to exam. This indicates that students who viewed
be college seniors, were taking fewer credit hours, interacting with their instructor as an important part
and were working a greater number of hours per of being successful in a course tended to do worse
week. These factors indicate a substantially differ- on the exam than students who did not. As this vari-
ent composition of the student population who take able does not significantly differ across the two
online courses compared with those who take F2F. types of classrooms or substantially influence any of
Future research should examine these composi- our main effects, this effect is likely present in both
tional differences more fully and explore the moti- class settings. Although this finding is relatively
vations behind students’ preferences for different tangential to our larger arguments, it still merits a
learning environments. Furthermore, with the pos- brief discussion. One potential explanation is that
sibility that the increased presence of stronger stu- academically weaker students are more reliant upon
dents in the F2F sections of the course is driven by their instructors and require a higher degree of inter-
the specific policies of the university where the action with them. These would be students who tend
study was conducted, there is a need to conduct to possess a lower ability to learn independently or
similar studies at other universities with different to grasp expectations and instructions immediately.
policies. These students could possibly be both more likely

Downloaded from tso.sagepub.com at ASA - American Sociological Association on October 1, 2012

326 Teaching Sociology 40(4)

to seek interaction with their instructor and more scholars are increasingly obligated to understand
likely to perform worse on exams. It is also possible its use and improve upon its implementation. As a
that the exam is a less effective assessment tool for new teaching medium, online education faces
measuring the type of student learning that occurs many of the same challenges that accompanied the
through interaction. Regardless, what is relevant to introduction of the textbook, the chalkboard, and
our central arguments is that this finding is consist- the computer to the classroom. Adjustments to
ent with both online and F2F courses providing a teaching strategies and methods are necessary to
satisfactory amount of interaction with the instruc- use new educational tools effectively, and increased
tor. research and discussion of the strategies and meth-
Although this study represents a distinct meth- ods associated with online learning are required.
odological improvement over previous compari- Learning is a flexible and dynamic process that can
sons between online and F2F courses, there is room be successfully accomplished across a wide range
for further refinement and additional research. This of settings, methods, and strategies. The quality of
analysis constitutes a quasi-experiment due to the the learning experience is determined by the peda-
fact that students could self-select into which of gogy used, not the medium through which the
the two types of classes they preferred. Had we learning takes place. If designed properly, in a way
been able to randomly assign students to the online that stresses interaction, clear structure, and strong
or F2F sections (see Olson 2002 and Piccoli et al. content, online courses can offer a learning envi-
2001), the possibility of a selection effect would ronment that is as equally effective and enjoyable
have been eliminated and we would have been able as the traditional classroom.
to more directly observe the influence of the class-
room setting. Obviously, there are numerous logis-
tical challenges with such a design, and a true
Appendix
experiment of this fashion would fail to capture the Select Elements of Student Survey: Evaluating the
differences that exist between the two distinct Influence of Individual Student Characteristics and
populations of students. As these dissimilarities Social Structural Factors on Academic Outcomes
appear to be what drives the differences in student in Online Distance Education Courses versus
performance, they should be understood better Traditional Face-to-face Settings (To obtain a copy
rather than minimized. Furthermore, this study’s of the full survey, please contact corresponding
reliance upon self-reported GPA data could be author.)
improved upon through access to student records
that included actual GPA data. Finally, although
this study used data obtained from multiple sec-
Section B: Residential and Employment
tions offered over multiple terms, our arguments Status
would be further enhanced by similar systematic B7. On the average, how many hours per week do
analyses of other courses taught by instructors at you work for pay? _______ Hours per week
different universities.
Online learning is a rapidly growing feature
of higher education, and as it increases in both
prevalence and importance, we as instructors and (continued)

Downloaded from tso.sagepub.com at ASA - American Sociological Association on October 1, 2012

Driscoll et al. 327

Appendix (continued)

Section C: Student Self-Perceptions

Instructions: For the following questions, please indicate how much you agree or disagree with the follow-
ing statements by circling the corresponding number. 5 = Strongly Agree, 4 = Agree, 3 = Neither Agree nor
Disagree, 2 = Disagree, 1 = Strongly Disagree.

Neither
Strongly Agree nor Strongly
Agree (5) Agree (4) Disagree (3) Disagree (2) Disagree (1)
C3. I usually get assignments 5 4 3 2 1
done ahead of time.
C4. Feeling like I am part of a 5 4 3 2 1
class is important to me.
C5. Classroom discussion 5 4 3 2 1
(including online discussion
boards) is helpful to me.
C6. I feel comfortable taking 5 4 3 2 1
part in discussions in an
actual classroom setting.
C7. I prefer figuring out the 5 4 3 2 1
instructions for an assign-
ment on my own without
the instructor explaining
them to me.
C8. I always read all of the 5 4 3 2 1
assigned readings.
C9. I enjoy working with oth- 5 4 3 2 1
er classmates on projects
or in study groups.
C10. I usually participate in 5 4 3 2 1
study groups when they
are available.
C11. Part of doing well in a 5 4 3 2 1
course involves frequent
interaction with the
instructor.
C12. I usually read the online 5 4 3 2 1
readings on the computer
rather than printing them
out.

Section E: Experience with Online Courses

(Please respond if you have ever taken an online distance education course, including this semester.)
E1. Prior to this semester, how many DE online courses have you taken? _______

(continued)

Downloaded from tso.sagepub.com at ASA - American Sociological Association on October 1, 2012

328 Teaching Sociology 40(4)

Appendix (continued)

Section F: Course/Instructor Evaluation

Neither
Strongly Agree or Strongly
Agree (5) Agree (4) Disagree (3) Disagree (2) Disagree (1)
F1. The instructor presented 5 4 3 2 1
course expectations very
clearly.
F2. The instructor for the 5 4 3 2 1
course provided prompt
feedback on assignments.
F3. The instructor was actively 5 4 3 2 1
involved in course discus-
sions.
F4. The instructor made him/ 5 4 3 2 1
herself accessible to stu-
dents.
F5. The interaction I had with 5 4 3 2 1
my instructor was very help-
ful for me in understanding
the course material.
F6. Instructions and due dates 5 4 3 2 1
for assignments and exams
were very clear.

Section G: Demographics
G1. Gender
1. Male
2. Female
G2. How old are you (actual age in years)? ______ years
G5. What is your year in school?
1. Freshman
2. Sophomore
3. Junior
4. Senior
5. Continuing education
G7. How many credit hours are you taking this semester (including courses you are
auditing)?
_________ credit hours
G8. What is your current GPA (If unsure, please estimate)?
1. 1.00-1.49
2. 1.50-1.99
3. 2.00-2.49
4. 2.50-2.99
5. 3.00-3.49
6. 3.50-3.99
7. 4.00 or above

Downloaded from tso.sagepub.com at ASA - American Sociological Association on October 1, 2012

Driscoll et al. 329

Acknowledgments 6. A quick comparison between the grades of the

The authors thank Maxine Atkinson, Richard students surveyed (our sample) and the overall
Slatta, Kathleen S. Lowney, and the three anony- grades for the course (our population) reveals
mous reviewers from Teaching Sociology for their no substantial differences, which supports our
kind and insightful comments on earlier drafts of sample being an unbiased representation of the
this article. student population. Means for the second exam
grades were 81.88 to 81.36 (sample to popula-
tion) for the F2F students and 78.94 to 77.70 for
Notes the online students. Means for the IDA assign-
Reviewers for this manuscript were, in alphabetical ment were 87.53 to 87.35 for the F2F students
order, Carol Jenkins, Timothy Kubal, and Daphne and 84.99 to 84.25 for the online students.
Pedersen.
7. Age was also initially tested as a control vari-
able in all analyses but had to be dropped due to
1. Although this course drew from both anthropol- multicollinearity issues where it was measuring
ogy and sociology (see Methods section), the the same effects accounted for by other vari-
instructor is a sociologist and the course heavily ables. Older students were more likely to be
favored sociological materials and perspectives. seniors in college, to be working more hours
2. Much of the literature on the efficacy of online per week, to have experience with online edu-
education dichotomizes all courses into one of cation, and to be enrolled in the online section
two extremes (online vs. F2F). In actuality, of the course. As further justification for its
many courses fall somewhere in between and removal, age did not demonstrate a significant
can be characterized as “hybrid” courses due to effect on any of the dependent variables in ini-
their use of some manner of online learning tial bivariate analysis.
strategy (e.g., e-mail correspondence, online
distribution of materials, etc.). “True hybrid”
courses incorporate online learning techniques
References
Allen, I. Elaine and Jeff Seaman. 2010. Class Differ-
into the structure of an F2F course to a substan-
ences: Online Education in the United States, 2010.
tial degree and can be viewed as a third, The Sloan Consortium. Retrieved October 25, 2011
middle-ground type of course. (http://sloanconsortium.org/publications/survey/
3. The course appears in the catalogue as both an class_differences).
anthropology and a sociology course. Although Allen, Mike, John Bourhis, Nancy Burrell, and Edward
Mabry. 2002. “Comparing Student Satisfaction with
both classes meet together and the students are
Distance Education to Traditional Classrooms in
treated as a homogenous whole, technically half Higher Education: A Meta-analysis.” The American
the students are enrolled in an anthropology Journal of Distance Education 16(2):83-97.
course and half in a sociology course. Allen, Mike, Edward Mabry, Michelle Mattrey, John
4. Learning management systems (LMSs) are Bourhis, Scott Titsworth, and Nancy Burrell. 2004.
“Evaluating the Effectiveness of Distance Learning:
Web-based software applications that are used
A Comparison Using Meta-analysis.” International
to coordinate course materials and activities. Communication Association 54(3):402-20.
Most LMSs possess features that allow for Atkinson, Maxine P. and Andrea N. Hunt. 2008. “Inquiry-
scheduling, posting readings and assignments, guided Learning in Sociology.” Teaching Sociology
conducting discussion boards and live chats, 36(1):1-7.
Bennett, Gregg and Frederick P. Green. 2001. “Student
running an online grade book, conducting
Learning in the Online Environment: No Significant
online quizzes and assignment, and much more. Difference?” Quest 53(1):1-13.
Common LMSs include WebCT, BlackBoard, Bernard, Robert M., Philip C. Abrami, Yiping Lou,
Desire2Learn, Angel, and Moodle. Evgueni Borokhovski, Anne Wade, Lori Wozney,
5. In the online section taught during the summer Peter Andrew Wallet, Manon Fiset, and Binru Huang.
2004. “How Does Distance Education Compare to
term, some material was dropped. Accordingly, 5
Classroom Instruction? A Meta-analysis of Empiri-
of 25 multiple choice questions and one essay cal Literature.” Review of Educational Research
question were changed on the second unit exam. 74(3):379-439.

Downloaded from tso.sagepub.com at ASA - American Sociological Association on October 1, 2012

330 Teaching Sociology 40(4)

Bok, Derek. 2003. Universities in the Marketplace: The opment, U.S. Department of Education. Retrieved
Commercialization of Higher Education. Princeton: January 5, 2012 (http://www2.ed.gov/rschstat/eval/
Princeton University Press. tech/evidence-based-practices/finalreport.pdf).
Brooks, J. Michael. 1997. “Beyond Teaching and Learn- Moore, Janet C. 2005. “The Sloan Consortium Quality
ing Paradigms: Trekking into the Virtual University.” Framework and the Five Pillars.” The Sloan Con-
Teaching Sociology 25(1):1-14. sortium. Retrieved April 4, 2011 (http://sloanconsor
Clark-Ibáñez, Marisol and Linda Scott. 2008. “Learning tium.org/publications/freedownloads).
to Teach Online.” Teaching Sociology 36(1):34-41. Olson, David. 2002. “A Comparison of Online and Lec-
Gallagher, Peggy A. and Katherine McCormick. 1999. ture Methods for Delivering the CS 1 Course.” Jour-
“Student Satisfaction with Two-way Interactive nal of Computing Sciences in Colleges 18(2):57-63.
Distance Learning for Delivery of Early Childhood Parkhurst, Rosamond, Barbara M. Moskal, Gary Lee
Special Education Coursework.” Journal of Special Downey, Juan Lucena, Thomas Bigley, and Sharon
Educational Technology 14(1):32-47. Elberb. 2008. “Engineering Cultures: Comparing
Hair, Joseph F., Jr., Rolph E. Anderson, Ronald L. Student Learning in Online and Classroom Based
Tatham, and William C. Black. 1998. Multivariate Implementations.” International Journal of Engi-
Data Analysis with Readings. 5th ed. Englewood neering Education 24(5):955-64.
Cliffs, NJ: Prentice-Hall. Pearson, A. Fiona. 2010. “Real Problems, Virtual Solu-
Hilal, Susan M. and Meredith Redlin. 2004. Integrating tions: Engaging Students Online.” Teaching Sociol-
Data Analysis: Instructional Materials for Integrat- ogy 38(3):207-14.
ing Data Analysis into Sociology Courses. Resource Piccoli, Gabriele, Rami Ahmad, and Blake Ives. 2001.
Materials for Teaching. Washington, DC: American “Web-based Virtual Learning Environments: A
Sociological Association. Research Framework and a Preliminary Assessment
Jaffee, David. 1997. “Asynchronous Learning: Technol- of Effectiveness in Basic IT Skills Training.” MIS
ogy and Pedagogical Strategy in a Distance Learning Quarterly 25(4):401-26.
Course.” Teaching Sociology 25(4):262-77. Ritzer, George. 2004. The McDonaldization of Society.
Jahng, Namsook, Don Krug, and Zuochen Zhang. 2007. 4th ed. Thousand Oaks, CA: Pine Forge Press.
“Student Achievement in Online Distance Education Rovai, Alfred P. and Kirk T. Barnum. 2003. “On-line
Compared to Face-to-face Education.” European Course Effectiveness: An Analysis of Student Inter-
Journal of Open, Distance, and E-Learning 2007(1). actions and Perceptions of Learning.” Journal of Dis-
Retrieved June 9, 2009 (http://www.eurodl.org/ tance Education 18(1):57-73.
materials/contrib/2007/Jahng_Krug_Zhang.htm). Russell, Thomas L. 1999. The No Significant Difference
Ko, Susan and Steve Rossen. 2008. Teaching Online: A Phenomenon. Chapel Hill, NC: Office of Instructional
Practical Guide. 2nd ed. New York: Routledge. Telecommunications, University of North Carolina.
Little, Craig B., Larissa Titarenko, and Mira Bergelson. Sitzmann, Traci, Kurt Kraiger, David Steward, and
2005. “Creating a Successful International Distance- Robert Wisher. 2006. “The Comparative Effectiveness
learning Classroom.” Teaching Sociology 33(4):355-70. of Web-based and Classroom Instruction: A Meta-
Logan, Elisabeth, Rebecca Augustyniak, and Alison Rees. analysis.” Personnel Psychology 59(3):623-64.
2002. “Distance Education as Different Education: A Summers, Jessica J., Alexander Waigandt, and Tiffany A.
Student-centered Investigation of Distance Learning Whittaker. 2005. “A Comparison of Student Achieve-
Experience.” Journal of Education for Library and ment and Satisfaction in an Online Versus a Traditional
Information Science 43(1):32-42. Face-to-face Statistics Class.” Innovative Higher Edu-
Lombardi, Marilyn M. 2008. Making the Grade: The Role cation 29(3):233-50.
of Assessment in Authentic Learning. EDUCAUSE Sumner, Jennifer. 2000. “Serving the System: A Criti-
Learning Initiative. Retrieved March 27, 2012 (http:// cal History of Distance Education.” Open Learning
net.educause.edu/ir/library/pdf/ELI3019.pdf). 15(3):267-85.
McFarland, Daniel and Diane Hamilton. 2005. “Factors Suskie, Linda. 2004. Assessing Student Learning: A
Affecting Student Performance and Satisfaction: Common Sense Guide. Bolton, MA: Anker.
Online versus Traditional Course Delivery.” Journal Tucker, Sheila. 2001. “Distance Education: Better,
of Computer Information Systems 46(2):25-32. Worse, or As Good As Traditional Education?”
McIsaac, Marina Stock, J. Michael Blocher, Veena Online Journal of Distance Learning Administra-
Mahes, and Charalambos Vrasidas. 1999. “Student tion 4(4). Retrieved June 6, 2009 (http://www.westga
and Teacher Perceptions of Interaction in Online .edu/~distance/ojdla/winter44/tucker44.html).
Computer-mediated Communication.” Educational Urtel, Mark G. 2008. “Assessing Academic Perfor-
Media International 36(2):121-31. mance Between Traditional and Distance Education
Means, Barbara, Yukie Toyama, Robert Murphy, Marianne Course Formats.” Educational Technology & Society
Bakia, and Karla Jones. 2009. Evaluation of Evidence- 11(1):322-30.
Based Practices in Online Learning: A Meta-analysis York, Reginald O. 2008. “Comparing Three Modes
and Review of Online Learning Studies. Washington, Instruction in a Graduate Social Work Program.”
DC: Office of Planning, Evaluation, and Policy Devel- Journal of Social Work Education 44(2):157-71.

Downloaded from tso.sagepub.com at ASA - American Sociological Association on October 1, 2012

Driscoll et al. 331

Andrea N. Hunt is a doctoral candidate in sociology at

Bios North Carolina State University. Her research in family
Adam Driscoll is a doctoral student in sociology at North sociology examines the employment and mental health
Carolina State University. His research in environmental outcomes over time of families of children with disabili-
sociology explores the various ways through which the ties. Her other research is in the Scholarship of Teaching
structure of human social relations and the natural world and Learning, where she is interested in faculty develop-
mutually shape one another, particularly within the realm ment, program evaluation, assessment of student learning
of agriculture. He also conducts research within the outcomes, implementing problem-based learning, and
Scholarship of Teaching and Learning, exploring the effi- teaching with technology.
cacy of online education and gender dynamics within the
online environment. Lisa Tichavsky is a doctoral student and lecturer in the
Department of Sociology and Anthropology at North
Carolina State University where she teaches both tradi-
Karl Jicha is a recent doctoral graduate and instructor of tional and online classes. Her research interests include
sociology at North Carolina State University. His spe- pedagogy, criminology, and family, with a special focus
cialty areas are rural and community sociology, on gender and crime, family violence, and formal social
technology and social and cultural change, and medical control policies and practices.
sociology. His dissertation research examined the influ-
ence of multiple dimensions of social capital and Gretchen Thompson is a recent doctoral graduate from
confidence in government institutions on participation in North Carolina State University. Her substantive areas
collective action in developing countries. Current include rural, community, global development and
research interests also include the Scholarship of Teach- hunger. She also conducts research on the Scholarship of
ing and Learning with a focus on social interaction in Teaching and Learning, publishing and presenting on
college classrooms and the effectiveness of distance topics that include prison education and the effectiveness
education. of distance learning.

Downloaded from tso.sagepub.com at ASA - American Sociological Association on October 1, 2012

View publication stats