Home Search Collections Journals About Contact us My IOPscience

Polarization of physics on global courses

This content has been downloaded from IOPscience. Please scroll down to see the full text.

2015 Phys. Educ. 50 210

(http://iopscience.iop.org/0031-9120/50/2/210)

View the table of contents for this issue, or go to the journal homepage for more

Download details:

IP Address: 203.29.104.15
This content was downloaded on 07/07/2016 at 07:40

Please note that terms and conditions apply.

 Papers
iopscience.org/ped

Polarization of physics on global

courses
AllanLAlinea1 and WadeNaylor2
1
Department of Physics, Osaka University, Toyonaka, Osaka 560 0043, Japan
2
International College and Department of Physics, Osaka University, Toyonaka,
Osaka 560 0043, Japan

E-mail: alinea@het.phys.sci.osaka-u.ac.jp and naylor@phys.sci.osaka-u.ac.jp

Abstract
Since October 2010, the ChemistryBiology Combined Major Program,
an international course taught in English at Osaka University, has been
teaching small classes (no more than 20 in size). We present data from
the Force Concept Inventory (FCI) given to first-year classical mechanics
students (N=47 students over three years) pre and post score, for a class that
predominantly uses interactive engagement, such as MasteringPhysics. Our
findings show a G-factor improved score of about0.18, which is marginally
about the average of a traditional-based course. Furthermore, we analyze
in detail a set of six questions from the FCI, involving the identification of
forces acting on a body. We find that student answers tend to cluster about
polarizing choicesa pair of choices containing the correct choice and
a wrong choice, with the latter corresponding to a superset of forces in the
former. Our results are suggestive that students have a good idea of the right
set of forces acting on a given system, but the inclusion of extra force(s)
brings about confusion; something that may be explained by misleading
ontological categorization of forces. In an appendix A we also comment on
possible correlations between the pre/post score and the level of English
ability on entry to the course.

1.Introduction the student will have difficulty in gaining a deep

In a typical calculus-based introductory mechan- understanding of Newtonian mechanics. For this
ics course, a significant time is allotted to applying reason, the need to look into this very simple matter
Newtons laws of motion. It follows that the skill of identifying forces cannot be overemphasized.
in making free body diagrams (FBD) (whether In the past couple of decades, much of the
on paper or just inside ones head) is repetitively research involving students understanding or the
invoked throughout much of the course in solving misconception of force and other related con-
mathematical and conceptual problems, and in try- cepts in Newtonian mechanics focused on the
ing to have a deeper understanding of the dynam- cure in the form of interactive engagement (see
ics of several systems. Making a FBD is thus one for instance, [15]). Although this is an equally
of the foundational skills a student is expected to important area of research, the subject itself of
acquire. Part of this foundational skill is the abil- student misconception regarding force is far
ity to identify the forces acting on a given body. If from being a closed case. As we shall see, there
this very basic ability is not properly established, are still avenues (e.g. the existence of polarizing

210 Physics Education 50 (2) 2015 IOP Publishing Ltd 0031-9120/15/020210+8$33.00

 Polarization of physics on global courses
choices (section 3.2)) that need to be explored. from MasteringPhysics. While answering the
New insights can be gained from this exploration problems, the instructor and teaching assistant
through the administration of a concept inven- attend to student concerns about the problems
tory. In this work, we revisit the subject matter they are solving. They are given one week to fin-
of understanding force with a special focus on ish the assigned set of problems, where students
the identification of forces through the use of the on average spend about one to two hours doing
Force Concept Inventory (FCI). their homework.
The FCI developed by Hestenes, Wells and We administered the FCI at the beginning and
Swackhamer in the early 1990s is an instrument at the end of the course on calculus-based intro-
to assess students understanding or misconcep- ductory mechanics. The data corresponding to a
tion of force [6, 7]. It consists of 30 multiple- total of 47 students were taken over three years
choice questions each with five choices. After from October 2011 until February 2014. The
more than two decades of existence, there must threeyear data were subjected to statistical analy-
be good reasons why it is still one of the most sis mainly involving t-test, graphical analysis and
widely used concept inventories. Contained in calculation of the standard statistical parameters
the FCI are questions that suit our need to look (e.g. mean, standard deviation, etc).
into the subject of identifying force. This allows
us to set aside the need to develop a new set of
3. Results and discussion
questions and focus on the analysis of the result
of the FCI. The subjects of our testing are the stu- 3.1. Result of the FCI as a whole
dents of the ChemistryBiology Combined Major
Table 13 shows the result of the FCI for three
Program (CBCMP).
batches of entrants to the CBCMP program. We
would like to stress that given the number of stu-
2.Methodology dents, we have we found an overall difference in
the means of the pre- and post-test scores in favour
The CBCMP is a program offered by Osaka of the latter. To further verify this we used R (sta-
University (Japan) that is geared for students tistical software) [9] to find the paired student
coming from outside Japan. The student compo- t-test result. At the 95% confidence level we found
sition is multiracial in nature, with approximately a p-value of 4.1105 for 46 degrees of freedom
50% coming from China (including on and off with a mean difference of 8.6%. The null hypoth-
the mainland), 25% from Southeast Asia and 25% esis was not satisfied and suggests the results do
from other countries including Japan. Classes are lead to a positive increase in the post scores.
conducted in English, with the instructor meeting The t-test only indicates whether there is a
the class once a week for a period of 1.5 h and significant difference between the pre- and post-
a total of 20 h in class (this is to be contrasted test scores. A better measure of student perfor-
with a standard physics course, which typically mance before and after instruction is provided
has 30/40h). by the normalized gain, G [2]. It is the difference
Having a limited number of class hours per between the pre- and post-test scores divided by
week, traditional lectures can be both ineffective the maximum possible score in the post-test rela-
and impractical, so to deal with this shortage of tive to the score in the pre-test. Symbolically,
hours, we have employed MasteringPhysics [8].
In a typical class the students are given prelec- %Sf %Si
 G= , (1)
ture assignments (10 questions each) to read in 100 %Si
advance before going to class, and at every meet
where % Sf and % Si are the final and initial scores,
the class starts with a set of prelecture conceptual
respectively. Table 1 shows the normalized gain
questions, some of which are discussed by the
for the three batches of students. On average, we
instructor. A short lecture then follows highlight-
find a result of G = 0.18. At first we found this
ing the main points of the scheduled subject of
result to be a little surprising, as we expected a
the meeting. For the remaining 60% of the class
hours, students are paired to discuss and answer 3
For a recent article on which kind of concept test to imple-
the assigned problems (conceptual or quantitative) ment and how to interpret the data, see [10].

March 2015 P h y s i c s E d u c at i o n 211

A L Alinea and W Naylor
Table 1. The results of the FCI for the three batches of entrants from 20112013.
Group 2011 (n=11) 2012 (n=18) 2013 (n=18) Total (N=47)
Mean (SD) Mean (SD) Mean (SD) Mean (SD)
Pre-test 17.1 (6.6) 16.6 (7.4) 14.7 (6.7) 16.0 (6.9)
Post-test 20.2 (7.0) 19.7 (6.3) 16.4 (7.0) 18.6 (6.8)
Gaina 0.24 0.23 0.12 0.18
a
For the normalized gain standard deviation (SD) is not applicable.

higher gain in line with interactive engagement

(IE) based courses [2, 11, 12]. In appendix A, we
look into the possible correlation of the normal-
ized gain with English language ability.

3.2. Identifying forces

The relatively low normalized gain may be
attributed to a variety of factors. The complex-
ity of the FCI, however, does not give us much
freedom to investigate a lot of things. The FCI
centering around the idea of force connects to so
many other structures of human understanding
regarding force that we are bound to focus only Figure 1. Normalized gain (G) (expressed in percent)
on a small manageable part. Beyond that, we can for selected questions involving the identification of
forces acting on a given body.
say a lot of things from logic alone, but scarcity
of experimental results on our part would simply number of students who answered each letter for
hinder us in drawing a sound conclusion. a given question is counted and the result is plot-
What we do in this work, as already men- ted in the figure. Careful analysis of the results
tioned, is focus on the identification of forces. For indicate that:
this subject, we choose only segments of the FCI
(a) there are two dominant answers for both the
(labeled here as questions IVI)4 that fall under
pre-test and the post-test,
the mentioned category. The hope is to see an
(b) the pair of dominant answers is the same for
obvious pattern on how students identify force.
both the pre-test and the post-test for each
Let us now look into the pre- and post-test
question,
scores. Figure 1 shows the percent of students
(c) the pair of dominant answers contain the
who got the correct answer for the six questions
correct answer.
labelled IVI involving identification of forces
acting on a given body. The percent of students By a pair of dominant answers, we mean a pair
who got the correct answer in the post-test is of choices with the highest frequency. Consider for
higher than that of the pre-test for all questions. instance, question I. The pair of dominant answers
However, similar to that of the overall FCI result are choices B and D. For the pre-test, the share of
presented in the previous subsection, the improve- the pair of dominant answers is 68% and it rises to
ment is small. The normalized gain expressed in 71% in the post-test. Such a large share of the pair
percent ranges only from 13% to 30%, which is of dominant answers justifies our use of the term
classified as low G in the work of [2] (medium G: dominant. Figure3 shows the wide gap between
30%<G<70%, high G:>70%). the percent of students who answered the pair of
The seemingly uninteresting result above dominant answers and those who answered the
prompted us to look closely at how the students remaining choices for both the pre-test and post-
answered the six questions. Figure 2 shows the test. The percent of students who answered the
result of the item analysis for each question. The dominant answers ranges from 64% to 89% for
the pre-test and from 81% to 98% for the post-test.
4
These correspond to questions 5, 11, 13, 18, 29 and 30.

212 P h y s i c s E d u c at i o n March 2015

 Polarization of physics on global courses

Figure 2. Distribution of student answers for questions (I)(VI). The five choices are labeled AE (NA means no
answer). The letter of the correct answer is labeled with a check mark. The red circles indicate the top two choices
polarizing student answers.

The curious pattern that we found above The existence of polarizing choices brings the
regarding the existence of a pair of dominant idea of misleading ontological categorization [13].
answers raises the following concerns. When facing a problem requiring the identification
of forces acting on a given body, several forces may
(a) Why is there a pair of dominant answers? come to mind. When there are only a few forces
(b) What is the nature of this pair of dominant acting on a given body, it is possible to have a good
answers? idea that includes all the valid forces plus the extra
(c) Why do students choose the pair of domi- forces that are wrong. Whereas the matter of pick-
nant answers? ing up the valid forces and putting it in the category
of valid forces may be easily done after a careful
Observation of the plots in figure2 in connection to
analysis, the matter of categorizing the extra forces
the six FCI questions tells us that the pair of domi-
as correctly acting or not can be quite problematic.
nant answers always contain the correct choice;
The idea of polarizing choices can be made
symbolically, it is (CC, X) where CC is the correct
clearer through an example. Consider a snapshot
choice while X is the other choice. Furthermore,
of a swinging pendulum shown in figure4. A stu-
both CC and X contain the right set of forces acting
dent may be asked to identify the forces acting
on a given body but X contains one or more extra
on the pendulum bob with the following choices:
forces that do/does not legitimately act on a given
body. In other words, X is a superset of CC. The set (a) downward force of gravity,
of extra fictitious force(s) in X confuse(s) the stu- (b) the tensional force exerted by the string from
dent causing a sort of polarization; that is, in anal- A to O,
ogy to charge polarization, the majority of student (c) the force in the direction of the pendulums
answers are divided into two, namely, choices CC motion.
and X. For this reason, the more appropriate term
for CC and X seems to be polarizing choices and 1. (a) only,
we are going to use this term from hereon in. 2. (b) only,

March 2015 P h y s i c s E d u c at i o n 213

A L Alinea and W Naylor
3. (c) only,
4. (a) and (b) only,
5. (a), (b) and (c).
The correct answer is letter (4) corresponding
to forces in (a) and (b). It is worth noting that
choice (5) offers the same set of forces as that of
choice (4) but with an additional fictitious force
given by (c). Choices (4) and (5) are our polar-
izing choices for this question. An average stu-
dent may be able to easily discard choices (1)(3)
but may find choices (4) and (5) rather confusing.
As we have seen in the results above, the inclu-
sion of extra fictitious force for questions of this
type can cause a serious divide in the answers of
students.
One curious thing that caught our attention
regarding this matter of polarization is that most
students did well in their homework (average
88%). The immediate question is How does the
system of MasteringPhysics guide the students
to the correct answer? MasteringPhysics [8] has
a corrective measure by way of hints and trial-
and-error. When it comes to answering problems Figure 3. Distribution of student answers to questions
IVI.
requiring the identification of forces, the exist-
ence of hints together with the limited number of
a polarization, where student answers tend to
tries the students are allowed to make, narrows
cluster about two competing choices with the
down the number of possible pathways to the cor-
wrong choice being a superset of the right choice.
rect answer in their homework. Although we may
Our results are suggestive that students have a
need more data, our preliminary study here sug-
good idea of the right set of forces acting on a
gests that the corrective measure of the system
given system but the inclusion of extra force(s)
of MasteringPhysics may not be able to correct
brings about confusion; something that may be
deeply ingrained student misconceptions/precon-
explained by misleading ontological categoriza-
ceptions about force. However, we are not ignor-
tion of forces. Progressive hints and the limited
ing the possibility that we might also be able to
number of tries offered by interactive web-based
use such a system to combat the misconception/
physics education software in answering prob-
preconception of extra force(s).
lems involving proper identification of forces
could help students get to the correct answer.
4. Summary and concluding remarks This, however, may not be enough to significantly
In this article we have discussed several points alter misleading preconceptions or misconcep-
involving (a) the notion of polarization as it arises tions of students.
in answering questions about the identification of In appendix A, we raised the issue of English
forces acting on a body, (b) the FCI as an instru- ability and performance on the FCI (more generally
ment to assess student understanding of forces, (c) any science-based test) and appeared to find no cor-
the interactive web-based homework and tutorial relation, most likely arising from the requirement of
software MasteringPhysics, and (d) the English a Test of English as a Foreign Language (TOEFL)
language ability as it relates to student learning score above80. In Japan, at least, there appears to
(see appendix A). be no minimum reading or writing score required and
We looked at six particular questions on the bottom panel of figureA1 clearly appears to show
the FCI to establish if there were any pat- a correlation (as might the total TOEFL score if any
terns in answering questions. We came across range of students were accepted: TOEFL 0120).

214 P h y s i c s E d u c at i o n March 2015

 Polarization of physics on global courses
Acknowledgments
We would like to thank current and past students
on the ChemistryBiology Combined Major Pro-
gram (CBCMP) for creating stimulating interac-
tive engagement classes.

Appendix A. English language versus

reading ability
Given that the course is predominantly for non-
native speakers of English (the CBCMP entrance
requirement is a TOEFL score above80), we con-
sidered the possibility that English ability leads to
a low G. We checked to find a very low correlation
between pre/post-test performance on the FCI and
English ability (r2=0.0048/0.00041, respectively),
see figureA1, including students classed as native.
It may be worth mentioning that the distinc-
Figure 4.A typical question used to test conceptual tion between native and non-native here is blurred
ability in identifying forces. Consider a pendulum
attached at O swinging from a previous higher point because, for administrative reasons, a TOEFL/
to A. The students have to distinguish what are real International English Language Testing System
and fictitious forces from: (a) the downward force (IELTS) type score was required for some students
of gravity, (b) the tensional force exerted by the string who might be considered native (for example even
from A to O, and (c) the force in the direction of the though their language of instruction at high school
pendulums motion.
was English). Actually it is well known that natives
This certainly warrants the implementation of a mini- do not score full marks on TOEFL/IELTS tests
mum reading and writing score on TOEFL and simi- [14]. However, rather than use some kind of com-
lar tests, for entrance to global courses in Japan. plicated probabilistic Monte Carlo type simulation
What we have found out in this study opens [15] to randomly model native scores for those
up new avenues for future investigation. Firstly, with TOEFL scores greater than 115 out 120, for
with regard to the idea of polarizing choices, we simplicity, we just give full marks to those classed
have so far only scratched the surface of answer- as native administratively. This also applies to
ing intriguing questions such as (a) how students reading ability, where in some cases a student is
ontologically categorize forces acting on a given exceptionally better (or worse) than the average
body, (b) why polarizing choices exist (it may taken from their total TOEFL/IELTS score.
be constructed by design, such as via the use of Perhaps not surprisingly we did find a cor-
distractors [10], but only experiment can tell relation with reading ability on pre/post FCI with
us whether there really is a resulting polariza- r2 = 0.076/0.097, respectively. This is an order
tion) and how such an idea relates to our way of of magnitude larger than the total TOEFL score
thinking, (c) how we can cure misconceptions/ correlation. The scatter plot in figure A1 (lower
preconceptions related to polarizing choices and panel) clearly shows some kind of correlation for
polarization, and (d) whether polarizing choices non-natives in reading ability, with two distinct
and the resulting polarization can also be induced/ groups. We stress here that this pattern is not the
found for other physics concepts aside from force. same problem that occurs for native speakers in
Secondly, the issue of English language ability conceptual tests, who have confusion between
FCI for international courses is far from being every day and physics usage of words. This can
settled. Although intuitively, there should be a clearly be seen from the fact that we have two
positive correlation between the two, questions separate groups in the scatter plots in figureA1.
on how far English ability is a factor in learning There are then two possible reasons for
physics concepts for a multiracial class are some- the fact that the overall score had no correla-
thing worthy of further investigation. tion. Either, this is because we are taking a

March 2015 P h y s i c s E d u c at i o n 215

A L Alinea and W Naylor
References
[1] HellerP, KeithR and AndersonS 1991 Teaching
problem solving through cooperative
grouping. Part 1: group versus individual
problem solving Am. J. Phys. 60 62736
[2] HakeR R 1998 Interactive-engagement versus
traditional methods: a six-thousand-student
survey of mechanics test data for introductory
physics courses Am. J. Phys. 66 6474
[3] FagenA P, CrouchC H and MazurE 2002 Peer
instruction: results from a range of classrooms
Phys. Teacher 40 2069
[4] MoroteE S and PritchardD E 2009 What
course elements with improvement on tests
in introductory Newtonian mechanics Am. J.
Phys. 77 746
[5] BrhaneH and AbebeG 2014 Comparison of
teaching methods in terms of conceptual
understanding of Newtonian mechanics:
the case of freshman students at Jigjiga
University, Ethiopia Middle Eastern African J.
Educ. Res. 10 4456
[6] HestenesD, WellsM and SwackhamerG 1992
Force concept inventory Phys. Teacher
30 14158
[7] SavinainenA and ScottP 2002 The Force
concept inventory: a tool for monitoring
student learning Phys. Educ. 37 4552
[8] MasteringPhysics and Pearson Publishing
Group www.pearsonmylabandmastering.com/
masteringphysics/
[9] R Foundation www.r-project.org/
[10] MadsenA, McKaganS B and SayreE C 2014
Best practices for administering concept
inventories arXiv:1404.6500
[11] SokoloffD R and ThorntonR K 1997 Using
Figure A1. Scatter plot of English (top) and reading interactive lecture demonstrations to create an
ability (bottom) versus post-test FCI. We clearly see two active learning environment Phys. Teacher
groups: natives and non natives, and although we found 35 3407
no correlation between the overall TOEFL and the [12] ColettaV P and PhillipsJ A 2005 Interpreting
post FCI score, there appears to be one for non-natives FCI scores: normalized gain, preinstruction
(similar correlations were found for pre-test FCI). scores and scientific reasoning ability Am. J.
Phys. 73 117282
small high-end sample of TOEFL in the range [13] JohnstonA T and SoutherlandS A 2000 A
80120, while the entrance requirement did not reconsideration of science misconceptions
require a minimum reading score on TOEFL; or using ontological categories Annual Meeting
that reading ability is the more important factor of the National Association for Research in
Science Teaching (New Orleans, LA, April
in conceptual tests on global courses. In Japan 2000) http://physics.weber.edu/johnston/
most institutions only ask for an overall score on research/ontological_categories_
English tests, while the evidence here suggests reconsidered.pdf
that lower bounds on reading and writing abil- [14] StrickerL J 2002 The Performance of Native
ity are very important for students to succeed on Speakers of English and ESL Speakers on the
Computer-based TOEFL and GRE General
university courses in general (we hope to discuss Tests (Princeton, NJ: Educational Testing
further evidence for this in future work). Service) www.ets.org/Media/Research/pdf/
RR-02-16.pdf
Received 22 August 2014 [15] RobertC and CasellaG 2005 Monte Carlo
Accepted for publication 3 December 2014 Statistical Methods 2nd edn (New York:
doi:10.1088/0031-9120/50/2/210 Springer)

216 P h y s i c s E d u c at i o n March 2015

 Polarization of physics on global courses
Allan L Alinea is a doctoral student in Wade Naylor is a specially appointed
particle physics at Osaka University in associate professor at Osaka University
Japan. He is currently working on the in Japan. He is a faculty member of
interdisciplinary fields of cosmology and the International College and has been
particle physics, but finds enough time teaching physics to students on the
to do research in the field of physics ChemistryBiology Combined Major
education. Prior to taking doctoral Program since its inception in 2010. He
studies in Japan, he taught fundamental college physics for is also a member of the high-energy particle physics group
about six years. at Osaka University. After his PhD in theoretical physics
(from Newcastle University, UK) he ventured to Japan in
late 2001, first on a couple of postdoctoral fellowships, until
becoming a lecturer in physics at Ritsumeikan University in
2005. His research interests, besides physics education, relate
to quantum effects and particle creation in dielectrics (optics)
and cosmology.

March 2015 P h y s i c s E d u c at i o n 217