FIsme Scientific Library

Ariyadi Wijaya
Invitation
Ariyadi Wijaya
You are kindly invited to attend

Toward better practice and achievement

Context-based mathematics tasks in Indonesia:
Context-based mathematics the public defense of my thesis:

The Indonesian national curriculum mandates that mathematics education tasks in Indonesia: Context-based mathematics
tasks in Indonesia:
must be relevant to the needs of life and should offer students opportunities
Toward better practice and
Toward better practice and
to develop the ability to apply their knowledge. Nevertheless, the low results achievement
of Indonesian students on the PISA studies (Programme for International
Student Assessment) indicate the students’ poor performance in applying
mathematics to solve problems situated in real-world contexts. This situation
achievement on January 21, 2015
at 10:30 hrs
induced the start of the CoMTI project (‘Context-based Mathematics Tasks
Indonesia’). The aim of the project was to investigate the reason of this In the Academiegebouw
poor performance and to explore ways for improvement. The results of the of Utrecht University,
CoMTI project showed that Indonesian students mainly have difficulties Domplein 29, Utrecht
in comprehending context-based tasks and in transforming these into a
mathematical problem. With respect to possible reasons for these difficulties,
After the defense, you are
a relation was found between students’ difficulties and the opportunity-to-
welcome at the reception
learn to solve context-based tasks that they got from textbooks and teachers’
teaching practice.
Paranymphs:
Domesia Novi Handayani
domesia.novi@gmail.com
Richo Andi Wibowo
r.a.wibowo@uu.nl

Ariyadi Wijaya
A.Wijaya@uu.nl

Faculteit Bètawetenschappen FIsme

88

2014119_Ariyadi_Wijaya_001.indd 2-3 12/7/2014 4:46:19 PM

 Context-based mathematics tasks in Indonesia:
Toward better practice and achievement

Thesis Ari_141201_FINAL.pdf 1 1-12-2014 16:26:11

 This research was funded by a grant from the Better Education through Reformed
Management and Universal Teacher Upgrading (BERMUTU) project of the
Indonesian Ministry of Education and Culture (project number: IDA CREDIT
NO.4349-IND, LOAN NO.7476-IND).

This research was carried out in the context of the Dutch Interuniversity Centre
for Educational Research.

Wijaya, Ariyadi

Context-based mathematics tasks in Indonesia: Toward better practice and

achievement / A. Wijaya – Utrecht: Freudenthal Institute for Science and
Mathematics Education, Faculty of Science, Utrecht University / FIsme Scientific
Library (formerly published as CD-Ƣ Scientific Library), no. 88, 2015.

Dissertation Utrecht University. With references. Met een samenvatting in het

Nederlands.

ISBN: 978-90-70786-26-7
Key words: mathematics education / context-based tasks / students’ difficulties /
opportunity-to-learn / Indonesia

Cover design: Vormgeving Faculteit Bètawetenschappen

Printed by: Drukkerij Wilco BV

© 2015 Ariyadi Wijaya, Utrecht, the Netherlands.

Thesis Ari_141201_FINAL.pdf 2 1-12-2014 16:26:21

 CONTEXT-BASED MATHEMATICS TASKS IN INDONESIA:
TOWARD BETTER PRACTICE AND ACHIEVEMENT

CONTEXTOPGAVEN IN WISKUNDE:
NAAR EEN BETERE ONDERWIJSPRAKTIJK
EN BETERE PRESTATIES
IN INDONESIË

(met een samenvatting in het Nederlands)

Proefschrift

ter verkrijging van de graad van doctor aan de Universiteit Utrecht op gezag
van de rector magnificus, prof. dr. G. J. van der Zwaan, ingevolge het
besluit van het college voor promoties in het openbaar te verdedigen op
woensdag 21 januari 2015 des ochtends te 10.30 uur

door
Ariyadi Wijaya

geboren op 16 juli 1982

te Banjarnegara, Indonesië

Thesis Ari_141201_FINAL.pdf 3 1-12-2014 16:26:21

 Promotor: Prof. dr. M. H. A. M. van den Heuvel-Panhuizen
Copromotor: Dr. L. M. Doorman

Thesis Ari_141201_FINAL.pdf 4 1-12-2014 16:26:21

 Table of contents

1 Introduction 9

2 Difficulties in solving context-based PISA 21

mathematics tasks: An analysis of students’ errors
Published in The Mathematics Enthusiast, 2014, 11(3), 555-584.
A. Wijaya, M. van den Heuvel-Panhuizen, M. Doorman, and A. Robitzsch

3 Opportunity-to-learn context-based tasks provided 53

by mathematics textbooks
Will be published in Educational Studies in Mathematics
A. Wijaya, M. van den Heuvel-Panhuizen, and M. Doorman

4 Teachers’ teaching practices and beliefs regarding 89

context-based tasks and their relation with students’
difficulties in solving these tasks
Submitted
A. Wijaya, M. van den Heuvel-Panhuizen, and M. Doorman

5 Opportunity-to-learn to solve context-based 125

mathematics tasks and students’ performance in
solving these tasks – lessons from Indonesia
Submitted
A. Wijaya, M. van den Heuvel-Panhuizen, M. Doorman, and M. Veldhuis

6 Conclusion 161

Summary 179
Samenvatting 185
Ringkasan 191
Acknowledgements 197
Curriculum Vitae 199
List of presentations related to this thesis 201
FIsme Scientific Library 203
ICO Dissertation Series 207

Thesis Ari_141201_FINAL.pdf 5 1-12-2014 16:26:21

Thesis Ari_141201_FINAL.pdf 6 1-12-2014 16:26:21
 For Elmi and Arka

Thesis Ari_141201_FINAL.pdf 7 1-12-2014 16:26:21

Thesis Ari_141201_FINAL.pdf 8 1-12-2014 16:26:21
 Chapter 1

Introduction

Thesis Ari_141201_FINAL.pdf 9 1-12-2014 16:26:21

 Chapter 1

Introduction

1 Background of the study

1.1 Ability to apply mathematics: An essential goal of mathematics

education
A number of studies have reported that modern society requires more than
content knowledge (see e.g., Ananiadou & Claro, 2009; Organisation for
Economic Co-operation and Development [OECD], 2013; Partnership for 21st
Century Skills [P21], 2002). To cope with the demands of modern life people need
to be able to apply their knowledge. This situation has led to an emphasis on
connecting education to students’ lives (Graumann, 2011; P21, 2002), which
implies that education should have as its goal to close the gap between how
students learn in school and how they deal with everyday life. Educational
practices, therefore, should provide students with not only knowledge, but also
skills they need for life and community (Griffin, Care, & McGraw, 2012;
Tomlinson, 2004).

In mathematics education a great deal of attention has been attached to developing

students’ ability to apply mathematics. The National Council of Teachers of
Mathematics (NCTM), for example, states that the mathematics curriculum should
focus on “mathematics that will prepare students for continued study and for
solving problems in a variety of school, home and work settings” (NCTM, 2000, p.
14–15). Ability to apply mathematics in real-world contexts is also considered as a
key area of mathematical competence in many European countries (Eurydice,
2011; Tomlinson, 2004). At a global level, the utilitarian purpose of mathematics in
everyday life is also taken into consideration by the OECD through its Programme
for International Student Assessment (PISA).

PISA is a large-scale assessment that examines students’ ability to apply

mathematics in a variety of situations. PISA introduces the term ‘mathematical
literacy’ that is defined as “an individual’s capacity to identify and understand the
role that mathematics plays in the world, to make well-founded judgments and to
use and engage with mathematics in ways that meet the needs of that individual’s
life as constructive, concerned and reflective citizens” (OECD, 2003, p. 24).

A focus on the application of mathematics implies the use of mathematics

problems situated in real-world contexts in mathematics teaching and learning. De

10

Thesis Ari_141201_FINAL.pdf 10 1-12-2014 16:26:21

 Introduction

Lange (2003) argued that students need to experience solving mathematics

problems in different situations and contexts to develop their ability to transfer
their knowledge from one area of application to another. Similarly, NCTM (2000)
also pointed out the importance of providing students with opportunities to work
on problems arising in contexts outside mathematics, which could be other subject
disciplines or students’ daily experiences. In the PISA studies, great value is also
attached to “tasks that could be encountered in a variety of real-world situations,
and that have a context in which the use of mathematics to solve the problem
would be authentic” (OECD, 2003, p. 34). Such tasks are used in the PISA studies
to assess mathematical literacy.

1.2 A discrepancy between the Indonesian mathematics curriculum

and student achievement
Students’ ability to apply mathematics is also considered as an important goal of
mathematics education in Indonesia. In 2004, the Indonesian Ministry of National
Education1 started implementing a competence-based curriculum (Kurikulum
Berbasis Kompetensi – KBK), which was revised into a school-based curriculum
(Kurikulum Tingkat Satuan Pendidikan2 – KTSP) in 2006. In contrast to previous
curricula that were oriented on the content of subject matter and emphasized
students’ acquisition of knowledge, both KBK and KTSP placed a premium on
developing students’ ability to apply their knowledge. These two curricula
considered that the subject of mathematics should target developing students’
ability to: (1) understand the concepts of mathematics, explain the relevance of
concepts, and apply the concepts or algorithms in a flexible way in problem-
solving; (2) solve problems that require the ability to understand a problem, design
and complete a mathematical model to solve it, and interpret the solution; and (3)
appreciate the purpose of mathematics in life (Pusat Kurikulum [The National
Curriculum Center], 2003; the Regulation of the Minister of National Education,
No. 22, Year 2006, about Standards of Content). This educational goal is also
considered in the newly implemented Curriculum 2013 in which the Indonesian
government clearly mandates that education must be relevant to the needs of life
and offers students opportunities to apply their knowledge in society (Kementerian

1 The Ministry of National Education was renamed the Ministry of Education and Culture in 2011.
2 As a consequence of the decentralization of education, the Indonesian Ministry of National Education
revised KBK into KTSP by adding the principle of school-based management. In KTSP, the government
determined the minimum standards (i.e. competences, content, and targeted performance) and every school
had the freedom to develop or adjust the minimum standards based on the school’s resources, students’
needs, and district (local) educational goals.

11

Thesis Ari_141201_FINAL.pdf 11 1-12-2014 16:26:21

 Chapter 1

Pendidikan dan Kebudayaan [the Ministry of Education and Culture], 2012). In

relation to the objectives of Indonesian mathematics education, KBK and KTSP
recommended teachers using mathematics problems situated in real-world contexts
in their classroom practices. In line with these curricula, there have been
educational movements in Indonesia that emphasize the application of
mathematics and promote the use of real-world problems in learning mathematics:
i.e. Pendidikan Matematika Realistik Indonesia (Indonesian Realistic Mathematics
Education) (see e.g., Sembiring, Hadi, & Dolk, 2008; Sembiring, Hoogland, &
Dolk, 2010) and Pembelajaran Kontekstual (Contextual Teaching and Learning)
(Supinah, 2008).

The attention for students’ ability to apply mathematics is also reflected in

Indonesia’s regular participation in PISA since the first survey in 2000. Despite
strong attention for and interest in the application of mathematics in Indonesian
curricula, however, the PISA results show Indonesian students’ low performance
on mathematics problems situated in real-world contexts. In the latest PISA 2012
study, for example, less than 1% of Indonesian students could solve mathematics
problems that are situated in complex situation and require mathematical modeling
and well-developed reasoning skills (OECD, 2013). Three quarters of Indonesian
students could only solve mathematics problems that use familiar contexts, have
clear questions, and present all relevant information. These students could identify
relevant information and carry out routine mathematical procedures if explicit
instructions were given.

The low performance of Indonesian students on mathematics problems situated in

real-world contexts, which in this thesis are called ‘context-based tasks’, prompted
us to carry out this PhD research in the Context-based Mathematics Tasks
Indonesia (CoMTI) project. The main purpose of this PhD research was to gain a
better insight into Indonesian students´ low performance on context-based tasks
and to generate recommendations for improving student performance.

2 What does a context-based task mean in this PhD thesis?

A variety of terms has been used to refer to problems that are situated in real-world
contexts; for example ‘word problems’ (see, e.g. Bernardo, 1999; Verschaffel, Greer,
& De Corte, 2000; Verschaffel, Van Dooren, Greer, & Mukhopadhyay, 2010),
‘authentic tasks’ (see, e.g. Palm, 2008; Kramarski, Mevarech, & Arami, 2002),
‘context problems’ (see, e.g. Van den Heuvel-Panhuizen, 1996; 2005), or ‘modeling

12

Thesis Ari_141201_FINAL.pdf 12 1-12-2014 16:26:21

 Introduction

tasks’ (see, e.g. Blum, 2011; Maass, 2010; Tasova & Delice, 2012). According to
Maass (2010), ‘word problems’ are often seen as “very simple, often artificial tasks
with text” (p. 290). Daily experience can often be ignored when solving a word
problem because the contexts are artificial. Such a conception leads to an
impression that the solution of a word problem can be obtained by operating all
numbers given in the text with procedures that are provided explicitly (Maass,
2010). However, there is a more open definition of word problems that refers to
mathematical modeling (see, e.g. Verschaffel et al., 2000; 2010). According to these
studies, word problems should be used to support the development of modeling
competences and cannot be solved by simply taking and combining all numbers
provided in the text. The latter definition of a word problem is close to a so-called
‘authentic task’ that also refers to mathematical modeling. An authentic task is a
mathematics task that represents a real life problem or situation (Palm, 2008) and
does not have a ready-made algorithm (Kramarski et al., 2002). Furthermore, an
authentic task often employs realistic data that are incomplete or inconsistent and
asks solvers to use different representations in their solutions. The authenticity of
contexts is also highlighted by Van den Heuvel-Panhuizen (1996; 2005) who
argued that a ‘context problem’ should be authentic for students and should
require them to think within the context of the problem. “By imagining themselves
in the situation to which the problem refers, the students can solve the problem in
a way that was inspired, as it were, by the situation“ (Van den Heuvel-Panhuizen,
2005, p. 6). It means that context problems can suggest strategies to find a
solution, such as using drawings, tables, or graphs. Another term that is used to
refer to problems situated in real-world contexts is ‘modeling tasks’ (see, e.g. Blum,
2011; Maass, 2010; Tasova & Delice, 2012). Modeling tasks emphasize modeling or
translation between real-world contexts and mathematics.

Considering the abovementioned descriptions of problems that use real-world

contexts, in this thesis we define a context-based mathematics task as a
mathematics task that is situated in real-world settings and provides elements or
information that need to be organized and modeled mathematically. We do not use
the term word problems or story problems because contexts can also be depicted
in non-verbal descriptions such as pictures. A consequence of our definition is that
we attach a high value to the following characteristics of context-based tasks: the
tasks have more or less information than what is needed to find the solution and
should not explicitly provide the required mathematical procedures.

13

Thesis Ari_141201_FINAL.pdf 13 1-12-2014 16:26:21

 Chapter 1

3 Context-based tasks and modeling

Solving a problem situated in a real-world context requires interplay between the
real world and mathematics (Blum et al., 2002; Blum, 2011), which is often referred
to as modeling (Greer, 1997). According to Maass (2010), modeling is not simply
the process of making a mathematical representation of a real-world problem, but
it is a whole process that includes “understanding a realistic problem, setting up a
model of the problem and finding a solution by working on the model
mathematically” (p. 287). This process is in general similar to what in PISA is
called ‘mathematization’, which is considered as the fundamental process to solve
real-life problems (OECD, 2003).

Modeling has been considered in many studies on problems situated in real-world

contexts. Maass (2010) used the modeling process as a key reference to make a
classification of tasks that have real-world contexts. She distinguished seven
categories of tasks based on the focus of the tasks on the stages of the modeling
process; i.e. tasks that focus on: (1) the whole process of modeling, (2)
understanding the real-world situation; (3) setting up a real model; (4)
mathematizing; (5) working within the mathematical model; (6) interpreting; and
(7) validating. With respect to the teaching of modelling, Blum (2011) and Blum
and Ferri (2009) considered the modeling process as a specific strategic tool that
needs to be acquired and used by teachers to support their students’ learning. For
this a four-step schema was developed, which comprised understanding the task,
establishing a model, using mathematics, and explaining the result, to be used as
“an aid for difficulties that might occur in the course of the solution process”
(Blum & Ferri, 2009, p. 55)

Considering the relevance of the modeling process to problems situated in real-

world contexts, in this PhD research we use the modeling process as one of the
main references to get a better insight into Indonesian students’ process of solving
context-based tasks.

4 Structure of the thesis

The PhD research that forms the basis of this PhD thesis comprised four studies
that were set up to investigate Indonesian students´ low performance on context-
based tasks and to provide recommendations to improve student performance.
The four studies are presented as the chapters of this thesis. Table 1 illustrates the
structure of the thesis and the topic addressed in each chapter.

14

Thesis Ari_141201_FINAL.pdf 14 1-12-2014 16:26:21

 Introduction

Table 1
Structure of thesis
Topic that is addressed in the chapter
Chapter 1 Introduction
Chapter 2 What difficulties do Indonesian students have when solving context-
based tasks?
Chapter 3
What are possible reasons for Indonesian students’ difficulties when
Chapter 4 solving context-based tasks?

Chapter 5 How can Indonesian student performance on context-based tasks be

improved?
Chapter 6 Conclusion

Chapter 2 reports a study in which we investigated Indonesian students’ difficulties

in solving context-based tasks. To identify these difficulties the types of errors
made by students were examined. The study addressed the following research
questions:
x What errors do Indonesian students make when solving context-based mathematics
tasks?
x What is the relation between the types of errors, the types of context-based tasks, and
the student performance level?

After examining the difficulties experienced by Indonesian students, two studies

were conducted to investigate possible reasons for the difficulties. In these studies,
which are reported in Chapter 3 and Chapter 4, we used the concept of opportunity-
to-learn.

Chapter 3 describes a study in which the opportunity-to-learn to solve context-

based tasks offered in Indonesian mathematics textbooks was examined. The
investigation was done by analyzing the characteristics of tasks in the textbooks.
This study addressed the following research questions:
x What are the amount of exposure to and the purpose of context-based tasks in
Indonesian mathematics textbooks?
x To what extent are different types of information provided in tasks in Indonesian
mathematics textbooks?
x What are the cognitive demands of tasks in Indonesian mathematics textbooks?

15

Thesis Ari_141201_FINAL.pdf 15 1-12-2014 16:26:21

 Chapter 1

Another issue dealt with in this study was the relation between opportunity-to-
learn to solve context-based tasks provided in the textbooks and students’
difficulties in solving such tasks. For this, we addressed the following research
question:
x What is the connection between students’ errors when solving context-based tasks and
the characteristics of tasks in Indonesian mathematics textbooks?

Textbooks might be not the only factor influencing student performance, because
how a textbook is used in the classroom is determined by teachers. Therefore, to
get a better insight into possible explanations for students’ difficulties we also took
the teaching practices of Indonesian mathematics teachers into consideration.

Chapter 4 describes our study on examining a possible explanation for students’

difficulties from the perspective of the opportunity-to-learn (OTL) to solve
context-based tasks offered by teachers through their teaching practices. In this
study the teachers’ teaching practices were seen from two perspectives, i.e. the
teachers’ reported practices and observed practices. The former focused on the
characteristics of context-based tasks presented by teachers in their lessons,
whereas the latter involved the teaching approaches used by teachers to help their
students learn to solve context-based tasks. Considering the potential influence of
beliefs on teachers’ teaching practices, we also investigated teachers’ beliefs about
mathematics, the learning and teaching of mathematics, and context-based tasks.
The following research questions were addressed:
x What beliefs do Indonesian teachers have regarding mathematics, teaching and learning
of mathematics, and context-based tasks?
x What OTL to solve context-based tasks do teachers offer students in their classroom
practice?
- What kinds of context-based tasks do Indonesian teachers offer their students?
- What teaching approach do Indonesian teachers use to teach context-based tasks?
x Is there a relationship between the OTL to solve context-based tasks offered by teachers
and students’ errors when solving such tasks?

Chapter 5 describes a study that investigated whether Indonesian students’

performance in solving context-based tasks can be improved by offering students
opportunity-to-learn to solve such tasks. This opportunity-to-learn, which
comprised a set of context-based tasks and a consultative teaching approach, was
developed based on our findings about what was lacking in Indonesian

16

Thesis Ari_141201_FINAL.pdf 16 1-12-2014 16:26:21

 Introduction

mathematics textbooks (reported in Chapter 3), and in Indonesian teachers’

teaching practices (reported in Chapter 4). In this final study we focused on
answering the following question:
x Does providing students with opportunity-to-learn to solve context-based tasks contribute
to students’ performance when solving these tasks and, more specifically, is there any
effect on the correctness of the answers and on the types of errors?

Chapter 6 summarizes the findings from the four studies and discusses practical
implications of the findings on the teaching and learning of context-based tasks in
Indonesia. In this chapter we also provide suggestions for further research on
context-based tasks.

References
Ananiadou, K., & Claro, M. (2009). 21st Century Skills and competencies for new
millennium in OECD countries. OECD Education Working Papers, No. 41. OECD
Publishing. httpx://dx.doi.org/10.1787/218525261154
Bernardo, A. B. I. (1999). Overcoming obstacles to understanding and solving word
problems in mathematics. Educational Psychology, 19(2), 149–163.
Blum, W. et al. (2002). ICMI Study 14: Application and modelling in mathematics
education – Discussion Document. Educational Studies in Mathematics, 51(1), 149–
171.
Blum, W., & Ferri, R. B. (2009). Mathematical modelling: Can it be taught and learnt?
Journal of Mathematical Modelling and Application, 1(1), 45–58.
Blum, W. (2011). Can modelling be taught and learnt? Some answers from empirical
research. In G. Kaiser, W. Blum, R. B. Ferri & G. Stillman (Eds.), Trends in teaching
and learning of mathematical modelling (pp. 15–30). New York: Springer.
De Lange, J. (2003). Mathematics for literacy. In B. L. Madison & L. A. Steen (Eds.),
Quantitative Literacy: Why numeracy matters for schools and colleges (pp. 75–89).
Princeton, NJ: National Council on Education and Disciplines.
Eurydice. (2011). Mathematics education in Europe: Common challenges and national policies.
Brussels: Education, Audiovisual and Culture Executive Agency.
Graumann, G. (2011). Mathematics for problem in the everyday world. In J. Maasz &
J. O’Donoghue (Eds.), Real-world problems for secondary school mathematics students: Case
studies (pp. 113–122). Rotterdam: Sense Publishers.
Greer, B. (1997). Modelling reality in mathematics classrooms: The case of word
problems. Learning and Instruction, 7(4), 293–307.

17

Thesis Ari_141201_FINAL.pdf 17 1-12-2014 16:26:21

 Chapter 1

Griffin, P., Care, E., & McGaw, B. (2012). The changing role of education and schools.
In P. Griffin, B. McGraw & E. Care (Eds.), Assessment and Teaching of 21st Century
Skills (pp. 1–16). New York: Springer.
Kementerian Pendidikan dan Kebudayaan. (2012). Dokumen Kurikulum 2013 (The
document of the Curriculum 2013). Jakarta: Author.
Kramarski, B., Mevarech, Z. R., & Arami, M. (2002). The effects of metacognitive
instruction on solving mathematical authentic tasks. Educational Studies in
Mathematics, 49(2), 225–250.
Maass, K. (2010). Classification scheme for modelling tasks. Journal für Mathematik-
Didaktik, 31(2), 285–311.
National Council of Teachers of Mathematics. (2000). Principles and standard for school
mathematics. Reston: Author.
OECD. (2003). The PISA 2003 assessment framework - Mathematics, reading, science, and
problem solving knowledge and skills. Paris: Author.
OECD. (2013). PISA 2012 Results: What students know and can do. Student performance in
mathematics, reading and science. Paris: Author.
Palm, T. (2008). Impact of authenticity on sense making in word problem solving.
Educational Studies in Mathematics, 67(1), 37–58.
Partnership for 21st Century Skills. (2002). Learning for the 21st century. A report and mile
guide for 21st century skills. Tucson, AZ: Author
Pusat Kurikulum. (2003). Kurikulum 2004. Standar kompetensi mata pelajaran matematika
Sekolah Menengah Pertama dan Madrasah Tsanawiyah (The curriculum 2004: The
competence standards for mathematics in Junior High School and Islamic Junior
High School). Jakarta: Departemen Pendidikan Nasional.
Sembiring, R., Hoogland, K., & Dolk, M. (2010). A decade of PMRI in Indonesia. Utrecht:
APS.
Sembiring, R. K., Hadi, S., & Dolk, M. (2008). Reforming mathematics learning in
Indonesian classrooms through RME. ZDM The International Journal on Mathematics
Education, 40(6), 927–939.
Supinah. (2008). Pembelajaran matematika SD dengan pendekatan kontekstual dalam
melaksanakan KTSP (Contextual approach in teaching elementary mathematics as
a means to implement KTSP). Yogyakarta: PPPPTK Matematika.
Tasova, H. I., & Delice, A. (2012). An analysis of pre-service mathematics teachers'
performance in modelling tasks in terms of spatial visualisation ability. Research in
Mathematics Education, 14(3), 297–298.

18

Thesis Ari_141201_FINAL.pdf 18 1-12-2014 16:26:21

 Introduction

The Regulation of the Minister of National Education, No. 22, Year 2006, about
Standards of Content.
Tomlinson, M. (2004). 14-19 Curriculum and qualifications reform. Final report of the Working
Group on 14-19 Reform. London: Department for Education and Skills.
Van den Heuvel-Panhuizen, M. (1996). Assessment and Realistic Mathematics Education.
Utrecht: CD-Ƣ Press, Utrecht University.
Van den Heuvel-Panhuizen, M. (2005). The role of context in assessment problems in
mathematics. For the Learning of Mathematics, 25(2), 2–9 + 23.
Verschaffel, L., Greer, B., & De Corte, E. (2000). Making sense of word problems. Lisse:
Swets & Zeitlinger.
Verschaffel, L., Van Dooren, W., Greer, B., & Mukhopadhyay, S. (2010).
Reconceptualising word problems as exercises in mathematical modelling. Journal
für Mathematik-Didaktik, 31(1), 9–29.

19

Thesis Ari_141201_FINAL.pdf 19 1-12-2014 16:26:21

 Chapter 1

20

Thesis Ari_141201_FINAL.pdf 20 1-12-2014 16:26:21

 Chapter 2

Difficulties in solving context-based PISA mathematics

tasks: An analysis of students’ errors

Wijaya, A., Van den Heuvel-Panhuizen, M., Doorman, M., & Robitzsch, A. (2014).
Difficulties in solving context-based PISA mathematics tasks: An analysis of students’
errors. The Mathematics Enthusiast, 11(3), 555-584.

Thesis Ari_141201_FINAL.pdf 21 1-12-2014 16:26:22

 Chapter 2

Difficulties in solving context-based PISA mathematics

tasks: An analysis of students’ errors

1 Introduction
Employer dissatisfaction with school graduates’ inability to apply mathematics
stimulated a movement favoring using mathematics in everyday situations (Boaler,
1993a) and a practice-orientated mathematics education (Graumann, 2011). The
main objective of this movement is to develop students’ ability to apply
mathematics in everyday life (Graumann, 2011) which is seen as a core goal of
mathematics education (Biembengut, 2007; Greer, Verschaffel, & Mukhopadhyay,
2007). In addition, this innovation was also motivated by theoretical developments
in educational psychology such as situated cognition theory (Henning, 2004;
Nunez, Edwards, & Matos, 1999) and socio-cultural theory (Henning, 2004).
Finally, this context-connected approach to mathematics education emerged from
studies of mathematics in out-of-school settings such as supermarkets (Lave, 1988)
and street markets (Nunes, Schliemann, & Carraher, 1993).

In line with this emphasis on application in mathematics education, the utilitarian

purpose of mathematics in everyday life has also become a concern of the
Programme for International Student Assessment (PISA) which is organized by
the Organisation for Economic Co-operation and Development (OECD). PISA is
a large-scale assessment which aims to determine whether students can apply
mathematics in a variety of situations. For this purpose, PISA uses real-world
problems that require quantitative reasoning, spatial reasoning or problem solving
(OECD, 2003b). An analysis of PISA results showed that the competencies
measured in PISA surveys are better predictors for 15 year-old students’ later
success than the qualifications reflected in school marks (Schleicher, 2007).
Therefore, the PISA survey has become an influential factor in reforming
educational practices (Liang, 2010) and making decisions about educational policy
(Grek, 2009; Yore, Anderson, & Hung Chiu, 2010).

Despite the importance of contexts for learning mathematics, several studies

(Cummins, Kintsch, Reusser, & Weimer, 1988; Palm, 2008; Verschaffel, Greer, &
De Corte, 2000) indicate that contexts can also be problematic for students when
they are used in mathematics tasks. Students often miscomprehend the meaning of
context-based tasks (Cummins et al., 1988) and give solutions that are not relevant
to the situation described in the tasks (Palm, 2008). Considering these findings, the

22

Thesis Ari_141201_FINAL.pdf 22 1-12-2014 16:26:22

 Context-based tasks: Students’ difficulties

intention of this study was to clarify students’ obstacles or difficulties when solving
context-based tasks.

As a focus we chose the difficulties students in Indonesia have with context-based

problems. The PISA 2009 study (OECD, 2010) showed that only one third of the
Indonesian students could answer the types of mathematics tasks involving
familiar contexts including all information necessary to solve the tasks and having
clearly defined questions. Furthermore, less than one percent of the Indonesian
students could work with tasks situated in complex situations which require
mathematical modeling, and well-developed thinking and reasoning skills. These
poor results ask for further research because the characteristics of PISA tasks are
relevant to the mathematics learning goals mandated in the Indonesian Curriculum
2004. For example, one of the goals is that students are able to solve problems that
require students to understand the problem, and design and complete a
mathematical model of it, and interpret the solution (Pusat Kurikulum, 2003).

Although the Indonesian Curriculum 2004 takes the application aspect of

mathematical concepts in daily life into account (Pusat Kurikulum, 2003), the PISA
results clearly showed that this curriculum did not yet raise Indonesian students’
achievement in solving context-based mathematics tasks. This finding was the
main reason to set up the ‘Context-based Mathematics Tasks Indonesia’ project, or
in short the CoMTI project. The general goal of this project is to contribute to the
improvement of the Indonesian students’ performance on context-based
mathematics tasks. The present study is the first part of this project and aimed at
clarifying Indonesian students’ difficulties or obstacles when solving context-based
mathematics tasks. Having insight in where students get stuck will provide us with
a key to improve their achievement. Moreover, this insight can contribute to the
theoretical knowledge about the teaching and learning of mathematics in context.

2 Theoretical background

2.1 Learning mathematics in context

Contexts are recognized as important levers for mathematics learning because they
offer various opportunities for students to learn mathematics. The use of contexts
reduces students’ perception of mathematics as a remote body of knowledge
(Boaler, 1993b), and by means of contexts students can develop a better insight
about the usefulness of mathematics for solving daily-life problems (De Lange,

23

Thesis Ari_141201_FINAL.pdf 23 1-12-2014 16:26:22

 Chapter 2

1987). Another benefit of contexts is that they provide students with strategies to
solve mathematical problems (Van den Heuvel-Panhuizen, 1996). When solving a
context-based problem, students might connect the situation of the problem to
their experiences. As a result, students might use not only formal mathematical
procedures, but also informal strategies, such as using repeated subtraction instead
of a formal digit-based long division. In the teaching and learning process,
students’ daily experiences and informal strategies can also be used as a starting
point to introduce mathematics concepts. For example, covering a floor with
squared tiles can be used as the starting point to discuss the formula for the area of
a rectangle. In this way, contexts support the development of students’
mathematical understanding (De Lange, 1987; Gravemeijer & Doorman, 1999;
Van den Heuvel-Panhuizen, 1996).

In mathematics education, the use of contexts can imply different types of

contexts. According to Van den Heuvel-Panhuizen (2005) contexts may refer to
real-world settings, fantasy situations or even to the formal world of mathematics.
This is a wide interpretation of context in which contexts are not restricted to real-
world settings. What is important is that contexts create situations for students that
are experienced as real and related to their common-sense understanding. In
addition, a crucial characteristic of a context for learning mathematics is that there
are possibilities for mathematization. A context should provide information that
can be organized mathematically and should offer opportunities for students to
work within the context by using their pre-existing knowledge and experiences
(Van den Heuvel-Panhuizen, 2005).

The PISA study also uses a broad interpretation of context, defining it as a specific
setting within a ‘situation’ which includes all detailed elements used to formulate
the problem (OECD, 2003b, p. 32). In this definition, ‘situation’ refers to the part
of the students’ world in which the tasks are placed. This includes personal,
educational/occupational, public, and scientific situation types. As well as Van den
Heuvel-Panhuizen (2005), the PISA researchers also see that a formal mathematics
setting can be seen as a context. Such context is called an ‘intra-mathematical
context’ (OECD, 2003b, p. 33) and refers only to mathematical objects, symbols,
or structures without any reference to the real world. However, PISA only uses a
limited number of such contexts in its surveys and places most value on real-world
contexts, which are called ‘extra-mathematical contexts’ (OECD, 2003b, p. 33). To
solve tasks which use extra-mathematical contexts, students need to translate the

24

Thesis Ari_141201_FINAL.pdf 24 1-12-2014 16:26:22

 Context-based tasks: Students’ difficulties

contexts into a mathematical form through the process of mathematization

(OECD, 2003b).

The extra-mathematical contexts defined by PISA are similar to Roth’s (1996)

definition of contexts, which also focuses on the modeling perspective. Roth
(1996, p. 491) defined a context as “a real-world phenomenon that can be modeled
by mathematical form.” In comparison to Van den Heuvel-Panhuizen and the
PISA researchers, Roth takes a narrower perspective on contexts, because he
restricts contexts only to real-world phenomena. However, despite this restriction,
Roth’s focus on the mathematical modeling of the context is close to the idea of
mathematization as used in PISA.

Based on the aforementioned definitions of context, in the present study we

restricted contexts to situations which provide opportunities for mathematization
and are connected to daily life. This aligns with the aim of PISA to assess students’
abilities to apply mathematics in everyday life. In conclusion, we defined context-
based mathematics tasks as tasks situated in real-world settings which provide
elements or information that need to be organized and modeled mathematically.

2.2 Solving context-based mathematics tasks

Solving context-based mathematics tasks requires an interplay between the real
world and mathematics (Schwarzkopf, 2007), which is often described as a
modeling process (Maass, 2010) or mathematization (OECD, 2003b). The process
of modeling begins with a real-world problem, ends with a real-world solution
(Maass, 2010) and is considered to be carried out in seven steps (Blum & Leiss, as
cited in Maass, 2010). As the first step, a solver needs to establish a ‘situation
model’ to understand the real-world problem. The situation model is then
developed into a ‘real model’ through the process of simplifying and structuring. In
the next step, the solver needs to construct a ‘mathematical model’ by
mathematizing the real model. After the mathematical model is established, the
solver can carry out mathematical procedures to get a mathematical solution of the
problem. Then, the mathematical solution has to be interpreted and validated in
terms of the real-world problem. As the final step, the real-world solution has to
be presented in terms of the real-world situation of the problem.

In PISA, the process required to solve a real-world problem is called

‘mathematization’ (OECD, 2003b). This involves: understanding the problem
situated in reality; organizing the real-world problem according to mathematical

25

Thesis Ari_141201_FINAL.pdf 25 1-12-2014 16:26:22

 Chapter 2

concepts and identifying the relevant mathematics; transforming the real-world

problem into a mathematical problem which represents the situation; solving the
mathematical problem; and interpreting the mathematical solution in terms of the
real situation (OECD, 2003b). In general, the stages of PISA’s mathematization are
similar to those of the modeling process. To successfully mathematize, a student
needs to possess mathematical competences which are related to the cognitive
demands of context-based tasks (OECD, 2003b). Concerning the cognitive
demands of a context-based task, PISA defines three types of tasks:
1. Reproduction tasks: These tasks require recalling mathematical objects and
properties, performing routine procedures, applying standard algorithms,
and applying technical skills.
2. Connection tasks: These tasks require the integration and connection from
different mathematical curriculum strands, or the linking of different
representations of a problem. The tasks are non-routine and ask for
transformation between the context and the mathematical world.
3. Reflection tasks: These tasks include complex problem situations in which
it is not obvious in advance which mathematical procedures have to be
carried out.

Regarding students’ performance on context-based tasks, PISA (OECD, 2009a)

found that cognitive demands are crucial aspects of context-based tasks because
they are – among other task characteristics, such as the length of text, the item
format, the mathematical content, and the contexts – the most important factors
influencing item difficulty.

2.3 Analyzing students’ errors in solving context-based mathematics

tasks
To analyze students’ difficulties when solving mathematical word problems,
Newman (1977, 1983) developed a model which is known as Newman Error
Analysis (see also Clarkson, 1991; Clements, 1980). Newman proposed five
categories of errors based on the process of solving mathematical word problems,
namely errors of reading, comprehension, transformation, process skills, and
encoding. To figure out whether Newman’s error categories are also suitable for
analyzing students’ errors in solving context-based tasks which provide
information that needs to be organized and modeled mathematically, we compared
Newman’s error categories with the stages of Blum and Leiss’ modeling process (as
cited in Maass, 2010) and the PISA’s mathematization stages (OECD, 2003b).

26

Thesis Ari_141201_FINAL.pdf 26 1-12-2014 16:26:22

 Context-based tasks: Students’ difficulties

Table 1 shows that of Newman’s five error categories, only the first category that
refers to the technical aspect of reading cannot be matched to a modeling or
mathematization stage of the solution process. The category comprehension
errors, which focuses on students’ inability to understand a problem, corresponds
to the first stage of the modeling process (“understanding problem by establishing
situational model”) and to the first phase of the mathematization process
(“understanding problem situated in reality”). The transformation errors refer to
errors in constructing a mathematical problem or mathematical model of a real-
world problem, which is also a stage in the modeling process and in
mathematization. Newman’s category of errors in mathematical procedures relates
to the modeling stage of working mathematically and the mathematization stage of
solving mathematical problems. Lastly, Newman’s encoding errors correspond to
the final stage of the modeling process and mathematization at which the
mathematical solution is interpreted in terms of the real-world problem situation.
Considering these similarities, it might be concluded that Newman’s error
categories can be used to analyze students’ errors in solving context-based
mathematics tasks.

2.4 Research questions

The CoMTI project aims at improving Indonesian students’ performance in
solving context-based mathematics tasks. To find indications of how to improve
this performance, the first CoMTI study looked for explanations for the low scores
in the PISA surveys by investigating, on the basis of Newman’s error categories,
the difficulties students have when solving context-based mathematics tasks such
as used in the PISA surveys.

Generally expressed our first research question was:

1. What errors do Indonesian students make when solving context-based mathematics

tasks?

A further goal of this study was to investigate the students’ errors in connection
with the cognitive demands of the tasks and the student performance level.
Therefore, our second research question was:

2. What is the relation between the types of errors, the types of context-based tasks (in the
sense of cognitive demands), and the student performance level?

27

Thesis Ari_141201_FINAL.pdf 27 1-12-2014 16:26:22

 Chapter 2

Table 1
Newman’s error categories and stages in solving context-based mathematics tasks
Stages in solving context-based mathematics tasks
Newman’s error
Stages in Blum and Leiss’ Stages in PISA’s
categoriesa
Modelingb Mathematizationc
Reading: -- --
Error in simple
recognition of words
and symbols
Comprehension: Understanding problem by Understanding problem
Error in understanding establishing situational situated in reality
the meaning of a model
problem
-- Establishing real model by --
simplifying situational
model
-- -- Organizing real-world
problems according to
mathematical concepts
and identifying relevant
mathematics
Transformation: Constructing mathematical Transforming real-world
Error in transforming a model by mathematizing problem into
word problem into an real model mathematical problem
appropriate which represents the
mathematical problem problem situation
Process skills: Working mathematically to Solving mathematical
Error in performing get mathematical solution problems
mathematical
procedures
Encoding: Interpreting mathematical Interpreting mathematical
Error in representing solution in relation to solution in terms of real
the mathematical original problem situation situation
solution into acceptable Validating interpreted
written form mathematical solution by
checking whether this is
appropriate and reasonable
for its purpose
-- Communicating the real- --
world solution
a (Newman, 1977, 1983; Clarkson, 1991; Clements, 1980); b (as cited in Maass, 2010); c (OECD,
2003b)

28

Thesis Ari_141201_FINAL.pdf 28 1-12-2014 16:26:22

 Context-based tasks: Students’ difficulties

3 Method

3.1 Mathematics tasks

A so-called ‘CoMTI test’ was administered to collect data about students’ errors
when solving context-based mathematics tasks. The test items were selected from
the PISA’s (OECD, 2009b) released ‘mathematics units’1 and included only those
units which were situated in extra-mathematical context. Furthermore, to get a
broad view of the kinds of difficulties Indonesian students encounter, units were
selected in which Indonesian students in the PISA 2003 survey (OECD, 2009b)
had either a notably low or high percentage of correct answer. In total we arrived
at 19 mathematics units consisting of 34 questions. Hereafter, we will call these
questions ‘tasks’, because they are not just additional questions to a main problem
but complete problems on their own, which can be solved independently of each
other. Based on the PISA qualification of the tasks we included 15 reproduction,
15 connection and 4 reflection tasks. The tasks were equally distributed over four
different booklets according to their difficulty level, as reflected in the percentage
correct answers found in the PISA 2003 survey (OECD, 2009b). Six of the tasks
were used as anchor tasks and were included in all booklets. Every student took
one booklet consisting of 12 to 14 tasks.

The CoMTI test was administered in the period from 16 May to 2 July, 2011,
which is in agreement with the testing period of PISA (which is generally between
March 1 and August 31) (OECD, 2005). In the CoMTI test the students were
asked to show how they solved each task, while in the PISA survey this was only
asked for the open constructed-response tasks. Consequently, the time allocated
for solving the tasks in the CoMTI test was longer (50 minutes for 12 to 14 tasks)
than in the PISA surveys (35 minutes for 16 tasks) (OECD, 2005).

3.2 Participants
The total sample in this CoMTI study included 362 students who were recruited
from eleven schools located in rural and urban areas in the province of Yogyakarta,
Indonesia. Although this selection might have as a consequence that the students

1 In PISA (see OECD, 2003b, p. 52) a ‘mathematics unit’ consists of one or more questions which
can be answered independently. These questions are based on the same context which is generally
presented by a written description and a graphic or another representation.

29

Thesis Ari_141201_FINAL.pdf 29 1-12-2014 16:26:22

 Chapter 2

in our sample were at a higher academic level than the national average2, we chose
this province for carrying out the present study for reasons of convenience (the
first author originates from this province).

To have the sample of this study close to the age range of 15 years and 3 months
to 16 years and 2 months, which is taken in the PISA surveys as the
operationalization of fifteen-year-olds (OECD, 2005), and which also applies to
the Indonesian sample in the PISA surveys, we did this study with Grade 9 and
Grade 10 students who generally are of this age. However, it turned out that in our
sample the students were in the age range from 14 years and 2 months to 18 years
and 6 months (see Table 2), which means that our sample had younger and older
students than in the PISA sample.

Table 2
Composition of the sample
Grade Boys Girls Total Min. age Max. age Mean age (SD)
Grade 9 85 148 233 14 y; 2 m 16 y; 7 m 15 y; 3m (5 m)a
Grade 10 59 70 129 14 y; 10 m 18 y; 6 m 16 y; 4 m (7 m)
Total 144 218 362
a y= year; m = month

Before analyzing students’ errors, we also checked whether the ability level of the
students in our sample was comparable to the level of the Indonesian students
who participated in the PISA surveys. For this purpose, we compared the
percentages of correct answers of 17 tasks included in the PISA 2003 survey
(OECD, 2009b), with the scores we found in our sample.

To obtain the percentages of correct answers in our sample, we scored the

students’ responses according to the PISA marking scheme, which uses three
categories: full credit, partial credit, and no credit (OECD, 2009b). The interrater
reliability of this scoring was checked by conducting a second scoring by an
external judge for approximately 15% of students’ responses to the open
constructed-response tasks. The multiple-choice and closed constructed-response
tasks were not included in the check of the interrater reliability, because the scoring

2 For example, of the 33 provinces in Indonesia the province of Yogyakarta occupied the 6th place in
the national examination in the academic year of 2007/2008 (Mardapi & Kartowagiran, 2009).

30

Thesis Ari_141201_FINAL.pdf 30 1-12-2014 16:26:22

 Context-based tasks: Students’ difficulties

for these tasks was straightforward. We obtained a Cohen’s Kappa of .76, which
indicates that the scoring was reliable (Landis & Koch, 1977).

The calculation of the Pearson correlation coefficient between the percentages

correct answers of the 17 tasks in the PISA 2003 survey and in the CoMTI sample
revealed a significant correlation, r (15) = .83, p < .05. This result indicates that
the tasks which were difficult for Indonesian students in the PISA 2003 survey
were also difficult for the students participating in the CoMTI study (see Figure 1).
However, the students in the CoMTI study performed better than the Indonesian
students in the PISA 2003 survey. The mean percentage of correct answers of the
students in the present study was 61%, which is a remarkably higher result than the
29% correct answers of the Indonesian students in the PISA survey. We assume
that this result was due to the higher academic performance of students in the
province of Yogyakarta compared to the performance of Indonesian students in
general (see Footnote 2).

Figure 1. Percentage correct answers in the CoMTI sample and the Indonesia PISA
2003 sample

3.3 Procedure of coding the errors

To investigate the errors, only the students’ incorrect responses, i.e., the responses
with no credit or partial credit, were coded. Missing responses which were also

31

Thesis Ari_141201_FINAL.pdf 31 1-12-2014 16:26:22

 Chapter 2

categorized as no credit were not coded and were excluded from the analysis
because students’ errors cannot be identified from a blank response.

The scheme that was used to code students’ errors (see Table 3) was based on
Newman’s error categories and in agreement with Blum and Leiss’ modeling
process and PISA’s mathematization stages. However, in this coding scheme we
included only four of Newman’s error categories, namely ‘comprehension’,
‘transformation’, ‘mathematical processing’, and ‘encoding’ errors. Instead of
Newman’s error category of ‘process skills’, we used the term ‘mathematical
processing’, because in this way it is more clear that errors in process skills concern
errors in processing mathematical procedures. The technical error type of ‘reading’
was not used because this type of error does not refer to understanding the
meaning of a task. Moreover, the code ‘unknown’ was added in the coding scheme
because in about 8% of the incorrect responses, the written responses did not
provide enough information for coding the errors. These responses with the code
‘unknown’ were not included in the analysis.

To make the coding more fine-grained the four error types were specified into a
number of sub-types (see Table 3), which was done on the basis of a first
exploration of the data and a further literature review. For example, a study of
Leinhardt, Zaslavsky, and Stein (1990) was used to establish sub-types related to
the use of graphs, which resulted in three sub-types: ‘treating a graph as a picture’,
‘point-interval confusion’, and ‘slope-height confusion’. The last two sub-types
belonged clearly to the error type of mathematical processing. The sub-type
‘treating a graph as a picture’ was classified under the error type of transformation
because it indicates that the students do not think about the mathematical
properties of a graph. Because students can make more than one error when
solving a task, a multiple coding was applied in which a response could be coded
with more than one code.

The coding was carried out by the first author and afterwards the reliability of the
coding was checked through an additional coding by an external coder. This extra
coding was done on the basis of 22% of students’ incorrect responses which were
randomly selected from all mathematics units. In agreement with the multiple
coding procedure, we calculated the interrater reliability for each error type and the
code unknown, which resulted in Cohen’s Kappa of .72 for comprehension errors,
.73 for transformation errors, .79 for errors in mathematical processing, .89 for

32

Thesis Ari_141201_FINAL.pdf 32 1-12-2014 16:26:22

 Context-based tasks: Students’ difficulties

encoding errors, and .80 for unknown errors, which indicate that the coding was
reliable (Landis & Koch, 1977).

3.4 Statistical analyses

To investigate the relationship between error types and task types, a chi-square test
of independence was conducted on the basis of the students’ responses. Because
these responses are nested within students, a chi-square with a Rao-Scott
adjustment for clustered data in the R survey package was used (Lumley, 2004;
2012).

For studying the relationship of student performance and error type, we applied a
Rasch analysis to obtain scale scores of the students’ performance. The reason for
choosing this analysis is that it can take into account an incomplete test design
(different students got different test booklets with a different set of tasks). A partial
credit model was specified in ConQuest (Wu, Adams, Wilson, & Haldane, 2007).
The scale scores were estimated within this item response model by weighted
likelihood estimates (Warm, 1989) and were categorized into four almost equally
distributed performance levels where Level 1 indicates the lowest performance and
Level 4 the highest performance. To test whether the frequency of a specific error
type differed between performance levels, we applied an analysis of variance based
on linear mixed models (Bates, Maechler, & Bolker, 2011). This analysis was based
on all responses where an error could be coded and treated the nesting of task
responses within students by specifying a random effect for students.

4 Results

4.1 Overview of the observed types of errors

In total, there were 4707 possible responses (number of tasks done by all students
in total) which included 2472 correct responses (53%), 1532 incorrect responses
(33%), i.e., no credit or partial credit, and 703 missing responses (15%). The error
analysis was carried out for the 1532 incorrect responses. The analysis of these
responses, based on the multiple coding, revealed that 1718 errors were made by
the students. Of these errors 38% were comprehension errors and 42% were
transformation errors. Mathematical processing errors were less frequently made
(17%) and encoding errors only occurred a few times (3%) (see Table 4).

33

Thesis Ari_141201_FINAL.pdf 33 1-12-2014 16:26:22

 Table 3

34
Coding Scheme for error types when solving context-based mathematics tasks

Thesis Ari_141201_FINAL.pdf 34
Chapter 2

Error type Sub-type Explanation

Comprehension Misunderstanding the instruction Student incorrectly interprets what (s)he is asked to do.
Misunderstanding a keyword Student misunderstands a keyword, which is usually a mathematical term.
Error in selecting information Student is unable to distinguish between relevant and irrelevant information (e.g.
using all information provided in a task or neglecting relevant information) or is
unable to gather required information which is not provided in the task.

Transformation Procedural tendency Student tends to directly use a mathematical procedure such as formula or
algorithm without analyzing whether or not it is needed.
Taking too much account of the Student’s answer only refers to the context or real-world situation without taking
context the perspective of the mathematics.
Wrong mathematical Student uses mathematical procedures or concepts which are not relevant to the
operation/concept tasks.
Treating a graph as a picture Student treats a graph as a literal picture of a situation. (S)he interprets and
focuses on the shape of the graph, instead of on the properties of the graph.

Mathematical Algebraic error Error in solving algebraic expression or function.

Processing
Arithmetical error Error in calculation.

1-12-2014 16:26:22
 Error type Sub-type Explanation

Thesis Ari_141201_FINAL.pdf 35
Mathematical Error in mathematical
Processing interpretation of graph:
- Point-interval confusion Student mistakenly focuses on a single point rather than on an interval.
- Slope-height confusion Student does not use the slope of the graph but only focuses on the vertical
distance.
Measurement error Student cannot convert between standard units (e.g. from m/minute to km/h)
or from a non-standard unit to a standard unit (e.g. from step/minute to
m/minute).
Improper use of scale Student cannot select and use the scale of a map properly.
Unfinished answer Student uses a correct procedure, but (s)he does not finish it.

Encoding Student is unable to correctly interpret and validate the mathematical solution in
terms of the real-world problem. This error is reflected by an impossible or not
realistic answer

Unknown Type of error could not be identified due to limited information from student’s
work.
Context-based tasks: Students’ difficulties

35

1-12-2014 16:26:22
 Chapter 2

Table 4
Frequencies of error types
Type of error N %
Comprehension (C) 653 38
Transformation (T) 723 42
Mathematical processing (M) 291 17
Encoding (E) 51 3
Total of observed errors 1718a 100
a Because of multiple coding, the total of observed errors exceeds the number of incorrect
responses (i.e. n = 1532). In total we had 13 coding categories (including combinations of error
types); the six most frequently coded categories were C, CM, CT, M, ME, and T.

4.1.1 Observed comprehension errors

When making comprehension errors, students had problems with understanding
the meaning of a task. This was because they misunderstood the instruction or a
particular keyword, or they had difficulties in selecting the correct information.
Errors in selecting information included half of the 653 comprehension errors and
indicate that students had difficulty in distinguishing between relevant and
irrelevant information provided in the task or in gathering required information
which was not provided in the task (see Table 5).

Table 5.
Frequencies of sub-types of comprehension errors
Sub-type of comprehension error N %
Misunderstanding the instruction 227 35
Misunderstanding a keyword 100 15
Error in selecting information 326 50
Total of observed errors 653 100

Figure 2 shows an example of student work which contains an error in selecting

information. The student had to solve the Staircase task, which was about finding
the height of each step of a staircase consisting of 14 steps.

36

Thesis Ari_141201_FINAL.pdf 36 1-12-2014 16:26:22

 Context-based tasks: Students’ difficulties

Figure 2. Example of comprehension error

The student seemed to have deduced correctly that the height of the staircase had
to be divided by the number of steps in order to get the height of each step.
However, he did not divide the total height 252 (cm) by 14. Instead, he took the
400 and subtracted 252 from it and then he divided the result of it, which is 148,
by 14. So, the student made a calculation with the given total depth of the staircase,
though this was irrelevant for solving the task.

4.1.2 Observed transformation errors

Within the transformation errors, the most dominant sub-type was using a wrong
mathematical operation or concept. Of the 723 transformation errors, two thirds
belonged to this sub-type (see Table 6).

Table 6.
Frequencies of sub-types of transformation errors
Sub-type of transformation error n %
Procedural tendency 90 12
Taking too much account of the context 56 8
Wrong mathematical operation/concept 489 68
Treating a graph as a picture 88 12
Total of observed errors 723 100

37

Thesis Ari_141201_FINAL.pdf 37 1-12-2014 16:26:22

 Chapter 2

Figure 3 shows the response of a student who made a transformation error. The
task was about the concept of direct proportion situated in the context of money
exchange. The student was asked to convert 3900 ZAR to Singapore dollars with
an exchange rate of 1 SGD = 4.0 ZAR. Instead of dividing 3900 by 4.0, the
student multiplied 3900 by 4.0. This means the student chose the wrong
mathematical procedure for solving the task.

Figure 3. Example of transformation error

4.1.3 Observed mathematical processing errors

Mathematical processing errors correspond to students’ failure in carrying out
mathematical procedures (for an overview of these errors, see Table 3). This type
of errors is mostly dependent on the mathematical topic addressed in a task. For
example, errors in interpreting a graph do not occur when there is no graph in the
task. Consequently, the frequencies of the sub-types of mathematical processing
errors were calculated only for the related tasks, i.e. tasks in which such errors may
occur (see Table 7).

38

Thesis Ari_141201_FINAL.pdf 38 1-12-2014 16:26:22

 Context-based tasks: Students’ difficulties

Table 7.
Frequencies of sub-types of mathematical processing errors
Mathematical
Related All errors in
Sub-type of processing errors
tasks related tasks
mathematical processing error in related tasks
n n n %
Algebraic error 8 243 33 14
Arithmetical error 20 956 94 10
Error in interpreting graph 6 155 43 28
Measurement error 1 74 15 20
Error related to improper use of scale 1 177 49 28
Unfinished answer 26 1125 79 7

An example of a mathematical processing error is shown in a task in Figure 4. The

n
task is about finding a man’s pace length (P) by using the formula 140 in
P
which n, the number of steps per minute, is given. The student correctly
substituted the given information into the formula and came to 70
140 . In the
P
next step, however, instead of dividing 70 by 140 the student subtracted 140 by 70.
This response indicates that the student had difficulty to work with an equation in
which the unknown was the divisor and the dividend is smaller than the quotient.

4.1.4 Observed encoding errors

Encoding errors were not divided into sub-types. They comprise all errors that are
related to students’ inability to interpret a mathematical answer as a solution that
fits to the real-world context of a task. As mentioned earlier only 3% of the total
errors belonged to this category. The response of the student in Figure 4, discussed
in the previous section, also contains an encoding error. His answer of 70 is, within
the context of this task, an answer that does not make sense. A human’s pace
length of 70 meter is a rather unrealistic answer.

39

Thesis Ari_141201_FINAL.pdf 39 1-12-2014 16:26:22

 Chapter 2

Figure 4. Example of mathematical processing error and encoding error

4.2 The relation between the types of errors and the types of tasks
In agreement with the PISA findings (OECD, 2009a), we found that the
reproduction tasks were the easiest for the students, whereas the tasks with a
higher cognitive demand, the connection and the reflection tasks, had a higher
percentage of completely wrong answers (which means no credit) (see Figure 5).

To investigate the relation between the error types and the tasks types we
performed a chi-square test of independence based on the six most frequently
coded error categories (see the note in Table 4). The test showed that there was a
significant relation (F2(10, N = 1393) = 91.385, p < .001). Furthermore, we found
a moderate association between the error types and the types of tasks (Phi
coefficient = .256; Cramer’s V = .181).

40

Thesis Ari_141201_FINAL.pdf 40 1-12-2014 16:26:22

 Context-based tasks: Students’ difficulties

Figure 5. Percentage of full credit, partial credit, no credit, and missing response per
task type

When examining the proportion of error types within every task type, it was found
that in the reproduction tasks, mostly comprehension errors (37%) and
transformation errors (34%) were made (see Table 8). Also in the connection tasks
students made mostly comprehension errors (41%) and transformation errors
(43%). However, in the reflection tasks mostly transformation errors (66%) were
made. Furthermore, the analysis revealed that out of the three types of tasks the
connection tasks had the highest average number of errors per task.

41

Thesis Ari_141201_FINAL.pdf 41 1-12-2014 16:26:22

 Chapter 2

Table 8.

Thesis Ari_141201_FINAL.pdf 42
Error types within task type
Compre- Transfor- Mathematical Encoding
hension mation Processing error Total errors
Incorrect error error error
Tasks
Type of task responses average
n
na number of
n % n % n % n % n %a
errors per
task
Reproduction 15 518 211 37 194 34 136 24 23 4 564 99 38
Connection 15 852 404 41 424 43 140 14 28 3 996 101 66
Reflection 4 162 38 24 105 66 15 9 0 0 158 99 40
Total 34 1532 653 723 291 51 1718b
a Because of rounding off, the total percentages are not equal to 100%
b Because of multiple coding, the total errors exceeds the total number of incorrect responses

42

1-12-2014 16:26:22
 Context-based tasks: Students’ difficulties

4.3 The relation between the types of errors, the types of tasks, and
the students’ performance level
When testing whether students on different performance levels differed with
respect to the error types they made, it was found, for all task types together, that
the low performing students (Level 1 and Level 2) made more transformation
errors than the high performing students (Level 3 and Level 4) (see Figure 6)3. For 5

the mathematical processing errors the pattern was opposite. Here, more errors
were made by the high performing students than by the low performing students.
With respect to the comprehension errors there was no such a difference. The low
and high performing students made about the same number of comprehension
errors.

70

60

50
Percentage

40 Comprehension
Transformation
30
Mathematical processing
20 Encoding
n = Number of incorrect responses
10

0
1 (n=541) 2 (n=458) 3 (n=316) 4 (n=217)

Student performance level

Figure 6. Types of error in all tasks for different performance levels

A nearly similar pattern of error type and performance level was also observed
when the analysis was zoomed in on the connection tasks (see Figure 7). For the
reproduction tasks (see Figure 8) the pattern was also quite comparable. The only
difference was that the high performing students made less comprehension errors
than the low performing students.

35The diagram in Figure 6 (and similarly in Figure 7, Figure 8, and Figure 9) can be read as follows:
the students at Level 1 gave in total 541 incorrect responses of which 45% contained comprehension
errors, 47% transformation errors, 15% mathematical processing errors, and 5% encoding errors.
Because of multiple coding, the total percentage exceeds 100%.

43

Thesis Ari_141201_FINAL.pdf 43 1-12-2014 16:26:22

 Chapter 2

70

60

50
Percentage

40 Comprehension
30 Transformation

20 Mathematical processing
Encoding
10
n = Number of incorrect responses
0
1 (n=247) 2 (n=238) 3 (n=211) 4 (n=156)

Student performance level

Figure 7. Types of errors in connection tasks for different performance levels

70

60

50
Percentage

40 Comprehension
30 Transformation

20 Mathematical processing
Encoding
10
n = Number of incorrect responses
0
1 (n=241) 2 (n=160) 3 (n=78) 4 (n=39)

Student performance level

Figure 8. Types of errors in reproduction tasks for different performance levels

For the reflection tasks it was found that the high performing students made more
mathematical processing errors than the low performing students (see Figure 9).
For the other error types, there was no remarkable difference across student
performance levels.

44

Thesis Ari_141201_FINAL.pdf 44 1-12-2014 16:26:23

 Context-based tasks: Students’ difficulties

80
70
Percentage

60 Comprehension
50 Transformation
40
Mathematical processing
30
Encoding
20
n = Number of incorrect responses
10
0
1 (n=53) 2 (n=60) 3 (n=27) 4 (n=22)
Student performance level

Figure 9. Types of errors in reflection tasks for different performance levels

5 Conclusions and discussion

The present study was aimed at getting a better understanding of students’ errors
when solving context-based mathematics tasks. Figure 10 summarizes the findings
regarding the types of errors Indonesian nine- and ten-graders made when solving
these tasks. Out of the four types of errors that were derived from Newman (1977;
1983), it was found that comprehension and transformation errors were most
dominant and that students made fewer errors in mathematical processing and in
the interpretation of the mathematical solution in terms of the original real-world
situation. This implies that the students involved in the present study mostly
experienced difficulties in the early stages of solving context-based mathematics
tasks as described by Blum and Leiss (cited in Maass, 2010) and PISA (OECD,
2003b), i.e. comprehending a real-world problem and transforming it into a
mathematical problem.

Furthermore, within the category of comprehension errors our analysis revealed

that most students were unable to select relevant information. Students tended to
use all numbers given in the text without considering their relevance to solving the
task. This finding provides a new perspective on students’ errors in understanding
real-world tasks because previous studies (Bernardo, 1999; Cummins et al., 1988)
mainly concerned students’ errors in relation to the language used in the
presentation of the task. For example, they found that students had difficulties to
understand the mathematical connotation of particular words.

45

Thesis Ari_141201_FINAL.pdf 45 1-12-2014 16:26:23

 Chapter 2

Type of errors
Comprehension Transformation Math. Processing Encoding
Inability to understand the Inability to Inability to perform the Inability to interpret the
meaning of a task transformation the mathematical procedures mathematical solution
context problem into a in terms of the real
mathematical model situation
All tasks
All students All students All students All students

L H L H L H L H

Reproduction tasks
Recalling mathematical All students All students All students All students
properties, performing routine
procedures or standard
algorithms, and applying L H L H L H L H
technical skills

Connection tasks
Integrating and connecting All students All students All students All students
different mathematical
curriculum strands, or
linking different L H L H L H L H
representations of a problem

Reflection tasks
All students All students All students All students
Using complex problem
situations in which the
relevant mathematical L H L H L H L H
procedures are not obvious

Percentage of errors L = low performing students

57% - 70% H = high performing students
43% - 56%
29% - 42%
15% - 28%
0 < 14%

a The percentages of errors range from 0% to 70% because the maximum percentage of students’ errors
is 66%. This range is divided into five equal levels. In the figure, these levels are represented in cells
with different degrees of shading.

Figure 10. Summary of research findings

The above findings suggest that focusing on the early stages of modeling process
or mathematization might be an important key to improve students’ performance
on context-based tasks. In particular for comprehending a real-world problem,
much attention should be given to tasks with lacking or superfluous information in
which students have to use their daily-life knowledge or have to select the
information that is relevant to solve a particular task.

A further focus of the present study was the relation between the type of tasks and
the types of students’ errors. In agreement with the PISA findings (OECD, 2009a)
we found that the cognitive demands of the tasks are an important factor
influencing the difficulty level of context-based tasks. Because we did not only

46

Thesis Ari_141201_FINAL.pdf 46 1-12-2014 16:26:23

 Context-based tasks: Students’ difficulties

look at the correctness of the answers but also to the errors made by the students,
we could reveal that most errors were not made in the reflection tasks, i.e. the tasks
with the highest cognitive demand, but in the connection tasks. It seems as if these
tasks are most vulnerable for mistakes. Furthermore, our analysis revealed that in
the reflections tasks students made less comprehension errors than in the
connection tasks and the reproduction tasks. One possible reason might be that
most reflection tasks used in the present study did not provide either more or less
information than needed to solve the task. Consequently, students in the present
study did hardly have to deal with selecting relevant information.

Regarding the relation between the types of errors and the student performance
level, we found that generally the low performing students made more
comprehension errors and transformation errors than the high performing
students. For the mathematical processing errors the opposite was found. The high
performing students made more mathematical processing errors than the low
performing students. A possible explanation for this result is that the low
performing students, in contrast to the high performing students, might get stuck
in the first two stages of solving context-based mathematics tasks and therefore are
not arriving at the stage of carrying out mathematical procedures. These findings
confirm Newman’s (1977) argument that the error types might have a hierarchical
structure: failures on a particular step of solving a task prevents a student from
progressing to the next step.

In sum, the present study gave a better insight into the errors students make when
solving context-based tasks and provided us with indications for improving their
achievement. The results signify that paying more attention to comprehending a
task, in particular selecting relevant data, and to transforming a task, which means
identifying an adequate mathematical procedure, both might improve low
performing students’ ability to solve context-based tasks. For the high performing
students, our results show that they may benefit from paying more attention to
performing mathematical procedures.

However, when making use of the findings of the present study this should be
done with prudence, because this study has some limitations that need to be taken
into account. What we found in this Indonesian sample does not necessarily apply
to students in other countries with different educational practices. In addition, the
classification of task types – reproduction, connection and reflection – as
determined in the PISA study might not always be experienced by the students in a

47

Thesis Ari_141201_FINAL.pdf 47 1-12-2014 16:26:23

 Chapter 2

similar way. For example, whether a reproduction task is a reproduction task for
the students also depends on their prior knowledge and experiences. For instance,
as described by Kolovou, Van den Heuvel-Panhuizen, and Bakker (2009), students
who have not learned algebra cannot use a routine algebraic procedure to split a
number into several successive numbers (such as splitting 51 into 16, 17, and 18).
Instead, they might use an informal reasoning strategy with trial-and-error to solve
it. In this case, for these students the task is a connection task, whereas for
students who have learned algebra it might be a reproduction task.

Notwithstanding the aforementioned limitations, the results of the present study

provide a basis for further research into the possible causes of students’ difficulties
in solving context-based mathematics tasks. For finding causes of the difficulties
that students encounter, in addition to analyzing students’ errors, it is also essential
to examine what opportunities-to-learn students are offered in solving these kinds
of tasks. Investigating these learning opportunities will be our next step in the
CoMTI project.

References
Bates, D., Maechler, M., & Bolker, B. (2011). lme4: Linear mixed-effects models using
S4 classes. R package version 0.999375-42.
Bernardo, A. B. I. (1999). Overcoming obstacles to understanding and solving word
problems in mathematics. Educational Psychology, 19(2), 149–163.
Biembengut, M. S. (2007). Modelling and applications in primary education. In W.
Blum, P. L. Galbraith, H.-W. Henn, & M. Niss (Eds.), Modelling and applications in
mathematics education (pp. 451–456). New York: Springer.
Boaler, J. (1993a). Encouraging transfer of ‘school’ mathematics to the ‘real world’
through the integration of process and content, context and culture. Educational
Studies in Mathematics, 25(4), 341–373.
Boaler, J. (1993b). The role of contexts in the mathematics classroom: Do they make
mathematics more real? For the Learning of Mathematics, 13(2), 12–17.
Clarkson, P. C. (1991). Language comprehension errors: A further investigation.
Mathematics Education Research Journal, 3(2), 24–33.
Clements, M. A. (1980). Analyzing children’s errors on written mathematical task.
Educational Studies in Mathematics, 11(1), 1–21.
De Lange, J. (1987). Mathematics, insight and meaning. Utrecht: OW & OC,
Rijksuniversiteit Utrecht.

48

Thesis Ari_141201_FINAL.pdf 48 1-12-2014 16:26:23

 Context-based tasks: Students’ difficulties

Graumann, G. (2011). Mathematics for problem in the everyday world. In J. Maasz &
J. O'Donoghue (Eds.), Real-world problems for secondary school mathematics students: case
studies (pp. 113–122). Rotterdam: Sense Publishers.
Gravemeijer, K.P.E. & Doorman, L.M. (1999). Context problems in realistic
mathematics education: A calculus course as an example. Educational Studies in
Mathematics, 39(1-3), pp. 111–129.
Greer, B., Verschaffel, L., & Mukhopadhyay, S. (2007). Modelling for life: Mathematics
and children’s experience. In W. Blum, P. L. Galbraith, H.-W. Henn & M. Niss
(Eds.), Modelling and applications in mathematics education (pp. 89–98). New York:
Springer.
Grek, S. (2009). Governing by numbers: The PISA ‘effect’ in Europe. Journal of
Education Policy, 24(1), 23–37.
Henning, P. H. (2004). Everyday cognition and situated learning. In D. H. Jonassen
(Ed.), Handbook of research for educational communications and technology: A project of the
Association for Educational Communications and Technology (2nd) (pp. 143–168). New
Jersey: Lawrence Erlbaum Associates, Inc.
Kolovou, A., Van den Heuvel-Panhuizen, M., & Bakker, A. (2009). Non-routine
problem solving tasks in primary school mathematics textbooks – A Needle in a
Haystack. Mediterranean Journal for Research in Mathematics Education, 8(2),
31–68.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for
categorical data. Biometrics, 33(1), 159–174.
Lave, J. (1988). Cognition in practice: Mind, mathematics, and culture in everyday life. New York:
Cambridge University Press.
Leinhardt, G., Zaslavsky, O., & Stein, M. K. (1990). Functions, graphs, and graphing:
Tasks, learning, and teaching. Review of Educational Research, 60(1), 1–64.
Liang, X. (2010). Assessment use, self-efficacy and mathematics achievement:
Comparative analysis of PISA 2003. Data of Finland, Canada and the USA.
Evaluation & Research in Education, 23(3), 213–229.
Lumley, T. (2004). Analysis of complex survey samples. Journal of Statistical Software,
9(1), 1–19.
Lumley, T. (2012). Survey: Analysis of complex survey samples. R Package version 3.28-1.
Maass, K. (2010). Classification scheme for modelling tasks. Journal für Mathematik-
Didaktik, 31(2), 285–311.
Mardapi, D., & Kartowagiran, B. (2009). Dampak Ujian Nasional. Yogyakarta:
Universitas Negeri Yogyakarta.

49

Thesis Ari_141201_FINAL.pdf 49 1-12-2014 16:26:23

 Chapter 2

Newman, M. A. (1977). An analysis of sixth-grade pupils’ errors on written

mathematical tasks. Victorian Institute for Educational Research Bulletin, 39, 31–43.
Newman, M. A. (1983). Strategies for diagnosis and remediation. Sydney: Harcourt Brace
Jovanovich.
Nunes, T., Schliemann, A. D., & Carraher, D. W. (1993). Street mathematics and school
mathematics. New York: Cambridge University Press.
Nunez, R. E., Edwards, L. D., & Matos, J. F. (1999). Embodied cognition as
grounding for situatedness and context in mathematics education. Educational
Studies in Mathematics, 39(1–3), 45–65.
OECD. (2003a). Literacy skills for the world of tomorrow. Further results from PISA 2000.
Paris: Author.
OECD. (2003b). The PISA 2003 assessment framework – Mathematics, reading, science, and
problem solving knowledge and skills. Paris: Author.
OECD. (2004). Learning for tomorrow’s world. First results from PISA 2003. Paris: Author.
OECD. (2005). PISA 2003. Technical report. Paris: Author.
OECD. (2007). PISA 2006: Science competencies for tomorrow’s world. Paris: Author.
OECD. (2009a). Learning mathematics for life. A view perspective from PISA. Paris: Author.
OECD. (2009b). Take the test. Sample questions from OECD’s PISA assessments. Paris:
Author.
OECD. (2010). PISA 2009 results: What students know and can do. Student performance in
reading, mathematics, and science (Vol. I). Paris: Author.
Pusat Kurikulum. (2003). Kurikulum 2004. Standar Kompetensi mata pelajaran matematika
Sekolah Menengah Pertama dan Madrasah Tsanawiyah. Jakarta: Departemen
Pendidikan Nasional.
Roth, W.-M. (1996). Where is the context in contextual word problems? Mathematical
practices and products in grade 8 students’ answers to story problems. Cognition
and Instruction, 14(4), 487–527.
Schleicher, A. (2007). Can competencies assessed by PISA be considered the
fundamental school knowledge 15-year-olds should possess? Educational Change,
8(4), 349–357.
Schwarzkopf, R. (2007). Elementary modelling in mathematics lessons: The interplay
between “real-world” knowledge and “mathematical structures”. In W. Blum, P.
L. Galbraith, H.-W. Henn, & M. Niss (Eds.), Modelling and applications in mathematics
education (pp. 209–216). New York: Springer.
Van den Heuvel-Panhuizen, M. (1996). Assessment and Realistic Mathematics Education.
Utrecht: CD-Ƣ Press, Utrecht University.

50

Thesis Ari_141201_FINAL.pdf 50 1-12-2014 16:26:23

 Context-based tasks: Students’ difficulties

Van den Heuvel-Panhuizen, M. (2005). The role of context in assessment problems in

mathematics. For the Learning of Mathematics, 25(2), 2–9, and 23.
Warm, T. A. (1989). Weighted likelihood estimation of ability in item response theory.
Psychometrika, 54(3), 427–450.
Wu, M. L., Adams, R. J., Wilson, M. R., & Haldane, S. (2007). ACER conquest version
2.0. Mulgrave: ACER Press.
Yore, L. D., Anderson, J. O., & Hung Chiu, M. (2010). Moving PISA results into the
policy arena: Perspectives on knowledge transfer for future considerations and
preparations. International Journal of Science and Mathematics Education, 8(3), 593–603.
Yusuf, S. (2004). Analisis tes PISA. Literasi membaca, matematika, dan sains. Jakarta: Pusat
Penelitian Pendidikan. Departemen Pendidikan Nasional.

51

Thesis Ari_141201_FINAL.pdf 51 1-12-2014 16:26:23

 Chapter 2

52

Thesis Ari_141201_FINAL.pdf 52 1-12-2014 16:26:23

 Chapter 3

Opportunity-to-learn context-based tasks provided by

mathematics textbooks

Wijaya, A., Van den Heuvel-Panhuizen, M., & Doorman, M.. Opportunity-to-learn
context-based tasks provided by mathematics textbooks. Will be published in
Educational Studies in Mathematics.

Thesis Ari_141201_FINAL.pdf 53 1-12-2014 16:26:23

 Chapter 3

Opportunity-to-learn context-based tasks provided by

mathematics textbooks

1 Introduction
Students’ ability to apply mathematics in various contexts in daily life is seen as a
core goal of mathematics education (e.g., Boaler, 1993; De Lange, 2003;
Graumann, 2011; Muller & Burkhardt, 2007; Niss, Blum, & Galbraith, 2007). This
core goal of mathematics education is also reflected in the Programme for
International Student Assessment (PISA) (OECD, 2003b) in which the assessment
of students’ mathematics achievement focuses on students’ ability to solve
mathematics problems situated in real-world contexts. According to the PISA
Framework (OECD, 2003b), such ability should be an educational core goal
because today and in the future every country needs mathematically literate citizens
to deal with complex everyday surroundings and rapidly changing professional
environments.

Contexts from daily life can also be used as a didactical tool to support the learning
of mathematics. Students’ experiences with these contexts give a meaningful basis
to the mathematical concepts they have to learn (Cooper & Harries, 2002; Van den
Heuvel-Panhuizen, 1996). Furthermore, contexts can provide context-connected
solution strategies (Van den Heuvel-Panhuizen, 1996, 2005). However, this does
not mean that context-based tasks are always easy to solve for students. Several
studies revealed that many students have low performance on such tasks (e.g.,
Clements, 1980; Cummins, Kintsch, Reusser, & Weimer, 1988;Schwarzkopf, 2007;
Verschaffel, Greer, & De Corte, 2000). When solving context-based tasks, students
have difficulties in (1) understanding what a problem is about (Bernardo, 1999;
Cummins et al., 1988), (2) distinguishing between relevant and irrelevant
information (Cummins et al., 1988; Verschaffel et al., 2000), and (3) identifying the
mathematical procedures required to solve a problem (Clements, 1980; Verschaffel
et al., 2000).

As shown by the PISA surveys (OECD, 2003a, 2004, 2007, 2010), one country in
which students have low performance on context-based mathematics tasks is
Indonesia. For example, in PISA 2009 (OECD, 2010) only one third of Indonesian
students could answer mathematics tasks embedded in familiar contexts and less
than 1% of the students could work with context-based tasks in complex situations
which require well-developed thinking and reasoning skills. Furthermore, in a

54

Thesis Ari_141201_FINAL.pdf 54 1-12-2014 16:26:23

 Opportunity-to-learn provided by textbooks

recent study (Wijaya, Van den Heuvel-Panhuizen, Doorman, & Robitzsch, 2014) –
which was the first study of the Context-based Mathematics Tasks Indonesia
(CoMTI) project – we found that the errors of Indonesian students when solving
context-based tasks were mostly related to comprehending the tasks, particularly to
selecting relevant information. Many mistakes were also made in choosing the
correct mathematical operation or concept when transforming a context-based
task into a mathematical problem.

The present study was set up to find an explanation for the difficulties Indonesian
students experience when solving context-based tasks. The approach taken in the
study was investigating what opportunity-to-learn Indonesian mathematics
textbooks offer Indonesian students to develop the ability to solve context-based
tasks. Although the study was situated in Indonesia, we think it is relevant for an
international audience because it contributes to existing knowledge about the
relation between textbooks’ content and what students learn. Moreover, this study
brings in a new aspect of this relation by focusing on the difficulties students
experience when solving context-based tasks.

2 Theoretical background and research questions

2.1 The concept of opportuity-to-learn

A plausible question when particular educational goals are not achieved by
students is whether they have received the education enabling them to reach the
competences expressed in these goals. Therefore, it is no wonder that the concept
of opportunity-to-learn (OTL) came into being. Fifty years ago this concept was
coined by Carroll (1963) when referring to sufficient time for students to learn
(Liu, 2009). OTL was also introduced to ensure the validity of international
comparative studies of students’ achievement. Researchers became aware that
when comparing the achievement of students from different countries curricular
differences across national systems had to be taken into account (Liu, 2009;
McDonnell, 1995). In the report of the First International Mathematics Studies
(Husén, 1967), OTL was defined as “whether or not […] students have had the
opportunity to study a particular topic or learn how to solve a particular type of
problem” (pp. 162-163).

55

Thesis Ari_141201_FINAL.pdf 55 1-12-2014 16:26:23

 Chapter 3

2.2 Assessing opportunity-to-learn

To examine OTL several approaches are possible for which various aspects of
OTL can be assessed. Liu (2009) considered four OTL variables: (a) content
coverage, that is the match between the curriculum taught and the content tested,
(b) content exposure, that is the time spent on the content tested, (c) content
emphasis, that is the emphasis the teachers have placed on the content tested, and
(d) quality of instructional delivery, that is the adequacy of teaching the content.
Another approach was offered by Brewer and Stasz (1996) who distinguished three
categories of concern when assessing OTL. The first category is the curriculum
content, implying the assessment of whether the students have been taught the
subjects and topics that are essential to attain the standards. The second category
includes the instructional strategies, assessing whether students have experience
with particular kinds of tasks and solution processes. The third category refers to
the instructional resources. Here are assessed, for example, issues of teacher
preparation and quality of instructional materials.

Besides different aspects of OTL that can be assessed, different methods can also
reveal the OTL students receive. These methods vary from teacher and student
surveys through questionnaires, to carrying out classroom observations, and to
analyzing instructional materials. For example, the first measurements of OTL are
based on questionnaires in which teachers have to indicate whether particular
mathematical topics or kinds of problems have been taught to students. Such
questionnaires were used in the international comparative studies FIMS, SIMS and
TIMSS (Floden, 2002). Furthermore, the TIMSS video studies are an example of
revealing the OTL based on classroom observations (Hiebert et al., 2003). Another
approach applied in TIMSS was to look at what content curricula offer. Schmidt et
al. (1997, p. 4) see the curriculum – and in connection with this – textbook series
as “a kind of underlying ‘skeleton’ that gives characteristic shape and direction to
mathematics instruction in educational systems around the world” and that
provides “a basic outline of planned and sequenced educational opportunities.”

2.3 Opportunity-to-learn and the role of textbooks

Compared to the influence of curricula, textbooks play an even more direct role in
what is addressed in instruction. Teachers’ decisions about the selection of content
and teaching strategies are often directly set by the textbooks teachers use
(Freeman & Porter, 1989; Reys, Reys, &Chavez, 2004). Therefore, textbooks are
considered to determine largely the degree of students’ OTL (Schmidt et al., 1997;

56

Thesis Ari_141201_FINAL.pdf 56 1-12-2014 16:26:23

 Opportunity-to-learn provided by textbooks

Tornroos, 2005). This means that if textbooks differ students will get a different
OTL (Haggarty & Pepin, 2002). As a result, different student outcomes will
appear, which is confirmed by several studies which found a strong relation
between the textbook used and the mathematics performance of the students (see,
e.g. Tornroos, 2005; Xin, 2007).

The recognition of the role of textbooks in mathematics learning in students’

chances to be taught particular mathematical topics and skills has recently led to a
large amount of studies examining OTL offered in textbooks. For example,
textbook analysis was applied to the distributive property (Ding & Li, 2010), the
equal sign (Li, Ding, Capraro, & Capraro, 2008; McNeil et al., 2006), fractions
(Charalambous, Delaney, Hsu, & Mesa, 2010), subtraction up to 100 (Van Zanten
& Van den Heuvel-Panhuizen, 2014), non-routine problem solving (Kolovou, Van
den Heuvel-Panhuizen, & Bakker, 2009; Xin, 2007), and mathematical modeling
(Gatabi, Stacey, & Gooya, 2012).

2.4 Textbook analysis to identify the opportunity-to-learn

To disclose what content is intended to be taught to students, textbooks can be
analyzed in several ways. In TIMSS, textbook analysis initially focused on
investigating the content profiles of textbooks (Schmidt et al., 1997). Later, they
were examined based on five measures (Valverde et al., 2002). The first measure is
the classroom activities proposed by the textbook. The second measure
corresponds to the amount of content covered in textbooks and the mode of
presentation, whether abstract or concrete. The third measure deals with the
sequencing of content. The fourth measure focuses on physical characteristics of
textbooks, for example the size of the book and the number of pages. The fifth
measure characterizes the complexity of the demands for student performance.

Another approach to textbook analysis was proposed by Pepin and Haggarty

(2001). They distinguished four areas on which a textbook analysis can focus,
namely (a) the mathematics topics presented in textbooks and the beliefs about the
nature of mathematics that underlie the textbooks’ content, (b) the methods
suggested in textbooks to help students understand the textbooks’ content, (c) the
sociological contexts of textbooks which examines whether textbooks are adaptive
to students with different performance levels, and (d) the cultural traditions in
textbooks, focusing on how textbooks reflect the cultural traditions and values.

57

Thesis Ari_141201_FINAL.pdf 57 1-12-2014 16:26:23

 Chapter 3

Charalambous et al. (2010) classified the approaches to textbook analysis in three

categories, namely horizontal, vertical, and contextual. The horizontal analysis
examines the general characteristics of textbooks, such as physical characteristics
and the organization of the textbooks’ content. This analysis gives a first
impression of the OTL because it can provide information about the quantity of
exposure of textbooks’ content. However, information about the quality and the
didactical aspects of the textbooks’ content is not revealed by a horizontal analysis.
Therefore, a vertical analysis is needed to address how textbooks present and treat
the content. Such an analysis offers an in-depth understanding of the mathematical
content. The third category, the contextual analysis, focuses on how textbooks are
used in instructional activities. Therefore, Charalambous et al. (2010) argued that,
in fact, only the first two categories are appropriate to analyze the characteristics of
textbooks.

2.5 Opportunity-to-learn required for solving context-based tasks

The primary requirement for students’ learning to solve context-based tasks is that
students should be offered experiences to deal with essential characteristics of
context-based tasks and should be given the necessary practice in handling these
characteristics.

2.5.1 Nature of the context

A critical characteristic of context-based tasks is the nature of the context.
Concerning mathematical problems, there are several views on what a context
means (De Lange, 1995; OECD, 2003b; Van den Heuvel-Panhuizen, 2005). In the
present study the focus is on real-world contexts, which in PISA are called ‘extra-
mathematical contexts’ (OECD, 2003b). In PISA context-based tasks are defined
as problems presented within a ‘situation’ which can refer to a real-world or
fantasy setting, can be imagined by students, and can include personal,
occupational, scientific and public information. This interpretation of a context
matches what De Lange (1995) called a ‘relevant and essential context’, which he
contrasted with a ‘camouflage context’. Tasks with the latter context are merely
dressed-up bare problems, which do not require modeling because the
mathematical operations needed to solve the task are obvious.

2.5.2 Mathematization and modeling process

To solve tasks which include relevant and essential contexts, students need to
transform the context situation into a mathematical form through the process of

58

Thesis Ari_141201_FINAL.pdf 58 1-12-2014 16:26:23

 Opportunity-to-learn provided by textbooks

mathematization (OECD, 2003b). Therefore, it is important that context-based

tasks use setting or situation that gives access to and supports the process of
mathematization. In other words, it is crucial that the tasks provide information
that can be organized mathematically and offer opportunities for students to use
their knowledge and experiences (Van den Heuvel-Panhuizen, 2005).

The process of solving context-based tasks requires interplay between the real
world and mathematics (Schwarzkopf, 2007) and is often described as a modeling
process (Blum & Leiss, 2007), which in general contains the following steps (1)
understanding the problem situated in reality; (2) transforming the real-world
problem into a mathematical problem; (3) solving the mathematical problem; and
(4) interpreting the mathematical solution in terms of the real situation.

2.5.3 Adequate mathematical procedures

A further characteristic of a context-based task is that the task cannot be solved by
simply translating the task into a mathematical procedure (Verschaffel, Dooren,
Greer, & Mukhopadhyay, 2010). This means that standard problems which have a
straightforward relation between the problem context and the necessary
mathematics do not help students building up experience to transform a real-world
problem into a mathematical problem. Therefore, students should be set tasks in
which the necessary mathematical procedures are more implicit.

2.5.4 Different information types

Solving a context-based task is not just combining all the information given in the
task (Verschaffel et al., 2010). A context-based task may contain more information
than needed for solving the problem or may even lack necessary information.
Providing more or less information than needed for solving a context-based task is
a way to encourage students to consider the context used in the task and not just
take the numbers out of the context and process them mathematically in an
automatic way (Maass, 2007). Therefore, students should be offered opportunities
to deal with different types of information such as matching, missing, and
superfluous information (Maass, 2010). In this way they can learn to select relevant
information, and to add and ignore information.

2.5.5 Cognitive demands

A final requirement to support students’ learning to solve context-based tasks is
that students can build experience with tasks covering the full range of levels of
cognitive demands, including reproduction, connection, and reflection tasks

59

Thesis Ari_141201_FINAL.pdf 59 1-12-2014 16:26:23

 Chapter 3

(OECD, 2009a). Reproduction tasks require the recall of mathematical properties

and the application of routine procedures or standard algorithms. Such tasks do
not require mathematical modeling. Connection tasks require integrating and
linking different mathematical curriculum strands or different representations of a
problem. These tasks also require interpreting a problem situation and engaging
students in simple mathematical reasoning. Reflection tasks include complex
problem situations in which it is not obvious in advance what mathematical
procedures have to be carried out. In fact, this latter category of tasks is the closest
to our definition of context-based tasks. What competences students will
eventually master depends on the cognitive demands of mathematics tasks they
have been engaged in (Stein &Smith, 1998), therefore these reflection tasks should
not be lacking in instruction.

2.5.6 Crucial aspects of opportunity-to-learn context-based tasks

In sum, three aspects of OTL are crucial to develop the competence of solving
context-based tasks. The first aspect is giving students experience to work on tasks
with real-world contexts and implicit mathematical procedures. The second aspect
is giving students tasks with missing or superfluous information. The last aspect is
offering students experience to work on tasks with high cognitive demands.

2.6 Difficulties of Indonesian students in solving context-based tasks

The aforementioned aspects of OTL also emerged as relevant in the first study of
the CoMTI project (Wijaya et al., 2014) in which we investigated Indonesian
students’ difficulties when solving context-based tasks. In the study, which
involved a total of 362 Indonesian students (233 ninth graders and 129 tenth
graders), four types of students’ errors were identified, that is comprehension,
transformation, mathematical processing, and encoding errors. Comprehension
errors correspond to students’ inability to understand a context-based task,
including the inability to select relevant information. Transformation errors are
related to students’ inability to identify the correct mathematical procedure to solve
a problem. The mathematical processing errors refer to mistakes in carrying out
mathematical procedures. Encoding errors refer to answers that are unrealistic and
do not fit the real-world situation described in the task.

Table 1 shows that comprehension and transformation errors were the most
dominant errors made by the Indonesian ninth- and tenth-graders when solving
context-based mathematics tasks. Within the former category most errors were

60

Thesis Ari_141201_FINAL.pdf 60 1-12-2014 16:26:23

 Opportunity-to-learn provided by textbooks

made in selecting the relevant information; whereas in the latter category the
students mostly used wrong procedures.

Table 1.
Frequency of students’ error when solving context-based mathematics tasks (Wijaya et al., 2014)
Type of error Frequency Sub-type of error Frequency
(total errors =
1718)
Comprehension 38% - Misunderstanding the instruction 35%
- Misunderstanding a keyword 15%
- Error in selecting relevant information 50%
Total comprehension error = 653 100%
Transformation 42% - Using a common mathematical 12%
procedure that does not apply to the
problem situation
- Taking too much account of the 8%
context
- Treating a graph as a picture 12%
- Otherwise using wrong mathematical 68%
procedure
Total transformation error = 723 100%
Mathematical 17% --a
processing
Total mathematical processing error =
291
Encoding 3% --b
Total encoding error = 51
a The sub-categories of mathematical processing error are task-specific.
b There is no sub-type for the encoding error.

In addition, in agreement with the results of the PISA study 2003 (OECD, 2009a),
we found in our study that the reproduction tasks were the easiest for the students
(67% of the responses to these tasks got a full credit), whereas the tasks with
higher cognitive demand, the connection and the reflection tasks, had lower
percentages of correct answers (in both types of tasks 39% of the responses got a
full credit).

2.7 Research questions

The purpose of the present study was to disclose what OTL Indonesian
mathematics textbooks offer to Indonesian students for developing the ability to
solve context-based tasks. Based on the literature review the focus was on four

61

Thesis Ari_141201_FINAL.pdf 61 1-12-2014 16:26:23

 Chapter 3

aspects of OTL: the exposure to context-based tasks, the purpose of the context-
based tasks, the type of information provided in tasks, and the type of cognitive
demand required by tasks. Therefore, the following research questions were
addressed:

1. What are the amount of exposure to and the purpose of context-based tasks in
Indonesian mathematics textbooks?
2. To what extent are different types of information provided in tasks in Indonesian
mathematics textbooks?
3. What are the cognitive demands of tasks in Indonesian mathematics textbooks?

The reason for this study was to find an explanation for the difficulties Indonesian
students experience when solving context-based tasks. Therefore, we also
investigated the connection between students’ difficulties when solving context-
based tasks with the OTL in Indonesian mathematics textbooks. This resulted in
the next research question:

4. What is the connection between students’ errors when solving context-based tasks
with the characteristics of tasks in Indonesian mathematics textbooks?

3 Method

3.1 Mathematics textbooks analyzed

To answer the research questions we carried out a textbook analysis in which we
focused on grade 8. Although according to the National Curriculum (Pusat
Kurikulum, 2003a, 2003b), the topics dealt with in the PISA tasks included in the
CoMTI test are taught from grade 7 to 10, the main emphasis on these topics lies
in grade 8. Of the topics, almost half are taught in grade 8 and the remaining part
is distributed over the three other grades. Moreover, grade 8 can be considered as a
relevant grade year to prepare students for being able to solve context-based tasks
as assessed in the PISA studies.

62

Thesis Ari_141201_FINAL.pdf 62 1-12-2014 16:26:23

 Opportunity-to-learn provided by textbooks

The mathematics textbooks chosen for this analysis are shown in Table 2. These
are the textbook series that were used in the schools involved in the first study of
the CoMTI project (Wijaya et al., 2014). All the schools either used a combination
of two of these textbook series or used all three. Of the textbook series MJHS,
which is bilingual, we only analyzed the Indonesian pages.

Table 2.
Analyzed textbooks and material
Textbook series Abbreviation Material involved in Publisher
analysis
Matematika: Konsep MKA “For Junior High School Indonesian Ministry of
dan Aplikasinya grade VIII” National Education

Matematika MS “For Junior High School Private publisher

grade VIII: 2A & 2B”

Mathematics for MJHS “Part 2” Private publisher

Junior High School

The three textbooks have a similar main structure. All have nine chapters, each
dealing with one mathematics topic. These chapters contain several sub-chapters
discussing specific aspects of the mathematics topic covered in the chapter. For
example, the chapter on ‘Equations of straight lines’ contains the sub-chapters: (a)
the general form of equations of straight lines, (b) the gradient, (c) the relation
between a gradient and the equation of a straight line, and (d) the application of
equations of straight lines.

Each sub-chapter consists of an explanation section, followed by one or more

worked example sections, and a task section with regular tasks which the students
have to solve themselves. The explanation section discusses a particular
mathematics concept, for example, how a rule or formula is obtained (see
Figure 1). The worked example section contains one or more tasks for which an
answer is given (see Figure 2). This section serves as a bridge between the
explanation section and the task section (Figure 3).

Although the three textbooks have the same number of chapters, they have
different numbers of sub-chapters because they discuss the mathematics topics in
different levels of detail. For example, MJHS discusses the topic of ‘Equations of
straight lines’ in three sub-chapters, whereas the other textbooks spend four sub-
chapters on this topic.

63

Thesis Ari_141201_FINAL.pdf 63 1-12-2014 16:26:23

 Chapter 3

Figure 1. Explanation section (MS 2A, p. 86).

Figure 2. Worked example section (MS 2A, p. 87).

64

Thesis Ari_141201_FINAL.pdf 64 1-12-2014 16:26:23

 Opportunity-to-learn provided by textbooks

Figure 3. Task section (MS 2B, p. 88).

3.2 Procedure of textbook analysis

Following Charalambous et al. (2010), we analyzed the textbooks from two
perspectives, namely their physical characteristics and instructional components
(horizontal analysis) and the characteristics of the tasks (vertical analysis).The
physical characteristics and instructional components of the textbooks were
investigated to provide information about the quantity of exposure to textbook
content. We collected data about the page size, the number of pages, and the page
surface area. Furthermore, we counted the number of explanation sections, the
worked example sections, the task sections with regular tasks, the tasks in worked
example sections, and the regular tasks in task sections. In this analysis we
considered as a task every question or problem with the answer provided (tasks in
the worked example sections) or for which the students have to give an answer
(regular tasks in the task sections). For example, although the task section in
Figure 3 contains two task numbers (7 and 8), in our approach this section has five
tasks (7a, 7b, 8a, 8b, and 8c). The tasks in the worked example sections were
counted in a similar way; therefore Figure 2 contains three tasks (1a, 1b, and 2).
The tasks in the worked example sections and in the task sections could be in a
bare format (with only symbols) or context-based format.

65

Thesis Ari_141201_FINAL.pdf 65 1-12-2014 16:26:24

 Table 3

66
Analysis framework for textbook analysis

Thesis Ari_141201_FINAL.pdf 66
Chapter 3

Task characteristic Sub-category Explanation

Type of context No context - Refers only to mathematical objects, symbols, or structures.
Camouflage context - Experiences from everyday life or common sense reasoning are not needed.
- The mathematical operations needed to solve the problems are already obvious.
- The solution can be found by combining all numbers given in the text.
Relevant and essential - Common sense reasoning within the context is needed to understand and solve
context the problem.
- The mathematical operation is not explicitly given.
- Mathematical modeling is needed.

Purpose of Application - The task is given after the explanation section

context-based task Modeling - The task is given before the explanation section.

Type of information Matching - The tasks contain exactly the information needed to find the solution.
Missing - The tasks contain less information than needed so students need to find missing
information.
Superfluous - The tasks contain more information than needed so students need to select
information.

1-12-2014 16:26:24
 Task characteristic Sub-category Explanation

Thesis Ari_141201_FINAL.pdf 67
Type of cognitive Reproduction - Reproducing representations, definitions or facts.
demand - Interpreting simple and familiar representations.
- Memorization or performing explicit routine computations/procedures.
Connection - Integrating and connecting across content, situations or representations.
- Non-routine problem solving.
- Interpretation of problem situations and mathematical statements.
- Engaging in simple mathematical reasoning.
Reflection - Reflecting on, and gaining insight into, mathematics.
- Constructing original mathematical approaches.
- Communicating complex arguments and complex reasoning.
- Making generalizations.
Opportunity-to-learn provided by textbooks

67

1-12-2014 16:26:24
 Chapter 3

3.2.1 Framework for textbook analysis focusing on OTL context-based

tasks
The vertical analysis was meant to investigate the OTL to solve context-based
tasks. For this purpose an analysis framework (see Table 3) was developed
addressing the task characteristics: type of context, type of information, and type
of cognitive demand. To investigate the amount of exposure to context-based
tasks, first we identified the type of context used in the tasks in Indonesian
mathematics textbooks. We used the categories distinguished by De Lange (1995)
including ‘no context’, ‘camouflage context’, and ‘relevant and essential context’.
As an extension to the type of context, we also included the ‘purpose of context-
based task’. The reason for including this characteristic was to distinguish whether
a context-based task is used for applying mathematics or for mathematical
modeling (Muller & Burkhardt, 2007; Niss, Blum, & Galbraith, 2007). In the
former the solvers know the mathematics they should apply because the task is
given after an explanation section. In the latter, the solvers start with a real-world
problem and have to identify what mathematics is suitable to solve the problem.

For types of information, we used three types described by Maass (2010):

‘matching information’, ‘missing information’, and ‘superfluous information’. The
categories for the cognitive demands of the tasks were established based on PISA’s
competence clusters (OECD, 2003b) which included reproduction, connection,
and reflection tasks. These categories were used to identify the characteristics of
both the tasks in the worked example sections and in the task sections.

3.2.2 Coding procedure

All tasks in the three textbooks were coded by the first author using the analysis
framework as shown in Table 3. Afterwards the reliability of the coding was
checked through an additional coding by an external coder who coded a random
selection of about 15% of the tasks. This extra coding resulted in a Cohen’s Kappa
of .75 for the type of context, 1.00 for the purpose of the context-based task, .74
for the type of information, and .84 for the type of cognitive demands. These
results indicate the coding was reliable (Landis & Koch, 1977).

68

Thesis Ari_141201_FINAL.pdf 68 1-12-2014 16:26:24

 Opportunity-to-learn provided by textbooks

4 Results

4.1 Physical characteristics and instructional components of the

textbooks
As shown in Table 4 MS has the largest page number and page surface area;
whereas MJHS has the lowest number for these physical characteristics. MKA
takes a middle position and has a page number and page surface area close to the
average of mathematics textbooks from other countries. As reported by Valverde
et al. (2002), the international average number of pages for grade eight
mathematics textbooks is 225 pages and the international median of page surface
area is 115,000 cm2.

Table 4.
Physical characteristics and instructional components of Indonesian mathematics textbooks
Textbook
MJHSa MKA MS
Physical characteristic
Page size (in mm) 205 × 176 × 176 × 250
277 250
Number of pages 146 252 336
Page surface areab (in cm2) 82,906 110,880 147,840
Instructional components
Number of explanation sections 25 42 56
Number of worked example 80 73 90
sections
Number of task sections 79 63 68
Total number of tasks 437 531 1187
Number of tasks in worked 119 91 218
example sections
Number of tasks in task sections 318 440 969
a Only the Indonesian pages of this textbook were analyzed.

b Multiplication of the page number and the area of a page (Valverde et al., 2002).

Regarding instructional components, MS has the largest number of explanation

sections, which is more than double that in MJHS. The largest worked example
sections are also in MS, but for this instructional component the difference
between the three textbooks is small. For the task sections, MJHS has slightly
more than the two other textbooks. We found a large difference between the

69

Thesis Ari_141201_FINAL.pdf 69 1-12-2014 16:26:24

 Chapter 3

textbooks for the number of tasks. MS has a total of 1187 tasks, which is double
the number in MKA and triple that in MJHS. A similar ratio was found for the
regular tasks in the task sections, with respectively 318, 440, and 969 tasks for
MJHS, MKA, and MS.

4.2 The amount of exposure and the purpose of context-based tasks

In relation to Research question 1 it was found that only 8% to 16% of the tasks in
the three textbooks are context-based, with the highest proportion of these tasks in
MS (see Table 5). In MS the proportion of context-based tasks in the worked
example sections and the task sections is about the same, whereas in MJHS and
MKA context-based tasks are used more often in the task sections.

Table 5.

Frequency of types of context

Textbook
Type of context MJHS MKA MS
n % N % n %
Relevant and 1 1 2 2 10 5
Tasks in essential
worked context
example Camouflage context 4 3 1 1 28 13
sections No context 114 96 88 97 180 83
Total 119 100 91 100 218 101

Relevant and 7 2 17 4 29 3
Regular essential
tasks in context
task Camouflage context 24 8 26 6 127 13
sections No context 287 90 397 90 813 84
Total 318 100 440 100 813 100

Relevant and 8 2 19 4 39 3
essential
context
All tasks
Camouflage context 28 6 27 5 155 13
No context 401 92 485 91 993 84
Total 437 100 531 100 1187 100

70

Thesis Ari_141201_FINAL.pdf 70 1-12-2014 16:26:24

 Opportunity-to-learn provided by textbooks

In all three textbooks most of the contexts belong to the category of camouflage
context. Regarding the purpose of the context-based tasks, we found that all these
tasks were intended for application as indicated by their position after the
explanation sections.

Examples of each type of context are given in Figure 4, Figure 5, and Figure 6.
Both tasks in Figure 4 and Figure 5 are related to the concept of gradient. The task
in Figure 4 only uses mathematical objects and symbols.

Figure 4. Task with no context (MKA, p. 70).

The task in Figure 5 is set in the context of a ski slope, which is not a daily-life
situation for Indonesian students. This context is an example of a camouflage
context, because it can be neglected in solving the problem. Although the task
includes a real world situation, the photograph of the ski slope is cut and arranged
so that it exactly resembles a straight and the arrows informing the vertical and
horizontal differences also resemble a coordinate system. Furthermore, the words
‘slope’ and ‘gradient’ are mentioned explicitly and the numbers in the picture are
given in such a way that students can immediately interpret the problem as a
mathematical problem and follow the common procedure for calculating the slope
or gradient.

71

Thesis Ari_141201_FINAL.pdf 71 1-12-2014 16:26:24

 Chapter 3

Figure 5. Task with camouflage context (MS 2A, p. 79).

Figure 6. Task situated in relevant and essential context (MS 2A, p. 132).

A task with a relevant and essential context is shown in Figure 6. This task asks to
determine the price of four pairs of shoes and five pairs of sandals. To solve this
task students are expected to transform the ‘price problem’ into linear equations
with two variables. However, this mathematical procedure is not explicitly
mentioned in the task, nor are the numbers presented so that a solution procedure
is afforded. The variables for setting up the equations, that is, the number of pairs
of shoes and the number of pairs of sandals, are not explicitly indicated in the task.
Consequently, students need to identify the relevant information and a solution
strategy for solving the task.

72

Thesis Ari_141201_FINAL.pdf 72 1-12-2014 16:26:24

 Opportunity-to-learn provided by textbooks

4.3 Types of information provided in tasks

As an answer to Research question 2 we found that for the bare tasks , that is tasks
with no context, almost all (98%, 99%, and 99% of the bare tasks in MJHS, MKA,
and MS respectively) give students exactly the information needed to solve the
problems. However, for the context-based tasks the proportions of tasks with
matching information are lower (i.e. 86%, 85%, and 89% of the context-based
tasks in respectively MJHS, MKA, and MS) (see Table 6). The remaining tasks
have missing information (i.e., 14%, 15%, and 10% of the context-based tasks in
MJHS, MKA, and MS respectively), whereas, except one task in MS, no tasks with
superfluous information were found.

Table 6.
Frequency of types of information in context-based tasks
Textbook
Type of
MJHS MKA MS
information
n % n % n %
Context- Matching 4 80 2 67 35 92
based tasks in Missing 1 20 1 33 2 5
worked Superfluous 0 0 0 0 1 3
example Total 5 100 3 100 38 100
sections

Matching 27 77 37 86 138 88
Context-
Missing 4 23 6 14 18 12
based tasks in
Superfluous 0 0 0 0 0 0
task sections
Total 31 100 43 100 156 100

Matching 31 86 39 85 173 89
All context- Missing 5 14 7 15 20 10
based tasks Superfluous 0 0 0 0 1 1
Total 36 100 46 100 194 100

An example of a task with matching information is shown in Figure 7. This task

involves finding the width of a river. The width of the river is mathematically the
leg of a right-angled triangle which can be found by applying the Pythagorean
Theorem. All the information required to apply the Pythagorean Theorem, that is
the lengths of the hypotenuse and another leg, was given.

73

Thesis Ari_141201_FINAL.pdf 73 1-12-2014 16:26:24

 Chapter 3

Figure 7. Task with matching information (MJHS, p. 156).

In the task in Figure 8 the students are asked to find the minimum length of rope
needed to tie up the three pipes. This task is an example of a task with missing
information, in the sense that not all data are directly given to carry out a
mathematical procedure leading to the answer. In fact, for solving this task
students have to add the lengths of the three external tangents and the three arcs.
However, these lengths are not given. Thus the students must first generate this
data by using further knowledge that can be derived from the contextual situation,
such as the three arcs together constituting precisely a full circle and that one
tangent equals two radii, that is, equals the diameter of a pipe.

Figure 8. Task with missing information (MS 2B, p. 143).

74

Thesis Ari_141201_FINAL.pdf 74 1-12-2014 16:26:24

 Opportunity-to-learn provided by textbooks

The task in Figure 9 is an example of a task with superfluous information. The task
was about finding the height of a flashlight. The task provided the diameter of the
top circle and the bottom circle of the flashlight, and the shape and volume of the
flashlight box. The shape of the flashlight box was cuboid; therefore the diameter
of the bottom of the flashlight was not needed to solve the task. Noteworthy is
that we only found one task with superfluous information, which is in the worked
example section.

Figure 9. Task with superfluous information (MS 2B, p. 106).

4.4 Cognitive demands required for solving the tasks

Table 7 shows the answers to Research question 3. In all three textbooks almost all
bare tasks were identified as reproduction tasks (95%, 93%, and 93% of the bare
tasks in MJHS, MKA, and MS respectively).The three textbooks have a small
proportion of connection tasks, ranging from 5 to 7%. Only MS includes reflection
tasks, that is 1% of bare tasks.

75

Thesis Ari_141201_FINAL.pdf 75 1-12-2014 16:26:25

 Chapter 3

Table 7.
Frequency of types of cognitive demands of bare tasks
Textbook
Type of cognitive
MJHS MKA MS
demands
N % n % n %
Bare tasks in Reproduction 110 96 83 94 172 96
worked Connection 4 4 5 6 8 4
example Reflection 0 0 0 0 0 0
sections Total 114 100 88 100 180 100

Reproduction 271 96 370 93 756 93

Bare tasks in Connection 16 4 27 7 48 6
task sections Reflection 0 0 0 0 9 1
Total 287 100 397 100 813 100

Reproduction 381 95 453 93 928 93

All bare Connection 20 5 32 7 56 6
tasks Reflection 0 0 0 0 9 1
Total 401 100 485 100 993 100

Focusing only on the context-based tasks (see Table 8), including a relevant and
essential context or a camouflage context, the proportions of task types according
to their cognitive demand changed remarkably. The proportions of reproduction
tasks in the context-based tasks (47%, 33%, and 56% of the context-based tasks in
MJHS, MKA, and MS respectively) are much lower than in the bare tasks. In the
context-based tasks substantial proportions of connection tasks were found (50%,
67%, and 42% of the context-based tasks in MJHS, MKA, and MS respectively).
However, reflection tasks are still either a minority or absent. Only one such task
was found in MJHS and three in MS. Differences between the three textbooks
were also found for the balance in the type of tasks. MKA contains more
connection tasks than reproduction tasks, whereas MJHS and MS have about the
same proportion of reproduction and connection tasks.

76

Thesis Ari_141201_FINAL.pdf 76 1-12-2014 16:26:25

 Opportunity-to-learn provided by textbooks

Table 8.
Frequency of types of cognitive demands of context-based tasks
Textbook
Type of cognitive
MJHS MKA MS
demands
n % n % n %
Context- Reproduction 2 40 1 33 22 58
based tasks Connection 3 60 2 67 16 42
in worked Reflection 0 0 0 0 0 0
example Total 5 100 3 100 38 100
sections

Context- Reproduction 15 48 14 33 87 56
based tasks Connection 15 48 29 67 66 42
in task Reflection 1 3 0 0 3 2
sections Total 31 99 43 100 156 100

Reproduction 17 47 15 33 109 56
All context- Connection 18 50 31 67 82 42
based tasks Reflection 1 3 0 0 3 2
Total 36 100 46 100 194 100

Figure 4 and Figure 5 show examples of reproduction tasks. The word ‘gradient’
was explicitly mentioned in the tasks so students could easily identify the required
mathematics procedure and apply it by using all the information. Figure 8, the task
about the three pipes, is an example of a connection task situated in a camouflage
context. The main focus of this task was finding the minimum length of rope to tie
up three pipes, which mathematically was related to the concept of common
tangent of two circles. To find the minimum length of rope students need to find
not only the length of the three common tangents, but also the length of the arcs.
Here, a connection to the concept of an equilateral triangle is needed to find the
measure of the central angle required to calculate the length of the arcs.

A connection task could also be assigned to a bare task, for example the task in
Figure 10. This task was about determining the perimeter and the area of a non-
regular shape. Connecting across representations, that is, circles with different sizes
was required to solve this task.

77

Thesis Ari_141201_FINAL.pdf 77 1-12-2014 16:26:25

 Chapter 3

Figure 10. Connection task situated in no context (MKA, p. 168).

Figure 11 shows an example of a reflection task in a relevant and essential context.

The task involved gaining insight into the mathematical meaning of ‘horizontal
floor’. Although the figures of triangles and their measures are provided, the
mathematical concept related to ‘horizontal floor’ was not explicitly given. Here,
students needed to identify that the task was related to the Pythagorean Theorem.

Figure 11. Reflection task situated in a relevant and essential context (MS 2A, p. 151).

4.5 Opportunity-to-learn provided in textbooks and students’ errors

To answer Research question 4 we related the results from the textbook analysis to
the errors students made when solving context-based tasks, as found in our first
study in the CoMTI project (Wijaya et al., 2014). Because we only knew that the
group of ninth-graders used the textbooks involved in our analysis, we decided to
focus on the results from the ninth-graders and exclude the tenth-graders.
Moreover, because the schools in our first study used different combinations of
textbooks, we could not exactly classify students’ errors according to the particular
textbooks with which they were taught. Therefore, in Table 9 we included the total
of errors over all schools and for each textbook the proportions of tasks that relate
to these errors.

78

Thesis Ari_141201_FINAL.pdf 78 1-12-2014 16:26:25

 Opportunity-to-learn provided by textbooks

Table 9.
Relation between students’ errors and task characteristics in textbooks
Ninth-graders’ errors Task characteristic Textbook
found in Wijaya et al. MJHS MKA MS
(2014) (934 errors)
Proportion of tasks (%)
Comprehension Type of Matching 86 85 89
errors: information in
Proportion of errors context-based
in selecting relevant tasks Missing 14 15 10
information:
21% of all errors Superfluous 0 0 1

Transformation Type of context No context 92 91 84

errors: in all tasks
Proportion of these Camouflage 6 5 13
errors: context
45% of all errors Relevant and 2 4 3
essential
context
Percentage of wrong Type of Reproduction 47 33 56
answers for each type cognitive
of cognitive demand demand in
- Reproduction: 32% Connection 50 67 42
context-based
- Connection: 61% tasks
- Reflection: 61% Reflection 3 0 2

Combining the findings from the textbook analysis and the error analysis, a
recognizable similarity between these two findings emerged. In 21% of the total of
934 errors (made by 233 ninth-graders) students made comprehension errors by
not selecting the relevant information for solving the tasks. Correspondingly, the
textbook analysis revealed that the textbooks mainly provide context-based tasks
with matching information, and only 11% to 15% of the context-based tasks had
missing or superfluous information.

In relation to the high proportion of transformation errors, the textbook analysis

disclosed that the proportions of context-based tasks in the three textbooks only
range from 8% to 16%. It was also found that only 2% to 4% of these tasks use
relevant and essential contexts. Furthermore, all context-based tasks are located
after the explanation sections in which a particular mathematics topic is discussed.

79

Thesis Ari_141201_FINAL.pdf 79 1-12-2014 16:26:25

 Chapter 3

This means these tasks are just meant to apply what has been demonstrated in the
explanation section. Thus, from the textbooks’ content, students can scarcely build
up experience in constructing a mathematical model by mathematizing a real-world
situation.

Lastly, regarding the percentage of wrong answers for the context-based tasks of
the various types of cognitive demand we also found a match with what is offered
in the textbooks. The lowest percentage of wrong answers was obtained for the
reproduction tasks which cover 33% to 56% of the context-based tasks in the
three textbooks. For the connections tasks, which form 42% to 67% of the
context-based tasks, about half of the answers were wrong. The largest percentage
of wrong answers belonged to the reflection tasks which include only 0% to 3% of
the context-based tasks in the textbooks.

5 Discussion
As the PISA studies (OECD, 2003a, 2004, 2007, 2010) have shown and our first
CoMTI study (Wijaya et al., 2014) has confirmed, Indonesian students have
difficulties in solving context-based tasks. The present study was meant to disclose
a possible reason for these difficulties by conducting an analysis of three
Indonesian mathematics textbooks. This analysis focused on the OTL to solve
context-based tasks offered by the textbooks. For identifying the characteristics of
the tasks in the textbooks we developed an analysis framework including four
perspectives: the types of contexts in tasks, the purpose of the context-based tasks,
the information used in tasks, and the types of cognitive demands in tasks.

5.1 Opportunity-to-learn and students’ difficulties

Our analysis revealed that context-based tasks are hardly available in Indonesian
mathematics textbooks. The textbooks mostly provide tasks without a context,
which do not require mathematization or modeling activities from the students.
Furthermore, the few context-based tasks in the textbooks do mostly not have a
real relevant and essential context. In addition, the context-based tasks in the
Indonesian textbooks are all located after the explanation sections. This means the
mathematics procedure to be applied is more or less given and students do not
have to identify an appropriate mathematics procedure to solve the tasks; and
consequently not getting enough experience to develop their ability to transform a
context-based task into a mathematical problem. This lack of experience is a

80

Thesis Ari_141201_FINAL.pdf 80 1-12-2014 16:26:26

 Opportunity-to-learn provided by textbooks

plausible explanation for the high number of transformation errors made by

Indonesian students. The foregoing conclusion is in agreement with several studies
which showed that students’ lack of experience in a particular type of task
corresponds to their difficulties with the task. For example, Haines and Crouch
(2007) mentioned that particular difficulties of students in mathematical modeling
are due to unfamiliarity with tasks in which students have to identify what
mathematics is appropriate to solve the problem. Similarly, Stein and Smith (1998)
reported that students’ lack of prior experience with open-ended tasks leads to
difficulties when solving tasks in which the mathematical procedure is implicit. A
lack of particular experience is also directly related to a missing OTL in textbooks
as shown in the studies by Li et al. (2008) and McNeil et al. (2006) which revealed
that students have difficulties in interpreting the equal sign as a relation because
the textbooks they use rarely provide equal signs with operations on both sides.

A specific characteristic of context-based tasks that we found missing in the three

Indonesian textbooks is the use of incomplete or irrelevant information, which,
according to Forman and Steen (2001), and Greer, Verschaffel, and
Mukhopadhyay (2007), is crucial for developing students’ ability to apply
mathematics in real-world problems. Of the 276 context-based tasks in the three
textbooks, only 32 have missing information and just one task contains
superfluous information. This means Indonesian students who worked with these
textbooks could not really build up experience in selecting relevant information or
using knowledge of the context to add missing information. Moreover, taking the
freedom to include one’s own knowledge or to neglect given information is
something with which the students should be familiar. Chapman (2006)
emphasized that when students are encouraged to use their own real-world
experiences and to relate school experiences to life outside school, they will
consider contextual information in a task as important to comprehend and solve
the task. The lack of tasks which give students such experiences could explain why
the students made so many comprehension errors.

The substantial number of errors made by Indonesian students in tasks with high
cognitive demands can also be traced back to the content of the Indonesian
textbooks. Of the context-based tasks, less than 3% were reflection tasks.
However, the connection tasks were found in about half the context-based tasks.
This proportion is similar to the proportion of connection tasks in the PISA study
2003 (OECD, 2009a), which might give an impression that Indonesian students
have OTL to solve these tasks. However, we should take into account that the

81

Thesis Ari_141201_FINAL.pdf 81 1-12-2014 16:26:26

 Chapter 3

context-based connection tasks in Indonesian textbooks are low in number. Over

the three textbooks, only 131 context-based tasks (see Table 8; MJHS: 18; MKA:
31; MS: 82) out of all the 2155 tasks together (see Table 4; MJHS: 437; MKA: 531;
MS: 1187) ask for integrating and connecting different mathematical curriculum
strands, or linking different representations of a problem.

In sum, the results from our analysis of three Indonesian textbooks provide
evidence of a relation between the errors Indonesian students make when solving
context-based tasks and the content offered by the textbooks they use. This
conclusion adds to earlier studies which showed a positive relation between OTL
provided in textbooks and student achievement: the students learn what is
“taught” by the textbook. For example, Tornroos (2005) found a high correlation
between student achievement in a test and the amount of textbook content related
to the test items. Also, Xin (2007) revealed that the algorithmic strategy used by
Chinese students to solve word problem tasks was the strategy suggested in their
textbooks.

5.2 Educational implications

Based on our findings, we recommend including more context-based tasks in
textbooks. Moreover, these tasks should not only be given after an explanation
section, because then the mathematical procedure to be chosen is more or less
fixed (see also De Lange, 2003). The quality of the context-based tasks should also
be of concern. Textbooks should include context-based tasks which offer students
opportunities for mathematization (Freudenthal, 1986; Van den Heuvel-
Panhuizen, 2005). This means that instead of camouflage contexts, relevant and
essential contexts should be used which demand mathematical organization or ask
for mathematization. The context-based tasks to be included in textbooks should
also have superfluous or missing information. Giving such tasks will provide
students not only OTL to select relevant information (Maass, 2010), but also to
identify appropriate mathematics procedures (Greer et al., 2007). Lastly, attention
should also be paid to the cognitive demands of context-based tasks. The
investigated textbooks contain too few reflection tasks to make it possible for
students to develop their ability in complex reasoning. Including more reflection
tasks is essential because they stimulate mathematical thinking and reasoning
related to authentic settings (OECD, 2003b).

Although this study was situated in Indonesia, the results of the study may also be
beneficial for other countries where students have a low performance in context-

82

Thesis Ari_141201_FINAL.pdf 82 1-12-2014 16:26:26

 Opportunity-to-learn provided by textbooks

based tasks. For such countries, our study gives strong indications for examining
the OTL to solve context-based tasks as offered in the textbooks in use in these
countries.

5.3 Limitations of the study and directions for future research

To be able to provide strong indications for the relation of OTL as offered in
textbooks and the achievements of students, it is necessary to know at least of
every student which textbook was used to teach him or her. Only then it is
possible to obtain a direct proof of this relation. However, in our study we did not
have a one-to-one link between students’ errors and the textbook the students
worked with because the schools involved in our study used a combination of
textbooks. Not knowing which textbook(s) each student used can be seen as a
limitation of our study; yet we found that in the three textbooks the number of
context-based tasks offered and the nature of these tasks were quite similar.
However, for getting more robust evidence for our findings, it would be necessary
to conduct a further study in which it is known for all students with which
textbook(s) they are taught. In addition, including more grades in a textbook
analysis would provide a better overview of the OTL in textbooks and the relation
with students’ achievements. Moreover, for the purpose of generalizability, our
study could be repeated in other countries where students have low performance
on, for example, context-based tasks used in the PISA studies.

A final limitation of our study is that the relation between student difficulties and
OTL was only investigated from the perspective of OTL in textbooks. Although
textbooks might be used as the main teaching and learning resources in the
classroom (see, e.g., Reys et al., 2004; Valverde et al., 2002), it is clear that teachers
also have an important role. For example, Pepin and Haggarty (2001) found that
how textbooks are used in classrooms is determined by the teachers. Teachers
determine the textbook sections used for students’ exercises, when the textbooks
should be used, and the ways students work with the textbooks. Regarding the use
of context-based tasks for modeling, Ikeda (2007) emphasized the important role
of teachers by arguing that the obstacle in teaching modeling is not only a lack of
modeling tasks in textbooks, but also teachers’ perceptions of mathematics and
understanding of modeling. As noted by Bishop (1988), teachers have a crucial role
in integrating students’ experiences and cultures in mathematics learning.
Furthermore, following Lampert (1990) and Boaler (1993), if teachers assume that
mathematics is a static body of knowledge and learning is the reproduction of

83

Thesis Ari_141201_FINAL.pdf 83 1-12-2014 16:26:26

 Chapter 3

facts, procedures and truths, teachers will fail to engage students with context-
based problems or problems with missing or superfluous information. Considering
these facts, we believe that revealing possible reasons for students’ difficulties in
solving context-based tasks should also include investigations into teachers’
teaching practice and teachers’ beliefs and knowledge about context-based tasks.
Consequently, the OTL offered by teachers will be the next focus of the CoMTI
project. Another issue that also needs further attention is the influence of the
classroom culture on the OTL offered to students (see Pepin & Haggerty, 2001).

Acknowledgements
This research was supported by the Indonesian Ministry of National Education
under the Better Education through Reformed Management and Universal Teacher
Upgrading (BERMUTU) Scholarship (No. 7476-IND Grant Ref. TF090784).

References
Bernardo, A. B. I. (1999). Overcoming obstacles to understanding and solving word
problems in mathematics. Educational Psychology, 19(2), 149-163.
Bishop, A. J. (1988). Mathematics education in its cultural context. Educational Studies in
Mathematics, 19(2), 179-191.
Blum, W., & Leiss, D. (2007). How do students and teachers deal with mathematical
modelling problems? The example “Sugarloaf”. In Haines, C., Galbraith, P., Blum,
W., & Khan, S., Mathematical modelling (ICTMA 12): Education, engineering and economics
(pp. 222-231). Chichester: Horwood Publishing.
Boaler, J. (1993). Encouraging the transfer of ‘school’ mathematics to the ‘Real world’
through the integration of process and content, context and culture. Educational
Studies in Mathematics, 25, 341-373.
Brewer, D. J., & Stasz, C. (1996). Enhancing opportunity to learn measures in NCES data.
Santa Monica, CA: RAND.
Carroll, J. (1963). A model of school learning. Teachers College Record, 64, 723-733.
Chapman, O. (2006). Classroom practices for context of mathematics word problems.
Educational studies in mathematics, 62(2), 211-230.
Charalambous, C. Y., Delaney, S., Hsu, H.-Y., & Mesa, V. (2010). A comparative
analysis of the addition and subtraction of fractions in textbooks from three
countries. Mathematical Thinking and Learning, 12(2), 117-151.
Clements, M. A. (1980). Analyzing children’s errors on written mathematical task.
Educational Studies in Mathematics, 11(1), 1-21.

84

Thesis Ari_141201_FINAL.pdf 84 1-12-2014 16:26:26

 Opportunity-to-learn provided by textbooks

Cooper, B., & Harries, T. (2002). Children’s responses to contrasting ‘realistic’

mathematics problems: Just how realistic are children ready to be? Educational
Studies in Mathematics, 49(1), 1-23.
Cummins, D. D., Kintsch, W., Reusser, K., & Weimer, R. (1988). The role of
understanding in solving word problems. Cognitive Psychology, 20(4), 405-438.
De Lange, J. (1995). Assessment: No change without problems. In T. A. Romberg
(Ed.), Reform in school mathematics (pp. 87-172). Albany: SUNY Press.
De Lange, J. (2003). Mathematics for literacy. In B. L. Madison & L. A. Steen (Eds.),
Quantitative literacy: Why numeracy matters for schools and colleges (pp. 75-89). Princeton,
NJ: National Council on Education and Disciplines.
Ding, M., & Li, X. (2010). A Comparative analysis of the distributive property in U.S.
and Chinese elementary mathematics textbooks. Cognition and Instruction, 28(2), 146-
180.
Floden, R. E. (2002). The measurement of opportunity to learn. In A. C. Porter & A.
Gamoran (Eds.), Methodological advances in cross-national surveys of educational achievement
(pp. 231-266). Washington, D.C.: National Academy Press.
Forman, S. L., & Steen, L. A. (2001). Why math? Applications in science, engineering,
and technological programs. Research Brief, American Association of Community Colleges.
Freeman, D. J., & Porter, A. C. (1989). Do textbooks dictate the content of
mathematics instruction in elementary schools? American Educational Research Journal,
26(3), 403-421.
Freudenthal, H. (1986). Didactical principles in mathematics instruction. In J. A.
Barroso, Aspects of mathematics and its applications (pp. 351-357). Amsterdam: Elsevier
Science Publishers BV.
Gatabi, A. R., Stacey, K., & Gooya, Z. (2012). Investigating grade nine textbook
problems for characteristics related to mathematical literacy. Mathematics Education
Research Journal, 24(4), 403-421.
Graumann, G. (2011). Mathematics for problem in the everyday world. In J. Maasz &
J. O'Donoghue (Eds.), Real-world problems for secondary school mathematics students: case
studies (pp. 113-122). Rotterdam: Sense Publishers.
Greer, B., Verschaffel, L., & Mukhopadhyay, S. (2007). Modelling for life: Mathematics
and children's experience. In W. Blum, P. L. Galbraith, H.-W. Henn & M. Niss
(Eds.), Modelling and applications in mathematics education: The 14th ICMI Study (pp. 89-
98). New York: Springer.
Haggarty, L., & Pepin, B. (2002). An investigation of mathematics textbooks in
England, France and Germany: Some challenges for England. Research in
Mathematics Education, 4(1), 127-144.
Haines, C., & Crouch, R. (2007). Mathematical modelling and applications: Ability and
competence frameworks. In W. Blum, P. L. Galbraith, H.-W. Henn & M. Niss

85

Thesis Ari_141201_FINAL.pdf 85 1-12-2014 16:26:26

 Chapter 3

(Eds.), Modelling and Applications in Mathematics Education: The 14th ICMI Study (pp.
89-98). New York: Springer.
Hiebert, J., Gallimore, R., Garnier, H., Givvin, K. B., Hollingsworth, H., Jacobs, J., et
al. (2003). Teaching mathematics in seven countries: Results from the TIMSS 1999 Video
Study. NCES 2003-013. Washington, D.C.: U.S. Department of Education,
National Center for Education Statistics.
Husén, T. (Ed). (1967). International study of achievement in mathematics: A comparison of
twelve countries (Vol. II). New York: John Wiley & Sons.
Ikeda, T. (2007). Possibilities for, and obstacles to teaching applications and modelling
in the lower secondary levels. In W. Blum, P. L. Galbraith, H.-W. Henn & M. Niss
(Eds.), Modelling and applications in mathematics education: The 14th ICMI
Study. New York: Springer.
Kolovou, A., Van den Heuvel-Panhuizen, M., & Bakker, A. (2009). Non-routine
problem solving tasks in primary school mathematics textbooks – A Needle in a
Haystack. Mediterranean Journal for Research in Mathematics Education, 8(2), 31-68.
Lampert, M. (1990). When the problem in not the question and the solution is not the
answer: Mathematical knowing and teaching. American Educational Research Journal,
27, 29-63.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for
categorical data. Biometrics, 33(1), 159-174.
Li, X., Ding, M., Capraro, M. M., & Capraro, R. M. (2008). Sources of differences in
children's understandings of mathematical equality: Comparative analysis of teacher
guides and student texts in China and the United States. Cognition and Instruction,
26(2), 195-217.
Liu, X. (2009). Linking competence to opportunities to learn: Models of competence and data
mining. New York: Springer.
Maass, K. (2007). Modelling tasks for low achieving students – first results of an
empirical study. In D. Pitta-Pantazi & G. Philippou (Eds.), Proceedings of the Fifth
Congress of the European Society for Research in Mathematics Education CERME 5 (pp.
2120-2129). Larnaca, Cyprus.
Maass, K. (2010). Classification scheme for modelling tasks. Journal für Mathematik-
Didaktik, 31(2), 285-311.
McDonnell, L. M. (1995). Opportunity to learn as a research concept and a policy
instrument. Educational Evaluation and Policy Analysis, 17(3), 305-322.
McNeil, N. M., Grandau, L., Knuth, E. J., Alibali, M. W., Stephens, A. C., Hattikudur,
S., & Krill, D. E. (2006). Middle-School students’ understanding of the equal sign:
The books they read can’t help. Cognition and Instruction, 24(3), 367-385.
Muller, E., & Burkhardt, H. (2007). Applications and modelling for mathematics -
Overview. In W. Blum, P. L. Galbraith, H.-W. Henn & M. Niss (Eds.), Modelling

86

Thesis Ari_141201_FINAL.pdf 86 1-12-2014 16:26:26

 Opportunity-to-learn provided by textbooks

and applications in mathematics education: The 14th ICMI study (pp. 267-274). New York:
Springer.
Newman, M. A. (1977). An analysis of sixth-grade pupils’ errors on written
mathematical tasks. Victorian Institute for Educational Research Bulletin, 39, 31-43.
Niss, M., Blum, W., & Galbraith, P. (2007). Introduction. In W. Blum, P. L. Galbraith,
H.-W. Henn & M. Niss (Eds.), Modelling and Applications in Mathematics Education: The
14th ICMI Study (pp. 3-32). New York: Springer.
OECD. (2003a). Literacy skills for the world of tomorrow. Further results from PISA 2000.
Paris: OECD.
OECD. (2003b). The PISA 2003 assessment framework - Mathematics, reading, science, and
problem solving knowledge and skills. Paris: OECD.
OECD. (2004). Learning for tomorrow’s world. First results from PISA 2003. Paris: OECD.
OECD. (2007). PISA 2006: Science competencies for tomorrow's world. Paris: OECD.
OECD. (2009a). Learning mathematics for life. A view perspective from PISA. Paris: OECD.
OECD. (2009b). Take the test. Sample questions from OECD’s PISA assessments. Paris:
OECD.
OECD. (2010). PISA 2009 Results: What students know and can do. Student performance in
reading, mathematics, and science (Vol. I). Paris: OECD.
Pepin, B., & Haggarty, L. (2001). Mathematics textbooks and their use in English,
French and German classrooms: A way to understand teaching and learning
cultures. ZDM – The International Journal on Mathematics Education, 33(5), 158-175.
Pusat Kurikulum. (2003a). Kurikulum 2004. Standar Kompetensi mata pelajaran matematika
Sekolah Menengah Pertama dan Madrasah Tsanawiyah. Jakarta: Departemen Pendidikan
Nasional.
Pusat Kurikulum. (2003b). Kurikulum 2004. Standar Kompetensi mata pelajaran matematika
Sekolah Menengah Atas dan Madrasah Aliyah. Jakarta: Departemen Pendidikan
Nasional.
Reys, B. J., Reys, R. E., & Chavez, O. (2004). Why mathematics textbooks matter.
Educational Leadership, 61(5), 61-66.
Schmidt, W. H., McKnight, C. C., Valverde, G. A., Houang, R. T., & Wiley, D. E.
(1997). Many visions, many aims: A cross-national investigation of curricular intentions in school
mathematics. Dordrecht: Kluwer Academic Publishers.
Schwarzkopf, R. (2007). Elementary modeling in mathematics lessons: The interplay
between “real-world” knowledge and “mathematics structures”. In W. Blum, P. L.
Galbraith, H.-W. Henn, & M. Niss (Eds), Modelling and applications in mathematics
education: The 14th ICMI Study (pp. 209-216). New York: Springer.
Stein, M. K., & Smith, M.S. (1998). Mathematical tasks as a framework for reflection:
From research to practice. Mathematics Teaching in the Middle School, 3(4), 268-275.

87

Thesis Ari_141201_FINAL.pdf 87 1-12-2014 16:26:26

 Chapter 3

Tornroos, J. (2005). Mathematics textbooks, opportunity to learn and student

achievement. Studies in Educational Evaluation, 31(4), 315-327.
Valverde, G. A., Bianchi, L. J., Wolfe, R. G., Schmid, W. H., & Houang, R. T. (2002).
According to the book. Using TIMSS to investigate the translation of policy into practice through
the world of textbooks. Dordrecht: Kluwer Academic Publishers.
Van den Heuvel-Panhuizen, M. (1996). Assessment and realistic mathematics education.
Utrecht: CD-Ƣ Press, Utrecht University.
Van den Heuvel-Panhuizen, M. (2005). The role of context in assessment problems in
mathematics. For the Learning of Mathematics, 25(2), 2-9 + 23.
Van Zanten, M., & Van den Heuvel-Panhuizen, M. (2014). Freedom of design: The
multiple faces of subtraction in Dutch primary school textbooks. In Y. Li & G.
Lappan (Eds.), Mathematics curriculum in school education (pp. 231-259). Heidelberg
/Dordrecht /London /NewYork: Springer.
Verschaffel, L., Dooren, W. V., Greer, B., & Mukhopadhyay, S. (2010).
Reconceptualising word problems as exercises in mathematical modelling. Journal
für Mathematik-Didaktik, 31(1), 9-29.
Verschaffel, L., Greer, B., & De Corte, E. (2000). Making sense of word problems. Lisse:
Swets & Zeitlinger.
Wijaya, A., Van den Heuvel-Panhuizen, M., Doorman, M., & Robitzsch, A. (2014).
Difficulties in solving context-based PISA mathematics tasks: An analysis of
students’ errors. The Mathematics Enthusiast, 11(3), 555-584.
Xin, Y. P. (2007). Word problem solving tasks in textbooks and their relation to
student performance. Journal of Educational Research, 100(6), 347-359.

88

Thesis Ari_141201_FINAL.pdf 88 1-12-2014 16:26:26

 Chapter 4

Teachers’ teaching practices and beliefs regarding context-

based tasks and their relation with students’ difficulties in
solving these tasks

Wijaya, A., Van den Heuvel-Panhuizen, M., & Doorman, M. (submitted). Teachers’
teaching practices and beliefs regarding context-based tasks and students’ difficulties
in solving these tasks.

Thesis Ari_141201_FINAL.pdf 89 1-12-2014 16:26:26

 Chapter 4

Teachers’ teaching practices and beliefs regarding context-

based tasks and their relation with students’ difficulties in
solving these tasks

1 Introduction
Currently, the ability to apply mathematics is considered a core goal of
mathematics education all around the world (see, e.g., Graumann, 2011; NCTM,
2000; OECD, 2003; Scardamalia, Bransford, Kozma, & Quellmalz, 2012;
Schleicher, 2007; Tomlinson, 2004). In the United States this goal is explicitly
stated in the Principles and Standard for School Mathematics (National Council of
Teachers of Mathematics [NCTM], 2000). According to the NCTM (2000, p. 64),
mathematics teaching should enable students to “recognize and apply mathematics
in contexts outside of mathematics.” In the United Kingdom, mathematics
education is aimed at ensuring that young people acquire functional mathematics
(Tomlinson, 2004, p. 29), which includes the knowledge and the capacity to apply
mathematics. This goal is similar to what, for example, in the Programme for
International Student Assessment (PISA) is called mathematical literacy, which
refers to students’ ability “to identify, and understand, the role that mathematics
plays in the world, to make well-founded judgments and to use and engage with
mathematics in ways that meet the needs of that individual’s life as a constructive,
concerned, and reflective citizen” (Organisation for Economic Co-operation and
Development [OECD], 2003, p. 24).

However, despite the significance attributed to applying mathematics in various

contexts, several studies have revealed that student performance in solving
context-based tasks is rather low (Cooper & Dunne, 2000; Sam, Lourdusamy, &
Ghazali, 2001). The PISA 2012 study (OECD, 2013) showed that about 32% of
the total group of students in the 65 participating countries had a performance
below Level 2 and could only answer “questions involving familiar contexts where
all relevant information is present and the questions are clearly defined” (OECD,
2013, p. 61). A further result from the PISA 2012 study (OECD, 2013) was that
there were fifteen countries in which more than half the students performed below
this level.

With respect to students’ difficulties when solving context-based tasks, various

studies showed that students have problems with understanding the wording of
context-based tasks and identifying relevant information (Klymchuk, Zverkova,

90

Thesis Ari_141201_FINAL.pdf 90 1-12-2014 16:26:26

 Teachers’ teaching practices and beliefs

Gruenwald, & Sauerbier, 2010; Prakitipong & Nakamura, 2006). It was also found
that students struggle to select adequate mathematical procedures (Clements, 1980;
Klymchuk et al., 2010) and often just apply a routine procedure without taking
realistic considerations into account (Sepeng, 2013; Verschaffel, Greer, & De
Corte, 2000; Xin, Lin, Zhang, & Yan, 2007).

Indonesia, similar to many other countries, attaches high value to applying

mathematics as a core goal of mathematics education. This is clearly stated in the
Indonesian national curriculum (Pusat Kurikulum, 2003). Nevertheless, in
Indonesia there also is an apparent discrepancy between this goal and outcomes in
student achievement. The PISA results repeatedly showed that Indonesian
students perform low in solving context-based tasks. For example, in the PISA
2009 study (OECD, 2010) 77% of Indonesian students did not reach Level 2.

The result from this PISA study prompted us to set up a project called “Context-
based Mathematics Tasks Indonesia” (CoMTI) to investigate how Indonesian
students’ performance can be improved. The first step in this project was a study
into Indonesian students’ difficulties when solving context-based tasks. We found
that the students mostly got stuck in the early stages of the solving process, i.e.,
they had particular difficulties with comprehending what a problem, situated in a
context, was about and made errors in transforming a context-based problem into
a mathematical problem (Wijaya, Van den Heuvel-Panhuizen, Doorman, &
Robitzsch, 2014). The next step in the CoMTI project was identifying possible
causes of these difficulties. For this, we took the concept of opportunity-to-learn
(OTL) into account, because this is seen as crucial in investigating possible reasons
for students’ low performance (Brewer & Stasz, 1996; Hiebert & Grouws, 2007).
Our first focus was on the OTL offered by textbooks. An analysis of Indonesian
mathematics textbooks revealed that only 11% of tasks were context-based
(Wijaya, Van den Heuvel-Panhuizen, & Doorman, submitted).

Although this low percentage of context-based tasks in textbooks gave a possible

explanation for Indonesian students’ low scores on these tasks, it might not be the
only reason for the low scores, because what is offered in textbooks is not always
identical to what is taught by teachers (Pepin & Haggarty, 2001). For instance,
teachers may leave out and add content, emphasize aspects of applying
mathematics that are not addressed in textbooks, or neglect specific textbook
guidelines. This means that students’ performance can be influenced by teachers’
teaching practices (Grouws & Cebulla, 2000; Hiebert & Grouws, 2007). In other

91

Thesis Ari_141201_FINAL.pdf 91 1-12-2014 16:26:26

 Chapter 4

words, the teaching practice of teachers can also contribute to the students’ OTL.
Moreover, teaching practice is often found to be affected by what teachers think
about mathematics (Ernest, 1989; Stipek, Givvin, Salmon, & MacGyvers, 2001;
Wilkins, 2008). It means teachers’ beliefs may also play a role in the OTL offered
by teachers.

Therefore, to further investigate the OTL to solve context-based tasks given to

students, the present study addressed teachers’ teaching practices in relation with
teachers’ beliefs. In addition, we analyzed whether our findings regarding the OTL
offered by Indonesian teachers correspond to the kinds of errors that Indonesian
students make when solving context-based tasks.

2 Theoretical background

2.1 OTL offered by teachers

To find an explanation for the differences in students’ mathematics performance in
different countries Husén (1967, p. 162-163) introduced the concept of OTL,
which was defined as “whether or not […] students have had the opportunity to
study a particular topic or learn how to solve a particular type of problem.” This
definition is specified by Brewer and Stasz (1996) who distinguished three aspects
of OTL. The first is the curriculum content which focuses on the scope of the
topics offered to students. The second refers to the teaching strategies teachers use
to present the topics and engage students. The third aspect concerns the
instructional resources, for instance textbooks used to teach the students.

Indeed, several studies (e.g., Husén, 1967; Tornroos, 2005) have found that
curriculum and instructional materials are important for students’ OTL. Yet, there
were also studies (e.g., Grouws & Cebulla, 2000; Hiebert & Grouws, 2007) which
revealed that students’ mathematical performance is largely influenced by teachers’
teaching practices. These studies showed that the strategies used by teachers to
teach particular topics, the kinds of tasks they presented to students, and the
nature of the discussions they organized in class are important factors influencing
students’ OTL.

92

Thesis Ari_141201_FINAL.pdf 92 1-12-2014 16:26:26

 Teachers’ teaching practices and beliefs

2.2 Teaching practices contributing to students’ OTL to solve context-

based tasks
From the perspective of mathematical modeling, Antonius, Haines, Jensen, Niss,
and Burkhardt (2007, p. 295) argued that teaching context-based tasks requires
more than an “explanation-example-exercise ritual”, i.e. teacher explains a concept,
gives an example of a problem in which this concept is applied and finally offers
students exercises for practicing problems with this concept. Such a directive
approach does not offer students the opportunity to develop strategic
competences which are necessary to solve context-based tasks (Antonius et al.,
2007), such as understanding the text of problems and selecting relevant
information, identifying and applying appropriate mathematical procedures,
connecting different representations or mathematics concepts, and interpreting
solutions (Blum, 2011; OECD, 2003). To achieve these competences, several
authors (Antonius et al., 2007; Forman & Steen, 2001) emphasized that teachers,
instead of using a directive teaching method, should teach context-based tasks
through a student-centered and investigative teaching approach in which students
are actively involved and the teacher’s role is consultative rather than directive.
Similarly, Blum (2011, p. 25) pleaded for consultative teaching, which he called
“operative-strategic teaching” that emphasizes guiding students to actively and
independently construct new knowledge by using their prior knowledge and
experiences. A key aspect of such teaching is that teachers should keep a balance
between providing guidance and fostering students’ independence for which they
can use flexible interventions and metacognitive prompts to elicit students to
reflect on their own understanding of the problem and on how they selected the
mathematical procedures to solve the problem. Promoting students to reflect can
also be done by providing them with opportunities to assess and (if necessary)
revise their own work (Bell & Pape, 2012).

Further specific recommendations can be given when zooming in on the four

stages of solving context-based tasks: comprehension stage, transformation stage,
mathematical processing stage, and encoding stage (see Blum, 2011).

2.2.1 OTL connected to the comprehension stage

In the comprehension stage, students have to figure out what the problem is about
and to identify the information that is relevant to solving it. Directly telling
students what a context-based task means and what information is required is,
according to Barnes (2000, p. 41), not supportive for their learning because such

93

Thesis Ari_141201_FINAL.pdf 93 1-12-2014 16:26:26

 Chapter 4

practice causes students to “have less need to struggle and less occasion to make
efforts of their own to achieve understanding.”

Several studies (e.g., Hagaman, Casey, & Reid, 2013; Karbalei & Amoli, 2011) have
shown that the so-called three-step RAP (Read-Ask-Paraphrase) strategy can
stimulate students’ active involvement in getting to know what is asked in the
problem and can improve reading comprehension. The first step of this RAP
strategy is in agreement with Blum’s (2011, p. 24) suggestion to ask students to
“read the text precisely and imagine the situation clearly.” In the second RAP step,
students are encouraged to figure out what the problem is really asking them. This
approach is similar to Kramarski, Mevarech, and Arami’s (2002, p. 228) suggestion
to train students to formulate and answer self-addressed metacognitive
comprehension questions such as “what is the problem about?” The last RAP step
implies the teacher should ask students to formulate the problem in their own
words. Paraphrasing is helpful for students because it makes problems more
familiar and, consequently, more understandable for them (Karbalei & Amoli,
2011; Kletzien, 2009).

Another important aspect of comprehending a problem is awareness of the

information needed to solve the problem. To achieve this, several authors (Blum,
2011; Forman & Steen, 2001; Lingefjard & Meier, 2010) suggested letting students
discuss the information presented in a problem in relation to what is asked in the
problem. Other approaches are asking students to figure out whether particular
information is missing (Forman & Steen, 2001) or to formulate a self-addressed
question like “Do I already have enough information to solve the problem?”

2.2.2 OTL connected to the transformation stage

In the transformation stage students have to transform a context-based task into a
mathematical problem. Again, directly telling the students what to do might not
offer them an OTL. It is more helpful when students are involved in this process
and can themselves explore different ways of transforming a context-based task
into a mathematical problem which they can use to solve this task. To stimulate
this exploration, students can be asked to formulate and answer questions such as
“What might be a possible mathematical procedure to solve this problem?”
Another approach to achieve this exploration is to call up earlier experiences of the
students when solving similar context-based tasks. In line with this, Kramarski et
al. (2002, p. 228) suggested that teachers should encourage students to formulate
and answer self-addressed questions such as “What are the similarities or

94

Thesis Ari_141201_FINAL.pdf 94 1-12-2014 16:26:26

 Teachers’ teaching practices and beliefs

differences between this problem and the problems I have ever solved?” What all
these approaches have in common is that OTL in the transformation stage needs
to orient students toward identifying relevant mathematical procedures for solving
the problem (Galbraith & Stillman, 2006).

2.2.3 OTL connected to the mathematical processing stage

In the mathematical processing stage, students do not in fact have to deal with the
context-based character of a problem, but only carry out the mathematical
procedure(s) resulting from transforming a context-based problem into a
mathematical problem. Therefore, it is not surprising that for the mathematical
processing stage, none of the aforementioned studies – e.g. Forman and Steen
(2001); Kramarski et al. (2002); Lingefjard and Meier (2010) – gave suggestions
that offer students OTL. Furthermore, the mathematical processing stage can
cover various mathematics topics which might make it difficult to provide a fixed
suggestion or direction. Nevertheless, having fewer mistakes in performing
mathematical procedures would eventually also help students to become better in
solving context-based tasks. Therefore, a teaching practice in which the teacher
stimulates students to check their mathematical procedures can also be considered
an OTL to solve context-based tasks.

2.2.4 OTL connected to the encoding stage

In the encoding stage students have to interpret a mathematical solution in terms
of the situation of the context-based task and take realistic and critical
considerations into account. For this, the students should be encouraged to link
their solution to the situation of the task and to verify the reasonableness of the
solution (Blum, 2011; Forman & Steen, 2001). Kramarski et al. (2002, p. 228)
proposed that teachers should stimulate their students to ask themselves whether
the solution makes sense. Such a teaching practice contrasts with just focusing on
the correctness of the mathematical solution.

2.3 Teacher beliefs contributing to students’ OTL to solve context-

based tasks
Several studies (Beswick, 2005; Ernest, 1989; Stipek, Givvin, Salmon, &
MacGyvers, 2001; Wilkins, 2008) showed the influence of teachers’ beliefs on
teachers’ practice. For example, Beswick found a relation between teachers’
positive beliefs toward the importance of problem solving in mathematics and
their constructivist teaching practice in which students are actively involved in the

95

Thesis Ari_141201_FINAL.pdf 95 1-12-2014 16:26:26

 Chapter 4

teaching-learning process. According to Ernest (1989), a teacher’s belief system

consists of three key components; i.e. beliefs (1) about the nature of mathematics,
(2) about the nature of mathematics teaching, and (3) about the process of learning
mathematics. With respect to the belief about the nature of mathematics, Lakatos
(1976) distinguished two philosophical views on mathematics: the absolutist and
the fallibilist. The absolutist view considers that mathematical truth is absolute and
represents objective and certain knowledge (Ernest, 1991) that “somehow exists
apart from everyday human affairs” (Toumasis, 1997, p. 319). Within this view,
mathematical truth is seen as resulting from a deduction process and consisting of
a priori knowledge without observations of the real world. In contrast, the fallibilist
view considers mathematical truth as fallible (Ernest, 1991). According to this
view, mathematical knowledge is open to revision and is related to everyday life,
because it results from human activity in trying to organize patterns and structures
in phenomena outside of and within mathematics (Ernest, 1991; Toumasis, 1997).

Toumasis (1997) and Tuge (2008) extended these different views on mathematics
to views on teaching and learning of mathematics. Their studies pointed out that
teachers with an absolutist view believe mathematics teaching is primarily oriented
toward mathematical content and the practice of procedural skills in which
teachers have the role of demonstrating how students should proceed. In contrast,
teachers with a fallibilist view focus their teaching of mathematics on mathematical
reasoning, the process of organizing patterns and structures, rather than on
practicing procedures and dealing with mathematics as a fixed system. Regarding
beliefs about the process of learning mathematics, within the absolutist view it is
argued that learning mathematics depends on drill or hard exercise of mathematical
procedures (Toumasis, 1997) and that mathematical concepts do not need to be
connected to the real-world context (Tuge, 2008). Teachers with a fallibilist view
are student-centered and believe learning mathematics should involve students in
investigation, exploration, problem solving and discussion (Toumasis, 1997; Tuge,
2008).

Comparing the characteristics of these two views, we may conclude that teachers
with a fallibilist view are well-placed to create teaching practices which provide
students OTL to solve context-based tasks: these teachers see mathematics as
related to everyday life; they emphasize student-centered learning through
investigation, exploration, and discussion, which is also in line with the consultative
teaching described earlier. In contrast, the absolutist view is at risk of not giving
students the chance to solve context-based tasks, because it considers mathematics

96

Thesis Ari_141201_FINAL.pdf 96 1-12-2014 16:26:26

 Teachers’ teaching practices and beliefs

as a set of abstract concepts and focuses on teaching practices which offer students
opportunities to exercise mathematical procedures without connection to a real-
world context.

The relation between teachers’ conceptions on context-based tasks and how

teachers use such tasks in their teaching is also emphasized by Chapman (2009).
Teachers who are mostly oriented toward mathematics as computational and
algorithmic problem solving conceptualize context-based tasks as word problems
meant to practice earlier learned procedures. According to Chapman (2009), these
teachers prefer context-based tasks which have a clear question, contain only the
information that is relevant for solving the task, and have an explicit suggestion
about the mathematical procedure to use.

A different conception on context-based tasks is reflected by teachers who use

contexts which have realistic value to students. Such teachers use context-based
tasks as a tool to help students “experience the world” and develop in-depth
mathematical thinking (Chapman, 2009, p. 232). These teachers favor context-
based tasks that offer various opportunities for students to create models to
structure the problems, to explore and connect different mathematical procedures
or concepts, and to select information. In line with this, the teachers in Chapman’s
study (2009) emphasized that students can play with a large amount of
information, which is also in agreement with Verschaffel, Van Dooren, Greer, and
Mukhopadhyay (2010), who pointed out that context-based tasks which do not
include irrelevant information and do not require students to look for additional
information will not support the development of students’ modeling competences
and in-depth thinking.

2.4 Research questions

After first investigating the kinds of errors Indonesian students make when solving
context-based tasks (Wijaya et al., 2014), and then looking for a possible
explanation for these errors by examining the OTL provided by Indonesian
textbooks (Wijaya et al., submitted), the present study is researching students’ OTL
from the perspective of teachers; focusing on teaching practices and underlying
beliefs. Although the main emphasis was on teaching practices we started with
teachers’ beliefs, because having beliefs that support using context-based
mathematics tasks can be considered necessary for a teaching practice that offers
students OTL to solve such tasks. In the present study we addressed the following
research questions:

97

Thesis Ari_141201_FINAL.pdf 97 1-12-2014 16:26:26

 Chapter 4

1. What beliefs do Indonesian teachers have regarding mathematics, teaching and learning
of mathematics, and context-based tasks?
2. What OTL to solve context-based tasks do teachers offer students in their classroom
practice?
2.a. What kinds of context-based tasks do Indonesian teachers offer their students?
2.b. What teaching approach do Indonesian teachers use to teach context-based tasks?
3. Is there a relationship between the OTL to solve context-based tasks offered by
Indonesian teachers and the errors Indonesian students make when solving such tasks?

3 Method

3.1 Design of the study

To investigate the students’ OTL to solve context-based tasks offered by teachers,
a teacher survey study was carried out through a written questionnaire and a series
of classroom observations. The results from from the questionnaire and the
observations were compared with the findings from our earlier study (Wijaya et al.,
2014) in which we identified the kinds of errors Indonesian students made when
solving context-based tasks.

3.2 Participants
The participants of the study were Junior High School mathematics teachers
working at seven schools in rural and urban areas in the province of Yogyakarta,
Indonesia. The seven schools were the same schools in which we collected the data
for investigating students’ errors when solving context-based tasks (see Wijaya et
al., 2014). We chose the province of Yogyakarta for the CoMTI project for reasons
of convenience (the first author originates from this province). All Junior High
School mathematics teachers of the participating schools filled in the
questionnaire. This resulted in a sample of 27 teachers (14 male and 13 female),
including nine Grade 7 teachers, twelve Grade 8 teachers, and eleven Grade 9
teachers1. The teachers had two to 34 years of teaching experience (M = 19,
SD = 10.02).

1 The total number is more than 27, because some teachers taught in two grades.

98

Thesis Ari_141201_FINAL.pdf 98 1-12-2014 16:26:26

 Teachers’ teaching practices and beliefs

For the classroom observations we asked all Grade 8 teachers2 whether they were
willing to be observed and video recorded during two mathematics lessons in
which they had to address context-based tasks for which we would provide them a
set of tasks. In total, four of the twelve Grade 8 teachers volunteered. The other
teachers either did not feel confident to be observed or argued that spending two
lessons on additional tasks would not fit their time schedule. The teachers whose
lessons were observed were from three different schools, had moderate to long
teaching experience, and all had a Bachelor degree in mathematics education (see
Table 1). They also used the same textbook, Matematika (Textbook for Junior High
School, Grade VIII: 2A & 2B).

Table 1
Teachers whose lessons were observed
Age Teaching experience Education
Name1 School Gender
(year) (year) background
Siti A Female 47 26 B.A. (math. education)
Ihsan B Male 44 17 B.A. (math. education)
Leni B Female 42 17 B.A. (math. education)
Ratih C Female 30 4 B.A. (math. education)
1 These names are pseudonyms.

3.3 Teacher questionnaire

To gather information about teachers’ beliefs and teaching practices we developed
a written questionnaire. As an addition to five questions about general background
information (age, gender, years of teaching experience, grade taught, educational
background), this questionnaire contained (1) a six-item section regarding teachers’
beliefs about mathematics, mathematics teaching, and mathematics learning, (2) a
four-item section addressing teachers’ beliefs about context-based tasks and one
item on how teachers think about the number of context-based tasks in the
textbooks they use, and (3) a seven-item section on how teachers see their teaching
practices related to context-based tasks.

2 Although Grade 9 students (the 15-year-olds) are the target group of the PISA studies, we decided
to do the observations in Grade 8 because that is where the basis for the performance in Grade 9 is
laid. Moreover, the schools did not give permission to do observations in Grade 9 classes because the
preparations for the National Exam took place in these classes.

99

Thesis Ari_141201_FINAL.pdf 99 1-12-2014 16:26:26

 Chapter 4

3.3.1 Teachers’ beliefs about mathematics and teaching and learning

mathematics
Of the six items concerning teachers’ beliefs about mathematics and its teaching
and learning (see Table 2), items 1-3 were derived from a questionnaire developed
by Adamson et al. (2002), who investigated teachers’ beliefs by focusing on their
tendency toward an absolutist or fallibilist view. We took these three items because
an absolutist or fallibilist view on mathematics and its teaching might be related to
whether teachers offer students OTL to solve context-based tasks.

We developed items 4-6 ourselves, based on the aforementioned studies of

Toumasis (1997) and Tuge (2008). Item 4 addresses teachers’ beliefs about the
teaching of mathematics and items 5-6 concern beliefs about the learning of
mathematics.

For all six items in this section we used the format of paired opposite statements as
employed by Adamson et al. (2002). We asked the teachers to rate their position on
a five-point rating scale ranging from 1 to 5, in which 1 represents a strong
absolutist view and 5 a strong fallibilist view.

3.3.2 Teachers’ beliefs and reported teaching practice regarding

context-based tasks
The four items in the questionnaire that was used to measure teachers’ beliefs
about context-based mathematics tasks focused on the explicitness of
mathematical procedures and the type of information provided in these tasks. In
addition to these four items, we included an item addressing teachers’ opinions
about the sufficiency of context-based tasks in the textbooks they used. All five
items in this section (see Figure 4) contained a five-point rating scale (ranging from
1 “strongly disagree” to 5 “strongly agree”).

The last part of our questionnaire, which contained seven items, investigated how
teachers think about their teaching practices (see Figure 5). The first three items
focused on how frequently teachers give context-based tasks, make their own
context-based tasks, and on modifying teaching materials. The last four items
examined the characteristics of the context-based tasks teachers offer to their
students. The teachers were asked to indicate on a five-point rating scale the
frequency of presenting particular context-based mathematics tasks (ranging from
1 “never”, 2 “once or twice in a semester”, 3 “monthly”, 4 “weekly”, to 5 “every
lesson”).

100

Thesis Ari_141201_FINAL.pdf 100 1-12-2014 16:26:26

 Teachers’ teaching practices and beliefs

Table 2

Questionnaire for investigating teachers’ beliefs about mathematics and the teaching and learning
of mathematics
Mathematics
Mathematics consists of Mathematics consists of objects
objects and patterns that and patterns that can be seen
have no existence outside the in real objects and natural
mind. phenomena.
The beauty of mathematics is The beauty of mathematics is
found mostly when complex found mostly in how it is
problems are solved using found in the patterns of
abstract concepts. nature.
Mathematics teaching
The most important part of The most important part of
instruction is the content of instruction is how it
the curriculum. encourages thinking among
students.
School mathematics is about School mathematics is about
learning skills which are learning skills that students
needed to understand higher will need in daily life.
levels mathematics.
Mathematics learning
Mathematics learning is more Mathematics learning is more
likely to occur through drill likely to occur when students
or practices of mathematical actively engage in problem
procedures in abstract or situated in various contexts.
formal forms.
Mathematics learning aims at Mathematics learning aims at
motivating students to learn motivating students to learn
mathematics as a subject skills they need in daily life.
matter.

3.4 Classroom Observations

3.4.1 Procedure
Classroom observations were conducted to further investigate teachers’ teaching
practice regarding context-based tasks. We observed two lessons for each of the
four Grade 8 teachers who volunteered to participate in this part of the study.
Because these classroom observations were intended to investigate how teachers
helped their students to learn solving context-based tasks, rather than to examine
the frequency of dealing with context-based tasks (this was covered by the

101

Thesis Ari_141201_FINAL.pdf 101 1-12-2014 16:26:26

 Chapter 4

questionnaire), we wanted to be sure the teachers would teach context-based tasks

in their lessons. Therefore, we provided them with a set of seven context-based
tasks consisting of twelve questions on graphs of linear equations. This topic was
chosen because it was taught in these schools during the observation period. The
tasks included different kinds of contexts such as internet fees (see Internet task in
Figure 1) and travelling by bike (see Journey task in Figure 2). To offer students
OTL to select relevant information most tasks contained superfluous or missing
information. For example, in the Internet task the relevant information the
students have to identify is 18,000 IDR (the monthly fee for the program Shine),
45 IDR (the fee per 1 MB), and 550 MB (the internet usage in May). Another
characteristic of the tasks was that they did not explicitly provide suggestions about
the mathematical procedure needed to solve them. For example, to solve Question
b in the Internet task the students needed to decide whether making a calculation
or drawing a graph would be helpful to find the answer.

To ensure the ownership of the teachers when teaching context-based tasks the
teachers could employ their own teaching strategies in the observed lessons.
Moreover, they were free to choose which and how many of the provided seven
tasks they would use. In addition they could also include other context-based tasks,
either from their textbook or designed by themselves. The observations were made
by the first author and the lessons were video recorded by using two cameras, a
static camera to record whole class activity and a dynamic camera to record the
interaction between teacher and particular students.

An internet company offers two different programs, i.e. Smile and Shine. Program
Smile charges customers 31,500 IDR for monthly fee and 30 IDR/1 Megabyte (MB).
Program Shine charges customers 18,000 IDR for monthly fee and 45 IDR/1 MB. The
registration fees including the price of modem for both programs are the same, i.e.
300,000 IDR. In January Doni subscribed for program Shine. In May Doni used 550
MB of internet data.
a. How much money should be paid by Doni in May?

Doni’s internet usage is increasing. He wonders whether it will be beneficial to

change the internet program.
b. When Doni’s internet usage is increasing, when should he change the internet
program?

Figure 1. Internet problem

102

Thesis Ari_141201_FINAL.pdf 102 1-12-2014 16:26:26

 Teachers’ teaching practices and beliefs

Last Saturday, Joni and his friends went biking for Time Distance
three hours nonstop. (in minutes) (in km)
During the journey, Joni frequently checked the 10 2.5
odometer on his bike. The table shows the time 20 6
and the distance travelled by Joni and his friends. 35 11
a. Estimated how far Joni travelled after: 60 20.5
- a half hour 90 32
- two hours 110 38.5
b. Estimate when Joni reached the fastest speed. 150 50

Figure 2. Journey problem

3.4.2 Data analysis

In total, we video recorded eight lessons. As the unit of analysis we chose an
activity-based chunk of video data. It means that a chunk was not based on a
particular time slot but on an activity that was carried out in class. In our case, this
was a teaching activity related to a context-based task. Because most tasks
consisted of more than one question, we treated an activity related to a question as
a unit. Activities related to bare mathematics tasks were excluded from our
analysis.

The approach we chose for analyzing the video data units was based on what
Erickson (2006) called the part-to-whole deductive approach; an approach that is
used when the analysis has a particular theoretical orientation. As discussed earlier,
there is evidence from literature that teaching practices which reflect consultative
teaching offer students more OTL to solve context-based tasks than directive
teaching approaches. Therefore, we developed a framework for coding teaching
practices (see Table 3) which included characteristics of both approaches for all
four stages of solving a context-based task. When a teacher did not pay attention
to a particular stage, it was coded as “no instruction”.

The coding was carried out by the first author and afterwards the reliability of the
coding was checked through an additional coding by a researcher of mathematics
education not involved in this study. This extra coding was done based on two
randomly selected lessons from the eight video-recorded lessons. The Cohen’s
Kappa for the coding of these two lessons was .89, which indicates that the coding
was reliable (Landis & Koch, 1977).

103

Thesis Ari_141201_FINAL.pdf 103 1-12-2014 16:26:26

 Table 3

104
Framework for coding teachers’ teaching practice related to context-based tasks

Thesis Ari_141201_FINAL.pdf 104

Chapter 4

Teaching practice supportive for Teaching practice not supportive for

Stage of solving a No
providing students OTL to solve context-based tasks providing students OTL to solve context-
context-based task instruction
(consultative teaching) based tasks (directive teaching)
COMPRE- - Asking students to read the text precisely and imagine - Reading aloud the problem by the
HENSION the context of the problem. teacher.
STAGE - Stimulating students to formulate and answer - Telling students what the problem is
comprehension questions for themselves, e.g. “what is about.
the problem about?” - Telling students the information needed
- Asking students to paraphrase the text. to solve the problem.
- Stimulating discussion of available data in relation to
what is asked; e.g. by asking students to underline only
the relevant information.
- Encouraging students to identify and seek out missing
information; for example, by asking them to formulate
and answer self-addressed metacognitive questions,
such as “Do I already have enough information to solve the
problem?”

TRANSFOR- - Encouraging students to explore various procedures by - Telling students the mathematics
MATION considering their existing knowledge and experiences; concept or procedure required to solve
STAGE for example, by asking them to formulate and answer the problem.
self-addressed metacognitive questions, such as “What - Directly translating a real-world problem
are the similarities or differences between this problem with the into a mathematical problem.
problems I have ever solved?”

1-12-2014 16:26:26
 Teaching practice supportive for Teaching practice not supportive for
Stage of solving a No
providing students OTL to solve context-based tasks providing students OTL to solve context-

Thesis Ari_141201_FINAL.pdf 105

context-based task instruction
(consultative teaching) based tasks (directive teaching)
TRANSFOR- - Encouraging students to identify the relevant
MATION mathematical procedures; for example, by asking them
STAGE to formulate and answer self-addressed metacognitive
questions, such as “What procedures are appropriate for
solving the problem?”

MATH. - Stimulating students to check the mathematical - Directly correcting students’ mistake at a
PROCESSING procedures they perform; e.g. by saying “Check your particular step of the mathematical
STAGE calculation thoroughly”, “Do you already perform the algebraic procedures.
operation correctly?”

ENCODING - Asking students to verify the reasonableness of - Directly telling students if their solution
STAGE solutions in terms of the original problem; for example, is unrealistic or does not make sense.
by asking them to formulate and answer self-addressed - Focusing only on the correctness of the
metacognitive questions, which focus on reflecting on mathematical solution without
the solution: “Does the solution make sense?” connecting it to the context of the
- When students only give mathematical solution, asking problem.
them to interpret the solution in terms of the context
of the problem; for example, by asking: “What does this
number mean?”

105
Teachers’ teaching practicesand beliefs

1-12-2014 16:26:26
 Chapter 4

4 Results

4.1 Teachers’ beliefs

4.1.1 Beliefs about mathematics and mathematics teaching and

learning
Regarding teachers’ beliefs, we found the teachers had a tendency towards the
fallibilist view on the nature of mathematics and the teaching and learning of it (see
Figure 3). We found that 48% of the teachers (13 out of 27) rated 4 or 5, which
indicates they tended to believe that mathematics consists of objects and patterns
that can be seen in real objects and natural phenomena and only 7% of the
teachers (two out of 27) tended to believe mathematics consists of objects and
patterns that have no existence outside the mind (Item 1). Regarding beliefs about
teaching mathematics (Item 3), more than 90% of the teachers (25 out of 27)
tended to believe that mathematics teaching should focus on encouraging thinking
and reasoning among students and not only on teaching mathematics content. A
tendency toward the fallibilist view was also found for two beliefs about
mathematics learning. We found that almost 60% of the teachers (16 out of 27)
tended to believe that students’ learning is more likely to occur when students are
actively engaged in problems situated in various contexts (Item 5) and more than
half of the teachers (15 out of 27) indicated believing that mathematics learning has
as an objective to motivate students to learn skills needed in daily life (Item 6).
However, only 26% of the teachers (seven out of 27) agreed that school
mathematics is about teaching skills that students will need in daily life and almost
half (13 out of 27) believed that school mathematics is about teaching pure
mathematics (Item 4). Here, the teachers showed a rather absolutist view on
learning mathematics.

106

Thesis Ari_141201_FINAL.pdf 106 1-12-2014 16:26:26

 Item Absolutist view on Fallibilist view on
Mathematics

Thesis Ari_141201_FINAL.pdf 107

1 Mathematics consists of objects and patterns Mathematics consists of objects and patterns that can
that have no existence outside the mind. be seen in real objects and natural phenomena.

2 The beauty of mathematics is found mostly when The beauty of mathematics is found mostly in
mathematics problems are solved using abstract how it is found in the patterns of nature.
concepts.

Mathematics teaching
3 The most important part of teaching is the The most important part of teaching mathematics is
mathematics content. how it encourages thinking and reasoning among
students.

4 School mathematics is about teaching topics, School mathematics is about teaching skills that
skills, and procedures which are needed to students will need in daily life.
understand pure mathematics.

Mathematics learning
5 Mathematics learning is more likely to occur Mathematics learning is more likely to occur
through drill or practices of mathematical when students actively engage in problem
procedures in abstract form. situated in various contexts.

6 Mathematics learning is aimed at motivating Mathematics learning is aimed at motivating

students to learn mathematics as a subject students to learn skills they need in daily life.
matter.

Strong Strong
absolutist fallibilist
view view

Figure 3. Teachers’ tendency towards a n absolutist or fallibilist view on mathematics, mathematics teaching, and mathematics
learning

107
Teachers’ teaching practices and beliefs

1-12-2014 16:26:26
 Chapter 4

4.1.2 Beliefs about context-based tasks

Regarding their beliefs about context-based tasks most teachers reflected ideas that
are not considered to be supportive for students’ learning to solve context-based
tasks. Figure 4 shows that three quarters of the teachers (20 out of 27) (strongly)
agreed with explicitly providing mathematical procedures in context-based tasks.
With respect to the type information that is included in a task the teachers’ beliefs
seemed to be more supportive for students’ learning to solve context-based tasks.
Only 41% of the teachers (11 out of 27) (strongly) agreed that context-based tasks
should provide precisely the information that is needed to find the solution;
whereas half of the teachers (14 out of 27) (strongly) disagreed. Yet teachers were
not particularly in favor of including superfluous information; barely 30% of the
teachers (eight out of 27) (strongly) agreed. However, for having less information
than needed the situation was different in which three quarters of the teachers (21
out of 27) (strongly) agreed.

Finally, we asked whether or not the teachers agreed with the statement that the
number of context-based tasks provided in textbooks is sufficient. Almost half (13
out of 27) (strongly) disagreed and only a quarter (seven out of 27) (strongly)
agreed.

Context-based tasks should provide explicit suggestion about

the mathematical procedures required to find the solution

Context-based tasks should provide only information which is

needed to find the solution

Context-based tasks should also include information that is not

needed to find the solution

Context-based tasks should contain less information than what

is needed to find the solution

The amount of context-based tasks provided in the textbook(s)

that I use is already sufficient

Figure 4. Teachers’ beliefs about context-based tasks

108

Thesis Ari_141201_FINAL.pdf 108 1-12-2014 16:26:27

 Teachers’ teaching practices and beliefs

4.2 Teachers’ reported teaching practices

4.2.1 Frequency of offering context-based tasks

As shown in Figure 5, according to the teachers’ own judgment, they frequently
present context-based tasks to their students. 81% of them (22 out of 27) reported
giving context-based tasks at least weekly. Moreover, three quarters (20 out of 27)
stated they make their own context-based tasks at least weekly when such tasks are
not available in the textbooks. Regarding modifying the textbook approach and
supplementing it with additional problems, 81% of the teachers (22 out of 27)
reported doing so at least weekly.

4.2.2 Characteristics of context-based tasks

With respect to the characteristics of context-based mathematics tasks, the
questionnaire data revealed that what the teachers offer to their students are mostly
only plain word problems (see the last four items in Figure 5). Two thirds (18 out
of 27) stated they present context-based tasks with explicit suggestions about the
mathematical procedures at least weekly. The same result was found for context-
based tasks which provide only the information needed to find a solution.
Furthermore, 41% (11 out of 27) reported never giving context-based tasks with
superfluous information. Regarding context-based tasks with missing information,
33% (nine out of 27) reported that they never give such tasks and an equal
percentage of the teachers said to give such tasks weekly.

I give context-based tasks to my students

I make my own context-based tasks when such tasks are

not available in the textbooks I use

I modify the approach suggested in the textbooks and

supplement it with additional problems and/or activities

I give context-based tasks that provide explicit suggestion

about the required mathematical procedures

I give context-based tasks that provide only information

needed to find the solution

I give context-based tasks which include information that

is not needed to find the solution
I give context-based tasks that contain less information
than what is needed to find the solution

Figure 5. Teachers’ reported teaching practices

109

Thesis Ari_141201_FINAL.pdf 109 1-12-2014 16:26:27

 Chapter 4

4.3 Observed teaching practices

Table 4 shows the number of cases in which mathematics tasks were dealt with in
the eight lessons observed in the four classrooms. Because tasks can consist of one
or more questions we took questions as our unit of analysis. A mathematics
question can be used as either a worked example (a question with a given answer)
or an exercise (a question to be solved by the students). Unlike worked example
questions which are by definition discussed in class, exercise questions are not
always discussed with the students.

Table 4
Number of Cases a Mathematics Question was Presented in the Four Observed Classrooms
Number of cases related to Number of cases related to
a bare mathematics question a context-based mathematics question
Teacher Worked Exercise question Worked Exercise question
example example
question question
Siti 3 5 (all discussed) - 6 ( 4 discussed)
Ihsan 2 2 (all discussed) 2 11 ( 6 discussed)
Leni 3 6 ( 4 discussed) - 6 ( 3 discussed)
Ratih 3 5 (all discussed) - 4 ( 2 discussed)
Total 11 18 (16 discussed) 2 27 (15 discussed)

A total of 29 cases related to a context-based mathematics question were presented

to the students, of which 27 involved exercises and 2 involved worked examples.
The same number of cases was found for the bare mathematics questions, either
designed by the teachers themselves or taken from the textbook. Of these 29 cases,
11 were presented as worked examples. The number of cases in which the teachers
gave a context-based mathematics question ranged from 4 in Ratih’s classroom, to
13 in Ihsan’s classroom. Ihsan was also the only teacher who included his own
context-based mathematics questions. Another observation related to teaching
practice is that in only half the cases related to a context-based mathematics
question, i.e. 15 cases, the question was discussed in class.

In the following we describe the observed teaching practice in these 15 cases in

more detail. As shown in Figure 6, to help students learn to solve context-based
tasks, the teachers more frequently used a directive than a consultative teaching
approach. Directive teaching was mostly used in the comprehension stage, whereas
consultative teaching was mostly used in the mathematical processing stage. We

110

Thesis Ari_141201_FINAL.pdf 110 1-12-2014 16:26:27

 Teachers’ teaching practices and beliefs

also found a substantial number of cases where no instruction was given. This
happened in all stages, but most often in the encoding and transformation stages.

Stages of solving context-based tasks

Comprehension stage

Transformation stage

Mathematical processing stage

Encoding stage

(N = 4 x 15 cases)

All stages together

Figure 6. Overview per stage and over all stages of the teaching approaches used by
the four teachers in the 15 found cases that were related to a context-based
mathematics question

4.3.1 Comprehension stage

In the comprehension stage, attention was paid to 12 of the 15 cases in which a
question was discussed in class. All teachers used directive teaching, i.e. reading the
tasks aloud and telling students what they were about. Furthermore, no teacher
asked students to paraphrase the tasks and explain what they understood from the
tasks. For the remaining three cases, the teachers did not discuss what the tasks
were about, but directly asked students the mathematical procedure(s) that is
required to find the solution. This means the teachers skipped the comprehension
stage and directly focused on the transformation stage.

An example of directive teaching in the comprehension stage is when Leni and her
students were working on the Journey task (see Figure 2 and Excerpt 1). After
distributing the student worksheet, Leni directly read aloud the text to her students
(see lines 1-4) and demonstrated how to read information in the table (see lines 5-

111

Thesis Ari_141201_FINAL.pdf 111 1-12-2014 16:26:27

 Chapter 4

7). The students were not given opportunities to paraphrase the task and to derive
information from the table by themselves.

Excerpt 1. Teacher Leni: Journey task

Leni : For task 2 (the Journey task), I will read it for you. [Reading the text] [1]
Last Saturday, Joni and his friends went biking for three hours [2]
nonstop. […] The table shows the time and the distance travelled by [3]
Joni and his friends. [4]
[Reading the information in the table] After biking for 10 minutes, [5]
Joni checked his odometer showing 2.5 km. After 20 minutes, he [6]
checked his odometer again which showed 6 km; and so on. [7]

4.3.2 Transformation stage

In the transformation stage, nine of the 15 questions were discussed in class. In
eight of these questions directive teaching was observed. For example, Ihsan did
not ask his students to interpret the question “if water is poured into the tank in 10
minutes, how much water is in the tank?” by themselves (see Question c in
Figure 7), but he translated this question into “if the value of x is 10, then what is
the value of y?” (lines 1-2 in Excerpt 2).

The water in the backyard is filled in with water every day. Time (x) Volume of water
The relation between the time of filling water and the volume in minutes in the tank (y)
of water poured into the tank is shown in the table. in litre
a. Let x be the time of filling water and y be the volume of 0 2
Water poured into the tank. Does every pair of time and the 1 7
related volume given in the table satisfy the equation 2 12
y = 5x + 2? 3 17
4 22
b. Plot the points representing the pairs of (x,y) on the Cartesian
5 27
coordinate and sketch a graph passing all these points.
c. If water is filled into the tank in 10 minutes, how much water
is in the tank?

Figure 7. Water tank task (this task was made by Ihsan)

Excerpt 2. Teacher Ihsan: Water tank task, Question c

Ihsan : Okay, now discuss Question c. The question means that [1]
“if the value of x is 10, then what is the value of y?” [2]

Another kind of directive teaching observed in the transformation stage was telling
the students what mathematical procedure to carry out. For example, when Leni
and her students discussed a question about estimating the time Joni reached the

112

Thesis Ari_141201_FINAL.pdf 112 1-12-2014 16:26:27

 Teachers’ teaching practices and beliefs

fastest speed (see Question b in Figure 2). Although Leni encouraged her students
to share their answers (see lines 3 and 5 in Excerpt 3), her focus was on the
answers and not on stimulating them to identify relevant mathematical procedures.
Furthermore, Leni directly told her students the fastest speed was the steepest line
(see lines 5-9).

Excerpt 3. Teacher Leni; Journey task: Question b

Leni : Now Question b. Estimate when Joni reached the fastest speed. [1]
Student 1 : At (the period of) 60-90 minutes. [2]
Leni : Any other opinion? [3]
Student 2 : 110-150. [4]
Leni : Any other (opinion)? … The fastest speed means that in a short [5]
time he (Joni) travelled the furthest distance. Among these lines [6]
[pointing at the segments on the graph], this (the fastest speed) is [7]
the line (segment) which is the most …? … The fastest speed is [8]
the steepest line. Which is the steepest line? [9]
In the transformation stage a consultative teaching approach was observed only in
Ihsan’s class, when he discussed Question a of the Water tank task (Figure 7). Ihsan
posed questions such as “How do we check it?” and “Which formula?” to
stimulate students to think about strategies to solve the task (see line 2 and 5 in
Excerpt 4).

Excerpt 4. Teacher Ihsan: Water tank task, Question a

Ihsan : For Question a you are asked to check whether they (the pairs of [1]
values in the table) satisfy the equation y = 5x + 2. How do we [2]
check it?
Students : (By using) subtraction and addition. [3]
Other Using that formula. The formula of y. [4]
students:
Ihsan : Which formula? [5]
Students : The formula y = 5x + 2 [6]
Ihsan : Okay. If the x is substituted by 0, is it correct that y is 2? [7]
Students : Yes, it is. [8]
Ihsan : Now, let’s try another value … for x = 3. What is the value of y? [9]
Student 1 5× 3 + 2 … 18 (this answer is incorrect; the correct answer is 17). [10]
Ihsan : [Ignoring the student’s answer and directly explaining the steps] [11]
Take x = 3. What is the formula? y = 5x + 2. This means [Writing [12]
on the board: y = 5× 3 + 2; y = 17]. [13]

In the transformation stage it was further observed that when students had already
found an adequate transformation into a mathematical problem – which was the

113

Thesis Ari_141201_FINAL.pdf 113 1-12-2014 16:26:27

 Chapter 4

case in six of the 15 questions – the teachers did not discuss with their students
how they arrived at this mathematical problem and whether other procedures
would have also been possible.

4.3.3 Mathematical processing stage

In comparison to the other stages, in the mathematical processing stage a
consultative teaching approach was observed more frequently. In this stage the
teachers posed questions to engage students in discussing mathematical
procedures. Excerpt 4 illustrates interactions between Ihsan and his students
discussing Question a of the Water tank task (Figure 7) which was about checking
whether pairs of values in the table satisfy a given linear equation. In line 7 Ihsan
engaged his students in the solving process by asking them to check whether the
result of the substitution was correct. Furthermore, he encouraged students to do
the substitution by themselves (see line 9).

4.3.4 Encoding stage

Our results clearly revealed that the teachers tended to ignore the encoding stage
of solving context-based tasks. They only focused on the correctness of students’
mathematical solutions without connecting the answers to the task’s context. A
consultative approach was observed in only one out of 15 cases, i.e. in Ihsan’s
classroom for Question c of the Water tank task (see Figure 7). When a student
gave a number without any measurement unit, Ihsan asked her “Has this already
solved the task?” (see line 2 in Excerpt 5). Contrary to Ihsan’s teaching approaches
in the other stages for this question, directive teaching was not used in the
encoding stage. Here he did not directly tell the students to connect the answer to
the context, but stimulated them to reflect on the answer. Furthermore, Ihsan also
asked students to explain their opinion (see line 5-6). Moreover, in the end, Ihsan
asked his students to conclude the correct answer in terms of the context of the
task (see line 10).

Excerpt 5. Teacher Ihsan: Water tank task, Question c

Ihsan : Now let’s check Dina’s answer. This … if we substitute x [1]
with 0 then y is 52. Has this already solved the task? [2]
Student 1 : Yes. [3]
Student 2 and 3 : Not yet. [4]
Ihsan : Some of you said “yes”, but some others said “no”. Please [5]
explain why this has not yet solved the task. [6]
Student 2 : Because there is no ‘liter’ [7]
Ihsan : Yes, you are right. So, this is 52 .... what? [8]

114

Thesis Ari_141201_FINAL.pdf 114 1-12-2014 16:26:27

 Teachers’ teaching practices and beliefs

Students : Liter. [9]

Ihsan : So, what is the conclusion? [10]
Students : The volume of water in the tank is 52 liters. [11]

4.4 Relation between teachers’ beliefs and teaching practices and

students’ errors
Combining the findings of the present study with our earlier findings about the
errors Indonesian ninth-graders made when solving context-based tasks (Wijaya et
al., 2014)3, we found correspondences between students’ errors and teachers’
1

beliefs and teaching practices regarding context-based tasks.

Table 5 shows that the students made a substantial number of errors in the
comprehension stage, which mostly were errors in selecting relevant information.
In relation to this earlier finding, 41% of the teachers (strongly) agreed that
context-based tasks should only provide matching information. Furthermore, the
same number of teachers (strongly) disagreed that superfluous information should
be included in a context-based task. Regarding their conceived practice, 67% of the
teachers reported that they provide in every lesson or weekly context-based tasks
that contain only the information needed to find the solution. Finally, observed
teaching practice revealed that consultative teaching was not used in this stage.

With respect to the transformation stage, the findings were quite similar. In this
stage students made a high number of errors in identifying the required procedures
and, correspondingly, three quarters of teachers (strongly) agreed that context-
based tasks should state explicitly what mathematical procedure is required.
Moreover, two thirds of the teachers offered students such context-based tasks
weekly or in every lesson. The observed teaching practice showed that in 40% of
the cases no instruction was given related to this stage and that in half the cases
teaching was directive. Hardly any consultative teaching was provided to offer
students opportunities to develop their ability to transform a real-world problem
into a mathematical problem.

31 In total, 233 ninth-graders were involved in this previous study that was carried out in school year
2011-2012. The students came from the same schools as the teachers in the present study which took
place in the school year 2012-2013 and involved teachers from Grade 7 to Grade 9. The test that was
administered contained 34 questions distributed over four different booklets. Every student made 13
items. The analyzed data consisted of 3027 responses (students × items). Of these responses 1855
were correct, 346 were missing and 826 were incorrect which included 934 errors (because of the
multiple coding, the number of errors is larger than the number of incorrect responses).

115

Thesis Ari_141201_FINAL.pdf 115 1-12-2014 16:26:27

 Table 5

116
Teachers’ beliefs and teaching practices per stage of solving context-based tasks and proportion of students’ errors in this stage

Thesis Ari_141201_FINAL.pdf 116

Chapter 4

Stage in Proportion of type of Teachers’ beliefs Teachers’ reported teaching Observed type of teaching
solving a errors in the stage about context-based practice for teaching context- approach when teaching
context- (233 ninth-graders tasks based tasks context-based tasks
based task made 934 errors)a (N = 27 teachers) (N = 27 teachers) (N = 15 cases)
Compre- 38% Matching Strongly 41% Weekly/Every lesson 67% No instruction 20%
hension (50% of errors in this information agree/Agree
stage stage were errors in Superfluous Strongly 41% Never/Once-twice in 52% Directive instruction 80%
selecting relevant information disagree/Disagree semester
information) Missing Strongly 22% Never/Once-twice in 44% Consultative instruction 0%
information disagree/Disagree semester

Transfor- 39% Explicitly Strongly 74% Weekly/Every lesson 67% No instruction 40%
mation (68% of errors in this providing agree/Agree Directive instruction 53%
stage stage were errors in required Consultative instruction 7%
identifying required mathematical
procedures) procedures

Math. 20% No instruction 13%

processing Directive instruction 53%
stage Consultative instruction 43%

Encoding 3% No instruction 93%

stage Directive instruction 0%
Consultative instruction 7%
a See Note 3 and Wijaya et al. (2014)

1-12-2014 16:26:27
 Teachers’ teaching practices and beliefs

In the mathematical processing stage the situation was different. Here consultative
teaching was observed in 43% of the cases, which might explain why students
made fewer mathematical processing errors than comprehension and
transformation errors. Lastly, we found that in the encoding stage, where students
only made a few errors, in almost all cases the teachers did not give any instruction
to students. Obviously, the teachers mostly ignored the interpretation of a
mathematical answer in terms of the context of a problem.

5 Conclusions and discussion

5.1 Teachers’ OTL provided to students to solve context-based tasks

In the present study, the OTL to solve context-based tasks that was offered by
teachers was examined in order to find possible causes of students’ difficulties in
solving context-based tasks. This study focused on teachers’ teaching practices and
the underlying beliefs. Data were collected by a teacher survey based on a written
questionnaire and classroom observations.

The first focus of the study was to investigate Indonesian teachers’ beliefs about
the nature of mathematics, the teaching and learning of mathematics, and context-
based tasks (Research question 1). It was found that in general the Indonesian
teachers in the study had a tendency toward a fallibilist view on mathematics, its
learning and teaching. This view suggests that they had ideas that are supportive
for offering students OTL to solve context-based tasks. However, teachers were
not always consistent in their responses. Almost half of them believed that school
mathematics is teaching pure mathematics, which clearly reflects an absolutist view.

Regarding the teachers’ beliefs about context-based tasks, we found the teachers
tended to perceive context-based tasks as merely plain word problems. Most
teachers thought that context-based tasks should provide only the information
needed to find the solution and should explicitly provide the required mathematical
procedures. In line with other researchers (Chapman, 2009; Galbraith & Stillman,
2006; Maas, 2010; Verschaffel et al., 2010), we argue that having such beliefs about
context-based tasks and perceiving context-based tasks as straightforward word
problems will not be supportive for providing students OTL to solve context-
based tasks. Teachers who have such beliefs might only focus on the mathematical
properties or structures of a context-based tasks without attaching great value to
the problems’ context (Chapman, 2009). Furthermore, they might abandon daily

117

Thesis Ari_141201_FINAL.pdf 117 1-12-2014 16:26:27

 Chapter 4

life knowledge and experiences during the solving process (Galbraith & Stillman,
2006) and might not contribute to the students’ sense-making of a problem
(Verschaffel et al., 2010). Moreover, word problems usually provide mathematical
procedures and therefore do not offer enough opportunities for students to learn
to identify an appropriate procedure or make a mathematical model of the
problem situation (Maass, 2010).

When investigating the kinds of context-based tasks the Indonesian teachers offer
their students (Research question 2a) the questionnaire data indicated a relation
with the teachers’ beliefs. In agreement with their beliefs, the teachers reported
that they mostly gave context-based tasks which explicitly provide the needed
procedures and contain only the information that is relevant for solving the tasks.
Furthermore, most teachers stated that they rarely gave context-based tasks with
superfluous information.

With respect to how context-based tasks were taught (Research question 2b) the
classroom observations revealed that the Indonesian teachers in our sample mainly
used a directive teaching approach. The teachers mainly told the students what the
problem is about, what information they have to use, and what mathematical
problem they have to solve. The teachers also immediately corrected their
students’ mistakes when performing a mathematical procedure, and focused on the
mathematical solution without connecting it to the context of the problem. In
agreement with Antonius et al. (2007) who argued that teaching context-based
tasks requires more than telling what students should do and offering exercises to
practice, we argue that the observed teaching practice in the investigated
Indonesian classrooms cannot be considered to be supportive for providing
students OTL to solve context-based tasks. More specifically, our observation that
in the comprehension stage the teachers did not give their students opportunities
to paraphrase the tasks, might contribute to students’ difficulty in comprehending
a context-based task (see Hagaman, Casey, & Reid, 2008). Paraphrasing would help
students to understand the text of a task, and to get access to what they already
know about the task (Kletzien, 2009). The directive teaching observed in the
transformation stage is also not beneficial to teaching students to solve context-
based tasks, because as Barnes (2000) stressed, this teaching discourages students
to think about mathematical concepts involved in tasks. Only in the mathematical
processing stage did we find the teachers using consultative teaching, which may
be so because the teachers have more experience in teaching mathematical
procedures than in dealing with real-world problems. Therefore they might have

118

Thesis Ari_141201_FINAL.pdf 118 1-12-2014 16:26:27

 Teachers’ teaching practices and beliefs

more flexibility in supporting their students’ learning in the mathematical

processing stage. Lastly, in the encoding stage the teachers tended to completely
ignore the interpretation of mathematical solution(s) in terms of the context of the
problem.

Our finding about teachers’ preference for the directive teaching approach is in
line with results from other studies which also examined teaching practices in
mathematics classrooms in Indonesia (see, e.g., Human Development Department
East Asia and Pacific Region, 2010; Maulana, Opdenakker, Den Brok, & Bosker,
2012). These studies revealed that Indonesian mathematics teachers tended to take
a directive role in which they mostly explain while students write, listen, and
answer closed questions. Maulana et al. (2012) argued that such directive practices
might be caused by a cultural aspect of Indonesian society that considers the
teacher profession as highly respected, so the teacher is considered as the source of
knowledge, whereas students are the recipients.

Lastly, we can conclude that there is a relationship between Indonesian teachers’

teaching practices and their underlying beliefs regarding context-based tasks and
the errors which the Indonesian students, involved in our earlier study, made when
solving these tasks (Research question 3). The teachers’ beliefs about the kind of
information to be included in context-based tasks and the frequent use of tasks
with matching information correspond to the high occurrences of comprehension
errors, especially errors in selecting information. Similarly, the teachers’ beliefs and
practices regarding context-based tasks with clear indications for the required
mathematical procedures match to students’ transformation errors. Furthermore,
the dominance of directive teaching in the comprehension and transformation
stages might also explain the high number of comprehension and transformation
errors made by the students. The findings of this study indicate that the shortage in
the OTL to solve context-based tasks offered by the teachers is a possible
explanation for students’ difficulties solving these tasks.

5.2 Limitations and recommendations

Due to some limitations of this study, the conclusions should be interpreted with
caution. To begin with, teachers’ reports about their teaching practices regarding
context-based tasks should be considered with prudence because the data were
based on self-report. Another limitation is that the classroom observations were
only conducted in four classrooms and in each of these classrooms only two
lessons were observed. This means that only a snapshot of the teachers’ teaching

119

Thesis Ari_141201_FINAL.pdf 119 1-12-2014 16:26:27

 Chapter 4

practices was captured. Moreover, in this selection the focus was only on one
mathematics topic. The teachers might have shown different teaching practices if
they were observed for more lessons addressing various mathematics topics. In any
case a larger sample of teachers would have given a more reliable picture of
Indonesian teachers’ teaching practice regarding context-based tasks. Finally, to
answer the research question about the relationship between OTL offered by
teachers and errors made by students we used data, as explained in the Results
section, which came from the same schools, but not from the same cohorts.

These limitations make it clear that to have a more robust understanding of the
teachers’ role in the difficulties students have when solving context-based tasks, it
is necessary to conduct a further research which has a wider scope and includes
more teachers that are followed, with their students, over a long time.
Nevertheless, the present study gives a first understanding of the importance of
the OTL offered by the teachers and added to our earlier study about the OTL to
solve context-based tasks offered in Indonesian textbooks (Wijaya et al.,
submitted).

Based on this first understanding we have the following recommendations for

educational practice. When confronted with students’ low performance in solving
context-based tasks teachers (and prospective teachers) should look critically at
their own role in students’ learning processes. Did they really offer their students
opportunity-to-learn to solve context-based tasks? And were they aware of the
different stages of solving context-based tasks, each requiring specific
opportunities-to-learn?

References
Adamson, S., Burtch, M., Cox, T., Banks, D., Judson, E., & Lawson, T. (2002). Nature
of Mathematics Survey. Retrieved June 1, 2012, from
http://www.amatyc.org/publications/Electronic-
proceedings/Phoenix28/Adamson/NatureOfMathSurvey.pdf.
Antonius, S., Haines, C., Jensen, T. H., Niss, M., & Burkhardt, H. (2007). Classroom
activities and the teacher. In W. Blum, P. L. Galbraith, H.-W. Henn & M. Niss
(Eds.), Modelling and applications in mathematics education (pp. 295–308). New York:
Springer.
Barnes, M. (2000). 'Magical' moments in mathematics: Insights into the process of
coming to know. For the Learning of Mathematics, 20(1), 33–43.

120

Thesis Ari_141201_FINAL.pdf 120 1-12-2014 16:26:27

 Teachers’ teaching practices and beliefs

Bell, C. V., & Pape, S. J. (2012). Scaffolding students’ opportunities to learn

mathematics through social interactions. Mathematics Education Research Journal,
24(4), 423–445.
Beswick, K. (2005). The beliefs/practice connection in broadly defined contexts.
Mathematics Education Research Journal, 17(2), 39–68.
Blum, W. (2011). Can modelling be taught and learnt? Some answers from empirical
research. In G. Kaiser, W. Blum, R. B. Ferri & G. Stillman (Eds.), Trends in teaching
and learning of mathematical modelling (pp. 15–30). New York: Springer.
Brewer, D. J., & Stasz, C. (1996). Enhancing opportunity to learn measures in NCES
data. In G. Hoachlander, J. E. Griffith & J. H. Ralph (Eds.), From data to
information: New directions for the National Center for Education Statistics (pp. 3.1–3.28).
Washington, D.C.: U.S. Department of Education.
Chapman, O. (2009). Teachers’ conceptions and use of mathematical contextual
problems in Canada. In L. Verschaffel, B. Greer, W. Van Dooren & S.
Mukhopadhyay (Eds.), Words and worlds. Modelling verbal descriptions of situations (pp.
227–244). Rotterdam: Sense Publishers.
Clements, M. A. (1980). Analyzing children's errors on written mathematical task.
Educational Studies in Mathematics, 11(1), 1–21.
Cooper, B., & Dunne, M. (2000). Assessing children’s mathematical knowledge: Social class, sex
and problem-solving. Buckingham: Open University Press.
Erickson, F. (2006). Definition and analysis of data from videotape: Some research
procedures and their rationales. In J. L. Green, G. Camili & P. B. Ellmore (Eds.),
Handbook of complementary methods in education research (pp. 177–192). Mahwah:
Lawrence Erlbaum.
Ernest, P. (1989). The impacts of beliefs on the teaching of mathematics. In P. Ernest
(Ed.), Mathematics Teaching: The state of the art (pp. 249–253). New York: Falmer.
Ernest, P. (1991). The Philosophy of Mathematics Education. Studies in Mathematics Education.
London: Routledge Falmer.
Forman, S. L., & Steen, L. A. (2001). Why math? Applications in science, engineering,
and technological programs. Research Brief, American Association of Community
Colleges.
Galbraith, P., & Stillman, G. (2006). A framework for identifying student blockages
during transitions in the modelling process. ZDM The International Journal on
Mathematics Education, 38(2), 143–162.
Graumann, G. (2011). Mathematics for problem in the everyday world. In J. Maasz &
J. O'Donoghue (Eds.), Real-world problems for secondary school mathematics students: Case
studies (pp. 113–122). Rotterdam: Sense Publishers.

121

Thesis Ari_141201_FINAL.pdf 121 1-12-2014 16:26:27

 Chapter 4

Grouws, D. A., & Cebulla, K. J. (2000). Improving student achievement in mathematics.

Brussels: International Academy of Education.
Hagaman, J. L., Casey, K. J., & Reid, R. (2012). The effects of the paraphrasing strategy
on the reading comprehension of young students. Remedial and Special Education,
33(2), 110–123.
Hiebert, J., & Grouws, D. A. (2007). The effects of classroom mathematics teaching
on students’ learning. In F. Lester (Ed.), Second handbook of research on mathematics
teaching and learning (pp. 371–404). Charlotte: Information Age Publishing.
Human Development Department East Asia and Pacific Region. (2010). Inside
Indonesia’s mathematics classrooms: A TIMSS Video Study of teaching and practices and
student achievement. Jakarta: The World Bank.
Husén, T. (Ed). (1967). International study of achievement in mathematics: A comparison of
twelve countries (Vol. II). New York: John Wiley & Sons.
Karbalei, A., & Amoli, F. A. (2011). The effect of paraphrasing strategy training on the
reading comprehension of college students at the undergraduate level. Asian EFL
Journal, 13(3), 229–344.
Kletzien, S. B. (2009). Paraphrasing: An effective comprehension strategy. The Reading
Teacher, 63(1), 73–77.
Klymchuk, S., Zverkova, T., Gruenwald, N., & Sauerbier, G. (2010). University
students’ difficulties in solving application problems in calculus: Student
perspectives. Mathematics Education Research Journal, 22(1), 81–91.
Kramarski, B., Mevarech, Z. R., & Arami, M. (2002). The effects of metacognitive
instruction on solving mathematical authentic tasks. Educational Studies in
Mathematics, 49(2), 225–250.
Lakatos, I. (1976). A renaissance of empiricism in the recent philosophy of
mathematics. The British Journal for the Philosophy of Science, 27(3), 201–223.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for
categorical data. Biometrics, 33(1), 159–174.
Lingefjärd, T., & Meier, S. (2010). Teachers as managers of the modelling process.
Mathematics Education Research Journal, 22(2), 92–107.
Maass, K. (2010). Classification scheme for modelling tasks. Journal fur Mathematik-
Didaktik, 31(2), 285–311.
Maulana, R., Opdenakker, M.-C., Den Brok, P., & Bosker. (2012). Teacher-student
interpersonal behavior in secondary mathematics classes in Indonesia. International
Journal of Science and Mathematics Education, 10(1), 21–47.
NCTM. (2000). Principles and Standard for School Mathematics. Reston: Author.

122

Thesis Ari_141201_FINAL.pdf 122 1-12-2014 16:26:27

 Teachers’ teaching practices and beliefs

OECD. (2003). The PISA 2003 Assessment Framework – Mathematics, Reading, Science, and
Problem Solving Knowledge and Skills. Paris: Author.
OECD. (2010). PISA 2009 results: What students know and can do. Student performance in
reading, mathematics, and science (Vo. I). Paris: Author.
OECD. (2013). PISA 2012 results: What students know and can do. Student performance in
mathematics, reading and science (Vol. I). Paris: Author.
Pepin, B., & Haggarty, L. (2001). Mathematics textbooks and their use in English,
French and German classrooms: A way to understand teaching and learning
cultures. ZDM – The International Journal on Mathematics Education, 33(5), 158–175.
Prakitipong, N., & Nakamura, S. (2006). Analysis of mathematics performance of
grade five students in Thailand using Newman procedure. Journal of International
Cooperation in Education, 9(1), 111–122.
Pusat Kurikulum. (2003). Kurikulum 2004. Standar kompetensi mata pelajaran matematika
Sekolah Menengah Pertama dan Madrasah Tsanawiyah. Jakarta: Departemen
Pendidikan Nasional.
Sam, L. C., Lourdusamy, A., & Ghazali, M. (2001). Factors affecting students’ abilities
to solve operational and word problems in mathematics. Journal of Science and
Mathematics Education in Southeast Asia, 24(1), 84–95.
Scardamalia, M., Bransford, J., Kozma, B., & Quellmalz, E. (2012). New assessments
and environments for knowledge building. In P. Griffin, B. McGraw & E. Care
(Eds.), Assessment and teaching of 21st century skills (pp. 231–300). New York:
Springer.
Schleicher, A. (2007). Can competencies assessed by PISA be considered the
fundamental school knowledge 15-year-olds should possess? Educational Change,
8(4), 349–357.
Sepeng, P. (2013). Use of unrealistic contexts and meaning in word problem solving: A
case of second language learners in Township schools. International Journal of
Research in Mathematics, 1(1), 8–14.
Stipek, D. J., Givvin, K. B., Salmon, J. M., & MacGyvers, V. L. (2001). Teachers'
beliefs and practices related to mathematics instruction. Teaching and Teacher
Education, 17(2), 213–226.
Tomlinson, M. (2004). 14-19 Curriculum and Qualifications Reform. Final report of the
Working Group on 14-19 Reform. London: Department for Education and Skills.
Tornroos, J. (2005). Mathematics textbooks, opportunity to learn and student
achievement. Studies in Educational Evaluation, 31(4), 315–327.

123

Thesis Ari_141201_FINAL.pdf 123 1-12-2014 16:26:27

 Chapter 4

Toumasis, C. (1997). The NCTM standards and the philosophy of mathematics. Studies
in Philosophy and Education, 16(3), 317–330.
Tuge, C. (2008). Mathematics curriculum, the philosophy of mathematics and its
implications on Ethiopian schools mathematics curriculum. Ethiopian Journal of
Education and Science, 4(1), 109–120.
Verschaffel, L., Greer, B., & De Corte, E. (2000). Making sense of word problems. Lisse:
Swets & Zeitlinger.
Verschaffel, L., Van Dooren, W., Greer, B., Mukhopadhyay, S. (2010).
Reconceptualising word problem as exercises in mathematical modelling. Journal
fur Mathematik-Didaktik, 31(1), 9–29.
Wijaya, A., Van den Heuvel-Panhuizen, M., Doorman, M., & Robitzsch, A. (2014).
Difficulties in solving context-based PISA mathematics tasks: An analysis of
students’ errors. The Mathematics Enthusiast, 11(3), 555-584.
Wijaya, A., Van den Heuvel-Panhuizen, M., & Doorman, M. (submitted). Opportunity-
to-learn context-based tasks provided by mathematics textbooks and the relation
with students’ difficulties in solving these tasks.
Wilkins, J. L. M. (2008). The relationship among elementary teachers’ content
knowledge, attitudes, beliefs, and practices. Journal of Mathematics Teacher Education,
11(2), 139–164.
Xin, Z., Lin, C., Zhang, L., & Yan, R. (2007). The performance of Chinese primary
school students on realistic arithmetic word problems. Educational Psychology in
Practice, 23(2), 145–159.

124

Thesis Ari_141201_FINAL.pdf 124 1-12-2014 16:26:27

 Chapter 5

Opportunity-to-learn to solve context-based mathematics

tasks and students’ performance in solving these tasks –
lessons from Indonesia

Wijaya, A., Van den Heuvel-Panhuizen, M., Doorman, M., & Veldhuis, M. (submitted).
Opportunity-to-learn to solve context-based mathematics tasks and students’
performance in solving these tasks – lessons from Indonesia.

Thesis Ari_141201_FINAL.pdf 125 1-12-2014 16:26:27

 Chapter 5

Opportunity-to-learn to solve context-based mathematics

tasks and students’ performance in solving these tasks –
lessons from Indonesia

1 Introduction
The broad recognition of the importance of mathematics for coping with the
demands of the 21st century has led to an emphasis on developing students’ ability
to apply mathematics in a real-world context as an important goal of mathematics
education (Eurydice, 2011; Graumann, 2011; National Council of Teachers of
Mathematics [NCTM], 2000; Tomlinson, 2004). This ability is also the focus in the
framework used in the Programme for International Student Assessment (PISA) of
the Organisation for Economic Co-operation and Development (OECD, 2003).
However, notwithstanding the high importance attached to teaching students to
apply mathematics, there is a discrepancy between this intended goal and the
achievement of students. A number of studies (e.g., Cooper & Dunne, 2000; Sam
et al., 2001) have shown students’ low performance in solving mathematics tasks
that are situated in real-world contexts. In the PISA studies (OECD, 2010; 2013) it
was found that many students cannot solve real-world problems that require
reasoning skills and mathematical modeling of complex situations.

Also in Indonesia, the application of mathematics in real-world situations is

considered a relevant aspect of the mathematics curriculum (Pusat Kurikulum,
2003). Yet the results of the PISA studies show that Indonesian students perform
low on context-based tasks (e.g., OECD, 2013). This situation prompted us to set
up the Context-based Mathematics Tasks Indonesia (CoMTI) project. The aim of
this project was to identify ways to improve Indonesian students’ ability to apply
mathematics in a real-world context. We began the project with a study which was
aimed at clarifying the difficulties Indonesian students have when dealing with
context-based tasks. In this first study (Wijaya et al., 2014), we analyzed students’
errors in which we found that Indonesian students mainly have difficulties in
comprehending context-based problems and in transforming them into
mathematical problems. Our second study (Wijaya et al., submitted a) revealed that
one of the reasons for these difficulties might be that Indonesian textbooks do not
offer the opportunity-to-learn how to solve context-based tasks. Indonesian
textbooks contain a low number of context-based tasks. Furthermore, these
context-based tasks are mostly plain word problems, which use dressed-up
contexts, are often very explicit about what mathematical procedures have to be

126

Thesis Ari_141201_FINAL.pdf 126 1-12-2014 16:26:27

 OTL to solve context-based tasks and students’ performance

carried out, and provide precisely the information that is needed to solve the task.
Finally, in our third study (Wijaya et al., submitted b) classroom observations were
carried out to identify what opportunities teachers offer their students to learn to
solve context-based tasks. Here we noticed that Indonesian teachers mostly used
directive and teacher-centered teaching and did not give students opportunities to
actively get involved in and reflect on the process of solving context-based tasks.

In the present study, the findings of these three earlier studies were synthesized
and used to develop an intervention that offers students opportunity-to-learn to
solve context-based tasks. This intervention was put to the test in a field
experiment in which we investigated whether student performance can be
enhanced by giving students experience in solving context-based tasks that require
the selection of relevant information and an adequate mathematical procedure, and
by offering them a teaching approach that involves them actively in the process of
solving context-based tasks.

2 Theoretical background

2.1 Students’ difficulties when solving context-based tasks

Solving context-based tasks involves a complex process in which students pass
through several stages. According to Blum and Ferri (2009) and Blum (2011), this
process comprises four main stages. The first stage relates to the process of
comprehending what a context-based task is about. In the second stage students
need to look for a mathematical concept or procedure required to solve the task.
In this stage the real-world problem is transformed into a mathematical problem.
The third stage is carrying out the mathematical procedure to solve the
mathematical problem. Finally, in the fourth stage the mathematical solution
obtained in the third stage is interpreted in terms of the context of the task.

In each of these stages students can face difficulties and make errors. Research has
shown that in the first stage students often misunderstand the meaning of the tasks
and misinterpret the terms used in the tasks (Bernardo, 1999; Klymchuk et al.,
2010). In the second stage students are struggle to identify the mathematical
concept or procedure that is needed to solve the tasks (Clements, 1980; Klymchuk
et al., 2010). This difficulty relates to students’ tendency either to apply routine
mathematical procedure without realistic considerations (Verschaffel et al., 2000;
Xin et al., 2007) or to take too much account of the context of the tasks so that no

127

Thesis Ari_141201_FINAL.pdf 127 1-12-2014 16:26:27

 Chapter 5

mathematical concept or procedure is used (Boaler, 1994). In the third stage where
the mathematical processing is carried out, students can make errors in carrying
out mathematical procedures. At the end of the solving process, in the fourth
stage, students often have difficulties in interpreting a solution in terms of the
context, and give solutions that are not relevant to the context of the tasks (Greer,
1997).

2.2 Opportunity-to-learn to solve context-based tasks

An initial question that can be asked when students perform low on a particular
competence is whether they have received the necessary education to develop this
competence. To cover this question, in the 1960s, the concept of opportunity-to-
learn emerged. Researchers involved in comparative studies became aware that
when comparing the achievements of students in different countries, students’
opportunity-to-learn resulting from curricular differences had to be taken into
account (McDonnell, 1995). In the report of the First International Mathematics
Studies (Husén, 1967), opportunity-to-learn was defined as “whether or not […]
students have had the opportunity to study a particular topic or learn how to solve
a particular type of problem” (pp. 162-163). Presently, as a result of increasing
requests for accountability and higher achievement standards there is concern
whether students have access to the instruction required to reach these standards.
Therefore, there is a renewed interest for investigating students’ opportunity-to-
learn (Abedi & Herman, 2010). This can generally be done in two ways: by
examining the instructional resources such as textbooks and by surveying the
instructional strategies employed by teachers (Brewer & Stasz, 1996; Grouws &
Cebulla, 2000; Herman et al., 2000).

2.2.1 Opportunity-to-learn offered by textbooks

Textbooks are considered as the main instructional material for teachers (Brewer &
Stasz, 1996), and mediate between the intended and the implemented curriculum
(Valverde et al., 2002). Research has shown that textbooks have a strong influence
on students’ learning (Schmidt et al., 1997; Tornroos, 2005). Therefore, what is
offered in textbooks can be regarded as an important measure for students’
opportunity-to-learn. In this respect several aspects of textbooks may be taken into
account. First, the degree of exposure to particular content included in a textbook
can influence students’ performance. Tornroos (2005) found a relation between
student achievement on a test and the amount of textbook content related to the
test items. Second, the method used in textbooks to help students understand the

128

Thesis Ari_141201_FINAL.pdf 128 1-12-2014 16:26:27

 OTL to solve context-based tasks and students’ performance

content is also an important aspect. As found by Xin (2007), students tend to solve
word problems by using the solution strategies suggested in the textbooks. Third,
students’ opportunity-to-learn is also determined by the characteristics of the tasks
in textbooks. In this respect, Charalambous et al. (2010) mention the cognitive
demands of tasks and the required types of responses in tasks. For example, it
makes a difference whether students can build up experience with tasks in which
they just have to recall facts and procedures, or that they are offered tasks in which
they have to model a problem situation or reflect on a solution. With respect to
opportunity-to-learn to solve real-world problems, Maass (2007) highlighted the
importance of giving students tasks that have superfluous and missing information.
Such tasks are necessary to direct students to pay attention to the context of the
tasks and to teach students to distinguish between relevant and irrelevant
information. Furthermore, it is also essential to provide students with real-worlds
problems that do not provide explicit suggestions about the required procedures to
develop students’ modeling competence (Maass, 2010).

2.2.2 Opportunity-to-learn offered by teachers’ instructional strategies

In addition to research that shows the influence of textbooks on students’ learning,
there are also many studies (e.g. Eurydice, 2011; Grouws & Cebulla, 2000; Hiebert
& Grouws, 2007) that have highlighted that student performance is affected by the
instructional strategies that teachers use. How teachers teach mathematics and
engage their students influences how well students learn. With respect to the
teaching of context-based tasks, several researchers (Antonius et al., 2007; Blum,
2011; Forman & Steen, 2001) suggested teachers using a teaching approach in
which they take a consultative role and give students opportunities to actively build
new knowledge and reflect on their learning process. Such consultative teaching
offers students opportunity to develop strategic competences that are necessary to
solve context-based tasks, such as understanding the text of problems, selecting
relevant information, identifying appropriate mathematical procedures, and
interpreting solutions. A key aspect of consultative teaching is to keep a balance
between teacher guidance and students’ independence. Both Antonius et al. (2007)
and Blum (2011) recommended the use of metacognitive prompts to create this
balance and Montague (2007) emphasized that these metacognitive prompts help
students to become active learners.

Metacognitive prompts can be provided in the form of self-addressed questions;

i.e. students are asked to question themselves while solving a problem. Self-
addressed questions are an important stimulus to help students regulate their

129

Thesis Ari_141201_FINAL.pdf 129 1-12-2014 16:26:27

 Chapter 5

solving process and reflect on it (Kramarski et al., 2002; Montague, 2007; 2008).
Another kind of metacognitive prompt is giving a verbal prompt or instruction to
help students focus attention on particular aspects of the solving process and to
assist them in carrying out the solving process (Goldman, 1989; Montague, 2007;
Montague et al., 2000). For example, the instruction to underline the important
information in a task (see e.g. Montague, 2007; 2008) can be used to guide students
to focus on identifying relevant information. Asking students to paraphrase a task
is also an important prompt. According to Karbalei and Amoli (2011) and Kletzien
(2009), students who explain in their own words what the task is about, gain a
better understanding of the task while doing this.

3 Results from our previous studies and the research question

3.1 Indonesian students’ difficulties when solving context-based

tasks
To investigate students’ difficulties when solving context-based tasks we (Wijaya et
al., 2014) analyzed the errors made by students. Four error types – comprehension,
transformation, mathematical processing, and encoding – were distinguished by
associating Newman’s (1977) error categories with Blum and Leiss’ (2007)
modeling process and PISA’s (OECD, 2003) mathematization stages.
Comprehension errors refer to students’ inability to understand a context-based
task, which includes the inability to select relevant information. Transformation
errors correspond to students’ failure in identifying the mathematical procedure
required to solve a task. Mathematical processing errors are related to mistakes in
carrying out mathematical procedures; for example, errors in calculating, solving
algebraic formulae, or interpreting a graph. Encoding errors refer to answers that
are unrealistic and do not match the context of the task.

Our error analysis revealed that 38% of Indonesian students’ errors when solving
context-based tasks were comprehension errors. Students often misunderstood the
question of a context-based task. Students also made remarkable errors in selecting
relevant information in which they tended to use all information given in the text
without considering the relevance of this information. Transformation errors were
found in 42% of students’ errors. Many students used the wrong procedure or
concept when solving a context-based task in which they tended to use a familiar
procedure. Regarding mathematical processing errors which are not directly related
to the context-based character of the tasks, but rather to the mathematical topic

130

Thesis Ari_141201_FINAL.pdf 130 1-12-2014 16:26:27

 OTL to solve context-based tasks and students’ performance

addressed in the tasks, we found that 17% of all errors were of this type. In our
study, students frequently made errors in interpreting graphs and in dealing with
algebraic formulas or expressions. Finally, encoding errors were found to be in the
minority, with only 3% of students’ errors belonging to this type. An example of
this error type is students giving 70 meter as the pace length of a human.

3.2 Opportunity-to-learn offered in Indonesian textbooks

To find possible explanations for Indonesian students’ difficulties when solving
context-based tasks, we first investigated the opportunity-to-learn to solve context-
based tasks as offered in Indonesian textbooks. For this investigation we carried
out a textbook analysis to get an overview of the characteristics of the tasks
included in the textbooks (see Wijaya et al., submitted a). The framework we
developed for this analysis contained four analysis entries. First, we examined the
type of context used in a task, for which we distinguished three types of context:
no context, camouflage context, and relevant and essential context. Second, a
further analysis was done for the context-based tasks to identify the purpose of the
tasks; which could be application or modeling. Third, we investigated the type of
information that was provided in a task. Three types of information were
considered: matching, missing, and superfluous information. The fourth
characteristic was the cognitive demand of a task. For this the tasks’ types of PISA
(OECD, 2003) were taken into account: reproduction, connection, and reflection.

Our textbook analysis revealed that only 10% of the tasks in the textbooks were
context-based tasks. When zooming in on the characteristics of these context-
based tasks, we found that three quarters of them used a camouflage context,
which means that the context can be neglected when solving the task and that the
procedures are explicitly provided. It also means that the students did not have to
think about transforming the task into a mathematical problem. Furthermore, this
lack of stimulus for transformation was amplified by the fact that all the context-
based tasks were given after the explanation sections in which a particular concept
or procedure was explained. With respect to the type of information, 85% of the
context-based tasks contained matching information, meaning that only the
relevant information was provided. Lastly, only 2% of the context-based tasks in
Indonesian textbooks were reflection tasks, i.e. tasks with the highest cognitive
demands that require constructing original mathematical approaches and
communicating complex arguments and reasoning.

131

Thesis Ari_141201_FINAL.pdf 131 1-12-2014 16:26:28

 Chapter 5

Combining these findings with our findings regarding the types of students’ errors
strongly indicates that both sets of findings are related. First of all, when only 10%
of the tasks in Indonesian textbooks are context-based one cannot expect
Indonesian students to be good at solving such tasks. Also, the characteristics of
the tasks included in the textbooks are clearly reflected in the students’ errors. For
example, the lack of opportunity students that get in dealing with superfluous and
missing information can be recognized in the large number of errors Indonesian
students made in selecting information. Similarly, the high percentage of students’
transformation errors corresponds to the low number of context-based tasks in
textbooks in which the students had to figure out the mathematical procedure by
themselves.

3.3 Opportunity-to-learn offered by the teaching practices of

Indonesian teachers
Our next step was to investigate to what degree students are offered opportunity-
to-learn to solve context-based tasks by Indonesian teachers in their teaching
practice. For this, we collected data about this teaching practice through a written
questionnaire and classroom observations (see Wijaya et al., submitted b). In the
questionnaire the focus was on the characteristics of context-based tasks that were
offered by teachers. It was found that 18 of the 27 Indonesian mathematics
teachers who were involved in our study gave context-based tasks with explicit
procedures in every lesson or weekly. The same number of teachers gave context-
based tasks with matching information in every lesson or weekly. Regarding
context-based tasks with superfluous information, 11 of the 27 teachers reported
that they never gave such tasks or did so once or twice in a semester. Context-
based tasks with missing information were never given or given only once or twice
in a semester by 9 of the 27 teachers.

To examine in more detail the teaching practice, observations were done in four
classrooms. As studies by Anthonius et al. (2007) and Blum (2011) highlighted that
for teaching mathematical modeling a consultative teaching practice is more
helpful than a directive teaching practice we took this distinction as the perspective
of our classroom observations. In consultative teaching a teacher plays a
consultative role and involves students to actively carry out and reflect on the
process of solving context-based tasks, while in directive teaching a teacher mainly
tells students what they should do to solve a context-based task. Our observations
covered in total eight lessons carried out in four classrooms of four teachers.

132

Thesis Ari_141201_FINAL.pdf 132 1-12-2014 16:26:28

 OTL to solve context-based tasks and students’ performance

During these observations a total of 15 questions was discussed in class, but we

found consultative teaching for only two questions. Directive teaching in which the
teachers directly told the students what the context-based task was about, or
explained what procedure was needed to solve the task was observed in 7 of the 15
questions. In addition, it was also observed that on some occasions the teachers
did not give attention to a particular stage of the solving process. This happened in
6 of the 15 questions, and mostly applied to the encoding stage in which the
teachers only focused on the correctness of students’ mathematical solutions
without connecting the solutions to the context of the task.

3.4 Research questions

The aim of the present study was to test whether students’ performance in solving
context-based tasks can be improved by offering them the opportunity-to-learn
these tasks. The ingredients for creating this opportunity-to-learn were derived
from what we found in research literature and our three earlier studies. The
opportunity-to-learn comprised context-based tasks with particular characteristics,
which were found to be lacking in Indonesian textbooks, and a consultative
teaching approach that is considered to be supportive in developing students’
ability to solve context-based tasks. To investigate whether offering students this
opportunity-to-learn is an effective way to improve their ability to solve context-
based tasks we looked into the effect of this opportunity-to-learn on both the
correctness of students’ answers and the errors made by them. This led to the
following research question:

Does providing students with an opportunity-to-learn to solve context-based tasks

contribute to students’ performance when solving these tasks and, more specifically, is
there any effect on the correctness of the answers and on the types of errors?

4 Method

4.1 Design of the study

To answer the research question, we carried out a field experiment with a pretest-
posttest control-group design. In the experimental group, the teachers used an
intervention program for their teaching, intended to offer students opportunity-to-
learn to solve context-based tasks. In the control group the students followed a

133

Thesis Ari_141201_FINAL.pdf 133 1-12-2014 16:26:28

 Chapter 5

teaching program that was developed on the basis of the textbook that they
regularly use.

4.2 Participants
The study took place in six junior high schools located in the province of
Yogyakarta, Indonesia. This province was chosen for reasons of convenience; the
first author originates from this part of the country. In every school an
experimental class and a control class were selected. In four schools there was only
one teacher for each grade; therefore in these schools one teacher taught both the
experimental and the control class. In the other two schools the experimental class
and the control class were taught by different teachers. We left the decision about
the allocation of these teachers to one of the two conditions to the school
principal. All eight participating teachers had a bachelor degree in mathematics
education and considerable teaching experience, ranging from 5 to 32 years
(M = 18.9 years; SD = 9.9 years). In each school several textbooks were in use, but
all schools also had one textbook in common, which was Matematika (Textbook
for Junior High School, Grade VIII: 2A & 2B).

In total, in the six schools 311 eight graders (M = 13.8 years; SD = 0.5 year)
participated in the study, involving 146 students in the experimental group and 165
in the control group. However, the data analysis was based on the 299 students
(M = 13.7 years; SD = 0.5 year) who were present during both the pretest and the
posttest. Of these students, 144 students were in the experimental group and 155
students in the control group.

4.3 Intervention program

4.3.1 The components of the intervention

The purpose of the intervention was to offer students opportunity-to-learn to
solve context-based tasks. The intervention program was designed based on the
findings of our earlier studies and comprised two components: a set of context-
based tasks with particular characteristics and a consultative teaching approach
with metacognitive prompts (see Figure 1). This intervention is illustrated using
one of the context-based tasks used in the intervention program: the Internet task
(see Figure 2).

134

Thesis Ari_141201_FINAL.pdf 134 1-12-2014 16:26:28

 OTL to solve context-based tasks and students’ performance

Figure 1. Intervention program informed by the findings of our previous studies

135

Thesis Ari_141201_FINAL.pdf 135 1-12-2014 16:26:28

 Chapter 5

Figure 2. Internet task

4.3.2 Context-based tasks

To compensate what was lacking in Indonesian textbooks the intervention
program offered students context-based tasks that had three characteristics. The
first characteristic was that the context-based tasks should have relevant and essential
contexts that require modeling. The tasks should be a real problem for students. For
example, in the Internet task students encounter a problem that is rather authentic
to them. Figuring out what internet fee Doni has to pay is a meaningful context for
students because there is really something at stake. Therefore students are
stimulated to take the context into account when solving the task instead of just
using the numbers involved to carry out more or less automatically a particular
calculation.

The second characteristic concerned the type of information provided in the

context-based tasks. To offer students opportunity-to-learn selecting relevant
information, context-based tasks were designed with missing or superfluous information,
i.e. tasks that have less or more information than needed to find the solution. For
example, to decide how much Doni has to pay in May, students do not need to use

136

Thesis Ari_141201_FINAL.pdf 136 1-12-2014 16:26:28

 OTL to solve context-based tasks and students’ performance

the monthly fee for the program Smile (30,500 IDR), the fee per 1 MB (40 IDR),
or the registration fee (300,000 IDR).

The third characteristic required for the context-based tasks was that they do not
contain apparent indications about the procedures that can be used to solve the problem. If
explicit information is given about what procedure to apply, then students are not
offered opportunity-to-learn to decide what would be a suitable mathematical
procedure. Therefore, we designed the Internet task so that students are not put
directly on the track of starting with a particular mathematical procedure, such as
making a calculation, formulating and solving an equation, or drawing a graph.
Instead, students are just asked to decide whether it is better for Doni to change
the internet program and, if yes, when it is better to change. This means that
students have to come up with a procedure by themselves.

4.3.3 Consultative teaching approach

The second component of the intervention program was a consultative teaching
approach. To conduct this approach, we provided the teachers with suggestions to
give metacognitive prompts to students and to stimulate whole-class and peer-to-
peer discussions that promote reflection for all the stages of solving context-based
tasks.

The metacognitive prompts were meant to point students to important aspects of the
tasks and the solving process. A first metacognitive prompt was asking students to
underline all the information included in a context-based task and to discuss the
included information. For example, in the Internet task this prompt was given in
the first assignment. As a second metacognitive prompt, students were asked to
use their own words to explain the Internet task. This paraphrasing of a given
problem was a second type of metacognitive prompt to help students to get a
better understanding of what the problem is about. Finally, a third type of
metacognitive prompt was to elicit self-questioning. For example, in the Internet
task students were stimulated to ask themselves questions, such as “What strategy
can we use to solve Doni’s problem?”.

To make the metacognitive prompts an integral part of the intervention program

they were embedded in the context-based tasks, yet the special character of these
prompts was indicated by printing them in italic. In this way the students (and
teachers) were made aware that the questions differed from the mathematical
questions. In the first two lessons metacognitive prompts were provided for every

137

Thesis Ari_141201_FINAL.pdf 137 1-12-2014 16:26:28

 Chapter 5

question in the context-based tasks, but in the later lessons the prompts gradually
faded out.

The suggestions for discussions to promote reflection were not embedded in the tasks, but
were only included in the teacher guide. This means that suggestions were
provided for additional instructions or guiding questions. For example, when
students had difficulties with only circling the relevant information in the Internet
task, the teacher could suggest students moving away from the question they had
to answer in this task (“How much money did he pay in May?”) and starting with
identifying the keywords of this question. Based on these keywords students can
look for relevant information in the remaining text.

4.3.4 Outline of the intervention program

The complete intervention program consisted of five 80-minutes lessons each
consisting of an explanation section and an exercise section. In fact, the actual
intervention took place in the exercise sections, which each lasted about 20 to 30
minutes. The intervention was conducted from September 18, 2013 to October 12,
2013. As all schools in Indonesia follow the national curriculum, during this time
period the teachers taught graphical representation of linear equations. Therefore
we chose this topic in designing the intervention program. An outline of the
intervention program is given in Appendix 1. It gives an overview of the sub-
topics of graphical representation of linear equations which were dealt with in the
five lessons, the mathematical goals addressed in each lesson, the goals related to
learning solving context-based tasks, and the context-based tasks that were
included in the program. In total, the program included nine context-based tasks;
in every lesson two and in the last lesson one. Moreover, in order not to lessen too
much the experience of students in solving tasks with bare mathematics, the lesson
series also contained five of such tasks.

4.3.5 Teacher training

Two weeks prior to the intervention, the experimental teachers were trained to
conduct the intervention. The teacher training was done per school because the six
schools were located in different districts and the teachers were busy with their
regular job, so that it was difficult to gather the teachers in one training. The
training consisted of two meetings of 90 to 120 minutes. During these meetings,
the overall goal of the intervention program, the structure of the program, and the
material belonging to this program (student material and teacher guide) were
explained. After this, attention was paid to the importance of the competence of

138

Thesis Ari_141201_FINAL.pdf 138 1-12-2014 16:26:28

 OTL to solve context-based tasks and students’ performance

solving context-based tasks and the characteristics of context-based tasks. Finally,

examples were discussed of how teachers could help students to deal with
superfluous or missing information and select relevant information, how teachers
could give metacognitive prompts, and how to stimulate students to discuss and
reflect on the solving process.

4.3.6 Fidelity of the implementation of the intervention program

To monitor whether the teachers conducted the intervention in the intended way,
classroom observations were carried out by the first author. Because the
intervention in the six schools took place in the same time period, not all lessons
could be observed. In general, two of each teacher’s five lessons were observed
and for one teacher observations were made in four lessons. After each observed
lesson, a discussion was held with the teacher to reflect on how the intervention
was carried out and to prepare for the intervention in the next lesson. For the
lessons that were not observed the teachers were asked to keep a log and report
how many tasks they discussed in class.

The observations and the teachers’ reports gave the impression that with respect to
the consultative teaching in general the teachers did the intervention according to
the plan. However, for the exposure of context-based tasks, on average the
teachers discussed only six out of the nine tasks that we gave them. Furthermore,
the observations revealed that during the lessons the teachers frequently read the
teacher guide before giving a metacognitive prompt or stimulating discussion and
reflection.

4.3.7 Regular program

Students in the control classes were taught as they were usually taught on the basis
of the textbook that the six schools had in common. This means that the teachers
were using a teacher-centered approach in which they mainly explained and
demonstrated how to solve tasks. In order to make the mathematics content the
same in all the control classes, the control teachers were asked to follow a program
consisting of 19 bare mathematics tasks and 3 context-based tasks. All these tasks
were taken from the common textbook used by the teachers. Consequently, the
context-based tasks all used a camouflage context and explicitly mentioned the
mathematical concepts related to the task. For example, the two tasks shown in
Figure 3 include a staircase and a ski slope, which are real-world contexts, yet
students do not have to think about a staircase and a ski slope in reality. They can
just do the calculation that is asked for, based on the figure of the staircase and the

139

Thesis Ari_141201_FINAL.pdf 139 1-12-2014 16:26:28

 Chapter 5

numbers provided in the picture of the mountainside (e.g. slope ൌ  ସହ

ସ଼
ൌ ͲǤͻ).
Furthermore, these two tasks explicitly mention the mathematical concept
gradient, which is relevant to solve these tasks.

Figure 3. Context-based tasks included in the regular program

4.4 Test for measuring students’ performance in solving context-

based tasks

4.4.1 Composition of the test

The test used for measuring students’ performance in solving context-based tasks
was made from released PISA tasks (OECD, 2009b). We selected a total of 17
PISA mathematics units9 consisting of 30 items. The 17 mathematics units were
divided over two different booklets. Booklet A contained eight mathematics units
(consisting of 15 items) and Booklet B contained nine mathematics units (also
consisting of 15 items). The items were equally distributed over the two booklets
according to: (1) the cognitive demand of the items as established in the PISA
studies, including reproduction, connection, and reflection (see OECD, 2003;
2009b), (2) the difficulty level of the items as indicated by the percentage of correct
answers found in the 2003 PISA study (OECD, 2009a), and (3) the mathematical
topics involved in the items. Although the intervention involved the topic of
graphical representation of linear equations, only 5 out of 30 items included in the
test were related to graphs, including non-linear graphs.

9A ‘mathematics unit’ consists of one or more questions that can be answered independently (see OECD,
2003, p. 52). These questions are based on the same context which is generally presented by a written
description and/or a graphic or another representation.

140

Thesis Ari_141201_FINAL.pdf 140 1-12-2014 16:26:28

 OTL to solve context-based tasks and students’ performance

To avoid a re-test effect due to administering two times the same items, the group
of students in each class was randomly split in half, leading to two groups of which
one got Booklet A as a pretest and Booklet B as a posttest, with the other group
getting these booklets in the reverse order.

4.4.2 Coding of the correctness of the answers

To code students’ responses the scoring scheme of the PISA studies (OECD,
2009b) was used. Of the total of 30 items, 24 items were coded as correct (1),
incorrect (0) or no answer (9). The other items had a partial credit scoring,
including 5 items that were coded as correct (2), partially correct (1), incorrect (0)
or no answer (9) and 1 item that was coded as correct (3), partially correct level 2
(2), partially correct level 1 (1), incorrect (0) or no answer (9). Based on this coding,
we calculated the students’ scores in the two booklets. The maximum score for
Booklet A was 18 and the maximum score for Booklet B was 19.

4.4.3 Psychometric properties of the test

To check the reliability of the two sets of items, which formed our test for
measuring students’ performance in solving context-based tasks, we calculated
Cronbach’s alpha. For the complete test, i.e. booklet A and B combined, this gave
a good ơ of .80. For the booklets separately we found ơ = .75 for booklet A, and
ơ = .69 for booklet B. As the complete sample was split into two groups and each
group got a different booklet as pretest and posttest, we also checked whether the
reliability estimates per booklet changed for whether it was used as a pretest or as a
posttest. Cronbach’s alpha for booklet A as pretest was ơ = .64 and as a posttest
ơ = .79, for booklet B as a pretest ơ = .68 and as a posttest ơ = .69. These changes
are relatively small and all alphas indicate acceptable to good reliabilities.

4.4.4 Coding of the errors made by students

To code students’ errors when solving the test items we used an analysis
framework that was developed in our earlier study (Wijaya et al., 2014) (see
Appendix 2). The coding was done for students’ an incorrect answer. Furthermore,
more than one code could be given to a student’s work because students could
make more than one type of error.

The coding was carried out by the first author. The interrater reliability of the
coding was checked through an extra coding by a mathematics teacher who was
not part of this study. The extra coding was done on the basis of 12% of the coded
responses, which were randomly selected. With a Cohen’s Kappa of .78, the

141

Thesis Ari_141201_FINAL.pdf 141 1-12-2014 16:26:28

 Chapter 5

agreement between the first author and the second coder was substantial (Landis &
Koch, 1977).

4.4.5 Statistical analysis

To investigate whether students in the experimental condition improved more than
those in the control condition we used a univariate analysis of variance (ANOVA).
We calculated a gain score for every student to use as dependent variable in the
ANOVA. In order to be able to compare students’ gain scores, taking into account
the different total scores they could obtain on the two booklets (A and B) they got
as pretest or posttest, we standardized the scores per booklet and per order of
presentation of the booklets. So the mean of all students that got booklet A as a
pretest was used to standardize the scores of the students who were in this group,
and for the students who got booklet B as a pretest the mean of all students who
were in this group was used for standardizing the scores. This same procedure was
repeated for the scores on the posttest. The obtained standardized scores were
used to calculate a standardized gain score for every student (“score on A – score
on B” for B as pretest and A as posttest; and “score on B – score on A” for A as
pretest and B as posttest).

5 Results

5.1 Effect of opportunity-to-learn on the correctness of students’

answers
To investigate the effect of the intervention program we carried out a univariate
ANOVA with the gain score (posttest score minus pretest score) as dependent
variable and intervention as a fixed factor. Contrary to our expectations, the
difference in gain scores between the students in the experimental group
(Mexperimental = 0.11, SDexperimental = 0.99) and the students in the control group
(Mcontrol = -0.09, SDcontrol = 0.95) was only marginally significant and the effect of the
intervention was small (p = .068; Ƨp2 = .011).

We also took an exploratory look at the effect of the school, finding a significant
main effect for the school students were in (p < .001, Ƨp2 = .114). In Figure 4 the
different gains for the students in the control and experimental condition in the six
schools are displayed.

142

Thesis Ari_141201_FINAL.pdf 142 1-12-2014 16:26:28

 OTL to solve context-based tasks and students’ performance

Figure 4. Mean gain scores for students in the control (white) and experimental (gray)
condition in the six schools

Furthermore, we also examined exploratorily whether there was an effect of the

booklet order, and whether there was a relation between the school and the order
in which the booklets were presented. The booklet order by itself did not turn out
to have an effect on the students’ gain scores (p = .884, Ƨp2 = .000), but for the
school and the booklet order we observed a significant interaction effect (p = .004,
Ƨp2 = .06). Depending on the school the students were in, they had either more
difficulty in solving the items in booklet A or those in booklet B (see Figure 5). For
example, in school S both groups showed a gain in performance when the booklet
order was A-B but not when it was B-A. For this school, booklet B was easier than
booklet A, whereas in school PR the opposite was the case. This difference in
difficulty level of the booklets is remarkable because they were constructed in such
a way that we expected them to be of comparable difficulty.

143

Thesis Ari_141201_FINAL.pdf 143 1-12-2014 16:26:28

 Chapter 5

Figure 4. Mean gain scores for students in the control (white) and experimental (gray)
condition in the six schools for both booklet orders

5.2 Effect of opportunity-to-learn on students’ errors

To investigate whether the intervention had an effect on the types of errors, we
carried out an error analysis based on the students’ incorrect responses. In total, we
found 1942 incorrect responses in the pretest (892 in the experimental group and
1050 in the control group) and 1705 incorrect responses in the posttest (744 in the
experimental group and 961 in the control group). In accordance with our finding
on the correctness of the students’ answers the decrease in incorrect responses was
not significant (Ʒ² (1, n = 3647) = 1.934, p = .164).

Table 1 presents the results of the error analysis. Looking at the changes in the
total number of errors between the experimental group and the control group we
found that the decrease in the number of errors made by the students in the
experimental group was about ten percent points larger than in the control group:
in the experimental group the decrease in errors was 18%, whereas in the control
group this was 7%. This showed that there was a significant difference between the
experimental group and the control group on the decrease in the total number of
errors (Ʒ² (1, n = 4127) = 4.149, p = .042).

144

Thesis Ari_141201_FINAL.pdf 144 1-12-2014 16:26:28

 OTL to solve context-based tasks and students’ performance

Table 1
The number of errors made by the students in the pretest and the posttest
Number of
Types of errors Percent of
Sub-types of errors Group
errors Pre- Post- change
test test
Compre- Errors in understanding Exp. 68 51 - 25% a
hension instruction Control 84 93 11%
Errors in understanding Exp. 22 36 64%
a keyword Control 14 39 179%
Errors in selecting Exp. 125 86 - 31%
information Control 125 127 2%
Exp. 215 173 - 20%
Total
Control 223 259 16%

Transfor- Exp. 7 15 114%

Procedural tendency
mation Control 2 9 350%
Taking too much Exp. 11 8 - 27%
account of context Control 20 10 - 50%
Wrong mathematical Exp. 487 376 - 23%
procedure Control 582 442 - 24%
Treating a graph as a Exp. 68 59 - 13%
picture Control 69 87 26%
Exp. 573 458 - 20%
Total
Control 673 548 - 19%

Math. Exp. 195 165 - 15%

processingb Control 230 239 4%

Encodingc Exp. 32 33 3%
Control 58 53 - 9%

Totald Exp. 1015 829 - 18%

Control 1184 1099 - 7%
a A negative value means a decrease
b The sub-types of mathematical processing error depend on the mathematics topics addressed in
the test items
c No sub-type for encoding error
d Because of a multiple coding, the number of errors were greater than the number of incorrect
responses.

145

Thesis Ari_141201_FINAL.pdf 145 1-12-2014 16:26:28

 Chapter 5

Regarding comprehension errors the intervention had a positive influence. In the

experimental group the number of errors decreased by 20%, whereas in the control
group the occurrence of these errors increased by 16%. Particularly, the finding
that there was a reduction of 31% in the number of selecting information errors
and of 25% in the number of understanding the instruction errors, provides
evidence that the metacognitive prompts helped. This was also supported by the
fact that in the posttest work of the students in the experimental group showed
clear signs of underlining and circling information. This was, for example, the case
in the Skateboard task in which students were asked to calculate the minimum and
the maximum price for self-assembled skateboards. This task provided a price list
that included irrelevant information. A typical error when solving this task was that
students included irrelevant information in their calculation. In the pretest this was
done by 19 out of 72 students in the experimental group and by 21 out of 80
students in the control group. In the posttest this error decreased to 6 in the
experimental group and to 12 in the control group.

For the transformation errors the results were different. Here, for both the
students in the experimental and the control group there was on average a decrease
of 20% in the number of errors, but at item level the change in errors differed
between the two groups. For example, in a follow-up question in the Skateboard
task students were asked to determine the number of different skateboards that
can be assembled based on a number of different skateboard components. The
transformation error that most students made was to simply add up the number of
components. This error decreased by 20% in the control group, but by only 3% in
the experimental group. However, for other items we found that for students in
the experimental group, in contrast with the students in the control group, the
number of transformation errors in the posttest was remarkably smaller than in the
pretest. This appeared to be the case for context-based tasks addressing the
interpretation of graphs. For example, in the Speed task in which a graph of the
varying speed of a racing car that drives along a flat 3-kilometer-track is presented,
students were asked to choose from five alternative race tracks along which the car
could have produced the speed graph that was shown. A usual transformation
error for this task was treating a graph as a literal picture of a situation. In the
experimental group the occurrence of this error decreased by 17%, whereas in the
control group it stayed the same. This result can be explained by the fact that in
the intervention program less explicit procedures were given for solving tasks
related to graphs than in the control group. In the regular program the tasks were
mostly cast in a particular mathematical language by which the students were

146

Thesis Ari_141201_FINAL.pdf 146 1-12-2014 16:26:28

 OTL to solve context-based tasks and students’ performance

immediately directed to the procedure of using linear equations to create tables and
use the numbers in the tables to draw graphs (with x horizontally and y vertically).
Consequently, students in the experimental group had had more opportunities to
learn to interpret and reason about graphs in context-based tasks.
Similar to comprehension errors, for mathematical processing errors the number
of errors decreased in the experimental group while it increased in the control
group. When taking a closer look at the sub-types of the mathematical processing
errors, we found that the largest difference between the two groups was in
arithmetical errors: in the posttest the number of arithmetical errors in the
experimental group barely changed, whereas in the control group the number
increased by 70%. Lastly, regarding the encoding errors we found only a small
number of errors and in both groups the number of these errors hardly changed
between pretest and posttest.

6 Discussion
Although our analysis showed that the students in the experimental group made
more progress than the students in the control group, the difference between the
two groups was only marginally significant and the effect was small. This result
might be due to the short duration of our intervention, which contained only five
30-minute lessons over a two-week period. In such a short time the students might
not have internalized the metacognitive prompts. As highlighted by Veenman et al.
(2006), prolonged training is fundamental for a successful metacognitive
instruction. Furthermore, because the teachers only offered and discussed six of
the nine tasks we provided, the students might not have received enough chances
to deal with the context-based tasks. Finally, following Kramarski et al. (2002),
who indicated that teacher experience in metacognitive instruction influences
student performance, the Indonesian teachers experience with this type of
instruction might have played a role. Several studies have shown that directive
teaching is still the dominant approach in mathematics lessons in Indonesia
(Maulana et al., 2012; Wijaya et al., submitted b; World Bank, 2010).

Based on a closer examination of students’ errors we found a positive effect of our

intervention on students’ ability to comprehend context-based tasks. This finding,
which is in agreement with other studies (e.g., Karbalei & Amoli, 2011; Kletzien,
2009), indicates the potential of the paraphrasing strategy to develop students’ task
comprehension. The improvement in students’ task comprehension was also

147

Thesis Ari_141201_FINAL.pdf 147 1-12-2014 16:26:28

 Chapter 5

reflected in the progress students made in selecting information. This result

signifies the power of context-based tasks with missing or superfluous information
to develop students’ ability to select information. Furthermore, we would also like
to highlight the benefit of asking students to circle only the relevant information.
This metacognitive prompt seems to be effective in guiding students to thoroughly
look at the information provided in the task.

Despite the promising result regarding task comprehension, in general no effect of

the intervention program on students’ performance in transforming a real-world
problem to a mathematical problem was found. A possible explanation for this
finding is that the intervention took place in teaching a particular topic. The
students might already have been put on the track of the procedure or concept that
is needed to solve the context-based tasks. Therefore, the students in the
experimental group might not have had enough chances to deal with identifying
the needed procedure. Also, Howson (2013) underlined recently that including
context-based tasks in a particular chapter of a textbook will discourage students
from thinking about the required procedure before they solve the tasks.

Because of limited financial resources available for this study there were
restrictions in the way we could set up our research. This resulted in some
limitations that should be taken into account in the interpretation of our findings.
A first limitation is that we could not include a large number of schools to achieve
sufficient statistical power. Furthermore, because of the limited number of schools
that we could take on board, we could not compose two groups of matched
schools followed by a random allocation to the experimental or the control group.
In our design, the two conditions were situated within every school. However,
most schools had only one teacher for each grade, which means that in these
schools the experimental students and the control students were taught by the
same teacher. This is not an ideal situation, because what the teachers did in the
experimental group might have had an influence on their teaching practices in the
control group. Another shortcoming of the study is that we could not include a
measure of the students’ general achievement level in mathematics in the analysis,
because the different districts where the schools were located administered their
own tests for Grade 8, which made the scores of students in different schools not
comparable.

These limitations ask for further research to get a more thorough understanding of
whether and how students’ ability in solving context-based tasks can be improved

148

Thesis Ari_141201_FINAL.pdf 148 1-12-2014 16:26:28

 OTL to solve context-based tasks and students’ performance

by offering them a particular opportunity-to-learn. Such research should include a

large sample of schools and teachers. In addition to our study, new research should
also include data on the quality of the schools, the teachers’ teaching experience
and the students’ achievement level in mathematics. All these factors might have
an influence on the effect of an intervention program aimed at improving students’
performance in context-based tasks.

7 Conclusion and policy implications

In Indonesia great attention is paid to developing students’ ability to apply
mathematics. This is reflected not only in the Indonesian national curriculum, but
also in Indonesia’s regular participation in the PISA study and in the nationwide
educational movement implementing Realistic Mathematics Education (see e.g.,
Sembiring, Hadi, & Dolk, 2008). With respect to the application of mathematics as
an educational goal, we would like to highlight two relevant recommendations
based on the findings of our study. First, our results showed the need for a
textbook quality improvement program. Because textbooks are the primary
resources in the Indonesian classroom practice and are found to be of influence on
what students learn, textbooks give the Indonesian government a powerful steering
instrument for improving students’ achievement in the application of mathematics.
Through the publication of freely available (electronic) textbooks the government
can reach all teachers, and offer students more opportunity-to-learn. However, this
necessitates that textbook authors include more context-based tasks in textbooks,
especially more tasks that have superfluous or missing information, and more tasks
with non-explicit procedures. With respect to the latter we also advice to revise the
chapter structure of textbooks somewhat by including in the exercise sections not
only context-based tasks that are related to the topic discussed in the chapter, but
also context-based tasks that refer to topics discussed in other chapters. Of course,
commercial publishers are also invited to improve their textbooks in the same
direction.

Our second recommendation is related to one of the quality requirements of the

Teacher Law set by Indonesian government, i.e. the need for higher standards in
classroom teachings, in which teachers are required to engage students in the
learning process (Jalal et al., 2009). Based on our experiences with the consultative
teaching approach (giving metacognitive prompts and room for discussion and
reflection) as included in the intervention program, we found that such an
approach is helpful to get students involved in the learning process of solving

149

Thesis Ari_141201_FINAL.pdf 149 1-12-2014 16:26:28

 Chapter 5

context-based tasks. Moreover, it turned out to be also feasible for teachers to

carry out as an alternative for their teaching practices. However, as was the case in
our project, professional development will be necessary to guarantee an
appropriate implementation and a widespread and prolonged practice of
consultative teaching. Therefore, we suggest including consultative teaching as a
subject in the two major teacher training programs in Indonesia, i.e. in-service
teacher profession training, which is called PLPG, and professional education for
pre-service teachers that is called PPG.

With the abovementioned recommendations we would like to underline the

significance of the availability of appropriate instructional resources and
professional development for teachers as key aspects for a sound implementation
of a national curriculum and the improvement of student achievement.

References
Abedi, J., & Herman, J. L. (2010). Assessing English language learners’ opportunity to
learn mathematics: Issues and limitations. Teacher’s College Record 112(3), 723–746.
Antonius, S., Haines, C., Jensen, T. H., Niss, M., & Burkhardt, H. (2007). Classroom
activities and the teacher. In W. Blum, P. L. Galbraith, H.-W. Henn & M. Niss
(Eds.), Modelling and Applications in Mathematics Education: The 14th ICMI Study (pp.
295–308). New York: Springer.
Bernardo, A. B. I. (1999). Overcoming obstacles to understanding and solving word
problems in mathematics. Educational Psychology, 19(2), 149–163.
Blum, W. (2011). Can modelling be taught and learnt? Some answers from empirical
research. In G. Kaiser, W. Blum, R. B. Ferri & G. Stillman (Eds.), Trends in teaching
and learning of mathematical modelling (pp. 15–30). New York: Springer.
Blum, W., & Ferri, R. B. (2009). Mathematical modelling: Can it be taught and learnt?
Journal of Mathematical Modelling and Application, 1(1), 45–58.
Blum, W., & Leiss, D. (2007). How do students and teachers deal with mathematical
modelling problems? The example “Sugarloaf”. In C. Haines, P. Galbraith, W.
Blum & S. Khan (Eds.), Mathematical modelling (ICTMA 12): Education, engineering
and economics (pp. 222–231). Chichester: Horwood Publishing.
Boaler, J. (1994). When girls prefer football to fashion? An analysis of female under
achievement in relation to realistic mathematics contexts. British Educational
Research Journal, 20(5), 551–564.

150

Thesis Ari_141201_FINAL.pdf 150 1-12-2014 16:26:28

 OTL to solve context-based tasks and students’ performance

Brewer, D. J., & Stacz, C. (1996). Enhancing opportunity to learn measures in NCES data.
Santa Monica, CA: RAND.
Charalambous, C. Y., Delaney, S., Hsuc, H.-Y., & Mesac, V. (2010). A comparative
analysis of the addition and subtraction of fractions in textbooks from three
countries. Mathematical Thinking and Learning, 12(2), 117–151.
Clements, M. A. (1980). Analyzing children’s errors on written mathematical task.
Educational Studies in Mathematics, 11(1), 1–21.
Cooper, B., & Dunne, M. (2000). Assessing children’s mathematical knowledge: Social class, sex
and problem-solving. Buckingham: Open University Press.
Eurydice. (2011). Mathematics education in Europe: Common challenges and national policies.
Brussels: Education, Audiovisual and Culture Executive Agency.
Forman, S. L., & Steen, L. A. (2001). Why math? Applications in science, engineering,
and technological programs. Research Brief, American Association of Community
Colleges.
Goldman, S. R. (1989). Strategy instruction in mathematics. Learning Disability Quarterly,
12(1), 43–55.
Graumann, G. (2011). Mathematics for problem in the everyday world. In J. Maasz &
J. O’Donoghue (Eds.), Real-world problems for secondary school mathematics students: Case
studies (pp. 113–122). Rotterdam: Sense Publishers.
Greer, B. (1997). Modeling reality in mathematics classrooms: The case of word
problems. Learning and Instruction, 7(4), 293–307.
Grouws, D. A., & Cebulla, K. J. (2000). Improving student achievement in mathematics.
Brussels: International Academy of Education.
Herman, J. L., Klein, D. C. D., & Abedi, J. (2000). Assessing students’ opportunity to
learn: Teacher and student perspectives. Educational Measurement: Issue and Practice,
19(4), 16-24.
Hiebert, J., & Grouws, D. A. (2007). The effects of classroom mathematics teaching
on students' learning. In F. Lester (Ed.), Second handbook of research on mathematics
teaching and learning (pp. 371–404). Charlotte: Information Age Publishing.
Howson, G. (2013). The development of mathematics textbooks: historical reflections
from personal perspective. ZDM – The International Journal on Mathematics
Education, 45(5), 647–658.
Husén, T. (Ed). (1967). International study of achievement in mathematics: A comparison of
twelve countries (Vol. II). New York: John Wiley & Sons.

151

Thesis Ari_141201_FINAL.pdf 151 1-12-2014 16:26:28

 Chapter 5

Jalal, F., Samani, M., Chang, M. C., Stevenson, R., Ragatz, A. B., & Negara, S. D.
(2009). Teacher Certification in Indonesia: A strategy for teacher quality improvement.
Jakarta: Indonesian Ministry of National Education and the World Bank.
Karbalei, A., & Amoli, F. A. (2011). The effect of paraphrasing strategy training on the
reading comprehension of college students at the undergraduate level. Asian EFL
Journal, 13(3), 229–344.
Kletzien, S. B. (2009). Paraphrasing: An effective comprehension strategy. The Reading
Teacher, 63(1), 73–77.
Klymchuk, S., Zverkova, T., Gruenwald, N., & Sauerbier, G. (2010). University
students’ difficulties in solving application problems in calculus: Student
perspectives. Mathematics Education Research Journal, 22(1), 81–91.
Kramarski, B., Mevarech, Z. R., & Arami, M. (2002). The effects of metacognitive
instruction on solving mathematical authentic tasks. Educational Studies in
Mathematics, 49(2), 225–250.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for
categorical data. Biometrics, 33(1), 159–174.
Maass, K. (2007). Modelling tasks for low achieving students – first results of an
empirical study. In D. Pitta-Pantazi & G. Philippou (Eds.), Proceedings of the Fifth
Congress of the European Society for Research in Mathematics Education CERME 5
(pp. 2120–2129). Larnaca, Cyprus.
Maass, K. (2010). Classification scheme for modelling tasks. Journal fur Mathematik-
Didaktik, 31(2), 285–311.
Maulana, R., Opdenakker, M.-C., Den Brok, P., & Bosket, R. J. (2012). Teacher-
student interpersonal behavior in secondary mathematics classes in Indonesia.
International Journal of Science and Mathematics Education, 10(1), 21–47.
McDonnell, L. M. (1995). Opportunity to learn as a research concept and a policy
instrument. Educational Evaluation and Policy Analysis, 17(3), 305-322.
Montague, M. (2007). Self-regulation and mathematics instruction. Learning Disabilities
Research and Practice, 22(1), 75–83.
Montague, M. (2008). Self-regulation strategies to improve mathematical problem
solving for students with learning disabilities. Learning Disability Quarterly, 31(1),
37–44.
Montague, M., Warger, C., & Morgan, T. H. (2000). Solve it! Strategy instruction to
improve mathematical problem solving. Learning Disabilities Research and Practices,
15(2), 110–116.

152

Thesis Ari_141201_FINAL.pdf 152 1-12-2014 16:26:28

 OTL to solve context-based tasks and students’ performance

National Council of Teachers of Mathematics. (2000). Principles and Standard for School
Mathematics. Reston: Author.
Newman, M. A. (1977). An analysis of sixth-grade pupils’ errors on written
mathematical tasks. Victorian Institute for Educational Research Bulletin, 39, 31–43.
OECD. (2003). The PISA 2003 Assessment Framework - Mathematics, Reading, Science, and
Problem Solving Knowledge and Skills. Paris: Author.
OECD. (2009a). Learning Mathematics for Life. A View Perspective from PISA. Paris:
Author.
OECD. (2009b). Take the Test. Sample Questions from OECD's PISA Assessments. Paris:
Author.
OECD (2010). PISA 2009 Results: What students know and can do. Student performance in
reading, mathematics, and science (Vol. I). Paris: Author.
OECD. (2013). PISA 2012 Results: What students know and can do. Student performance in
mathematics, reading and science. Paris: Author.
Pusat Kurikulum (2003). Kurikulum 2004: Standar kompetensi mata pelajaran matematika
Sekolah Menengah Pertama dan Madrasah Tsanawiyah. Jakarta: Departemen
Pendidikan nasional.
Sam, L. C., Lourdusamy, A., & Ghazali, M. (2001). Factors affecting students’ abilities
to solve operational and word problems in mathematics. Journal of Science and
Mathematics Education in Southeast Asia, 24(1), 84–95.
Schmidt, W. H., McKnight, C. C., Valverde, G. A., Houang, R. T., & Wiley, D. E.
(1997). Many visions, many aims: A cross-national investigation of curricular intentions in
school mathematics. Dordrecht: Kluwer Academic Publishers.
Sembiring, R. K., Hadi, S., & Dolk, M. (2008). Reforming mathematics learning in
Indonesian classrooms through RME. ZDM The International Journal on Mathematics
Education, 40(6), 927–939.
Tomlinson, M. (2004). 14-19 Curriculum and Qualifications Reform. Final report of the
Working Group on 14-19 Reform. London: Department for Education and Skills.
Tornroos, J. (2005). Mathematics textbooks, opportunity to learn and student
achievement. Studies in Educational Evaluation, 31(4), 315–327.
Valverde. G. A., Bianchi, L. J., Wolfe, R. G., Schmidt, W. H., & Houang, R. T. (2002).
According to the book. Using TIMSS to investigate the translation of policy into practice
through the world of textbooks. Dordrecht: Kluwer Academic Publishers.
Veenman, M. V. J., Van Hout-Wolters, B. H. A. M., & Afflerbach, P. (2006).
Metacognition and learning: conceptual and methodological considerations.
Metacognition Learning, 1(1), 3–14.

153

Thesis Ari_141201_FINAL.pdf 153 1-12-2014 16:26:29

 Chapter 5

Verschaffel, L., Greer, B., & De Corte, E. (2000). Making sense of word problems. Lisse:
Swets & Zeitlinger.
Wijaya, A., Van den Heuvel-Panhuizen, M., Doorman, M., & Robitzsch, A. (2014).
Difficulties in solving context-based PISA mathematics tasks: An analysis of
students’ errors. The Mathematics Enthusiast, 11(3), 555-584.
Wijaya, A., Van den Heuvel-Panhuizen, M., & Doorman, M. (submitted a).
Opportunity-to-learn context-based tasks provided by mathematics textbooks
and the relation with students’ difficulties in solving these tasks.
Wijaya, A., Van den Heuvel-Panhuizen, M., & Doorman, M. (submitted b). Teachers’
teaching practices and beliefs regarding context-based tasks and students’
difficulties in solving these tasks.
World Bank. (2010). Inside Indonesia’s mathematics classrooms: A TIMSS Video Study of
teaching practices and student achievement. Jakarta: Author.
Xin, Y. P. (2007). Word problem solving tasks in textbooks and their relation to
student performance. Journal of Educational Research, 100(6), 347–359.

154

Thesis Ari_141201_FINAL.pdf 154 1-12-2014 16:26:29

 OTL to solve context-based tasks and students’ performance

155

Thesis Ari_141201_FINAL.pdf 155 1-12-2014 16:26:29

 Appendix 1

156
Outline of the intervention program
Chapter 5

Thesis Ari_141201_FINAL.pdf 156

Bare
Mathematical Goal related to Goal related to
Lesson Context-based task math.
sub-topic mathematical content solving context-based tasks
task
1 Sketching - Making table of ordered pairs - Identifying and completing missing - Taxi task: 1
graph of of linear equation information. Comparing taxi
straight line - Sketching graph of equation on - Selecting relevant information. tariffs
Cartesian coordinate by - Internet task:
- Paraphrasing the tasks.
plotting ordered pairs Comparing
- Identifying appropriate mathematics internet fees
procedure.

2 Gradient (1) - Understanding definition of - Identifying and completing missing - Experiment task: 1
gradient information. Determining the
- Determining gradient of a - Selecting relevant information. fastest speed of
straight line through two - Identifying appropriate mathematics bike simulation
points procedure. - Price task:
Determining the
increase rate of
oil price

3 Gradient (2) - Determining gradient of parallel - Identifying and completing missing - Bus task: 1
lines information. Comparing bus
- Determining gradient of - Identifying appropriate mathematics speed
perpendicular lines procedure. - Water pump task
- Interpreting the solution in terms of the (1):
problem situation. Visualizing the
rates of filling
water tanks

1-12-2014 16:26:29
 Bare
Mathematical Goal related to Goal related to
Lesson Context-based task math.

Thesis Ari_141201_FINAL.pdf 157

sub-topic mathematical content solving context-based tasks
task
4 Equation of Determining the equation and - Selecting relevant information - Olympic task: -
straight line sketching the graph of: Predicting the
(1) - A line with a gradient of m and number of
through point (x1,y1) future
- A line through points (x1,y1) participants
and (x2,y2) - Hospital task:
Estimating drug
dosage

5 Equation of Determining the equation and - Selecting relevant information - Water pump task 2
straight line sketching the graph of: - Interpreting the solution in terms of the (2):
(2) - A line through point (x1,y1) and problem situation Estimating
parallel to another line filling rates
- A line through point (x1,y1) and
perpendicular to another line
OTL to solve context-based tasks and students’ performance

157

1-12-2014 16:26:29
 Appendix 2

158
Coding scheme for error types when solving context-based mathematics tasks
Chapter 5

Thesis Ari_141201_FINAL.pdf 158

Error type Sub-type Explanation
Comprehension Misunderstanding the instruction Student incorrectly interprets what (s)he is asked to do.
Misunderstanding a keyword Student misunderstands a keyword, which is usually a mathematical term.
Error in selecting information Student is unable to distinguish between relevant and irrelevant information (e.g.
using all information provided in a task or neglecting relevant information) or is
unable to gather required information which is not provided in the task.

Transformation Procedural tendency Student tends to directly use a mathematical procedure such as formula or
algorithm without analyzing whether or not it is needed.
Taking too much account of the Student’s answer only refers to the context or real-world situation without taking
context the perspective of the mathematics.
Wrong mathematical Student uses mathematical procedures or concepts which are not relevant to the
operation/concept tasks.
Treating a graph as a picture Student treats a graph as a literal picture of a situation. (S)he interprets and
focuses on the shape of the graph, instead of on the properties of the graph.

Mathematical Algebraic error Error in solving algebraic expression or function.

Processing
Arithmetical error Error in calculation.

1-12-2014 16:26:29
 Error type Sub-type Explanation

Thesis Ari_141201_FINAL.pdf 159

Mathematical Error in mathematical
Processing interpretation of graph:
- Point-interval confusion Student mistakenly focuses on a single point rather than on an interval.
- Slope-height confusion Student does not use the slope of the graph but only focuses on the vertical
distance.
Measurement error Student cannot convert between standard units (e.g. from m/minute to km/h)
or from a non-standard unit to a standard unit (e.g. from step/minute to
m/minute).
Improper use of scale Student cannot select and use the scale of a map properly.
Unfinished answer Student uses a correct procedure, but (s)he does not finish it.

Encoding Student is unable to correctly interpret and validate the mathematical solution in
terms of the real-world problem. This error is reflected by an impossible or not
realistic answer.
OTL to solve context-based tasks and students’ performance

159

1-12-2014 16:26:29
 Chapter 2

160

Thesis Ari_141201_FINAL.pdf 160 1-12-2014 16:26:29

 Chapter 6

Conclusion

Thesis Ari_141201_FINAL.pdf 161 1-12-2014 16:26:29

 Chapter 6

Conclusion

1 Gains from the study

The main purpose of this PhD research was to gain a better insight into
Indonesian students´ low performance on context-based mathematics tasks and to
generate recommendations for improving student performance. To achieve this
goal, three interrelated issues regarding context-based mathematics tasks in
Indonesia were studied. First, it was investigated what difficulties Indonesian
students experience when solving context-based tasks. Second, it was examined
why Indonesian students struggle with context-based tasks, for which the focus was
on analyzing textbooks and investigating teachers’ teaching practices. Lastly, to
study how student performance can be improved, students were provided with
opportunity-to-learn to solve context-based tasks and afterward the effects of this
opportunity-to-learn on students’ performance was examined.

1.1 Indonesian students’ difficulties when solving context-based

tasks
In the first study of this PhD research the difficulties experienced by Indonesian
students when solving context-based tasks were examined through an analysis of
students’ errors. This approach was chosen because students’ errors provide access
to students’ reasoning (Brodie, 2014) and are considered a powerful source to
diagnose learning difficulties (Batanero, Godino, Vallecillos, Green, & Holmes,
1994; Borasi, 1987; Seng, 2010).

The error analysis revealed that Indonesian students primarily had difficulties in
comprehending context-based tasks and in transforming real-world problems into
mathematical problems. In the process of comprehending a context-based task,
students often made errors in selecting relevant information, i.e. they tended to use
all information provided in the task. This finding adds to the results of the PISA
studies (e.g. OECD, 2013) that most Indonesian students can solve context-based
tasks if all relevant information is given and the questions are clearly stated. In line
with Jupri, Drijvers, and Van den Heuvel-Panhuizen (2014), it was found that
when Indonesian students had to transform a real-world problem into a
mathematical problem many of them were unable to identify the required
mathematical concept or procedure. The students had a tendency to apply a

162

Thesis Ari_141201_FINAL.pdf 162 1-12-2014 16:26:29

 Conclusion

familiar procedure or calculation without considering its relevance for the context
of the task.

In addition to these specific results, this study showed how analyzing students’
difficulties can be a crucial preliminary step in the process of improving student
performance because it sheds light on key aspects of solving context-based tasks
that need to be developed. For example, the findings of this study suggest that
improving the task comprehension of Indonesian students requires a focus not
only on students’ language competence, but also on the ability to select relevant
information. Furthermore, the ability to identify the required procedure or concept
was found to be another key competence that needs to be improved.

1.2 Identifying why Indonesian students struggle with context-based

tasks: The opportunity-to-learn as a key concept
The next step to find ways to improve student performance on context-based tasks
was identifying possible explanations for students’ difficulties. The concept of
‘opportunity-to-learn’ was used as a key reference to investigate whether students
have received education and experiences that are supportive for them to learn to
solve context-based tasks. This investigation was focused on two aspects: the
opportunity-to-learn that is provided in Indonesian mathematics textbooks and the
opportunity-to-learn that is offered by teachers through their teaching practices.

1.2.1 Context-based tasks in Indonesian mathematics textbooks

In order to find possible explanations for Indonesian students’ difficulties, a
second study was conducted to analyze the mathematics textbooks that were used
in the schools participating in the first study. This textbook analysis study focused
on the mathematics tasks that were provided in the textbooks.

The analysis revealed that Indonesian mathematics textbooks contained an

insufficient number of context-based tasks. Only 10% of the tasks in the textbooks
were context-based. To get a better picture of the relation between textbooks and
student performance, an in-depth analysis of the characteristics of these context-
based tasks was carried out. The results of this analysis provided indications for a
relation between the characteristics of the context-based tasks in Indonesian
mathematics textbooks and the types of errors made by students when solving
such tasks. First, most of the context-based tasks in the textbooks used camouflage
context and provided explicit suggestions about the procedure required to solve
the tasks. This characteristic corresponds to the large number of students’ errors in

163

Thesis Ari_141201_FINAL.pdf 163 1-12-2014 16:26:29

 Chapter 6

transforming a real-world problem into a mathematical problem that includes

errors in identifying the required procedure. As underlined by Haines and Crouch
(2007), students’ unfamiliarity with tasks that require identifying the necessary
procedure leads to students’ difficulties in mathematical modeling. Second, it was
found that most of the context-based tasks in the textbooks provided the exact
information that was necessary to solve the tasks; so students had no experience in
selecting relevant information. This finding matches the high number of students’
errors in selecting information.

In conclusion, the findings of this study indicate that Indonesian mathematics

textbooks do not provide students enough opportunity-to-learn to solve context-
based tasks. It also means that the textbooks do not seem to optimally put into
practice the mandate of the national curriculum with regard to developing
students’ ability to apply mathematics.

1.2.2 Indonesian teachers’ teaching practices of context-based tasks

In the third study, teachers’ teaching practices were investigated to identify from
this perspective the opportunity-to-learn to solve context-based tasks that is
offered to Indonesian students. Moreover, considering the potential influence of
teachers’ beliefs on teaching practices, the beliefs of Indonesian teachers were also
investigated.

Teachers’ beliefs
The findings of this study showed that the involved teachers tended to perceive
context-based tasks merely as plain word problems. The teachers focused only on
the use of simple real-world contexts without really taking mathematical modeling
into account and argued that the mathematical procedure required to solve a
context-based task should be given explicitly. Furthermore, the teachers also did
not consider missing and superfluous information as an important characteristic of
a context-based task. In sum, the teachers did not hold beliefs that are supportive
towards providing students with opportunity-to-learn to solve context-based tasks.

Teachers’ teaching practices

Two aspects of teachers’ teaching practices were investigated in this third study.
The first aspect is the characteristics of the context-based tasks offered by teachers
to students in which an indication for the influence of teachers’ beliefs on teaching
practices was found. In line with their beliefs about context-based tasks, the

164

Thesis Ari_141201_FINAL.pdf 164 1-12-2014 16:26:29

 Conclusion

teachers mostly gave students context-based tasks that had explicit indications
about the required procedure and provided only the relevant information. The
second aspect is the teaching approach used by teachers to help students learn to
solve context-based tasks. It was revealed that the teachers mostly used directive
teaching. In directive teaching, students are not actively involved in reflecting on
and carrying out the stages of solving context-based tasks. With respect to the
process of comprehending a context-based task the teachers frequently told
students what the task was about and what information was needed to solve it.
This directive teaching approach was also observed in the process of transforming
a real-world problem into a mathematical problem. It was visible in how the
teachers directly told their students the mathematical procedure required to solve
the task. In contrast to directive teaching, a consultative teaching approach, in
which students get opportunities to actively perform and reflect on the solving
process, was rarely used by the teachers. This teaching approach was mostly
observed in the process of carrying out mathematical procedures, which, in fact, is
a process in which students do not have to deal with the contexts of a task.

Correspondences were indicated between teachers’ teaching practices and students’

difficulties in solving context-based tasks. A lack of opportunities for students to
paraphrase a context-based task seems to be related to students’ difficulty in
comprehending a context-based task because, as pointed out by Kletzien (2009),
paraphrasing helps students understand the text of a task. Teachers’ direct advice
regarding the procedures to be carried out might correspond to students’
transformation errors because, according to Barness (2000), it will discourage
students from thinking about the mathematics concepts addressed in the task.

In conclusion, the observed teaching practices of the teachers seem to be not

supportive for students to learn solving context-based tasks because, according to
Antonius, Haines, Jensen, Niss, and Burkhardt (2007) and Forman and Steen
(2001), teaching context-based tasks requires more than telling students what they
should do. This finding suggests that improving the teaching practices of context-
based tasks might contribute to improving student performance on such tasks.

1.3 Opportunity-to-learn to solve context-based mathematics tasks

and students’ performance in solving such tasks
In the fourth and final study of this PhD research, a field experiment was carried
out to investigate whether providing students opportunity-to-learn to solve
context-based tasks can help them improve their performance on such tasks. For

165

Thesis Ari_141201_FINAL.pdf 165 1-12-2014 16:26:29

 Chapter 6

this field experiment an intervention program was developed to compensate what

was considered to be lacking in Indonesian mathematics textbooks and in teachers’
teaching practices. The intervention comprised two components of opportunity-
to-learn; first, context-based tasks that have superfluous and missing information
and do not explicitly provide the required procedures, and second, a consultative
teaching approach that uses metacognitive prompts. This opportunity-to-learn was
offered to students, and its effects on students’ performance in solving context-
based tasks were examined from the perspectives of students’ gain scores and
students’ errors.

The field experiment showed that there was only a small and marginally significant
effect of the opportunity-to-learn on students’ gain scores. Nevertheless, a closer
examination of the effect of the opportunity-to-learn on students’ errors revealed a
positive influence. Students who received the opportunity-to-learn could better
understand the instruction for a context-based task and had improved
performance in selecting relevant information. With respect to transforming a real-
world problem into a mathematical problem and identifying the required
procedure in general no influence of the opportunity-to-learn was found.
However, a positive influence was found for context-based tasks addressing graphs
– i.e. the topic taught during the intervention period – in which students who got
the opportunity-to-learn were better able to explain a mathematical interpretation
of a graph. Reflecting upon this finding and referring to Howson (2010), it can be
learned that to improve students’ ability to identify the required procedure it is
essential to provide not only context-based tasks that are related to the topic being
taught, but also context-based tasks that address other topics.

To sum up, offering students context-based tasks that have non-explicit

procedures and provide superfluous and missing information, and a consultative
teaching approach with metacognitive prompts is promising to support students’
learning of context-based tasks. However, in agreement with OECD (2014), it
should be noted that a prolonged and continuous implementation of such practice
might be needed in order to have optimum results.

1.4 Results at a glance

In summary, the results of this PhD research cover three interrelated issues
regarding context-based mathematics tasks in Indonesia (see Figure 1). First, the
investigation into students’ difficulties provided information about key
competences that need to be improved. Second, possible reasons for the students’

166

Thesis Ari_141201_FINAL.pdf 166 1-12-2014 16:26:29

 Conclusion

Figure 1. Results at a glance

167

Thesis Ari_141201_FINAL.pdf 167 1-12-2014 16:26:29

 Chapter 6

difficulties were identified when the difficulties were connected to opportunity-to-

learn offered in Indonesian textbooks and teachers’ teaching practices. Lastly, these
possible reasons served as an important basis for designing an intervention
program to offer students opportunity-to-learn to solve context-based tasks. This
intervention program was found to have a positive influence on students’ ability to
comprehend a context-based task.

2 Toward a better performance of Indonesian students on

context-based tasks
It is widely acknowledged that there are different ways to improve educational
achievement (see e.g. Pearson, 2014; Stacey, 2011). For example, as mentioned by
Stacey (2011), Finland attributes its success in PISA to teacher quality, whereas
Singapore and the Netherlands point to their curriculum. The way a lesson is
constructed can also contribute to improving student achievement. As a case in
point, Japan is a country that refers its success in education to carefully constructed
lessons and its culture of lesson study.

Reflecting upon these different ways to improve educational achievement, in this

final chapter I would like to raise and answer the following question:

How could the performance of Indonesian students on context-based tasks be improved?

Improving Indonesian students’ performance cannot be simply done by applying

an educational practice that is used in other countries because, according to
Pearson (2014) and Stacey (2011), what works in one particular country will not
necessarily give the same result in other countries. Careful thought about what is
missing in current educational practices and what might be needed by students is
necessary. In this respect, this PhD research provides a better insight into
educational practices in Indonesia regarding context-based tasks. Although the
research took place in one province in Indonesia, I argue that its findings can be
taken into consideration for a broader scope because the issues addressed in the
study were quite general; for example, the textbooks that were analyzed are widely
used in Indonesia. Furthermore, the finding about the dominance of directive
teaching is in agreement with other studies on mathematics classrooms in
Indonesia (e.g. Maulana, Opdenakker, Den Brok, & Bosker, 2012; World Bank,
2010). Therefore, on the basis of the results of this PhD research I would like to

168

Thesis Ari_141201_FINAL.pdf 168 1-12-2014 16:26:29

 Conclusion

offer some recommendations for the practice of teaching context-based tasks in

Indonesia that might contribute to improving student performance.

2.1 Learning materials: Adequate and appropriate context-based

tasks
My first recommendation addresses learning materials, for which I recommend
providing students with adequate and appropriate context-based tasks. ‘Adequate’
refers to the exposure to context-based tasks. Indonesian students need to get
more experiences and chances to deal with context-based tasks; therefore it is
essential to increase the quantity of such tasks in classroom practices.

What I mean with ‘appropriate’ is the quality of context-based tasks. Although

some studies (OECD, 2014; World Bank, 2010) report an increased connection to
real-life problems in mathematics lessons in Indonesia, this PhD research revealed
that what students received were mainly tasks with dressed-up contexts and explicit
indication about the required procedure. Such tasks do not really require students
to transform a real-world problem into a mathematical problem and are therefore
not sufficient to support students learning to solve context-based tasks. Therefore,
as an addition to increasing the exposure of context-based tasks, I recommend
improving the quality of the tasks by considering the following characteristics:

Relevant and essential context. A context-based task should use a real-world context
that is essential and needs to be considered in the solving process. Such tasks offer
students more opportunities to transform a real-world problem to a mathematical
problem, which is an important aspect of applying mathematics.

Non-explicit procedures. As an addition to relevant contexts, I recommend giving

students context-based tasks that do not imply the mathematical procedures or
concepts needed to find the solution. Such tasks are important to develop
students’ ability to identify the procedure that is required to solve a context-based
task. The results of this research suggest that not implying the required procedures
is not only a property of a context-based task, but also a matter of how and when a
context-based task is used. It was found that students did not really benefit from
context-based tasks with non-explicit procedures when the tasks were given after
an explanation of particular mathematics topics. Therefore, my further
recommendation is to provide students with a ‘mixed exercises’ section that
includes context-based tasks related to various mathematics topics.

169

Thesis Ari_141201_FINAL.pdf 169 1-12-2014 16:26:29

 Chapter 6

Superfluous and missing information. The ability to distinguish between relevant and
irrelevant information is a key aspect of solving problems in real life. However, this
PhD research shows that most students were not able to select information and,
correspondingly, received mainly tasks that contained only the information
required to find the solution. With respect to this finding, I recommend offering
students context-based tasks that have superfluous or missing information; i.e.
tasks that have more or less information than what is needed to find the solution.

Lastly, I would like to emphasize that the abovementioned recommendation is not

only for teachers, but also for textbook authors because OECD (2014) reported an
increased use of textbooks as a primary resource for mathematics lessons in
Indonesia. This means that improving the quality of textbooks might be followed
by improved teaching practice, at the very least with respect to the quality of tasks
offered to students. Furthermore, textbooks are essential for a widespread
dissemination of an educational innovation because textbooks are widely used in
many schools.

2.2 Teaching practices: Consultative teaching approach with meta-

cognitive prompts
Several researchers (e.g. Antonius et al., 2007; Forman & Steen, 2001; Lingefjard &
Meier, 2010) underlined the importance of a student-centered approach for
teaching students mathematics problems situated in real-world contexts. However,
the term ‘student-centered’ is still rather general and, therefore, concrete
suggestions about how to conduct student-centered teaching might be necessary
for teachers. Here I would like to recommend the consultative teaching approach
developed in this PhD research as an alternative for student-centered teaching. In
order to have student-centered teaching, metacognitive prompts can be used to
keep a balance between teacher guidance and students’ independence. Self-
questioning, which is a kind of metacognitive prompt, is important not only to
guide students to independently carry out the process of solving context-based
tasks, but also to reflect on their work and understanding.

Teaching approach is not the only important aspect of teaching practices, because
in order to optimally support their students’ acquiring particular competences,
teachers also need to know students’ learning difficulties. This research suggests
that diagnosing students’ difficulties could provide essential information for
improving student performance. Therefore, I recommend teachers including
diagnosing students’ difficulties as an integral part of their teaching practices.

170

Thesis Ari_141201_FINAL.pdf 170 1-12-2014 16:26:29

 Conclusion

2.3 Teachers
In relation to the previous two recommendations, I would like to highlight three
essential roles of a teacher. First, a teacher is required to select tasks that match the
learning goals and the students’ ability level, because not every task in a textbook
might be appropriate for his/her classroom practice. Second, a teacher should
design tasks when textbooks do not accommodate all learning goals set in the
curriculum or when the teacher has his/her own specific learning goals. The third
role of a teacher is to convert tasks to learning opportunities; which means the
teacher should use the tasks to support students’ learning.

The findings about the low exposure of context-based tasks in textbooks and that
teachers mostly offered students mere plain word problems indicate a need to
increase the teachers’ role in selecting and designing context-based tasks.
Designing one’s own tasks could also lead to better task ownership for teachers
and also contribute to teachers’ flexibility and creativity in using the tasks to
support students’ learning, which is in fact related to the third role of a teacher. To
increase the three roles of a teacher I recommend paying attention to teachers’
beliefs about context-based tasks. A correspondence between teachers’ beliefs
about context-based tasks and the characteristics of tasks they offered to students
indicates that having appropriate beliefs might be crucial for selecting and
developing good context-based tasks. Furthermore, appropriate beliefs, according
to Blum (2011), are also necessary for teaching context-based tasks, which is the
third role of a teacher. As an addition to beliefs, knowledge is also an important
element for increasing the roles of a teacher in teaching context-based tasks. A
teacher needs to have knowledge of context-based tasks, which includes: (1) the
type of context; (2) non-explicit procedures; (3) the type of information; and (4)
cognitive demands of tasks.

Above all, as my final note I would like to highlight that we cannot rely solely on
teachers to improve student performance. Teachers need professional
development for which support from other parties, especially teacher training
institutions, is highly required. Professional development for teachers has been
recognized as a key aspect to improve the quality of education and student
achievement (Guskey, 2002; Porter, Garet, Desimone, Yoon, & Birman, 2000;
Supovitz & Turner, 2000). Therefore, I would like to recommend teacher training
institutions to consider beliefs and knowledge about context-based tasks in their
training programs.

171

Thesis Ari_141201_FINAL.pdf 171 1-12-2014 16:26:29

 Chapter 6

2.4 Recommendations at a glance

To sum up, in Figure 2 I illustrate how the aforementioned recommendations can
contribute to improving student performance. Learning materials and teaching
practices are key elements to improve students’ performance. Here, the role of
teachers is to select and design learning materials and to convert the learning
materials into learning opportunities through the use of appropriate teaching
practices. Despite the importance of their roles, teachers are not the only key actor
to improve students’ performance. Support from textbook authors and teacher
training centers is required.

Figure 2. Recommendations at a glance

3 Suggestions for further research

Reflecting upon the results and the processes of this PhD research, it is realized
that there is a need for further study. The research took place only in the Province

172

Thesis Ari_141201_FINAL.pdf 172 1-12-2014 16:26:29

 Conclusion

of Yogyakarta. Therefore, to get a more thorough understanding of how the

performance of Indonesian students on context-based tasks can be improved, it is
necessary to extend this research to a broader area by involving more schools in
more provinces. Having more schools might provide greater chances to investigate
more crucial aspects such as school quality, teacher quality, school facility, and
student background.

The investigation into possible reasons for student difficulties only emphasized
factors that are related to cognitive aspects, i.e. textbooks and teachers’ teaching
practices. However, as pointed out by Leron and Hazzan (1997), students’ thinking
is influenced not only by cognitive factors, but also by affective factors. Therefore,
investigating affective factors such as students’ motivation might provide a
comprehensive picture about possible factors that influence student performance
on context-based tasks.

With regard to teachers’ teaching practices the focus was only on teachers’ beliefs
and teaching approaches. Although the research findings regarding these two
aspects already provide recommendations for improving educational practices in
Indonesia, it is important to take teachers’ knowledge about context-based tasks
into consideration. Teachers’ knowledge about context-based tasks might also have
influence on teaching practices and, therefore, on student performance.

Another issue that needs further consideration is teacher quality, because teachers
play a key role in the learning process of students. In attempting to improve
teacher quality, the Indonesian government, through the Teacher Law 2005, sets
minimum academic and professional requirements for teachers. One of the
requirements is the need for higher standards in classroom teaching, in which
teachers are required to engage students in the learning process (Jalal et al., 2009). I
argue that the consultative teaching that was developed in this PhD research meets
this requirement. Therefore, as a further study it might be valuable to incorporate
and collaborate the knowledge generated by this research into two major teacher
training programs that are positioned by the Indonesian government as strategies
for teacher quality improvement; i.e. Pendidikan dan Latihan Profesi Guru (in-service
teacher profession training) and Pendidikan Profesi Guru (professional education for
pre-service teachers).

173

Thesis Ari_141201_FINAL.pdf 173 1-12-2014 16:26:29

 Chapter 6

References
Antonius, S., Haines, C., Jensen, T. H., Niss, M., & Burkhardt, H. (2007). Classroom
activities and the teacher. In W. Blum, P. L. Galbraith, H.-W. Henn & M. Niss
(Eds.), Modelling and applications in mathematics education (pp. 295–308). New York:
Springer
Barnes, M. (2000). 'Magical' moments in mathematics: Insights into the process of
coming to know. For the Learning of Mathematics, 20(1), 33–43.
Batanero, C., Godino, J. D., Vallecillos, A., Green, D. R., & Holmes, P. (1994). Errors
and difficulties in understanding elementary statistical concepts. International
Journal of Mathematical Education in Science and Technology, 25(4), 527–547.
Blum, W. (2011). Can modelling be taught and learnt? Some answers from empirical
research. In G. Kaiser, W. Blum, R. B. Ferri & G. Stillman (Eds.), Trends in teaching
and learning of mathematical modelling (pp. 15 – 30). New York: Springer.
Borasi, R. (1987). Exploring mathematics through the analysis of errors. For the
Learning of Mathematics, 7(3), 2–8.
Brodie, K. (2014). Learning about learner errors in professional learning communities.
Educational Studies in Mathematics, 85(2), 221–239.
Guskey, T. R. (2002). Professional development and teacher change. Teachers and
Teaching: Theory and Practice, 8(3-4), 381–391.
Haines, C., & Crouch, R. (2007). Mathematical modelling and applications: Ability and
competence frameworks. In W. Blum, P. L. Galbraith, H.-W. Henn & M. Niss
(Eds.), Modelling and applications in mathematics education: The 14th ICMI study (pp. 89-
98). New York: Springer.
Howson, G. (2010). The development of mathematics textbooks: historical reflections
from a personal perspective. ZDM – The International Journal on Mathematics
Education, 45(5), 647–658.
Jalal, F., Samani, M., Chang, M. C., Stevenson, R., Ragatz, A. B., & Negara, S. D.
(2009). Teacher Certification in Indonesia: A strategy for teacher quality improvement.
Jakarta: Indonesian Ministry of National Education and the World Bank.
Jupri, A., Drijvers, P., & Van den Heuvel-Panhuizen, M. (2014). Difficulties in initial
algebra learning in Indonesia. Mathematics Education Research Journal, 26(4), 683–
710.
Kletzien, S. B. (2009). Paraphrasing: An effective comprehension strategy. The Reading
Teacher, 63(1), 73–77.
Leron, U., & Hazzan, O. (1997). The world according to Johnny: A coping perspective
in mathematics education. Educational Studies in Mathematics, 32(3), 265–292.

174

Thesis Ari_141201_FINAL.pdf 174 1-12-2014 16:26:29

 Conclusion

Maulana, R., Opdenakker, M.-C., Den Brok, P., & Bosker. (2012). Teacher-student
interpersonal behavior in secondary mathematics classes in Indonesia. International
Journal of Science and Mathematics Education, 10(1), 21–47.
OECD. (2013). PISA 2012 results: What students know and can do. Student performance in
mathematics, reading and science. PISA: OECD Publishing.
OECD. (2014). Measuring innovation in education: A new perspectives. Educational Research
and Innovation, OECD Publishing. doi: 10.1787/9789264215696-en.
Pearson. (2014). The Learning Curve 2014: Education and skills for life. Retrieved from
http://thelearningcurve.pearson.com/reports/the-learning-curve-report-2014 on
June 24, 2014.
Porter, A. C., Garet, M. S., Desimone, L., Yoon, K. S., & Birman, B. F. (2000). Does
professional development change teaching practice? Results from a three-year study.
Washington D. C.: U. S. Department of Education.
Stacey, K. (2011). The PISA view of mathematical literacy in Indonesia. IndoMS Journal
on Mathematics Education, 2(2), 95–126.
Seng, L. K. (2010). An error analysis of Form 2 (Grade 7) students in simplifying
algebraic expressions: A descriptive study. Electronic Journal of Research in Educational
Psychology, 8(1), 139–162.
Supovitz, J. A., & Turner, H. M. (2000). The effects of professional development on
science teaching practices and classroom culture. Journal of Research in Science
Teaching, 37(9), 963–980.
World Bank. (2010). Inside Indonesia’s mathematics classrooms: A TIMSS Video Study of
teaching practices and student achievement. Jakarta: Author.

175

Thesis Ari_141201_FINAL.pdf 175 1-12-2014 16:26:29

 Chapter 6

176

Thesis Ari_141201_FINAL.pdf 176 1-12-2014 16:26:29

 Summary

Samenvatting

Ringkasan

Acknowledgments

Curriculum vitae

List of presentations related to this thesis

FIsme Scientific Library

ICO Dissertation Series

Thesis Ari_141201_FINAL.pdf 177 1-12-2014 16:26:29

Thesis Ari_141201_FINAL.pdf 178 1-12-2014 16:26:30
 Summary

Summary

The Indonesian national curriculum emphasizes that mathematics education must

be relevant to the needs of life and should offer students opportunities to develop
the ability to apply their knowledge in society. Furthermore, there are educational
movements in Indonesia that emphasize the application of mathematics and
promote the use of context-based tasks, i.e. Pendidikan Matematika Realistik Indonesia
(Indonesian Realistic Mathematics Education) and Pembelajaran Kontekstual
(Contextual Teaching and Learning). Nevertheless, the results of PISA studies
showed that Indonesian students have poor performance on context-based tasks.
Considering this situation, this PhD research was carried out to gain a better
insight into Indonesian students´ low performance on context-based mathematics
tasks and to generate recommendations for improving student performance.

This PhD research addresses three main research questions:

1. What difficulties do Indonesian students have when solving context-based tasks?
2. What are possible reasons for Indonesian students’ difficulties when solving context-based
tasks?
3. How can Indonesian student performance on context-based tasks be improved?

Four studies were carried out to answer these research questions. The studies are
reported in chapters 2 to 5 of this thesis. Chapter 2 contributes to answering the
first research question, Chapter 3 and Chapter 4 address the second research
question, and Chapter 5 provides an answer to the last research question. Finally,
the findings of the four studies are synthesized in Chapter 6.

Chapter 2 describes the study into Indonesian students’ difficulties when solving
context-based tasks. In this first study, a total of 362 ninth- and tenth-grade
students from eleven schools from rural and urban areas in the Province of
Yogyakarta in Indonesia took a paper-and-pencil test on context-based PISA
mathematics tasks. To investigate the kinds of difficulties students have when
solving these tasks an error analysis was performed on the students’ responses. For
this purpose, an analysis framework was developed by combining Newman’s error
categories with the stages of modeling when solving context-based tasks and the
stages of mathematization as used in PISA. The analysis framework comprised
four types of errors: comprehension, transformation, mathematical processing, and
encoding.

179

Thesis Ari_141201_FINAL.pdf 179 1-12-2014 16:26:30

 The error analysis revealed that the most dominant errors made by the students
were comprehension and transformation errors. This finding indicates that
Indonesian students mostly have difficulties with the first two stages of solving
context-based tasks: comprehending a context-based task and transforming it into
a mathematical problem. With respect to the first stage it was found that the
students mainly made errors in selecting relevant information; whereas for the
second stage the students were mostly unable to select the procedures that are
suitable to solve the tasks.

After investigating students’ difficulties, the next step in this PhD research was
identifying possible explanations for these difficulties. For this purpose the
concept of ‘opportunity-to-learn’ was used as the key perspective. The
investigation focused on opportunities-to-learn in the used mathematics textbooks
as well as in the teaching practice of the teachers.

Chapter 3 reports on the study into the opportunity-to-learn offered in Indonesian

mathematics textbooks. Three mathematics textbooks were analyzed that are used
in the schools participating in the first study (see Chapter 2). This textbook analysis
study focused on investigating the mathematics tasks that are included in the
textbooks.

The analysis revealed that the investigated Indonesian mathematics textbooks have
a very low number of context-based tasks. Only 10% of tasks could be labeled as
context-based tasks. To get a better picture of the relation between opportunity-to-
learn offered in textbooks and student performance, an in-depth analysis of the
characteristics of these context-based tasks was carried out. It was found that three
quarters of the context-based tasks in the textbooks used a camouflage context
(which means that the tasks are merely dressed-up bare problems) and provided an
explicit indication about the mathematical procedure to be used. This finding
shows that Indonesian mathematics textbooks do not offer students enough
opportunity-to-learn to identify a suitable mathematical procedure to solve a
context-based task, which might explain the high number of transformation errors
made by the students. Moreover, the textbook analysis also disclosed that most
context-based tasks in Indonesian textbooks provided just precisely the
information needed to solve a task. This result signifies a lack of opportunities for
students to learn to select relevant information, which, therefore, might have
contributed to students’ comprehension errors, in particular errors in selecting
information.

180

Thesis Ari_141201_FINAL.pdf 180 1-12-2014 16:26:30

 Summary

Chapter 4 describes the study that focused on investigating the opportunity-to-learn

to solve context-based tasks offered by the teachers’ teaching practice. Teachers’
beliefs were also taken into account because they often influence the teaching
practice. The study started by surveying 27 teachers from seven schools which also
participated in the first study (see Chapter 2) through a written questionnaire. Then,
to further examine teachers’ teaching practice related to context-based tasks, four
teachers of this group of 27 teachers were observed and video-recorded in two
mathematics lessons in which they were asked to deal with context-based tasks.
The focus of the observations was to investigate what approach the teachers used
to help their students learn to solve context-based tasks.

The questionnaire revealed that the teachers tended to perceive context-based

tasks merely as plain word problems. They believed that the mathematical
procedure needed to solve a context-based task should be given explicitly. The
teachers also did not consider missing and superfluous information as an
important characteristic of a context-based task. Moreover, these beliefs were
reflected in their reported teaching practice. In agreement with their beliefs, the
teachers stated that they frequently offered their students context-based tasks with
explicit procedures and that they rarely gave context-based tasks that provided
superfluous or missing information. Such practice might explain Indonesian
students’ difficulties in identifying a suitable procedure and in selecting relevant
information.

The classroom observations showed that the teachers mainly used a directive
teaching approach in which they told the students what a context-based task is
about, translated the task into a mathematical problem, and explained what
mathematical procedure had to be carried out. Students were not encouraged to be
actively involved in and reflect on the process of solving context-based tasks. This
directive teaching approach was mostly used in the comprehension and
transformation stages. Consultative teaching in which students were actively
engaged in the process of solving context-based tasks was barely used by the
teachers. Remarkably, this consultative teaching approach was mostly used in the
mathematical processing stage; the stage in which students factually do not have to
deal with the context of a task.

Chapter 5 describes the final study of this PhD research in which the findings of the
first three studies were used to develop an intervention intended to offer students
opportunity-to-learn to solve context-based tasks. This intervention comprised two
components: a set of context-based tasks and guidelines for a consultative teaching

181

Thesis Ari_141201_FINAL.pdf 181 1-12-2014 16:26:30

 approach including giving students metacognitive prompts. The context-based
tasks used in the intervention had three important characteristics: including a
relevant and essential context, containing superfluous or missing information, and
not giving explicit suggestions about a suitable mathematical procedure. The
effects of this opportunity-to-learn on students’ performance in solving context-
based tasks were examined through a field experiment involving 299 eight-graders
from six schools. The students’ performance was seen from the perspectives of
students’ gain scores and students’ errors.

This study revealed that there was only a small and marginally significant effect of
the opportunity-to-learn on the students’ gain scores. Nevertheless, a closer
examination of the effect of the intervention on the students’ errors revealed a
positive influence. The students who received the opportunity-to-learn could
better understand the wording of a context-based task and had improved
performance in selecting relevant information. With regard to the students’
performance in transforming a real-world problem into a mathematical problem
and in identifying a suitable procedure no influence of the intervention was found
in general. However, a positive influence was found for context-based tasks
addressing graphs, which was the topic taught during the intervention period.
Students who were involved in the intervention were better able to give a
mathematical interpretation of graphs. This finding indicates that to improve
students’ ability to identify a suitable mathematical procedure it is important to
provide students with context-based tasks that address all kinds of topics and not
only offer them context-based tasks that are related to the topic that is currently
taught, because then the mathematical procedure is more or less given.

Chapter 6 provides an overview of the findings of the four studies and offers
recommendations for educational practice and for further research. The main
conclusions that can be drawn from this PhD research are that on the one hand
Indonesian students mainly had difficulties in comprehending context-based tasks
and in transforming them into a mathematical problem, and that on the other hand
it was found that there was a lack of opportunity-to-learn to solve context-based
tasks in textbooks and teachers’ teaching practice. On the basis of these research
findings three recommendations were given for improving the practice of teaching
context-based tasks. The first recommendation is to include more context-based
tasks in the learning materials; especially context-based tasks that use relevant and
essential contexts, have superfluous or missing information, and do not explicitly
signify what mathematical procedure is suitable. The second recommendation

182

Thesis Ari_141201_FINAL.pdf 182 1-12-2014 16:26:30

 Summary

addresses the teaching practice; that is, the use of a consultative approach and
make use in the teaching practice of the knowledge about students’ difficulties
when they have to solve context-based tasks. The third recommendation is to pay
attention in teacher education and professional development to teachers’ beliefs
and their knowledge about context-based tasks, because these two aspects are
essential for selecting good tasks (or designing them) and creating learning
opportunities with them.

183

Thesis Ari_141201_FINAL.pdf 183 1-12-2014 16:26:30

 184

Thesis Ari_141201_FINAL.pdf 184 1-12-2014 16:26:30

 Samenvatting

Samenvatting

In het Indonesische wiskundecurriculum wordt benadrukt dat het onderwijs moet

aansluiten bij de behoeften van de huidige maatschappij en aan leerlingen de
gelegenheid moet bieden om kennis en vaardigheden te ontwikkelen die ze in de
maatschappij kunnen gebruiken. Dit is in lijn met vernieuwingsprojecten zoals
Pendidikan Matematika Realistik Indonesia (Realistisch Wiskunde Onderwijs
Indonesië) en Pembelajaran Kontekstual (Contextrijk Onderwijs), die beide gericht
zijn op het leren toepassen van wiskunde en het gebruik van contextopgaven
stimuleren. Ondanks deze initiatieven laten de resultaten van PISA (Programme
for International Student Assessment) zien dat Indonesische leerlingen zwak
presteren op contextopgaven. Het onderhavige promotieonderzoek is opgezet om
te onderzoeken hoe dit komt en hoe die prestaties verbeterd kunnen worden.

De volgende drie vragen zijn onderzocht:

1. Welke problemen hebben Indonesische leerlingen bij het oplossen van contextopgaven?
2. Wat zijn mogelijke verklaringen voor de problemen van Indonesische leerlingen bij het
oplossen van contextopgaven?
3. Hoe kunnen de prestaties van Indonesische leerlingen bij contextopgaven verbeterd
worden?

De hoofdstukken 2, 3, 4 en 5 rapporteren over de vier studies die zijn uitgevoerd

om deze vragen te beantwoorden. De eerste onderzoeksvraag en de
beantwoording ervan zijn het onderwerp van hoofdstuk 2. De studies die
besproken worden in hoofdstuk 3 en hoofdstuk 4 richten zich op de tweede
onderzoeksvraag en hoofdstuk 5 heeft betrekking op de derde en laatste
onderzoeksvraag. Hoofdstuk 6 omvat een synthese van de resultaten van deze vier
studies.

Hoofdstuk 2 beschrijft het onderzoek naar de problemen die Indonesische

leerlingen hebben bij het oplossen van contextopgaven. Aan deze eerste studie
deden 362 leerlingen mee uit de derde en vierde klas voortgezet onderwijs van elf
Indonesische scholen gelegen in de provincie Yogyakarta. Het betrof zowel
scholen uit een landelijke omgeving als scholen in steden. De schriftelijk toets die
de leerlingen hebben gemaakt, bestond uit een aantal contextopgaven die ontleend
waren aan de vrijgegeven wiskundeopgaven van PISA. Om te onderzoeken wat nu
precies de problemen zijn die Indonesiche leerlingen tegenkomen bij deze

185

Thesis Ari_141201_FINAL.pdf 185 1-12-2014 16:26:30

 opgaven, is een foutenanalyse van het leerlingenwerk uitgevoerd. Hierbij is
gebruikgemaakt van een speciaal hiervoor ontwikkeld raamwerk. Bij dit raamwerk
is enerzijds uitgegaan van de foutencategorieën van Newman en anderzijds van de
verschillende fasen die bij het modelleren van wiskundeopgaven en bij de door
PISA gehanteerde definitie van mathematiseren onderscheiden worden. Dit
raamwerk omvatte vier type fouten: fouten in het begrijpen van de (tekst van de)
opgave, in het omzetten van de opgave naar een wiskundig probleem, in het
uitvoeren van de wiskundige bewerking en in het terugvertalen naar de context.

De foutenanalyse liet zien dat de meeste fouten van de leerlingen gemaakt werden
in het begrijpen van de opgaven en in het omzetten naar een wiskundige
bewerking. Deze bevinding impliceert dat de Indonesische leerlingen vooral moeite
hadden met de eerste twee fasen in het proces van het oplossen van
contextopgaven. Bij het begrijpen van de opgaven maakten de leerlingen vooral
fouten in het selecteren van de relevante gegevens en bij het omzetten naar een
wiskundige bewerking lukte het ze vaak niet om een passende wiskundige
procedure te vinden.

De volgende stap in het promotieonderzoek was het zoeken naar mogelijke

verklaringen voor deze fouten. Hierbij is nagegaan of aan de leerlingen wel
voldoende leermogelijkheden (opportunities-to-learn) worden geboden om
contextopgaven op te lossen. Dit is op twee manieren nagegaan. Eerst is naar
Indonesische wiskundemethodes gekeken en daarna naar de leermogelijkheden die
Indonesische leraren in hun lessen bieden.

Hoofdstuk 3 beschrijft het onderzoek naar de leermogelijkheden die in Indonesische

wiskundemethodes geboden worden voor het oplossen van contextopgaven. In dit
onderzoek zijn drie wiskundemethodes geanalyseerd die gebruikt werden op de
scholen die betrokken waren bij de eerste studie (zie hoofdstuk 2). De analyse
richtte zich op de opgaven die in deze wiskundemethodes zijn opgenomen. De
methode-analyse maakte duidelijk dat de onderzochte Indonesische methodes
nauwelijks contextopgaven bevatten. Slechts 10% van de opgaven waren
contextopgaven. Vervolgens zijn de kenmerken van deze contextopgaven verder
geanalyseerd met de bedoeling om te onderzoeken of er een relatie kon worden
vastgesteld tussen de leermogelijkheden die de wiskundemethodes bieden en de
fouten van de leerlingen. Hierbij kwam naar voren dat driekwart van de
contextopgaven in feite opgaven waren met een camouflagecontext (aangeklede
kale opgaven) en opgaven waarin expliciet is aangegeven welke wiskundige

186

Thesis Ari_141201_FINAL.pdf 186 1-12-2014 16:26:30

 Samenvatting

procedure nodig is om de opgaven te kunnen oplossen. Met andere woorden, de

onderzochte Indonesische methodes geven de leerlingen nauwelijks
leermogelijkheden voor het identificeren van de wiskundige procedure die nodig is
om een probleem op te lossen. Dit tekort in de leermogelijkheden die de methodes
bieden, kan dus als een mogelijke verklaring worden gezien voor het grote aantal
fouten die de Indonesische leerlingen maakten bij de omzetting van contextopgave
naar wiskundige bewerking. Een ander punt dat bij de methode-analyse naar voren
kwam, is dat de meeste contextopgaven in de Indonesische methodes precies die
gegevens bleken te bevatten die nodig zijn om de opgaven te kunnen oplossen,
waardoor leerlingen die met deze methodes werken dus weinig gelegenheid krijgen
om het selecteren van de relevante gegevens te leren, hetgeen weer de vele fouten
kan verklaren die de Indonesische leerlingen hierbij maakten.

Hoofdstuk 4 beschrijft de studie naar de leermogelijkheden die leraren in hun

onderwijs aan leerlingen bieden voor het oplossen van contextopgaven. Hierbij
werd niet alleen gekeken naar wat leraren in hun lessen doen, maar ook naar hun
opvattingen en naar hoe ze zelf tegen hun lespraktijk aankijken. Van de scholen die
betrokken waren bij de eerste studie (zie hoofdstuk 2), hebben 27 wiskundeleraren
aan dit verdere onderzoek meegedaan. Met een vragenlijst werden hun opvattingen
over allerlei aspecten van het wiskundeonderwijs en hun lespraktijk in kaart
gebracht. Daarnaast zijn bij vier leraren lesobservaties uitgevoerd om een dieper
inzicht in hun onderwijsaanpak te krijgen. Hierbij is vooral gekeken naar de manier
waarop de leraren hun leerlingen ondersteunden bij het leren oplossen van
contextopgaven.

De analyse van de vragenlijstgegevens liet zien dat de betrokken leraren de

contextopgaven vooral benaderden als wiskundeopgaven die in gewone taal
beschreven zijn (redactieopgaven) in plaats van met formele wiskundetaal. De
leraren waren de mening toegedaan dat bij zo’n opgave de uit te voeren wiskundige
procedure expliciet gegeven moet worden. De leraren zagen niet het belang in van
het gebruik van contextopgaven met te weinig of met te veel gegevens. Deze
opvattingen waren ook zichtbaar in datgene wat de leraren over hun lespraktijk
rapporteerden. In overeenstemming met hun opvattingen meldden de lerarendat
ze voornamelijk contextopgaven aan hun leerlingen gaven die expliciete
aanwijzingen bevatten over de wiskundige procedures en dat ze nauwelijks
opgaven gaven met ontbrekende of overbodige gegevens. Gezien deze lespraktijk
is het dus heel aannemelijk dat de leerlingen bij contextopgaven problemen hadden

187

Thesis Ari_141201_FINAL.pdf 187 1-12-2014 16:26:30

 met het identificeren van de wiskundige procedure en het selecteren van de
relevante gegevens.

De lesobservaties brachten naar voren dat de leraren vooral op een directieve

manier les gaven waarbij ze de leerlingen vertelden waar een contextopgave over
ging, welke gegevens gebruikt moesten worden en welke wiskundige procedure
uitgevoerd moest worden. De leerlingen werden niet aangemoedigd om zelf na te
denken over het oplossen van de opgaven. Een onderwijsaanpak waarbij
deleerlingen wel werden betrokken bij het selecteren van de relevante gegevens en
het kiezen van de wiskundige procedure, en waarbij vragen werden gesteld om
reflectie te stimuleren werd nauwelijks toegepast. Echter, de leraren lieten de
leerlingen wel meedenken bij het uitvoeren van de wiskundige procedures, terwijl
dit juist de fase is die los staat van de context.

Hoofdstuk 5 beschrijft de laatste studie die in het kader van dit promotieonderzoek
is uitgevoerd. In deze studie is een veldexperiment uitgevoerd om de bevindingen
van de tweede studie (leermogelijkheden in wiskundemethodes, zie hoofdstuk 3)
en de derde studie (leermogelijkheden in de lespraktijk, zie hoofdstuk 4) te toetsen.
Daarvoor is een interventie ontwikkeld waarin de leerlingen uitdrukkelijk
leermogelijkheden werden geboden voor het oplossen van contextopgaven. De
interventie bestond uit twee componenten: een verzameling contextopgaven en
een onderwijsaanpak waarbij de leerlingen door meta-cognitieve hints werden
gestimuleerd actief mee te denken bij het oplossen van contextopgaven. Deze
contextopgaven hadden drie belangrijke kenmerken: een relevante en essentiële
context, overbodige of ontbrekende gegevens, en geen expliciete aanwijzingen
voor de uit te voeren wiskundige procedures. Om het effect van de interventie te
meten is een voortoets en een natoets afgenomen bestaande uit contextopgaven
die ontleend waren aan vrijgegeven PISA-opgaven. Het experiment is uitgevoerd
met derde klassen voortgezet onderwijs van zes scholen, waarbij op elke school
een experimentele klas en een controleklas heeft meegedaan. In totaal zijn de
gegevens van 299 leerlingen (met een gemiddelde leeftijd van 13.7 jaar)
geanalyseerd waarbij zowel gekeken is naar het verschil in goedscore tussen de
voortoets en natoets als naar het type gemaakte fouten.

De analyses lieten zien dat een onderwijsaanpak waarbij de leraren de leerlingen

ervaring lieten opdoen in het zelf selecteren van relevante gegevens en het zelf
kiezen van de wiskundige bewerking, en waarbij de leraren vragen stelden om
reflectie bij de leerlingen te stimuleren, inderdaad tot minder fouten leidde bij het

188

Thesis Ari_141201_FINAL.pdf 188 1-12-2014 16:26:30

 Samenvatting

maken van de contextopgaven. De leerlingen uit de experimentele groep begrepen

de contextopgaven beter en waren beter in staat om de relevante gegevenste
selecteren dan de leerlingen uit de controlegroep. De interventie had echter geen
effect op de vaardigheid van de leerlingen om contextopgaven om te zetten naar
een wiskundig probleem. Daarentegen werd wel een positieve invloed van de
interventie gevonden op het oplossen van contextopgaven waarin grafieken een rol
speelden. Dit heeft waarschijnlijk te maken met het feit dat de interventie
betrekking had op het onderwerp grafieken en de leerlingen in de experimentele
groep voor dit onderwerp dus meer actieve leermogelijkheden hebben gehad dan
de leerlingen in de controlegroep. Deze bevinding geeft wel aan dat bij het bieden
van leermogelijkheden voor het oplossen van contextopgaven voorkomen moet
worden dat de contextopgaven per onderwerp worden aangeboden. Bij een
hoofdstuk over grafieken is het voor de leerlingen duidelijk dat de contextopgave
die na zo’n hoofdstuk gegeven wordt met een grafiek moet worden opgelost. De
leerlingen krijgen op die manier geen ervaring in het omzetten van een
contextopgave naar een wiskundig probleem. Beter is het dus om bij het bieden
van leermogelijkheden voor het oplossen van contextopgaven de wiskundige
onderwerpen af te wisselen.

Hoofdstuk 6 levert een overzicht van de resultaten van de vier studies en geeft
aanbevelingen voor de lespraktijk en voor vervolgonderzoek. De belangrijkste
conclusies van dit promotieonderzoek zijn dat Indonesische leerlingen van de ene
kant problemen hebben met het begrijpen van wat precies in een contextopgave
gevraagd wordt en met het vertalen van contextopgaven naar een wiskundig
probleem en dat van de andere kant wiskundemethodes en de lespraktijk van
leraren de leerlingen nauwelijks gelegenheid bieden om het oplossen van
contextopgaven te leren. Daarom zijn op basis van dit onderzoek drie
aanbevelingen geformuleerd. De eerste aanbeveling is het opnemen van meer
contextopgaven in het lesmateriaal, in het bijzonder moeten daarin contextopgaven
worden opgenomen met relevante en essentiële contexten, met ontbrekende of
overbodige gegevens en zonder expliciete aanwijzingen voor de te gebruiken
wiskundige procedure. De tweede aanbeveling heeft betrekking op de manier
waarop contextopgaven onderwezen moeten worden. Leraren zouden hun
leerlingen actief mee moeten laten denken bij het oplossen van de opgaven. Ook
zouden leraren meer oog moeten hebben voor de fouten die de leerlingen in de
verschillende fasen van het oplossingsproces maken. Hiermee zouden de leraren
rekening moeten houden bij hun onderwijsaanpak. De derde aanbeveling is dat er
in de opleiding en begeleiding van leraren aandacht moet zijn voor de opvattingen

189

Thesis Ari_141201_FINAL.pdf 189 1-12-2014 16:26:30

 en kennis die leraren hebben over contextopgaven en het belang ervan, omdat
deze beide uiteindelijk bepalend zijn voor het kiezen (of ontwerpen) van geschikte
contextopgaven en voor het creëren van leermogelijkheden.

190

Thesis Ari_141201_FINAL.pdf 190 1-12-2014 16:26:30

 Ringkasan

Ringkasan

Di dalam kurikulum nasional Indonesia ditekankan bahwa pendidikan matematika

harus relevan untuk kebutuhan kehidupan dan memberikan siswa kesempatan
untuk mengembangkan kemampuan dalam menerapkan pengetahuan. Sejalan
dengan hal tersebut, di Indonesia terdapat gerakan pendidikan yang menekankan
pada aplikasi matematika dan penggunaan soal berbasis konteks, yaituPendidikan
Matematika Realistik Indonesia dan Pembelajaran Kontekstual. Namun demikian, hasil
tes PISA menunjukkan rendahnya kemampuan siswa Indonesia dalam
menyelesaikan soal berbasis konteks. Mempertimbangkan situasi tersebut,
penelitian PhD ini ditujukan untuk memperoleh pemahaman yang lebih baik
tentang rendahnya pencapaian siswa Indonesia dalam menyelesaikan soal berbasis
konteks serta untuk merumuskan rekomendasi untuk meningkatkan pencapaian
siswa.

Dalam PhD tesis ini terdapat tiga pertanyaan utama yang dibahas:
1. Kesulitan apakah yang dihadapi siswa Indonesia saat mengerjakan soal berbasis konteks?
2. Apakah kemungkinan penyebab kesulitan siswa dalam mengerjakan soal berbasis
konteks?
3. Bagaimana kemampuan siswa Indonesia dalam mengerjakan soal berbasis konteks dapat
ditingkatkan?

Bab 2 sampai 5 dalam PhD tesis ini menyajikan hasil dari empat penelitian yang
dilaksanakan untuk menjawab ketiga pertanyaan utama penelitian tersebut di atas.
Bab 2 ditujukan untuk menjawab pertanyaan pertama, Bab 3 dan Bab 4 membahas
pertanyaan kedua, dan Bab 5 menampilkan jawaban untuk pertanyaan ketiga. Hasil
dari empat penelitian tersebut dirangkum dan disintesis di Bab 6.

Bab 2 membahas penelitian pertama yang ditujukan untuk mengidentifikasi

kesulitan siswa dalam mengerjakan soal berbasis konteks. Penelitian ini
dilaksanakan di 11 sekolah di provinsi Dareah Istimewa Yogyakarta dan melibatkan
362 siswa kelas 9 dan 10. Siswa tersebut berpartisipasi dalam suatu tes yang
menggunakan soal berbasis konteks. Analisis kesalahan (error analysis) dilakukan
untuk mengidentifikasi jenis kesulitan siswa saat mengerjakan soal berbasis
konteks. Untuk hal ini, suatu kerangka analisis dikembangkan dengan mengaitkan
kategori kesalahan siswa yang dirumuskan Newman (Newman’s error categories)
dengan tahapan pada pemodelan (modeling process) dan matematisasi PISA.

191

Thesis Ari_141201_FINAL.pdf 191 1-12-2014 16:26:30

 Kerangka analisis ini memuat empat jenis kesalahan, yaitu: pemahaman
(comprehension), transformasi (transformation), pemrosesan matematis (mathematical
processing), dan penafsiran (encoding).

Analisis kesalahan mengungkapkan bahwa jenis kesalahan yang paling sering

dilakukan siswa adalah kesalahan terkait pemahaman dan transformasi. Temuan ini
menunjukkan bahwa kesulitan terbesar dialami siswa Indonesia di dua tahap awal
penyelesaian soal berbasis konteks, yaitu dalam memahami maksud soal dan dalam
mengubah soal tersebut menjadi permasalahan matematika. Terkait tahap pertama
penyelesaian soal banyak siswa yang membuat kesalahan dalam memilih informasi
yang relevan, sedangkan pada tahap kedua siswa tidak bisa memilih prosedur atau
konsep matematika yang tepat untuk menyelesaikan soal.

Setelah meneliti kesulitan siswa, langkah selanjutnya dalam penelitian PhD ini
adalah mengidentifikasi kemungkinan penyebab kesulitan tersebut. Untuk hal ini
konsep ‘kesempatan belajar’ (opportunity-to-learn) digunakan sebagai perspeksi utama.

Bab 3 memuat penelitian tentang ‘kesempatan belajar’ mengerjakan soal berbasis

konteks yang ditawarkan buku teks di Indonesia. Dalam penelitian ini dilakukan
analisis terhadap tiga buku teks yang digunakan di sekolah yang berpartisipasi
dalam penelitian pertama (yaitu Bab 2). Fokus analisis buku (textbook analysis) ini
adalah soal-soal matematika yang tersedia dalam ketiga buku tersebut.

Hasil analisis menunjukkan kurangnya soal berbasis konteks di ketiga buku teks
matematika yang dianalisis. Soal berbasis konteks hanya sebanyak 10% dari
keseluruhan soal yang ada dalam buku. Untuk mendapatkan pemahaman yang
lebih baik tentang hubungan antara ‘kesempatan belajar’ dengan tingkat pencapaian
siswa, dilakukan analisis lanjutan untuk mengidentifikasi karakteristik soal berbasis
konteks. Analisis ini mengungkap bahwa tiga perempat dari soal berbasis konteks
yang ada di buku menggunakan konteks kamuflase (yaitu konteks dapat diacuhkan
dalam penyelesaian soal) dan menampilkan prosedur penyelesaian secara jelas.
Temuan ini menunjukkan bahwa buku teks matematika di Indonesia tidak
menyediakan kesempatan yang cukup kepada siswa untuk belajar mengidentifikasi
prosedur atau konsep matematika yang dibutuhkan untuk menyelesaikan suatu soal
berbasis konteks. Hal ini mungkin merupakan salah satu alasan atas banyaknya
kesalahan transformasi yang dilakukan siswa. Lebih lanjut lagi, hasil analisis buku
juga mengungkap bahwa mayoritas soal berbasis konteks yang ada di buku teks
hanya menyediakan informasi yang dibutuhkan untuk menjawab soal. Hal ini
menunjukkan kurangnya kesempatan untuk siswa belajar memilih informasi yang

192

Thesis Ari_141201_FINAL.pdf 192 1-12-2014 16:26:30

 Ringkasan

relevan. Hal tersebut berperan pada banyaknya kesalahan pemahaman yang dibuat
siswa, khususnya kesalahan dalam memilih informasi.

Bab 4 memuat penelitian tentang ‘kesempatan belajar’ menyelesaikan soal berbasis

konteks yang ditawarkan melalui praktik mengajar guru. Keyakinan (beliefs) guru
juga diteliti karena hal tersebut sering mempengaruhi bentuk praktik mengajar
guru. Penelitian ini melibatkan 27 guru matematika dari sekolah yang berpartisipasi
dalam penelitian pertama (Bab 2). Seluruh guru tersebut diminta mengisi angket
tentang keyakinan dan praktik mengajar guru. Untuk meneliti lebih jauh tentang
praktik mengajar guru, observasi kelas dilakukan di empat kelas dimana para guru
tersebut diminta menggunakan soal berbasis konteks dalam kegiatan belajar
mengajar. Fokus observasi kelas adalah meneliti strategi mengajar yang digunakan
guru untuk membantu siswanya belajar menyelesaikan soal berbasis konteks.

Angket guru menunjukkan bahwa para guru menganggap soal berbasis konteks
sekadar soal cerita biasa (word problems), yaitu soal yang menyediakan prosedur
penyelesaian secara jelas. Selain itu, mereka juga menganggap bahwa soal berbasis
kontek tidak perlu memuat informasi berlebih (superfluous information) maupun
memiliki informasi terselubung (missing information). Keyakinan guru tersebut juga
tercermin pada praktik mengajar guru. Para guru melaporkan bahwa mereka sering
memberi siswa soal yang memuat prosedur penyelesaian secara jelas tetapi jarang
memberi siswa soal yang memuat informasi berlebih ataupun informasi
terselubung. Praktik mengajar semacam itu merupakan salah satu kemungkinan
penyebab kesulitan siswa dalam mengidentifikasi prosedur matematika untuk
menyelesaikan soal dan dalam memilih informasi yang relevan.

Hasil observasi kelas menunjukkan bahwa para guru lebih sering menggunakan
‘pengajaran langsung’ (directive teaching) dimana mereka: memberi tahu siswa apa
yang dimaksud dalam soal, mengubah soal menjadi bentuk permasalahan
matematika, dan menjelaskan prosedur atau konsep matematika apa yang
dibutuhkan. Siswa tidak diminta untuk terlibat secara aktif dalam menyelesaikan
dan melakukan refleksi atas proses penyelesaian soal. Praktik pengajaran secara
langsung tersebut paling sering digunakan di dua tahap awal penyelesaian soal:
pemahaman (comprehension) dan transformasi (transformation). Pengajaran konsultatif
(consultative teaching), dimana siswa terlibat secara aktif dalam penyelesaian soal,
sangat jarang digunakan oleh guru. Pengajaran konsultatif tersebut digunakan guru
di tahap pemrosesan matematis (mathematical processing), yaitu tahapan dimana
konteks dunia nyata (real-world contexts) bukan menjadi fokus utama.

193

Thesis Ari_141201_FINAL.pdf 193 1-12-2014 16:26:30

 Bab 5 memuat penelitian terakhir dalam proyek PhD ini dimana hasil dari tiga
penelitian awal dijadikan sebagai dasar untuk mengembangkan suatu program
intervensi untuk memberi siswa kesempatan belajar menyelesaikan soal berbasis
konteks. Intervensi ini terdiri dari dua komponen: soal berbasis konteks dan
pengajaran konsultatif (consultative teaching). Soal berbasis konteks yang digunakan
dalam intervensi memiliki tiga karakter penting: menggunakan konteks yang
relevan, memuat informasi berlebih dan informasi terselubung, serta tidak
menampilkan prosedur penyelesaian secara langsung. Pengaruh pemberian
kesempatan belajar ini pada kemampuan siswa dalam menyelesaikan soal berbasis
konteks diteliti melalui suatu eksperimen yang melibatkan 299 siswa kelas 8 dari
enam sekolah. Pencapaian siswa dilihat dari dua aspek, yaitu peningkatan skor (score
gains) dan kesalahan siswa (students’ errors).

Hasil eksperimen menunjukkan bahwa efek intervensi pada peningkatan skor siswa
hanya signifikan secara marginal (marginally significant). Namun demikian, penelitian
lanjutan pada efek intervensi terhadap kesalahan siswa menunjukkan adanya
pengaruh yang positif. Siswa yang menerima intervensi memiliki peningkatan
kemampuan dalam memahami soal dan memilah informasi. Terkait kemampuan
siswa dalam melakukan transformasi soal berbasis konteks menjadi bentuk
permasalahan matematika, untuk keseluruhan soal tidak ditemukan pengaruh dari
intervensi tetapi untuk soal yang berkaitan dengan grafik, yaitu soal yang diajarkan
saat intervensi, ditemukan pengaruh positif. Untuk soal yang terkait grafik siswa di
kelas eksperimen memiliki kemampuan yang lebih baik dalam memberikan
interpretasi matematis untuk suatu grafik yang terkait konteks dunia nyata.
Berdasarkan temuan tersebut, dapat disimpulkan bahwa untuk meningkatkan
kemampuan siswa dalam mengidentifikasi konsep atau prosedur matematis untuk
menyelesaikan soal berbasis konteks tidaklah cukup hanya memberikan soal yang
berkaitan dengan topik yang sedang diajarkan, melainkan juga soal yang terkait
dengan berbagai macam topik berbeda.

Bab 6 menampilkan ikhtisar dari hasil penelitian PhD ini serta menawarkan
rekomendasi untuk praktik pendidikan dan penelitian lanjutan. Kesimpulan utama
dari hasil penelitian PhD ini adalah: (1) kesulitan terbesar yang dialami siswa
Indonesia dalam menyelesaikan soal berbasis konteks adalah dalam proses
memahami soal dan mengubahnya ke dalam permasalahan matematika, dan (2)
kurangnya kesempatan belajar soal berbasis konteks yang diperoleh siswa, baik di
buku teks maupun dalam praktik mengajar guru. Hasil tersebut menunjukkan
adanya hubungan antara kesulitan siswa dengan kesempatan belajar yang diperoleh

194

Thesis Ari_141201_FINAL.pdf 194 1-12-2014 16:26:30

 Ringkasan

siswa. Hasil penelitian juga menunjukkan bahwa mendiagnosis kesulitan siswa dan
mengidentifikasi kesempatan belajar yang diperoleh siswa merupakan dua langkah
penting untuk merumuskan cara meningkatkan pencapaian siswa. Berdasarkan
hasil penelitian PhD ini, dapat dirumuskan beberapa rekomendasi untuk
meningkatkan praktik pengajaran soal berbasis konteks. Pertama, memberi siswa
lebih banyak soal berbasis konteks, khususnya soal yang: (1) menggunakan konteks
yang relevan, (2) memuat informasi berlebih dan informasi terselubung, dan (3)
tidak menampilkan secara langsung prosedur matematika yang dibutuhkan untuk
menjawab soal. Kedua, menggunakan pendekatan konsultatif serta memasukkan
kegiatan mendiagnosis kesulitan siswa sebagai bagian terpadu dari praktik mengajar
guru. Ketiga, pentingnya memberi perhatian pada keyakinan dan pengetahuan guru
tentang soal berbasis konteks pada program pendidikan dan pelatihan guru karena
kedua hal tersebut merupakan modal penting untuk meningkatkan kemampuan
guru dalam memilih (atau mengembangkan) soal serta dalam menciptakan
kesempatan belajar untuk siswa.

195

Thesis Ari_141201_FINAL.pdf 195 1-12-2014 16:26:30

 196

Thesis Ari_141201_FINAL.pdf 196 1-12-2014 16:26:30

 Acknowledgements

Acknowledgements

“The Milky Way is nothing else but a mass of innumerable stars planted together in clusters.”

The abovementioned statement of Galileo Galilei has always been one of my

favorite quotes because it reflects my academic journey. This quote also perfectly
represents the completion of this thesis which would not have been possible
without support, guidance, and help from many people around me. Therefore, I
would like to dedicate this section to acknowledge these people.

First, I would like to express my sincere gratitude to my promotor, Marja van den
Heuvel-Panhuizen, who has supported and guided me with her impressive
knowledge and expertise. It was a great opportunity for me to do my PhD research
under her guidance. She guided me to see things in a comprehensive and coherent
way that was not only important for my PhD work, but will also be essential for
my future professional life. The way she guided me reminds me of another quote
from Galileo Galilei: “we cannot teach people anything; we can only help them discover it
within themselves.” She gave me this opportunity to learn.

I also owe a debt of gratitude to Michiel Doorman, my copromotor, for his

valuable guidance and support. He often posed questions that stimulated me to
further reflect on my own thought and works. Furthermore, he did not only give
me professional and academic supports, but also personal ones. He even took me
to the Assen circuit – one of the places in my dream list – to watch British
Superbike. Thank you very much for this experience, Michiel.

I would like to acknowledge the Indonesian Ministry of Education and Culture for
the scholarship I received. I would also like to express my gratitude to the
authorities of Yogyakarta State University, the university where I work as a
lecturer, for giving me permission to pursue my doctoral degree.

During my PhD study I received support and help from my colleagues at the
Freudenthal Institute for Science and Mathematics Education. Special thanks to
my officemates, Michiel Veldhuis and Xiaoyan Zhao, for the encouragement,
fruitful discussion, and also our routine ‘lunch 'meeting’. I would also like to thank
Al Jupri, my fellow Indonesian PhD student, for helping me with interrater
reliability and many other things. I am grateful to Nathalie Kuijpers for correcting
my written English and for her editorial assistance. Thanks to Mark Uwland and

197

Thesis Ari_141201_FINAL.pdf 197 1-12-2014 16:26:30

 Heleen Verhage for arranging the financial matter, especially in the final year of my
study.

When dealing with statistical analyses of my data I also got help from Alexander
Robitzsch and Michiel Veldhuis. Thanks to both of you. I am also deeply indebted
to the teachers and the students who participated in my studies. Completion of my
PhD research would have never been possible without their help and cooperation.

I would like to extend my gratitude to my Indonesian friends who helped me

during my stay in the Netherlands. Special thanks to oom Richard and tante Nanda
who have given me a lot of help since my arrival in the Netherlands. They were
really like my parents in the Netherlands. I owe my gratitude to Domesia Novi for
designing the cover of this PhD thesis. I am also grateful to: Eko Handoyo, Richo,
Siswanto, Gustiandi, Supardi, Arie Fiandri, Bambang, Yusuf Setiyono, Bangaip,
and Citra Dewi.

Above all, I would like to express my immeasurable gratitude to my wife, Endang

Ilmi NH, for her love and understanding. Without her unconditional support I
would never have completed my study. Although she was busy with our son and
her teaching job, she was always there whenever I needed her. Furthermore, she
did not only support my personal life, but also helped me with the data collection
and data analysis in some of my studies. To my son, Arkapraba Wijaya, I am sorry
for the time we spent apart. I was not even there when he took his first steps. I
dedicate this book to the two of you. Special thanks to my parents, my parents in
law, my sisters, and my brothers who always supported and encouraged me.

198

Thesis Ari_141201_FINAL.pdf 198 1-12-2014 16:26:30

 Curriculum Vitae

Curriculum Vitae

Ariyadi Wijaya was born on July 16, 1982 in Banjarnegara, Indonesia. In 2000, he
completed his secondary schooling at Sandhy Putra Technical High School of
Telecommunication in Purwokerto and then started his university studies in
Mathematics Education at Yogyakarta State University, Indonesia. After obtaining
a bachelor degree, with honors, in 2004, he immediately worked as a senior high
mathematics teacher. In 2005 he started a new job as a lecturer at the Department
of Mathematics Education, the Faculty of Mathematics and Natural Sciences,
Yogyakarta State University (YSU). He pursued a master degree in Science,
Communication and Education at Utrecht University, the Netherlands in 2006 and
graduated with honors in 2008.

In 2009, he was appointed by YSU to take a short course on ‘Content Language

Integrated Learning’ (CLIL) at the Melbourne Graduate School of Education, the
University of Melbourne. In addition to his duty as a lecturer at YSU, in the period
from 2009 to 2010 he also frequently gave lectures in short courses which were
organized by the Southeast Asian Ministers of Education Organization
(SEAMEO) Regional Centre for Quality Improvement of Teachers and Education
Personnel (QITEP) in Mathematics. He published three books about mathematics
education and frequently participated in conferences on mathematics education.

In 2011, he was awarded a scholarship from the Indonesian Directorate General of

Higher Education. In the same year he started his PhD study at Utrecht University
under supervision of prof. dr. Marja van den Heuvel-Panhuizen and dr. Michiel
Doorman. His PhD researchfocused on the teaching and learning of context-based
mathematics tasks in Indonesia. His research proposal was accepted by the
Interuniversity Center for Educational Sciences (ICO) Research School in the
Netherlands and he fulfilled all requirements to be qualified as an ICO PhD
member.

In February 2015 he will go back to his position as a lecturer at the YSU to teach
in the Undergraduate Program and Graduate Program of mathematics education.

199

Thesis Ari_141201_FINAL.pdf 199 1-12-2014 16:26:30

 200

Thesis Ari_141201_FINAL.pdf 200 1-12-2014 16:26:30

 Presentations

List of presentations related to this thesis

Wijaya, A., Van den Heuvel-Panhuizen, M., & Doorman, M. Identifying ways to
improve student performance on context-based mathematics tasks. Paper that will be
presented at the 9th Congress of European Research in Mathematics
Education (CERME 9), Prague, Czech Republic, 4-8 February 2015.
Wijaya, A., Van den Heuvel-Panhuizen, M., Doorman, M., & Robitzsch, A. (2014,
November). Identifying (Indonesian) students’ difficulties in solving context-based
(PISA) mathematics tasks. Paper presented at International Seminar on
Innovation in Mathematics and Mathematics Education 2014, Yogyakarta,
Indonesia.
Wijaya, A., Van den Heuvel-Panhuizen, M., & Doorman, M. (2013, November).
Analyzing Indonesian mathematics textbooks. Do they provide opportunity-to-learn
context-based tasks? Paper presented at ICO National Fall School 2013,
Maastricht, the Netherlands.
Wijaya, A., Van den Heuvel-Panhuizen, M., & Doorman, M. (2012, November).
An investigation of opportunities to learn solving context-based tasks in Indonesian
mathematics textbooks and classroom practices. Paper presented at ICO International
Fall School 2012, Girona, in Cataluña, Spain.

201

Thesis Ari_141201_FINAL.pdf 201 1-12-2014 16:26:30

 202

Thesis Ari_141201_FINAL.pdf 202 1-12-2014 16:26:30

 FIsme Scientific Library

FIsme Scientific Library

(formerly published as CD-Ⱦ Scientific Library)

87. Klerk, Saskia (2015). Galen reconsidered. Studying drug properties and the foundations of medicine in the
Dutch Republic ca. 1550-1700.
86. Krüger, Jenneke (2014). Actoren en factoren achter het wiskundecurriculum sinds 1600.
85. Lijnse, P.L. (2014). Omzien in verwondering. Een persoonlijke terugblik op 40 jaar werken in de
natuurkundedidactiek.
84 Van Weelie, D. (2014). Recontextualiseren van het concept biodiversiteit.
83 Bakker, M. (2014). Using mini-games for learning multiplication and division: A longitudinal effect study.
82. HĈng, N. V. T. (2014). Design of a social constructivism-based curriculum for primary science education in
Confucian heritage culture.
81. Sun, L. (2014). From rhetoric to practice: enhancing environmental literacy of pupils in China.
80. Mazereeuw, M. (2013). The functionality of biological knowledge in the workplace. Integrating school and
workplace learning about reproduction.
79. Dierdorp, A. (2013). Learning correlation and regression within authentic contexts.
78. Dolfing, R. (2013). Teachers’ Professional Development in Context-based Chemistry Education. Strategies
to Support Teachers in Developing Domain-specific Expertise.
77. Mil, M. H. W. van (2013). Learning and teaching the molecular basis of life.
76. Antwi, V. (2013). Interactive teaching of mechanics in a Ghanaian university context.
75. Smit, J. (2013). Scaffolding language in multilingual mathematics classrooms.
74. Stolk, M. J. (2013). Empowering chemistry teachers for context-based education. Towards a framework for
design and evaluation of a teacher professional development programme in curriculum innovations.
73. Agung, S. (2013). Facilitating professional development of Madrasah chemistry teachers. Analysis of its
establishment in the decentralized educational system of Indonesia.
72. Wierdsma, M. (2012). Recontextualising cellular respiration.
71. Peltenburg, M. (2012). Mathematical potential of special education students.
70. Moolenbroek, A. van (2012). Be aware of behaviour. Learning and teaching behavioural biology in
secondary education.
69. Prins, G. T., Vos, M. A. J. & Pilot, A. (2011). Leerlingpercepties van onderzoek & ontwerpen in het
technasium.
68. Bokhove, Chr. (2011). Use of ICT for acquiring, practicing and assessing algebraic expertise.
67. Boerwinkel, D. J. & Waarlo, A. J. (2011). Genomics education for decision-making. Proceedings of the
second invitational workshop on genomics education, 2-3 December 2010.
66. Kolovou, A. (2011). Mathematical problem solving in primary school.
65. Meijer, M. R. (2011). Macro-meso-micro thinking with structure-property relations for chemistry. An
explorative design-based study.
64. Kortland, J. & Klaassen, C. J. W. M. (2010). Designing theory-based teaching-learning sequences for
science. Proceedings of the symposium in honour of Piet Lijnse at the time of his retirement as professor of
Physics Didactics at Utrecht University.
63. Prins, G. T. (2010).Teaching and learning of modelling in chemistry education. Authentic practices as
contexts for learning.

203

Thesis Ari_141201_FINAL.pdf 203 1-12-2014 16:26:30

 62. Boerwinkel, D. J. & Waarlo, A. J. (2010). Rethinking science curricula in the genomics era. Proceedings
of an invitational workshop.
61. Ormel, B. J. B. (2010). Het natuurwetenschappelijk modelleren van dynamische systemen. Naar een
didactiek voor het voortgezet onderwijs.
60. Hammann, M., Waarlo, A. J., & Boersma, K. Th. (Eds.) (2010). The nature of research in biological
education: Old and new perspectives on theoretical and methodological issues – A selection of papers presented
at the VIIth Conference of European Researchers in Didactics of Biology.
59. Van Nes, F. (2009). Young children's spatial structuring ability and emerging number sense.
58. Engelbarts, M. (2009). Op weg naar een didactiek voor natuurkunde-experimenten op afstand. Ontwerp
en evaluatie van een via internet uitvoerbaar experiment voor leerlingen uit het voortgezet onderwijs.
57. Buijs, K. (2008). Leren vermenigvuldigen met meercijferige getallen.
56. Westra, R. H. V. (2008). Learning and teaching ecosystem behaviour in secondary education: Systems
thinking and modelling in authentic practices.
55. Hovinga, D. (2007). Ont-dekken en toe-dekken: Leren over de veelvormige relatie van mensen met natuur
in NME-leertrajecten duurzame ontwikkeling.
54. Westra, A. S. (2006). A new approach to teaching and learning mechanics.
53. Van Berkel, B. (2005). The structure of school chemistry: A quest for conditions for escape.
52. Westbroek, H. B. (2005). Characteristics of meaningful chemistry education: The case of water quality.
51. Doorman, L. M. (2005). Modelling motion: from trace graphs to instantaneous change.
50. Bakker, A. (2004). Design research in statistics education: on symbolizing and computer tools.
49. Verhoeff, R. P. (2003). Towards systems thinking in cell biology education.
48. Drijvers, P. (2003). Learning algebra in a computer algebra environment. Design research on the
understanding of the concept of parameter.
47. Van den Boer, C. (2003). Een zoektocht naar verklaringen voor achterblijvende prestaties van allochtone
leerlingen in het wiskundeonderwijs.
46. Boerwinkel, D. J. (2003). Het vormfunctieperspectief als leerdoel van natuuronderwijs. Leren kijken door
de ontwerpersbril.
45. Keijzer, R. (2003). Teaching formal mathematics in primary education. Fraction learning as mathematising
process.
44. Smits, Th. J. M. (2003). Werken aan kwaliteitsverbetering van leerlingonderzoek: Een studie naar de
ontwikkeling en het resultaat van een scholing voor docenten.
43. Knippels, M. C. P. J. (2002). Coping with the abstract and complex nature of genetics in biology education
– The yo-yo learning and teaching strategy.
42. Dressler, M. (2002). Education in Israel on collaborative management of shared water resources.
41. Van Amerom, B. A. (2002). Reinvention of early algebra: Developmental research on the transition from
arithmetic to algebra.
40. Van Groenestijn, M. (2002). A gateway to numeracy. A study of numeracy in adult basic education.
39. Menne, J. J. M. (2001). Met sprongen vooruit: een productief oefenprogramma voor zwakke rekenaars in
het getallengebied tot 100 – een onderwijsexperiment.
38. De Jong, O., Savelsbergh, E. R. & Alblas, A. (2001). Teaching for scientific literacy: context,
competency, and curriculum.
37. Kortland, J. (2001). A problem-posing approach to teaching decision making about the waste issue.
36. Lijmbach, S., Broens, M., & Hovinga, D. (2000). Duurzaamheid als leergebied; conceptuele analyse en
educatieve uitwerking.

204

Thesis Ari_141201_FINAL.pdf 204 1-12-2014 16:26:31

 FIsme Scientific Library

35. Margadant-van Arcken, M. & Van den Berg, C. (2000). Natuur in pluralistisch perspectief –
Theoretisch kader en voorbeeldlesmateriaal voor het omgaan met een veelheid aan natuurbeelden.
34. Janssen, F. J. J. M. (1999). Ontwerpend leren in het biologieonderwijs. Uitgewerkt en beproefd voor
immunologie in het voortgezet onderwijs.
33. De Moor, E. W. A. (1999). Van vormleer naar realistische meetkunde – Een historisch-didactisch
onderzoek van het meetkundeonderwijs aan kinderen van vier tot veertien jaar in Nederland gedurende de
negentiende en twintigste eeuw.
32. Van den Heuvel-Panhuizen, M. & Vermeer, H. J. (1999). Verschillen tussen meisjes en jongens bij
het vak rekenen-wiskunde op de basisschool – Eindrapport MOOJ-onderzoek.
31. Beeftink, C. (2000). Met het oog op integratie – Een studie over integratie van leerstof uit de
natuurwetenschappelijke vakken in de tweede fase van het voortgezet onderwijs.
30. Vollebregt, M. J. (1998). A problem posing approach to teaching an initial particle model.
29. Klein, A. S. (1998). Flexibilization of mental arithmeticsstrategies on a different knowledge base – The
empty number line in a realistic versus gradual program design.
28. Genseberger, R. (1997). Interessegeoriënteerd natuur- en scheikundeonderwijs – Een studie naar
onderwijsontwikkeling op de Open Schoolgemeenschap Bijlmer.
27. Kaper, W. H. (1997). Thermodynamica leren onderwijzen.
26. Gravemeijer, K. (1997). The role of context and models in the development of mathematical strategies and
procedures.
25. Acampo, J. J. C. (1997). Teaching electrochemical cells – A study on teachers’ conceptions and teaching
problems in secondary education.
24. Reygel, P. C. F. (1997). Het thema 'reproductie' in het schoolvak biologie.
23. Roebertsen, H. (1996). Integratie en toepassing van biologische kennis – Ontwikkeling en onderzoek van
een curriculum rond het thema 'Lichaamsprocessen en Vergift'.
22. Lijnse, P. L. & Wubbels, T. (1996). Over natuurkundedidactiek, curriculumontwikkeling en
lerarenopleiding.
21. Buddingh', J. (1997). Regulatie en homeostase als onderwijsthema: een biologie-didactisch onderzoek.
20. Van Hoeve-Brouwer G. M. (1996). Teaching structures in chemistry – An educational structure for
chemical bonding.
19. Van den Heuvel-Panhuizen, M. (1996). Assessment and realistic mathematics education.
18. Klaassen, C. W. J. M. (1995). A problem-posing approach to teaching the topic of radioactivity.
17. De Jong, O., Van Roon, P. H. & De Vos, W. (1995). Perspectives on research in chemical education.
16. Van Keulen, H. (1995). Making sense – Simulation-of-research in organic chemistry education.
15. Doorman, L. M., Drijvers, P. & Kindt, M. (1994). De grafische rekenmachine in het
wiskundeonderwijs.
14. Gravemeijer, K. (1994). Realistic mathematics education.
13. Lijnse, P. L. (Ed.) (1993). European research in science education.
12. Zuidema, J. & Van der Gaag, L. (1993). De volgende opgave van de computer.
11. Gravemeijer, K, Van den Heuvel Panhuizen, M., Van Donselaar, G., Ruesink, N., Streefland,
L., Vermeulen, W., Te Woerd, E., & Van der Ploeg, D. (1993). Methoden in het reken-
wiskundeonderwijs, een rijke context voor vergelijkend onderzoek.
10. Van der Valk, A. E. (1992). Ontwikkeling in Energieonderwijs.
9. Streefland, L. (Ed.) (1991). Realistic mathematics education in primary schools.

205

Thesis Ari_141201_FINAL.pdf 205 1-12-2014 16:26:31

 8. Van Galen, F., Dolk, M., Feijs, E., & Jonker, V. (1991). Interactieve video in de nascholing reken-
wiskunde.
7. Elzenga, H. E. (1991). Kwaliteit van kwantiteit.
6. Lijnse, P. L., Licht, P., De Vos, W. & Waarlo, A. J. (Eds.) (1990). Relating macroscopic phenomena
to microscopic particles: a central problem in secondary science education.
5. Van Driel, J. H. (1990). Betrokken bij evenwicht.
4. Vogelezang, M. J. (1990). Een onverdeelbare eenheid.
3. Wierstra, R. F. A. (1990). Natuurkunde-onderwijs tussen leefwereld en vakstructuur.
2. Eijkelhof, H. M. C. (1990). Radiation and risk in physics education.
1. Lijnse, P. L. & De Vos, W. (Eds.) (1990). Didactiek in perspectief.

206

Thesis Ari_141201_FINAL.pdf 206 1-12-2014 16:26:31

 ICO Dissertation Series

ICO Dissertation Series

In the ICO Dissertation Series dissertations are published of graduate students from
faculties and institutes on educational research within the ICO Partner Universities:
Eindhoven University of Technology, Leiden University, Maastricht University, Open
University of the Netherlands, University of Amsterdam, University of Twente, Utrecht
University, VU University Amsterdam, and Wageningen University, and formerly
University of Groningen (until 2006), Radboud University Nijmegen (until 2004), and
Tilburg University (until 2002). The University of Groningen, University of Antwerp,
University of Ghent, and the Erasmus University Rotterdam have been ‘ICO ‘Network
partner’ in 2010 and 2011. From 2012 onwards, these ICO Network partners are full ICO
partners, and from that period their dissertations will be added to this dissertation series.

(Partial list – updated in January 2014)

281. Cviko, A. (19-12-2013). Teacher Roles and Pupil Outcomes. In technology-rich early literacy
learning. Enschede: University of Twente.
280. Kamp, R.J.A. (28-11-2013). Peer feedback to enhance learning in problem-based tutorial
groups. Maastricht: Maastricht University.
279. Lucero, M.L. (21-11-2013). Considering teacher cognitions in teacher professional
development: Studies involving Ecuadorian primary school teachers. Ghent: Ghent
University.
278. Dolfing, R. (23-10-2013). Teachers' Professional Development in Context-based Chemistry
Education. Strategies to Support Teachers in Developing Domain-specific Expertise. Utrecht:
Utrecht University.
277. Popov, V. (8-10-2013). Scripting Intercultural Computer-Supported Collaborative Learning
in Higher Education. Wageningen: Wageningen University.
276. Bronkhorst, L.H. (4-10-2013). Research-based teacher education: Interactions between
research and teaching. Utrecht: Utrecht University.
275. Bezdan, E. ( 04-10-2013). Graphical Overviews in Hypertext Learning Environments:
When One Size Does Not Fit All. Heerlen: Open University of the Netherlands.
274. Kleijn, R.A.M. de, (27-09-2013). Master’s thesis supervision. Feedback, interpersonal
relationships and adaptivity. Utrecht: Utrecht University.
273. Pillen, M.T. (12-09-2013). Professional identity tensions of beginning teachers. Eindhoven:
Eindhoven University of Technology.
272. Meeuwen, L.W. van (06-09-13). Visual Problem Solving and Self-regulation in Training
Air Traffic Control. Heerlen: Open University of the Netherlands.
271. Keuvelaar-Van den Bergh, L. (26-06-2013). Teacher Feedback during Active Learning:
The Development and Evaluation of a Professional Development Programme. Eindhoven:
Eindhoven University of Technology.
270. Hornstra, T.E. (17-06-2013). Motivational developments in primary school. Group-specific
differences in varying learning contexts. Amsterdam: University of Amsterdam.
269. Vandyck, I.J.J. (17-06-2013). Fostering Community Development in School-University
Partnerships. Amsterdam: VU Universtiy Amsterdam.

207

Thesis Ari_141201_FINAL.pdf 207 1-12-2014 16:26:31

 268. Milliano, I.I.C.M. de (24-05-2013). Literacy development of low-achieving adolescents. The
role of engagement in academic reading and writing. Amsterdam: University of
Amsterdam.
267. Taminiau, E.M.C. (24-05-2013). Advisory Models for On-Demand Learning. Heerlen:
Open University of the Netherlands.
266. Azkiyah, S.N. (23-5-2013). The effects of Two Interventions - on Teaching Quality and
Student Outcome. Groningen: University of Groningen.
265. Diggelen, M.R. van (21-05-2013). Effects of a self-assessment procedure on VET teachers’
competencies in coaching students’ reflection skills. Eindhoven: Eindhoven University of
Technology.
264. M.H. Knol (19-04-2013). Improving university lectures with feedback and consultation.
Amsterdam: University of Amsterdam.
263. Dekker-Groen, A. (19-04-2013). Teacher competences for supporting students’ reflection.
Standards, training, and practice. Utrecht: Utrecht University.
262. Verberg, C.P.M. (18-04-2013). The characteristics of a negotiated assessment procedure to
promote teacher learning. Leiden: Leiden University.
261. Jong, R.J. de (11-04-2013). Student teachers' practical knowledge, discipline strategies, and
the teacher-class relationship. Leiden: Leiden University.
260. Belo, N.A.H. (27-03-2013). Engaging students in the study of physics. Leiden: Leiden
University.
259. Bijker, M.M. (22-03-2013). Understanding the gap between business school and the
workplace: Overconfidence, maximizing benefits, and the curriculum. Heerlen: Open
University of the Netherlands.
258. Noroozi, O. (11-01-2013). Fostering Argumentation-Based Computer-Supported
Collaborative Learning in Higher Education. Wageningen: Wageningen University.
257. Van der Linden, P.W.J. (20-12-2012). A design-based approach to introducing student
teachers in conducting and using research. Eindhoven: Eindhoven University of
Technology.
256. Timmermans, A.C. (27-11-2012). Value added in educational accountability: Possible, fair
and useful? Groningen: University of Groningen.
255. De Feijter, J.M. (09-11-2012). Learning from error to improve patient safety. Maastricht:
Maastricht University.
254. Kenbeek, W.K. (31-10-2012). Back to the drawing board. Creating drawing or text
summaries in support of System Dynamics modeling. Enschede: University of Twente.
253. Doppenberg, J.J. (24-10-2012). Collaborative teacher learning: settings, foci and powerful
moments. Eindhoven: Eindhoven University of Technology.
252. Peltenburg, M.C. (24-10-2012). Mathematical potential of special education students.
Utrecht: Utrecht University.
251. Phielix, C. (28-09-2012). Enhancing Collaboration through Assessment & Reflection
[Samenwerking Verbeteren door middel van Beoordeling en Reflectie]. Utrecht: Utrecht
University
250. Van de Pol, J.E. (28-09-2012). Scaffolding in teacher-student interaction. Exploring,
measuring promoting and evaluating scaffolding. Amsterdam: University of Amsterdam.
249. Vrijnsen-de Corte, M.C.W. Researching the Teacher-Researcher. Practice-based research in
Dutch Professional Development. Schools Eindhoven: Eindhoven University of
Technology.

208

Thesis Ari_141201_FINAL.pdf 208 1-12-2014 16:26:31