Journal of the Korea Society of Mathematical Education Series D: D:

Research in Mathematical Education <>

Vol. 7, No. 3, September 2003, 163189 7 3 2003 9 , 163189

A Development of the Test for Mathematical Creative

Problem Solving Ability 1
Lee, Kang Sup
Department of Mathematics Education, College of Education, Dankook University,
Hannam-dong, Yongsan-ku, Seoul 140-714, Korea; Email: leeks@dankook.ac.kr

Hwang, Dong-jou
Department of Mathematics Education, College of Education, Dankook University,
Hannam-dong, Yongsan-ku, Seoul 140-714, Korea; Email: hdj0719@chollian.net

Seo, Jong Jin

Department of Mathematics Education, College of Education, Dankook University,
Hannam-dong, Yongsan-ku, Seoul 140-714, Korea; Email: sjj8483@hanmail.net

(Received September 15, 2003)

The purpose of this study is to develop a test, which can be used in creative problem
solving ability in mathematics of the mathematically gifted and the regular students.
This test tool is composed of three categories; fluency (number of responses), flexibility
(number of different kinds of responses), and originality (degree of uniqueness of
responses) which are the factors of the creativity. After applying to 462 middle school
students, this test was analyzed into item analysis. As a results of item analysis, it turned
out to be meaningful (reliability: 0.80, validity: item 1(1.05), item 2(1.10), item 3(0.85),
item 4(0.90), item 5(1.08), item difficulty: item 1(0.22), item 2(0.41), item 3(0.23),
item 4(0.40), item 5(0.01), item discriminating power: item 1(0.73), item 2(0.73), item
3(0.67), item 4(0.51), item 5(0.56)), over the level of a standard basis. This means that
the test tool was useful in the test process of creative problem solving ability in
mathematics

Key words: mathematical development, problem solving, creative problems.

ZDM classification: D50
MSC2000 classification: 97D50

This research was supported by Research Fund of Dankook University in 2002.

1
This paper will be presented at the Eighth International Seminar on Education of Talented Children in
Mathematics at the Daegu National University of Education, Daegu, Korea, October 3, 2003.

163
164 Lee, Kang Sup; Hwang, Dong-jou & Seo, Jong Jin

1. INTRODUCTION

School Education has long focused on problem solving (cf. Dillon 1982; Ramirez
2002). Cognitivism and Constructivism, both of which have been providing a basic
framework for school education, stressing the importance of improving students problem
solving ability. Cognitivists insist that school education should concentrate on enhancing
students ability to solve structured problems.
On the other hand, constructivists and problem-based learning theorists advocate the
use of unstructured problems in learning, emphasizing that the process of solving an
unstructured problem is different from that of solving a structured problem. They also
hold that the process of solving an unstructured problem should be preceded by the
process of redefining or restructuring it (cf. Reiter-Palmon, Mumford, Boes & Runco
1997). However, they argue that problem solving is more important than any other part
of learning and the process of structuring a problem is simply a process of solving it.
Although constructivists have been attempting to link the process of structuring an
unstructured problem to creativity, they still fail to broaden their interest and make a
comprehensive study of problem finding ability. Runco & Chand (1995) explain that
finding a problem is the starting point and the key to producing creative products. In
current years, finding a problem has sometimes been considered as a creative process in
itself (cf. Dillon 1988; Voss & Means 1989).
NCTM (2000) Standards suggests that, in order to prepare for the 21st century, todays
students should identify themselves with the ability to use mathematical knowledge for
problem solving, with the ability to communicate mathematically, and with the ability to
reason mathematically and a mathematical propensity. It also states that students need to
be provided with challenging problems that can stimulate students to develop diverse and
sound ways of mathematical thinking and to think creatively. It adds that guiding
students to solve a problem using several methods and strategies help students develop
and extend their mathematical thinking. Creative thinking ability and expressive ability
in the field of mathematics can be measured by open-ended or open-response
problems and questions that require more than one answer.
Based on previous studies as described above, this study aims to examine and analyze
how differently gifted and regular second-grade middle-school students respond to open-
ended problems, which can be used as essential vehicles to measure mathematical
creativity. In order to investigate how creatively they solve problems, they were
presented with problems that have several answers that can be solved with original and
unique ideas.
 A Development of the Test for Mathematical Creative Problem Solving Ability 165

1.1. The concept of open problem

One of the aim of the PME discussion group was to find answers to the question,
What are open-ended problems?. This was becase the group of open-ended problems
does not seem to be well defined. In the course of the discussion, several types of
problems were put forward: Investigations, problem posing, real-life situations, projects,
problem fields (or problem sequences), problems without question, and problem
variations (what-if method). Examples of these groups of problems can be found in the
papers published on this subject (cf. Nohda 1995; Silver 1995; Stacey 1995). A
relationship between ill-structured problems and problem finding was found in Voss
(1990). Indeed, problem finding, that is, how individuals formulate and identify a
problem in itself is an ill-structured problem (cf. Voss 1990, p.12).
The concept open problem can be explained as follows: We will begin by its
opposite, supposing that if its starting situation and goal situation are closed, i.e. exactly
explained, a problem is closed. If the starting situation and / or the goal situation are
open, i.e. they are not closed, we have an open problem (Table 1). Problems dealt with in
school mathematics are usually closed problems (or more generally closed tasks) that
leave not much room for creative thinking (cf. Pehkonen 1995a).

Table 1: The classification of problems according to their starting and goal situations
goal situation CLOSED
OPEN
starting situation (i.e. exactly explained)
open-ended problems, real-life
CLOSED
closed problems situations investigations, problem fields
(i.e. exactly explained)
problem variations

real-life situations real-life situations, problem variations

OPEN
problem variations projects, problem posing

1.2. A prior study and relation to mathematical creativity

Creativity is similar to problem posing in its multiplicity in nature. Psychologists

identified what as a special construct other than intelligence. Problem posing, or problem
finding, has long been viewed as a characteristic of creative activity or exceptional talent
in many fields of human endeavor. For example, Getzels & Csikszentmihalyi (1976)
studied artistic creativity and characterized problem finding as a center of creative artistic
experience. The notions of fluency, flexibility and novelty were adapted and applied in
the domain of mathematics by Balka (1974), who asked subjects to pose mathematical
problems that could be answered on the basis of information provided in a set of stories
about real world situations. Problem posing, along with problem solving, is central to the
166 Lee, Kang Sup; Hwang, Dong-jou & Seo, Jong Jin

discipline of mathematics and the nature of mathematical thinking (cf. Silver 1994).
When mathematicians engage in the intellectual work of the discipline, it can be argued
that the self-directed posing of problems to be solved is an important characteristic (cf.
Plya 1954).
In fact, problem finding has sometimes been considered as a creative process in itself
(cf. Dillon 1988; Voss & Means 1989). Studies in mathematical creativity were reviewed
(cf. Haylock 1987) and one may see problem posing ability as a creative ability. In a
recent comprehensive review paper on problem posing, however, Silver (1994) extended
the discussion and commented that a general relationship between problem posing and
creativity was still unknown. For a more detailed report of the study see Leung and
Silver (1997). From the first result, fluency is general in both verbal creativity and
problem posing but according to the second, flexibility is not. Finally, the third result
suggests that flexibility is specific within arithmetic problem posing. The third result was
also obtained in a replicated study in Taiwan (r = 0.286; P < 0.01; Leung 1995). Studies
on the relationship between general creativity and mathematical creativity (cf. Evans
1964; Haylock 1978; Lee & Hwang 2003) were reviewed and there was a correlation
between general creativity and mathematical creativity. We examined relations between
Mathematical Creative Problem Solving Ability Test (MCPSAT: Kim et al. 1997) and
Torrance Test of Creative Thinking Figural A (TTCT; adapted for Korea by Kim 1998).
The results of the study can be summarized as follows; first, there was a correlation
between the originality of general creativity and the three elements-fluency, flexibility,
and the total-of mathematical creativity (significant at p < 0.01 ). Second, there was also
a correlation between the total of general creativity and the three elements of
mathematical creativity (significant at p < 0.05) (cf. Lee & Hwang 2003). Yoshihiko
(1997) think that openness like open-ended approach and from problem to problem is
one aspect of fostering mathematical creativity. Because, open-ended approach means
end products are open, and from problem to problem means ways to develop a problem
are open.
In the 1980s, the idea of using some form of open-ended problems in classroom
spreaded all over the world, and research on its possibilities was very active in many
countries (cf. Nohda 1988, 1991; Pehkonen 1995a, 1995b; Silver & Mamona 1989;
Williams 1989; Mason 1991; Stacey 1991, 1995; Zimmermann 1991; Clarke & Sullivan
1992; Silver 1993, 1995; Leung 1993; Silver & Cai 1996). Studies are currently under
the way on the relationship between mathematical creativity and open-ended problems (cf.
Kwon et al. 1999; Min 1999; Byun 2001; Moon 2002). This means that the test process
of creative problem solving ability in mathematics should be considered with open-ended
problems.
 A Development of the Test for Mathematical Creative Problem Solving Ability 167

1.3. Creativity in school mathematics

Mathematician considered mathematical creativity a major element of mathematical

bility and have tried to define it. After searching literature and research on mathematical
creativity Aiken (1973) concluded that mathematical creativity is always defined on the
basis of process and various products. Judging from literature and studies on mathe-
matical creativity, the nature of creativity can be classified into two perspectives: Firstly,
mathematical creativity is regarded as cognitive ability that leads to emphasize creative
thinking. Secondly, mathematical creativity is essentially defined with focus on products.

1.4. Creativity as measuring factor

Mathematical creativity and divergent products can be summarized by measuring

factors as follows: first, fluency is used as measuring factor (cf. Foster 1970; Baur 1971;
Maxwell 1974; Dunn 1976). Second, flexibility is used (cf. Krutetskii 1976). Third,
fluency and originality are used (cf. Mainville 1972). Fourth, fluency, flexibility and
originality are used (cf. Evans 1964; Zosa 1978; Balka 1974; Kim et al. 1997; Song 1998).

2. DESIGN OF ANALYSIS

For this research, we examined and analyzed the responses to open-ended problems of
the mathematically gifted and the regular students with three categories; fluency,
flexibility, and originality which are the factors of the creativity. Purposes for this study
included (a) the reliability, validity, difficulty, relevance and discrimination of open-ended
problems; (b) a comparative study on the characteristic of responses to open-ended
problems of the mathematically gifted and the regular students.
Because it is unreasonable to generalize test results from a single open-ended problem,
more items are required for detailed analysis of test results. In this study, internal validity
and difficulty were assessed based on Raschs 1-parameter item-response model.

2.1. Participants

The subjects of this study are 53 volunteers from the Gifted Education Center of
Hanbat National University in Daejeon and 409 students from middle-schools in Daejeon.

2.2. Instrumentation

Five following problems were selected as the open-ended problems for this study.
Problem 1. Sixteen dot problem, a transformed version of the nine dot problem in
168 Lee, Kang Sup; Hwang, Dong-jou & Seo, Jong Jin

Haylock (1984), Kim et al. (1997) and Song(1988).

Problem 2. Regular hexagon problem, a transformed version of the quadrangle problem

in Kim et al. (1997).
Problem 3. Water-flask problem in Becker & Shimada (1997).
Problem 4. Marble problem in Becker & Shimada (1997).
Problem 5. Classifying several solid figure problems in Becker & Shimada (1997).

2.3. Design and Procedure

Data collection. Sampling was done in May 2003. Prior to conducting the test, the
subjects were instructed by the tester for 5 minutes on how to complete their answer
sheets. Subjects were given 50 minutes to present various types of original and unique
answers.
Marking method and standard. The method and standard of marking the responses are
as follows.
1) All types of responses to items are analyzed and recorded.
2) Same types of responses are selected and classified.
3) Scores are given by categorized responses where score of fluency, flexibility, and
originality are analyzed. Each scoring method is suggested below.
(1) Flexibility: how many types of categorized response a student can made.
Students are allowed to write a maximum of 15 answers for one problem thus
maximum score of flexibility is 15. For example, if a student's answers are
classified into 3 categories of responses, then flexibility score is 3.
(2) Fluency: how many correct answers exist with a categorized response type.
When a student made multiple correct answers in a category, the score can be
given to the maximum of 5.
(3) Originality: how original response an answer is which no other students could
think of. That is, originality score reflects the relative rarity of response.
Originality is measured as the following procedure.
The frequency is analyzed in that how many students have given the same type of
response categorized in sub-level.
The percentage of the frequency is calculated that the response type belongs to
The score is given as below according to the percentage of frequency.
* 3% above: 0 * 2% above3% below: 1
* 1% above2% below: 2 * 1% below: 3
 A Development of the Test for Mathematical Creative Problem Solving Ability 169

The originality score has no upper limit.

This test does not suggest total score due to the following reasons:
If regular score for each item is given, then regular score for 3 sub-ability factors is
also fixed that leads to incorrect discrimination of students ability. For instance, 5
flexibility scores are decided and reference marks are given such as frequency
12 score 1, 34 score 2, 56 score 3,
then both students giving 3 responses and 4 responses have the same score. This is true
to other ability factors.
Data analyses. A reference table for scoring the responses was developed by selecting
and classifying all relevant responses to each item according to their types and identifying
the frequency of each type. In order to evaluate item-internal consistence reliability and
discrimination, Cronbach . was calculated using SPSS 10.0K. Internal validity and
difficulty were calculated using BIGSTEPS (cf. Livacre & Wright 1994, 2003) based on
Raschs 1-parameter item-response model.

3. RESULTS AND CONCLUSIONS

3.1. Analysis of quality of test instruments

Item-internal Consistence Reliability. To evaluate the reliability of the test, Cronbach

was calculated, which indicates item-internal consistence reliability. Cronbach
was .80, suggesting that the test is fairly reliable.
Internal Validity by Item Relevance Index. The internal validity of each test item was
calculated using BIGSTEPS, a computer program designed to measure parameter values
and conduct item analysis based on Raschs 1-parameter response model. The analysis
model used in this study was the Partial Credit model. Every item relevance index was
less than 1.2, which implies that all items are relevant for an analysis model.

Table 2: Open-ended Item relevance indexes

Item 1 2 3 4 5 Total
Infit 1.05 1.10 0.85 0.90 1.08 1.00
Outfit 1.01 1.02 0.83 0.90 1.05 0.96

Difficulty. Item difficulty refers to the degree of difficulty of an item. In this study,
item difficulty was calculated based on Raschs 1-parameter item response model. The
item difficulty of 0.0 means average. A higher positive number indicates a higher
difficulty. On a difficulty scale, the differences in difficulty between items are evenly
170 Lee, Kang Sup; Hwang, Dong-jou & Seo, Jong Jin

distributed as far as the logit score does not exceed 0.6. Every item reliability index was
higher than 0.80, which implies that the used items are well separated and highly relevant
for discriminating between students on the basis of creative problem solving ability.

Table 3: Open-ended item difficulties

Item 1 2 3 4 5 Total
Difficulties .22 .41 .23 .40 .0.1 0.00
Discrimination. The discrimination of each open-ended item was analyzed by point-
biserial correlation. Point-biserial correlation represents the correlation between the score
for a single item and the total score for the remaining items. An item with a negative
value is not suitable for discriminating between high and low ability students. Most of
the items with point-biserial correlation of less than 0 are not relevant for discriminating
between students because they allow students to get good marks easily based on their
previous knowledge. However, considering that there was no item calculated as less than
0, all items seem to be more or less relevant for discriminating between students on the
basis of mathematical creativity.

Table 4: Open-ended item discriminations

Item 1 2 3 4 5 Total
Discriminations .73 .73 .67 .51 .56 1.00

3.2. Analysis of answers by type

Considering that there can be multiple numbers and types of answers to each item, all
possible responses were selected and classified by their types and the frequency of each
type was measured. In order to develop a reference table for scoring the responses, a
table of response types was firstly prepared by analyzing the number of responses and the
number of response types, then a criteria for giving marks for originality was established
by identifying the frequency of each type and its mathematical utility. The detailed
analysis of items are showed in appendix 2.
Totally 462 students participated in this test. While numbers with parenthesis show
percentage in proportion to 462. The frequency is calculated under the same standard
between of gifted students and regular students. Fluency, flexibility, and originality
scores are resulted from the same reference table.

3.3. Open-ended test results

The following table shows results of the test by items. Each score for fluency,
flexibility, and originality is resulted from minimal and maximal score of each item.
 A Development of the Test for Mathematical Creative Problem Solving Ability 171

Correlation means Pearsons correlation coefficient of each item score and total score.
Almost all items show the correlation of 0.690.74. Scores represented in coefficient are
marks of fluency, flexibility, originality, and minimum/maximum of total score. For
example, fluency score ranges from minimum score of 0 to maximum score of 40.
As items 1 and 2 have a maximal fluency limit of 15 responses, they are familiar items
to students. However fewer the maximal flexibility score, more difficult items that give
various types of ideas the students were offered. Items 3 to 5 have fewer maximal scores
in both fluency and flexibility, which indicates items 3 and 5 are difficult items to
stimulate various ideas. This is either of the two: the nature of the question itself makes
difficulty in suggesting various responses or various types of responses are possible, but
students have difficulty thinking of it. Some items have the latter case.
The number shown in the note is a ratio of students who got at least one point higher
score in each item. The lower ratio, more difficult response to give, and vice versa.

Table 5: Scores by items

(N=462)
Fluency Flexibility Originality Total M SD Correlation Note
Items
G R T G R T G R T G R T G R T G R T T G R T

1 0-12 0-15 0-15 0-6 0-6 0-6 0-15 0-3 0-15 0-32 0-20 0-32 11.77 4.92 5.71 5.93 4.71 5.33 .69 94.3 76.0 78.1

2 0-15 0-15 0-15 0-6 0-5 0-6 0 0 0 0-20 0-20 0-20 12.72 6.99 7.65 5.28 5.50 5.76 .69 96.2 78.2 80.3

3 0-7 0-4 0-7 0-6 0-4 0-6 0-8 0-8 0-8 0-21 0-14 0-21 5.38 1.57 2.00 4.86 2.65 3.22 .72 73.6 36.4 40.7

4 0-5 0-4 0-5 0-4 0-4 0-4 0-9 0-9 0-9 0-18 0-14 0-18 2.94 1.14 1.34 3.53 2.43 2.64 .75 62.3 24.7 29.0

5 0-6 0-7 0-7 0-4 0-5 0-5 0-6 0-9 0-9 0-12 0-18 0-18 5.02 3.70 3.85 2.97 3.47 3.44 .74 96.2 71.4 74.2

Total 3-40 0-38 0-40 2-19 0-18 0-19 0-16 0-23 0-23 5-74 0-68 0-74 37.83 18.31 20.55 12.71 11.93 13.53 .75 100 97.1 97.4

M 23.00 12.02 12.39 11.28 6.11 6.70 4.26 3.55 1.46 37.83 18.31 20.55

SD 7.57 7.34 8.29 3.31 3.42 3.78 2.38 1.19 2.77 12.71 11.93 13.53

**, p < 0.01

* correlation is a coefficient of correlation with total score.
* note is a ratio of students having over score 1.
* denote by G: gifted students, R: regular students, T: Total
The results of analysis of the differences between students responses are as follows:
1) Item 1. Responses are categorized largely into four types:
(1) Using a single basic figure.
(2) Using more than two figures.
(3) Using the median point.
(4) Using curved lines.
172 Lee, Kang Sup; Hwang, Dong-jou & Seo, Jong Jin

Gifted students gave an even distribution of response types, showing an extraordinary

ability in giving response using curves.
2) Item 2. Responses are categorized largely into five types as showen in Appendix 2.
Responses were so numerous that the score for originality was 0 (cf. every relative
frequency of response type to item 2 exceeds 0.03). Item 2 was the easiest to solve, and
its difficulty was 0.41. Students showed various of point of views, but most of their
expressions were not advanced level. In these problems, the approach were already given
to the students and they did not need to devise their own. Item 2 is narrowed simply to
identifying some figures that have a certain property.
3) Item 3. Responses are categorized lloughly into six types:
(1) Linking each dots.
(2) Using a diagonal line.
(3) Using an inner dot.
(4) Using circle.
(5) Using square.
(6) Others.
As the difficulty of item 3 was 0.23, gifted students gave more active response than
regular students did. Most of the gifted students scored for originality. Regular students
lacked ability to express their originality clearly.
4) Item 4. It was the most difficult one to solve.
Few of regular students could solve item 4, though students with higher scores were
more or less responsive to it. Item 4 was found to be very difficult for the subjects.
However, the discrimination power of item 4 was at the acceptable level. In these
problems, the approach is already given to the students and they did not need to
devise their own.
5) Item 5. It is simply narrowed to identifying some figures that have certain property.
A large number of students ansewered to item 5, which was answered in more various
ways than items 3 and 4 were.

3.4. Analysis of scores by answers

The frequency is calculated under same standard with a difference of gifted students
and regular students. Fluency, flexibility, and originality scores are resulted from the
same reference table (cf. Table 6 and 7).
 A Development of the Test for Mathematical Creative Problem Solving Ability 173

Table 6: Reference table for total and frequency of fluency, flexibility and originality
174 Lee, Kang Sup; Hwang, Dong-jou & Seo, Jong Jin

Table 7: Reference table for standard of frequency

fluency flexibility originality Total
Scores accumulation accumulation accumulation accumulation
frequency percent frequencypercent frequency percent frequency percent
percent percent percent percent

.00 12 2.6 2.6 12 2.6 2.6 306 66.2 66.2 12 2.6 2.6

1.00 11 2.4 5.0 21 4.5 7.1 19 4.1 70.3

2.00 23 5.0 10.0 30 6.5 13.6 12 2.6 72.9 11 2.4 5.0

3.00 17 3.7 13.6 39 8.4 22.1 62 13.4 86.4 8 1.7 6.7

4.00 20 4.3 18.0 40 8.7 30.7 12 2.6 89.0 14 3.0 9.7

5.00 31 6.7 24.7 47 10.2 40.9 6 1.3 90.3 12 2.6 12.3

6.00 21 4.5 29.2 47 10.2 51.1 20 4.3 94.6 8 1.7 14.1

7.00 19 4.1 33.3 52 11.3 62.3 10 2.2 96.8 15 3.2 17.3

8.00 22 4.8 38.1 32 6.9 69.3 3 .6 97.4 16 3.5 20.8

9.00 21 4.5 42.6 32 6.9 76.2 3 .6 98.1 17 3.7 24.5

10.00 23 5.0 47.6 36 7.8 84.0 1 .2 98.3 16 3.5 27.9

11.00 20 4.3 51.9 18 3.9 87.9 2 .4 98.7 5 1.1 29.0

12.00 18 3.9 55.8 25 5.4 93.3 2 .4 99.1 10 2.2 31.2

13.00 26 5.6 61.5 9 1.9 95.2 15 3.2 34.4

14.00 16 3.5 64.9 8 1.7 97.0 16 3.5 37.9

15.00 14 3.0 68.0 5 1.1 98.1 1 .2 99.4 16 3.5 41.3

16.00 13 2.8 70.8 5 1.1 99.1 2 .4 99.8 18 3.9 45.2

17.00 15 3.2 74.0 1 .2 99.4 9 1.9 47.2

18.00 8 1.7 75.8 2 .4 99.8 18 3.9 51.1

19.00 16 3.5 79.2 1 .2 100.0 12 2.6 53.7

20.00 17 3.7 82.9 12 2.6 56.3

21.00 8 1.7 84.6 11 2.4 58.7

22.00 9 1.9 86.6 17 3.7 62.3

23.00 8 1.7 88.3 1 .2 100.0 12 2.6 64.9

24.00 9 1.9 90.3 6 1.3 66.2

25.00 9 1.9 92.2 9 1.9 68.2

26.00 9 1.9 94.2 7 1.5 69.7

27.00 1 .2 94.4 11 2.4 72.1

28.00 5 1.1 95.5 6 1.3 73.4

29.00 5 1.1 96.5 4 .9 74.2

30.00 5 1.1 97.6 9 1.9 76.2

 A Development of the Test for Mathematical Creative Problem Solving Ability 175

Tabel 1 (continued.)
31.00 1 .2 97.8 8 1.7 77.9

32.00 4 .9 98.7 10 2.2 80.1

33.00 10 2.2 82.3

34.00 1 .2 98.9 9 1.9 84.2

35.00 1 .2 99.1 7 1.5 85.7

36.00 1 .2 99.4 2 .4 86.1

37.00 1 .2 99.6 3 .6 86.8

38.00 1 .2 99.8 9 1.9 88.7

39.00 1 .2 100.0 4 .9 89.6

40.00 6 1.3 90.9

41.00 7 1.5 92.4

42.00 4 .9 93.3

43.00 6 1.3 94.6

44.00 4 .9 95.5

45.00 2 .4 95.9

46.00 2 .4 96.3

47.00 2 .4 96.8

48.00 1 .2 97.0

49.00

50.00

51.00 2 .4 97.4

52.00 2 .4 97.8

53.00 1 .2 98.1

54.00 2 .4 98.5

55.00 1 .2 98.7

56.00

57.00 1 .2 98.9

58.00

59.00

60.00 2 .4 99.4

65.00 1 .2 99.6

68.00 1 .2 99.8

74.00 1 .2 100.0
176 Lee, Kang Sup; Hwang, Dong-jou & Seo, Jong Jin

3.5. Conclusions

Five open-ended problems were designed to evaluate students creative problem

solving ability in terms of fluency, flexibility and originality, which are the sub-elements
of creativity. For this study, a total of 462 gifted and average second-grade middle school
students were tested and then each item's reliability, validity, difficulty, discrimination, etc.
were analyzed based on item-response theory. The results are as follows:
First, the coefficient of item-internal consistency reliability (Cronbach ) was 0.80.
Second, every item relevance index based on item-response theory was less than 1.2,
which implies that all items are relevant for an analysis model. The differences in
difficulty among items are evenly distributed as far as the logit score does not exceed 0.6.
Third, every item reliability index was higher than 0.8, which implies that the used
items are well separated and highly relevant for discriminating between students on the
basis of creative problem solving ability.
Fourth, considering that there was no item calculated as less than 0, all items are seem
to be more or less relevant for discriminating between students on the basis of
mathematical creativity.
The major targets of these tests were students whose achievement level in
mathematics is to top 080%. This test was found to be easy for the subjects. However,
the discrimination power of the test was at the acceptable level.
The research on the elaboration of creative problem solving in mathematics should be
performed in other studies in future.
It is highly recommended that the test for mathematical creative problem solving
ability to be introduced in school for mathematics education, since it can stimulate
students' mathematical creativity and divergent thinking as well as enhancing their
interest in mathematics. Also a teaching based on open-ended learning methods can
greatly help students develop their mathematical creativity. This teaching method does
not mean simply applying and practicing algorithms presented by the teacher, but
encouraging students to challenge new problems and develop flexible thinking and
mathematical power.

REFERENCE

Aiken, L. R. (1973): Ability and Creativity in Mathematics. Mathematics Education Reports in

Guilford College. (ERIC Document Reproduction Service No. ED077730).
Balka, D. S. (1974): Creativity ability in mathematics. Arithmetic Teacher 21(7), 633636.
Bauer, G. R. (1971): A study of the effects of a creative classroom, creative problems, and
 A Development of the Test for Mathematical Creative Problem Solving Ability 177

mathematics educators on the creative ability in mathematics of prospective elementary

teachers. Doctorial Dissertation. Bloomington, IN: Indiana University. (Dissertation Abstracts,
No. 71-11, 338).
Becker, J. P. & Shimada, S. (1997): The open-ended approach: a new proposal for teaching
mathematics. Reston, VA: NCTM. MATHDI 1998b.01596
Byun, E. J. (2001): The Effects of Academic Assessment Using Open-Ended Problems on the
Mathematical Creativity (in Korean). Masterial Dissertation, Chongju, Chungbuk: Korea
National University of Education.
Clarke, D. J. & Sullivan, P. A. (1992): Responses to open-ended tasks in mathematics:
characteristics and implications. In: W. Geesline. & K. Graham (Eds.), Processings of the
PME 16. Vol. 2 (pp. 137144). Durham, NH: University of New Hampshire.
Dillon, J. T. (1982): Problem finding and solving. Journal of Creative Behavior 16, 97111.
_____ (1988): Levels of problem finding vs problem solving. Questioning Exchange 2(2), 105
115.
Dunn, J. A. (1976): Discovery, creativity and school mathematics: a review of research.
Educational Review 28(2), 102117.
Evans, E. W. (1964): Measuring the Ability of Students to Respond to Creative Mathematical
Situations at the Late Elementary and Early Junior High School Level. Doctorial Dissertation,
Ann Arhor, MI: University of Michigan. (Dissertation Abstracts, 1965, 25, 7108-7109).
Foster, J. (1970): An Exploratory attempt to assess creative ability in mathematics. Primary
Mathematics 8, 27.
Getzels, J. W. & Csikszentmihalyi, M. (1976): The creative vision. New York: Wiley.
Haylock, D. W. (1978): An investigation into the relationship between divergent thinking in non-
mathematical and mathematical situations. Math. School 7(2) p. 25. MATHDI 1978d.01772
Haylock, D. W. (1987): A framework for assessing mathematical creativity in schoolchildren.
Educational Studies in Mathematics 18, 5974. MATHDI 1989c.00290
Kim, H. W.; Kim, M. S.; Bang, S. J. & Hwang, D. J. (1997): Development of the Test for
Mathematical Creative Problem Solving Ability (II) (in Korean). Report CR97-50, Seoul:
Korea Education Development Institute (KEDI).
Kim, Y. C. (1998): The Torrance tests of creative thinking: Norms-technical Manual of Korean
version (in Korean). ChungAng Aptitude Press.
Krutetskii, V. A. (1976): The psychology of mathematical abilities in school children. Chicago,
IL: Univ. of Chicago Press.
Kwon, O. N.; Bang, S. J. & Song, S. H. (1999): A Study on Characteristic of Responses to
Multiple Soultion Problems of Middle School Students Gifted in Mathematics (in Korean). J.
Korea Soc. Math. Educ. Ser. A Mathematics Education 38(1), 3748.
Lee, K. S. & Hwang, D. J. (2003): A Study on the Relationship between General Creativity and
Mathematical Creativity-Based on the TTCT; Figural A and the MCPSAT; A. J. Korea Soc.
178 Lee, Kang Sup; Hwang, Dong-jou & Seo, Jong Jin

Math. Educ. Ser. A Mathematics Education 42(1), 19. (in Korean)

Leung, S. S. (1993): Mathematical problem posing: the influence of task formats, mathematics
knowledge, and creative thinking. In: Proceedings of the 17th PME Conference. Vol. III (pp.
3340). Tsukuba: University of Tsukuba
_____ (1995): On analyzing problem posing brhaviors of elementary school teachers in Taiwan
(in Chinese). Technical Report of research project funded by National Science Council of
Taiwan under contract number NSC 84-2511-23-001.
Leung, S. S. & Silver, E. A. (1997): The role of task format, mathematics knowledge and creative
thinking on the arithmetic problem posing of prospective elementary school teacher.
Mathematics Education Research Journal 9(1), 520. MATHDI 1997f.04172
Livacre, J. M. & Wright, B. D. (1994, 2003): A Users Guide to BIGSTEPS Rasch-Model
Computer Programs. Winsteps.com.
Mainville, W. E. Jr. (1972): A study of the effects of mathematics activity materials upon certain
aspects of creative thinking ability of prospective elementary school teacher. Unpublished
Doctorial Dissertation. Farmington, ME: University of Maine, (Dissertation Abstracts, No. 72-
29, 994).
Mason, J. (1991): Mathematical problem solving: open, closed and exploratory in UK. ZDM,
Zentralbl. Didakt. Math.( International Reviews on Mathematical Education) 23(1), 1419.
Maxwell, A. A. (1974): An exploratory study of secondary school geometry students: problem
solving related to convergent-divergent productivity. Doctoral dissertation, Knoxville, TN:
University of Tennessee. (ERIC Document Reproduction Service No. ED110328).
Min, D. J. (1999): A Study on a Method of Teaching Mathematics by Open-ended Problems-
Focusing on Sixth Grades in the Primary Schoo. Masterial Dissertation, Chongju, Chungbuk:
Chinju National University of Education. (in Korean)
Mun, J. Y. (2002): The Effects of Open-Ended Approach on Mathematical Creativity in
Elementary Mathematics Education. Masterial Dissertation, Korea National University of
Education. (in Korean)
National Council of Teachers of Mathematics (NCTM): Principles and Standard for School
Mathematics. Reston, VA: NCTM. MATHDI 1999f.04754
Nohda, N. (1988): Problem solving using open-ended problems in mathematics teaching. In:
Problem Solving; A World View. (pp. 225234). Nottingham, VA: Shell Centre.
_____ (1991): Paradigm of the open-approach method in mathematics teaching: Focus on
mathematical problem solving. ZDM, Zentralbl. Didakt. Math. (International Reviews on
Mathematical Education) 23(2), 3237. MATHDI 1991c.00860
_____ (1995): Teaching and Evaluating Using Open-Ended Problem in Class. ZDM, Zentralbl.
Didakt. Math. (International Reviews on Mathematical Education) 27(2), 5761. MATHDI
1995c.01902
Pehkonen, E. (1995a): Use of open-ended problems in mathematics classroom. Research Report
 A Development of the Test for Mathematical Creative Problem Solving Ability 179

176. Helsinki University, Finland. Dept. of Teacher Education. (ERIC Document Reproduction
Service No. ED419 714). MATHDI 1998b.01386
_____ (1995b): On pupils reactions to the use of open-ended problems in mathematics. Nordisk
Matematikkdidaktikk. Nomad. (Nordic Studies in Mathematics Education) 3(40), 4357.
MATHDI 1996f.04350
Plya, G. (1954): Mathematics and plausible reasoning. Princeton, NJ: Princeton University
Press. MATHDI 1988f.00369
Ramirez, V. E. (2002): Finding the right problem. Asia Pacific Education Review 3, 1823.
Reiter-Palmon, R.; Mumford, M. D.; Boes, J. O. & Runco, M. A. (1997): Problem construction
and creativity: The role of ability, cue consistency, and active processing. Creativity Research
Journal 10, 923.
Runco, M. A. & Chand, I. (1995): Cognition and creativity. Educational Psychology Review 7,
243267.
Silver, E. A. & Cai, J. (1996): An analysis of arithmetic problem posing by middle school
students. Journal for Research in Mathematics Education 27(5), 521539. MATHDI
1997f.04550
Silver, E. A. & Mamona, J. (1989): Problem posing by middle school mathematics teachers. In:
Proceedings of PME-NA 11(1) (pp. 263269). New Brunswick, NJ: Rutgers University.
Silver, E. A. (1993): On mathematical problem posing. In: Proceedings of the 17th PME
Conference. Vol. I (pp. 6685). Tsukuba: University of Tsukuba. MATHDI 1998b.01720
_____ (1994): On mathematical problem posing. For the learning of mathematic 14(1), 1928.
MATHDI 1994f.02882
_____ (1995): The Nature and Use of Open Problems in Mathematics Education: Mathematical
and Pedagogical Perspectives. ZDM. Zentralblatt fuer Didaktik der Mathematik (International
Reviews on Mathematical Education) 27(2), 6772. MATHDI 1995c.01900
Song, S. H. (1998): A study on the Measurement and Discrimination of the Mathematical
Giftedness (in Korean). Doctorial Dissertation, Seoul: Seoul National University.
Stacey, K. (1991): Linking application and acquisition of mathematical ideas through problem
solving. ZDM, Zentralbl. Didakt. Math. (International Reviews on Mathematical Education)
23(1), 814. MATHDI 1991b.01376
Stacey, K. (1995): The Challenges of Keeping Open Problem Solving Open in School Mathe-
matics. ZDM, Zentralbl. Didakt. Math. 27(2), 6267. MATHDI 1995c.01901
Voss, J. F. (1990): Das Losen schlecht struckturierter Problem-cin uberbhck. (On the solving of
ill-structured problems: A review). Unterrichts Wissenschaft 18, 313337.
Voss, J. F. & Means, M. L. (1989): Towards a model of creativity based upon problem solving in
social sciences. In: Glover, J. A., Ronning R. R. & Reynolds, C. R. (Eds.), Handbook of
creativity: New York, NY: Plenum Press, 399410.
Williams, D. (1989): Assessing authentic tasks: alternatives to mark-schemes. Nordic Studies in
180 Lee, Kang Sup; Hwang, Dong-jou & Seo, Jong Jin

Mathematics Education 2(1), 4868. MATHDI 1989j.01925

Yoshihiko, H. (1997): The methods of fostering creativity through mathematical problem solving.
ZDM, Zentralbl. Didakt. Math. 29(3), 8687. MATHDI 1997e.03339
Zimmermann, B. (1991): Offene Probleme fuer den Mathematikunterricht und ein Ausblick auf
Forschungsfragen. ZDM, Zentralbl. Didakt. Math. 23(2), 3846. MATHDI 1991c.00861
Zosa, E. D. (1978): The construction of a test to measure creative ability in mathematics.
Doctoral dissertation, New York, NY: Columbia University. (Dissertation Abstracts
International, 1979, 39A, 6009).
 A Development of the Test for Mathematical Creative Problem Solving Ability 181

Appendix 1
Open-ended Test

Name: _____________, Date of Birth: _______________, Sex: Male__ Female__

School: ______________________________________________________

Attention
Please read the following explanation before getting to the questions below.
Every question allows multiple answers. The time given is 50 minutes.
(1) Write a maximum of 15 answers that you think are pertinent to the question.
(2) Give answers that are different from but not similar to one another.
(3) Give answers that can not be easily found.
(4) Present answers in as exact and detailed way as possible.
(5) If you need more space to write, get another answer sheet from the teacher.
Do not turn to the next page until instructed by the teacher.

[1] As shown below, there are 16 dots which are arranged with 1cm spacings between
them.

Draw lines between the dots to make as many figures as possible with the area of 2
2
cm . (If two or more figures are overlapped when turned around or over, they are
considered as identical. No figure should be split in two or have one point in common
with another.)
[2] As shown in the Example below, 3 sheets of paper in the shape of a regular hexagon
can be joined together along the sides in 3 ways.
182 Lee, Kang Sup; Hwang, Dong-jou & Seo, Jong Jin

and are considered as identical.

Then, make all drawings of how to join together 6 sheets of paper in the shape of a
regular hexagon along the sides, as in the Example below. (If two or more figures are
overlapped when turned around or over, they are considered as identical.)
[3] Three students, A, B, C, each threw five marbles, Which came to rest as shown. In
this game, the winner is the student with the smallest scattering of marbles. The degree
of scattering seems to decrease in the order A, B, C. Devise as many ways as you can to
express numerically the degree of scattering.

[4] A transparent flask in the shape of a right rectangular prism is partially filled with
water. When the flask is placed on a table and titled, with one edge of its base being
fixed, several geometric shapes of various sizes are formed by the cuboids faces and the
surface of the water. The shapes and sizes may vary according to the degree of tilt or
inclination. Try to discover as many invariant relations (rules) concerning these shapes
and sizes as possible. Write down all your findings.

Figure 2.1 Figure 2.2

 A Development of the Test for Mathematical Creative Problem Solving Ability 183

[5] Consider the solid figures as shown. Choose one or more figures that share the same
characteristics with figure B and write down those characteristics.

[Solutions]

Characteristics A B C D E F G H
184 Lee, Kang Sup; Hwang, Dong-jou & Seo, Jong Jin

Appendix 2
Characteristic of Responses to Open-Ended Problems

Table 8: Reference table for scoring Item 1 and the frequency of response type
Number of Responses
Students Observations Gifted Regular
Classification Originality Total
(Rules) students Students
(N=462)
(N=53) (N=409)
line 879
0 151 728
segment (190.3)
Using a
single point 318
0 86 232
basic symmetry (68.8)
figure
10
asymmetry 1 6 4
(2.2)

line 81
0 30 51
segment (17.5)
Using
more than point 18
0 8 10
two symmetry (3.9)
figures
401
asymmetry 0 143 258
(87)

Using the
line 4
middle 3 1 3
segment (0.9)
point

line 3
3 3 0
Using segment (0.7)
Curved
lines 2
asymmetry 3 2 0
(0.4)
 A Development of the Test for Mathematical Creative Problem Solving Ability 185

Table 9: Reference table for scoring Item 2 and the frequency of response type

Number of Responses
Students
Classification Observations Originality Gifted Regular
Total
(Rules) students Students
(N=462)
(N=53) (N=409)
350
line segment 0 80 270
(76)
146
asymmetry 0 37 109
(32)
260
line segment 0 64 196
(56)
point 126
0 26 100
symmetry (27)
318
asymmetry 0 74 244
(69)
504
line segment 0 86 418
(109)
point 218
0 45 173
symmetry (47)
427
asymmetry 0 63 364
(92)
67
line segment 0 16 51
(14.5)
point 63
0 11 52
symmetry (14)
16
asymmetry 0 4 12
(3.5)
78
0 0 15 63
(17)
186 Lee, Kang Sup; Hwang, Dong-jou & Seo, Jong Jin

Table 10: Reference table for scoring Item 3 and the frequency of response type
Number of Responses
Students Observations Gifted Regular
Classification Originality Total
(Rules) students Students
(N=462)
(N=53) (N=409)
measure the area of pentagon made 74
0 34 40
from linking each dots. (16.0)
measure the circumference of pentagon 61
0 23 38
made from linking each dots. (13.2)
Linking each
dots measure the circumference of star made
0 1 6 5(1.1)
from linking every dots.
measure and add the distance between
dots after linking each dots, and then 2 0 5 5(1.1)
divide it by the number of lines
measure all lengths of diagonal lines
0 4 10 14(3.0)
and sort out the longest value.
add all lengths of diagonal lines and
2 3 2 5(1.1)
measure mean value of them.
sort out the longest one of 4 lines
Diagonal
linking one of 5 dots and the other 4 1 9 3 12(2.6)
Line
dots.
add the lengths of 4 lines linking one of
0 9 6 15(3.3)
5 dots and the other 4 dots.
measure it by the total lengths of
3 0 1 1(0.2)
diagonal lines.
add the lengths of lines connecting one
dot inside pentagon and the other 5 2 5 3 8(1.7)
dots.
Inner dot
measure mean length of lines
connecting one dot inside pentagon and 3 3 0 3(0.7)
the other 5 dots.
measure the radius(circumference) of
3 0 1 1(0.2)
the smallest circle including all dots.
Using circle count the number of marbles outside a
circle when the same size of circle is 3 0 1 1(0.2)
projected on it.
measure the area of big square
2 0 5 5(1.1)
excluding pentagon in it.
Using square draw rectangles of same proportion and
give point 10, 9, 8 for each rectangle
2 3 2 5(1.1)
and then add all the points.
e. g. 10*1+9*2+8*2 = 44(points)
 A Development of the Test for Mathematical Creative Problem Solving Ability 187

Table 10 (continued.)
draw triangles and calculate the largest
3 0 2 2(0.4)
area of them.
measure the deviation and the standard
3 2 1 3(0.7)
deviation using the coordinates system.
draw figures linking dots, then put the
same size of figures on A, B, C, and 3 0 1 1(0.2)
Others
compare the rest they make.
put some kinds of objects in figure
producing marbles one after another, 2 5 2 7(1.5)
and find out the number of the objects.
measure it by the number of objects
3 0 1 1(0.2)
between marble and marble.

Table 11: Reference table for scoring Item 4 and the frequency of response type
Number of Responses
Students Observations Gifted Regular
Classification Originality Total
(Rules) students Students
(N=462)
(N=53) (N=409)
a+b is constant 1 8 2 10(2.2)
Constant
The mean value of a and b is constant 3 2 0 2(0.4)
Sum b+c is constant 2 2 3 5(1.1)
The sum of the lengths of the edges
above the water surface is constant
3 1 1 2(0.4)
One edge decreases by the amount
3 0 1 1(0.2)
the other increases
When one edge increases, the
1 0 11 11(2.4)
other decreases
The lengths of the edges vary 3 0 1 1(0.2)
Variation
The length of the edge of the water
3 0 2 2(0.4)
surface becomes greater
When one edge becomes 0, the
other edge becomes twice its 3 1 0 1(0.2)
original length
when the slope smeller, the area of
3 0 1 1(0.2)
the water surface becomes smeller
BDE=CAF 3 0 1 1(0.2)
Slope A transparent flask in the shape of
a right rectangular prism is not 3 3 0 3(0.7)
circle
The side plane is a rectangular,
3 0 1 1(0.2)
when the slope is right angle
The limit of the length of an edge
Range 3 0 1 1(0.2)
is 15cm
188 Lee, Kang Sup; Hwang, Dong-jou & Seo, Jong Jin

Table 11 (continued.)

The water surface (upper) and the

3 2 2 4(0.9)
base are rectangles
The water surface is a rectangle or
1 1 12 13(2.8)
a quadrangle
Shape of
The shape of the side plane
water surface 0 8 20 28(6.1)
changes from trapezoid to triangle
The side view is a trapezoid 1 2 5 7(1.5)
The shape of the water surface
3 0 2 2(0.4)
changes
The total area of the side faces
3 2 0 2(0.4)
does not change
The area of the water surface
3 2 0 2(0.4)
Area changes
The area of the water surface
2 0 5 5(1.1)
becomes larger
The total surface area changes 3 0 1 1(0.2)
Volume The volume does not changes 0 17 27 44(9.5)
The weight of the water does not
0 8 22 30(6.5)
change
The is a fixed point, when viewed
1 0 8 8(1.7)
horizontally
The form of the water is a
3 1 1 2(0.4)
quadrangular prism
Others
The form of the water changes
from a cuboid to a triangular 3 2 0 2(0.4)
prism
The form of the water changes 1 0 7 7(1.5)
The segment MN is a fixed
3 0 2 2(0.4)
segment

Table 12: Reference table for scoring Item 5 and the frequency of response type
Number of Responses
Classification Characteristics / Solids A B C D E F G H Originality Gifted Regular
Total
students Students
(N=462)
(N=53) (N=409)
Having only one base / / 0 23 47 70(15.1)
Side being a triangle / / 0 36 148 184(39.8)
Surface being flat / / / / 1 3 10 13(2.8)
Shape of Having four faces / / 0 11 28 39(8.4)
faces(side Viewed shape from the top
and bases) / / / / 0 8 8 16(3.5)
being a polygon
Base is not circle / / / / / 3 1 0 1(0.2)
Base being same to side
/ / 1 6 0 6(1.3)
face
 A Development of the Test for Mathematical Creative Problem Solving Ability 189

Table 12 (continued.)
Number of edges=(Number
/ / 3 0 4 4(0.9)
of edges of the base) 2
Edges having straight lines
/ / / 1 2 10 12(2.6)
only
Having Vertices / / / / 0 4 49 53(11.5)
Having Edges / / / / 3 1 0 1(0.2)
The length of side edges
Number of / / / / / / / 3 1 0 1(0.2)
being the same
edges,
vertices, Number of Vertices =
faces, angeles (Number of edges of the / / 3 0 2 2(0.4)
and relations base) + 1
among them Number of face = (Number / / 3 0 2 2(0.4)
of edges of the base) + 1
Number of edge of the base
/ / 3 1 2 3(0.7)
= Number of side faces
Number of vertices =
/ / 3 1 0 1(0.2)
(Number of edges) 2/3
Number of vertex of the
/ / 3 1 0 1(0.2)
base is odd
Shape of shadow being a
/ / 0 2 22 24(5.2)
Shape of a triangle
projection Viewed shape from the top
/ / / / 3 0 4 4(0.9)
being a polygon
Cross section parallel to
/ / / 3 0 2 2(0.4)
the base being similar
Cross section
perpendicular to the base / / / 3 3 0 3(0.7)
being a rectangle
Shape of a
cross section Cross section not parallel / / 3 0 2 2(0.4)
to the base being an ellipse
Cross section
perpendicular to the base
/ / 3 0 4 4(0.9)
through the vertex being a
triangle
Pyramid Pyramid / / 0 22 173 195(42.2)
Shape of a
development Not a solid of revolution / / / / / 1 7 5 12(2.6)
of the solid
Volume Having volumed / / / / / / / / 1 8 4 12(2.6)
Having development figure / / / / / / / 3 1 0 1(0.2)
Viewed shape from
Others development figure being a
/ / 3 0 3 3(0.7)
triangle, the number is the
same