JPII 12 (4) (2023) 564-574

Jurnal Pendidikan IPA Indonesia

http://journal.unnes.ac.id/index.php/jpii

GAMIFICATION IN THE STEM DOMAIN SUBJECT:

THE PROSPECTIVE METHOD TO STRENGTHEN
TEACHING AND LEARNING

J. Arlinwibowo1, N. Ishartono2,3*, Y. Linguistika4, D. Purwoko5, Suprapto1

1
Research Center for Education, National Research and Innovation Agency, Indonesia
Department of Mathematics Education, Universitas Muhammadiyah Surakarta, Indonesia
2

3
Department of Curriculum and Instructional Technology,
University of Malaya, Kuala Lumpur, Malaysia
4
School of Education, The University of Adelaide, Australia
5
Research Center for Society and Culture,
National Research and Innovation Agency, Indonesia

DOI: 10.15294/jpii.v12i4.48388

Accepted: October 26th, 2023. Approved: December 29th, 2023. Published: December 31st 2023

ABSTRACT

Several meta-analysis studies related to game-based learning or Gamification have been carried out by various
researchers. However, there is still a scarcity of studies specifically elucidating the extent of Gamification’s impact
on STEM domain subjects. Therefore, this research aims to summarize various research results related to the in-
fluence of game-based learning in improving student learning outcomes in STEM domain subjects. This research
was a meta-analysis using the standardized group contrast design with a random model. Research data were from
articles published in Scopus-indexed journals or proceedings. The inclusion criteria in this research were articles
published between 2014 – 2023 in English, quantitative type data with a contrast group design containing control
and experimental groups, complete data (n, mean, SD), and research focused on STEM domain subjects. The
data collection process used the PRISMA method. To ensure data quality, researchers conducted publication bias
analysis using the fail-safe N method. The moderator variables were continent, stem domain, era, developing
competency, game type, and education level. The results find that Gamification has a positive impact on student
learning outcomes, as indicated by an effect size value of 0.5492 [0.3943; 0.7041] with a confidence interval of
95%. Gamification does not provide a significant positive difference in the moderator variables continent, era,
developing competence, and game type. For STEM domain subjects, Gamification is highly recommended to be
applied. For the educational level, Gamification is most recommended to be implemented in elementary school.

© 2023 Science Education Study Program FMIPA UNNES Semarang

Keywords: gamification; meta-analysis; subject of learning; teaching and learning; STEM domain

INTRODUCTION sized in its education system (Chesky & Wolf-

meyer, 2015; Bicer et al., 2017). Then, various
STEM, which stands for Science, Techno- countries also emphasize strengthening mastery
logy, Engineering, and Mathematics, is a learning of STEM scientific domains through educatio-
approach widely used in the world (Arlinwibowo nal fields (Khaeroningtyas et al., 2016; Ong et
et al., 2020c; Arlinwibowo et al., 2021b; Arlinwi- al., 2016; Wan Husin et al., 2016; Çevik, 2018;
bowo et al., 2023). It was begun by America to Wisudawati, 2018). In light of the technological
increase the global competitiveness of its society, advancements in the 21st century, it is not an
and STEM scientific domains must be empha- overstatement to regard STEM as a profoundly
*Correspondence Address important discipline that significantly influences
E-mail: naufal.ishartono@ums.ac.id competitiveness.
 J. Arlinwibowo, N. Ishartono, Y. Linguistika,
565
D. Purwoko, S. Suprapto/ JPII 12 (4) (2023) 564-574
However, students face various challenges The synthesis of diverse study findings will offer
in mastering the STEM (Science, Technology, readers a general overview to use as a foundation
Engineering, and Mathematics) domain (Ret- when considering game-based learning imple-
nawati et al., 2017). Mathematics is considered mentation. Meta-analysis is one of the quantitati-
the most challenging domain for students due ve tools that can help researchers summarize data
to its abstract nature, where only some students from prior studies (Utami et al., 2022; Hernan-
possess strong abstraction skills (Arlinwibowo & da et al., 2023). As a result, the purpose of this
Rernawati, 2015; Marsigit et al., 2020). Additio- study is to summarize diverse research findings
nally, the science domain presents challenges, as concerning the impact of game-based learning on
some students perceive it as overly theoretical and improving student learning outcomes in STEM
difficult to comprehend (Retnawati et al., 2017). topics.
Lastly, engineering and technology domains are Until now, several meta-analysis studies
based on complex concepts, requiring students related to game-based learning or Gamification
to master high-level thinking skills for effective have been conducted by various researchers. The
learning (Arlinwibowo et al., 2020b; Arlinwi- common findings suggest that gamification or
bowo et al., 2021a). Despite these challenges, game-based learning improves learning outcomes
many students are capable of mastering STEM (Kim & Castelli, 2021; Mazeas et al., 2022). Re-
subjects. One of the key determinants is the high searchers argue that game-based learning makes
motivation of these students to learn the domain. students more enthusiastic and happier (Mazeas
Empirically, games have been proven to enhance et al., 2022) and helps them understand contexts
students’ learning motivation in STEM subjects (Kim & Castelli, 2021), thereby enhancing lear-
(Ibrahim et al., 2020; Janković & Lambić, 2022). ning motivation (Mamekova et al., 2021) and imp-
Game is considered one of the platforms roving learning outcomes (Fadhli et al., 2020). A
that can enhance an individual’s cognition (Pi- meta-analysis study specifically focuses on the re-
aget, 2013), increase learning appeal (Öztürk & lationship between gamification and STEM lear-
Korkmaz, 2020), make students enjoy learning ning but only concentrates on virtual games with
(Papastergiou, 2009), and provide opportunities data from 2010 to 2020 (Wang et al., 2022). This
to incorporate various real-world contexts (Chu recent meta-analysis study aims to analyze the
& Chang, 2014). However, the implementation impact of games (both virtual and non-virtual)
of game-based learning still faces various challen- on learning outcomes (attitudes, knowledge, and
ges. The most common constraints include infra- skills) with the most recent data from the last ten
structure issues, funding, school policies (Kaima- years to provide a more realistic picture of the si-
ra et al., 2021), the complexity of instructional tuation before and after COVID-19. The research
design, teachers’ ability to organize learning (Ma- is also enriched with several moderator variables
nesis, 2020), and students’ gaming skills (Stanko- that can describe the dataset in more detail, allo-
va et al., 2021). Therefore, it becomes interesting wing readers to understand the impact of game-
to question whether game-based learning can based learning.
improve students’ learning outcomes. In theory, METHODS
games can enhance the appeal and motivation
of STEM subjects (Papastergiou, 2009; Piaget, This was a meta-analysis study with the
2013; Chu & Chang, 2014). However, many chal- goal of determining the possible enhancement
lenges must be faced (Manesis, 2020; Kaimara et of learning quality by Gamification in STEM
al., 2021; Stankova et al., 2021). The negative ef- (Science, Technology, Engineering, and Mathe-
fects of Gamification found in previous research matics) subjects. Meta-analysis is a systematic
are disparity in facilities, too much time allocati- study of quantitative data that allows researchers
on, too many students in class, and the difficulty to summarize the findings of prior studies (Ret-
of integrating games into learning according to nawati et al., 2018). It will produce a numerical
the curriculum (Lee et al., 2022), such as the tight conclusion in the form of an effect index that rep-
ranking system (win/lose) in games (Toda et al., resents findings from a collection of studies with
2018). similar themes.
Some studies demonstrate that game-based This research was carried out in several
learning has a beneficial influence (Janković steps, namely: 1) Determine the research theme;
& Lambić, 2022; Legaki et al., 2020), while ot- 2) Determine the data population and research
hers show that it has a detrimental impact when variables; 3) Determine a complete data search
compared to traditional techniques (Deng et al., mechanism with inclusion and exclusion criteria;
2020; Fitriyana et al., 2021; Legaki et al., 2021). 4) Determine data networking procedures based
These opposing findings are worth summarizing. on the PRISMA model; 5) Determine the mo-
566 J. Arlinwibowo, N. Ishartono, Y. Linguistika,
D. Purwoko, S. Suprapto/ JPII 12 (4) (2023) 564-574
derator variable to be analyzed; 6) Carry out the words representing the research theme, including
analysis process; 7) Report findings. “gamification,” “STEM Education,” “game,”
This study used research data published “achievement,” and “learning outcome.” To gat-
online. The data search strategy involved using her more comprehensive articles, Boolean opera-
the sciencedirect.com website and the Publish or tors (AND, OR) were used.
Perish application on Scopus-indexed data. This The researchers followed inclusion and
approach aimed to include high-quality articles exclusion criteria during the data screening pha-
that meet the standards of Scopus-indexed jour- se. Articles that met all of the inclusion criteria
nals or proceedings. The strategy was based on were chosen. There were also exclusion criteria
the assumption that the review processes in the- that were the inverse of the inclusion criteria. If
se journals adhere to strict protocols, reducing an article fails to match any of the inclusion cri-
the potential for obtaining articles with research teria, it is removed from the dataset that can be
quality below standard. In the search process, analyzed. Table 1 displays the study’s inclusion
the researcher employed a combination of key- and exclusion criteria.
Table 1. Inclusion and Exclusion Criteria
Aspect Inclusion Criteria Exclusion Criteria
Time 2014 - 2023 Else
Language English Else
Data Type Quantitative Qualitative
Research Design Group Contrast Else
Data Set Control and experiment Does not load either or both
Data Component Sample Size, Mean, and Stan- Does not load one, two, or all
dard Deviation three
Analysis Data Quantitative Qualitative
Theme Gamification to improve the Else
quality of learning
Focus STEM Domain Subjects Else
Journal or Proceeding Index Scopus Else

For greater clarity, the data collection data according to analysis needs. The initial
process is shown in Figure 1. 20 articles met search found 234 articles. Then, through a
the inclusion requirements, making them strict screening process based on the speci-
eligible to enter the analysis stage. Figure 1 fied inclusion criteria, 20 articles that con-
explains the flow of the literature filtering tained complete data and appropriate themes
process to produce literature that contains were found.

Figure 1. PRISMA-based Database Selection Process

 J. Arlinwibowo, N. Ishartono, Y. Linguistika,
567
D. Purwoko, S. Suprapto/ JPII 12 (4) (2023) 564-574
From the 20 articles obtained in the PRIS- the meta-analysis findings. The total effect size
MA selection process, some articles provided of the meta-analysis indicated the general po-
diverse information, allowing for the extraction tential of Gamification to improve learning per-
of more than one set of data for analysis. For formance in STEM subjects, and the analysis of
example, the article by Chen et al. (2020) con- moderator variables produced more specific fin-
tained varied data, all relevant to the research the- dings to highlight other influencing factors. The
me; thus, one article yielded information on emo- moderator variables in this study included conti-
tions and achievement in game-based learning. nent, STEM domain, era, evolving competencies,
Additionally, the samples were divided into three game type, and educational level. The analysis of
categories: high achievers, middle achievers, and moderator variables utilized ANOVA-like models
low achievers. Consequently, six studies were de- to indicate the position of each moderator variab-
rived from this single article. Mapping the results le concerning the effect size. The technique for
of the 20 articles yielded 60 data available for inferring interactions in the moderating variable
meta-analysis to conclude the potential of Gami- was to look at the p-value. If the p-value <0.05,
fication in enhancing the quality of the learning then the interaction in the moderating variable
process in STEM subjects. Moderator variables has a significant difference.
were selected to provide a deeper description of

Table 2. Moderator Variables

Moderator Variable Identified Categories freq %
Continent Euro 18 30.0%
Asia 37 61.7%
America 5 8.3%
STEM domain Math 27 45.0%
Science 18 30.0%
Engineering 15 25.0%
Era Pre-Covid (published before 2020) 38 63.3%
Post-Covid (2020 publications and after) 22 36.7%
Developing Compe- Affective
tence 13 21.7%
Knowledge 20 33.3%
Skill 23 38.3%
Type game Virtual 34 56.7%
Non-Virtual 26 43.3%
Education level University 21 35.0%
Senior High School 7 11.7%
Junior High School 20 33.3%
Elementary 8 13.3%
Preschool 4 6.7%

This study utilized highly varied data sour- assumption of population variance was then
ces. This was demonstrated by the researcher col- clarified with a test of data heterogeneity (Bo-
lecting research results from all continents (Asia, renstein, 2009). The heterogeneity test used the
Europe, and America), educational levels (from Q, Ʈ2, and I2 methods. An effect size between
preschool to high school and beyond), game ty- studies was considered heterogeneous when the
pes (virtual and non-virtual), and evolving com- Q p-value was below 0.05 (for a 95% confidence
petencies (affective, cognitive, and skills). Thus, interval), Ʈ2 > 0 (Retnawati et al., 2018), and I2
this study used a random-effects model, assuming values indicating low (0-25%), moderate (25%-
that population variance contributes to variance 75%), and substantial heterogeneity (75%-100%)
in the effect size due to the different populations (Higgins, 2003).
(Borenstein, 2009; Retnawati et al., 2018). The
568 J. Arlinwibowo, N. Ishartono, Y. Linguistika,
D. Purwoko, S. Suprapto/ JPII 12 (4) (2023) 564-574
The meta-analysis in this study used a stan- sidered free from publication bias when the fail-
dardized group contrast design. Standardization safe N value > 5K + 10, where K is the number
was performed because the scale of values among of studies (Mullen et al., 2001)
data was highly diverse, requiring standardizati-
on to produce proportional aggregates among RESULTS AND DISCUSSION
studies (Arlinwibowo et al., 2022). To determine
the total effect size (d), the first step was to calcu- This research analyzed 60 studies taken
late it using the formula as follows (Retnawati et from 20 research results. This research was con-
al., 2018). ducted in various countries and continents, in-
volving preschool, elementary and high school,
and university students. Based on those data, this
research analyzed the data that was assumed to
be heterogeneous to conclude the potential for
Then, to determine the standard error of d using games to improve the quality of learning
(), the formula was: in STEM domain subjects. The following is an
explanation of the Meta-analysis results: proving
the heterogeneity assumption, exploring the pub-
lication of data bias, inferring effect sizes, and
analyzing moderator variables by utilizing the R
To minimize bias, the effect size and stan- application with the Meta and metafor packages.
dard error were then converted using the formula Three methods, namely Ʈ2, I2, and Q,
proposed by (Hedges, 1981): were used to prove the heterogeneity. These three
methods were used to guarantee the accuracy of
judgment to prove heterogeneity. The analysis re-
sults show that Ʈ2 = 0.2979 [0.2147; 0.5732] >
0 (Retnawati et al., 2018). Thus, based on the Ʈ2
value, it can be concluded that the data in this
study are heterogeneous. Analysis results I2 =
The process of calculating effect size and 77.9% [71.9%; 82.7%] fall into the substantially
standard error used the R application with the heterogeneous data category, so it can be conclu-
meta and metafor packages. The results of effect ded that based on I2, the data in this study are
size and standard error served as the basis for vi- heterogeneous (Higgins, 2003). The Q test results
sual representations in the form of forest plots show a p-value < 0.0001, which is below 0.05 (for
and funnel plots. The effect size results could be a confidence interval of 95%), so according to
classified into five categories: no effect, low ef- the Q test, it can be concluded that the data in this
fect, moderate effect, high effect, and very high study are heterogeneous (Retnawati et al., 2018).
effect (Sullivan & Feinn, 2012). The classification Thus, selecting a random model in Meta-analysis
table for effect size values is presented in Table 3. is appropriate regarding data characteristics and
statistics.
Identification of publication bias is proven
Table 3. Effect Size Categories Results of Meta- using the file-safe N method with the criteria that
Analysis of Contrast Group Model data is said to be free from publication bias when
Effect Size Category N > 5K + 10 (N is the fail-safe value of N and
0 – 0.19 No Effect K is the number of studies). The analysis output
shows Fail-safe N: 5332 for a confidence level of
0.20 – 0.49 Small 95%. Thus, N = 5332 > 5 × 60 +10 = 310. Based
0.50 – 0.79 Medium on these calculations, it is proven that the data is
0.80 – 1.29 Large free from publication bias (Mullen et al., 2001) so
> 1.30 Very Large that the meta-analysis results can be trusted.
After the random model selection is con-
firmed to be correct and devoid of publication
To ensure the quality of the gathered data,
bias, the meta-analysis process can be repeated
this study needed to demonstrate freedom from
to ascertain the conclusion of the differences bet-
publication bias. Techniques for demonstrating
ween the two groups. The effect size represents
bias intervention included funnel plots, Egger’s
the difference between the two groups. The study
test, and fail-safe N. A meta-analysis study is con-
findings suggest that the random effect size mo-
 J. Arlinwibowo, N. Ishartono, Y. Linguistika,
569
D. Purwoko, S. Suprapto/ JPII 12 (4) (2023) 564-574
del has an effect size of 0.5492 [0.3943; 0.7041] research by Kim and Castelli (2021), Mazeas et
and a p-value of 0.0001 with a confidence level of al. (2022), and Wang et al. (2022)such as provi-
95% (Hedges, 1981). ding digital badges earned for specific accomp-
The p-value is 0.0001 0.05 based on this lishments, are related to student engagement in
result, indicating that there is a significant diffe- educational settings. The purpose of this study
rence between the two groups being compared. was to conduct a meta-analytic review to quanti-
The groups compared in this study are students fy the effects of gamified interventions on student
taught using non-game approaches and students behavioral change. Methods: A meta-analysis
taught using game-based learning. The interpre- was performed using the following databases:
table outcomes of the second analysis are 0.5492 The Academic Search Complete, Communicati-
[0.3943; 0.7041]. According to the results of on & Mass Media Complete, Education Source,
this examination, the total effect size is 0.5492. ERIC, Library Information Science & Techno-
A range of total effect size values is discovered logy Abstracts, and PsycINFO. Inclusion in the
between 0.3943 and 0.7041 with a confidence in- review required: (a.
terval of 95%. A positive result implies that the Forest plot visualization can show com-
game-based learning group outperforms the non- prehensive analysis results. The forest plot of the
game-based learning group in terms of learning analysis output using the R application contains
outcomes. The effect size indicates that the st- a summary of the raw data, the mean effect size
rength of the differences between the two groups for each study, the standard error for each study,
is moderate (Sullivan & Feinn, 2012). The results the total effect size, and the total standard error.
of this meta-analysis are consistent with earlier The forest plot can be seen in Figure 2 as follows.

Figure 2. Forest Plot

The general description of the impact of In this research, the moderator variables
game-based learning on improving the quality of analyzed are continent, STEM domain, era, com-
learning shows a positive effect with medium st- petencies developed, game type, and education
rength. Analysis of moderator variables can pro- level. The results of the moderator variable analy-
vide a more in-depth portrayal of various other sis are summarized in the forest plot in Figure 3.
factors that have an influence.
570 J. Arlinwibowo, N. Ishartono, Y. Linguistika,
D. Purwoko, S. Suprapto/ JPII 12 (4) (2023) 564-574

Figure 3. Interaction and Forest Plot of Moderator Variables

Based on the analysis output in Figure The second moderator variable is STEM
3,Based on the analysis output in Figure 3, we domain subjects. Identification based on this mo-
can interpret the influence of various moderator derating variable is intended to explore the effects
variables that have been selected in this research. of Gamification for each STEM domain subject.
The first moderator variable is the continent Each subject has its character (Arlinwibowo et
where the research is conducted. Many studies al., 2020c; Gale et al., 2020; Kelley & Knowles,
suggest that there are differences in the character 2016). Thus, it is possible for a strategy to produ-
of education in each location or country (Arlin- ce different results for each subject. The p-value,
wibowo et al., 2020c) or continent (Uysal, 2009; with a confidence interval of 95%, indicates that
Kartianom & Ndayizeye, 2017). Thus, it becomes there is a significant difference in the impact of
essential to show the impact of independent va- Gamification based on STEM domain subjects.
riables based on research location. When the p- As a result, it is possible to conclude that Gami-
value is less than 0.05 (a confidence interval of fication improves all aspects of the learning pro-
95%), each continent has a different effect of cess in STEM domain subjects. These findings
game-based learning on increasing the quality are consistent with previous research findings
of learning. The interaction P-value is 0.17, in- from a meta-analysis ( Sailer & Homner, 2020;
dicating that there are no significant differences Wang et al., 2022; Arztmann et al., 2023).
between the three continents: Europe, Asia, and The influence on science subjects is the
America. As a result, the utilization of game- highest, followed by mathematics, and the smal-
based learning has the same or a similar benefi- lest influence on engineering subjects. Based on
cial influence on the quality of learning in STEM these findings, it is possible for recommendations
domain subjects across the three continents. The to develop various products in the form of mo-
positive impacts of game-based learning are unaf- dels, methods, and teaching aids to support the
fected by the differences in characteristics bet- implementation of Gamification for science and
ween countries grouped by continent. mathematics learning.
 J. Arlinwibowo, N. Ishartono, Y. Linguistika,
571
D. Purwoko, S. Suprapto/ JPII 12 (4) (2023) 564-574
There are various opportunities to improve learning in STEM domain subjects. Teachers can
and strengthen the quality of mathematics and adapt game-based learning designs according to
science education through Gamification. their respective conditions with virtual and tradi-
The third moderator variable is the peri- tional games.
od when the research is carried out. The period The sixth or final moderator variable is the
is divided into two, namely pre-covid and post- level of students taught with game-based lear-
covid. This distribution is carried out based on ning. Levels are divided into 5; preschool (< 7
extraordinary dynamics that occurred during y.o.), elementary (grades 1-6), junior high school
the COVID-19 pandemic (Bozkurt et al., 2022). (grades 7-9), senior high school (grades 10-12),
The world of education has been impacted so and university (> 18 y.o.). Investigating the sui-
tremendously that there have been various mas- tability of game-based learning at a school level
sive transformations in the education system in is very important because students have different
all countries (Arlinwibowo et al., 2020d). Thus, characteristics at each level (Kraevskii, 2006).
it is important to know whether there are diffe- These character differences influence many ele-
rences in the impact of Gamification in the two ments, including learning style and orientation.
eras. The P-value of the interaction shows 0.49 > The p-value interaction value shows a very small
0.05, which indicates that there is no significant value, namely < 0.03 < 0.05. With a confidence
difference between the two periods. It means that interval of 95%, the p-value shows that there is a
Gamification can improve the quality of learning significant difference in the impact of Gamifica-
in STEM domain subjects before and after CO- tion based on school level. It means that Gami-
VID-19. These findings can be interpreted as a fication has a good impact at all levels, but app-
judgment that Gamification suits diverse learning lication at the elementary level has the greatest
conditions. Implementing game-based learning impact. Uniquely, Gamification does not have a
does not need to worry about its relevance at any very significant impact at the senior high school
time. level. This is because elementary school students
The fourth moderator variable is developed are very interested in games (Lucas, 2017). In
competence. Education has a mission to develop this way, integrating games into the learning pro-
student competencies comprehensively ( Zurqoni cess makes them like learning more (Nand et al.,
et al., 2018; Arlinwibowo et al., 2021b, 2021a). 2019) and even makes learning that was previous-
An in-depth study of these moderator variables is ly considered difficult easier (Udjaja et al., 2018).
crucial to see the suitability of game-based lear- Based on these findings, Gamification is highly
ning for the competency targets being developed. recommended to be applied to learning STEM
Competencies are divided into three categories: domain subjects at elementary school.
affective, knowledge, and skills (Arlinwibowo et
al., 2020a). The interaction p-value shows 0.35 > CONCLUSION
0.05, indicating no significant differences in the
three types of competencies. It means that Ga- In analytical research, the use of a ran-
mification has an equally good influence on the dom model has demonstrated to be adequate in
development of students’ affection, knowledge, terms of data features and statistics Ʈ2, I2, and
and skills. Thus, there is no need to hesitate in Q. Based on the results of the Fail-Safe N analy-
choosing the type of competency to be developed sis, the data obtained has been shown to be free
when implementing game-based learning. of biased publications. As a result, the analysis
The fifth moderator variable is the type
methodologies selected are appropriate, and the
of game, which is classified into virtual (digital)
results may be trusted because they have been
and non-virtual (physical). Not all locations and
ages are suited to virtual gaming (Retnawati et
demonstrated to be free of biased publishing.
al., 2017). Technology can improve the learning Gamification has beneficial effects on student
process (Marsigit et al., 2020) but can also cause learning outcomes, as evidenced by an effect
difficulties. Traditional games are more likely to size value of 0.5492 [0.3943; 0.7041] with a con-
be involved in the learning process in areas with fidence interval of 95%. In the moderator vari-
minimal facilities. Thus, the influence of both ty- ables of continent, era, improving competency,
pes of games needs to be explored. The P-value and game type, gamification does not give a
of the interaction shows 0.73 > 0.05, indicating substantial positive impact. It means that game-
no significant difference between the two types of based learning should be used regardless of loca-
games. It means that both types of games have the tion or competencies being cultivated. It can in-
same effect in improving the quality of student clude any form of game (virtual or non-virtual)
572 J. Arlinwibowo, N. Ishartono, Y. Linguistika,
D. Purwoko, S. Suprapto/ JPII 12 (4) (2023) 564-574
and is applicable both before and after the pan- Borenstein, M. (2009). Effect sizes for continuous data.
demic. Gamification is mainly recommended In The Handbook of Research Synthesis and Me-
ta-analysis, 2nd ed (pp. 221–235). Russell Sage
for STEM domain subjects to be applied to. It is
Foundation.
most commonly used in elementary schools for Bozkurt, A., Karakaya, K., Turk, M., Karakaya, Ö.,
educational purposes. & Castellanos-Reyes, D. (2022). The Impact of
COVID-19 on Education: A Meta-Narrative
REFERENCES Review. TechTrends, 66(5), 883–896.
Çevik, M. (2018). Impacts of the project-based (PBL)
Arlinwibowo, J., Kartowagiran, B., & Retnawati, H. science, technology, engineering, and math-
(2020a). Model penilaian capaian belajar matema- ematics (STEM) education on academic
tika dengan framework STEM [Mathematics learn- achievement and career interests of vocational
ing achievement assessment model using the STEM high school students. Pegem Eğitim ve Öğretim
framework]. UNY Press. Dergisi, 8(2), 281–306.
Arlinwibowo, J., Kistoro, H. C. A., Retnawati, H., Chen, S., Jamiatul Husnaini, S., & Chen, J.-J. (2020).
Kassymova, G. K., & Kenzhaliyev, B. K. Effects of games on students’ emotions of
(2020b). Differences between Indonesia and learning science and achievement in chemistry.
Singapore based on PISA 2015: Five-factor stu- International Journal of Science Education, 42(13),
dents’ perception in science education. Jurnal 2224–2245.
Inovasi Pendidikan IPA, 6(1), 79-87. Chesky, N. Z., & Wolfmeyer, M. R. (2015). Philoso-
Arlinwibowo, J., Retnawati, H., & Kartowagiran, B. phy of STEM education: A critical investigation.
(2020c). The Types of STEM Education Imple- Springer.
mentation in Indonesia. Journal of Xi’an Univer- Chu, H. C., & Chang, S. C. (2014). Developing an
sity of Architecture & Technology, XII(VIII). educational computer game for migratory bird
Arlinwibowo, J., Retnawati, H., Kartowagiran, B., & identification based on a two-tier test approach.
Kassymova, G. K. (2020d). Distance learning Educational Technology Research and Development,
policy in Indonesia for facing pandemic CO- 62(2), 147–161.
VID-19: School reaction and lesson plans. Jour- Deng, W., Pi, Z., Lei, W., Zhou, Q., & Zhang, W.
nal of Theoretical and Applied Information Technol- (2020). Pencil Code improves learners’ com-
ogy, 98(14), 2828–2838. putational thinking and computer learning at-
Arlinwibowo, J., & Rernawati, H. (2015). Developing titude. Computer Applications in Engineering Edu-
audio tactile for visually impaired students. In- cation, 28(1), 90–104.
ternational Journal on New Trends in Education Fadhli, M., Brick, B., Setyosari, P., Ulfa, S., & Kus-
and Their Implications, 6(4), 18–30. wandi, D. (2020). A Meta-Analysis of Selected
Arlinwibowo, J., Retnawati, H., & Kartowagiran, B. Studies on the Effectiveness of Gamification
(2021a). How to Integrate STEM Education Method for Children. International Journal of
in The Indonesian Curriculum? A Systematic Instruction, 13(1), 845–854.
Review. Challenges of Science, 18–25. Fitriyana, N., Wiyarsi, A., Sugiyarto, K. H., & Ikhsan,
Arlinwibowo, J., Retnawati, H., & Kartowagiran, B. J. (2021). The Influences of Hybrid Learning
(2021b). Item Response Theory Utilization for with Video Conference and Chemondro-Game
Developing the Student Collaboration Ability on Students Self-Efficacy, Self-regulated Learn-
Assessment Scale in STEM Classes. Ingénierie ing, and Achievement Toward Chemistry. Turk-
Des Systèmes d Information, 26(4), 409–415. ish Journal of Science Education, 18(2), 233–248.
Arlinwibowo, J., Retnawati, H., & Kartowagiran, B. Gale, J., Alemdar, M., Lingle, J., & Newton, S. (2020).
(2022). The impact of ICT utilization to im- Exploring critical components of an integrated
prove the learning outcome: A meta-analysis. STEM curriculum: An application of the inno-
International Journal of Evaluation and Research vation implementation framework. Internation-
in Education (IJERE), 11(2), 522. al Journal of STEM Education, 7(5), 1–17.
Arlinwibowo, J., Retnawati, H., Pradani, R. G., & Hedges, L. V. (1981). Distribution Theory for Glass’s
Fatima, G. N. (2023). STEM Implementation Estimator of Effect Size and Related Estima-
Issues in Indonesia: Identifying the Problems tors. Journal of Educational Statistics, 6(2), 107–
Source and Its Implications. Qualitative Report, 128.
28(8), 2213–2229. Hernanda, T., Absori, Azhari, A. F., Wardiono, K., &
Arztmann, M., Hornstra, L., Jeuring, J., & Kester, L. Arlinwibowo, J. (2023). Relationship Between
(2023). Effects of games in STEM education: Knowledge and Affection for the Environment:
A meta-analysis on the moderating role of stu- A Meta-Analysis. European Journal of Educa-
dent background characteristics. Studies in Sci- tional Research, 12(2), 1071–1084.
ence Education, 59(1), 109–145. Higgins, J. P. T. (2003). Measuring inconsistency in
Bicer, A., Capraro, R. M., & Capraro, M. M. (2017). meta-analyses. BMJ, 327(7414), 557–560.
Integrated STEM Assessment Model. EUR- Ibrahim, R. H., Ghanim, A. K., & Alkhaderjameel,
ASIA Journal of Mathematics, Science and Tech- H. A. (2020). Impact of Electronic Games on
nology Education, 13(7). the Behavior of Children and their Academic
 J. Arlinwibowo, N. Ishartono, Y. Linguistika,
573
D. Purwoko, S. Suprapto/ JPII 12 (4) (2023) 564-574
Achievement upon Schools in Mosul City. In- Gamification over Students’ Motivation. Jour-
dian Journal of Forensic Medicine & Toxicology, nal of Intellectual Disability - Diagnosis and Treat-
14(1), 374–379. ment, 9(4), 417-422.
Janković, A., & Lambić, D. (2022). The effect of game- Manesis, D. (2020). Barriers to the Use of Games-
based learning via Kahoot and Quizizz on the Based Learning in Preschool Settings: Inter-
academic achievement of third-grade primary national Journal of Game-Based Learning, 10(3),
school students. Journal of Baltic Science Educa- 47–61.
tion, 21(2), 224–231. Marsigit, M., Retnawati, H., Apino, E., Santoso, R.
Kaimara, P., Fokides, E., Oikonomou, A., & Deliyan- H., Arlinwibowo, J., Santoso, A., & Rasmuin,
nis, I. (2021). Potential Barriers to the Imple- R. (2020). Constructing Mathematical Con-
mentation of Digital Game-Based Learning in cepts through External Representations Utiliz-
the Classroom: Pre-service Teachers’ Views. ing Technology: An Implementation in IRT
Technology, Knowledge and Learning, 26(4), 825– Course. TEM Journal, 9(1).
844. Mazeas, A., Duclos, M., Pereira, B., & Chalabaev, A.
Kartianom, K., & Ndayizeye, O. (2017). What’s wrong (2022). Evaluating the Effectiveness of Gamifi-
with the Asian and African Students’ math- cation on Physical Activity: Systematic Review
ematics learning achievement? The multilevel and Meta-analysis of Randomized Controlled
PISA 2015 data analysis for Indonesia, Japan, Trials. Journal of Medical Internet Research, 24(1),
and Algeria. Jurnal Riset Pendidikan Matematika, 1–19.
4(2), 200-210. Mullen, B., Muellerleile, P., & Bryant, B. (2001). Cu-
Kelley, T. R., & Knowles, J. G. (2016). A conceptual mulative Meta-Analysis: A Consideration of
framework for integrated STEM education. Indicators of Sufficiency and Stability. Personal-
International Journal of STEM Education, 3(11), ity and Social Psychology Bulletin, 27(11), 1450–
1–11. 1462.
Khaeroningtyas, N., Permanasari, A., & Hamidah, I. Nand, K., Baghaei, N., Casey, J., Barmada, B., Me-
(2016). STEM learning in material of tempera- hdipour, F., & Liang, H.-N. (2019). Engaging
ture and its change to improve scientific literacy children with educational content via Gamifi-
of junior high school. Jurnal Pendidikan IPA In- cation. Smart Learning Environments, 6(1).
donesia, 5(1), 94-100. Ong, E. T., Ayob, A., Ibrahim, M. N., Adnan, M.,
Kim, J., & Castelli, D. M. (2021). Effects of Gamifi- Shariff, J., & Ishak, N. (2016). The Effective-
cation on Behavioral Change in Education: A ness of an In-Service Training of Early Child-
Meta-Analysis. International Journal of Environ- hood Teachers on STEM Integration through
mental Research and Public Health, 18(7), 1–13. Project-Based Inquiry Learning (PIL). Journal
Kraevskii, V. V., Filonov, G. N., Pidkasistyi, P. I., Mon- of Turkish Science Education, 13(special).
akhov, V. M., Ryzhakov, M. V., Khutorskoi, Öztürk, Ç., & Korkmaz, Ö (2020). The Effect of
A. V., ... & Makarov, A. M. (2006). The prin- Gamification Activities on Students’ Academic
ciples of science education in today’s schools-A Achievements in Social Studies Course,
roundtable. Russian Education and Society, 48(1), Attitudes towards the Course and Cooperative
37-71. Learning Skills. Participatory Educational Re-
Lee, W. H., Shim, H. M., & Kim, H. G. (2022). Effect search, 7(1), 1–15.
of Game-based Learning Using Live Stream- Papastergiou, M. (2009). Digital Game-Based Learn-
ing on Learners’ Interest, Immersion, Satisfac- ing in high school Computer Science educa-
tion, and Instructors’ Perception. International tion: Impact on educational effectiveness and
Journal of Serious Games, 9(2), 3–26. student motivation. Computers & Education,
Legaki, N.-Z., Karpouzis, K., Assimakopoulos, V., & 52(1), 1–12.
Hamari, J. (2021). Gamification to avoid cog- Piaget, J. (2013). Play, Dreams And Imitation In Child-
nitive biases: An experiment of gamifying a hood (1st ed.). Routledge.
forecasting course. Technological Forecasting and Retnawati, H., Apino, E., Kartianom, Djidu, H., &
Social Change, 167, 1–19. Anafiza, R. D. (2018). Pengantar meta analisis.
Legaki, N.-Z., Xi, N., Hamari, J., Karpouzis, K., & Parama Publishing.
Assimakopoulos, V. (2020). The effect of chal- Retnawati, H., Hadi, S., Nugraha, A. C., Arlinwibo-
lenge-based Gamification on learning: An ex- wo, J., Sulistyaningsih, E., Djidu, H., Apino,
periment in the context of statistics education. E., & Iryanti, H. D. (2017). Implementing the
International Journal of Human-Computer Studies, computer-based national examination in Indo-
144, 1–15. nesian schools: The challenges and strategies.
Lucas, F. M. M. (2017). The Game as an Early Child- Problems of Education in the 21st century, 75(6),
hood Learning Resource for Intercultural Edu- 612–633.
cation. Procedia - Social and Behavioral Sciences, Retnawati, H., Kartowagiran, B., Arlinwibowo, J., &
237, 908–913. Sulistyaningsih, E. (2017). Why are the Math-
Mamekova, A. T., Toxanbayeva, N. K., Naubaeva, K. ematics National Examination Items Difficult
T., Ongarbayeva, S. S., & Akhmediyeva, K. and What Is Teachers’ Strategy to Overcome
N. (2021). A Meta-Analysis on the Impact of It? International Journal of Instruction, 10(3),
574 J. Arlinwibowo, N. Ishartono, Y. Linguistika,
D. Purwoko, S. Suprapto/ JPII 12 (4) (2023) 564-574
257–276. Uysal, G. (2009). Human Resource Management in
Sailer, M., & Homner, L. (2020). The Gamification of US, Europe and Asia: Differences and Charac-
Learning: A Meta-analysis. Educational Psychol- teristics. Journal of American Acade
ogy Review, 32(1), 77–112. Business, Cambridge, 14(1), 112-117.
Stankova, M., Tuparova, D., Tuparov, G., & Mihova, Wan Husin, W. N. F., Mohamad Arsad, N., Oth-
P. (2021). Barriers to the Use of Serious Com- man, O., Rasul, M. S., Osman, K., & Iksan,
puter Games in the Practical Work with Chil- Z. (2016). Fostering students’ 21st-century
dren with Educational Difficulties. TEM Jour- skills through Project Oriented Problem-Based
nal, 10(3), 1175–1183. Learning (POPBL) in integrated STEMedu-
Sullivan, G. M., & Feinn, R. (2012). Using Effect cation program. Asia-Pacific Forum on Science
Size—Or Why the P Value Is Not Enough. Learning and Teaching, 17(1), 1–18.
Journal of Graduate Medical Education, 4(3), Wang, L.-H., Chen, B., Hwang, G.-J., Guan, J.-Q., &
279–282. Wang, Y.-Q. (2022). Effects of digital game-
Toda, A. M., Valle, P. H. D., & Isotani, S. (2018). The based STEM education on students’ learning
dark side of Gamification: An overview of achievement: A meta-analysis. International
negative effects of Gamification in education. Journal of STEM Education, 9(26), 1–13.
Communications in Computer and Information Sci- Wisudawati, A. W. (2018). Science Technology Engi-
ence, 832(August), 143–156. neering and Mathematics (STEM) Education
Udjaja, Y., Guizot, V. S., & Chandra, N. (2018). Gami- Approach against a Microscopic Representa-
fication for elementary mathematics learning tion Skill in Atom and Molecule Concept.
in Indonesia. International Journal of Electrical International Journal of Chemistry Education Re-
and Computer Engineering, 8(5), 3860–3865. search, 2(1), 1–5.
Utami, R., Rosyida, A., Arlinwibowo, J., & Fatima, Zurqoni, Z., Retnawati, H., Arlinwibowo, J., & Api-
G. N. (2022). The effectivity of problem-based no, E. (2018). Strategy and Implementation of
learning to improve the HOTS: A meta-analy- Character Education in Senior High Schools
sis. Psychology, Evaluation, and Technology in Edu- and Vocational High Schools. Journal of Social
cational Research, 5(1), 43–53. Studies Education Research, 9(3), 370–397.