Topic Evolution in the Research on Educational Gamification

education sciences Article Topic Evolution in the Research on Educational Gamification Jakub Swacha Department of IT in Management, University of Szczecin, 71-004 Szczecin, Poland; jakub.swacha@usz.edu.pl Abstract: The research on educational gamification spans many topics of interest. As the total volume of research in this area has greatly increased in the last 10 years, it is interesting to see how the interest in the respective topics has changed over the same period. In this paper, we answer this question by means of keyword analysis performed on 7572 unique keywords extracted from 2203 papers. The obtained results reveal (1) the high popularity of keywords that are non-obviously relevant to gamification, (2) vast disproportions in the volume of research dedicated to different aspects of the same research sub-area, and (3) differing patterns of popularity among the most frequent keywords, as well as keywords introduced and abandoned in recent years (4). The presented findings bear a number of implications for the future of research on educational gamification. Keywords: gamification in education; literature survey; keyword analysis 1. Introduction Citation: Swacha, J. Topic Evolution in the Research on Educational Gamification. Educ. Sci. 2022, 12, 640. https://doi.org/10.3390/ educsci12100640 Academic Editor: Huei Tse Hou Received: 31 August 2022 Accepted: 19 September 2022 Published: 22 September 2022 Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. Copyright: © 2022 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ Educational gamification strives to increase learners’ motivation and engagement by incorporating game design elements into educational environments [1]. There is an ongoing debate about the extent and in which educational contexts it delivers what it is expected to [1]. The empirical data indicate that, while not always, more often than not, it does; the most extensive-to-date survey on educational-gamification outcomes reported positive results for 71.4% of the 91 analyzed studies [2]. While educational gamification builds upon the success of gamification in business environments, making use of many techniques first developed for business purposes (primarily, employee and customer engagement [3]), as the interests of teachers and students are much more aligned than those of employers and employees, or vendors and customers, the educational context finds itself free from some of its known drawbacks–in particular, there is no risk of exploitation often associated with gamification in the workplace [4–6]. Once a niche topic, the research on gamification in education amassed in recent years into a large body of knowledge, amounting to 105,000 items reported by Google Scholar for the search terms “education gamification” [7], thus, creating a large opportunity for secondary research. This opportunity did not go unnoticed, and a number of reviews on the research on educational gamification have been published in the last five years. These include systematic literature reviews [1,2,8–32], meta-analyses [33,34], systematic mappings [35–37], and bibliometric surveys [38–45]. None of them, however, focused on how the popularity of the covered topics changed over time. The gap described above is addressed in this paper, which pursues the aim of understanding the evolution of the topics chosen in the research on educational gamification. The presented study is based on keyword analysis, a capable tool for tracking the evolution of a research area. The potential of this approach stems from the fact that keywords are deliberately selected by authors to correctly express the subject matter of papers ([46] and works cited therein). Keyword analysis has been widely implemented for providing an overview of the development of various research areas, including such diverse fields as animal behavior [47], epidemiology [48], psychiatry [49], management information systems [50], technology forecasting [51], and interdisciplinary research [52]. It has also been 4.0/). Educ. Sci. 2022, 12, 640. https://doi.org/10.3390/educsci12100640 https://www.mdpi.com/journal/education Educ. Sci. 2022, 12, 640 2 of 13 successfully applied in the fields of education [53] and educational technology [54–56], game-based learning [57,58], and gamification in general [40,59]. The following research questions have been stated: RQ1. How did the yearly number of publications on educational gamification change over the analyzed period? RQ2. What were the keywords most frequently used throughout the analyzed period? RQ3. How did the set of the most frequently used keywords change over the subsequent years of the analyzed period? RQ4. Which pairs of keywords were most often used together? RQ5. How did the popularity of respective keywords change over the subsequent years of the analyzed period? RQ6. What have been the keywords that have become popular most recently and the keywords that lost their popularity in recent years? The research questions listed above are answered in the six respective subsections of Section 3. Before that, in Section 2, the applied research method and used data sources are described. The final Section 4 concludes the main findings and presents their implications for future research within the area of educational gamification. 2. Materials and Methods As the publications on educational gamification are not limited to a few venues or journals, rather than focusing on a preselected set of these, we decided to query bibliographic databases covering thousands of potentially relevant sources. Three providers of such databases have been chosen, two of which because of their overall extensive coverage of the scientific literature (Scopus [60] and Web of Science [61]) and one because of its thematic focus on education (ERIC [62]). Considering the abundance of publications on educational gamification, in a strive to avoid false positives more than false negatives in the search results, we followed the example of [45] and used just two search keywords: “education” and “gamification”. Unless specified below, the standard query options have been used. As only incomplete data are available for the year 2022, and the earliest indexed publications were published in 2011, the search period has been limited to 10 publication years: 2012–2021. In both Scopus and Web of Science, the allowed language has been set to English, and the search was performed on Title, Author Keywords, and Abstract fields. The Index Terms field containing keywords generated by the respective database provider was not included in the search, as doing so resulted in a high number of false positives. From Scopus, the allowed document types have been limited to “Conference Paper” (1915 items), “Article” (1103 items), and “Book Chapter” (142 items), giving a total of 3160 papers found. From Web of Science, the allowed document types have been limited to “Proceeding Paper” (1518 items), “Article” (701 items), and “Book Chapter” (21 items); a total of 2203 papers has been found (some papers were assigned to more than one document type simultaneously). From ERIC, the “Peer reviewed only” option has been set. The query results comprised 227 publications. The three publication data files were then combined with the duplicates removed. The resulting dataset contained 4324 items, including 3138 that were found in Scopus, 1105 that were found in Web of Science (excluding those also found in Scopus), and 81 that were found in ERIC (excluding those also found in Scopus or Web of Science). The Author Keywords field for the publications found in Scopus or Web of Science and the Manual Tags field for the publications found in ERIC were then processed to obtain a list of keywords. The first processing step involved converting all keywords and titles into lowercase, in order to allow for case-insensitive matching. Educ. Sci. 2022, 12, 640 3 of 13 Next, the identified keywords were looked for in each publication’s title, and matching publications were marked as relevant to a given keyword even if it was not specified in its own Author Keywords or Manual Tags field. Note that, in the case of keywords containing other keywords, only the longest matching keyword found in the title was considered (e.g., if a title contained “mobile learning”, only the “mobile learning” keyword has been assigned to the work, and not “learning”). Next, the keywords used as the search terms (and their variant forms) were removed, as all analyzed works were required to use them to be included in the set, so there was no point in considering them for further analysis. The list of removed keywords comprised the following: “gamification”, “education”, “gamification in education”, “educational gamification”, and “gamifying”. Then, the counts for those terms having different spelling variants or forms were aggregated. In total, 56 such alternative forms were identified, including, e.g., “game” and “games”; “Technology Acceptance Model”, “TAM”, and “Technology Acceptance Model (TAM)”; or “game-based learning”, “game based learning”, “games-based learning”, and “GBL”. Finally, the uppercase letters were restored to the proper nouns and acronyms identified in the keywords. After the above-described pruning, 7572 unique keywords were identified, which occurred 27,272 times in total during the analyzed period. There were 21 keywords which occurred at least 100 times, 58 keywords which occurred at least 50 times, and 158 keywords which occurred at least 25 times during the analyzed period. Looking at the other end of the list, there were 4677 keywords which were encountered only once over the whole analyzed period, 1053 keywords which were encountered only twice, and 448 keywords which were encountered 3 times each. The large number of single-instance keywords is clearly an indicator of a very wide thematic span of the analyzed dataset; among such keywords, there were, e.g., “3d digital storybook”, “Ada”, “business process simulation”, “certification systems”, “digital citizenship education”, “egyptology”, “Filipino sign language”, “global warming”, “honeymoon effect”, “information ecology”, “joyful learning”, “Kolb cycle”, “language fluency”, “Mandarin as second language”, “nutritional knowledge”, “on-the-hour effect”, “passive haptics”, “quality education”, “recurrent neural networks”, “scale development”, “team formation”, “user research”, “virtual library”, “WordNet”, “xMOOC”, “young workers”, and “zero-cost implementation”. All bibliographic data processing, unless specified otherwise, was performed using dedicated scripts written in Python. 3. Results and Discussion 3.1. The Evolution of the Yearly Number of Publications Before we proceed to the keyword analysis, Figure 1 shows the quantitative development of the educational gamification field during the last 10 years. As can be observed in Figure 1, the field was characterized by a very fast consistent growth from its beginnings until the year 2019. There is a small drop in 2020 (which could be possibly linked to the general decrease in research efforts due to the COVID-19 pandemic and the turmoil it caused in higher education institutions [63]) and an even smaller rebound in 2021 (possibly indicating that the educational gamification research is slowly getting back on its pre-COVID-19 level). 3.2. The Overview of Educational Gamification Research Topics In order to provide an overview of the topics appearing in educational gamification research, a word cloud has been drawn based on the list of the most frequent keywords. The WordClouds.com tool has been used for this purpose [64]. The resulting word cloud is shown in Figure 2. Educ. Sci. 2022, 12, 640 4 of 13 800 700 600 500 400 300 200 100 0 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 Figure 1. Number of works included in the analyzed dataset per publication year. Figure 2. The world cloud of the keywords most often used in the educational gamification research. Based on the observations in Figure 2, the following groups of keywords used in the research on educational gamification could be identified: 1. 2. 3. Keywords denoting the purpose of applying gamification (keywords such as “motiva“motition”, “engagement”, “learning”, “teaching”, or “training”); vation”, “engagement”, “learning”, “teaching” “training” Keywords denoting the means, using which type of gamification is being applied (keywords such as “game”, “serious games”, “game-based learning”, or “educational (keywords such as “game”, “serious games”, “game based learning”, or “educagames”); tional games”) Keywords denoting the specific area or the subject of applying the gamification (keywords such as “higher education”, “e-learning”, “blended learning”, “mobile words such as “higher education”, “e learning”, “blended learning”, “mobile learnlearning”, “MOOC”, “course”, “university”, “children”, or “engineering education”); ing”, “MOOC”, “course”, “university”, “children”, “engineering ” ” ” and ” ” “augmented reality” “virtual reality” s “Kahoot!” Educ. Sci. 2022, 12, 640 5 of 13 4. 5. Keywords denoting the technological aspect of applying gamification (from general keywords such as ”technology” and ”educational technology”, across keywords indicating the kind of used technology, such as “augmented reality” or “virtual reality” to the names of specific solutions, the only visible example of which is “Kahoot!”); Keywords denoting the character of the contribution of particular research (keywords such as “design”, “model”, “development”, “evaluation”, “case study”, “analysis”, or “review”). Note that some keywords can be assigned to more than one group, as they can be used for different meanings, e.g., “application” could mean either software (i.e., member of group 4) or the act of applying some method to solve a given problem (i.e., member of group 5). The identified groups of keywords allow for the indication of the main directions of research on educational gamification: 1. 2. 3. 4. 5. Verifying the effectiveness of gamification in education, with regard to known purposes of applying gamification, as well as finding new purposes of applying it; Proposing and analyzing the means using which gamification can be applied in educational domain; Identifying the specifics of applying gamification to specific educational areas or subjects; Adapting or developing new technologies supporting educational gamification; Designing, developing, and validating new models of educational gamification, as well as reporting case studies and analyzing prior research results relevant to educational gamification. 3.3. The Most Frequently Used Keywords Throughout the entire analyzed period, the top 10 keywords were as follows: “higher education” (used 383 times), “game-based learning” (363), “motivation” (353), “serious games” (314), “game” (298), “learning” (273), “e-learning” (224), “teaching” (204), “engagement” (186), and “use” (149). These 10 keywords together were used 2747 times, which constitutes about 10% of all counted uses. The top 25 keywords (covered in Section 3.5) were used 4393 times, which constitutes about 16% of all counted uses. Table 1 lists the 5 most frequent keywords for each year covered in the analysis. Table 1. The most frequently used keywords for each year covered in the analysis. Year 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 1st Most Frequent 2nd Most Frequent 3rd Most Frequent 4th Most Frequent 5th Most Frequent e-learning, game-based learning, motivation (3 each) educational games, serious games (2 each) e-learning (13) motivation, learning, game-based learning (12 each) serious games (11) motivation (19) game, game-based learning (14 each) educational games, serious games (12 each) serious games (35) game (28) e-learning (24) motivation (22) higher education (21) game-based learning (42) serious games (41) e-learning (34) game (32) motivation (30) learning (56) higher education, motivation (43 each) game-based learning (39) serious games (34) motivation (73) higher education (57) game-based learning (46) game (43) serious games (42) higher education (61) game-based learning (60) motivation (57) game (51) serious games (50) higher education (69) game-based learning (56) game (48) motivation (46) teaching (42) higher education (91) game-based learning (70) serious games (50) game (49) motivation (48) As can be observed, with the growing number of papers published each year, the number of appearances of the top keywords grew as well; having three uses was enough to become the most frequent keyword in the area of educational gamification in 2012, whereas it required over 90 uses in 2021. Only one keyword appeared in the top five list throughout the entire analyzed period: “motivation”. As motivation is mentioned in the presented definition of educational gamification [1], this could be expected; what was not expected is the much lower popularity of the second term mentioned in the definition: “engagement”, which, although included among the ten most frequent keywords, did not make it into the top five list in any year of the analyzed period. This indicates that engagement (or the rising thereof) is a gamification purpose that is tackled more rarely than motivation (or the building thereof) by educational gamification researchers, despite the fact that, in education, there are multiple sources Educ. Sci. 2022, 12, 640 6 of 13 of motivation [65], whereas the lack of engagement is often a serious issue, especially in formal education [66]. Interestingly, the two keywords which were missing from the top five (only once, in 2015 and 2020, respectively) were “game-based learning” and “serious games”. Together with “game” (having six appearances in Table 1), these keywords have a non-obvious relationship to gamification, which is usually portrayed as a concept different to fullyfledged games [67]. After screening the publications using these keywords, it was found that, while some of them are actually devoted to both serious games and strictly defined gamification, far more of them simply denote the application of games in education as gamification, clearly indicating that not all researchers are aware of the distinction between the two concepts or agree with it. This point of view can be easily understood if we consider gamification as a process of making non-game activities resemble games, with the most extensive form of this process consisting in turning non-game activities into fully-fledged games. “Higher education” also made it into the top five keywords six times, but was the keyword which took the top spot most often (three times). No other keyword denoting a level of education made it into the top five for any year during the analyzed period. For a comparison, during the whole period, “higher education” was used 383 times as a keyword, whereas “primary education” was used only 39 times and “secondary education” only 33 times. Such a huge difference suggests that educational gamification researchers are most interested in implementing gamification where they themselves teach, and this is most often in higher-education institutions. This is in contrast to the evolution of students’ interest in games, which was reported to peak at the middle-school age [68]. While “learning” appears twice in Table 1 (both times in the top spot in the respective years), “teaching” appears only once (and in the fifth spot), and “training” did not qualify for the top five list during any year. This indicates that the educational gamification literature is much more focused on the student’s perspective than the teacher’s. Only one keyword denoting a mode of education made it to the top five keyword list: “e-learning” (in four years: 2012, 2013, 2015, and 2016). It was used 224 times during the whole analyzed period; for comparison, in total, only five uses of “traditional learning” and three uses of “face-to-face learning” were found, and not a single instance of “brick-andmortar learning”, whereas “blended learning” was used 71 times. This clearly indicates that, despite gamification being used in offline contexts, it is most often applied to computermediated instruction and learning. 3.4. Keyword Co-Occurrence Figure 3 presents 40 pairs of keywords with the highest measured value for the Pearson correlation coefficient (the beige line between two keywords indicates such a pair). While most of these pairs consist of two obviously-related concepts (e.g., “educational games” and “educational technology”, “engagement” and “motivation”, “augmented reality” and “virtual reality”, “active learning” and “flipped classroom”, “innovation” and “technology”, or “game” and “serious games”), some connections bring some new insights, as it stated in the following. • • • • The links between “design” and “evaluation” as well as “development” and “evaluation” suggests that proposed designs and developed solutions were often evaluated; The link between "Kahoot!” and “use” suggests that the papers on Kahoot! most often merely reported its use; The link between “augmented reality” and “mobile learning” suggests that the former technology was often applied to the latter form of learning; The link between “training” and “virtual reality” suggests that the latter technology was often used for training (rather than, e.g., primary education). Educ. Sci. 2022, 12, 640 7 of 13 Figure 3. The most often co-occurring keywords. 3.5. The Change in Keyword Usage Figure 4 compares the number of occurrences for the top 25 keywords in each year during the analyzed period. We can clearly observe the large difference in the number of occurrences between the least frequently used keywords and the remaining ones, and for the first part of the analyzed period, the fast growth in the number of occurrences. Figure 4. The number of keyword occurrences per year during the analyzed period. In order to make it easier to spot the change in the number of occurrences (also for the less frequent keywords, Figure 5), the data for the same top 25 keywords are presented using relative values (100% denotes the maximum number of occurrences of a given keyword in any year during the analyzed period). Educ. Sci. 2022, 12, 640 8 of 13 Figure 5. The relative number of keyword occurrences per year. Based on usage trends visible in Figure 5, we can assign each of the top 25 keywords to one of the following groups: (“higher education”) 1. Consistent fast growth in popularity (“higher education”);(“game based learning”, with minor fluctuations 2. Consistent growth in popularity, with minor fluctuations learning”, “game”, “virtual reality”, “ ”, “ ”, “training”, (“game-based “ ”, “ed“game”, “virtual reality”, “course”, “evaluation”, “training”, “active learning”, “eduucational technology”, “effects”) cational technology”, “effects”); “learning”, “motivation”, “teaching”, “engagement”, Passed of popularity (“learning”, “motivation”, “engagement”, “use”, 3. “use”, peak “educational games”, “case study”, “MOOC”,“teaching”, “application”, “mobile learn“educational games”, “case study”, “MOOC”, “application”, “mobile learning”); ”) 4. Rebound of (“e-learning”, “design”,“development”) “development”); of popularity popularity (“e learning”, “design”, 5. High plateau (“seriousgames”, games”,“augmented “augmented reality”). plateau of of popularity popularity (“serious reality”). Regarding group 1, and its only representative, it is startling to see how fast and consistent the interest in gamification in higher education was, growing throughout the 10 analyzed years. While this finding is consistent with prior research [31,38,41], none of these indicated the unique status of this research topic. It is interesting to think about how long this trend may continue, and whether the interest in gamification of higher education will form a plateau, staying at a high level, or will decrease more or less rapidly. Regarding the keywords of group 2, despite there being some fluctuations in popularity (i.e., individual years with the number of publications over or below the trend line), they generally kept growing throughout the analyzed period. Especially worth noting s list: “ ”, indicating a growing interest in is one member from this list: “evaluation”, indicating a growing interest in evaluating educational gamification applications, which is a good sign considering that this aspect of the educational gamification research has been somewhat neglected in the past, as indicated by the fact that only 20 out of 179 educational gamification papers analyzed in [37] included evaluation. The popularity of the keywords belonging to group 3 seems to have passed its peak. It seems that the volume of writing on motivation and engagement in the educationalgamification context has reached some level of saturation. It is quite surprising to see a fall in the interest in the application of educational “learning” gamification Massive Open Online “MOOC” andto“teaching”, Courses (“MOOC”) and to mobile learning. As for “learning” and “teaching”, this may indicate just a change in the selection of the used keywords from very general (as the two mentioned keywords are) to more specific, which is understandable as the field matures. Educ. Sci. 2022, 12, 640 9 of 13 The group 4 keywords seem to have surpassed a decrease in interest and returned to the path of popularity growth. For “e-learning”, the explanation seems straightforward, as the rebound in its popularity can be clearly linked to the outburst of the COVID-19 pandemic and the emergency remote teaching it resulted in [69]. The popularity of keywords in group 5 reached their highest popularity level earlier (“serious games”) or later (“augmented reality”) and has stayed there since (with small fluctuations). The last member of this group could be seen as unexpected, as educational gamification applications using augmented reality, in spite of many of them being developed (see, e.g., [70]), have not been as widely used as non-educational augmented reality applications such as Pokémon GO (of course, we are aware of the educational use of Pokémon GO, as evidenced, e.g., in [71], but this is not a common practice either). 3.6. The Fresh and Abandoned Keywords The list of all keywords has been screened to find the most popular keywords that have started to appear only in the last three years, as well as the most popular keywords that have no longer appeared in the last three years. The results of this screening are shown in Tables 2 and 3, respectively (only keywords with a total of at least seven occurrences in the respective sub-period were included in order to keep the tables compact). Table 2. The most popular fresh keywords. Keyword Uses in 2019 Uses in 2020 Uses in 2021 Uses in 2019–2021 COVID-19 immersive virtual reality meta-analysis board games nutrition online gamified learning student behavior 0 1 1 2 3 1 4 18 3 5 4 1 3 0 34 5 3 2 3 3 3 52 9 9 8 7 7 7 Table 3. The most popular abandoned keywords. Keyword Uses in 2012–2018 persuasive technology pervasive games games and learning social networking virtual learning 9 9 8 8 7 As could be expected, “COVID-19” made it to the top of the fresh keywords list. This is understandable, considering the huge disruption it caused to the whole educational domain [72]. The remaining fresh keywords attained much less popularity. The most unexpected is the presence of “meta-analysis” on this list, indicating that there were no meta-analyses on the topic of educational gamification published prior to 2019. There are no highly popular keywords listed in Table 3, which means none of the wider sections of research on educational gamification ceased to exist. Quite unexpectedly, “persuasive technology” and “pervasive games” top this list, even though the general interest in these topics (i.e., outside of educational gamification) keeps increasing (see, e.g., [73]). Maybe even more puzzling, is the lack of continued interest in “social networking” among the educational gamification researchers, even though social network is often listed as one of primary gamification components (see, e.g., [74]). As for “games and learning”, both its component terms remain highly popular keywords, so, apparently, it is just that the educational gamification researchers simply no longer combine them into one keyword. Educ. Sci. 2022, 12, 640 10 of 13 4. Conclusions and Implications for Future Work Gamification in education is a quickly growing area of research, with hundreds of scientific papers devoted to it being published every year. Although this body of literature has been the subject of numerous secondary research studies [1,2,8–45], so far, none of these focused on how the popularity of respective covered topics changed over time. This paper contributes to addressing this gap via keyword analysis, helping to understand the evolution of the set of topics most often selected in the research on educational gamification. The obtained results provide a handful of novel insights. Already, the identification of the most frequent keywords throughout the entire analyzed period revealed that two are in a non-obvious relationship to gamification (“game-based learning” and “serious games”, which are perceived by some authors as distinct from gamification [67]) and a lack of balance both between student–teacher perspectives (with much more attention given to the student perspective) and the different levels of education (with much more attention given to higher education than to the other levels of education). This brings implications to future work: firstly, unless a significant share of gamification research is to be ignored, a more comprehensive definition of gamification is needed, clearly placing, under its umbrella, the adoption of fully-fledged games for non-entertainment purposes (rather than only game elements). Secondly, more research attention is needed on those aspects of educational gamification that were insufficiently addressed by prior studies, i.e., the gamification of teaching (rather than learning) and the application of gamification at various education levels (rather than higher education only). The analysis of keyword usage in subsequent years reveals that, despite the consistent growth in the volume of educational gamification research, the popularity of the most frequent keywords has evolved in differing ways. Only 10 of the top 25 most frequent keywords were characterized by steady growth in their yearly number of occurrences, whereas, for the same number of keywords, the number of occurrences decreased in the most recent years. The popularity level of three keywords has stabilized, and two keywords rebounded from a period of lower popularity. The analysis of these patterns also provides some implications for further research, such as the need for a meta-analysis of the models used in educational gamification or the need for the identification of the barriers hampering the spread of the numerous gamified augmented reality applications into educational practice. We have also identified keywords that came into use only recently and keywords that are no longer used in spite of having some popularity in the past. As the former group indicates the emerging research fronts (the main of which regards COVID-19), the latter directs the educational gamification researchers’ attention toward topics that have lost it (in particular, “persuasive technology”, “pervasive games,” and “social networking”). Funding: This research received no external funding. Institutional Review Board Statement: Not applicable. Informed Consent Statement: Not applicable. Data Availability Statement: Data are available from the authors on request. Conflicts of Interest: The authors declare no conflict of interest. References 1. 2. 3. 4. Dichev, C.; Dicheva, D. Gamifying Education: What Is Known, What Is Believed and What Remains Uncertain: A Critical Review. Int. J. Educ. Technol. High Educ. 2017, 14, 9. [CrossRef] Majuri, J.; Koivisto, J.; Hamari, J. Gamification of Education and Learning: A Review of Empirical Literature. In Proceedings of the 2nd International GamiFIN Conference—CEUR-WS.org, Pori, Finland, 21–23 May 2018; pp. 11–19. Robson, K.; Plangger, K.; Kietzmann, J.H.; McCarthy, I.; Pitt, L. Game on: Engaging Customers and Employees through Gamification. Bus. Horiz. 2016, 59, 29–36. [CrossRef] Kim, T.W. Gamification of Labor and the Charge of Exploitation. J. Bus. Ethics 2018, 152, 27–39. [CrossRef] Educ. Sci. 2022, 12, 640 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 11 of 13 Hammedi, W.; Leclercq, T.; Poncin, I.; Nasr, L.A. Née Uncovering the Dark Side of Gamification at Work: Impacts on Engagement and Well-Being. J. Bus. Res. 2021, 122, 256–269. [CrossRef] Fitzpatrick, S.; Marsh, T. The Dehumanising Consequences of Gamification: Recognising Coercion and Exploitation in Gamified Systems. In Advances in Business Strategy and Competitive Advantage; Bernardes, O., Amorim, V., Moreira, A.C., Eds.; IGI Global: Hershey, PA, USA, 2022; pp. 398–417. ISBN 978-1-79989-223-6. Google Scholar. Available online: https://scholar.google.com/scholar?q=education+gamification (accessed on 13 September 2022). Antonaci, A.; Klemke, R.; Specht, M. The Effects of Gamification in Online Learning Environments: A Systematic Literature Review. Informatics 2019, 6, 32. [CrossRef] Arufe-Giráldez, V.; Sanmiguel-Rodríguez, A.; Ramos-Álvarez, O.; Navarro-Patón, R. Gamification in Physical Education: A Systematic Review. Educ. Sci. 2022, 12, 540. [CrossRef] Dreimane, S. Gamification for Education: Review of Current Publications. In Didactics of Smart Pedagogy: Smart Pedagogy for Technology Enhanced Learning; Daniela, L., Ed.; Springer International Publishing: Cham, Switzerland, 2019; pp. 453–464. ISBN 978-3-030-01551-0. Gari, M.R.N.; Walia, G.S.; Radermacher, A. Gamification in Computer Science Education: A Systematic Literature Review. In Proceedings of the 2018 ASEE Annual Conference & Exposition, Salt Lake City, UT, USA, 23 June–27 July 2018; American Society for Engineering Education: Salt Lake City, UT, USA, 2018. González-Fernández, A.; Revuelta-Domínguez, F.-I.; Fernández-Sánchez, M.R. Models of Instructional Design in GaAmification: A Systematic Review of the Literature. Educ. Sci. 2022, 12, 44. [CrossRef] Hallifax, S.; Serna, A.; Marty, J.-C.; Lavoué, É. Adaptive Gamification in Education: A Literature Review of Current Trends and Developments. In Proceedings of the Transforming Learning with Meaningful Technologies, Delft, The Netherlands, 16–19 September 2019; Scheffel, M., Broisin, J., Pammer-Schindler, V., Ioannou, A., Schneider, J., Eds.; Springer International Publishing: Cham, Switzerland, 2019; pp. 294–307. Indriasari, T.D.; Luxton-Reilly, A.; Denny, P. Gamification of Student Peer Review in Education: A Systematic Literature Review. Educ. Inf. Technol. 2020, 25, 5205–5234. [CrossRef] Inocencio, F. Using Gamification in Education: A Systematic Literature Review. In Proceedings of the International Conference on Information Systems 2018 (ICIS 2018), San Francisco, CA, USA, 13–16 December 2018; AIS: San Francisco, CA, USA, 2018; p. 3. Jarnac de Freitas, M.; Mira da Silva, M. Systematic Literature Review about Gamification in MOOCs. Open Learn. J. Open Distance E-Learn. 2020, 1–23. [CrossRef] Kalogiannakis, M.; Papadakis, S.; Zourmpakis, A.-I. Gamification in Science Education. A Systematic Review of the Literature. Educ. Sci. 2021, 11, 22. [CrossRef] Klock, A.C.T.; Gasparini, I.; Pimenta, M.S.; Hamari, J. Tailored Gamification: A Review of Literature. Int. J. Hum. Comput. Stud. 2020, 144, 102495. [CrossRef] Kocakoyun, S.; Ozdamli, F. A Review of Research on Gamification Approach in Education. In Socialization—A Multidimensional Perspective; Morese, R., Palermo, S., Nervo, J., Eds.; InTech: London, UK, 2018; ISBN 978-1-78923-308-7. Legaki, N.Z.; Hamari, J. Gamification in Statistics Education: A Literature Review. In Proceedings of the CEUR Workshop Proceedings, Levi, Finland, 1–3 April 2020; Volume 2637, pp. 41–51. Limantara, N.; Meyliana; Gaol, F.L.; Prabowo, H. Factors Influencing the Implementation of Gamification for Learning in Information Systems Education. Int. J. Emerg. Technol. Learn. (IJET) 2022, 17, 32–41. [CrossRef] Limantara, N.; Meyliana; Hidayanto, A.N.; Prabowo, H. The Elements of Gamification Learning in Higher Education: A Systematic Literature Review. Int. J. Mech. Eng. Technol. 2019, 10, 982–991. Looyestyn, J.; Kernot, J.; Boshoff, K.; Ryan, J.; Edney, S.; Maher, C. Does Gamification Increase Engagement with Online Programs? A Systematic Review. PLoS ONE 2017, 12, e0173403. [CrossRef] Manzano-León, A.; Camacho-Lazarraga, P.; Guerrero, M.A.; Guerrero-Puerta, L.; Aguilar-Parra, J.M.; Trigueros, R.; Alias, A. Between Level Up and Game Over: A Systematic Literature Review of Gamification in Education. Sustainability 2021, 13, 2247. [CrossRef] Maryono, D.; Budiyono, S.; Akhyar, M. Implementation of Gamification in Programming Learning: Literature Review. Int. J. Inf. Educ. Technol. 2022, in press. Metwally, A.H.S.; Nacke, L.E.; Chang, M.; Wang, Y.; Yousef, A.M.F. Revealing the Hotspots of Educational Gamification: An Umbrella Review. Int. J. Educ. Res. 2021, 109, 101832. [CrossRef] Milosz, M.; Milosz, E. Gamification in Engineering Education—A Preliminary Literature Review. In Proceedings of the 2020 IEEE Global Engineering Education Conference (EDUCON), Porto, Portugal, 27–30 April 2020; IEEE: Porto, Portugal, 2020; pp. 1975–1979. Ortega-Arranz, A.; Muñoz-Cristóbal, J.A.; Martínez-Monés, A.; Bote-Lorenzo, M.L.; Asensio-Pérez, J.I. How Gamification Is Being Implemented in MOOCs? A Systematic Literature Review. In Data Driven Approaches in Digital Education; Lavoué, É., Drachsler, H., Verbert, K., Broisin, J., Pérez-Sanagustín, M., Eds.; Springer International Publishing: Cham, Switzerland, 2017; Volume 10474, pp. 441–447. ISBN 978-3-319-66609-9. Osatuyi, B.; Osatuyi, T.; de la Rosa, R. Systematic Review of Gamification Research in IS Education: A Multi-Method Approach. Commun. Assoc. Inf. Syst. 2018, 42, 95–124. [CrossRef] Educ. Sci. 2022, 12, 640 30. 31. 32. 33. 34. 35. 36. 37. 38. 39. 40. 41. 42. 43. 44. 45. 46. 47. 48. 49. 50. 51. 52. 53. 54. 55. 56. 57. 12 of 13 da Silva, R.J.R.; Rodrigues, R.G.; Leal, C.T.P. Gamification in Management Education: A Systematic Literature Review. BAR Braz. Adm. Rev. 2019, 16, e180103. [CrossRef] Subhash, S.; Cudney, E.A. Gamified Learning in Higher Education: A Systematic Review of the Literature. Comput. Hum. Behav. 2018, 87, 192–206. [CrossRef] Venter, M. Gamification in STEM Programming Courses: State of the Art. In Proceedings of the 2020 IEEE Global Engineering Education Conference (EDUCON), Porto, Portugal, 27–30 April 2020; IEEE: Porto, Portugal, 2020; pp. 859–866. Baptista, G.; Oliveira, T. Gamification and Serious Games: A Literature Meta-Analysis and Integrative Model. Comput. Hum. Behav. 2019, 92, 306–315. [CrossRef] Sailer, M.; Homner, L. The Gamification of Learning: A Meta-Analysis. Educ Psychol Rev. 2020, 32, 77–112. [CrossRef] Alhammad, M.M.; Moreno, A.M. Gamification in Software Engineering Education: A Systematic Mapping. J. Syst. Softw. 2018, 141, 131–150. [CrossRef] Kamunya, S.; Oboko, R.; Maina, E. A Systematic Mapping of Adaptive Gamification in E-Learning. Open J. Inf. Technol. 2019, 2, 53–68. [CrossRef] Klock, A.C.T.; Ogawa, A.N.; Gasparini, I.; Pimenta, M.S. Does Gamification Matter? A Systematic Mapping about the Evaluation of Gamification in Educational Environments. In Proceedings of the 33rd Annual ACM Symposium on Applied Computing, Pau, France, 9 April 2018. Bagher Khatibi, M.; Badeleh, A.; Khodabandelou, R. A Bibliometric Analysis on the Research Trends of Gamification in Higher Education: 2010–2020. New Educ. Rev. 2021, 65, 17–28. [CrossRef] Behl, A.; Jayawardena, N.; Pereira, V.; Islam, N.; Giudice, M.D.; Choudrie, J. Gamification and E-Learning for Young Learners: A Systematic Literature Review, Bibliometric Analysis, and Future Research Agenda. Technol. Forecast. Soc. Chang. 2022, 176, 121445. [CrossRef] Bassanelli, S.; Vasta, N.; Bucchiarone, A.; Marconi, A. Gamification for Behavior Change: A Scientometric Review. Acta Psychol. 2022, 228, 103657. [CrossRef] [PubMed] Grosseck, G.; Malita, L.; Sacha, G.M. Gamification in Higher Education: A Bibliometric Approach. In Proceedings of the 16th International Scientific Conference—ELearning and Software for Education, Bucharest, Romania, 22–23 April 2020; eLSE: Bucharest, Romania, 2020; pp. 20–30. Guerrero-Alcedo, J.M.; Espina-Romero, L.C.; Nava-Chirinos, Á.A. Gamification in the University Context: Bibliometric Review in Scopus (2012–2022). Int. J. Learn. Teach. Educ. Res. 2022, 21, 309–325. [CrossRef] Nadi-Ravandi, S.; Batooli, Z. Gamification in Education: A Scientometric, Content and Co-Occurrence Analysis of Systematic Review and Meta-Analysis Articles. Educ. Inf. Technol. 2022, 27, 10207–10238. [CrossRef] Schöbel, S.; Saqr, M.; Janson, A. Two Decades of Game Concepts in Digital Learning Environments—A Bibliometric Study and Research Agenda. Comput. Educ. 2021, 173, 104296. [CrossRef] Swacha, J. State of Research on Gamification in Education: A Bibliometric Survey. Educ. Sci. 2021, 11, 69. [CrossRef] Yang, J.; Bu, Y.; Lu, W.; Huang, Y.; Hu, J.; Huang, S.; Zhang, L. Identifying Keyword Sleeping Beauties: A Perspective on the Knowledge Diffusion Process. J. Informetr. 2022, 16, 101239. [CrossRef] Ord, T.J.; Martins, E.P.; Thakur, S.; Mane, K.K.; Börner, K. Trends in Animal Behaviour Research (1968–2002): Ethoinformatics and the Mining of Library Databases. Anim. Behav. 2005, 69, 1399–1413. [CrossRef] Zhang, Y.; Cai, X.; Fry, C.V.; Wu, M.; Wagner, C.S. Topic Evolution, Disruption and Resilience in Early COVID-19 Research. Scientometrics 2021, 126, 4225–4253. [CrossRef] Wu, Y.; Jin, X.; Xue, Y. Evaluation of Research Topic Evolution in Psychiatry Using Co-Word Analysis. Medicine 2017, 96, e7349. [CrossRef] [PubMed] Choi, J.; Yi, S.; Lee, K.C. Analysis of Keyword Networks in MIS Research and Implications for Predicting Knowledge Evolution. Inf. Manag. 2011, 48, 371–381. [CrossRef] Yoon, B.; Park, Y. A Systematic Approach for Identifying Technology Opportunities: Keyword-Based Morphology Analysis. Technol. Forecast. Soc. Chang. 2005, 72, 145–160. [CrossRef] Xu, J.; Bu, Y.; Ding, Y.; Yang, S.; Zhang, H.; Yu, C.; Sun, L. Understanding the Formation of Interdisciplinary Research from the Perspective of Keyword Evolution: A Case Study on Joint Attention. Scientometrics 2018, 117, 973–995. [CrossRef] Papamitsiou, Z.; Giannakos, M.; Simon; Luxton-Reilly, A. Computing Education Research Landscape through an Analysis of Keywords. In Proceedings of the 2020 ACM Conference on International Computing Education Research, Virtual Event, New Zealand, 10 August 2020. Chang, C.Y.; Hwang, G.J. Trends of Mobile Technology-Enhanced Medical Education: A Review of Journal Publications from 1998 to 2016. IJMLO 2018, 12, 373. [CrossRef] Chen, X.; Yu, G.; Cheng, G.; Hao, T. Research Topics, Author Profiles, and Collaboration Networks in the Top-Ranked Journal on Educational Technology over the Past 40 Years: A Bibliometric Analysis. J. Comput. Educ. 2019, 6, 563–585. [CrossRef] Chen, X.; Zou, D.; Xie, H.; Cheng, G.; Liu, C. Two Decades of Artificial Intelligence in Education. Educ. Technol. Soc. 2022, 25, 28–47. Chen, X.; Zou, D.; Kohnke, L.; Xie, H.; Cheng, G. Affective States in Digital Game-Based Learning: Thematic Evolution and Social Network Analysis. PLoS ONE 2021, 16, e0255184. [CrossRef] Educ. Sci. 2022, 12, 640 58. 59. 60. 61. 62. 63. 64. 65. 66. 67. 68. 69. 70. 71. 72. 73. 74. 13 of 13 Hwang, G.-J.; Chen, P.-Y. Interweaving Gaming and Educational Technologies: Clustering and Forecasting the Trends of Game-Based Learning Research by Bibliometric and Visual Analysis. Entertain. Comput. 2022, 40, 100459. [CrossRef] Albertazzi, D.; Ferreira, M.G.G.; Forcellini, F.A. A Wide View on Gamification. Technol. Knowl. Learn. 2019, 24, 191–202. [CrossRef] Scopus. Available online: https://www.scopus.com/ (accessed on 25 August 2022). Web of Science. Available online: https://webofscience.com/ (accessed on 25 August 2022). ERIC. Available online: https://eric.ed.gov/ (accessed on 25 August 2022). Swacha, J. Online Teaching in the Times of COVID-19 and the Change in Work Time Distribution of Faculty Members. In Proceedings of the AMCIS 2022 Proceedings, Minneapolis, MN, USA, 10–13 August 2022. WordClouds.com. Available online: https://www.wordclouds.com/ (accessed on 13 September 2022). Gorham, J.; Millette, D.M. A comparative analysis of teacher and student perceptions of sources of motivation and demotivation in college classes. Commun. Educ. 1997, 46, 245–261. [CrossRef] Bryson, C.; Hand, L. The role of engagement in inspiring teaching and learning. Innov. Educ. Teach. Int. 2007, 44, 349–362. [CrossRef] Prince, J.D. Gamification. J. Electron. Resour. Med. Libr. 2013, 10, 162–169. [CrossRef] Greenberg, B.S.; Sherry, J.; Lachlan, K.; Lucas, K.; Holmstrom, A. Orientations to Video Games Among Gender and Age Groups. Simul. Gaming 2008, 41, 238–259. [CrossRef] Hodges, C.; Moore, S.; Lockee, B.; Trust, T.; Bond, A. The difference between emergency remote teaching and online learning. Educause Review, 27 March 2020. Available online: https://er.educause.edu/articles/2020/3/the-difference-between-emergencyremote-teaching-and-online-learning (accessed on 13 September 2022). Lampropoulos, G.; Keramopoulos, E.; Diamantaras, K.; Evangelidis, G. Augmented Reality and Gamification in Education: A Systematic Literature Review of Research, Applications, and Empirical Studies. Appl. Sci. 2022, 12, 6809. [CrossRef] Soltani, P.; Morice, A.H. Augmented reality tools for sports education and training. Comput. Educ. 2020, 155, 103923. [CrossRef] d’Orville, H. COVID-19 causes unprecedented educational disruption: Is there a road towards a new normal? Prospects 2020, 49, 11–15. [CrossRef] [PubMed] Ndulue, C.; Orji, R. Games for Change—A Comparative Systematic Review of Persuasive Strategies in Games for Behaviour Change. IEEE Trans. Games 2022, 1. [CrossRef] Rodríguez, I.; Puig, A.; Rodríguez, À. Towards Adaptive Gamification: A Method Using Dynamic Player Profile and a Case Study. Appl. Sci. 2022, 12, 486. [CrossRef]