Gamification of MOOCs for increasing user engagement

Gamification of MOOCs for Increasing User Engagement Anant Vaibhav, Student Member, IEEE Pooja Gupta HCL Technologies, SEZ, Sector- 126, Noida, Uttar Pradesh, India anant.vaib@gmail.com Dept. of Computer Science Engineering Maharaja Agrasen Institute of Technology, Rohini, Delhi, India poojaguptamait@gmail.com Abstract—As the educational organizations worldwide are moving towards bringing education experience on internet, via Massive Open Online Courses (MOOCs) on Massive Open Online Education Platforms (MOOEPs), the retention of the enrolled students throughout the course is still a huge challenge. Although the online education has an immensely noble aim, i.e., to provide free\less expensive quality education to everyone, but if the existing MOOCs are upgraded to more captivating environment for users, a drastic increase in the user engagement in the course can be achieved, and hence MOOCs’ providers can reach their ultimate goal in a more ubiquitous manner. In this paper it is suggested that this divide between a MOOC and user’s engagement can be bridged with Gamification. Gamification is the use of game design elements in non-game contexts. Here an experimental environment was setup to compare the results of a small vocabulary course distributed among 100 candidates, and a comparative analysis between the gamified learning method and conventional learning method has been done. After the completion of the course a small survey was also taken from the candidates who had gamified tool. The results show that if the learning platform is gamified, it does not only drastically increase the user enrolment but also increase user engagement throughout the course. Gamification also makes learning, a unique and amusing experience. Index Terms— MOOCs, MOOEPs, Gamification, User engagement, Online Education I. INTRODUCTION Online education has been constantly modifying the concept of distance learning and distance education from the early 90s [1]-[2]. By 2008-2009, many colleges and universities in the United States considered online education as an integral part of their instruction model and strategic plan [3]. At around the same time, an interesting online education model started to evolve from several existing prototypes open courseware, virtual classroom, networked learning, distance learning and open education. This has led to the creation of what is now called a Massive Open Online Course (MOOC). A MOOC is a free web-based course with open registration, publicly shared curriculum and open-ended learning outcomes [4]. The year of 2012 saw rapid development and expansion of several Massive Open Online Education Platforms (MOOEPs) like Canvas [5], classToGo [6], Coursera [7], edX [8], NPTEL [9], Udacity[10], and many c 978-1-4799-6876-3/14/$31.00 2014 IEEE others. These MOOEPs provided a centralized domain for courses provided by various universities and industries of the world. The evolution and widespread of these MOOEPs can be ascribed to the concept of pervasive learning – the idea that learning can occur anywhere and not necessarily in classrooms. This pervasive learning removes chronological and spatial boundaries in education and enables continual growth and development of knowledge and skills [11]. The technology that enables this realization of ubiquity in learning is, cloud computing. In the context of education, cloud computing can be defined as a model that provides on demand access to an infrastructure that includes computing resources, storage and applications to readily deliver and manage a scalable online course. This central idea of providing a free knowledge platform to anyone, anytime, and anywhere, makes MOOCs a powerful learning technology. But as far as the process of knowledge gaining from a MOOC is concerned, retaining the enrolled students throughout the course is a challenge. Statistics: The first ever MOOC offered by Massachusetts Institute of Technology (MITx), instructed by the director of edX.org, Prof. Anant Aggarwal, “6.002x: Circuits & Electronics”, stated following statistics: 6.002x had 154,763 registrants. Of these, 69,221 people looked at the first problem set, and 26,349 earned at least one point on it. 13,569 people looked at the midterm while it was still open, 10,547 people got at least one point on the midterm, and 9,318 people got a passing score on the midterm. 10,262 people looked at the final exam while it was still open, 8,240 people got at least one point on the final exam, and 5,800 people got a passing score on the final exam. Finally, after completing 14 weeks of study, 7,157 people have earned the first certificate awarded by MITx. [12]. A course on edX.org taught by Prof. Wallter Lewin, “8.02x Electricity and Magnetism” provided by MIT, started on 18th February 2013. It was spread over 13 weeks. At the end of the course, the course staff notified students with following statistics, “The number of students that registered was about 40,000, but only about 8000 took the first homework. About 4000 took the first exam and of those 1721 passed the course” [13]. Another course on edX.org instructed by Dr. Michael E. Webber, “UT.1.01x: Energy 101”, provided by University of Texas at Austin started on 15th September 290 2013 saw similar statistics. Total enrolled students were around 40,000 and at the end only a 4,707 students qualified for a certificate [14]. All of these courses were of intermediate difficulty level. But according to the statistics, only 5-10% of the students passed and qualified for a certificate, even though the qualifying percentage for completion of these courses was 60%. The plausible reasons for these daunting statistics might be: (1) Students did not aim to take the tests and assignments but joined only for knowledge or out of interest. (2) Courses were too long and challenging and students discontinued due to decaying interests or time constraints or even boredom. The drop outs due to former reason cannot be reduced, because this is up to an individual. But the latter cause can be used to improve the retention of students throughout the course. This can be achieved by making the platform more interesting and interactive for the students. A promising approach to increase user participation can be achieved by applying Gamification methods. Deterding et al. [15] define Gamification as “the use of game design elements in non-game contexts”, i.e., game mechanics and concepts are applied on non-gaming environments to reach specific goals. Examples for goals include an improvement of user engagement, increased participation, enhanced motivation or just having more fun. Focusing more on the users’ perspective, Huotari and Hamari [16] define Gamification as “a process of enhancing a service with affordances for gameful experiences in order to support user’s overall value creation”. Gamification techniques strive to leverage people's natural desires for socializing, learning, mastery, competition, achievement, status, selfexpression, altruism, or closure. Early gamification strategies use rewards for players who accomplish desired tasks or competition to engage players. Types of rewards include points, achievement badges or levels, the filling of a progress bar, or providing the user with virtual currency. Making the rewards for accomplishing tasks visible to other players or providing leader boards are ways of encouraging players to compete. Due to potentially problematic consequences of competition, which can result in unethical behavior, low cooperation and low collaboration, or disadvantaging certain player demographics such as women, current gamification designs try to refrain from using this element. Another approach to gamification is to make existing tasks feel more like games. Some techniques used in this approach include adding meaningful choice, onboarding with a tutorial, increasing challenge, and adding narrative. In this paper, the analysis shows that Gamification of a learning tool does not only increase the user enrollment but it also increases the user retention and the success rate of the candidates enrolled in that learning tool. The paper is classified in two parts, first part (i.e. section II) is a case study based on edX.org and second part (i.e. section III & section IV) is the main experimental setup and results. The further details of paper organization are as follows, Section II shows a case study of edX.org, in which, all the courses available on edX.org have been organized discipline wise and geographywise. Section III describes the experimental setup, which contains a 3-day vocabulary building course. This setup has been categorized in two environments, one is non-Gamified, and second is Gamified through a learning platform called quizlet.com. In Section IV, the results of the experiments have been analyzed and comparisons have been drawn between gamified and non-gamified environments based on the experiment. And at the end, Section V presents the conclusions drawn from the results. II. CASE STUDY EdX is a massive open online course (MOOC) provider and online learning platform. It hosts online university-level courses in a wide range of disciplines to a worldwide audience at no charge. It also conducts research into learning based on how people use its platform. EdX differs from other MOOC platforms, such Coursera and Udacity, in that it is nonprofit and runs on an open-source software platform. EdX was founded by the Massachusetts Institute of Technology and Harvard University in May 2012. There are currently 56 schools, nonprofits, corporations, and international organizations that offer or plan to offer courses on the edX website. As of 23rd September 2014, edX has more than 2.5 million users taking over 270 courses online [8]. The purpose of this case study is to analyze various courses and course providers available on edX.org platform and summarize them discipline wise and location wise. Discipline-wise distribution: The courses on edX belong to various disciplines like science, engineering, arts, music, etc. Table 1 summarizes the percentage wise distribution of all the online courses based on discipline, provided by various organizations and institutions on edX.org. This table reflects that the amount of computer science courses is maximum on edX.org. Region-wise distribution: EdX provides a common platform for institutions and organizations present worldwide to present and start their courses on edX.org. The statistics for the institutes using edx.org for their courses is collected according to their location and a pie chart for the same distribution is shown in fig. 1. According to fig. 1, The American universities (60% courses) comprise of MIT, Harvard, Cornell University, University of Texas, Rice University, Stanford University, University of California Berkeley and Canadian universities like University of British Columbia, etc. Australian universities (4% courses) like Australian National University and University of Queensland is also a part of edX. Asian universities (12%) contain universities from China and Japan, like Tsingua University, Peking University, etc and recently Indian Institute of Technology-Bombay has collaborated with edX and started a course on Introduction to Programing. European universities (12% courses) like EPFL, ETH Zurich, Delft University of Technology, etc have their courses successfully running on edX.org. And some organizations like Linux Foundation, The International Monetary Fund, etc have been categorized in “Others” and 2014 IEEE International Conference on MOOC, Innovation and Technology in Education (MITE) 291 contain 12% of all the courses present on edX. Recently a high school initiative has also started on edX, which provides courses for high school students, these courses are provided by various schools and organizations worldwide. This analysis shows that American universities are the providers of maximum courses on edX.org. TABLE I DISCIPLINE-WISE DISTRIBUTION OF COURSES ON EDX.ORG Discipline Architecture Art & Culture Biology & Life Sciences Business & Management Chemistry Communication Computer Science Economics & Finance Education Electronics Energy & Earth Sciences Environmental studies Food & Nutrition Health & Safety History Law Literature Math Medicine Music Philanthropy Philosophy & Ethics Physics Stats & Data Analysis Percentage courses (%) 0.6 5.8 6.5 5 2.2 1.8 11.1 4.4 1.2 3.4 1.4 4 1.4 5.8 9.9 2.6 4.8 6.9 4 0.8 0.2 4.2 6.5 5.2 III. EXPERIMENTAL SETUP Learning vocabulary words in conventional method i.e. reading them in books or as form of word lists or even online is sometimes a tedious and dreary task. Learning vocabulary is not just about memorizing the words and their meanings, but the real time usage of these words helps in developing vocabulary and retaining the difficult words. Quizlet.com is an online learning website in which helps users to create and share their vocabulary word lists [17]. At first user enters the words and their meaning and after this a database is generated, users are free to learn through various gamified techniques such as flash cards, Speller, space race and scatter. These techniques designed by developers of quizlet.com allow real time learning and makes it an enjoyable experience. The objective of this section is to show that gamified environment increases the user interest in learning even the boring concepts. An experiment was set up for this purpose. For this Experiment, 5 different wordlists, each list containing 25 challenging English words, were created. The test process was performed via two environments. A. Group-A : Non-gamified environment B. Group-B: Gamified environment A. Non-Gamified Environment (Group-A): For the testing of non-gamified environment, this was adopted by conventional method of learning, i.e. “reading” a document was adopted. For this purpose, PDF of the word lists were randomly mailed within a group of 50 volunteering candidates comprising of various ages and backgrounds. A time limit of 3 days was given for this task, after which they had to take a test based on their respective list and passing percentage was set at an optimal 65%. B. Gamified environment (Group-B): This was adopted by the unconventional way of teaching, i.e., through a web based gamified learning platform called quizlet.com. For this purpose, the links of these quizlet lists were mailed to a group of 50 volunteering candidates comprising different ages and backgrounds. This group was also given a 3 day window, after which they had to appear for a test based on respective lists but the passing percentage in this case was set to 70%. Fig. 2 shows the learning interface of quizlet.com, which was used for group-B. As an example, out of few gamified applications available on quizlet, the “scatter” application has been selected, which enables user to drag and drop the words to their appropriate meaning with the time being recorded, this is shown in fig. 3. Fig. 1. Region wise distribution of courses on edX.org 292 2014 IEEE International Conference on MOOC, Innovation and Technology in Education (MITE) 1. Non-Gamified Environment (Group-A): Out of 50 candidates who were asked to take up the conventional word learning course for 3 days, only 35 candidates actually agreed to take up the final test. In this test only 22 were able to pass the course, with the passing limit of 65%. A pie chart for the same has been shown in fig. 4. Fig. 2. Interface of quizlet.com platform Fig. 4. Percentage distribution of outcomes of non-gamified environment 2. Gamified Environment (Group-B): Out of 50 candidates who were asked to take up the gamified word learning course for 3 days on quizlet platform, 42 candidates actually agreed to take up the final test. In this test, surprisingly 36 were able to pass the course, even with the passing limit of 70%. A pie chart for the same has been shown in fig. 5. Fig. 3. “scatter” tool for learning on quizlet.com platform IV. RESULTS AND ANALYSIS As the learning and testing of group-A and group-B is conducted over same content but on different environments (i.e. non-gamified and gamified), their responses have been analyzed based on two factors: • • Number of candidates appearing for final test. Number of candidates passing the final test according to the passing limit. After successful completion of the experiment stated above, following results were found: Fig. 5. Percentage distribution of outcomes of gamified environment These results show that there is definitely an increase in the interest of the users if the learning platform is gamified. In 2014 IEEE International Conference on MOOC, Innovation and Technology in Education (MITE) 293 fact in this study, the percentage increase in the number of candidates who completed the task successfully was 28%. It can also be noted that the number of candidates who took up the challenge and gave the final test increased when the platform was gamified. In fact an increase of 14% was observed. After the tests when candidates were asked to complete a feedback survey based on the course, the users who got gamified platforms had more positive reviews. Whereas the reviews received by candidates who were allocated to the conventional learning method were negative. Fig. 6 summarizes the results of post test survey which was taken from the candidates who appeared for gamified environment. The motive of this survey was to know the personal experiences of using gamified platform. Parameters like Fun, Challenge and Improved learning were included in the survey. Out of the users who got gamified learning platform, 76% experienced fun during the course, 45% felt a challenge which inspired them to complete the course and 79% felt that gamified platform improved their learning abilities. Due to these reasons candidates were engaged in the course and had a higher success rate. On the other hand, candidates who got conventional learning method gave feedback that after a point of time the task became boring and tedious. This is why nongamified environment had lesser success rate. in large variety of MOOCs for high school students, as a part of “High School Initiatives”, has been noted. Further, as discussed in section III, an experimental setup was developed for analyzing the differences between Gamified and Non-Gamified platforms. Following paragraph summarizes the results of experimental setup as explained in section IV. • The increase in the candidates successfully completing the experimental task from non-gamified platform to gamified platform was found to be around 28%. • The reduction in failure cases for the stated experiment from non-gamified to gamified platform was around 14%. • The candidates who denied taking up the final test also decreased from 30% in non-gamified platform to 16% in gamified platform. This shows a decrease of 14%. • Also, in the survey taken from candidates who worked on gamified platform, 76% candidates had fun during the course, 45% candidates felt challenge and 79% candidates observed an improvement in their learning. Considering all the above results, it can be concluded that if the learning tool or platform is gamified, there is a noticeable increase in the number of users succeeding the learning process and a similar decrease is found in the candidates failing the learning process. Also, a significant decrease in the candidates who drop out of the course is observed if the learning platform is gamified. Although the primary factor of any learning platform is its course content, its presentation is also an important factor for successful retention of the users using that online learning platform. And a significant increase in the user engagement can be achieved if the learning platform is gamified, as shown in this paper. Fig. 6. Summary of post-test survey from candidates of gamified platform based on their personal experience. Similarly, if the MOOCs providers like edX, couresera, udacity, etc. are gamified to some extent, the user engagement can be increased. And hence the ultimate aim of these MOOCs platforms, i.e., imparting knowledge globally on a unified platform to as many people as possible, can be achieved successfully. Because learning with fun and challenge is always a positive and retentive learning. REFERENCES V. CONCLUSIONS It can be concluded from the case study of discipline wise distribution of the MOOCs available on the edX platform that, the number of courses for computer science related fields is maximum, i.e. about 11-15% of the total courses. And for regional distribution, American universities are the largest provider of MOOCs on edX.org (America constitutes around 60% of MOOCs on edX.org), although several Asian, European and Australian universities have also started deploying MOOCs on edX.org. A noticeable boom of MOOCs from Chinese universities has been observed. Also, an increase 294 [1] Volery, T., and Lord, D., “Critical Success Factors in Online Education”, International Journal of Educational Management,vol. 14, no.5, pp. 216-223, 2000 [2] Harasim, L., “Shift happens: Online Education as a New Paradigm in Learning”, The Internet and higher education, vol. 3 no. 1, PP. 41-61, 2000. [3] Allen, I. E., and Seaman, J., “Class Differences: Online Education in the United States”, Sloan Consortium, 2010. [4] McAuley, A., Stewart, B., Siemens, G., and Cormier, D, “The MOOC Model for Digital Practice”, SSHRC Knowledge Synthesis Grant on the Digital Economy, 2010. [5] “Canvas” https://www.canvas.net/. 2014 IEEE International Conference on MOOC, Innovation and Technology in Education (MITE) [6] [7] [8] [9] “ClassToGo” http://class2go.stanford.edu/. “Coursera” https://www.coursera.org/. “edX,” https://www.edx.org/ “National Programme on Technology Enhanced Learning (NPTEL) Live Online Courses,” http://nptel.iitm.ac.in/Onlinecourses/ [10] “Udacity” http://www.udacity.com/ [11] Cope, B., and Kalantzis, M. (Eds.), Ubiquitous learning, University of Illinois Press, 2009. [12] https://6002x.mitx.mit.edu/info. [13] https://courses.edx.org/courses/MITx/8.02x/2013_Spring/info [14] https://courses.edx.org/courses/UTAustinX/UT.1.01x/2013_Sep t/info [15] Deterding, S., Dixon, D., Khaled, R., and Nacke, L. From game design elements to gamefulness: defining gamification. Proc. Int. Academic MindTrek Conference (2011), 9–15. [16] Huotari, K., and Hamari, J. Defining gamification: a service marketing perspective. Proceeding of the 16th International Academic MindTrek Conference (2012), 17–22. [17] https://www.quizlet.com/. 2014 IEEE International Conference on MOOC, Innovation and Technology in Education (MITE) View publication stats 295