LEARNING AND ASSESSMENT BASED ON GAMIFIED E-COURSE IN MOODLE

Математика и информатика Volume 61, Number 5, 2018 Mathematics and Informatics Educational Matters Научно-методически статии LEARNING AND ASSESSMENT BASED ON GAMIFIED E-COURSE IN MOODLE Mariya Gachkova, Martin Takev, Elena Somova University “Paisii Hilendarski” – Plovdiv (Bulgaria) Abstract. The paper presents game-like elements suitable for the realization of a gamified e-course and their representation as components in the e-learning environment Moodle. An example of the design of a gamified e-learning course in Moodle is shown. The paper contains comparison charts for the assessment of the course participants. Feedback of the effectiveness of certain aspects of the gamified e-course is collected by a survey. Keywords: gamification; e-learning course design; e-learning I. Introduction In accordance with the development of technology and the evolution of the learning environment, the needs of modern learners have changed. Educators often use games and game-like environments (Gachkova & Somova, 2016) to attract learner’s attention during the instructional process (Kocadere & Caglar, 2018). Gamification, the use of game elements in non-game contexts, has become a popular technique to enhance instructional outcomes in both education and organizational learning, and many recommendations have appeared regarding such learning environments1) (Kapp, 2012), (Landers & Callan, 2012). Gamification of learning is an educational approach to motivate students to learn by using video game design and game elements in learning environments (Kapp, 2012). Gamification, broadly defined, is the process of defining the elements, comprising games, make those games fun and motivate the players to play them continually, and then using those same elements in a non-game context to influence behavior (Deterding, Dixon & Khaled, 2011). Gamification can be provided in two variants (Kapp, 2012): structural gamification (without any changes to the learning content) and content gamification (consisting of altering the learning content to make it more game-like). Some Learning Management Systems are using methods and ideas from games and propose gamification, like GENIE, TalentLMS, Moodle, Frog and Expertus One. For example, GENIE2 implements the following game-like elements: rewarding with badges and points for achieving learning goals, preparation of lead444 Learning and Assessment... erboards for stimulating the competitive spirit, putting deadlines to tasks, and learning through gradually passing through levels. The cloud-based platform for e-learning TalentLMS4) supports gamification through: giving points for performed actions (going through learning resources for example), collecting various badges for passed tests, receiving certificates and awards for finishing a course, re-certificate on certain time periods, leaderboards – displayed by charts and diagrams and passing the course by levels. Although Moodle is not a platform designed for gamification, it has most of the popular game-like elements, which were explained in (Somova & Gachkova, 2016) and we will use in the current study. The aim of the paper is to make a research on learner’s opinion about gamified approach in learning and the obtained impact in learning quality and efficiency, according to students. For this purpose, a gamified e-course has to be created. Section 2 presents the used methodology approach in this case study. Section 3 shows the design of our e-course, including game-like elements and assignments. Participants in the course and their user types are determined in Section 4. Analysis of the of assignments’ assessment is presented in Section 5. A COLLES survey with participants and an analysis of the survey results are given in Section 6. The paper ends with a conclusion summarizing the contributions of the authors as well as with some ideas for future works. II. Methodological Approach The study is designed as a case study based on structural gamification (Kapp, 2012) of a standard learning environment. (Kapp, 2012) describes structural gamification as the application of game elements to propel a learner through content with no changes to the content itself. The content does not become gamelike, only the structure around the content. The primary focus behind this type of gamification is to motivate the learners to go through the content and to engage them in the process of learning through rewards (for example: gaining points for watching a video or completing an assignment). Structural gamification in our study is achieved by including some gamebased elements in the e-course sources, that are explained in the next section. Participants use the gamified learning environment for ten weeks. Based on weekly assignments, the participants are evaluated. The research process consists of 4 phases: – The design of the gamified e-course in e-learning environment; – Determination of participant types in the case study; – Analysis of the assessment of assignments included in the gamified e-course; – Analysis of the answers collected by COLLES Survey included in the gamified e-course. 445 Mariya Gachkova, Martin Takev, Elena Somova III. The Design of Gamified e-Learning Course A gamified e-learning course is designed for use in this study. The course supports the discipline “Modeling and management of business processes” in the bachelor’s degree program “Business information technologies” at University of Plovdiv “Paisii Hilendarski”, Bulgaria. The e-course is realized in the learning management system Moodle3). In Table 1 gamification elements are presented as Moodle components. The e-learning course is arranged so that each learning week corresponds to one level (totally 10 levels). The course uses 4 levels with entry requirements and rules. Levels include quests (assignments), suitable to the current week’s subject. There are various quests, such as preparing a project, building a wiki, taking a test or reading an article. Some of the assignments are set as individual tasks and others have to be finished as a team work. During the learning process, learners are awarded points for completing assignments. Only those with a certain score are able to pass onto the next level. The learners without sufficient score to unlock the next content, could re-send corrected assignments a second time (game-based learning model uses game cycle by (Garris, Ahlers & Driskell, 2002)). 8 assignments are set in the current gamified e-course (see Table 2), and 7 of them need assessment. 7 of the assignments are individual (marked with I in Table 2) and 1 is a group one (marked with G in Table 2). Levels 2, 4, 8 and 9 have entry requirements – a set of nested restrictions to apply complex logic, every level can start only if the following 3 conditions are met: – The previous assignment(s) is finished; – The previous assignment(s) is finished with the minimum required points in assessment (e.g. equal or greater than 30%); – The deadline is met (timelines are set). Table 1. Gamification elements used during the design of gamified e-course in Moodle Gamification element Level Quest Badge Leader board Bonus Reward Combo 446 Moodle realization Section/topic object in e-course (different activities are required to unlock the levels) Individual or Group assignment Moodle custom badge Visible report with the assignment’s points and ranks Points for finishing an assignment Additional recourses with interesting facts, etc., unexpectedly received after completing some work in the course Additional instructions/help for the next assignment or for reaching special requirements Learning and Assessment... Game Rules Story/history Game progress Status Team Time frame Learning process rules Description of the learning resources and activities Course progress Different character is attached to each level (section) Moodle Group Time restrictions for tests and assignments Out of the levels (marked with level 0 in Table 2), the participants have a group assignment to build a wiki resource. In the last level, learners have to answer the questions in a COLLES Survey. Table 2. Gamified e-course components (by levels) Level Entry Requirements Assignment (I/G) Forum Badges (B1/B2/B3) Point Bonuses Combos Rewards 0 – 1 – 2 yes 3 – 4 yes 5 – 6 – 7 – 8 yes 9 yes 10 – G yes B3 I yes B1, B2 yes yes yes I yes B1, B2 yes yes yes – yes – I yes B1, B2 yes yes yes I yes B1, B2 yes yes yes I yes B1, B2 yes yes yes – yes – I yes B1, B2 yes – yes – yes – I yes – – – – – – yes yes – yes – – – – – – The gamified e-course is designed to give badges for a set of requirements. On Table 3 the used three types of badges (B1/B2/B3) are presented, and each one of them has different requirements. There are two methods to achieve the badges in the course – automatically, when the user meets some certain criteria and manually – by the teacher, who can decide to give the badge to the student. Students can receive different prizes: bonuses like points for finishing an assignment, special combos like additional instructions and help for the next assignment, or unexpected rewards like additional recourses with interesting facts, etc. To provoke competitive spirit, learners’ scores (ordered from highest to lowest) are placed on a leader board, visible for all learners, with the points they earn throughout the course. As a social component, students have forums in each level where they can share their accomplishments with other learners, share how they have achieved them or discuss the problems in quests. 447 Mariya Gachkova, Martin Takev, Elena Somova Table 3. Types of badges set in gamified e-course Type of Badge First Finished (B1) Description Achievement Type The learner should be the first that finished Automated and sent the assignment. Best result (B2) The learner should have more than 80% from Automated the maximum points. Best group work (B3) The team should have the best result in the Manual assignment. Each learner from the awarded team receives the badge. Students receive different character (status) on each level (for example: Worker – level 1, Team Leader – level 2, …, President – level 10). Some story/history is added in the description of the learning resources and activities. For example, the story of assignment 2 is: To obtain the higher position “Team Leader” in the company “Progress Ltd” you have to model a diagram for the business process “Purchasing a property” with following activities and requirements: …. At any time, students can see their course progress (current position in the course, to where are checked passed resources and activities by teacher or students themselves). All the students, who have started with the gamified e-course and did not pass some of the levels, are left to continue with the standard e-course without gamified elements. IV. Participants and User Types 1. Participants in the case study For the analysis of the results and statistics, data is obtained from 113 undergraduate students, enrolled in the education program “Business information technologies” at the University of Plovdiv “Paisii Hilendarski”, Faculty of Mathematics and Informatics. Opportunity to choose between standard and gamified e-learning environment is given to the students. A total of 41 students have preferred to participate in the designed gamified e-course and 27 of them have completed the full list of assignments. There are three reasons for the remaining 14 participants to discontinue the gamified e-course: – The participants did not meet the time criteria – the time limits are set for 4 assignments in the e-course; – The participants did not meet the points criteria – for some of the assignments the participants could not meet the target for minimum points required; – The participants prefer to continue learning in a standard e-course. 2. Determination of user types One of the theoretical models outside the field of gamification suggests that the use of new technologies in training, including gamification, may lead to poor in448 Learning and Assessment... structional outcomes when people are not comfortable and experienced with them (Landers & Callan, 2012). (Landers & Armstrong, 2017) concludes that the learner’s lack of experience with virtual worlds and poor attitude towards the value of virtual worlds for training would minimize any benefit from using virtual worlds. Based on these studies, at the beginning of the course students have the ability to choose between gamified or standard e-learning environment. 41 of 113 students have preferred to participate in the gamified variant of the course. (Bartle, 1996) examined players expectations and produced the first effort at player categorization, classifying players using their act/interact preferences and orientation (world/player). He considers 4 categories (killer, achiever, explorer and socializer) described in Table 4. Table 4. Game elements according to the user type User type Killer Achiever Explorer Socializer Definition (Bartle, 1996) Game elements Tries to dominate other players by acting Leader board, Points, Course on the people in the environment progress, Status, Combos Acts in the world and care about the Badges, Levels, Course assignments in the environment in order progress, Bonuses to win Interacts with the world. They want to Quests (Individual explore the environment and discover as assignments), Rewards, many new things as possible History Interacts with the people in an environment Forums, Quests (Group and usually takes advantage of the Assignments), History communication function to socialize According to (Kocadere & Caglar, 2018), learning environments should be designed to incorporate a variety of elements so that each player type is able to encounter those that attract them. We suggest some appropriate game elements in Table 4 to attract each learning (player) type. V. Analysis of the Assessment of Assignments The main assessment of the students in the gamified e-course is done by the first six assignments in levels 1, 2, 4, 5, 6 and 8. At the end all students (from both variants of the course) fill in a test. The total number of students that passed the exam (in their first attempt) is 59, where 27 of them have finished the gamified course and 32 – the standard e-course. Fig. 1 shows the extended chart with average students’ scores (in points, where the maximal points per assignment are 100) for each assignment and the count of participating students. 449 Mariya Gachkova, Martin Takev, Elena Somova Figure 1. Average students’ scores vs. count of participating students Fig. 2 shows the average scores of only those students, who have successfully completed the whole gamified course. It is obvious that the lower scores have been achieved for the first three assignments. We think this has mainly happened, because the previous students’ experience with gamification is poor or is missing. Figure 2. Average assessment of students that entirely completed the gamified e-course (by levels) This is also one of the reasons that many students discontinue the gamified course. The other reasons are lack of time to follow time requirements, preferences for easier learning process without many activities (regardless of the learning quality), not attracted of the game approach, etc. The percentages of discontinued students are as follows: after level 1 – 50%, level 2 – 29%, level 4 – 0%, level 5 – 14%, level 6 – 7% and level 8 – 0%. Thus, most of the participants (79%) discontinue the course after the first two assignments. 450 Learning and Assessment... VI. Analysis of the Colles Survey The gamified e-course is designed including COLLES (The Constructive On-Line Learning Environment Survey)5). The survey provides feedback on the effectiveness of certain aspects of online learning. By definition COLLES survey is carried out twice with the same questions; the first time, in the beginning of the course – to find out what the student wants (so called “Preferred”) and the second time, at the end of the learning – to find out what actually happened (so called “Actual”). Current survey is divided into six categories: Practical importance, Self-criticism, Dialogic, Teaching help, Mutual assistance and Understandability. Students can choose from five-points Likert scale to answer the questions in COLLES survey. Possible values in the scale are scored as follows: 1 – Almost Never, 2 – Seldom, 3 – Sometimes, 4 – Often, 5 – Almost Always. Figure 3 shows summarized results, divided into the six categories of the conducted COLLES survey. According to students, the achieved “Teaching help” is higher than expected. In four categories (” Practical importance”, “Self-criticism”, “Dialogic” and “Mutual assistance”), students’ “preferred” answer is scored higher than the “actual” answer. But we have to take into account that the maximum difference calculated between both answers for “Practical importance” category is 0,91 (total of 5). In category “Understandability” of the gamified course, the “actual” average score is equal to the “preferred” score of the involved students. This means that the participants are satisfied with the understandability of the course. Figure 4 presents in details the results of COLLES survey for the category “Understandability”. The students’ opinions are shown in 4 categories: “understanding other students”, “other students understand me”, “understanding teacher” and “teacher understands me”. Figure 3. COLLES survey results for gamified e-course by categories 451 Mariya Gachkova, Martin Takev, Elena Somova The average results for “Actual” and “Preferred” answers in each category is more than 3 (Sometimes) total of 5. Presented as a percentage, average “Understandability” is more than 75%. Figure 4. COLLES survey results for the category “Understandability” of the gamified e-course Students give higher assessment to “Preferred” according to “Actual” state for the categories “teacher understands me” and “understanding other students” and lower assessment to others. It is obvious that students have obtained more understanding from teachers in the learning process than they expected. The same situation is with “other students understand me”. Something more, these values strongly incline to the answer “Almost Always”. VII. Conclusions The presented gamified e-learning course is proposed together with a standard e-learning course to give students options to choose the more appropriate environment for their learning. In the game-like environment there are 4 types of learners (players): killer, achiever, explorer and socializer, which are attracted to different activities and environment components. Our gamified learning course is intended for all learning (player) types, because the course design incorporates a variety of elements, so that each learning type is able to encounter those that attract them. In conclusion, the current study is thought to provide further clarity about the result of designing and applying gamified learning courses. Due to the given analysis in Section 5 and 6, the results of this study can be summarized in 4 main points: – Students, that participate in the gamified e-course, spend more time working on assignments in the learning environment, than the students who preferred the standard e-course; 452 Learning and Assessment... – Students in the gamified e-course have higher grades than the students, attending the standard e-course; – Students in the gamified e-course are searching for less explanations than students in the standard e-course; – The average understandability of students in the gamified course is equal to their preferences and higher than 75%. Our future work in this field continues with design and realization of a Moodle module for gamified learning, which will make the realization of a gamified course in Moodle with all game-like elements (some of them now missing) easier. NOTES 1. Brousell, L. (2013). How gamification reshapes corporate training. http://www. cio.com/article/728268/How_Gamification_Reshapes_Corporate_Training. Accessed 09.07.2018 2. GENIE. http://www.growthengineering.co.uk/genie-content-authoring-tool/. Accessed 09.07.2018 3. Moodle. https://moodle.org. Accessed 09.07.2018 4. TalentLMS. http://www.talentlms.com/. Accessed 09.07.2018 5. The Constructivist On-Line Learning Environment Survey (COLLES). https:// surveylearning.moodle.com/colles/. Accessed 09.07.2018 REFERENCES Bartle, R. (1996). Hearts, clubs, diamonds, spades: Players who suit MUDs, Journal of MUD research, 1(1), 19 – 58. Deterding, S., Dixon, D. & Khaled, R. (2011). From game design elements to gamefulness: defining ‘gamification’, 15th International MindTrek Conference. New York: ACM, 2011, 9 – 15. Gachkova, M. & Somova, E. (2016). Game-based approach in E-learning, IX National Conference „Education and Researches in Information Society”, 26 – 27.05.2016, Plovdiv, 2016. Garris, R., Ahlers, R. & Driskell, J. E. (2002). Games, motivation, and learning: A research and practice model. Simulation & Gaming, 33(4), 441 – 467. Kapp, K. M. (2012). The gamification of learning and instruction: Gamebased methods for trainig and education. San Francisco: Pfeiffer. Kocadere, S. A. & Caglar, S. (2018). Gamification from player type perspective: A case study, Educational Technology & Society, (21) 3. Landers, R. N. & Armstrong, M. B. (2017). Enhancing instructional outcomes with gamification: An empirical test of the Technology453 Mariya Gachkova, Martin Takev, Elena Somova Enhanced Training Effectiveness Model, Computers in Human Behavior, Vol. 71, June 2017, 499 – 507. Landers, R. N. & Callan, R. C. (2012). Training evaluation in virtual world: Development of a model, Journal for Virtual Worlds. Research, 5(3), 1 – 20. Somova, E. & Gachkova, M. (2016). An Attempt for Gamification of Learning in Moodle, International Conference on e-Learning (eLearning’16), 08 – 09 September 2016, Bratislava, Slovakia, ISSN: 2367-6787 (online), ISSN: 2367-6698 (print), ISSN: 2367-6701 (cdrom), 2016, 201 – 207. ОБУЧЕНИЕ И ОЦЕНЯВАНЕ НА ОСНОВАТА НА ИГРОВИЗИРАН Е-КУРС В MOODLE Резюме. Статията представя игровите елементи, подходящи за реализиране на игровизиран е-курс, и тяхното представяне като компоненти в средата за е-обучение Moodle. Показан е пример за дизайн на игровизиран е-курс в Moodle. Статията съдържа диаграми за сравняване на оценяването на участниците в курса. Чрез проучване е събрана обратна информация за ефективността на определени аспекти от игровизирания е-курс.  Ms. Mariya Gachkova, PhD student Faculty of Mathematics and Informatics University of Plovdiv “Paisii Hilendarski” 236, Bulgaria Blvd. 4003 Plovdiv, Bulgaria E-mail: mariq.gachkova@gmail.com  Mr. Martin Takev, PhD student Faculty of Mathematics and Informatics University of Plovdiv “Paisii Hilendarski” 236, Bulgaria Blvd. 4003 Plovdiv, Bulgaria E-mail: takevm@gmail.com  Dr. Elena Somova, Assoc. Prof. Faculty of Mathematics and Informatics Plovdiv University “Paisii Hilendarski” 236, Bulgaria Blvd. 4003 Plovdiv, Bulgaria E-mail: eledel@uni-plovdiv.bg 454