Face-to-Face Peer Assessment in Secondary Education: Does Anonymity Matter?

Available online at www.sciencedirect.com Procedia - Social and Behavioral Sciences 69 (2012) 1340 – 1347 International Conference on Education and Educational Psychology (ICEEPSY 2012) Face-to-face peer assessment in secondary education: does anonymity matter? Ellen Vanderhovena, Annelies Raesa, Tammy Schellensa , Hannelore Montrieuxa* a Ghent University, Henri Dunantlaan 2, Ghent 9000, Belgium Abstract Pupils involved in peer assessment often state that they do not feel entirely comfortable with publicly evaluating their peers. It is found that peer-pressure might cause stress and a lack of accuracy of the assessment (Falchikov, 2003). Based on social impact theories (e.g. Latané, 1981), anonymity within peer assessment can be considered as a solution. Yet, although the benefits of anonymity are previously investigated in the context of classroom voting and debating in higher education (Ainsworth et al., 2011), no research is found regarding anonymity in the context of peer assessment in a face-to-face secondary education setting. To fill this gap, the present study was set up to investigate whether anonymity of the assessors in peer assessment can counter undesirable social effects. It was hypothesized that anonymous modes of peer assessment will induce a reduced perception of peer pressure, a reduced fear of failure, and more positive attitudes towards peer assessment. Classroom response technology (CRT) was introduced as a tool that enables anonymity within face-to-face settings (Kay & Knaack, 2009). A quasi-experimental study was set up in four secondary classes in Belgium (n=69). In all classes, pupils had to assess each others’ group presentations on different criteria in a face-to-face classroom setting. In the control group (2 classes) a traditional peer assessment approach was used, i.e. raising score cards-, while in the experimental group (2 classes) CRT was used to give scores anonymously. In the latter, score distributions were presented on a screen in front of the class. Feelings of peer-pressure, fear of failure towards the other, and pupils’ general attitudes towards peer assessment were measured using a post-questionnaire. It was found that the pupils who used CRT as a tool to give scores anonymously, felt less peer pressure and fear of failure than those in the classic peer assessment condition. They also reported more positive attitudes towards this kind of evaluation. Implications of these results are discussed. © 2012 Elsevier Ltd.Ltd. Selection and/or peer-review under responsibility of Dr. Zafer Bekirogullari © 2012Published Publishedbyby Elsevier Selection and/or peer-review under responsibility of Dr. Zafer Bekirogullari of Cognitive – Counselling, Research & Conference Services C-crcs.of Cognitive – Counselling, Research & Conference Services C-crcs. Keywords: Peer assessment; secondary education; Classroom response technology; social effects; anonimity 1. Introduction It is generally accepted by learning scientists that assessment is an important drive of the learning process (Hunt, Hughes, & Rowe, 2002) and that changes in educational approaches often require new forms of assessment (Dochy, Heylen, & Van de Mosselaar, 2002). Therefore, in the context of an emerging learning approach infused with social 1877-0428 © 2012 Published by Elsevier Ltd. Selection and/or peer-review under responsibility of Dr. Zafer Bekirogullari of Cognitive – Counselling, Research & Conference Services C-crcs. doi:10.1016/j.sbspro.2012.12.071 Ellen Vanderhoven et al. / Procedia - Social and Behavioral Sciences 69 (2012) 1340 – 1347 constructivism, different modes of peer assessment have become increasingly popular in higher education. Peer assessment is defined by Topping (1998, 2003) as an educational arrangement for learners to consider and specify the level, value or quality of a product or performance of other equal-status learners, by providing the peer with oral and/or written feedback. This way, it presumes students’ active involvement in the assessment process (Kollar & Fisher, 2010). This kind of evaluation has proven to be accurate, with high correlations between the ratings of peers and those of teachers (Dochy, Segers, & Sluijsmans, 1999), when certain conditions are met, such as the presence of unambiguous criteria on which to evaluate (Falchikov & Goldfinch, 2000) and a necessary training in peerassessment (Sluijsmans, Brand-Gruwel, van Merriënboer, & Martens, 2004; Van Zundert, Sluijsmans, van Merriënboer, 2010). Moreover, research has indicated that peer assessment leads to higher quality performances, as a consequence of better understanding the assessment criteria by playing the role of assessor (Li, Liu, & Steckelberg, 2010; Smith, Cooper, & Lancaster, 2002; Topping 2003). However, since peer assessment is fundamentally a collaborative activity that occurs between at least two peers (Kollar & Fischer, 2010; van Gennip, Segers, & Tillema, 2010), there is evidence that the development of, and the interplay between interpersonal variables can affect the outcomes of this activity (Van den Bossche, Gijselaers, Segers, & Kirschner, 2006). More specific, interpersonal variables which are relevant in peer assessment practices are peer pressure due to friendship bonds, enmity or other power processes and the social discomfort about being critical (Cartney, 2010; Stepanyan, Mather, Jones, & Lusuardi, 2009; Topping, 2003). It has also been found that students experience more stress, because they don’t feel entirely comfortable with publicly evaluating their peers (Pope, 2005; Stepanyan et al., 2009). Indeed, according to social impact theory (Latané, 1981), social influence, e.g. peer pressure, will increase with the immediacy of its members (their proximity in space or time), as is the case in a face-to-face classroom setting. Yet, Stepanyan and colleagues (2009) pointed out that the allocation of marks and inclass activities are important in encouraging student involvement. Despite the various indications that these interpersonal variables might play a significant role in causing a lack of accuracy, reliability and acceptance of peer assessment (Falchikov, 2003; Sung, Chang, Chang & Yu, 2010), the question how we can decrease these undesirable social effects has to date hardly been studied. It might be expected that when peers perceive the learning environment as safe for interpersonal risk-taking they will be less prone to peer pressure and will put effort into obtaining a fair peer assessment process (van Gennip et al., 2010). This safe environment can be linked with the possibility for learners to make their views known in a way that does not reveal their identity (Ainsworth et al., 2011). For example, Deutsch & Gerard (1955) showed in an experiment that normative social influence upon the individual judgment is reduced, when the judgment can be given anonymous. They define normative social influence as an influence to conform with the positive expectations of another. In the context of peer assessment, the peers can be interpreted as the others, and peer pressure as a form of normative social influence. Following the theory of normative influence, we might therefore hypothesize that when the pupils cannot be identified and thereby do not perceive any pressure from others to conform, peer assessment will be more accurate and will be evaluated more positive. Similar predictions can be derived from social impact theory (Latané, 1981), which states that identifiability, as opposed to anonymity, would be expected to facilitate social influence. Moreover, researchers suggest that anonymity within peer assessment is one of the several factors that encourage student participation (Ballantyne, Hughes, & Mylonas, 2002; Vickerman, 2009; Yang & Tsai, 2010). However, Ainsworth et al. already mentioned that the (limited) studies conducted in educational contexts with attention to anonymity have typically focused on online learning and have employed a case study method. Anonymous assessment is much more difficult to orchestrate within a face-to-face co-present classroom setting. In this respect, innovative technology that can provide immediate anonymous feedback even within face-to-face settings can be put forth. Classroom response technology (CRT), e.g. the electronic voting system TurningPoint, may provide a solution to the given objections and counter some undesirable social effects. A classroom response system is a system used in a face-to-face setting to poll students by means of individual infrared handset transmitters. The aggregated totals of votes are subsequently displayed on the screen as immediate feedback. An extensive review of literature done by Kay and LeSage (2009) shows that CRT has been proven to be an effective educational tool when used during classroom practices: students attended more classes, paid more attention, were more engaged and there seemed to be a positive effect on learning performance. Moreover, it has been found that students typically report that it is the anonymous nature of the response that encourages them to participate (Draper & Brown, 2004). Using CRT, students can score their peers anonymously and the assessees can immediately get the visual feedback by means of the aggregated totals shown as a graph on a screen. Raes, Vanderhoven & Schellens 1341 1342 Ellen Vanderhoven et al. / Procedia - Social and Behavioral Sciences 69 (2012) 1340 – 1347 (submitted) found that in higher education, CRT is a valuable and valid tool for peer assessment. It was found that students expressed positive attitudes and they felt more comfortable than in non-anonymous modes of peer assessment. Concerning peer pressure, however, no positive effects were found in favour of the anonymous peer assessment using CRT. However, the students in this study experienced little or no peer pressure in general. This might be caused by the nature of class groups in higher education. Classroom variables might have an influence on the level of peer pressure students experience, e.g. how students feel in their class group (classroom culture). Deutsch and Gerard (1955) indeed found that normative social influence upon individual judgments is greater among individuals forming a group, than among an aggregation of individuals who do not compose a group. It might therefore be expected that higher levels of peer pressure would be observed in secondary education, where pupils are part of a class group more than they are in higher education. Moreover, Sumter, Bokhorst, Steinberg and Westenberg (2009) found that the resistance to peer influence increases over age during adolescence and Steinberg and Monahan (2007) found that this resistance reaches adult levels around the age of 18. These findings increase the importance of reducing undesirable social effects during peer assessment in secondary schools. There is however no research that investigates the influence of anonymity during peer assessment in a face-to-face classroom setting in secondary education. To counter these shortcomings, a quasi-experimental study was conducted in which traditional non-anonymous peer assessment was compared with anonymous peer assessment using CRT in a secondary classroom setting with regard to different social effects. The following research questions drove this study: RQ1: Do pupils experience peer assessment using CRT as an anonymous form of peer assessment, as compared to raising score cards? RQ2: Do pupils feel more positive towards peer assessment using CRT, as compared to raising cards? RQ3: Do pupils experience less undesirable social effects, such as peer pressure and fear of failure towards others, when using CRT for peer assessment, as compared to raising cards? 2. Method 2.1 Participants and setting Out of four different classes, 69 pupils participated in this study. Classes were selected out of grade 9 and 10 of secondary school, including pupils aged 15-16 years old. 72% of them were girls, 28% were boys. All pupils received a group assignment to prepare and give a presentation about a specific topic. Groups consisted of two or three students. 2.2 Procedure As depicted in Figure 1, the whole procedure existed of different steps. In advance of the presentations, pupils got an intensive training which is seen as a necessity to guarantee successful peer assessment (Dochy et al., 1999; Tsivitanidou et al., 2011). The first part of the training consisted of a theoretical background about the advantages and disadvantages of peer assessment. Secondly, students were assigned to formulate criteria to assess each other. Final rubrics were then given to the pupils, and comprised descriptions of observable behaviour for every possible score on a certain criterion (1-5). Using this procedure, the criteria were transparent and clear for all pupils, increasing the chances of a reliable and valid assessment (Dochy et al., 1999). Following, in the last part of the training, students got the chance to practice these rubrics. After the training, pupils had to give their presentations. Every presentation was followed by a peer assessment procedure. The mode of peer assessment differed between groups. Two classes were assigned to the anonymous CRT-condition. Pupils in these classes had to score their peers for every criterion on the rubric, using the CRTtransmitters. Summarized results of this scoring became visible immediately after everyone judged, on a screen in front of the class. No one knew what scores were given by whom. The two other classes were assigned to the nonanonymous traditional peer assessment condition. In these classes, pupils had to give their score by raising a scorecard. As it has been proven to be valuable (Hattie, 2003; Raes et al., submitted), there was an additional oral feedback moment based on the given scores that was moderated by the teacher. The aim was to provide each group with some general strengths and weaknesses to take into account in the future. Finally, after all presentations and assessment, pupils had to fill out a survey in which different variables were questioned. 6LOLQPLú Ellen Vanderhoven et al. / Procedia - Social and Behavioral Sciences 69 (2012) 1340 – 1347 WĞĞƌ 'ƌŽƵƉ WĞĞƌĂƐƐĞƐƐŵĞŶƚ ĂƐƐĞƐƐŵĞŶƚ ƉƌĞƐĞŶƚĂƚŝŽŶƐ džƉĞƌŝŵĞŶƚĂůĐŽŶĚŝƚŝŽŶ͗WĞĞƌĂƐƐĞƐƐŵĞŶƚƵƐŝŶŐZd ƚƌĂŝŶŝŶŐ  WŽƐƚͲƐƵƌǀĞLJ ŵĞĂƐƵƌŝŶŐ͗ Ͳ ŶŽŶLJŵŝƚLJ Ͳ WŽƐŝƚŝǀĞ ĂƚƚŝƚƵĚĞƐ Ͳ WĞĞƌƉƌĞƐƐƵƌĞ Ͳ &ĂŝƌŽĨĨĂŝůƵƌĞ ŽŶƚƌŽůĐŽŶĚŝƚŝŽŶ͗ƚƌĂĚŝƚŝŽŶĂůƉĞĞƌĂƐƐĞƐƐŵĞŶƚ  Figure 1. The procedure of the study. 2.3 Measures After the peer assessment procedure, pupils had to fill out a survey, which contained items measuring different variables. All items were measured on a 5-point Likert scale (1= totally disagree, 5= totally agree). First of all, we measured the experienced anonymity of the peer assessment form and their overall attitudes towards peer assessment as they experienced it, with scales partly based on previous research (Cheng & Warren, 1997; Sluijsmans, 2002; Sluijsmans et al., 2004). To measure the undesirable social effects that might accompany the peer assessment procedure, items were added that measured general experienced peer pressure (Raes et al., submitted), and experienced fear of failure associated with a negative evaluation (fear of losing the interest of important others, and of upsetting important others, PFAI, Conroy, 2002) during the peer assessment process. The scales are presented in Table 1 together with an example item for each scale and the corresponding Cronbach’s α reliability coefficients. The reliability of all scales was satisfactory. Table 1. Reliabillity of different scales. Scale Example item Cronbach’s α Anonymity “During the peer assessment procedure, I could give scores anonymously” 0,88 Positive attitudes “It is nice to score my peers during peer assessment” 0,90 Experienced peer pressure “During the peer assessment procedure, my score was influenced by my peers” 0,79 Experienced fear of failure “By giving low scores to my peers, I fear that the others would have less appreciation for me” 0,84 1343 1344 Ellen Vanderhoven et al. / Procedia - Social and Behavioral Sciences 69 (2012) 1340 – 1347 3. Results In what follows, the results with regard to the three research questions are described. These results are displayed in Figure 2. Figure 2. Means in both modes of peer assessment on the different measured dependent variables. 3.1 RQ1: Anonymity To find an answer to our first research question, an ANOVA analysis was conducted, with the mode of peer assessment as a factor, and the anonymity-scale as a dependent variable. As expected, it was found that the condition in which pupils had to give scores to their peers using CRT was experienced more anonymous than the condition in which pupils had to give their scores using cards (F(1,63)=62,51, p<.001). 3.2 RQ2: Positive attitudes To find an answer to our second research question, an ANOVA analysis was conducted, with the form of peer assessment as a factor and the attitude-scale as a dependent variable. As expected, it was found that pupils feel more positive about peer assessment when they use CRT, than when they have to raise cards (F(1,63)=4.51, p<.05). 3.3 RQ3: Undesirable social effects To find an answer to our third research question, a MANOVA analysis was conducted, with the form of peer assessment as a factor and the two different scales that measure undesirable effects as dependent variables, i.e. experienced peer pressure and fear of failure towards others. The MANOVA analysis showed a significant effect of the peer assessment mode (F(2,62)=4,75, p<.05). Univariate analysis showed that pupils experienced less peer pressure (F(1,63)=5,31, p<.05) and less fear of failure when giving lower scores (F(1,63)=6,91, p<.05) in the anonymous CRT condition, than they did in the non-anonymous card-raising condition. It was also found that lower scores were given by pupils in the anonymous condition than in the non-anonymous condition (F(1,67)=9,89, p<.05), while there was no difference in the scores given by the teacher (F(1,67)=2,84, p>.05). 4. Conclusion & Discussion As could be expected out of different social influence theories (Deutsch & Gerard, 1955; Latané, 1981), a lack of identifiability of the assessors decreases the negative social effects that are typically associated with peer assessment procedures. Pupils in secondary education report to experience less peer pressure and less fear of failure when giving low scores, when they can give their scores anonymously. Moreover, they like the anonymous CRT- 1345 Ellen Vanderhoven et al. / Procedia - Social and Behavioral Sciences 69 (2012) 1340 – 1347 procedure better than the non-anonymous traditional way of scoring. Therefore, we can conclude that CRT is a good solution to enable anonymous peer assessment in a face-to-face classroom setting. Moreover, as expected, this anonymous peer assessment is preferred over non-anonymous assessment, due to the positive effects on attitudes and undesirable social effects. These results confirm the results found by Raes et al. (submitted) about anonymous peer assessment in higher education. Moreover, these results extent their results, and confirm the hypothesis that anonymous peer assessment is even more important in secondary education, to reduce peer pressure. Our results also suggest that the peer assessment procedure will be more accurate when given anonymous, due to a reduced fear of failure when giving low scores. This might be the reason why it was also found that lower scores were given in the anonymous condition. Further research on the effects of anonymity on accuracy of the assessment in secondary education should clarify these findings. We should however point out some limitations of this study. First of all, only four classes participated. As depicted in Figure 2, it can be seen that general levels of peer pressure and fear of failure are still rather low. This might be due to specific class-group characteristics. As Deutsch and Gerard (1955) stated, individuals that form close groups are more susceptible of peer pressure and social influences than individuals not forming a close group. Therefore, a large scale study is necessary to confirm and deepen these results, taking into account different class cultures. Secondly, the manipulation of our peer assessment modes, was a between-subject manipulation. Therefore, no direct comparison could be made between the two conditions, by the same student. It might be interesting to replicate this study with a within-subject manipulation. That way, students can easily point out their preferred way of assessment. To conclude, we might state that anonymous peer assessment using CRT is a valuable method, with various positive effects. Therefore, more research about the details of the implementation, the accuracy and the optimal conditions of the context, are invaluable. 5. References Ainsworth, S., Gelmini-Hornsby, G., Threapleton, K., Crook, C., O’Malley, C., & Buda, M. (2011). Anonymity in classroom voting and debating. Learning and Instruction, 21, 365-378. Ballantyne, R., Hughes, K., & Mylonas, A. (2002). Developing Procedures for Implementing Peer Assessment in Large Classes Using an Action Research Process. Assessment and Evaluation in Higher Education, 27(5), pp. 427-441. Cartney, P. (2010). Exploring the use of peer assessment as a vehicle for closing the gap between feedback given and feedback used. Assessment & Evaluation in Higher Education, 35(5), 551-564. doi: Pii 926292288 Cheng, W. N., & Warren, M. (1997). Having second thoughts: Student perceptions before and after a peer assessment exercise. Studies in Higher Education, 22(2), 233-239. Conroy, D.E. (2002). The Performance Failure Appraisal Inventory. Retrieved.April.21,2011,..from http://www.personal.psu.edu/dec9/uploads/3/0/4/0/304067/2003_pfai_users_manual.pdf User’s Manual. Deutsch, M., & Gerard, H.B. (1995). A study of normative and informational social influences upon individual judgement. Journal of Personality and Social Psychology, 51, 629-636. Dochy, F., Heylen, L., & Van de Mosselaer, H. (2002). Assessment in onderwijs. Utrecht: Lemma. Dochy, F., Segers, M., & Sluijsmans, D. (1999). The Use of Self-, Peer and Co-assessment in Higher Education: a review. Studies in Higher Education, 24(3), 331. Draper, S. W., & Brown, M. I. (2004). Increasing interactivity in lectures using an electronic voting system. Journal of Computer Assisted Learning, 20(2), 81-94. Falchikov, N. (2003). Involving students in assessment. Psychology Learning and Teaching, 3(2), 102-108. 1346 Ellen Vanderhoven et al. / Procedia - Social and Behavioral Sciences 69 (2012) 1340 – 1347 Falchikov, N., & Goldfinch, J. (2000). Student Peer Assessment in Higher Education: A Meta-Analysis Comparing Peer and Teacher Marks. Review of Educational Research, 70(3), 287-322. Hattie, J. (2003). Formative and summative interpretations of assessment information. Retrieved September 28, 2011, from http://www.education.auckland.ac.nz/webdav/site/education/shared/hattie/docs/formative-andsummative-assessment-%282003%29.pdf Hunt, N., Hughes, J., & Rowe, G. (2002). Formative Automated Computer Testing (FACT). British Journal of Educational Technology, 33(5), 525-535. Kay, R., & Knaack, L. (2009). Exploring the use of audience response systems in secondary school science classrooms. Journal of Science Education and Technology, 18(5), 382-392. Kay, R. H., & LeSage, A. (2009). Examining the benefits and challenges of using audience response systems: A review of the literature. Computers & Education, 53(3), 819-827. doi: DOI 10.1016/j.compedu.2009.05.001 Kollar, I., & Fischer, F. (2010). Peer assessment as collaborative learning: A cognitive perspective. Learning and Instruction, 20(4), 344-348. doi: DOI 10.1016/j.learninstruc.2009.08.005 Latané, B. (1981), The psychology of social impact. American Psychologist, 36, 343-356. Li, L., Liu, X. Y., & Steckelberg, A. L. (2010). Assessor or assessee: How student learning improves by giving and receiving peer feedback. British Journal of Educational Technology, 41(3), 525-536. doi: DOI 10.1111/j.14678535.2009.00968.x Pope, N. K. L. (2005). The impact of stress in self- and peer assessment. Assessment & Evaluation in Higher Education, 30(1), 51-63. Raes, A., Vanderhoven, E., & Schellens, T. (submitted). Increasing anonymity in peer assessment by using Classroom Response Technology within face-to-face higher education . Sluijsmans, D. M. A., Brand-Gruwel, S., van Merrienboer, J. J. G., & Martens, R. L. (2004). Training teachers in peer-assessment skills: effects on performance and perceptions. Innovations in Education and Teaching International, 41(1), 59-78. doi: Doi 10.1080/1470329032000172720 Sluijsmans, D.M.A. (2002). Student involvement in assessment: The training of peer assessment skills. (Unpublished doctoral dissertation). Open University of the Netherlands, Heerlen. Smith, H., Cooper, A., & Lancaster, L. (2002). Improving the Quality of Undergraduate Peer Assessment: A Case for Student and Staff Development. Innovations in Education and Teaching International, 39(1), 71 - 81. Steinberg, L., & Monahan, K.C. (2007). Age differences in resistance to peer influence. Developmental Psychology, 43, 1531-1543. Stepanyan, K., Mather, R., Jones, H., & Lusuardi, C. (2009). Student Engagement with Peer Assessment: A Review of Pedagogical Design and Technologies. Lecture Notes in Computer Science, 5686, 367-375. Sumter, S. R., Bokhorst, C. L., Steinberg, L., & Westenberg, P. M. (2009). The developmental pattern of resistance to peer influence in adolescence: Will the teenager ever be able to resist? Journal of Adolescence, 32(4), 10091021. doi: DOI 10.1016/j.adolescence.2008.08.010 Sung, Y.-T., Chang, K.-E., Chang, T.-H., & Yu, W.-C. (2010). How many heads are better than one? The reliability and validity of teenagers' self- and peer assessments. Journal of Adolescence, 33(1), 135-145. Topping, K. J. (1998). Peer assessment between students in colleges and universities. Review of Educational Research, 68, 249-276. Topping, K. J. (2003). Self and peer assessment in school and university: Reliability, validity and utility. In M. Segers, F. Dochy, & E. Cascallar (Eds.), Optimizing new modes of assessment: In search of qualities and standards (pp. 55-87). Dordrecht: Kluwer Academic. Ellen Vanderhoven et al. / Procedia - Social and Behavioral Sciences 69 (2012) 1340 – 1347 Tsivitanidou, O.E., Zachaia, Z.C., & Hovardos, T. (2011). Investigating secondary …school students’ unmediated peer assessment skills. Learning and Instruction, 21, 506-519. Van den Bossche, P., Gijselaers, W., Segers, M., & Kirschner, P. A. (2006). Social and cognitive factors driving teamwork in collaborative learning environments: team learning beliefs and behaviors. Small Group Research, 37, 490-521. van Gennip, N. A. E., Segers, M. S. R., & Tillema, H. H. (2010). Peer assessment as a collaborative learning activity: The role of interpersonal variables and conceptions. Learning and Instuction, 20, 280-290. van Zundert, M., Sluijsmans, D., & van Merriënboer, J. (2010). Effective peer assessment processes: Research findings and future directions. Learning and Instruction, 20, 270-279. Vickerman, P. (2009). Student perspectives on formative peer assessment: an attempt to deepen learning? Assessment & Evaluation in Higher Education, 34(2), 221-230 Yang, Y.-F., & Tsai, C.-C. (2010). Conceptions of and approaches to learning through online peer assessment. Learning and Instruction, 20(1), 72-83. View publication stats 1347