Teaching evaluation and student response rate

PSU Research Review Teaching evaluat ion and st udent response rat e Tashfeen Ahmad, Article information: Downloaded by University of The West Indies at Mona At 08:45 30 October 2018 (PT) To cite this document: Tashfeen Ahmad, (2018) "Teaching evaluation and student response rate", PSU Research Review, https://doi.org/10.1108/PRR-03-2018-0008 Permanent link t o t his document : https://doi.org/10.1108/PRR-03-2018-0008 Downloaded on: 30 Oct ober 2018, At : 08: 45 (PT) Ref erences: t his document cont ains ref erences t o 39 ot her document s. Access t o t his document was grant ed t hrough an Emerald subscript ion provided by All users group For Authors If you would like t o writ e f or t his, or any ot her Emerald publicat ion, t hen please use our Emerald f or Aut hors service inf ormat ion about how t o choose which publicat ion t o writ e f or and submission guidelines are available f or all. Please visit www. emeraldinsight . com/ aut hors f or more inf ormat ion. About Emerald www.emeraldinsight.com Emerald is a global publisher linking research and pract ice t o t he benef it of societ y. The company manages a port f olio of more t han 290 j ournals and over 2, 350 books and book series volumes, as well as providing an ext ensive range of online product s and addit ional cust omer resources and services. Emerald is both COUNTER 4 and TRANSFER compliant. The organization is a partner of the Committee on Publication Ethics (COPE) and also works with Portico and the LOCKSS initiative for digital archive preservation. *Relat ed cont ent and download inf ormat ion correct at t ime of download. The current issue and full text archive of this journal is available on Emerald Insight at: www.emeraldinsight.com/2399-1747.htm Teaching evaluation and student response rate Tashfeen Ahmad Teaching evaluation and student response Downloaded by University of The West Indies at Mona At 08:45 30 October 2018 (PT) University Project Management Office, University of the West Indies, Kingston, Jamaica Abstract Purpose – The purpose of this paper is to share the author’s viewpoint on how to increase student response Received 4 March 2018 Revised 21 June 2018 6 July 2018 Accepted 10 July 2018 rate in course evaluation surveys. Design/methodology/approach – The approach is to highlight measures which increased student response rate in online surveys of the author’s teaching evaluation at The University of the West Indies, Jamaica. Findings – This viewpoint suggests that student response rate to course evaluation can be improved by the lecturer’s effective communication. The examples of effective communication are given in this paper. Originality/value – This work will encourage the lecturers to initiate more student engagement to improve response rate of their teaching evaluation. Keywords Online survey, Course evaluation, Paper survey, Response rate, Teaching evaluation Paper type Viewpoint Importance of student evaluation of teaching Student evaluation of teaching is important for a number of reasons. These evaluations ensure quality in university teaching, provide an independent method of gauging teacher’s effectiveness, guide in making decisions for major curriculum changes and professional development for faculty and help in establishing a framework to better quantify and reward good teaching outcomes. Shift in student evaluation of teaching from paper-based to online surveys Paper-based assessment has been the most common form of student evaluation of teaching worldwide. However, over the past decade, there has been a shift away from paper-based to online assessment. As internet is becoming more available and affordable, traditional paper-based data collection methods, seem expensive, time consuming and less efficient. Positives of this shift to online evaluation One of the most important positives is efficiency gains, in terms of turnaround time from students and significant cost savings. In addition, online evaluations allow students the time, ease and ability to refine, expand and reflect on responses without the constraint of an “in class” time bound environment to complete paper-based surveys. This increases student © Tashfeen Ahmad. Published in PSU Research Review: An International Journal. Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode PSU Research Review Emerald Publishing Limited 2399-1747 DOI 10.1108/PRR-03-2018-0008 Downloaded by University of The West Indies at Mona At 08:45 30 October 2018 (PT) PRR response to open-ended questions which provide qualitative data which is instrumental in improving teaching practices. A review of the major literature works over the period 2000-2013 (as depicted in Table I) summarizes the main advantages of online method of evaluation. Challenges of this shift to online evaluation One of the biggest challenges is the low response and return rate of students to online evaluations when compared to paper-based evaluation responses (Benton et al., 2010; Goodman et al., 2015; Guder and Malliaris, 2010; Nowell et al., 2010). A sample of the findings of response rates drawn from different research studies at various higher education institutions (as seen in Table II) over the years 1999-2013 indicates Authors (Year) Main advantages Hmieleski and Champagne (2000) Kasiar et al. (2002) More written feedback Refine, reflect, expand on responses More written feedback Refine, reflect, expand on responses Johnson (2002) Richer and higher data collection Hardy (2003) More written feedback Refine, reflect, expand on responses Ballantyne (2003) Richer and higher data collection Ballantyne (2004) Richer and higher data collection Anderson et al. (2006) Provide more feedback Richer and higher data collection Donovan et al. (2006) Provide more comments about Lecturer Laubsch (2006) More written feedback Refine, reflect, expand on responses Donovan et al. (2006) Richer and higher data collection Emery et al. (2008) Table I. Main advantages of using online course evaluation surveys Miller (2010) Samuels (2013) Authors (Year) Layne et al. (1999) Sax et al. (2003) Dommeyer et al. (2004) Table II. Anderson et al. (2005) Comparison of Avery et al. (2006) response rates (online Laubsch (2006) versus paper-based Nair et al. (2008) evaluation) Perrett (2013) Efficiency, cost savings, richer responses Time and cost savings, richer responses Richer responses, efficiency, quicker and cost savings Research focus areas Student course evaluations Comparison of traditional and Web-based evaluation processes Online student ratings Online student ratings Online evaluations of teaching Online student survey and comments Student course evaluations Student feedback on online vs traditional course evaluations Comparison of online and in person evaluations Constructive student feedback on online and traditional evaluations Open source online evaluation experiences Online evaluations Academic departments use of online course evaluations Institution Response rates Southeastern University Several US institutions California State University University of Kentucky Cornell University Fairleigh Dickinson University Monash University Large university in South US 47% – online vs 60% – paper 17% – online vs 24% – paper 43% – online vs 75% – paper 83% – online vs 80% – paper 47% – online vs 69% – paper 61% – online vs 82% – paper 31% – online vs 56% – paper 71% – online vs 68% – paper Downloaded by University of The West Indies at Mona At 08:45 30 October 2018 (PT) general lower return rates for online evaluations compared to paper-based evaluations in all except two cases (ranging as low as 17 per cent to a high of 83 per cent). This low number of response rates, in online surveys, makes the data invalid. To mitigate this challenge, Nulty’s (2008) research provides a set of guidelines for required response rates to be considered valid and useful measure of accuracy for online evaluation. Since the larger is the class size, the lower response rate is required, Nulty recommends an ideal required response rate of 58 (size <20) and 35 per cent (>50) for accuracy of online survey results and to achieve validity. Major reasons for the differences in response rates The reasons for the differences in response rates range from gender and age factors (Hatfield and Coyle, 2013); privacy and anonymity (Khorsandi et al., 2012; Nevo et al., 2010); social pressure; distraction and location issues (Mau and Opengart, 2012); lack of engagement; incentives; communication; perceived inaction with feedback or general “survey fatigue” (Bennett and Nair, 2010); and demographic and economic variables peculiar to the institution of country (Morrison, 2011). Solving the issue of low response rate Bennett and Nair (2010) in their study at an Australian University were able to register an overall 83 per cent online response rate, but this was in response to the deliberate strategies and measures implemented to increase student involvement. Using effective engagement, communication and teacher–student participation techniques led to greater and more sustained response rates. Measures to increase student online response rates A vast amount of literature has been written about the problems and the strategies which can be used to encourage and increase the response rates of student online evaluation (Crews and Curtis, 2011; Morrison, 2011; Stowell et al., 2012). The most comprehensive work done by Berk (2012) outlines a review of the problems and articulates an in-depth set of techniques and best practices which can be applied to increase online response rates. It should be noted however that he does not advocate a “one size fit all” solution but emphasizes that success in raising response rates will most likely be met by a combination of strategies and incentives over the long term. In my opinion, the most important and fundamental ingredients for raising online response rates depends to large extent on the commitment, engagement and buy in of both students and teaching administrators to the process. For example, studies indicate that the biggest determinant for student participation in online evaluation is the level of engagement they obtain from teachers (Gaillard et al., 2011). Those institutions which take the time to communicate and explain the process, how their responses will be used or incorporated to improve course delivery and outcomes experience increase in response rates (Wode and Keiser, 2011). On the other hand, students who do not feel a part of the process or think their feedback will not be taken seriously or valued or teachers who do not effect changes consequent on feedback experience lower response rates (Beran and Rokosh, 2009). What can lecturers do to increase response rate? The response rates are important as these evaluations are frequently used for consideration in tenure and promotion, hiring and pay increase decisions (Hammonds et al., 2017). Teaching evaluation and student response Downloaded by University of The West Indies at Mona At 08:45 30 October 2018 (PT) PRR My viewpoint is that response rate can be increased if lecturers are informed about the timing of when the surveys are sent out, so they can also make a personal appeal (both in class and by email) to the students to complete their course evaluation surveys. In this communication, lecturers should explain to the students how their comments would be taken seriously, and how it will be used to improve teaching (Heinert and Roberts, 2016). The key is to inform students about the purpose of evaluations:  Let students know that you will use their feedback to make changes in the course.  Give students some examples of useful feedback you have received in the past, and how the course/pedagogy has benefited in response. This best practice will show you improved results, and if you also want to score better in these evaluations, start giving chocolate cookies to your students (ESA, 2018). References Anderson, H.M., Cain, J. and Bird, E. (2005), “Online student course evaluations: review of literature and a pilot study”, American Journal of Pharmaceutical Education, Vol. 69 No. 1, p. 5. Anderson, J., Brown, G. and Spaeth, S. (2006), “Online student evaluations and response rates reconsidered”, Innovate: Journal of Online Education, Vol. 2 No. 6. Avery, R.J., Bryant, W.K., Mathios, A., Kang, H. and Bell, D. (2006), “Electronic course evaluations: does an online delivery system influence student evaluations?”, The Journal of Economic Education, Vol. 37 No. 1, pp. 21-37. Ballantyne, C. (2003), “Online evaluations of teaching: an examination of current practice and considerations for the future”, New Directions for Teaching and Learning, Vol. 2003 No. 96, pp. 103-112. Ballantyne, C. (2004), “Online or on paper: an examination of the differences in response and respondents to a survey administered in two modes”, in International Conference of Australasian Evaluation Society, Adelaide, October. Bennett, L. and Nair, C.S. (2010), “A recipe for effective participation rates for web-based surveys”, Assessment & Evaluation in Higher Education, Vol. 35 No. 4, pp. 357-365. Benton, S.L., Webster, R., Gross, A.B. and Pallett, W.H. (2010), An Analysis of IDEA Student Ratings of Instruction Using Paper versus Online Survey Methods 2002-2008 Data, The IDEA Technical Center. Beran, T.N. and Rokosh, J.L. (2009), “Instructors’ perspectives on the utility of student ratings of instruction”, Instructional Science, Vol. 37 No. 2, pp. 171-184. Berk, R.A. (2012), “Top 20 strategies to increase the online response rates of student rating scales”, International Journal of Technology in Teaching and Learning, Vol. 8 No. 2, pp. 98-107. Crews, T.B. and Curtis, D.F. (2011), “Online course evaluations: faculty perspective and strategies for improved response rates”, Assessment & Evaluation in Higher Education, Vol. 36 No. 7, pp. 865-878. Dommeyer, C.J., Baum, P., Hanna, R.W. and Chapman, K.S. (2004), “Gathering faculty teaching evaluations by in-class and online surveys: their effects on response rates and evaluations”, Assessment & Evaluation in Higher Education, Vol. 29 No. 5, pp. 611-623. Donovan, J., Mader, C.E. and Shinsky, J. (2006), “Constructive student feedback: online vs traditional course evaluations”, Journal of Interactive Online Learning, Vol. 5 No. 3, pp. 283-296. Emery, L., Head, T., Zeckoski, A., Yu. and Borkowski, E. (2008), “Deploying an open source, online evaluation system: multiple experiences”, Presentation at Educause 2008, October 31, Orlando, FL. ESA (European Society of Anaesthesiology) (2018), “Teachers who give cookie rewards score better in evaluations”, ScienceDaily, 17 June, available at: www.sciencedaily.com/releases/2018/06/ 180604182502.htm Downloaded by University of The West Indies at Mona At 08:45 30 October 2018 (PT) Gaillard, F.D., Mitchell, S.P. and Kavota, V. (2011), “Students, faculty, and administrators perception of students evaluations of faculty in higher education business schools”, Journal of College Teaching & Learning (Tlc)), Vol. 3 No. 8. Goodman, J., Anson, R. and Belcheir, M. (2015), “The effect of incentives and other faculty-driven strategies to increase online student evaluation response rates”, Assessment & Evaluation in Higher Education, Vol. 40 No. 7, pp. 958-970, doi: 10.1080/02602938.2014.960364. Guder, F. and Malliaris, M. (2010), “Online and paper course evaluations”, American Journal of Business Education (AJBE), Vol. 3 No. 2, pp. 131-138. Hammonds, F., Mariano, G.J., Ammons, G. and Chambers, S. (2017), “Student evaluations of teaching: improving teaching quality in higher education”, Perspectives: Policy & Practice in Higher Education, Vol. 21 No. 1, pp. 26-33, doi: 10.1080/13603108.2016.1227388. Hardy, N. (2003), “Online ratings: fact and fiction”, New Directions for Teaching and Learning, Vol. 2003 No. 96, pp. 31-38. Hatfield, C.L. and Coyle, E.A. (2013), “Factors that influence student completion of course and faculty evaluations”, American Journal of Pharmaceutical Education, Vol. 77 No. 2, p. 27. Heinert, S. and Roberts, T.G. (2016), “Factors motivating students to respond to online course evaluations in the college of agricultural and life sciences at the university of Florida”, NACTA Journal, Vol. 60 No. 2, pp. 189-194, available at: http://eds.a.ebscohost.com.proxy1.ncu.edu Hmieleski, K. and Champagne, M.V. (2000), “Plugging in to course evaluation”, The Technology Source. Johnson, T. (2002), “Online student ratings: will students respond? Online student ratings of instruction”, New Directions for Teaching and Learning, Vol. 2003 No. 96, pp. 49-59. Kasiar, J.B., Schroeder, S.L. and Holstad, S.G. (2002), “Comparison of traditional and web-based course evaluation processes in a required, team-taught pharmacotherapy course”, American Journal of Pharmaceutical Education, Vol. 66 No. 3, pp. 268-270. Khorsandi, M., Kobra, A., Ghobadzadeh, M., Kalantari, M. and Seifei, M. (2012), “Online vs traditional teaching evaluation: a cross-sectional study”, Procedia – Social and Behavioral Sciences, Vol. 46 No. 2012, pp. 481-483. Laubsch, P. (2006), “Online and in-person evaluations: a literature review and exploratory comparison”, Journal of Online Learning and Teaching, Vol. 2 No. 2, pp. 62-73. Layne, B.H., DeCristoforo, J.R. and McGinty, D. (1999), “Electronic versus traditional student ratings of instruction”, Research in Higher Education, Vol. 40 No. 2, pp. 221-232. Mau, R.R. and Opengart, R.A. (2012), “Comparing ratings: in-class (paper) vs out of class (online) student evaluations”, Higher Education Studies, Vol. 2 No. 3, p. 55. Miller, M.H. (2010), “Online evaluations show same results, lower response rate”, The Chronicle of Higher Education. Morrison, R. (2011), “A comparison of online versus traditional student end-of-course critiques in resident courses”, Assessment & Evaluation in Higher Education, Vol. 36 No. 6, pp. 627-641. Nair, C.S., Adams, P. and Mertova, P. (2008), “Student engagement: the key to improving survey response rates”, Quality in Higher Education, Vol. 14 No. 3, pp. 225-232. Nevo, D., McClean, R. and Nevo, S. (2010), “Harnessing information technology to improve the process of students’ evaluations of teaching: an exploration of students’ critical success factors of online evaluations”, Journal of Information Systems Education, Vol. 21 No. 1, p. 99. Nowell, C., Gale, L.R. and Handley, B. (2010), “Assessing faculty performance using student evaluations of teaching in an uncontrolled setting”, Assessment & Evaluation in Higher Education, Vol. 35 No. 4, pp. 463-475. Nulty, D.D. (2008), “The adequacy of response rates to online and paper surveys: what can be done?”, Assessment & Evaluation in Higher Education, Vol. 33 No. 3, pp. 301-314, doi: 10.1080/02602930701293231. Teaching evaluation and student response Downloaded by University of The West Indies at Mona At 08:45 30 October 2018 (PT) PRR Perrett, J.J. (2013), “Exploring graduate and undergraduate course evaluations administered on paper and online: a case study”, Assessment & Evaluation in Higher Education, Vol. 38 No. 1, pp. 85-93. Samuels, B. (2013), Increasing Number of Academic Departments Use Online Course Evaluations, CampusWest, 30 April. Sax, L.J., Gilmartin, S.K. and Bryant, A.N. (2003), “Assessing response rates and nonresponse bias in web and paper surveys”, Research in Higher Education, Vol. 44 No. 4, pp. 409-432. Stowell, J.R., Addison, W.E. and Smith, J.L. (2012), “Comparison of online and classroom-based student evaluations of instruction”, Assessment & Evaluation in Higher Education, Vol. 37 No. 4, pp. 465-473. Wode, J. and Keiser, J. (2011), “Online course evaluation literature review and findings”. A Report from Academic Affairs, Columbia University, Chicago. About the author Tashfeen Ahmad expanded his understanding of Psychology at Harvard University and joined The University of the West Indies, Mona, Jamaica, with 10 years of general management experience. He has taught courses in International Business, Production Management, Operations Management, Quality Management and Change Management. His research work focuses on the future of higher education and learning technologies. Tashfeen Ahmad can be contacted at: mrtashfeen@hotmail.com For instructions on how to order reprints of this article, please visit our website: www.emeraldgrouppublishing.com/licensing/reprints.htm Or contact us for further details: permissions@emeraldinsight.com