Feedback: Part of a System

September 2012 | Volume 70 | Number 1 Feedback for Learning Pages 30-34 Feedback: Part of a System Dylan Wiliam Just as a thermostat adjusts room temperature, effective feedback helps maintain a supportive environment for learning. We're all familiar with the kind of feedback that occurs in an engineering system when a public address microphone is placed too near a loudspeaker; the output from the speaker is picked up by the microphone and amplified further, which in turn makes the output from the speaker even louder, and so on. This positive feedback loop ultimately results in an ear-splitting howl. Less commonly known is the negative feedback loop in which feedback operates as a component in a self-regulating system. A room thermostat is a good example. Each thermostat contains a thermometer, which measures the temperature of the air in the room, as well as an instrument enabling the user to set the desired temperature. Most important, the thermostat contains a mechanism that compares the desired temperature with the actual temperature; if the reading on the thermometer is below the desired temperature, this mechanism sends a signal to turn on the heating system. When the temperature in the room reaches the desired temperature, the signal to the heating system is turned off. Engineers call this a negative feedback loop because when the room gets colder, this information (feedback) triggers a message to the heater to warm up the room. So the effect of the information, turning on the heat, is to oppose the existing tendency for the room to cool down. The important thing about the concept of feedback in engineering is that the feedback is designed as part of a system, and the role of feedback is to keep the system under control. From Engineering to Psychology In the 1960s, there was great interest in the idea that schools could improve instructional design by adopting a more scientific approach. Borrowing the idea of feedback from engineering systems theory thus seemed like an obvious thing to try. Unfortunately, as psychologists quickly discovered, making feedback work for learning proved more complex than using feedback in engineering. At the time, psychologists' dominant view was that learning resulted from making associations between stimuli and responses. If students failed to learn something, that meant the links between stimuli and responses required further reinforcement. Many psychologists therefore assumed that feedback in the form of positive reinforcement (telling learners that their responses to questions were correct) would increase the likelihood that the students would make the same response on some future occasion (for example, when they took a test). One influential review defined feedback as "any of the numerous procedures that are used to tell a learner if an instructional response is right or wrong" (Kulhavy, 1977, p. 211). Unfortunately, many psychologists missed the importance of designing feedback as part of a system, instead assuming that just telling students whether their responses were correct or incorrect would improve learning. To an engineer, this would be nonsense, tantamount to installing a thermostat but forgetting to connect it to the furnace. In the 1980s and 1990s, several reports attempted to draw together the various research findings on the effects of feedback (see, for example, Bangert-Drowns, Kulik, Kulik, & Morgan, 1991; Crooks, 1988; Natriello, 1987); but simple conclusions were elusive. Studies that examined whether delaying feedback was beneficial or harmful obtained differing results. Studies that involved a pretest showed smaller effects than studies that did not, presumably because the pretest itself improved the learning somehow. Perhaps most surprising, a number of studies, notably those by Butler (1987, 1988), showed that feedback in the form of scores and grades could actually reduce student learning. In one extraordinary project, Avraham Kluger and Angelo DeNisi (1996) reviewed every research study on the effects of feedback that had been published between 1905 and 1995. They found 2,500 journal articles and 500 technical reports, but they soon realized that many of these studies were of dubious quality. Some, for example, involved only a single participant. In others, there was no control group. In still others, feedback was combined with some other form of intervention (such as goal setting), so it was not clear which intervention was having an effect. After paring down the studies to those that met some basic quality criteria—having at least 10 participants, containing at least one group of participants who received only feedback, comparing the performance of those receiving feedback with a group not receiving feedback, and measuring performance before and after so that it was possible to quantify the effects of the feedback—Kluger and DeNisi were left with just 131 welldesigned studies. They found that in these studies, on average, feedback did significantly improve learning. But surprisingly, in 50 of these studies, giving feedback made learners' performance worse. In other words, in more than two out of five carefully controlled scientific studies, you would have been better off shutting up than actually giving the feedback. In attempting to understand these results, Kluger and DeNisi realized that the effects of feedback depended on the reactions of the recipient. The nature of the feedback itself was less important than the kind of responses triggered in individual students. Of course, to engineers, this would have been no surprise, because they understood the importance of thinking about feedback as part of a system. What seems odd in retrospect is how long it took psychologists to realize that we cannot understand feedback without thinking about how recipients react to the feedback. Feedback in Education When we give feedback, there are two possible cases—the feedback might show that current performance falls short of the goal, or it might show that the goal has already been reached. There are also four responses an individual can make to the feedback— he or she can change the behavior, modify the goal, abandon the goal, or reject the feedback. Figure 1 shows all these possibilities, with the two desirable outcomes in bold. The figure makes it clear why it's so hard to get feedback right. When we give students feedback, there are eight things that can happen—and six of them are bad! FIGURE 1. Eight Ways Students May Respond to Feedback Recipients respond to feedback in four basic ways: If feedback indicates that performance has fallen short of the goal, the recipient may If feedback indicates that performance has exceeded the goal, the recipient may By changing behavior Increase effort* Exert less effort By modifying the goal Reduce aspiration Increase aspiration* By abandoning the goal Decide the goal is too hard Decide the goal is too easy By rejecting the feedback Ignore the feedback Ignore the feedback * = Desirable outcome When we try to determine what kind of feedback works, therefore, we are asking the wrong question. What matters is what response the feedback triggers in the recipient. A furnace doesn't care how many times it's turned on during the day, or what the other furnaces in the neighborhood are doing. But people aren't machines. The way they will react to feedback is difficult, if not impossible, to predict; it depends on not only the feedback given, but also the context in which the feedback is given, and even the relationship between the recipient and the person giving the feedback. The issue of competition illustrates the complexity of response to feedback. Competition can be powerful. When individuals receive feedback indicating that they are falling short of the goal but they feel the goal is within reach, their response is likely to be to increase effort. In an analysis by Berger & Pope (2011) of more than 18,000 professional basketball games and 45,000 collegiate games, teams that were one point behind at halftime ended up winning more often than did teams that were ahead by one point at halftime. The researchers concluded that these findings, together with the results of other studies, suggested that being just slightly behind increases effort and can lead to winning. Many teachers believe that competition for grades can increase performance, and to some extent they are right. Students who feel that the goals are within their grasp are likely to be motivated by competition to do even better. There are two problems with competition in education, however. First, many students do not feel they are able to compete, and therefore they give up—so competition produces gains for some students at the expense of others. This might be acceptable in the adult world in competition for jobs and other scarce resources, but it is unacceptable in primary and secondary education, where we want every student to achieve at high levels. And competition can be counterproductive for the winners as well as the losers. For many high-achieving students, grades become more important than what the grades are intended to signify. Psychologists call this a performance orientation to learning. Students with a performance orientation might cheat to get a particular grade—but more damagingly, they may avoid challenge, preferring easy work because they can get a high score. Students often adopt a performance orientation because of their views of the nature of ability. As the work of Carol Dweck (2006) has shown, many students believe that academic ability is more or less fixed—that there are smart kids and not-so-smart kids. When students with this view of learning are given a task in the classroom, they rapidly make a judgment about their chances of success. If they think there is a danger that they'll fail while many others in the class succeed, they are likely to disengage from the task. After all, it's better to be thought lazy than dumb. Students with an incremental view of ability, on the other hand, see challenging tasks as opportunities to get smarter. Because of the many factors affecting how recipients respond to feedback, research offers no simple prescription for making feedback work effectively. What works in one classroom for one teacher will not work for another teacher. Feedback given by a teacher to one student might motivate that student to strive harder to reach a goal, whereas exactly the same feedback given by the same teacher to another student might cause the student to give up. Designing Effective Feedback Although the existing research cannot provide teachers with a single "right" way to give feedback, it does suggest a number of important features that teachers can build into feedback that can increase the likelihood of a productive student response. As Figure 1 shows, we want students who fall short of a goal to strive to reach it, and we want those who have already reached it to aspire to higher goals. This outcome requires that we establish a classroom environment in which students focus on intellectual growth rather than on preserving their emotional well-being. There is no simple formula for establishing such an environment, but from my reading of the studies on feedback and my work with teachers over the years, I've found that two principles seem to be almost universally applied in the classrooms where feedback is used to maximum effect. First, teachers must establish the classroom as a safe place for making mistakes. As Alina Tugend (2011) shows in her book Better By Mistake: The Unexpected Benefits of Being Wrong, the best learners fail often. Second, and related to this, teachers who use feedback effectively convey the idea that smart is not something you just are; it's something you can become. In this regard, the most important word in a teacher's vocabulary is "yet." When a student says "I can't do this," the teacher adds, "yet." One way to emphasize an incremental mind-set is to refuse to award less-than-passing grades to work. If the work does not yet merit a passing grade, provide the student with the support needed to get the work up to a passing standard. Emphasize the idea that although some students may need more support than others, all students can succeed. The research also suggests that the most effective feedback 1 2 3 Focuses on the task at hand rather than the recipient's ego. When students receive both scores and comments, the first thing they look at is their score, and the second thing they look at is … someone else's score. Being compared with others triggers a concern for preserving well-being at the expense of growth. One high school language arts teacher writes comments about students' essays on strips of paper, rather than on the students' notebooks. The next day, each group of four students receives back their four essays and the four strips of paper, and the students' task is to match the comments to the essays. The ego involvement is minimized as students read and reflect on the comments before they know whose paper the comments refer to. Focuses on things that are within the recipient's control. Telling a student to "be more systematic" is likely to be no more helpful than telling an aspiring basketball player to be taller or an unsuccessful comedian to be funnier. Feedback can be true, but useless. Requires more work from the recipient than from the giver. If feedback highlights everything that is wrong in a piece of work, there's nothing left for the recipient to do. If a student has solved a number of equations, some correctly and some incorrectly, the teacher could say, "Five of these are incorrect. Your challenge is to find them and fix them." For students who have solved all of the equations correctly, the teacher could say, "Make up three equations for others to solve; one harder, one at about the same level, and one easier than the ones you've just solved." More Complex Than Thermostats Ultimately, we need to remember what engineers realized more than 60 years ago— feedback only works within a system. Because classrooms are much more complex than thermostats, you cannot give good feedback without understanding your students, their experiences with current and previous teachers, their attitudes about the subjects they are studying, and how they perceive you. This complexity means that the key to effective feedback is the judgment and creativity of teachers. References Bangert-Drowns, R. L., Kulik, C.-L. C., Kulik, J. A., & Morgan, M. (1991). The instructional effect of feedback in test-like events. Review of Educational Research, 61(2), 213–238. Berger, J., & Pope, D. G. (2011). Can losing lead to winning? Management Science, 57(5), 817–827. Butler, R. (1987). Task-involving and ego-involving properties of evaluation: Effects of different feedback conditions on motivational perceptions, interest and performance. Journal of Educational Psychology, 79(4), 474–482. Butler, R. (1988). Enhancing and undermining intrinsic motivation: The effects of task-involving and ego-involving evaluation on interest and performance. British Journal of Educational Psychology, 58, 1–14. Crooks, T. J. (1988). The impact of classroom evaluation practices on students. Review of Educational Research, 58(4), 438–481. Dweck, C. S. (2006). Mindset: The new psychology of success. New York: Random House. Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119(2), 254–284. Kulhavy, R. W. (1977). Feedback in written instruction. Review of Educational Research, 47(2), 211–232. Natriello, G. (1987). The impact of evaluation processes on students. Educational Psychologist, 22(2), 155–175. Tugend, A. (2011). Better by mistake: The unexpected benefits of being wrong. New York: Riverhead.