Biol. Rev. (2004), 79, pp. 301335. f Cambridge Philosophical Society DOI : 10.

1017/S1464793103006225 Printed in the United Kingdom

301

The evolution of learning

Bruce R. Moore
Department of Psychology, Dalhousie University, Halifax, NS, Canada B3H 4J1 (E-mail : bmoore@dal.ca) (Received 12 August 2002 ; revised 17 April 2003 ; accepted 6 May 2003 )

ABSTRACT Most processes or forms of learning have been treated almost as special creations, each as an independent process unrelated to others. This review oers an evolutionary cladogram linking nearly one hundred forms of learning and showing the paths through which they evolved. Many processes have multiple forms. There are at least ve imprinting processes, eleven varieties of Pavlovian conditioning, ten of instrumental conditioning, and eight forms of mimicry and imitation. Song learning evolved independently in at least six groups of animals, and movement imitation in three (great apes, cetaceans and psittacine birds). The cladogram also involves at least eight new processes : abstract concept formation, percussive mimicry, cross-modal imitation, apo-conditioning, hybrid conditioning, proto-pantomime, prosodic mimicry, and image-mediated learning. At least eight of the processes evolved from more than one source. Multiple sources are of course consistent with modern evolutionary theory, as seen in some obligate symbionts, and gene-swapping organisms. Song learning is believed to have evolved from two processes : auditory imprinting and skill learning. Many single words evolved from three sources: vocal mimicry, discrimination learning, and abstract concept formation. Key words : evolution, imprinting, navigation, Pavlovian, instrumental, mimicry, imitation, language.

CONTENTS I. Introduction ................................................................................................................................................. (1) Background information ...................................................................................................................... (2) Possible advantages of evolutionary analysis ..................................................................................... II. Habituation and sensitization .................................................................................................................... (1) Habituation phenomena ...................................................................................................................... (2) Sensitization ........................................................................................................................................... (3) The Pavlovian pathway ........................................................................................................................ (4) Extinction, discrimination learning, and concept formation .......................................................... III. Pavlovian processes and imprinting .......................................................................................................... (1) Alpha and Garcia conditioning ........................................................................................................... (2) Imprinting .............................................................................................................................................. (a) Filial imprinting (visual) ................................................................................................................. (b) Auditory imprinting ....................................................................................................................... (c) Sexual imprinting ............................................................................................................................ (d ) Is imprinting associative ? ............................................................................................................... (e) Home and habitat imprinting ....................................................................................................... (3) Observational conditioning ................................................................................................................. (4) Misbehaviour and superstition ............................................................................................................ (5) Auto-shaping .......................................................................................................................................... (6) Dierences between avian and mammalian conditioning .............................................................. (7) Associations between neutral stimuli .................................................................................................. IV. Homing and navigation .............................................................................................................................. (1) The primary compass ........................................................................................................................... (2) Magnetic compass or scientic false alarm ? ..................................................................................... (3) Navigational maps ............................................................................................................................. 302 302 303 303 303 305 305 306 306 306 307 307 308 308 308 309 309 309 312 312 314 314 314 314 315

302

Bruce R. Moore
V. Mimetic learning in birds ........................................................................................................................... (1) Song learning ......................................................................................................................................... (2) The neural basis of song learning in oscines ..................................................................................... (3) Vocal mimicry ....................................................................................................................................... (4) Movement imitation in birds ............................................................................................................... (5) Cross-modal imitation .......................................................................................................................... VI. Instrumental learning and tool use ........................................................................................................... (1) The Pavlovian Law of Eect ............................................................................................................... (2) Thorndikian conditioning .................................................................................................................... (3) Cross-drive conditioning ...................................................................................................................... (4) Operant conditioning ........................................................................................................................... (5) Skill learning and tool use .................................................................................................................... (6) Hybrid conditioning ............................................................................................................................. (7) Putting through ..................................................................................................................................... (8) Omission training .................................................................................................................................. VII. Instrumental conditioning with negative reinforcement ........................................................................ (1) Vicious-circle behaviour ....................................................................................................................... (2) Avoidance conditioning ....................................................................................................................... (3) Nested mechanisms ? ............................................................................................................................ (4) Punishment ............................................................................................................................................ VIII. Movement imitation in mammals ............................................................................................................. (1) Simpler processes that mimic imitation ............................................................................................. (2) Imitation in dolphins ............................................................................................................................ (3) Imitation in primates and humans ..................................................................................................... (4) The origins of anthropoid imitation ................................................................................................... (5) Cross-modal imitation in hominids .................................................................................................... (6) Imitation in cats and rats ? ................................................................................................................... (7) Are avian and hominoid imitation homologous? ............................................................................. IX. Language acquisition .................................................................................................................................. X. Image-mediated learning ............................................................................................................................ XI. Does learning re-capitulate ? ...................................................................................................................... XII. Evolutionary webs : cross-connections in the tree ................................................................................... XIII. Comparative evidence for evolutionary hypotheses ............................................................................... XIV. The vertebrate hierarchy ............................................................................................................................ XV. Acknowledgements ...................................................................................................................................... XVI. References .................................................................................................................................................... 315 315 316 316 317 317 317 317 319 319 319 319 320 320 321 321 321 321 322 322 323 323 324 324 325 325 326 326 326 327 327 327 328 328 329 329

I. INTRODUCTION (1 ) Background information The various forms of learning have traditionally been treated as separate creations, much as species were treated by pre-Darwinian biologists. This long-standing position is indefensible, however. Just as systematists, comparative zoologists, biochemists, and ethologists have reconstructed evolutionary hierarchies showing the origins of species, structures, proteins, and instincts, so also comparative psychologists (or others) should in principle be able to nd hierarchical relationships linking most or all known forms of learning. The present work addresses this question, and oers a cladogram linking ninety-seven processes (Fig. 1). The separate-creations approach to learning was not an isolated aberration. It was one aspect of a wider antibiological tradition that prevailed in learning for decades and led many researchers to fail to recognize, and sometimes grossly misinterpret, their animals most basic responses

(Moore, 1973; Moore & Stuttard, 1979). It is arguable that this tradition, which still persists in some quarters, retarded the study of learning throughout most of the twentieth century, and contributed to its recent decline. Ironically, nineteenth-century studies of learning were solidly grounded in biology. Spalding (1873), who rst studied imprinting, was a zoologist, as was Lloyd Morgan who described Pavlovian conditioning before Pavlov (Morgan, 1894, pp. 86, 89), Thorndikian conditioning before Thorndike (Morgan, 1896, p. 22), taste aversions before Garcia (Morgan, 1894, pp. 85, 90, 287 ; Morgan, 1896, pp. 150151), and observational conditioning before anyone else (Morgan, 1896, pp. 118119, 164). Further, Thorndikes (1898) monograph, which for more than a century provided the foundation of research in both instrumental learning and imitation, began by explaining that the primary reason for studying learning in animals was to seek the roots of complex processes and, in so doing, to trace the evolution of learning down through the phylum ( pp. 2, 3). That is, he hoped to provide an evolutionary framework

The evolution of learning

for a truly comparative study of learning. Regrettably, soon after announcing this goal Thorndike turned to other matters, and his vision was largely ignored by subsequent generations of experts. There were exceptions, of course, in the twentieth century. Wells (1967, 1968), and Kandel & Schwartz (1982), both suggested that Pavlovian conditioning had evolved from sensitization. Slater (1983) argued that vocal mimicry in birds had evolved from song learning. More recently, Moore (1992) suggested that movement imitation in parrots had evolved through the following progression : song learning p vocal mimicry p percussive mimicry p movement imitation p cross-modal imitation, and that imitation in hominoids (Moore, 1996) had evolved along a dierent path : Thorndikian learning p operant conditioning p skill learning p putting through p visual movement imitation p crossmodal imitation. Moore (1996) showed these two progressions as distant branches of an evolutionary tree containing a total of 36 learning processes. The other elements of the tree were not explained because of space limitations. The present work attempts to explain a much larger cladogram (Fig. 1). (2 ) Possible advantages of evolutionary analysis The present work undoubtedly contains some errors and omissions. Nevertheless, it may prove worthwhile for several reasons : Gaps in evolutionary hierarchies can lead to the identication of previously unrecognized forms of learning, intermediate between known mechanisms (Moore, 1992, 1996). In the present case such gaps led to the identication of at least eight new learning processes. The approach forces one to distinguish between many levels of Pavlovian, instrumental, and imitative learning. This should reduce the tendency to equate complex processes seen in great apes and humans with the simpler forms of learning found in lower vertebrates and invertebrates. Similarly, the cladogram shows that corresponding processes in distantly related creatures are often analogous rather than homologous. For example, as previously noted (Moore, 1992, 1996) movement imitation appears to have evolved independently in great apes, dolphins, and parrots. Their imitation is unrelated, and sometimes quite dierent. The approach might relieve recurrent arguments about which of two processes cause some eect, like whether termite shing in chimpanzees (Goodall, 1971) is due to imitative or instrumental learning, or whether other eects are due to avoidance or Pavlovian conditioning, true punishment or Pavlovian, true operant or Pavlovian, imprinting or Pavlovian, Pavlovian or sensitization, Garcia conditioning or sensitization, and so forth. Many such conicts arise because the processes in question are evolutionarily related, and therefore similar and easy to confuse. In some cases the contested process might be intermediate between the suggested forms, in which case the arguments would have been futile. The approach might provide, for the rst time, a rational basis on which to compare the intelligence of various species. It might also oer a synthesis of behaviouristic and

303 cognitive approaches, while discouraging the indiscriminate attribution of cognition to primitive organisms. Finally, it might prove a useful antidote to the assumption that learning in various mammals, birds, and shes diers quantitatively (cf. Skinner, 1938, 1959 ; MacPhail, 1982). Below we examine the evidence that generated the cladogram in Fig. 1, beginning with habituation and sensitization, which weaken or strengthen instinctive reexes. Habituation is only discussed briey, given that it may have evolved repeatedly in various phyla, and since it was presumably not ancestral to more complex learning processes.

II. HABITUATION AND SENSITIZATION (1 ) Habituation phenomena Habituation is widely considered to be the simplest of all learning processes. The relevant literature, however, is both vast and complex. Major reviews have been published by Harris (1943), Thompson & Spencer (1966), Horn & Hinde (1970), Peeke & Hertz (1973), Peeke & Petrinovich (1984) among others. A bibliography covering only work published in English between 1940 and 1970 listed more than 800 papers (Leibrecht, 1972). Habituation has been dened as a response decrement occurring as the result of repeated [or prolonged] stimulation, and not attributable to fatigue or sensory adaptation (Harris, 1943). Various control groups that might allow one to rule out those explanations are described by Hinde (1970, pp. 34). At least nine properties of habituation were listed by Thompson & Spencer (1966), but all were critically examined, and often challenged, by Hinde (1970). The process occurs throughout most of the animal kingdom (Wyers, Peeke & Hertz, 1973). It has also been shown in spinal preparations (Prosser & Hunter, 1936) and in isolated ganglia (Castellucci et al., 1970). But, in spite of its ubiquity and apparent simplicity, we cannot assume that all habituation is homologous. Thus, Prosser & Hunters (1936) classic research with rats led them to suggest that habituation was due to changes occurring between sensory neurones and motoneurones, a conclusion reached also by Thompson & Spencer (1966). But, as Harris (1943) pointed out, this explanation could scarcely apply to protozoa, or species with syncytia. The point was echoed by Sharpless & Jasper (1956), who speculated that dierent mechanisms might underlie habituation in the amoeba, the sea anemone, and the higher vertebrates (p. 43). For years it was generally accepted that [ habituation] was due to an inhibitory process (Konorski, 1967, p. 99). But Spencer, Thompson & Neilson (1966) argued, and Castellucci et al. (1970) showed, that (at least in their preparations) it was due simply to a reduced excitatory tendency. Sokolov (1955), Sharpless & Jasper (1956), and others distinguished between short- and long-term habituation. That distinction was challenged by Thorpe (1956), who treated all habituation as potentially long-term (see also Thompson & Spencer, 1966). But neuro-anatomical work

304
writing* reading* image-mediated learning* pantomime vocal language* syntax* telegraphic speech* phrase clusters* two-word phrases* formulaic two-word phrases* pivot words* rote syntax* non-syntactic grammar* rote grammar* word clusters* parts of speech* single words* associations between neutral stimuli abstract concept formation discrimination learning phonetics* primitive music* vocal mimicry prosodic mimicry instrumental mimicry primitive counting* primitive dance* pantomime sign language processes rudimentary sign production proto-pantomime cross-modal imitation deferred imitation visual movement imitation putting through hybrid conditioning associations between neutral stimuli tool use skill learning operant conditioning cross-drive conditioning Thorndikian conditioning primitive language comprehension associations between neutral stimuli Pavlovian conditioning apoconditioning avoidance conditioning (novel responses) avoidance conditioning (SSDRs) omission learning punishment Pavlovian punishment tool use skill learning operant conditioning tool use skill learning operant conditioning avian Thorndikian learning vicious-circle behavior Pavlovian effect learning auto-shaping observational conditioning movement imitation percussive mimicry? vocal mimicry song learning cross-modal imitation visual movement imitation percussive mimicry vocal mimicry (non-associative) vocal mimicry (associative) song learning skill learning (Psittaciformes) percussive mimicry? vocal mimicry (associative) song learning skill learning (Passeres) vocal mimicry (associative) song learning skill learning (Piciformes)? song learning skill learning (Apodiformes)? auditory imprinting wind compass? star compass sun compass avian navigational map pilotage habitat imprinting sexual imprinting filial imprinting food imprinting Garcia conditioning abstract concept formation concept formation conditional discrimination discrimination learning extinction alpha conditioning long-term sensitization sensitization dishabituation long-term habituation habituation reflex repertoire

Bruce R. Moore

Hominids

Hominids

Pongidae

Mammalia

Mammalia

Psittaciformes Oscines Aves Chondrichthyes?

Cetacea

Psittaciformes Sturnidae Passeres Ramphastidae Piciformes Apodiformes

Aves

Hominids Vertebrata

Metazoa Animalia

Fig. 1. For legend see facing page.

The evolution of learning

with Aplysia californica suggests that long-term habituation might indeed be dierent. At least in molluscs, it seems to arise from specic pre-synaptic changes : reductions in the number, size, and vesicle complement of pre-synaptic active zones (Bailey & Chen, 1983). Physiologically as well as behaviourally, therefore, it is the opposite of sensitization. This would seem to weaken the assumption that habituation was ancestral to other learning (Harlow, 1958; Razran, 1971). Dishabituation, in which abrupt stimuli reduce habituation, is rather like sensitization (Prosser & Hunter, 1936). But it brings habituated reactions back to normal, or nearnormal, whereas sensitization enhances normal reexes. To see whether they are related would require close analysis of both processes in many simple species. A detailed analysis of habituation is beyond the scope of this article. What is very clear is that habituation and sensitization are complementary processes : the former weakens reexive responses while the latter strengthens them. (2 ) Sensitization Sensitization is ordinarily an after-eect of negative reinforcement. It occurs when mere presentation of a reinforcing stimulus, typically an aversive one, potentiates reactions from the species-typical repertoire. An early description of sensitization is found in Bernsteins (1934) study of human eyelid conditioning. The nominal conditioned stimulus (CS) was a loud click delivered through earphones ; the unconditioned stimulus (US) was a brief facial shock which elicited blinking. In analysing his data Bernstein (1934) discovered that, in some subjects, clicks elicited blinking even before the rst CSUS pairing. These blinks were found to be short-term after-eects of four pre-tests in which shock had been presented alone. Bernstein called the unexpected eect facilitation, but also commented that his subjects had been sensitized by shock. Several years later, Grether (1938) reported similar eects in two rhesus monkeys. After being subjected to frightening stimuli (ignition of ash powder, or sudden ination of a snake blowout toy) the animals reacted violently to other noises that had not previously disturbed them. They behaved as if conditioned to the noises, even though the sounds had never been paired with reinforcers. Like Bernstein, Grether said that his monkeys had been sensitized by the reinforcers. The shocks, ash powder, and blow-out toys are typical of the reinforcers used in sensitization. Positive reinforcers are rarely used in this context, and when they are it is important to control for background conditioning, which is easily

305 confused with sensitization (Kimble, 1961, p. 61). One must test the animals in dierent environments. If the eect occurs only in the environment where the reinforcer was delivered, it should be attributed to background conditioning, not sensitization. Grether (1938) gave sensitization another, somewhat pejorative, name. He called it pseudo-conditioning, and stressed the problem that it posed for interpretation of conditioning studies. Thus began a period of several decades in which sensitization was regarded primarily as a nuisance an impediment to the study of more important associative learning processes. To distinguish between true conditioning and mere non-associative artifacts such as sensitization, a number of control groups were devised and their relative merits forcefully debated (see e.g. Rescorla, 1967; Kremer, 1971). While the importance of control groups cannot be denied, a very dierent, and equally important, approach was taken by more biologically oriented writers. Wells (1968), and Kandel & Schwartz (1982), treated sensitization not as a nuisance, but as an important process in its own right, and saw its similarity to conditioning not as an accident, but as a probable sign that conditioning had evolved from sensitization. The likely progression is that habituation and sensitization arose independently from the species-typical repertoire with one process weakening, and the other strengthening, particular reexes as follows : from reex repertoire p habituation p long-term habituation p dishabituation p sensitization (Fig. 1). Sensitization currently strengthens groups of related reexes (though the process might have evolved in creatures with only single reactions). Whatever its origin, sensitization was almost certainly the source of Pavlovian conditioning (Wells, 1967, 1968; Kandel & Schwartz, 1982 ; Kandel, 1991), although there may have been intermediate steps (see below). (3 ) The Pavlovian pathway While conditioning and sensitization are similar, they dier in at least two ways. Conditioning is associative, while sensitization is not, and it is a long-term (semi-permanent) eect, unlike simple sensitization. Conditioning sometimes persists for years unless reversed by pro- or retro-active interference arising from extinction, latent inhibition, or competing associations. Since evolution proceeds one step at a time, we can infer the existence of at least one intermediate form of learning linking these two processes. If the associative property evolved before the long-term one, then the intermediate

Fig. 1. Cladogram linking various aspects of learning and language. Solid lines show hypothesized evolutionary pathways. Broken lines indicate that one process is a special case of another. Asterisks indicate products of cultural, rather than biological, evolution. Some processes have multiple origins. In such cases secondary sources are indicated by L-shaped line on the left side of the cladogram. The number of steps shown in any pathway could be increased or decreased by making more, or fewer, distinctions. For example, the various forms of imprinting could all be lumped together, or various elements of language could be added. Path lengths are arbitrary. No inferences can be drawn about which of two processes is the more primitive, except where one descends directly from the other. The use of the (modied) cladogram does not imply endorsement of all aspects of the theory of cladistics (Hennig, 1966).

306 process would have been short-term associative learning. If, however, the long-term property came rst, then the intermediate process would have been long-term sensitization. Since long-term sensitization seems the simpler process, and has been shown in very primitive creatures (Pinsker et al., 1973), we shall tentatively treat it as the intermediate process. The progression then is from sensitization p long-term sensitization p Pavlovian processes (Fig. 1). (4 ) Extinction, discrimination learning, and concept formation It is important to distinguish various levels of Pavlovian conditioning. But rst we must note briey several processes related to such conditioning : extinction, discrimination learning, and concept formation. The evolution of conditioning clearly made possible extinction : the weakening of conditioned associations when the CS is presented without the US. The extinction process produces, as a corollary, true discrimination learning. If one stimulus (CS+) is followed by a reinforcer, while another (CSx) is not, then the animal may learn to discriminate between the two, e.g. salivating to the former but not the latter. Conditional discriminations are somewhat more demanding. Here, CSA is reinforced only in the presence of some other stimulus (CSX). The animal must learn the conditional relationship or, what is often equivalent, learn to react to relevant combinations or patterns of stimuli, only some of which, e.g. CSA+CSX, are reinforced. Basic conditional discriminations involve only external stimuli. More complex forms, which occur in the learning of grammar, for example, are conditional upon abstract concepts. Concept formation (Hull, 1920; Ach, 1921, cited in Hanfmann & Kasanin, 1937), is also based upon discriminations. An animal might be shown objects with all possible combinations of N properties : e.g. large or small, rough or smooth, red or blue, triangular or square. If it learned to discriminate, for example, rough objects from all the rest, ignoring size, colour, form, and other factors, it would be said to have learned the concept of roughness. A higher level of this process, involving abstract concepts rather than external stimuli, occurs in humans, and is critical in language. Abstract concepts range from fairly primitive to highly sophisticated. It is quite possible that several transitions are involved.

Bruce R. Moore
reinforcement-appropriate subset of its species-typical repertoire. In the classic example, a dog may salivate to the sound of a bell that signals food (Pavlov, 1927). There are, however, many levels of Pavlovian processes, and we will distinguish at least 11 : alpha and Garcia conditioning, skeletal and positive conditioning, observational conditioning, auto-shaping, apo-conditioning, Pavlovian punishment, anticipatory (Pavlovian) responding, associations between neutral stimuli, and hybrid conditioning. ( 1) Alpha and Garcia conditioning The most primitive form of conditioning does not produce new associations. Alpha conditioning simply strengthens pre-existing reactions i.e. those in which the CS innately elicits the same response evoked by the US (Hull, 1934 ; Kimble, 1961; Razran, 1971 ; Kandel, 1976). Two frequently cited demonstrations of alpha conditioning involve human eyelid responses (Weber & Wendt, 1942 ; Grant & Adams, 1944). Both examples are entirely unconvincing, however, because they lack controls for sensitization. Kandels work with Aplysia californica provides much better examples. Typically, weak tactile stimulation of the animals siphon or mantle reexively elicits defensive retraction of the siphon and gills (Kandel, 1976, pp. 84 ). When either of these tactile stimuli is followed by tail shock, it will elicit stronger reactions, demonstrating classic alpha conditioning (Kandel, 1991, p. 1016). More familiar than alpha conditioning, but only slightly less constrained, is Garcia conditioning. J. Garcia and others have shown that many animals nd it easiest to form associations involving biologically signicant combinations of stimulus and reinforcer : avour and illness, for example, or visual stimuli and pain (see Revusky & Garcia, 1970). Bitterman (1976) challenged the reality of Garcia conditioning, attributing the eects to sensitization and suggesting that the vast literature on this process was a scientic misunderstanding. He rst attacked the notion of selective associations. In the seminal paper by Garcia & Koelling (1966), a 2r2 or double dissociation design was introduced. Rats were either shocked or made ill by X-rays (or lithium chloride). These USs were preceded by gustatory and/or audio-visual CSs. The shocked animals appeared to condition to audio-visual, but not gustatory stimuli, while those made ill by X-rays or poison did the opposite. Selective associations of this sort make biological sense, since, in nature, illness is often preceded by eating the wrong food, but rarely preceded by bells, whistles, or lights. Conversely, painful surface events like shocks are often preceded by visual or auditory cues, but rarely by avours. It is obvious that associations between such natural and signicant events would have evolved long before more complex forms of conditioning. Bittermans (1976) argument was that the Garcia pattern could be produced by sensitization alone. Illness could sensitize reactions to avours, and shock to sudden sounds and lights. It is true that Garcia & Koellings (1966) classic study lacked the relevant controls, but many subsequent studies were very rigorously controlled, as Revusky (1977) pointed out.

III. PAVLOVIAN PROCESSES AND IMPRINTING Classical (Pavlovian) conditioning is said to occur when an animal forms an association between a signal (conditioned stimulus or CS) and a reinforcer (unconditioned stimulus or US). The reinforcer is typically a biologically signicant stimulus such as food or water, warmth, pain, nausea, the arrival of a mate or predator, etc. As a consequence of such associations the animal may begin to treat the signal as if it were the reinforcer, reacting to it with responses from the

The evolution of learning

In addition, most people have had the personal misfortune of experiencing Garcia conditioning. Consumption of a certain food followed by illness within a few hours often results in associations of that food with illness. Subsequent exposure to the food can induce extreme nausea. Bitterman (1976) also denied that avour aversions [might] develop with relatively long intervals between the CS and US. Thus he ignored Revuskys (1968) classic article, which had shown that when rats were X-irradiated even 7 h after consuming a sucrose solution, their preference for this substance was greatly reduced. Before conditioning, when given a choice between plain and sugar water, they preferred the latter in the ratio 4 : 1. However after a single conditioning trial, that preference vanished. Revusky (1968) included three large, well-controlled experiments, involving almost 200 rats. Previously it was widely believed that Pavlovian conditioning required CSUS intervals of approximately 0.5 (0.250.67) s (Kimble, 1961, pp. 156157). Revusky (1968) produced strong conditioning within a single trial using an interval 50 000 times greater. The evolution of Garcia conditioning appears to have been very simple. In alpha conditioning the initial reaction to the CS is slightly above threshold ; in Garcia conditioning it is slightly below. This suggests that Garcia conditioning arose from alpha conditioning in the following progression : from sensitization p long-term sensitization p alpha conditioning p Garcia conditioning (Fig. 1). It should be noted that the four processes sensitization, long-term sensitization, alpha conditioning and Garcia conditioning almost always involve aversive reinforcers. The near-absence of positive reinforcement in these processes in the literature strongly suggests that sensitization initially involved negative events. Later, sensitization presumably involved also positive reinforcers. However, we shall simplify the cladogram by using only two forms: short- and long-term sensitization (Fig. 1). It is not clear that Garcia conditioning always extends to positive reinforcers. Rats decient in B vitamins such as thiamine sometimes seem to show specic hungers. When given the opportunity, they may choose diets containing the needed substance (Garcia et al., 1967). P. Rozin and his associates have shown, however, that these preferences may result from avoidance of decient diets. Vitamin-deprived rats sometimes tend to try novel foods instead of familiar ones. Given a choice of several novel foods, they sample them one at a time, and reject each decient one after one or two meals (Rodgers & Rozin, 1966 ; Rozin & Rodgers, 1967 ; Rozin, 1967, 1969). Thus, while the original form of Garcia conditioning can generate preferences for positive substances, the underlying mechanism might still depend upon negative reinforcers. We should note that higher mammals, at least, such as monkeys or great apes, when made ill by intestinal parasites often feed upon such medicinal plants as Aspilia, Vernonia, or bitter leaf, as do native humans in the same environments (for review see Cowen, 1990). This might be a higher level of Garcia conditioning, where instead of associations between avour and illness, there would be associations between avour and curatives. The delay might be the same, but the reinforcer would be positive.

307 The transitions between Garcia conditioning and three more modern processes (imprinting, observational conditioning, and auto-shaping) are perhaps the most dicult in the cladogram, since Garcia conditioning was shown primarily with mammals. The other three processes require both skeletal (motor) behaviour and positive reinforcers ; the skeletal behaviour presumably came rst. The third transition in each case is more obvious : imprinting requires sensitive periods, while the others require relaxation of constraints. We shall tentatively use three primitive levels of conditioning : Garcia, skeletal, and positive. (2 ) Imprinting The literature on imprinting has been extensively reviewed by Bateson (1966), Immelmann (1972), Bolhuis (1991), and others. The process was described clearly by Sir Thomas More in 1551 (Kevan, 1976), but was demonstrated experimentally by Spalding (1873), who reported that if otherwise isolated chicks were exposed to humans within three days of hatching, they often formed strong social attachments, i.e. lial imprinting. They tended then to follow humans and utter contentment calls in their presence, but distress calls when separated from them. If not seen until the fourth day, however, humans evoked antagonistic reactions: escape instead of attraction, and distress calls instead of contentment. There seemed to be a three-day sensitive period. Our understanding of imprinting comes primarily from precocial birds such as chicks, geese, or ducklings, which are able to move about fairly soon after hatching and must therefore imprint very early. By contrast, altricial birds, such as songbirds, hawks, and pigeons, which remain in their nests for some weeks, often imprint later and more gradually (Klinghammer, 1967). Human children imprint later still. Other forms of imprinting can determine future choices of food, habitat, home, mother, sexual partner, etc. Imprinting in various species involves visual, auditory or chemical senses, and occurs in many vertebrates and some invertebrates. Not all of these processes can be assumed to be homologous. It is necessary to review them briey before attempting to characterize imprinting, and nd their place or places in the cladogram. ( a ) Filial imprinting (visual ) Lorenz (1937) asserted that imprinting diered from other sorts of learning in being totally irreversible. He compared its absolute rigidity to that seen in some aspects of embryology, where transplanted tissue cannot be transformed except during an early sensitive period ( pp. 264266). In fact, data contradicting his position had already been published (Craig, 1911), and many more such data followed (e.g. Steven, 1955; Boakes & Panter, 1985; Kent, 1987). Re-imprinting is presumably not common in the wild, but not because imprinting is intrinsically permanent. More likely, it simply perpetuates itself by causing the animal to remain near its own kind. Thus, Lorenz (1937) distinction was not really valid : imprinting is not always permanent, while conditioning sometimes may be so (Solomon & Wynne, 1953 ; Solomon, Kamin & Wynne, 1953; Skinner, 1959).

308 Clearly, imprinting cannot be dened on the basis of irreversibility. Sensitive periods are denitive, but permanence is not. Many mammals show very similar (visual, lial) imprinting. A harbour seal (Phoca vitulina) named Chimo and handreared by Stanhope, Foy, and Moore (see Moore, 1996) soon began to follow the three experimenters (with great diculty) even up stairways and ladders. He also attempted to nurse them, and showed separation anxiety, crying pitifully if they moved too far away (Moore, 1996). For other examples of apparent imprinting in mammals, see Hediger (1968). The process may be important even in primates. Monkeys deprived of normal contact with conspecics during an early sensitive period clearly fail to form normal social bonds (Harlow & Zimmerman, 1959; Harlow & Harlow, 1962). Failure to imprint on ones own species may provide a powerful model for human sociopathic behaviour. ( b ) Auditory imprinting The literature on auditory imprinting is far more complex than that of the visual process. To discuss it thoroughly would require many distinctions e.g. between precocial and altricial birds, pre- and post-natal procedures, articial and natural sounds, sounds associated (or not) with visible objects, and attachments to species or individuals. To summarize briey, on the basis of the literature and several unpublished studies from my laboratory, it is clear that pre-natal auditory imprinting in chicks (Guyomarch, 1974) is unlikely to occur. Two larger-scale studies failed to replicate his work ( J. Church, unpublished data), as did other studies involving hundreds of chicks. Several groups have shown that either natural or articial sounds can be imprinted if they emanate from, and become associated with, visual objects (Evans, 1972; Evans & Mattson, 1972 ; Cowan, 1974 ; van Kampen & Bolhuis, 1991). The visual stimulus may act as a US for [imprinting chicks to] sounds (Bolhuis & van Kampen, 1992). But there is no solid evidence that chicks can imprint to novel (articial) sounds in the absence of associated visual stimuli. When hens clucks are used instead of articial stimuli, associations with visual stimuli are not required according to van Kampen & Bolhuis (1991). The chicks following is said to be a short-term process. If the imprinted sound was not heard for several hours, it seemed to be forgotten (Kent, 1987). This might well be adaptive since, in nature, long separations would often mean that the chick had been orphaned, in which case its best chance of survival would lie in bonding with foster parents (Kent, 1987). Further the natural clucks might provide individual recognition, but clearly not species imprinting. More fundamentally, natural clucks elicit following without imprinting. In recent unpublished work sixteen of my students over two years attempted to imprint 270 chicks on clucks taped by J. P. Kent. Half of the chicks heard Kents hen 1441 for 24 h after hatching ; the other half heard hen 1452. They were then tested in a 1 m diameter circular arena, outside of which a slowly moving (0.5 rev/min) nonvisible speaker played each cluck for two 8 min periods with an A-B-B-A or B-A-A-B pattern. The novel clucks were

Bruce R. Moore
followed just as strongly as the imprinted ones. The chicks following behaviour was clearly instinctive, not learned. Auditory imprinting does occur in some altricial birds. Young songbirds often learn songs performed by neighbours of their species. The songbirds often imprint when the sound emanates from, and is associated with, a visible conspecic or surrogate. It is not yet clear, however, whether all songbirds require associative imprinting. ( c ) Sexual imprinting Bischof (1994) suggested that sexual imprinting diered from lial imprinting in being a two-stage process which begins with exposure to a model during an early sensitive period but is then consolidated (or not) at maturity in a process that occurs only if sexual behaviour is performed ( p. 92). This assertion was based upon two studies (Immelmann et al., 1991 ; Kruijt & Meeuwissen, 1991) with Zebra nches Taeniopygia guttata castanotis. But both studies showed only the eects of exposure to conspecics; neither actually showed that the occurrence of courtship or mating was critical. To the contrary, Kruijt & Meeuwissen (1991) showed explicitly that mating was not required. There is no evidence that avian sexual imprinting diers in this respect from lial. Sexual imprinting, like lial, occurs in birds but also in mammals. And, again, it is reversible. When Chimo the seal (see above) reached maturity he clearly courted human females, caressing them and displaying an erection. But when he was nally (at age seven) put with conspecic females, he mated with several within the rst year, and fathered several pups (Moore, Stanhope & Foy, cited in Moore, 1996), suggesting that imprinting need not be permanent. Several writers (Gould, 1982 ; Staddon, 1983; Baptista, Bell & Trail, 1993 ; ten Cate, Vos & Mann, 1993) have pointed out similarities between sexual imprinting and song learning. The cladogram (Fig. 1) shows the two processes to be closely related. ( d ) Is imprinting associative ? Imprinting has traditionally been considered a non-associative process arising from mere exposure to the model (Lorenz, 1937). The tradition may be mistaken. We have seen that auditory imprinting in songbirds might be associative. Further, James (1959, 1960) drew the same conclusion for classic lial imprinting. His explanation was framed in Pavlovian terms, but also resembled what Thorndike (1935, pp. 191197) called associative shifting. James theory was later elaborated by Homan and his associates (e.g. Homan, Eiserer & Singer, 1972 ; Homan, 1973; Homan & Ratner, 1978; see also Boakes & Panter, 1985 ; Bolhuis, de Vos & Kruijt, 1990 ; van Kampen, 1993; Bolhuis & Honey, 1994). The basic idea is that a moving hen (or other object) elicits following even in untrained chicks. Some aspect of the moving object is a US for following. The many neutral properties of the object (e.g. size, colour, and shape) are potential CSs. As the chick imprints, those stimuli acquire signicance through association with the USs (see e.g. James, 1959, 1960).

The evolution of learning

An early demonstration of associative imprinting was published by Homan et al. (1972). If imprinting was indeed associative, then presenting the CS without the US should extinguish the bond. This has been shown by Evans (1972) and Evans & Mattson (1972) ; see also Boakes & Panter (1985). The similarity between imprinting and conditioning will be discussed later. The light being courted by Rackhams (1977) sexually auto-shaped pigeon could just as easily have been imprinted. The eects of sexual conditioning and sexual imprinting are sometimes so similar that it may not be clear which has occurred without knowing the animals history. For this reason, we shall treat lial imprinting as a sister group of auto-shaping. Imprinting seems to dier from that process primarily in having one extra constraint : it typically occurs during sensitive periods. ( e ) Home and habitat imprinting Certain insects are said to show host or habitat imprinting, returning as adults to the same plant, habitat, or host that they experienced as larvae. At rst glance, such imprinting might appear to be non-associative, based simply on exposure to critical stimuli. But recent work has shown that associations with faecal chemicals are required for habitat imprinting in various wasps (Tumlinson, Lewis & Vet, 1993). We cannot assume that this process is homologous to any vertebrate imprinting, but it provides an extreme example of how associative imprinting can masquerade as a non-associative (exposure) process. The various forms of imprinting might well have evolved from others. But without strong evidence, we shall treat them all as descendants of Garcia conditioning. Home and habitat imprinting also occur in vertebrates. Many such animals imprint on home areas and later navigate to them. (3 ) Observational conditioning Research on observational conditioning goes back to Lloyd Morgan (1896, pp. 118119), who reported that some mammals learn to identify potential predators through observation of parental reactions. Kittens and lambs were said to react fearfully to dogs only after having seen their mothers do so. Similar learning was reported in blackbirds by Curio, Ernst & Veith (1978), and in monkeys by Hall (1968) and Mineka et al. (1984). Human prejudices are often learned in precisely the same way. A child may play happily with another of a dierent ethnic background until a parent says, in eect, Arggh, youre playing with a lthy Klingon. This process is, unfortunately, a key to human prejudice. The simplest explanation of observational conditioning, I believe, is that it reects a combination of two pre-existing processes : behavioural contagion and primitive conditioning. Behavioural contagion is a simple reex in which an instinctive process performed by one animal elicits the same reaction in others. In the case of Morgans kittens and lambs, the mothers panic reaction elicited panic in her ospring. The primitive conditioning component is equally

309 obvious. The kitten or lamb associates the fear reaction with the dog that arrived just before the mothers panic. Observational conditioning sometimes acts to produce quite unnatural preferences. Thus, when Wyrwicka (1996) used hypothalamic stimulation to condition mother cats to consume unnatural substances, their watching kittens began to treat the same substances as food, and therefore acquired long-term preferences for bananas, potatoes, and even quinine-laced agar gel. Wyrwicka (1996) cites even stranger cross-species eects, in which deer and giraes housed with pelicans sometimes began to eat sh (unpublished work of Y. Kawamura) ; a striking photograph of such behaviour was reprinted by Wyrwicka (1996, p. 41). The process can also teach young animals what foods not to eat. Thus, young monkeys may learn to avoid foods associated with the threat displays of large males (Fletemeyer, 1978). Some apparent examples of observational conditioning can be explained by simpler processes. For example, Morgan (1896, p. 39) pointed out that newly hatched chicks sometimes acquired food preferences by pecking and eating what their mothers pecked and ate. These eects resemble observational conditioning, but may be due only to stimulus enhancement. Moore & Rogers (1997) found that in newborn chicks food rejection movements are as eective as pecking in inducing the young birds to peck things. The pecking (or rejection) movements simply attract the chicks, and call attention to the target. It has long been known that young chicks reexively peck whatever small, nearby objects catch their eyes (Morgan, 1896). Pigeons show similar learning : they can learn to peck an object through seeing it pecked by another (Zentall & Hogan, 1976; Lefebvre & Giraldeau, 1994), or even by a surrogate beak (Neuringer, 1974). These eects have sometimes been called imitation, but that is not the case since they involve instinctive reactions, and also rewards. Again, they could be due to mere stimulus (or local) enhancement, as in the chick studies just described. The cladogram in Fig. 1 shows observational conditioning as one step further from primitive conditioning. The other component, behavioural contagion, is not shown since it is not a learning process. (4 ) Misbehaviour and superstition In 1961 Breland & Breland published a brief and entertaining paper which had serious implications for our understanding of both Pavlovian and instrumental conditioning. It also marked a signicant departure from the anti-biological tradition that had so long aicted the eld. The Brelands had studied with Skinner. But their article marked a clear departure from Skinnerian orthodoxy. Its title, The Misbehavior of Organisms, signalled a new point of view by parodying the title of Skinners (1938) principal work. The article described a pattern of breakdowns of operant technology a number of cases in which animals that had learned to perform operant responses, then began to drift away from them. In each instance the learned reaction was displaced by intrusions of instinctive behaviour.

310 The rst case that they described was disarmingly simple. Hens that had been trained with food reward to stand upon a platform, stopped standing passively and started to dance. The Brelands recognized this dancing as scratching for grain a species-typical food-searching reaction. They hypothesized that many Skinnerians would interpret the dancing as superstitious operant behaviour (Skinner, 1948). It was not clear, however, why every hen should have settled upon the same superstitious response. It should be noted that Skinners (1948) superstition procedure was identical to Pavlovs (1927) procedure of temporal conditioning. In each case, animals are given a reinforcer, typically food, every N seconds no matter what they are doing. Pavlovian animals typically learned the temporal pattern, and began to display species-typical foodrelated reactions (e.g. salivation) towards the end of each interval (Pavlov, 1927). Scratching by hens is also a speciestypical, food-related response. Thus, the most parsimonious explanation of the behaviour of the Brelands hens would be Pavlovian temporal conditioning. It should be noted that pigeons and rats in superstition experiments also typically behave in Pavlovian, rather than Skinnerian, fashion (Staddon & Simmelhag, 1971 ; Hurshman & Moore, 1976; Timberlake & Lucas, 1985). The Brelands went on to describe more dramatic forms of misbehaviour in pigs and raccoons. The pigs had been trained, again with food reward, to deposit articial coins into piggy banks. They quickly learned to do this, but later began to misbehave by tossing the coins into the air and rooting them into the ground. This greatly delayed the receipt of food. Raccoons, after similar training, began to rub the coins together as if washing them. Even more extreme examples were described in three other species (Breland & Breland, 1966). Turkeys trained to deposit coins learned quickly enough, but then the coins began to vanish. When one of the birds was later killed, some missing coins were found, worn virtually paper thin by the action of the gizzard (Breland & Breland, 1966, p. 68). A dolphin was trained with food rewards to play baseball i.e. to retrieve a ball and return it to the batter. Eventually the dolphin ate the ball and died. Similarly, a whale trained to manipulate an inner tube later swallowed it. It had to be purged with a whale-sized dose of mineral oil (Breland & Breland, 1966). Breland & Breland (1961, 1966) described this misbehaviour as instinctive drift. When manipulation of some object is followed by food, they said, it becomes virtually a food surrogate to the animal (Breland & Breland, 1966, p. 68). This was not a new process, however. When an object associated with food begins to be treated as a food surrogate, it conforms perfectly to Pavlovs denition of conditioning (Moore, 1971). The denitions in Pavlovs (1927) book were sometimes ambiguous. But his other writings contain many clear denitions :
Let us take any natural phenomenon that has never had any relation either to food motion or to food secretion. If this phenomenon precedes the act of eating, once or several times, it will later provoke a food reaction ; it will become, so to speak, a surrogate for food the animal moves toward it and may even take it into its

Bruce R. Moore
mouth, if the object is tangible (Pavlov, 1930, p. 209 ; also Pavlov, 1941, p. 47).

See also Pavlov [1913, pp. 973974; 1932, p. 95 ; 1934, p. 187 ; 1941, p. 62 ; 1955, p. 309 (original date 1936). The 1932 and 1934 references are reprinted in Pavlov, 1941, pp. 120, 150]. The Pavlovian nature of misbehaviour was widely overlooked, perhaps because of ambiguities in Pavlovs (1927) book, but also because of the long-standing belief that Pavlovian conditioning did not apply to skeletal responses :
Most of the experiments upon skeletal behaviour which have been [seen as Pavlovian] are capable of interpretation as discriminated operants .It is quite possible on the existing evidence that a strict topographical separation [between operant and Pavlovian conditioning can be based upon] the skeletal-autonomic distinction (Skinner, 1938, p. 112). Most (if not all) skeletal behaviour is operant Much of the plausibility [of Pavlovian explanations of skeletal behaviour] has come from confusion, (Skinner, 1938, p. 115).

The same view was expressed by Mowrer (1947). Such passages led many researchers to believe that Pavlovian conditioning applied only to autonomic responses, and that all skeletal (motor) conditioning was operant. It should be noted, however, that Skinners assertions contradicted not only Pavlovs (1927, p. 17 .) position, but also a great body of data, from Pavlovs laboratory and others, demonstrating classical conditioning of chewing, leg-exion, licking, running, blinking, pecking, and so forth. Skinner oered no evidence for his position ; the dogma was completely unsupported. Perhaps because of their Skinnerian background, Breland & Breland (1961, 1966) did not recognize the Pavlovian process. Nor, apparently, did Kelleher (1962), who attacked their work aggressively. Researchers who challenged prevailing beliefs with the hope of advancing science were frequently excoriated (see Revusky, 1977). Kelleher (1962) remarked that misbehaviour was observed occasionally even in operant laboratories, but that it could usually be controlled. This assertion was correct ; Misbehaviour [read : Pavlovian conditioning] does indeed occur in operant laboratories with rats, monkeys, and pigeons (Moore, 1971). Rats in Skinner boxes, for example, are typically trained to press levers for food. But they often abandon Skinnerian training and begin to chew the lever as if it were food (Fig. 2). In many cases operant apparatus and procedures have been modied to suppress misbehaviour (Moore, 1971). Consider the following passages from early operant-equipment catalogues :
The lever itself is made of stainless steel . The end is rounded and smoothly nished. This, in addition to the thickness and hardness of the lever, prevents the rat from chewing and biting it [Ralph Gerbrands Co., Catalogue, approximately 1967 (unpaginated)]. The PLS-100 Pedal Lever features a continuous curved, stainless steel [surface], designed to discourage the animal from biting . (Scientic Prototype Catalogue, 1970, p. 12).

The evolution of learning

311

Fig. 2. Bar biting in a rat. The animal had been trained to press the lever to obtain food, but it misbehaved (Breland & Breland, 1961) by grasping and biting the lever as if it were food.

In the early 1960s, while a graduate student at Indiana University, I noted that Skinner boxes in the undergraduate laboratories were equipped with thin (1/16 inch ?) brass rods, rather than thick, curved, stainless-steel levers. Strikingly, the brass levers were never chewed ; if food reward had been used Pavlovian conditioning might have resulted in the destruction of the thin brass rods in minutes. But the reinforcer was used with water, not food. The rats might have licked the levers, but never chewed them. Monkeys in Skinner boxes also show misbehaviour. If plastic keys extend into the box, the monkeys often chewed them in the N. J. Mackintosh lab. Most plastic keys are now placed behind small holes in the aluminium wall, to prevent the monkeys from biting them (Moore, 1971, pp. 14 .). With pigeons, misbehaviour is sometimes suppressed (Moore, 1971, p. 26), but more often it is exploited. Much of

the literature on operant conditioning has been generated by pigeons pecking keys for food. In order to misbehave, the birds would have only to keep pecking. It is therefore arguable that much of the pigeon literature has been generated by misbehaviour as dened by Breland & Breland (1961). To evaluate this possibility properly we must begin by examining Brown & Jenkins (1968) auto-shaping eect. The three creatures most often used in Skinnerian research (rats, pigeons, and monkeys) all showed Pavlovian, skeletal behaviour. In each case, however, apparatus, procedures, and/or explanations were modied in order to accommodate Skinners assertion that Pavlovian conditioning did not apply to skeletal reactions. In Breland & Breland (1961, 1966) the CS and US are implicitly paired when contact with some object is followed by food. The process can be better controlled, and more

312 clearly studied, however, if the pairings are explicit, as in auto-shaping. (5 ) Auto-shaping When a signal light (CS) is paired with food or water (US), birds typically show auto-shaping. Objectively, the light is a signal that informs the animal that a particular reinforcer will be delivered at a certain time and place. An animal that learned this relationship might be expected to use the information by approaching the reinforcement site in time for the food or water delivery ; i.e. it might perform an anticipatory response. But many birds and simpler vertebrates typically behave in a simpler fashion. Instead of using the CS as a signal for food or water, they try to eat or drink the signal itself, as shown in Fig. 3 ( Jenkins & Moore, 1973). In responding in this fashion, pigeons treat the CS as a surrogate for the US : the same primitive pattern seen in the misbehaviour eects of Breland & Breland (1961). Similarly, if the CS signals the arrival of a male pigeons mate, he typically approaches and courts the light itself : he bows, coos, nods, oscillates, and displays his tail feathers (Rackham, 1977). The conditioning shown in Fig. 3 could be described as a process of surrogate formation, as opposed to anticipatory responding, signal use, or semiosis (Moore, 1971). In each case the signal is treated as if it were the reinforcer. In proper signal use, by contrast, the animal would react to the CS by approaching the reinforcement site ; it would perform an anticipatory reaction. Only one of Rackhams (1977) 15 pigeons did that, as did only one of several dozen birds in my work of auto-shaping with food or water (B. R. Moore, 1971). (6 ) Dierences between avian and mammalian conditioning We shall see later that movement imitation is very dierent in birds and mammals (i.e. parrots and great apes). Conditioning is also quite dierent. We must begin by distinguishing three levels of mammalian conditioning : auto-shaping (which does sometimes occur), anticipatory responding, and an intermediate process called apo-conditioning. (1974) reported that dogs, cats, and monkeys, Ste pien which ordinarily showed more complex behaviour, reverted to primitive auto-shaping after ablation of certain forebrain structures (medial precruciate or dorsolateral prefrontal , 1974). This suggests that higher, forebrain cortex) (Ste pien mechanisms normally override the more primitive autoshaping mechanism, a not uncommon pattern. Occasionally, mammals in auto-shaping (and misbehaviour ) situations do appear to behave in the primitive, avian fashion, approaching CSs and directing consummatory responses to them (e.g. Pavlov, 1941, pp. 47, 62, 120, 150; Cowles, 1937 ; Kelleher, 1958 ; Breland & Breland, 1961). In many, though not all such cases, the primitive eects may be expressed, at least in part, because the CS objects have been manipulated, creating tactile CSs. An eect described by Peterson et al. (1972) was somewhat more complex. It is widely considered the most reliable way to produce full auto-shaping (CS-approach, contact, and consummatory behaviour) in mammals specically rats.

Bruce R. Moore
The essence of the procedure was to use a retractable lever as a CS. When the levers appearance and disappearance signalled food, rats typically learned to approach, seize and bite it. However, three experiments by Davey & Cleland (1982) found that when water reinforcement was used in place of food, the eect essentially vanished. They showed that the eect required food deprivation, food reward, and a retractable lever. An interesting explanation for this was oered by Thomas Sherrard, who suggested that the rats might simply have pursued the retracting lever as if it were prey. Most commercial retractable levers take 2 s to withdraw. Their velocity is well within the range of rats prey scurrying for cover. T. Sherrard & Moore (unpublished data) compared the eects of levers that retracted in 2 or 10 s. With the 2 s lever, we replicated the eect of Peterson et al. (1972). But with the slower lever the eect was greatly diminished. It was as if rats would pursue, seize, and bite a lever only if it seemed to retract. Thus, the auto-shaped responses of Peterson et al. (1972) appear to have been a combination of conditioned and unconditioned eects. The lever must be associated with food, but it must also seem to ee. At least nine other auto-shaping studies with mammals have been conducted in my laboratory, beginning in 1967 : ve with rats, three with cats, and one with a harbour seal. The mammals sometimes approached CSs, but none ever bit, licked or even unambiguously contacted the lights that signalled food or water (B. R. Moore, unpublished data). Dogs in auto-shaping situations typically behave similarly, approaching, but not contacting, the CS ( Jenkins et al., 1978). Hearst & Jenkins (1974) included such behaviour in what they called sign-tracking. But their term was dened far too broadly for present purposes: it included everything from mere orientation to attempts to consume the CS. We need a term for approaching, but not contacting, which may have been a signicant stage in the evolution of mammalian conditioning. We shall tentatively call it apo-conditioning, where apo stands for approach only, or appetitive only, and also reects the Greek prex meaning separated or detached. In summary, while exceptions do exist, it appears that complex mammals in Pavlovian situations do not usually show true auto-shaping, but rather behave in more sophisticated ways, showing apo-conditioning (approaching the CS without contacting it), or anticipatory responding (approaching the reinforcement site). Which of these forms of conditioning occurs depends upon the species and procedural details. Anticipatory responding is seen in humans and some other mammals. The dierence between auto-shaping and anticipatory responding seems vast. Auto-shaping seems extremely primitive, and can be characterized as surrogate formation, or fetishism. Anticipatory responding, by contrast, implies signal use or semiosis. In fact, the two processes dier in only two ways : response direction, and inhibition of CS contact and consummatory responses. Apo-conditioning therefore may represent the missing link between them.

The evolution of learning

313

Fig. 3. Reactions auto-shaped with food or water reinforcers. The key light on the left was paired with water, and elicited drinking reactions, while that on the right was paired with food, and elicited eating pecks (Jenkins & Moore, 1973). Reproduced with permission of the Journal of Experimental Analysis of Behavior.

It should be noted that auto-shaping can mimic both apo-conditioning and anticipatory responding. When autoshaping is weakly conditioned, as a result of long, inappropriate, or distant CSs, short intertrial intervals, or too few

reinforcers, sometimes only the early (appetitive ) elements of the response are conditioned. The animal thus approaches, but fails to contact, the CS. Weak auto-shaping can therefore mimic apo-conditioning.

314 It can also mimic anticipatory responding. When a visual CS illuminates the reinforcement site, or causes reections near it, the animal can then auto-shape, and approach, these reections instead of the nominal CS which would look like anticipatory responding. This can occur for either of two reasons. First, spatial contiguity with the reinforcer might strengthen the association between the reections and the US. Secondly, the reections would not persist when the CS was o, which would minimize intertrial extinction. The problem often arises with bright CSs and/or darkened chambers (Hearst & Jenkins, 1974, p. 9), or in aquatic environments, as when one conditions shes or molluscs. (7 ) Associations between neutral stimuli The least-constrained form of classical conditioning involves associations between neutral stimuli, i.e. the ability to learn, e.g. that lights follow tones. Since no reinforcer is involved, non-human animals do not always reveal such associations. The associations can be revealed, however, by using an A-B, B-C mediation procedure rst suggested by Thorndike (1898), and now called sensory preconditioning. Here the animal is rst exposed to pairings of two neutral stimuli, A p B. Later, the B stimulus is paired with a reinforcing stimulus, B p C. The reinforcer (C) is typically a shock. If stimulus A is then re-introduced, it may elicit a defensive reaction. It is as if A evoked the memory of B, which had become a danger signal, and which therefore elicited a defensive reaction. In any case, the mediation outcome shows that the animal had formed an association between the two neutral stimuli, A and B. See Mackintosh (1974) for review. Such associations presumably evolved from anticipatory Pavlovian conditioning in the following sequence: from auto-shaping p apo-conditioning p anticipatory responding p associations between neutral stimuli (Fig. 1). IV. HOMING AND NAVIGATION The navigation of salmon is based upon delayed home imprinting. As adolescents they imprint upon chemicals in the water while undergoing smolt transformation, a time of signicant morphological and hormonal change (Hasler & Scholz, 1983). Homing pigeons undergo a similar process. During their juvenile moult, which is also a time of hormonal change, they imprint visually upon their homes (Kramer, 1961), and on other stimuli associated with the home region. When beginning to y, they initially stay within sight of home, using it as a landmark ( pilotage). They then begin to y increasing distances beyond the sight of home, which requires the use of true navigation. But within a few kilometres of home, they revert to visual pilotage. The navigation of normal adult birds is a map-and-compass process (Kramer, 1953, 1961). The word map is not used here in a cognitive way; it simply refers to the animals sense of position. The evidence for this map is that birds released in unfamiliar territory often depart within minutes in a generally homeward direction (Keeton, 1974). Evidence for the existence of the compass, which provides their sense of direction, comes partly from clock-shift studies. ( 1) The primary compass

Bruce R. Moore

Clock-shift experiments (Schmidt-Koenig, 1960; Nauss & Wallra, 1988) show that, on clear days, birds derive directional information from the sun. The sun is not stationary but moves across the sky at an average rate of 15x hx1. Thus, using the sun as an accurate compass requires temporal adjustments. Resetting the birds circadian rhythms by manipulating light/dark cycles interferes with such adjustments and causes the animals to make predictable errors. For example, a 12 h clock-shift causes them to reorient a mean 180x, on average (Schmidt-Koenig, 1960). In fact, clock-shift eects are quite precise. The suns angular velocity may range from zero to innity, depending upon latitude, season, and time of day with extremes occurring in the tropics. Clock-shift eects reect these dierences in solar velocity (Nauss & Wallra, 1988). Such a complex compass, corrected for time and solar movement, is unlikely to have evolved all at once ; there must have been simpler, but adaptive, earlier stages. The obvious possibility is that the sun was rst used as a stationary beacon, perhaps to help the animal hold a steady course during pilotage, or to bring it back within pilotage range. The suns azimuth is essentially stationary early and late in the day, when its movement is primarily vertical. It could be used at mid-day for brief excursions when its movement would be negligible. If the animal lingered for hours, and the solar beacon moved, say 45 or 60x, the bird would not necessarily be lost. The initial ight segment would take it nearer to home. If it periodically redetermined its position, each segment would take it closer and it would essentially travel home along a spiralling path. Such a path would require more time and energy and expose the bird for longer to predation. This selection pressure would act to limit ight times and, eventually, to correct for solar movement. The sun compass might therefore have evolved gradually from pilotage. When the sun is hidden by clouds, but blue sky is visible elsewhere, birds may use polarized light for orientation (Kreithen & Keeton, 1974 a ; Able & Able, 1997). Further, many migratory birds y nocturnally, when the sun is not available. On clear nights these birds (in the northern hemisphere) use the stars as their compass (Emlen, 1970, 1971). ( 2) Magnetic compass or scientic false alarm ? Even under totally overcast skies, pigeons still navigate well (Keeton, 1969). In such conditions, they are unaected by clock-shifts, indicating that they are not using the sun in any way. There must therefore use a fourth compass. It is widely believed that this backup compass is magnetic. This belief originated from Keetons (1971) classic article, which appeared to show that attaching bar magnets to pigeons necks reduced the accuracy of their orientation. However Keeton was unable to replicate his own eect (Moore, 1988). When all of his relevant data from almost 700 birds were analysed, it was clear that magnets had no eect on homeward orientation, homing speed, loss rates, or three other measures of performance (Moore, 1988). The apparent reason for the early, positive, results, as Dr. Mel Kreithen explained to

The evolution of learning

me, was that the birds were not assigned randomly to experimental and control groups. Keetons assistant, who did not want the best birds to get lost, assigned them to the control group. The best birds wore brass bars. Losing the worst birds would not be a problem, however, so they were tted with magnets. Not surprisingly, the best birds did better than the worst which gave the impression that magnets interfered with homing. Elsewhere, at least 10 studies have shown that pigeons could not detect magnetic elds in laboratory settings (e.g. Kreithen & Keeton, 1974 b ; Moore, Stanhope & Wilcox, 1987). The few exceptions to this pattern were seriously awed (e.g. Guyomarch, 1974). In short, magnetic sensitivity in pigeons appears to be a scientic false alarm. How then do pigeons navigate under overcast skies? Grin (1944) revived the simple hypothesis that they might use wind directions, and this deserves to be tested. (3 ) Navigational maps Homing pigeons appear to imprint on their homes in two ways. Pilotage depends upon visual imprinting. But the positional sense, or map, is not visual. When tted with translucent goggles, which prevent them from seeing landmarks, birds appear to navigate normally until within a few kilometres of home, at which point they would normally switch to visual pilotage. They then began to mill about as if searching for the home they can see (Schmidt-Koenig & Schlichte, 1972; Schmidt-Koenig & Walcott, 1978). The map is clearly not visual. Since I have argued above that pigeons cannot detect magnetic elds, it is also not geomagnetic. There have been at least 10 other hypotheses concerning the physical basis of the avian map (Keeton, 1974 ; Papi & Wallra, 1982 ; Papi, 1992). Most are unconvincing, however. One possible exception is that infrasonic sound is the basis of the avian map. A number of researchers speculated that this might be so (Grin, 1969 ; Yodlowski, Kreithen & Keeton, 1977 ; Kreithen & Quine, 1979). Tectonic movements, pounding surf, temperature changes, and many other factors produce low-frequency vibrations in the crust of the earth. Such vibrations can travel great distances, but can be attenuated, ltered, reected, and refracted by geological formations. The vibrational spectrum therefore diers in dierent locations. These dierences are surprisingly stable from place to place (Fox, 1972 ; Frantti, Willis & Wilson, 1962 ; Murphy et al., 1972). The low-level atmospheric infrasounds due to vertical vibrations of the earths surface create consistent patterns. Humans rarely hear such sounds except during earthquakes. But human hearing is poor below 1520 Hz, whereas pigeons can hear down to at least 0.5 Hz (Yodlowski, Kreithen & Keeton, 1977). Further, birds are sensitive even to minute spectral dierences (Konishi & Kenuk, 1975). It therefore seems to be likely that dierent geographic regions sound very dierent to birds, and that they use infrasonic spectral dierences to tell them where they are (Moore, in progress). This hypothesis was greatly strengthened by Hagstrum (2000), who analysed four major pigeon races in which some 50 000 birds were lost. Hagstrum (2000) found that in at

315 least three, and perhaps all four races, many pigeons had crossed the path of the sonic boom generated by the supersonic aeroplane Concorde. The intensity of the boom sometimes reached 128 dB. Hagstrum suggested that many birds would have been permanently deafened, and these were lost, while others, temporarily deafened, might have reached home several weeks later. Such an infrasonic map (sense of position) might well have evolved from pilotage. Young birds might imprint upon not only visible landmarks, but also the sound of home. If infrasonic spectra changed systematically in various directions, then young birds ying near home might detect such gradients, which could allow them to navigate from places where they had not been before. Major geological discontinuities could interfere with such patterns, of course. It is therefore quite possible that both maps and compasses evolved from pilotage.

V. MIMETIC LEARNING IN BIRDS (1 ) Song learning Bird song has several functions, all of which are social. It is used primarily to attract mates and secondarily (in many species) to repel rivals from the singers territory (Thorpe, 1961). As a consequence of these eects it may also minimize cross-breeding. The vocalizations used in most orders of birds are genetically determined rather than learned (Lade & Thorpe, 1964 ; Konishi, 1963). In at least four orders, however, various levels of song learning have been reported (Nottebohm, 1972). Hummingbirds (Apodiformes) have been reported to show rudimentary song learning (Baptista & Schuchmann, 1990). Toucanets (Piciformes) do somewhat better (Wagner, 1944). Parrots (Psittaciformes) are capable of impressive vocal learning (Cruickshank, Gauthier & Chappius, 1993). But the process is most conspicuous, and best understood, in true songbirds (oscine Passeriformes). Some oscine species, e.g. chanches (Fringilla coelebs), show little or no song learning (Thorpe, 1958). At the other extreme, the songs of marsh warblers (Acrocephalus palustris) are almost entirely learned (Dowsett-Lemaire, 1979). These four orders of birds are not closely related (Sibley & Ahlquist, 1990), and one must be cautious in generalizing between them. Song learning is not unitary ; it is a composite of two distinct processes, one sensory, one motor, which sometimes occur months apart (Konishi, 1965 ; Marler, 1981 ; Marler & Peters, 1981 ; but see also Nottebohm, 1991). In many species, the sensory process occurs in an early sensitive period (Thorpe, 1961 ; Marler & Peters, 1981; Dowsett-Lemaire, 1979). Here the bird imprints on the songs of neighbouring males of its species, and thus learns what its own songs should sound like. This sensory learning can be said to provide, or modify, an auditory template (Marler, 1964 ; Konishi, 1965) which is later used to guide motor learning. Through motor learning, the bird learns to produce songs similar, though not quite identical, to those remembered from the sensory-learning stage.

316 The evolution of song learning has been regarded as problematic. It has proven dicult to identify a plausible linear progression through which it might have arisen (Nottebohm, 1972). But perhaps one should not look for a single progression. Since song learning is a composite of two separate processes (auditory and motor) it presumably evolved from two separate sources. The sensory component of song learning appears to be a specialized sort of auditory imprinting (cf. Thorpe, 1961; Nottebohm, 1972). The limits of the motor-learning process are dicult to determine, since some improvement may arise from simple maturation, and some from socially induced vocal exercise (Thorpe, 1966, pp. 258259). But if we accept Marlers (1976) model, the motor learning seems to resemble Thorndikian conditioning in that it selects between related species-typical responses. It diers, however, in that there is no external reinforcer. We shall refer to this process as avian Thorndikian learning, with the understanding that there may be important dierences among dierent orders of birds. Since song learning evolved from two processes, auditory imprinting and skill learning, Fig. 1 shows song learning arising from auditory imprinting, with an L-shaped gure indicating the second source, skill learning. (2 ) The neural basis of song learning in oscines Song learning in oscines is better understood at the neural level than most other forms of vertebrate learning and provides a useful model for understanding several others. An adequate summary of the relevant literature would be very lengthy. It would also be redundant with several published reviews (e.g. Marler, 1991 ; Nottebohm, 1991). We shall therefore focus on only one central point. In birds whose songs are entirely instinctive, singing is controlled by the twelfth cranial nerve, which originates in the hypoglossal nucleus, low in the brainstem. In oscines capable of song learning, this nucleus is innervated by a descending motor tract from three serially connected forebrain nuclei : the nucleus interfacialis (NIf ), the hyperstriatum ventrale pars caudale (HVc) (sometimes called the high vocal centre ), and the nucleus robustus archistriatalis (RA) (Nottebohm, 1980 ; McCasland, 1987). The HVc and RA have been extensively studied. They are known to be absent in at least some non-song-learning species (Konishi, 1985). But in male roller canaries (Serinus canarius), which learn new songs annually, these centres not only exist, they also greatly increase in size during each learning season (Nottebohm, 1981 a). The precise amount of swelling is apparently correlated with the level of song learning (Nottebohm, 1981 b ; Baker, Bottjer & Arnold, 1984). These seasonal changes occur in canaries (S. canarius), which learn new songs annually, but not in white-crowned sparrows (Zonotrichia leucophrys), which do not (Nottebohm, 1980; Baker et al., 1984). Destruction of the HVc does not silence birds but, more interestingly, makes them revert to juvenile (instinctive) subsong (Nottebohm, 1981 b). In short, the sub-song, which is innate, is controlled by a lower, and phylogenetically older nucleus, but the learning of adult songs is controlled by

Bruce R. Moore
higher (forebrain) nuclei, which send eerents to, and modify the action of, the lower centre. We have seen that a similar over-riding of lower (older) brain centres by higher (more recently evolved) ones was also important in the , 1974). evolution of conditioning (Ste pien ( 3) Vocal mimicry Vocal mimicry in birds is simply an extension of song learning (Slater, 1983). Primitive song learning is highly constrained. Birds of some species can reproduce faithfully only sounds of a certain tonal quality, duration, or phrasing (Thorpe, 1961 ; Guttinger, 1979 ; Marler, 1981). In such cases, the song learner simply polishes an innate pattern ; the concept of mimicry is not needed. But many song-learning birds are far less constrained, and can learn phrases from neighbours, or foster parents of other species. Toucanets (Piciformes) have been described as good mimics, able to copy the songs of neighbouring quetzals, motmots, and trogons (Wagner, 1944). Many oscines, such as mocking birds (Mimus polyglottos) and marsh warblers (Acrocephalus palustris) also have this ability. The songs of 20 marsh warblers studied by Dowsett-Lemaire (1979) contained identiable phrases copied from an average of 76 other species. This less-constrained learning is essentially vocal mimicry. The least constrained learners, such as mynahs (Gracula religiosa), lyre birds (Menura superba), and grey parrots (Psittacus erithacus), are able to copy not only the sounds of many other birds, but also mechanical sounds and human words and phrases. Thus, avian vocal mimicry is widely believed to have evolved from song learning (Slater, 1983 ; see also Witchell, 1896) through relaxation of constraints. More than 250 species of psittacine, oscine-passerine, and piciforme birds have been reported to show vocal mimicry (Kroodsma, 1982; Kroodsma & Baylis, 1982). Moreover the pattern of occurrence corroborates the assumption about its origins : Virtually every author who has reviewed the subject of vocal mimicry has noted that only birds [that] learn their songs [are able to mimic] (Baylis, 1982, p. 73). Thus, we infer the evolutionary progression shown in Fig. 1 : from skill learning p auditory imprinting p song learning (sensory component) p vocal mimicry. Vocal mimicry required two processes : auditory imprinting plus skill learning (Fig. 1). Note that the motor-learning component of vocal mimicry diers slightly from that of song learning in that it requires the acquisition of novel, rather than species-typical, responses. Many oscine and psittacine birds, e.g. crows (Corvus brachyrhynchos) and African grey parrots, are also capable of skill learning. They can acquire novel, non-vocal, motor responses without external reward. The parrot used in my own imitation research (Moore, 1992) demonstrated an amusing form of skill learning. The light cycle in his room was controlled by a standard wall timer (Paragon model 4001-00). The parrot taught itself to open the timers heavy sheet-metal door, which required far more force than one might expect from a 450-gram bird. It then swung the door open, and delicately unscrewed the two small thumb screws that controlled light onset and oset (540 threaded screws

The evolution of learning

with 5 mm caps). The bird learned to perform this act swiftly and eciently. Once removed, the thumb screws were dropped to the oor and ignored. Parrots exhibit tool use (Boswall, 1977, 1983 a, b), as do common crows (P. D. Cole, in preparation). This suggests the progression : from avian Thorndikian conditioning p operant conditioning p skill learning p tool use (Fig. 1). In parrots, and certain song birds, there may be two levels of vocal mimicry. The basic form may be associative. Many passerine birds, for example, copy songs emanating from neighbours of their own (imprinted) species. Some song birds and parrots, however, can copy isolated sounds. Moores (1992) parrot, for example, repeatedly mimicked a swear word that it had heard only once, through a closed door. (4 ) Movement imitation in birds Most species of birds are incapable of movement imitation. Thorndike (1898) tried and failed to nd imitation in chicks. The process has repeatedly been claimed for pigeons because they learn to peck some object after seeing it pecked by another (Zentall, Sutton & Sherburne, 1996; Palameta, 1989). But these studies violate all three elements of the denition of imitation (q.v.) : the pecking is typically reinforced, it is scarcely a novel response, and very few of the dozen simpler processes that can mimic imitation are ever ruled out. The only birds so far shown to be capable of true movechoro, a male ment imitation are parrots (Psittaciformes). Ok African grey (Psittacus erithacus) studied by Moore (1992) proved a master of imitation. The bird copied at least 14 dierent human movements, e.g. waving, showing his tongue, head shaking, nodding, and so forth. Further, each response was accompanied by a mimetic word or phrase. All responses were scored on closed-circuit television. Only two imitative responses were ever performed in the presence of humans. No subsequent occurrences of either were counted. The data were therefore uncontaminated by any possibility of accidental social reinforcement. During the period when it acquired the 14 imitative movements, the parrot learned to mimic 300 words. It cannot be mere coincidence that the avian species most renowned for vocal mimicry should also be able to imitate movements. But while these processes are clearly related, they dier in two basic ways. First, vocal mimicry involves vocal, and movement imitation non-vocal muscles. Secondly, vocal mimicry involves the matching of auditory models, whereas movement imitation involves matching visual models. Whichever of these two changes was rst to evolve would have produced a new form of learning, intermediate between vocal mimicry and movement imitation. It is unlikely that the use of visual models arose rst, for there is no obvious way that they could be copied using auditory muscles. But the use of non-vocal muscles could have come rst, for animals can, in principle, use some of these muscles to copy auditory models : they can strike things with their wings or beaks to mimic percussive sounds. One can therefore predict the existence of a new form of learning, percussive mimicry, dened by the use of movements to copy percussive sounds (Moore, 1992, 1996).

317 The parrot appeared to show this process: it mimicked human knocking sounds by drumming on various objects with the top of its beak, and even the top of its head (Moore, 1992). P. D. Cole ( personal communication, 1999) observed similar percussive mimicry in a sun conure (Aratinga solstitialis). (5 ) Cross-modal imitation choro, went beyond movement The African grey parrot, Ok imitation. He also showed cross-modal imitation by copying ve head movements which he could not see himself perform, including three shown in Fig. 4 ( nod, shake, and heads up ). This process presumably evolved from movement imitation, though the nature of the transformation is unclear. One may therefore suggest the following evolutionary path for psittacine imitation (Moore, 1992, 1996): from auditory imprinting p song learning p vocal mimicry p percussive mimicry p visual movement imitation p cross-modal imitation (Fig. 1). The ve transitions between auditory imprinting and cross-modal imitation could be described as learning to reproduce imprinted sounds, a reduction of constraints (allowing reproduction of other sounds), a shift from vocal to non-vocal muscles (for reproduction of percussive sounds), a shift from auditory to visual models, and copying visual models without visual feedback.

VI. INSTRUMENTAL LEARNING AND TOOL USE (1 ) The Pavlovian Law of Eect The most primitive sort of instrumental learning in vertebrates is a special case of Pavlovian conditioning. When a researcher rewards a pigeon with food for pecking a key, the key becomes associated with food, and therefore tends to elicit pecking as a Pavlovian conditioned response (CR). If these elicited pecks earn further reinforcement, the Pavlovian association will be further strengthened, and the key will become even more likely to elicit pecking. This positivefeedback loop causes the animal to repeat the rewarded response (Moore, 1971, 1973). Thus, a primitive form of instrumental (eect) learning can arise as a corollary of purely Pavlovian conditioning : it is a Pavlovian Law of Eect. This primitive process is highly constrained. It is limited, in the rst instance, to reinforcement-appropriate, species-typical reactions. But these are precisely the sorts of reactions that animals learn most readily. Moore (1973) examined six procedures for teaching pigeons to peck keys. (1) The experimenter may wait and reinforce only spontaneous pecking. (2) He or she may also shape the response through successive approximations, rewarding the bird rst for orienting toward the panel containing the key, then approaching it, then for approaching the key itself, then for pecking it lightly, and so forth. This method encourages pigeons to zoom in on the key, rst associating the entire panel, then the key area, and then the key itself, with food. (3) It is important to note that autoshaped pigeons go through precisely the same progression as those conditioned through successive approximations. In

318

Bruce R. Moore

Fig. 4. Imitation of human movements by a parrot. Each imitative movement was accompanied by a labelling phrase. While performing the various movements the bird said : Ciao (A and B), Look at my tongue (C), Turn (D), Peanut (E and F), Whoops, dropped the peanut (G and H), [Head] shake (I and J), Nod (K), and Heads up (L) (Moore, 1992). Reproduced with permission from E. J. Brill Publishers.

each case the food-getting response is expressed more fully as a conditioning process. It therefore seems likely that while the experimenter shapes the pigeon, the pigeon reciprocates by shaping the experimenter. (4) Observational conditioning (q.v.) provides a sixth way to teach pigeons to peck. As mentioned earlier, this process is also a form of Pavlovian conditioning. Two other methods for training pigeons include (5) taping grain to the key, or (6) pre-training the bird by letting it peck grain through torn places in white paper discs designed to resemble response keys. These techniques work by creating associations between response keys (or mock keys) and food. It is interesting that these techniques were developed by Ferster & Skinner (1957), since it was fundamental to Skinners (1938) theory that operant responses were never elicited. A seventh process may also redirect pecking from the food site to the response key (P. Seraganian, personal communication, 1970).

The key pecks recorded in operant situations are like those obtained in Pavlovian (auto-shaping) studies. Pecks rewarded with food closely resemble eating reactions (Smith, 1967; Wolin, 1968 ; Stanhope, 1989), and those with water resemble drinking (Wolin, 1968 ; Stanhope, 1989). The similarity is quantitative as well as qualitative : Stanhope (1989) found virtually identical responseforce distributions in auto-shaped and operant pecks ; this was true for both food and water reinforcers. Pavlovian eect learning can even generate a degree of response selection (Bindra, 1972 ; Moore, 1971, 1973). For example, a pigeon might learn to peck at the key itself, or an image projected onto part of the key, or the rim surrounding it (Fig. 1 in Moore, 1973), or the key plus some nearby feature, like the athead screw that holds the key in place (Moore, 1971, pp. 130132). Such stimulus selection aects the topography, force, and duration of the Pavlovian response, thus producing response dierentiation or selection which mimics true instrumental learning.

The evolution of learning

Similar Pavlovian eect learning may occur in many taxa. Wells (1964) pointed out that, except for arthropods, invertebrates typically lack joints and proprioceptors. Without such receptors, there would be limited central knowledge of limb positions, making it dicult to learn novel (operant) movement patterns. Thus, when such animals show what appears to be instrumental learning, Pavlovian eect learning should be suspected. This process did not have to evolve from Pavlovian conditioning. It is simply a corollary, or special case of it. (2 ) Thorndikian conditioning Thorndikian conditioning (Thorndike, 1898) involves species-typical, reinforcement-appropriate responses, elicited by environmental stimuli. Such responses, when reinforced, become more likely to recur in the particular situation. The process is often dicult, if not impossible, to distinguish from Pavlovian eect learning. The two processes involve the same sorts of reinforcers, and reinforcement-appropriate species-typical responses. Further, we have seen that Pavlovian procedures can produce response dierentiation and selection. Although the procedures dier, with reinforcers following particular stimuli (CSs) in the Pavlovian case, and particular responses in the Thorndikian, we have seen that Thorndikian procedures often produce Pavlovian associations between manipulanda and reinforcers. Similarly, Pavlovian procedures could, in principle, produce superstitious associations (Skinner, 1948) but between stimuli and reinforcers, not stimuli and responses. A number of other similarities between Pavlovian and instrumental learning have been listed by Miller & Konorski (1969), Kimble (1961, pp. 81 .), and others. Pavlovian acquisition, extinction, spontaneous recovery, generalization, discrimination, external inhibition, inhibition of delay, and disinhibition all have close counterparts in instrumental learning. Skinner (1938, pp. 97 .) denied that disinhibition occurred with instrumental responses, but Brimers (1970 a, b) more systematic work showed robust eects. The many similarities between Thorndikian and Pavlovian processes, and the frequent diculty of discriminating between them, might suggest that the processes are the same. But that is unlikely. The Thorndikian process can be viewed as a more precise, and less constrained, mechanism for selecting responses from within the Pavlovian repertoire (Schlossberg, 1937; Stokes & Balsam, 1991). Further, operant and cross-drive conditioning could easily have evolved from Thorndikian, but not Pavlovian conditioning at least in mammals. (3 ) Cross-drive conditioning Species-typical responses not appropriate to the reinforcer are often learned slowly, if at all, and extinguished rapidly. For example, a chick that was released from a box when it pecked his own feathers, and a cat that was released when it licked itself. Both failed to repeat these actions. Rather, their future movements were increasingly awkward and unnatural (Thorndike, 1898, pp. 2728). Konorski (1967) provided similar results.

319 The situation is dierent in certain higher mammals. Breland & Breland (1966) described several instances of such learning in dolphins (p. 87). Cross-drive conditioning, as they called it, was repeated precisely. The mechanism might well have evolved from Thorndikian conditioning through an expansion of possible neural connections. (4 ) Operant conditioning Some animals, e.g. dogs, raccoons, and primates, are able to transcend the Thorndikian process, and use novel (nonspecies-typical) responses. The emergence of such responses was not a trivial development. Indeed, Sherringtons Integrative Action of the Nervous System (1906) concludes by saying that a major function of the cerebral cortex is to allow species-typical responses to be modied [with] seeming independence [from] external stimuli . The controlling centres can pick out from an ancestrally given motor reaction some one part of it, so as to isolate that as a new separate movement (Sherrington, 1906, p. 386; see also Schlossberg, 1937). This is presumably a third example of higher neural sites modifying lower processes. It also clearly suggests that operant conditioning evolved from Thorndikian. Skinner (1938) stressed that his operant responses were not elicited by known stimuli. Many of his followers spoke of arbitrary operants i.e. novel reactions. Thus, although Thorndikian and operant conditioning are often treated as identical, it is essential to distinguish between them. It is likely that Skinners (1938) concept was inspired by Sherrington (1906). Although most of the literature on mammalian instrumental learning involves species-typical responses (Breland & Breland, 1966; Bolles, 1970 ; Vines, 1981), some orders of mammals, and a few of birds, are capable of learning novel reactions, and exhibiting true operant conditioning. (5 ) Skill learning and tool use Novel responses learned without external reward are called skill (or motor) learning. Learning to play a violin, operate a lathe, throw a curve ball, or sail close to the wind involves skill learning. Like operant conditioning, the process involves learning novel responses, but it diers in not requiring external reinforcement. That does not imply that all reinforcement is irrelevant, however. Subjectively, when one makes progress in such a skill, the outcome is satisfying, and one tries to repeat the successful movement. We may therefore characterize skill learning as a higher form of operant conditioning that requires implicit, rather than explicit, reinforcement. Here again, what probably happens is that Sherringtons (1906) controlling centres isolate small elements of speciestypical movement patterns, utilize them with seeming independence from external stimuli, and even arrange them into novel sequences if required. In some mammals (e.g. humans), the repertoire available for skill learning includes almost every micro-response of which the skeletal muscles are capable. Very few such muscles (e.g. the stapedius) are exempt. In humans the

320 process generates the subtle movement patterns used by surgeons, artists, musicians, craftsmen, dancers, and athletes. Some species of birds, especially crows, are capable of tool use. Some examples are instinctive, but most require skill learning (Beck, 1980 ; Boswall, 1977, 1983 a, b ; Hunt, 1996; Tebbich et al., 2001; Terkel, 1996 ; P. D. Cole, in preparation). The most sophisticated forms of mammalian tool use include termite-shing by chimpanzees on Gombe Stream (Goodall, 1971), and nut-cracking by Ivory-Coast (Tai) chimps (Boesch & Boesch, 1983). These cases are as dicult as complex human imitation [see Sherrington in Section VIII (4)]. Such forms of imitation require three processes : rst, evolution requires putting through (see section VI. 7) and movement imitation which has occurred in great apes and humans. Second, great apes must reach a certain age before they can imitate. Third, the chimpanzees must also use complex skill learning, which sometimes take years to perfect. An even higher form of skill learning that depends upon human language will be discussed later (see Section X, Image-mediated learning). (6 ) Hybrid conditioning Logan (1969) described several cases of everyday Pavlovian behaviour. One example was reaching for the keys to ones car while still a great distance from it. Another was a sherman jerking a line prematurely, at rst nibble. A third was a golfer raising his head too early, as if to watch the balls ight before hitting it (Logan, 1969, p. 46). None of these movements were purely Pavlovian, however. Learning originally to use the key, jerk the line, or club the golf ball would have been forms of instrumental learning. Sometimes the anticipatory response would be harmless, and sometimes not. Early Russian researchers called this anticipatory instructed conditioning. A simpler term might be hybrid conditioning. Later, having learned the responses, stimuli were anticipating two movements: association between neutral stimuli plus skill learning and tool use. Logans descriptions involved only humans. I attempted the same with pigeons in the early 1960s, but the eect was quite dierent : the tail-end of the distribution was markedly shortened, but anticipatory responding never occurred. Logans everyday process is common in humans, but it is not clear which other species show it. Finally, hybrid conditioning requires two processes : skill learning and associations between neutral stimuli (see Fig. 1). (7 ) Putting through Putting through (Moore, 1996) appears to be related to skill learning, from which it diers by beginning with a passive, rather than active, response (Sherrington, 1900). In putting through, a teacher simply guides the passive learner through the desired behaviour. Humans use putting through in a variety of teaching situations. Parents move the hands or feet of their infants to teach them to play pattycake, wave goodbye, or ride tricycles ; they mould the hands of slightly older children to teach them to hold pencils and

Bruce R. Moore
form letters. Again, the rst published work on teaching violin recommends, If the beginner does not succeed in holding the violin [properly] then arrange his hand (Mozart, 1951). The method is also used in dance instruction (Schlaich & DuPont, 1993). True putting through, as described above, involves novel responses, passively formed, and not explicitly reinforced. Gardner & Gardner (1969) routinely used this method when teaching Ameslan signs to the chimpanzee Washoe, and Fouts & Rigby (1977) pronounced it more useful than imitation for that purpose. Fouts, Fouts & van Cantfort (1989) reported anecdotally that Washoe herself once used putting through while teaching signs to an adopted infant. Premack conrms that great apes sometimes use putting through, teaching others with passive guidance placing the others body into the desired position a procedure humans likewise use when teaching dance, sports, and so forth (Premack, 1990, p. 755). Putting through is clearly important in humans, and apparently in great apes, but it has not yet been proven to exist in either old- or new-world monkeys. The method may be used by traditional monkey trainers in Thailand (see Bertrand, 1967), but perhaps only in conjunction with reinforcement (avoidance conditioning). Westergaard & Fragaszy (1987, p. 162) observed one incident of possible putting through in new-world monkeys. In that instance, the process appeared to be initiated by the learner, an 8-monthold capuchin (Cebus appella). There is need for further experiments with putting through and movement imitation in both old- and new-world monkeys ( perhaps baboons and capuchins). True putting through has not been shown in animals other than primates. Thorndike (1898) tried in vain to train cats by this method. I would take the cats paw and with it pull down the loop. The cat would then [escape from the puzzle box] and eat the sh. This would be done over and over (Thorndike, 1898, p. 68). The method proved entirely unsuccessful, however. Miller & Konorski (1969) initially believed that dogs could learn passive paw movements through a form of putting through but their assertion was later retracted. Konorskis nal position (Konorski, 1967, Ch. 11; Konorski, 1969) was that putting through was helpful only when it served to elicit a useful reexive reaction. Konorskis procedures also utilized explicit reinforcement, which is not required by true putting through. The combination of response elicitation and reinforcement is sometimes extremely successful (Denny, 1970; Bertrand, 1967), but does not constitute a new form of learning ; it is simply an ecient technique for producing Thorndikian conditioning. Thus, true putting through (involving passive, novel, non-reinforced responses) has so far been proven to exist only for humans and great apes. The process resembles skill learning insofar as novel responses are learned without external reinforcement. It diers in that the response is passive rather than active, which means that kinaesthetic stimuli cannot be used in response matching. That is, if a chimpanzee is taught the sign for drink by putting through, it sees its hand make the response, and receives some proprioceptive feedback, but it

The evolution of learning

does not experience kinaesthetic stimuli which are critical in skill learning (Moore, 1996). We shall examine this process further in Section VIII. So far, we may hypothesize the following progression : from Thorndikian conditioning p cross-drive conditioning p operant conditioning p skill learning p tool use p putting through (Fig. 1). Tool use may or may not be simply a special case of skill learning. The transformations required for the rest of the progression are these : an expansion of repertoire, a change from explicit to implicit reinforcement, and a loss of kinaesthetic cues. (8 ) Omission training Omission training is the positive-reinforcement analogue of punishment. The response is not punished by delivery of an aversive reinforcer but by withholding a positive response. The process is often used by animal trainers and by human parents, spouses, and employers, who withhold privileges, favours, approbation, or treats when displeased by some behaviour. Like many learning processes, it has been given many names. It was called Case IV of Type-II conditioning by Miller & Konorski (1969), omission training by Sheeld (1954, 1965), auto-maintenance by Williams & Williams (1969), and dierential reinforcement of other behaviour by Ferster & Skinner (1957). We shall use Sheelds term. We must distinguish between two parallel omission eects. An animal punished by omission of a positive reward might respond less because it was associated with nonreinforcement with either external or proprioceptive stimuli. The former would not be a new form of learning; it would simply be Pavlovian extinction. The latter would arise from a response-reinforcement contingency, and would therefore reect true instrumental learning. In principle, almost any level of Pavlovian conditioning might have been the evolutionary source of omission learning. We have, at present, very little data on any aspect of the process, and perhaps none that allow us to distinguish between true proprioceptive and Pavlovian extinction. Lacking the necessary data, we shall tentatively treat omission learning as a sister of true punishment, and one step further of auto-shaping. Omission training has most often been used or misused as a control group to distinguish between Pavlovian and instrumental learning (Sheeld, 1954, 1965 ; Patten & Rudy, 1967 ; Wasserman, 1974; Williams & Williams, 1969). When the auto-shaping eect was new, for example, Kirby (1968), Williams & Williams (1969), and Wasserman (1974) all employed omission controls. Instead of letting food (or heat) follow every key light, as in simple autoshaping, they let the reinforcer follow the light unless the bird responded to it. There are at least four major problems with such controls. First, had the animals learned to suppress pecking, that would have shown only that there was evidence of an instrumental contingency after the contingency had been introduced (Mackintosh, 1983). That is, superimposing an instrumental contingency upon what had been a purely

321 Pavlovian procedure introduces a Heisenbergian problem: it changes what it was meant to assess. The second diculty is that introducing an omission contingency also introduces partial reinforcement, which, in itself, may reduce the level of Pavlovian responding. Yoked procedures, which are meant to control this control, introduce a multitude of other confounding variables (Moore & Gormezano, 1961; Church, 1964, 1989 ; Church & Getty, 1972; Church & Lerner, 1976). The third problem is that omission controls can produce not response suppression, but response re-direction (Kirby, 1968 ; Barrerra, 1974) which, again, requires higher-order control groups. The fourth problem arises from evolutionary analysis. If, as Fig. 1 implies, Miller & Konorskis (1969) four cases of instrumental conditioning (Thorndikian learning, omission training, avoidance conditioning, and punishment) are distinct processes, each of which evolved independently, then the presence or absence of an omission eect would tell us nothing whatsoever about the presence or absence of Thorndikian conditioning. The processes would be independent. These four problems mean that omission training, while important in its own right, is quite useless as a control for dierential diagnoses.

VII. INSTRUMENTAL CONDITIONING WITH NEGATIVE REINFORCEMENT (1 ) Vicious-circle behaviour When Morse, Meade & Kelleher (1967) shocked monkeys for struggling, the unfortunate creatures struggled more and not less. Their struggling brought further shocks, which caused even more struggling, and so on. Many analogous eects occur in humans. For example, ridicule tends to strengthen, rather than weaken, such responses as stuttering, bed-wetting, clumsiness, stage-fright, and sexual disfunction. Eects of this nature are especially likely when unwanted behaviour is punished by negative reinforcers that elicit the same reaction (Church, 1963). (2 ) Avoidance conditioning We must distinguish between three levels of avoidance conditioning. The rst is purely Pavlovian ; the other two comprise true avoidance conditioning. Pavlovian eect learning occurs with negative as well as positive reinforcers (Moore, 1973). If, in an avoidance situation, some environmental cue elicits an instinctive defensive reaction, and if that reaction allows the animal to escape or avoid the noxious stimulus, then the eliciting cue may become associated with safety ; it therefore becomes more likely to be approached, to elicit the same response, and so on. This Pavlovian process explains, in part, the preeminence of species-typical defensive reactions in avoidance learning situations. Beyond this Pavlovian process, there is true (instrumental) avoidance conditioning, the essence of which was described in 1969 by Miller & Konorski (Case III of their Type-II conditioning). We should, however, distinguish between two

322 levels of this instrumental process, according to whether the selected responses are species-typical or novel. Paralleling the evolution of Thorndikian learning, some species are able to repeat species-typical defensive responses which avoid negative consequences. While such responses come from the repertoire potentiated by Pavlovian conditioning, they are selected on the basis of the avoidance contingency. The selected response may allow the animal to escape from the noxious stimulus, or from Pavlovian stimuli paired with it in which case the animal may avoid the noxious stimulus entirely. This form of true avoidance conditioning was described in Bolles (1970) classic article on species-specic defensive reactions (SSDRs). Some animals are able to go beyond this, however. They can transcend the species-typical repertoire and learn to use novel responses in avoidance situations. Humans (and great apes) can learn to ride bicycles without falling. Equestrians, skaters, and skiers also learn to minimize falls. Sailors learn to manoeuvre fast dinghies without turning over. These skills require a mixture of novel responses and species-typical balancing (righting) reactions. Humans can also learn to use clutches smoothly so that cars will not stall or buck, to land aeroplanes without bouncing, or dine with royalty without giving oence. At these levels, one learns novel responses that deliver one from fearful, uncomfortable, or embarrassing situations, as described in two-factor theory (Mowrer, 1947 ; Solomon & Wynne, 1953, 1954). The evolutionary progression here was presumably a transition from Pavlovian auto-shaping to avoidance conditioning (SSDRs) and thence to avoidance conditioning (novel responses) (see Fig. 1). The rst transition requires gaining the capacity to select from the Pavlovian repertoire on the basis of instrumental (avoidance) contingencies. That is, whereas the Pavlovian process potentiates a subset of the defensive repertoire elicited by a particular stimulus (CS), true avoidance conditioning potentiates particular responses from within that subset. We shall examine this more closely below. The second transition requires the ability to transcend the species-typical repertoire and utilize novel responses. Physiologically, this may involve the motor cortex, rather than subcortical centres. (3 ) Nested mechanisms ? One may conceptualize the progression from sensitization to Pavlovian to avoidance conditioning as a series of increasingly precise and complex ways of ne-tuning the animals species-typical repertoire to its particular environment. Sensitization potentiates responses from within the reinforcement-appropriate subset of the species-typical repertoire, e.g. species-typical defensive reactions. Pavlovian conditioning potentiates a subset of that subset just those species-typical, defensive responses elicited by a particular stimulus (the CS). Finally, basic avoidance conditioning potentiates a subset of that subset of a subset a particular species-typical defensive reaction, elicited by a particular stimulus. The three mechanisms are therefore nested, as shown in Fig. 5. We may therefore hypothesize the following major progression (suppressing intermediate steps) : from Sensitization p Pavlovian conditioning p avoidance

Bruce R. Moore
conditioning. The progression would be highly adaptive and physiologically straightforward. ( 4) Punishment The literature on punishment is anything but simple (Estes, 1944 ; Church, 1963 ; Solomon, 1964 ; Dunham, 1971). To begin with, the process can be mimicked by at least four others. Most fundamentally, merely introducing (or discontinuing) an aversive stimulus such as shock profoundly changes the stimulus context in which the animal operates (Church, 1963, Estes, 1969). This can disrupt ongoing behaviour, or restore an earlier situation and thus bring back an earlier mode of responding. Secondly, the mere occurrence of the negative reinforcer quite apart from any contingency can lead to sensitization, thus potentiating species-typical defensive or aggressive reactions, and inhibiting other sorts of behaviour. In some situations, non-contingent shocks are as eective as response-contingent ones ( punishment) in suppressing ongoing reactions (Estes, 1944). This outcome may be especially likely when very strong shocks are used, in which case both procedures may produce near total suppression (Church, 1963). Third, Pavlovian eect learning (Moore, 1973) can mimic true punishment. If some environmental stimulus elicits a reaction incompatible with the one that brings punishment, then shock may be avoided, and the stimulus may become associated with safety. The animal may therefore approach it again, thus avoiding further shocks, and so forth, making the stimulus ever more attractive. This amounts to a Pavlovian form of avoidance conditioning, which suppresses the (incompatible) target response, thus mimicking true punishment. Fourth, and most importantly, the punishing stimulus can also enter into Pavlovian associations with the manipulandum or surrounding cues. If a rat is shocked for pressing a lever, it may associate the lever with shock. Estes (1944, 1969) pointed out that such associations could produce conditioned emotional responses, or CERs (Estes & Skinner, 1941), and also withdrawal responses, either of which could suppress ongoing instrumental behaviour especially behaviour based upon positive reinforcement. We might call these exteroceptive conditioning eects Pavlovian punishment. True punishment might easily have evolved from that process. The animal would need only to associate responseproduced cues with shock. Such cues might be tactile, kinaesthetic, or other proprioceptive stimuli. Once such associations were formed, then whenever the animal started to make the response, that would restore the cues associated with shock, and the response could be inhibited (Mowrer, 1947; Church, 1963). So true punishment could easily have evolved from Estes (1944) Pavlovian punishment : the CS would simply be proprioceptive rather than external, and the sequence would have been from Pavlovian conditioning (auto-shaping) p Pavlovian punishment p punishment (Fig. 1).The fact that punishment can be mimicked by so many other processes makes it dicult to identify. In some cases control groups can help with diagnosis. It would also

The evolution of learning

Defensive responses

323

R1

R2

R3

Rj

Rj+1

Rn

Sn
S1

RR

S2

S3

Si

Av

Pa

Sn

Fig. 5. Sensitization Sn, Pavlovian conditioning Pa, and avoidance conditioning Av act within nested subsets of the species-typical repertoire. Sensitization potentiates a subset of the entire repertoire. Pavlovian conditioning potentiates only a subset of that subset, e.g. defensive reactions involving a particular (signalling) stimulus. Avoidance conditioning potentiates a subset of a subset of the subset, e.g. a particular defensive reaction, elicited by a particular stimulus.

be helpful, as pointed out by Hunt & Brady (1951, 1955), to break with the maladaptive tradition of not observing ones animals. If observation showed undirected responses, such as freezing and defecating, or crouching and urinating, this might signify sensitization, or conditioning to a diuse CS. Abortive lever pressing, by contrast, would not be expected from those processes, or from avoidance conditioning ; it would indicate either Pavlovian or true punishment. Animals might be governed by dierent mechanisms at dierent stages of learning. They might, for example, retrace evolutionary history, showing rst sensitization, then Pavlovian punishment, and, still later, true punishment. Note that true punishment, as dened above, is both Pavlovian and instrumental. The underlying stimulusreinforcer association is clearly Pavlovian, but it implies the learning of a response-reinforcement contingency. This duality bridges the traditional dichotomy between the two great families of conditioning. We shall encounter other processes that also belie what had seemed to be clear dichotomies. Omission learning, for example, is isomorphic to punishment, and therefore, also, both Pavlovian and instrumental. Similarly, putting through appears to be both instrumental and imitative.

VIII. MOVEMENT IMITATION IN MAMMALS (1 ) Simpler processes that mimic imitation It is generally understood that many simpler processes can mimic imitation, including artifacts of following, simple maturation, local and stimulus enhancement, behavioural contagion and circular reactions, audience eects, environmental after-eects of the models performance, percussive mimicry, observational conditioning, and sometimes even Pavlovian or instrumental conditioning ; see Galef (1988) and Moore (1992, 1996) for denitions and reviews of such processes. One might add two other mechanisms to this long and growing list : olfactory artifacts, where the model leaves odour traces that later elicit pseudo-imitation (Mitchell et al., 1999), and inverse imitation, where a model performs a certain response (e.g. tongue protrusion) while moving his face towards an infant. The proximity of the face might then elicit the same reaction because the infant attempts to nurse the experimenters face (Meltzo & Moore, 1983). It is also understood by specialists and non-specialists alike that (i) when normal humans imitate they are able to copy novel (not species-typical) responses, (ii) they do so

324 without being rewarded, and (iii) imitation is shown clearly by individual persons ; one need not assemble large groups and invoke statistical tests. Imitation is a qualitative, not quantitative, phenomenon. Any denition of imitation should reect the preceding facts. It should require the copying of novel (non-instinctive) responses, by individuals, in the absence of reinforcement, and in situations where simpler explanations are untenable. (2 ) Imitation in dolphins Imitation has also been shown in cetaceans. Tayler & Saayman (1973) documented the process in several Indian Ocean bottlenose dolphins (Tursiops aduncus). One clear example involved a captive animal named Daan. A human diver often entered its tank to clean an underwater window. The dolphin eventually copied the divers behaviour in four dierent ways : holding on to the window frame with one ipper while scrubbing the glass as the diver had done, releasing bubbles in bursts as he had, and vocally mimicking the sound of his air valve. A younger dolphin, Haig, bonded to a young Cape fur seal (Arctocephalus pusillus) and followed it about. After several months, she began to imitate the seals manner of swimming. Instead of using her natural dorsolateral uke movements, she was frequently seen moving forward very slowly on the surface, holding her ukes motionless and propelling herself with seal-like strokes of the ippers (Tayler & Saayman, 1973, p. 288). She also copied the seals belly-scratching, shaking movements, sleeping postures, wide-mouthed yawns, and an inverted-vertical posture during which they both blew bubbles. At other times Haig was seen imitating the swimming of turtles, skates, and penguins, or using a scraping tool to dislodge seaweed as a diver had done previously. Relatively little is known about learning in dolphins, but their imitation seems surprisingly bird-like in some ways. In addition to movement imitation, some cetaceans are capable of song learning (Payne, Tyack & Payne, 1983), and a degree of vocal mimicry. Further, their movement imitation is used socially: they often match each others jumps and spins ; they are the ultimate synchronized swimmers (see also Tayler & Saayman, 1973, pp. 287289). Given these capacities and their anity for slapping water with their ukes, it seems probable that they are also capable of percussive mimicry. The evolution of imitation of parrots [Section V (5)] and great apes [Section VIII (3)] is quite dierent. The dolphins imitation is like that of parrots, not primates: from song learning p vocal mimicry p percussive mimicry p movement imitation. (3 ) Imitation in primates and humans Thorndike (1898, 1901) was unable to demonstrate true movement imitation in experiments with chicks, cats, dogs, or monkeys. He nevertheless believed that the process occurred in certain non-human primates and that its appearance constituted a quantum jump in the evolution of learning.

Bruce R. Moore
In spite of the popular belief that monkeys can imitate (endorsed by Donald, 1991, p. 169), there is no good evidence for this. There have been countless relevant studies (see Moore, 1992), but none, so far, have been successful. To date the only study of lesser apes was done by one of the authors students, Eva Rogerson (Moore, 1996). The study animal, a 21 2-year-old female gibbon (Hylobates lar) had been raised by humans. Ms Rogersons younger sister, to whom the gibbon had bonded, modelled a series of gestures : putting nger to lips, clapping hands, nodding, patting head, shutting eyes, covering face or ears, waving, joining hands behind back, putting hands on hips, and folding arms across chest. These are responses that gibbons could easily make but did not. Each week, three actions were modelled in exaggerated form, six times per session in each of 14 sessions. But none of these responses was ever imitated. It would be worthwhile to repeat the experiment modelling tool use rather than gestures. Thus, among primates, imitation has been demonstrated so far in only great apes, and in humans at least 9 months old. Imitation does occur among great apes (Hayes & Hayes, 1952; de Waal, 1982; Goodall, 1971, 1986 ; Russon & Galdikas, 1993). Their responses were unrewarded, unambiguously novel, and typically involved the use of tools most dramatically hammers, saws, screwdrivers, shovels, brooms, brushes, keys, needles, lipstick, spray bottles, canes, outboard motors, or canoes (Moore, 1996). Meltzo & Moore (1983) claimed to show that human neonates could imitate at even 4 h old. But their (implicit) denition of imitation was inaccurate. The reactions involved, mouth opening and tongue protrusion, are scarcely novel or unusual, but are clearly species-typical nursing reactions (Jacobson, 1979 ; Masters, 1979) and thus fail to qualify as examples of true imitation. Newborn infants have very poor vision, and attempt to nurse almost anything that comes within range. It is therefore possible that the subjects were attempting to nurse the experimenters face. How the experimenter presented his face its distance, position, expression, etc. may have elicited dierentially one or other of these nursing movements. For further criticism, see Anisfeld (1991). Barr, Dowden & Hayne (1996) claimed to show deferred (delayed) imitation in 6-month-old children. The key response was to remove a large (9 cm) mitten from a puppet (a pink rabbit or grey mouse). This was, again, not a novel response ; it would occur often in the course of manipulating colourful objects. Also, in the course of modelling the movement to be imitated, the experimenters drew attention to the mitten in three dierent ways, any one of which might have produced stimulus enhancement : the mitten was removed from the puppet, and waved, and the waving caused a bell inside the mitten to ring. The likelihood of stimulus enhancement here seems to invalidate the claim of imitation. A follow-up study by Collie & Hayne (1999) again purported to show deferred imitation in six-month olds although the authors failed to address two control problems. The infants had watched experimenters manipulate selected objects on an activity board 50 cm away. Twenty-four hours later, the children were tested with the same board,

The evolution of learning

now close enough to reach. They did touch the demonstration objects more often than the controls, but, again, stimulus enhancement cannot be ruled out. The children manipulated the target objects properly, spinning or squeezing them, pulling strings, etc. However, objects without strings did not permit string pulling, those without noise-makers could not be made to produce sounds, and xed objects could not be spun. In such an experiment, all objects should be manipulable in all the same ways. One should then decide at random, for each infant, which objects to pull, squeeze, or spin. By contrast, Piagets (1951) claim that human infants begin to imitate at about 9 months old seems valid, suggesting that among humans, children can imitate novel movements towards the end of their rst year. Piaget made novel movements when his daughter J was 8, 9, and 10 months. He hit his knee, bent and straightened his nger, and waved good-bye; his daughter imitated these movements. When J was 11 months old, Piaget (i) rubbed his cheek with the back of his hand, (ii) raised his arm, clenched his st, and swung it around, (iii) bent forward, then straightened his head, (iv) struck the back of his hand with his other, (v) struck a xylophone with a hammer, (vi) seized a lock of his hair, then moved it about his temple, and her mother (vii) hit a toy duck with a comb. The daughter J imitated all of these. His other daughter, L, at 9 months, imitated a hand movement that moved out of view (Piaget, 1951, pp. 3755). (4 ) The origins of anthropoid imitation Any mammal capable of imitating the novel movements of another should also be capable of putting through (q.v.), since that process amounts to self-imitation. Skill learning, putting through, and imitation appear to be closely related processes. Functionally, all three lead to the acquisition of skilled movements, often movements involving tool use. Phylogenetically, all three are highly developed in humans and great apes. Behaviourally, all three involve responses of novel (non-elicited, non-speciestypical) forms. They are also similar in that none requires explicit though all may involve implicit reinforcement. Altogether, the three processes have many elements in common and seem to form a logical progression : skill learning involves copying movements that one has made by oneself ; putting through involves copying movements that a teacher has moulded or guided in ones own body ; movement imitation involves copying movements that the other has made with his/her body. Perhaps a more useful way to describe the progression, however, is to say that these three processes involve nested sets of supporting stimuli. Repeating a response in skill learning, putting through, or imitation presumably involves stimulus matching (King, 1979). In the case of skill learning, the animal might match kinaesthetic, other proprioceptive, or visual stimuli. In putting through, kinaesthetic matching would be lost, but the animal could match other proprioceptive or visual stimuli. In movement imitation there could only be visual matching. In some cases imitative movements are extremely complex. Sherrington (1906, p. 392) long ago noted that it was

325 impossible to learn skating or racquets by simple visual observation. Observation could show the learner what to do, and generally how to do it. But, with such complex responses, skill learning would then be required to shape the details. The same could be said of putting through. Both processes can teach only the rudiments of complex behaviour ; they direct and set the stage for subsequent skill learning. See also Bandura (1977, p. 28). The acquisition of termite shing by chimpanzees (Goodall, 1971 cf. Teleki, 1974) presumably involves both imitation and skill learning (Moore, 1996). So, altogether, these three processes (skill learning, putting through, and visual imitation) are linked in many ways : their possible controlling stimuli are nested as just described ; both putting through and imitation set the stage for skill learning. Putting through is like self-imitation. All three processes involve novel responses and possibly implicit reinforcement. And all three of them often serve tool use. Imitation is obviously adaptive in allowing hominoids to learn from the experience of others. In human huntergatherers, it is also used in deceptive communication. When African Bushmen stalk certain mammals, and are detected by them, they make no eort to hide themselves but go slowly, hands behind [their] backs, imitating the movements of ostriches pecking at food in the veld (Van der Post, 1958, p. 232). (5 ) Cross-Modal Imitation in Hominids Movement imitation has three daughter processes, two of which ( pantomime and instrumental mimicry) will be discussed later in the context of language. The third is cross-modal imitation. The distinction between ordinary (visual) and crossmodal imitation was rst noted in the eld of human development (Guillaume, 1971). If an infant copies its mothers hand movements, it can see its own movements and compare them to hers. That is visual imitation. If, however, it imitates blinking, wrinkling the nose, or nodding the head, then it cannot see its own movements, and that is non-visual (cross-modal) imitation. Piaget (1951) believed that crossmodal imitation evolved from simpler visual imitation. He reported that, at approximately 1315 months of age, the human infant often touches the mothers nose, then tactually locates its own. Symmetrically, mothers sometimes touch and name their own noses and then babys (E. Coscia, personal communication, 2000). After a few such repetitions, initiated by either party, if the mother touches her own nose, the baby can imitate the response. It has learned where to nd its corresponding part, and can therefore show crossmodal imitation. The same process occurs with eyes, ears, mouth, teeth, and hair. The baby can also feel, and therefore match, responses that use those structures, like opening the mouth, closing the eyes, or sticking out the tongue. Thus, Piaget showed that cross-modal imitation is not a giant step from ordinary, visual imitation. What makes it possible is a period of developmentally programmed tactile exploration. He also noted that cross-modal imitation occurs later in ontogeny than ordinary (visual) movement imitation. It thus appears that imitation in Anthropoidea evolved through the following progression : from

326 Thorndikian conditioning p cross-drive conditioning p operant conditioning p skill learning p putting through p visual imitation p cross-model imitation (Fig. 1). (6 ) Imitation in cats and rats ? A well-known study by John et al. (1968) purported to show imitation in cats, but their study contained a number of serious problems (cats in the control group were trained to hurdle to avoid shocks, but those in the imitation group were also conditioned to hurdle to avoid shocks. This was interpreted as imitating, although it was, in fact, punishment or avoidance conditioning. The experiments with the imitative cats were not called conditioning ; they were referred to as daily empathy trials. Several other confoundings are described in Moore (1992, 1996). Heyes, Dawson & Nokes (1992) claimed evidence for imitation in rats. Moore (1996) expressed reservations on their ndings on multiple grounds. Mitchell et al. (1999) subsequently showed that the imitating rats were simply reacting to odour deposits left by conspecic models on critical parts of the apparatus. (7 ) Are avian and hominoid imitation homologous ? Imitation in parrots seems very dierent from that seen in apes. Most strikingly, it has signicant incubation periods. Todt (1975) pointed out that parrots never copy new sounds in fewer than three days (see also Baldwin, 1914). The authors parrot conrmed this. It eventually learned 300 words and sounds, including several that it had heard only once. But no sound took less than three days to learn. The incubation period for new movements was measured in weeks. In primates, of course, there is no incubation period. In fact, in human infants immediate imitation is easier than delayed, and appears earlier in development (Piaget, 1951, pp. 45, 62). Another dierence is that in birds vocal mimicry is far commoner than movement imitation whereas great apes copy movements, but rarely copy novel sounds. As Beck (1974) pointed out, imitation by great apes usually involves tool use. But avian imitation never incorporates objects. When the parrot imitated a response that had involved some object, it always performed without the object. Its imitation seemed to be an extension of display behaviour, as suggested by the late Mel Kreithen (see Moore, 1992, and Groothius, 1993). There is some degree of overlap in function primates can imitate displays. But, even so, the dierence may be meaningful. Notice that all three distinctive features of avian mimetic learning the apparent social function (display), the predominance of sounds, and the incubation periods are also properties of song learning, their hypothesized source. Similarly, the properties of primate imitation all reect its hypothesized roots in skill learning and tool use. Thus, phylogenetic patterns of occurrence, dierences in evolutionary paths, incubation time, and apparent function, all suggest that avian and mammalian imitation are not homologous. This should not be surprising since the last common ancestors of parrots and primates, the so-called

Bruce R. Moore
stem reptiles, disappeared three hundred million years ago and one hundred million years before the rst dinosaurs. It is dicult to imagine imitation in such primitive creatures. But if the process had been present, then one would expect to see it now in certain orders of reptiles, and most or all birds and mammals which is far from the case. Of course, birds dier from most mammals in many fundamental ways. They have feathers rather than fur, and wings rather than arms. They can y. They lay eggs and lack external genitalia. Their kidneys, hearts and lungs are radically dierent, as are their forebrains. Where such basic structures are so fundamentally dierent it would be unlikely for learning to somehow remain the same. Psychologists have rarely distinguished between learning in birds and that in mammals. The supercial similarities between imitation in the parrot and that in chimpanzees or dolphins should not be allowed to reinforce this error. That the process might have evolved at least three times should not be surprising. Flight has evolved at least three times (in birds, bats, and insects), hearing at least 10 times, and photoreception even more often (Salvini-Plawen & Mayr, 1977), but see also Quiring et al. (1994) and Halder, Callaerts & Gehring (1995). The fact that the various forms of imitation in parrots, dolphins, and primates are not homologous does not mean that they are entirely unrelated. All appear to be distant descendants of Pavlovian conditioning. IX. LANGUAGE ACQUISITION The learning of language is extremely complex. To describe it fully is beyond the scope of the present article. It will simply be outlined later. Chomsky (1972) declared that language did not evolve. In fact, both spoken and gestural language develop almost entirely through cultural evolution. Further, Chomsky (1980) declared that all normal humans were born with Universal Grammar. That is, they were born with every form of earthly grammar : English, Chinese, Nthlakampx, and thousands of others. According to his theory, the learning of grammar is simply a product of parameter xing. Each child is exposed to limited samples of local language usage. They compare these samples to a vast array of innate grammatical rules, and eventually retain only those that correspond to local usage (Caplan & Chomsky, 1980 ; Piattelli-Palmerini, 1989). Chomsky tells us that he does not regard this theory as mere theory : We expect to nd [it] physically represented in the genetic code and the adult brain (Chomsky, 1980, pp. 8283). The rst problem with this theory is that it attributes amazing cognitive powers to 2- and 3-year-old infants. Secondly, evolution typically provides one adequate solution to a problem for any given group of humans. Wherever there seem to be two, they typically serve dierent functions, or provide important backups. A backup of thousands of options is unlikely to occur. Thirdly, the evolution of multiple grammars would be extremely maladaptive. If children were born with a single grammar, they might learn to speak promptly, and be able to learn other languages smoothly

The evolution of learning

and quickly. But thousands of grammars could greatly prolong individual language acquisition, and obstruct communication between dierent populations. Fourthly, we shall see that learning grammar (or syntax) involves memorizing many similar phrases, and eventually learning each pattern through abstract concept formation. Piattelli-Palmarini (1989) declared that learning did not exist, but that it was actually a process of parameter setting. In fact, language acquisition (gestural or vocal) requires multiple learning processes, including vocal mimicry, phonetics, discrimination learning, associations between neutral stimuli, skill learning, abstract concept formation, putting through, movement imitation, and pantomime. The various armchair language theories leave much to be desired. Most children begin by learning single words (Leopold, 1939, 1949). The phonetic sounds are simply a higher level of vocal mimicry. Somewhat later, children begin to use multiple words but spoken separately, with long spaces between them. Such word clusters, are also known as vertical constructions (Scollon, 1976). Leopolds bilingual daughter used phrases like Buch ? Away, or Mama ? Mit (Leopold, 1949). Later, much longer clusters were recorded by Scollon (1976), including Tape, tape, Brenda, hurt, Mommy [my ?], Mommy, band aid, hurt, hurt, hurt, sick, sick. Such phrases begin to incorporate various parts of speech. Infants do not, of course, learn at that age to categorize nouns, verbs, adjectives, etc. They do, however, begin to learn to memorize certain phrases, including rote grammar and rote syntax. After that they begin to use pivot words (Braine, 1963, 1976), such as my or o, as in my doll, my nose, my shoe, or boot o, light o, or water o. Such pivot words are very important, and clearly generate more complex speech. The children also begin to use formulaic two-word phrases, such as thank you, good night, or all gone. Such phrases are learned almost as single words. They are followed by true two-word phrases (Bloom, 1970), which are not simply memorized. Later they use phrase clusters, such as talk / corder talk / Brenda talk (Scollon, 1976). Months later they begin to use telegraphic speech (see Brown & Bellugi, 1964). Again, the early telegraphic phrases are simply memorized, but they eventually learn the patterns that generate more complex phrases (Tomasello, 1992 ; Olguin & Tomasello, 1993 ; Akhtar & Tomasello, 1997 ; Akhtar, 1999). Telegraphic phrases of course leave out certain words or parts of speech that have yet to be acquired. But telegraphic speech is serious communication. As the missing words are slowly added, syntactic phrases occur, followed by real sentences initially short, but gradually longer i.e. true vocal language. Reading and writing use most of the same processes, plus tool use. Clearly language is acquired through many stages of cultural evolution, each of which requires multiple learning processes. X. IMAGE-MEDIATED LEARNING When vocal language is acquired, a later process, which might be called image-mediated learning, is sometimes used to communicate complex movements. For example, a

327 violinist might be taught a certain ngering technique by simply telling him or her to release the string as if testing a very hot iron. (This is one of numerous techniques employed by David Zafer of Toronto.) This simple technique works more quickly and precisely than any demonstration or detailed description of movement. Other such images are used in equestrian riding (Swift, 1985), dancing, sport, and countless other situations. XI. DOES LEARNING RE-CAPITULATE ? Just as ontogeny sometimes recapitulates phylogeny (i.e. retraces ancestral embryonic forms), so also the course of learning sometimes retraces its evolutionary past. This can happen in particular learning situations and also in early development. Of course such comparisons are not meaningful unless one process is a linear descendant of another. In learning, Wahlsten & Cole (1972) said that their dogs appeared to show Pavlovian conditioning before they showed true avoidance. In fact, several seemed to show sensitization, then Pavlovian conditioning, and then avoidance : rst diuse fear, then fear of the warning signal, then precise avoidance. Similar data were reported by Cassady (1996) ; see also Schlossberg (1937), Razran (1938), and Lashley & Wade (1946) on human conditioning. Again, the chimpanzee Viki learned by putting through before movement imitation (Hayes & Hayes, 1952). And Moores (1992) parrot learned at least 14 combinations of mimetic words and movements. In all 14 cases, the vocal mimicry emerged before movement imitation. So learning can sometimes recapitulate evolutionary sequences. Recapitulation may also occur developmentally. Thus, Piaget (1951) found that human infants showed immediate, then deferred, and then cross-modal imitation. XII. EVOLUTIONARY WEBS : CROSS-CONNECTIONS IN THE TREE It was argued earlier that song learning had evolved from two sources, auditory imprinting and avian skill learning. Having two evolutionary paths come together in this fashion may seem to some readers to dier from the classical pattern seen in the evolution of species. But, in fact, there are important parallels (Lawrence & Ochmann, 1998; Doolittle, 1999). It is important to document this point. At the simplest level, fertile hybrids reect the convergence of two evolutionary lines. The lines would have to be closely related, of course. But in other cases nature is less strict. Some 18 % of the genes of Escherichia coli are said to have come from Salmonella enterica through lateral gene transfer (Lawrence & Ochmann, 1998). These two strains of intestinal bacteria diverged roughly 100 million years ago. But the transferred genes were, on average, only 14 million years old. So there has been a melding of genes from dierent genera. Lichen are symbiotic combinations of algae and fungi. This was rst discovered by Schwendener (1869), and championed by Beatrix Potter (1966). In 189697, Potter,

328 a ne self-trained naturalist, attempted to demonstrate this to several eminent botanists, whom she later described as misogynistic, rude, and stupid (Potter, 1966, p. 426). Her frustration led her nally to abandon science and turn to writing childrens books ; see also Ahmadjian (1967). Fungi provide shelter for algae and are much better at storing water ; algae, by contrast, photosynthesize, which fungi do not. So here we have a coming together not of two genera, but two kingdoms. Their genes do not occupy the same cells in this case. But that, too, can occur. The mitochondria that inhabit our own cells are aliens within. They govern aerobic respiration, and help sustain life, in most or all eukaryotes. But these essential organelles are not produced by our genes. They have chromosomes of their own circular, bacterial origins. Mitochondria are obligate endosymbionts : resident a-proteobacteria (Wallin, 1925; Margulis, 1993; Doolittle, 1999). Similarly, chloroplasts, the organelles that govern photosynthesis in green plants, are actually blue green algae, also known as cyanophyta, or cyanobacteria (Famintzyn, 1891, cited in Margulis, 1993 ; Doolittle, 1999). Eukaryotes with their mitochondria, and green plants with their chloroplasts, might be said to belong to two kingdoms. The genes of the two reside in the same cells. Their chromosomes are separate, but even that can change. Genes can move from the bacterial DNA to that of the hosts. Even more fundamentally, recent genetic research suggests widespread lateral gene transfer between the three domains (Ur-Kingdoms) of life : eukaryotes, bacteria, and archaea (archaebacteria) (Hilario & Gogarten, 1993; Doolittle, 1999). In short, a truly accurate evolutionary tree would show cross-connections at many levels, from subspecies to Ur-Kingdoms. It is therefore not improper to propose that bird song evolved from two sources, or human language from many. These conclusions might have been a problem for nineteenth-century theory, but are no problem at all in the twenty-rst. Evolutionary theory is evolving.

Bruce R. Moore
mimicry throughout the order, imprinting throughout the class, Garcia conditioning in most or all of the phylum, and sensitization in most multi-celled animals. Here, the phylogenetic pattern agrees perfectly with the hypothesized evolutionary sequence. The boundaries of the relevant mammalian processes are less precisely known for several reasons. Beach (1950) complained that research in mammalian learning involved primarily white rats. While the legitimacy of his complaint was widely accepted, the subsequent half century brought very little change. Further, researchers in operant conditioning have traditionally known far less about their subjects natural repertoires than did the ethologists who studied avian song learning and vocal mimicry. This lack of knowledge was exacerbated by the fact that many operant researchers chose deliberately not to observe their animals, preferring the convenience of automated research. Within the Skinnerian system, there was no reason to observe behaviour. Responses were not dened by their topographies but by their consequences, their eects upon the environment. Thus, it was important to know how many times a rat pressed a lever, but how it pressed were considered irrelevant. Indeed, distinguishing between dierent forms of pressing was explicitly discouraged (Skinner, 1938, p. 37). Skinners point was amplied by Ferster & Culbertson (1982, p. 16), who explained that it did not matter whether a rat pressed the lever by sitting on it, pressing it with its feet, grasping it with the teeth, or jumping on it. All of these performances are functionally equivalent . Moore (1973) and Moore & Stuttard (1979) pointed out several cases where this approach led directly to fundamental misinterpretations of what animals were doing in classic learning situations. For these reasons, the comparative data base involving mammals is not comparable to that of birds. Still, the author knows of no case where any mammal has shown any processes discussed here without also showing its hypothesized precursor. It should be noted that if Group B evolved from Group A, that does not imply that all present species of B should be more advanced than all present species of A. For example, the rst Amphibia evolved from shes in the Paleozoic era, but both groups demonstrably continued to evolve in the subsequent 400 million years (Romer, 1966). During that interval, some groups of shes may well have outstripped some groups of Amphibia in certain areas (Razran, 1971), acquiring processes that the latter did not. This would not invalidate an evolutionary theory. One would simply modify it to reect the reality : one branch of the cladogram should show the progress of certain shes, and another, parallel, branch, the progress of certain Amphibia.

XIII. COMPARATIVE EVIDENCE FOR EVOLUTIONARY HYPOTHESES One critical test of any evolutionary hypothesis is compatibility with comparative data. Species capable of process n should also be capable of process nx1, as should their immediate ancestors, and, in most cases, their nearest living relatives. Processes n, nx1, nx2, , should therefore occur in nested phylogenetic taxa : for example, process n might occur within a certain family, nx1 within the larger order (or other grouping), and so forth. In the case of avian mimetic learning, an impressive body of relevant data is available. Song learning and vocal mimicry, in particular, have been studied in hundreds of species, and much is known of their phyletic boundaries (Kroodsma, 1982; Kroodsma & Baylis, 1982). These extensive data are entirely compatible with the hypothesized path of avian mimetic learning (Moore, 1992). On a broader scale, movement imitation occurs within the parrot family, vocal

XIV. THE VERTEBRATE HIERARCHY In addition to suggesting various paths in the evolution of learning, Fig. 1 attempts to show, again to a rst approximation, the groups of animals in which some forms of learning rst emerged. Many gaps, and undoubtedly errors, are inevitable. In many cases, the required research has

The evolution of learning

simply not been done. In others, it may have escaped the writers attention. In still others the work has not been conclusive. The cladogram nevertheless includes hypotheses about the taxa within which some processes rst evolved. While possibly wrong in some cases, the guesses may constitute a qualitative improvement upon Skinners (1938) assumption that the laws of learning are suciently general to permit their study in a single species the rat. It should be understood that Fig. 1 reects, at best, a rst approximation of nature. The dierences between adjacent processes could involve multiple genes, thus many sequences could easily be lengthened. Alternatively, many could be shortened by combining categories. For these and other reasons, no inferences can be drawn about which of two processes is more recent, or higher, except where one has descended directly from the other. Again, some parts of the diagram are clearly incomplete, and others may simply be wrong. One hopes, however, that corrections required by future developments will allow the cladogram to evolve, much like the processes it represents. If that occurs, the very falsication of the Figure might arguably demonstrate its value. XV. ACKNOWLEDGEMENTS
The work described here began several decades ago, in 1972, when Pat Bateson called my attention to Martin Wells (1967) suggestion that conditioning had evolved from sensitization. It was clear that similar analyses could be done for all forms of learning, and might then be combined in a single evolutionary tree. The fact that the work took thirty years reected increasing teaching loads and minuscule nancial support. Most of the writing was limited to holidays and summers. When I reached the age of forced retirement, it progressed more rapidly. I thank Betty Moore, Sara Lees, and Drs Nameera Akhtar, Nick Mackintosh, and Marcia Spetch for suggesting numerous changes in the manuscript. All residual errors are, of course, my fault, not theirs.

329
BALDWIN, J. M. (1914). Deferred imitation in West-African Gray Parrots. Proceedings of the 9th International Congress of Zoology. Monaco. 536537. BANDURA, A. (1977). Social Learning Theory. Prentice-Hall, Englewood Clis, NJ. BAPTISTA, L. F., BELL, D. A. & TRAIL, P. W. (1993). Song learning and production in the white crowned sparrow : parallels with sexual imprinting. Netherlands Journal of Zoology 43, 1733. BAPTISTA, L. F. & SCHUCHMANN, K.-L. (1990). Song learning in the Anna Hummingbird (Calypte anna). Ethology 84, 1526. BARR, R., DOWDEN, A. & HAYNE, H. (1996). Developmental changes in deferred imitation by 6- and 24-month-old-infants. Infant Development and Behavior 19, 159170. BARRERRA, F. J. (1974). Centrifugal selection of signal-directed pecking. Journal of the Experimental Analysis of Behavior 22, 341355. BATESON, P. P. G. (1966). The characteristics and context of imprinting. Biological Review 41, 177220. BAYLIS, J. R. (1982). Avian vocal mimicry : its function and evolution. In Acoustic Communication in Birds (eds. D. E. Kroodsma, E. H. Miller and H. Ouellet), pp. 5183. Academic Press, New York. BEACH, F. A. (1950). The snark was a boojum. American Psychologist 5, 115124. BECK, B. B. (1974). Baboons, chimpanzees, and tools. Journal of Human Evolution 3, 509516. BECK, B. B. (1980). Animal Tool Behavior. Garland STPM Press, New York. BERNSTEIN, A. L. (1934). Temporal factors in the formation of conditioned eyelid reactions in human subjects. Journal of General Psychology 10, 173197. BERTRAND, M. (1967). Training without reward : traditional training of pig-tailed macaques as coconut harvesters. Science 155, 484 486. BINDRA, D. (1972). A unied account of classical conditioning and operant training. In Classical Conditioning II : Current Theory and Research (eds. A. H. Black and W. F. Prokasy), pp. 453 481. Appleton Century Crofts, New York. BISCHOF, H.-J. (1994). Sexual imprinting as a two-stage process. In Causal Mechanisms of Behavioural Development (eds. J. A. Hogan and J. J. Bolhuis), pp. 8297. Cambridge University Press, Cambridge. BITTERMAN, M. E. (1976). Technical comment. Flavor aversion studies. Science 192, 266267. BLOOM, L. (1970). Language Development : Form and Function in Emerging Grammars. MIT Press, Cambridge, MA. BOAKES, R. & PANTER, D. (1985). Secondary imprinting in the domestic chick blocked by previous exposure to a live hen. Animal Behaviour 33, 353365. BOESCH, C. & BOESCH, H. (1983). Optimisation of nut-cracking with natural hammers by wild chimpanzees. Behaviour 83, 265286. BOLHUIS, J. J. (1991). Mechanisms of avian imprinting : a review. Biological Review 66, 303345. BOLHUIS, J. J., DE VOS, G. J. & KRUIJT, J. (1990). Filial imprinting and associative learning. Quarterly Journal of Experimental Psychology 42B, 313329. BOLHUIS, J. J. & HONEY, R. C. (1994). Within-event learning during lial imprinting. Journal of Experimental Psychology : Animal Behavior Processes 20, 240248. BOLHUIS, J. J. & VAN KAMPEN, H. S. (1992). An evaluation of auditory learning in visual imprinting. Behaviour 122, 195230.

XVI. REFERENCES
ABLE, K. P. & ABLE, M. A. (1997). Development of sunset orientation in a migratory bird : no calibration by the magnetic eld. Animal Behaviour 53, 363368. AHMADJIAN, V. (1967). The Lichen Symbiosis. Blaisdell, London. AKHTAR, N. (1999). Acquiring basic word order : evidence for datadriven learning of syntactic structures. Journal of Child Language 26, 339356. AKHTAR, N. & TOMASELLO, M. (1997). Young childrens productivity with word order and verb morphology. Developmental Psychology 33, 952965. ANISFELD, M. (1991). Neonatal imitation. Developmental Review 11, 6097. BAILEY, C. H. & CHEN, M. (1983). Morphological basis of longterm habituation and sensitization in Aplysia. Science 220, 9193. BAKER, M. C., BOTTJER, S. W. & ARNOLD, A. P. (1984). Sexual dimorphism and lack of seasonal changes in vocal control regions of the white-crowned sparrow brain. Brain Research 295, 8589.

330
BOLLES, R. C. (1970). Species-specic defense reactions and avoidance learning. Psychological Review 77, 32 48. BOSWALL, J. (1977). Tool-using by birds and related behaviour. Avicultural Magazine 83, 146159, 220228. BOSWALL, J. (1983 a). Tool-using and related behaviour in birds : more notes. Avicultural Magazine 89, 94108. BOSWALL, J. (1983 b). Tool-using and related behaviour in birds : yet more notes. Avicultural Magazine 89, 170181. BRAINE, M. D. S. (1963). The ontogeny of English phrase structure. Language 39, 113. BRAINE, M. D. S. (1976). Childrens rst word combinations. Monographs of the Society for Research in Child Development 41, Serial No. 164, No. 1, 1104. BRELAND, K. & BRELAND, M. (1961). The misbehavior of organisms. American Psychologist 16, 681684. BRELAND, K. & BRELAND, M. (1966). Animal Behavior. Macmillan, New York. BRIMER, C. B. (1970 a). Disinhibition of an operant response. Learning and Motivation 1, 346371. BRIMER, C. B. (1970 b). Inhibition and disinhibition of an operant response as a function of the amount and type of prior training. Psychonomic Science 21, 191192. BROWN, P. L. & JENKINS, H. M. (1968). Auto-shaping of the pigeons key peck. Journal of the Experimental Analysis of Behavior 11, 18. BROWN, R. & BELLUGI, U. (1964). Three processes in the childs acquisition of syntax. Harvard Educational Review 34, 133151. CAPLAN, D. & CHOMSKY, N. (1980). Linguistic perspectives on language development. In Biological Studies of Mental Processes (ed. D. Caplan), pp. 97105. MIT Press, Cambridge, MA. CASSADY, J. M. (1996). Avoidance and classical conditioning of leg flexion in dogs. Behavioural Brain Research 77, 7999. CASTELLUCCI, V., PINSKER, H., KUPFERMANN, I. & KANDEL, E. R. (1970). Neuronal mechanisms of habituation and dishabituation of the gill-withdrawal reflex in Aplysis. Science 167, 17451748. CHOMSKY, N. (1972). Language and Mind (Enlarged edn). Harcourt Brace Jovanovich, New York. CHOMSKY, N. (1980). Rules and Representations. Columbia University Press, New York. CHURCH, R. M. (1963). The varied eects of punishment on behavior. Psychological Review 70, 369 402. CHURCH, R. M. (1964). Systematic eect of random error in the yoked control design. Psychological Bulletin 62, 122131. CHURCH, R. M. (1989). The yoked control design. In Aversion, Avoidance, and Anxiety (eds. T. Archer and L. Nilsson), pp. 403415. Erlbaum, Hillsdale, NJ. CHURCH, R. M. & GETTY, D. J. (1972). Some consequences of the reaction to an aversive event. Psychological Bulletin 78, 2127. CHURCH, R. M. & LERNER, N. D. (1976). Does the headless roach learn to avoid ? Physiological Psychology 4, 439 442. COLLIE, R. & HAYNE, H. (1999). Deferred imitation by 6 and 9-month-old infants : more evidence for declarative memory. Psychobiology 35, 8390. COWAN, P. J. (1974). Selected responses to the parental calls of dierent individual hens by young Gallus : auditory discrimination learning versus auditory imprinting. Behavioural Biology 10, 541545. COWEN, R. (1990). Medicine on the wild side. Science News 138, 280282. COWLES, J. T. (1937). Food tokens as incentives for learning in chimpanzees. Comparative Psychology Monographs 14, 196.

Bruce R. Moore
CRAIG, W. (1911). The expressions of emotion in the pigeons. II. The mourning dove (Zenaidura macroura Linn.). Auk 28, 398 407. CRUICKSHANK, A. J., GAUTHIER, J.-P. & CHAPPIUS, C. (1993). Vocal mimicry in wild African Grey parrots (Psittacus erithacus). Ibis 135, 293299. CURIO, E., ERNST, U. & VEITH, W. (1978). Cultural transmission of enemy recognition : one function of mobbing. Science 202, 899901. DAVEY, G. C. L. & CLELAND, G. G. (1982). Topography of signalcentered behavior in the rat : eects of deprivation state and reinforcement type. Journal of Experimental Analysis of Behavior 38, 291304. DENNY, M. R. (1970). A learning model. In Comparative Psychology (eds. M. R. Denny and S. C. Ratner), pp. 630640. Dorsey Press, Homewood, IL. DE WAAL, F. B. (1982). Chimpanzee politics : Power and Sex among Apes. Harper & Row, New York. DONALD, M. (1991). Origins of the Modern Mind. Harvard University Press, Cambridge, MA. DOOLITTLE, W. F. (1999). Phylogenetic classication and the universal tree. Science 284, 21242128. DOWSETT-LEMAIRE, F. (1979). The imitative range of the song of the marsh warbler Acrocephalus palustris with special reference to imitations of African birds. Ibis 121, 453 468. DUNHAM, P. J. (1971). Punishment : method and theory. Psychological Review 78, 5870. EMLEN, S. T. (1970). Celestial rotation : its importance in the development of migratory orientation. Science 170, 11981201. EMLEN, S. T. (1971). Celestial rotation and stellar orientation in migratory warblers. Science 173, 460 461. ESTES, W. K. (1944). An experimental study of punishment. Psychological Monographs 57, No. 3, Whole No. 263. ESTES, W. K. (1969). Outline of a theory of punishment. In Punishment and Aversive Behavior (eds. B. A. Campbell and R. M. Church), pp. 5782. Appleton-Century-Crofts, New York. ESTES, W. K. & SKINNER, B. F. (1941). Some quantitative properties of anxiety. Journal of Experimental Psychology 29, 390 400. EVANS, R. M. (1972). Development of an auditory discrimination in domestic chicks (Gallus). Animal Behaviour 20, 7787. EVANS, R. M. & MATTSON, M. E. (1972). Development of selective responses to individual maternal calls in young Gallus. Canadian Journal of Zoology 50, 777780. FERSTER, C. B. & CULBERTSON, S. A. (1982). Behavior Principles, 3rd Edn. Prentice-Hall, Englewood Clis, NJ. FERSTER, C. B. & SKINNER, B. F. (1957). Schedules of Reinforcement. Appleton-Century-Crofts, New York. FLETEMEYER, J. R. (1978). Communication about potentially harmful foods in free-ranging Chacma baboons, Papio ursinus. Primates 19, 223226. FOUTS, R. S., FOUTS, D. H. & VAN CANTFORT, T. E. (1989). The infant Loulis learns signs from cross-fostered chimpanzees. In Teaching Sign Language to Chimpanzees (eds. R. A. Gardner, B. T. Gardner and T. E. van Cantfort), pp. 280292. State University of New York Press, New York. FOUTS, R. S. & RIGBY, R. L. (1977). Manchimpanzee communication. In How Animals Communicate (ed. T. A. Sebeok), pp. 10341054. Indiana University Press, Bloomington, IN. FOX, J. E. (1972). Ambient earth motion in the period range from 0.1 to 2560 seconds. Bulletin of the Seismological Society of America 62, 17531760.

The evolution of learning

FRANTTI, G. E., WILLIS, D. E. & WILSON, J. T. (1962). The spectrum of seismic noise. Bulletin of the Seismological Society of America 52, 113121. GALEF, B. G., JR. (1988). Imitation in animals : history, denition and interpretation of data from the psychological laboratory. In Social Learning : Psychological and Biological Perspectives (eds. T. R. Zentall and B. G. Galef, Jr.), pp. 328. Erlbaum, Hillsdale, NJ. GARCIA, J., ERVIN, F. R., YORKE, C. H. & KOELLING, R. A. (1967). Conditioning with delayed vitamin injections. Science 155, 716718. GARCIA, J. & KOELLING, R. A. (1966). Relation of cue to consequence in avoidance learning. Psychonomic Science 4, 123124. GARDNER, R. A. & GARDNER, B. T. (1969). Teaching sign language to a chimpanzee. Science 165, 664672. GOODALL, J. VAN L. (1971). In the Shadow of Man. Collins, London. GOODALL, J. (1986). The Chimpanzees of Gombe. Belknap/Harvard University Press, Cambridge, MA. GOULD, J. L. (1982). Ethology. Norton, New York. GRANT, D. A. & ADAMS, J. K. (1944). Alpha conditioning in the eyelid. Journal of Experimental Psychology 34, 136142. GRETHER, W. F. (1938). Pseudo-conditioning without paired stimulation encountered in attempted backward conditioning. Journal of Comparative Psychology 25, 9196. GRIFFIN, D. R. (1944). The sensory basis of bird navigation. Quarterly Review of Biology 19, 1532. GRIFFIN, D. R. (1969). The physiology and geophysics of bird navigation. Quarterly Review of Biology 44, 255276. GROOTHIUS, T. G. G. (1993). A comparison between development of bird song and development of other displays. Netherlands Journal of Zoology 43, 172192. GUILLAUME, P. (1971). Imitation in Children (E. P. Halperin, translation). University of Chicago Press, Chicago (originally published in 1926). GUTTINGER, H. R. (1979). The integration of learnt and genetically programmed behaviour : a study of hierarchical organization in songs of canaries, greennches and their hybrids. Zeitschrift fu r Tierpsychologie 49, 285303. GUYOMARCH, J. C. (1974). Lempreinte auditive prenatale chez le poussin domestique. Revue Comparative Animal 8, 36. HAGSTRUM, J. T. (2000). Infrasound and the avian navigational map. Journal of Experimental Biology 203, 11031111. HALDER, G., CALLAERTS, P. & GEHRING, W. J. (1995). Induction of ectopic eyes by targeted expression of the eyeless gene in Drosophila. Science 267, 17881792. HALL, K. R. L. (1968). Social learning in monkeys. In Primates (ed. P. C. Jay), pp. 383397. Holt, Rinehart & Winston, New York. HANFMANN, E. & KASANIN, J. (1937). A method for the study of concept formation. Journal of Psychology 3, 521540. HARLOW, H. F. (1958). The evolution of learning. In Behavior and Evolution (eds. G. G. Simpson and A. Roe), pp. 259290. Yale University Press, New Haven. HARLOW, H. F. & HARLOW, M. K. (1962). Social deprivation in monkeys. Scientic American 207 ( July), 6976. HARLOW, H. F. & ZIMMERMAN, R. R. (1959). Aectional responses in the infant monkey. Science 130, 421 431. HARRIS, J. D. (1943). Habituatory response decrement in the intact organism. Psychological Bulletin 40, 385 422. HASLER, A. D. & SCHOLZ, A. T. (1983). Olfactory Imprinting and Homing in Salmon. Springer-Verlag, Berlin.

331
HAYES, K. J. & HAYES, C. (1952). Imitation in a home-reared chimpanzee. Journal of Comparative and Physiological Psychology 45, 450 459. HEARST, E. & JENKINS, H. M. (1974). Sign-Tracking : The Stimulus Reinforcer Relation and Directed Action. The Psychonomic Society, Austin, TX. HEDIGER, H. (1968). The Psychology and Behaviour of Animals in Zoos and Circuses. Dover, New York. HENNIG, W. (1966). Phylogenetic Systematics. University of Illinois Press, Urbana, IL. HEYES, C. M., DAWSON, G. R. & NOKES, T. (1992). Imitation in rats : initial responding and transfer evidence. Quarterly Journal of Experimental Psychology 45B, 8192. HILARIO, E. & GOGARTEN, J. P. (1993). Horizontal transfer of ATPase genes : the tree of life becomes a net of life. Biosystems 31, 111119. HINDE, R. A. (1970). Behavioural habituation. In Short-Term Changes in Neural Activity and Behaviour (eds. G. Horn and R. A. Hinde), pp. 339. Cambridge University Press, Cambridge. HOFFMAN, H. S. (1973). A reinforcement model of imprinting : implications for socialization of monkeys and men. Psychological Review 80, 527544. HOFFMAN, H. S., EISERER, L. A. & SINGER, D. (1972). Acquisition of behavioral control by a stationary imprinting stimulus. Psychonomic Science 26, 146148. HOFFMAN, H. S. & RATNER, A. M. (1978). A reinforcement model of imprinting : implications for socialization in monkeys and men. Psychological Review 80, 527544. HORN, G. & HINDE, R. A. (1970). Short-Term Changes in Neural Activity and Behaviour. Cambridge University Press, Cambridge. HULL, C. L. (1920). Quantitative aspects of the evolution of concepts. Psychological Monographs 28 (No. 123), 9 40. HULL, C. L. (1934). Learning : II. The factor of the conditioned reflex. In A Handbook of General Experimental Psychology (ed. C. Murchison), pp. 382 455. Russell & Russell, New York. HUNT, G. R. (1996). Manufacture and use of hook-tools by New Caledonian crows. Nature 379, 249251. HUNT, H. F. & BRADY, J. V. (1951). Some quantitative and qualitative dierences between anxiety and punishment conditioning. American Psychologist 6, 276277. HUNT, H. F. & BRADY, J. V. (1955). Some eects of punishment and intercurrent anxiety on a simple operant. Journal of Comparative and Physiological Psychology 48, 305310. HURSHMAN, A. & MOORE, B. R. (1976). Behaviour of rats under superstition schedules of food or water reinforcement. Paper presented at Canadian Psychological Association, Toronto. IMMELMANN, K. (1972). Sexual and other long-term aspects of imprinting in birds and other species. Advances in the Study of Behavior 4, 147174. IMMELMANN, K., PRO VE, R., LASSEK, R. & BISCHOF, H.-J. (1991). Influence of adult courtship on the development of adult sexual preferences in zebra nch males. Animal Behaviour 42, 8389. JACOBSON, S. W. (1979). Matching behavior in the young infant. Child Development 50, 425 430. JAMES, H. (1959). Flicker : an unconditioned stimulus for imprinting ? Canadian Journal of Psychology 13, 5967. JAMES, H. (1960). Imprinting with visual flicker : evidence for a critical period. Canadian Journal of Psychology 14, 1320. JENKINS, H. M., BARRERA, F. J., IRELAND, C. & WOODSIDE, B. (1978). Signal-centred action patterns of dogs in appetitive classical conditioning. Learning and Motivation 9, 272296.

332
JENKINS, H. M. & MOORE, B. R. (1973). The form of the autoshaped response with food or water reinforcers. Journal of the Experimental Analysis of Behavior 20, 163181. JOHN, E. R., CHESLER, P., BARTLETT, F. & VICTOR, I. (1968). Observation learning in cats. Science 159, 14891490. KANDEL, E. R. (1976). Cellular Basis of Behavior. Freeman, San Francisco. KANDEL, E. R. (1991). Cellular mechanisms of learning and the biological basis of individuality. In Principles of Neural Science, 3rd Edn (eds. E. R. Kandel, J. H. Schwartz and T. M. Jessell), pp. 10091031. Elsevier, Amsterdam. KANDEL, E. R. & SCHWARTZ, J. H. (1982). Molecular biology of learning : modulation of transmitter release. Science 218, 433 443. KEETON, W. T. (1969). Orientation by pigeons : is the sun necessary ? Science 165, 922928. KEETON, W. T. (1971). Magnets interfere with pigeon homing. Proceedings of the National Academy of Sciences USA 68, 102106. KEETON, W. T. (1974). The orientational and navigational basis of homing in birds. Advances in the Study of Behavior 5, 47132. KELLEHER, R. T. (1958). Fixed-ratio schedules of conditioned reinforcement. Journal of the Experimental Analysis of Behavior 1, 281289. KELLEHER, R. T. (1962). Variables and behavior. American Psychologist 17, 659660. KENT, J. P. (1987). Experiments of the relationship between the hen and the chick (Gallus) : the role of the auditory mode in recognition and the eects of maternal separation. Behaviour 102, 114. KEVAN, P. G. (1976). Sir Thomas More on imprinting : observations from the sixteenth century. Animal Behaviour 24, 1617. KIMBLE, G. A. (1961). Hilgard & Marquis Conditioning and Learning. Appleton-Century-Crofts, New York. KING, D. L. (1979). Conditioning, an Image Approach. Gardner Press, New York. KIRBY, A. J. (1968). Explorations of the BrownJenkins autoshaping phenomenon. M.A. thesis, Dalhousie University, Halifax. KLINGHAMMER, E. (1967). Factors influencing choice of mate in altricial birds. In Early Behavior : Comparative and Developmental Approaches (ed. H. W. Stevenson), pp. 5 42. Wiley, New York. KONISHI, M. (1963). The role of auditory feedback in the vocal behavior of the domestic fowl. Zeitschrift fu r Tierpyschologie 20, 349367. KONISHI, M. (1965). The role of auditory feedback in the control of vocalization in the White-crowned sparrow. Zeitschrift fu r Tierpyschologie 22, 770783. KONISHI, M. (1985). Birdsong : from behavior to neuron. Annual Review of Neuroscience 8, 125170. KONISHI, M. & KENUK, A. S. (1975). Discrimination of noise spectra in the barn owl. Journal of Comparative Physiology 97, 5558. KONORSKI, J. (1967). Integrative Activity of the Brain. University of Chicago Press, Chicago, IL. KONORSKI, J. (1969). Postscript. Journal of the Experimental Analysis of Behavior 12, 189. KRAMER, G. (1953). Wird die Sonnenho he bei der Heimndeorientierung verwertet ? [Is solar altitude utilized in homeward orientation ?]. Journal of Ornithology 94, 201219. KRAMER, G. (1961). Long-distance orientation. In Biology and Comparative Physiology of Birds, Vol. 2 (ed. Marshall), pp. 341371. Academic Press, New York.

Bruce R. Moore
KREITHEN, M. L. & KEETON, W. T. (1974 a). Detection of polarized light by the homing pigeon, Columba livia. Journal of Comparative Physiology A 89, 8392. KREITHEN, M. L. & KEETON, W. T. (1974 b). Attempts to condition homing pigeons to magnetic stimuli. Journal of Comparative Physiology A 91, 355362. KREITHEN, M. L. & QUINE, D. B. (1979). Infrasound detection by the homing pigeon : a behavioral audiogram. Journal of Comparative Physiology A 129, 1 4. KREMER, E. F. (1971). Truly random and traditional control procedures in CER conditioning in the rat. Journal of Comparative and Physiological Psychology 76, 441 448. KROODSMA, D. E. (1982). Learning and the ontogeny of sound signals in birds. In Acoustic Communication in Birds, Vol. 2 (eds. D. E. Kroodsma, E. H. Miller and H. Ouellet), pp. 123. Academic Press, New York. KROODSMA, D. E. & BAYLIS, J. R. (1982). Appendix : a world survey of evidence for vocal learning in birds. In Acoustic Communication in Birds : Vol. 2 (eds. D. E. Kroodsma, E. H. Miller and H. Ouellet), pp. 313337. Academic Press, New York. KRUIJT, J. P. & MEEUWISSEN, G. B. (1991). Sexual preferences of male zebra nches : eects of early and adult experience. Animal Behaviour 42, 91102. LADE, B. I. & THORPE, W. H. (1964). Dove songs as innately coded patterns of specic behaviour. Nature 202, 366368. LASHLEY, K. S. & WADE, M. (1946). The Pavlovian theory of generalization. Psychological Review 53, 7287. LAWRENCE, J. G. & OCHMANN, H. (1998). Molecular archaeology of the Escherichia coli genome. Proceedings of the National Academy of Sciences USA 95, 94139417. LEFEBVRE, L. & GIRALDEAU, L.-A. (1994). Cultural transmission in pigeons is aected by the number of tutors and bystanders present. Animal Behaviour 47, 331337. LEIBRECHT, B. C. (1972). Habituation, 19401970 : bibliography and key-word index. Psychonomic Monograph Supplement 4 (No. 11, Whole No. 59), 189217. LEOPOLD, W. F. (1939). Speech Development of a Bilingual Child, Volume 1. Vocabulary in the First Two Years. Northwestern University Press, Evanston, IL. LEOPOLD, W. F. (1949). Speech Development of a Bilingual Child, Volume 3. Grammar and General Problems. Northwestern University Press, Evanston, IL. LOGAN, F. A. (1969). Fundamentals of Learning and Motivation. Wm. C. Brown, Dubuque, IA. LORENZ, K. Z. (1937). The companion in the birds world. Auk 54, 245273. MACKINTOSH, N. J. (1974). The Psychology of Animal Learning. Academic Press, London. MACKINTOSH, N. J. (1983). Conditioning and Associative Learning. Clarendon Press, Oxford. MACPHAIL, E. M. (1982). Brain and Intelligence in Vertebrates. Clarendon Press, Oxford. MARGULIS, L. (1993). Symbiosis in Cell Evolution, 2nd Edn. Freeman, New York. MARLER, P. (1964). Inheritance and learning in the development of animal vocalizations. In Acoustic Behavior of Animals (ed. R. G. Busnel), pp. 228243. Elsevier, Amsterdam. MARLER, P. (1976). Sensory templates in species-specic behavior. In Simpler Networks and Behavior (ed. J. C. Fentress), pp. 314329. Sinauer, Sunderland, MA. MARLER, P. (1981). Bird song : the acquisition of a learned motor skill. Trends in Neuroscience 4, 8894.

The evolution of learning

MARLER, P. (1991). Song-learning behaviour : the interface with neuroethology. Trends in Neuroscience 14, 199205. MARLER, P. & PETERS, S. (1981). Sparrows learn adult song and more from memory. Science 213, 780782. MASTERS, J. C. (1979). Untitled letter. Science 205, 215. MCCASLAND, J. S. (1987). Neuronal control of bird song production. Journal of Neuroscience 7, 2339. MELTZOFF, A. N. & MOORE, M. K. (1983). Newborn infants imitate adult facial gestures. Child Development 54, 702709. MILLER, S. & KONORSKI, J. (1969). On a particular form of the conditioned reflex. Journal of the Experimental Analysis of Behavior 12, 187189 (Original work published in 1928). MINEKA, S., DAVIDSON, M., COOK, M. & KEIR, R. (1984). Observational conditioning of snake fear in rhesus monkeys. Journal of Abnormal Psychology 93, 355372. MITCHELL, C. J., HEYES, C. M., GARDNER, M. R. & DAWSON, G. R. (1999). Limitations of a bidirectional control procedure for the investigation of imitation : odour cues on the manipulandum. Quarterly Journal of Experimental Psychology 52b, 193202. MOORE, B. R. (1971). On directed respondents. Ph.D. thesis, Stanford University, Stanford. MOORE, B. R. (1973). The role of directed Pavlovian responding in simple instrumental learning in the pigeon. In Constraints on Learning (eds. R. A. Hinde and J. S. Hinde), pp. 159187. Academic Press, London. MOORE, B. R. (1988). Magnetic elds and orientation in homing pigeons : experiments of the late W. T. Keeton. Proceedings of the National Academy of Sciences USA 85, 4907 4909. MOORE, B. R. (1992). Avian movement imitation and a new form of mimicry : tracing the evolution of a complex form of learning. Behaviour 122, 231263. MOORE, B. R. (1996). The evolution of imitative learning. In Social Learning in Animals : The Roots of Culture (eds. C. M. Heyes and B. G. Galef, Jr.), pp. 245265. Academic Press, London. MOORE, B. R. & ROGERS, T. (1997). Observational conditioning ? Paper presented at Conference on Comparative Cognition, Melbourne, FL. MOORE, B. R., STANHOPE, K. J. & WILCOX, D. (1987). Pigeons fail to detect low-frequency magnetic elds. Animal Learning and Behavior 15, 115117. MOORE, B. R. & STUTTARD, S. (1979). Dr. Guthrie and Felis domesticus or : tripping over the cat. Science 205, 10311033. MOORE, J. W. & GORMEZANO, I. (1961). Yoked comparisons of instrumental and classical eyelid conditioning. Journal of Experimental Psychology 62, 552559. MORGAN, C. L. (1894). An Introduction to Comparative Psychology. Walter Scott, London. MORGAN, C. L. (1896). Habit and Instinct. Edward Arnold, London. MORSE, W. H., MEADE, R. M. & KELLEHER, R. T. (1967). Modulation of elicited behavior by a xed-interval schedule of electric shock presentation. Science 157, 215217. MOWRER, O. H. (1947). On the dual nature of learning a reinterpretation of conditioning and problem-solving . Harvard Educational Review 17, 102148. MOZART, L. (1951). A Treatise on the Fundamental Principles of Violin Playing (E. Knocker, translated). Oxford University Press, London (Original work published in 1756). MURPHY, A. J., SAVINO, J., RYNN, J. M. W., CHOY, G. L. & MCCAMY, K. (1972). Observation of long-period (10100 sec) noise at several worldwide locations. Geophysical Research 77, 50125019.

333
NAUSS, M. & WALLRAFF, H. G. (1988). Orientation of displaced homing pigeons with shifted circadian clocks : prediction vs observation. Naturwissenschaften 75, 363365. NEURINGER, A. (1974). Learning by following a food source. Science 184, 10051008. NOTTEBOHM, F. (1972). The origins of vocal learning. American Naturalist 106, 116140. NOTTEBOHM, F. (1980). Testosterone triggers growth of brain vocal control nuclei in adult female canaries. Brain Research 189, 429 436. NOTTEBOHM, F. (1981 a). A brain for all seasons : cyclical anatomical changes in song control nuclei of the canary brain. Science 124, 13681370. NOTTEBOHM, F. (1981 b). Laterality, seasons and space govern the learning of a motor skill. Trends in Neuroscience 4, 104106. NOTTEBOHM, F. (1991). Reassessing the mechanisms and origins of vocal learning in birds. Trends in Neuroscience 14, 206211. OLGUIN, R. & TOMASELLO, M. (1993). Twenty-ve-month-old children do not have a grammatical category of verb. Cognitive Development 8, 245272. PALAMETA, B. (1989). The importance of socially transmitted information in the acquisition of novel foraging skills by pigeons and canaries. Ph.D. thesis, University of Cambridge, Cambridge. PAPI, F. (1992). Animal Homing. Chapman & Hall, London. PAPI, F. & WALLRAFF, H. G. (1982). Avian Navigation. SpringerVerlag, Berlin. PATTEN, R. L. & RUDY, J. W. (1967). The Sheeld omission training procedure applied to the conditioning of the licking response in rats. Psychonomic Science 8, 463 464. PAVLOV, I. P. (1913). An address on the investigation of the higher nervous functions. British Medical Journal II, 973978. PAVLOV, I. P. (1927). Conditioned Reflexes (G. V. Anrep, translation). Oxford University Press, Oxford (Original work published 1926). PAVLOV, I. P. (1930). A brief outline of the higher nervous activity. In Psychologies of 1930 (ed. C. Murchison), pp. 207220. Clark University Press, Worcester, MA. PAVLOV, I. P. (1932). The reply of a physiologist to psychologists. Psychological Review 39, 91127. PAVLOV, I. P. (1934). An attempt at a physiological interpretation of obsessional neurosis and paranoia. Journal of Mental Science 80, 187197. PAVLOV, I. P. (1941). Lectures on Conditioned Reflexes : Vol. 2. Conditioned Reflexes and Psychiatry (W. H. Gantt, translation). International, New York. PAVLOV, I. P. (1955). Selected Works. Foreign Languages Publishing House, Moscow. PAYNE, K., TYACK, P. & PAYNE, R. (1983). Progressive changes in the songs of Humpback whales (Megaptera novaeangliae) : a detailed analysis of two seasons in Hawaii. In Communication and Behavior of Whales (ed. R. Payne), pp. 957. A.A.A.S., Washington, DC. PEEKE, H. V. S. & HERTZ, M. J. (1973). Habituation. Academic Press, New York. PEEKE, H. V. S. & PETRINOVICH, L. (1984). Habituation, Sensitization, and Behavior. Academic Press, New York. PETERSON, G. B., ACKIL, J. E., FROMMER, G. P. & HEARST, E. S. (1972). Conditioned approach and contact behavior toward signals of food or brain-stimulation reinforcement. Science 177, 10091011. PIAGET, J. (1951). Play, Dreams, and Imitation in Childhood (C. Gattegno & F. M. Hodgson, translation). Heinemann, London (Original work published 1945).

334
PIATTELLI-PALMARINI, M. (1989). Evolution, selection, and cognition : from learning to parameter setting in biology and in the study of language. Cognition 31, 1 44. PINSKER, H. M., HENING, W. A., CAREW, T. J. & KANDEL, E. R. (1973). Long-term sensitization of a defensive withdrawal reflex in Aplysia. Science 182, 10391042. POTTER, B. (1966). The Journal of Beatrix Potter from 1881 to 1897. Decoded by Leslie Linder. Frederick Warne, London (Original manuscript written 1896). PREMACK, D. (1990). On the coevolution of language and social competence. Behavioral and Brain Sciences 13, 754756. PROSSER, C. L. & HUNTER, W. S. (1936). The extinction of startle reflexes and spinal reflexes in the white rat. American Journal of Physiology 117, 609618. QUIRING, R., WALLDORF, U., KLOTER, U. & GEHRING, W. J. (1994). Homology of the eyeless gene of Drosophila to the Small eye gene in mice and Aniridia in humans. Science 265, 785789. RACKHAM, D. W. (1977). Discriminative courtship conditioning in the pigeon (Columba livia). Ph.D. thesis, Dalhousie University, Halifax. RALPH GERBRANDS CO. (ca. 1967). Scientic Instruments for Psychology and Physiology. Arlington, MA. RAZRAN, G. H. S. (1938). Transposition of relational responses and generalization of conditioned responses. Psychological Review 45, 532538. RAZRAN, G. (1971). Mind in Evolution. Houghton Milin, Boston. RESCORLA, R. A. (1967). Pavlovian conditioning and its proper control procedures. Psychological Review 74, 7180. REVUSKY, S. H. (1968). Aversion to sucrose produced by contingent X-irradiation. Journal of Comparative and Physiological Psychology 65, 1722. REVUSKY, S. (1977). Interference with progress by the scientic establishment : examples from flavor aversion learning. In Food Aversion Learning (eds. N. W. Milgram, L. Krames and T. M. Alloway), pp. 5371. Plenum, New York. REVUSKY, S. & GARCIA, J. (1970). Learned associations over long delays. In The Psychology of Learning and Motivation, Vol. 4 (ed. G. H. Bower), pp. 184. Academic Press, New York. RODGERS, W. & ROZIN, P. (1966). Novel food preference in thiamine-decient rats. Journal of Comparative and Physiological Psychology 61, 1 4. ROMER, A. S. (1966). Vertebrate Paleontology. University of Chicago, Chicago, IL. ROZIN, P. (1967). Specic aversions as a component of specic hungers. Journal of Comparative and Physiological Psychology 64, 237242. ROZIN, P. (1969). Adaptive food sampling patterns in vitamin decient rats. Journal of Comparative and Physiological Psychology 69, 126132. ROZIN, P. & RODGERS, W. (1967). Novel-diet preferences in vitamin-decient rats and rats recovered from vitamin deciency. Journal of Comparative and Physiological Psychology 63, 421 428. RUSSON, A. & GALDIKAS, B. (1993). Imitation in free-ranging orangutans (Pongo pygmaeus). Journal of Comparative Psychology 107, 147161. SALVINI-PLAWEN, L. VON & MAYR, E. (1977). On the evolution of photoreceptors and eyes. In Evolutionary Biology, Vol. 10 (eds. M. K. Hecht, W. C. Steere and B. Wallace), pp. 207263. Plenum, New York. SCHLAICH, J. & DUPONT, B. (1993). The Art of Teaching Dance Technique. National Dance Association, Reston, VA.

Bruce R. Moore
SCHLOSSBERG, H. (1937). The relationship between success and the laws of conditioning. Psychological Review 44, 379394. SCHMIDT-KOENIG, K. (1960). Internal clocks and homing. Cold Spring Harbor Symposium on Quantitative Biology 25, 389393. SCHMIDT-KOENIG, K. & SCHLICHTE, H. J. (1972). Homing in pigeons with impaired vision. Proceedings of the National Academy of Sciences USA 69, 24462447. SCHMIDT-KOENIG, K. & WALCOTT, C. (1978). Tracks of pigeons homing with frosted lenses. Animal Behaviour 26, 480 486. SCHWENDENER, S. (1869). Die Algentypen der Flechtgoniden. Programm fu r die Rectorsfeier der Universita t Basel 4, 1 42. SCIENTIFIC PROTOTYPE MANUFACTURING CORPORATION, EQUIPMENT CATALOGUE (1970). New York. SCOLLON, R. (1976). Conversations With a One Year Old. University of Hawaii Press, Honolulu. SHARPLESS, S. & JASPER, H. (1956). Habituation of the arousal reflex. Brain 79, 655680. SHEFFIELD, F. D. (1954). A drive induction theory of reinforcement. Unpublished manuscript, Yale University, New Haven, CT. SHEFFIELD, F. D. (1965). Relation between classical conditioning and instrumental learning. In Classical Conditioning : A Symposium (ed. W. F. Prokasy). Appleton-Century-Crofts, New York. SHERRINGTON, C. S. (1900). The muscular sense. In Textbook of fer), pp. 10021025. Pentland, London. Physiology (ed. E. A. Scha SHERRINGTON, C. S. (1906). The Integrative Action of the Nervous System. Oxford University Press, London. SIBLEY, C. G. & AHLQUIST, J. E. (1990). Phylogeny and Classication of Birds. Yale University Press, New Haven, CT. SKINNER, B. F. (1938). The Behavior of Organisms. Appleton-CenturyCrofts, New York. SKINNER, B. F. (1948). Superstition in the pigeon. Journal of Experimental Psychology 38, 168172. SKINNER, B. F. (1959). A case history in scientic method. In Psychology : A Study of a Science, Volume 2 (ed. S. Koch), pp. 359379. McGraw-Hill, New York. SLATER, P. J. B. (1983). Bird song learning : theme and variations. In Perspectives in Ornithology (eds. A. H. Bush and G. A. Clark, Jr.), pp. 475 499. Cambridge University Press, London. SMITH, R. F. (1967). Behavioral events other than key striking which are counted as responses during key pecking. Ph.D. thesis, Indiana University, Bloomington. SOKOLOV, YE. N. (1955). Perception and the Conditioned Reflex. Pergamon, London. SOLOMON, R. L. (1964). Punishment. Psychological Review 19, 239253. SOLOMON, R. L., KAMIN, L. J. & WYNNE, L. C. (1953). Traumatic avoidance learning : the outcomes of several extinction procedures with dogs. Journal of Abnormal and Social Psychology 48, 391 402. SOLOMON, R. L. & WYNNE, L. C. (1953). Traumatic avoidance learning : acquisition in normal dogs. Psychological Monographs 67, No. 4, Whole No. 354. SOLOMON, R. L. & WYNNE, L. C. (1954). Traumatic avoidance learning : the principles of anxiety conservation and partial irreversibility. Psychological Review 61, 353385. SPALDING, D. (1873). Instinct : with original observations on young animals. Macmillans Magazine 27, 282293. SPENCER, W. A., THOMPSON, R. F. & NEILSON, D. R., JR. (1966). Response decrement of the flexion reflex in the acute spinal cat and transient restoration by strong stimuli. Journal of Neurophysiology 29, 221239.

The evolution of learning

STADDON, J. E. R. (1983). Adaptive Behavior and Learning. Cambridge University Press, Cambridge. STADDON, J. E. R. & SIMMELHAG, V. L. (1971). The Superstition experiment : a reexamination of its implications for the principles of adaptive behavior. Psychological Review 78, 3 43. STANHOPE, K. J. (1989). Dissociation of the eect of reinforcer type and response strength on the force of a conditioned response. Animal Learning and Behavior 17, 311321. STANHOPE, K. J. (1989). Dissociation of the eect of reinforcer type and response strength on the force of a conditioned response. Animal Learning and Behavior 17, 311321. STE PIEN , I. (1974). The magnet reaction, a symptom of prefrontal ablation. Acta Neurobiologiae Experimentalis 34, 145160. STEVEN, D. M. (1955). Transference of imprinting in a wild gosling. British Journal of Animal Behaviour 3, 1416. STOKES, P. D. & BALSAM, P. D. (1991). Eects of reinforcing preselected approximations on the topography of the rats bar press. Journal of the Experimental Analysis of Behavior 55, 213231. SWIFT, S. (1985). Centered Riding. Trafalgar Square Farm Books, St. Martins/Marek, New York. TAYLER, C. K. & SAAYMAN, G. S. (1973). Imitative behaviour by Indian Ocean Bottlenose Dolphins (Tursiops aduncus) in captivity. Behaviour 44, 286298. TEBBICH, S., TABORSKY, M., FESSL, B. & BLOMQVIST, D. (2001). Do woodpecker nches acquire tool-use by social learning ? Proceedings of the Royal Society of London B 268, 21892193. TELEKI, G. (1974). Chimpanzee subsistence technology : materials and skills. Journal of Human Evolution 3, 575594. TEN CATE, C., VOS, D. R. & MANN, N. (1993). Sexual imprinting and song learning : two of one kind ? Netherlands Journal of Zoology 43, 3454. TERKEL, J. (1996). Cultural transmission of feeding behavior in the black rat (Rattus). In Social Learning in Animals : The Roots of Culture (eds. B. G. Galef, Jr. and C. M. Heyes), pp. 17 47. Academic Press, London. THOMPSON, R. F. & SPENCER, W. A. (1966). Habituation : a model phenomenon for the study of neuronal substrates of behavior. Psychological Review 73, 16 43. THORNDIKE, E. L. (1898). Animal intelligence. Psychological Review, Supplement 2, 1109. THORNDIKE, E. L. (1901). Mental life of the monkeys. Psychological Monograph Supplements 3 (5, Whole No. 15). THORNDIKE, E. L. (1935). The Psychology of Wants, Interests and Attitudes. D. Appleton-Century, New York. THORPE, W. H. (1956). Learning and Instinct in Animals. Harvard University Press, Cambridge, MA. THORPE, W. H. (1958). The learning of song patterns by birds, with special reference to the song of the chanch Fringilla coelebs. Ibis 100, 535570. THORPE, W. H. (1961). Bird-Song. Cambridge University Press, Cambridge. THORPE, W. H. (1966). Vocal imitation and antiphonal song and its implications. In Proceedings of the XIV International Ornithological Congress (ed. D. W. Snow), pp. 245263. Blackwell, Oxford. TIMBERLAKE, W. & LUCAS, G. A. (1985). The basis of superstitious behavior : chance contingency, stimulus substitution, or appetitive behavior ? Journal of the Experimental Analysis of Behavior 44, 279199. TODT, D. (1975). Social learning of vocal patterns and modes of their application in Grey parrots (Psittacus erithacus). Zeitschrift fu r Tierpsychologie 39, 178188.

335
TOMASELLO, M. (1992). First Verbs. A Case Study of Early Grammatical Development. Cambridge University Press, Cambridge. TUMLINSON, J. H., LEWIS, W. J. & VET, L. E. M. (1993). How parasitic wasps nd their hosts. Scientic American March, 100106. VAN DER POST, L. (1958). The Lost World of the Kalahari. Hogarth Press, London. VAN KAMPEN, H. S. (1993). Filial imprinting and associative learning : similar mechanisms ? Netherlands Journal of Zoology 43, 143154. VAN KAMPEN, H. S. & BOLHUIS, J. J. (1991). Auditory learning and lial imprinting in the chick. Behaviour 117, 303319. VINES, G. (1981). Wolves in dogs clothing. New Scientist September, 648652. WAGNER, H. O. (1944). Notes on the life history of the Emerald toucanet. Wilson Bulletin 56, 6576. WAHLSTEN, D. L. & COLE, M. (1972). Classical and avoidance training of leg flexion in the dog. In Classical Conditioning II : Current Research and Theory (eds. A. H. Black and W. F. Prokasy), pp. 379 408. Appleton-Century-Crofts, New York. WALLIN, I. E. (1925). On the nature of mitochondria. IX. Demonstration of the bacterial nature of mitochondria. American Journal of Anatomy 36, 131149. WASSERMAN, E. A. (1974). Pavlovian conditioning with heat reinforcement produces heat-directed pecking in chicks. Science 181, 875877. WEBER, H. & WENDT, G. R. (1942). Conditioning of eyelid closure with various conditions of reinforcement. Journal of Experimental Psychology 30, 114124. WELLS, M. J. (1964). Learning and movement in octopuses. In Learning and Associated Phenomena in Invertebrates, Animal Behaviour, Supplement I (eds. W. H. Thorpe and D. Davenport), ` re, Tindall & Cassell, London. pp. 115128. Ballie WELLS, M. J. (1967). Sensitization and the evolution of associative learning. In Symposium on Neurobiology of Invertebrates (ed. J. Salanki), pp. 391 411. Plenum, New York. WELLS, M. J. (1968). Lower Animals. World University Library, McGraw Hill, New York. WESTERGAARD, G. C. & FRAGASZY, D. M. (1987). The manipulation and use of tools by Capuchin monkeys. Journal of Comparative Psychology 101, 159168. WILLIAMS, D. R. & WILLIAMS, H. (1969). Auto-maintenance in the pigeon : sustained pecking despite contingent non-reinforcement. Journal of the Experimental Analysis of Behavior 12, 511520. WITCHELL, C. A. (1896). The Evolution of Bird Song. Adam and Charles Black, London. WOLIN, B. R. (1968). Dierence in manner of pecking a key between pigeons reinforced with food and with water. (Paper read at the Conference on the Experimental Analysis of Behavior, 1948.) In Contemporary Research in Operant Behavior (ed. A. C. Catania), p. 286. Scott Foresman, Glenview, IL. WYERS, E. J., PEEKE, H. V. S. & HERTZ, M. J. (1973). Behavioral habituation in invertebrates. In Habituation (eds. H. V. S. Peeke and M. J. Hertz), pp. 157. Academic Press, New York. WYRWICKA, W. (1996). Imitation in Human and Animal Behavior. Transaction Publishers, London. YODLOWSKI, M. L., KREITHEN, M. L. & KEETON, W. T. (1977). Detection of atmospheric infrasound by homing pigeons. Nature 265, 725726. ZENTALL, T. R. & HOGAN, D. E. (1976). Imitation and social facilitation in the pigeon. Animal Learning and Behavior 4, 427 430. ZENTALL, T., SUTTON, J. E. & SHERBURNE, L. M. (1996). True imitative learning in pigeons. Psychological Science 7, 343346.