See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/264501060

The origin of human multi-modal communication

Article  in  Philosophical Transactions of The Royal Society B Biological Sciences · September 2014

DOI: 10.1098/rstb.2013.0302 · Source: PubMed

CITATIONS READS

114 1,239

2 authors:

Stephen C Levinson Judith Holler

Max Planck Institute for Psycholinguistics Radboud University
344 PUBLICATIONS   30,063 CITATIONS    67 PUBLICATIONS   1,237 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Speech Planning and Turn Timing in Conversation View project

Gestural communication in pain-focused interactions View project

All content following this page was uploaded by Judith Holler on 12 August 2014.

The user has requested enhancement of the downloaded file.

 Downloaded from rstb.royalsocietypublishing.org on August 5, 2014

The origin of human multi-modal

communication
Stephen C. Levinson1,2 and Judith Holler1
1
Language and Cognition Department, Max Planck Institute for Psycholinguistics, Wundtlaan 1,
6525XD Nijmegen, The Netherlands
rstb.royalsocietypublishing.org 2
Linguistics Department, Radboud University Nijmegen, The Netherlands

One reason for the apparent gulf between animal and human communi-
cation systems is that the focus has been on the presence or the absence of
language as a complex expressive system built on speech. But language nor-
Opinion piece mally occurs embedded within an interactional exchange of multi-modal
signals. If this larger perspective takes central focus, then it becomes appar-
ent that human communication has a layered structure, where the layers
Cite this article: Levinson SC, Holler J. 2014
may be plausibly assigned different phylogenetic and evolutionary
The origin of human multi-modal
origins—especially in the light of recent thoughts on the emergence of
communication. Phil. Trans. R. Soc. B 369: voluntary breathing and spoken language. This perspective helps us to
20130302. appreciate the different roles that the different modalities play in human
http://dx.doi.org/10.1098/rstb.2013.0302 communication, as well as how they function as one integrated system
despite their different roles and origins. It also offers possibilities for recon-
ciling the ‘gesture-first hypothesis’ with that of gesture and speech having
One contribution of 12 to a Theme Issue
evolved together, hand in hand—or hand in mouth, rather—as one system.
‘Language as a multimodal phenomenon:
implications for language learning, processing
and evolution’.
1. Introduction
Human communication is unusual in the animal world on at least two principal
Subject Areas: counts: it has an unrivalled complexity and expressivity on the one hand, and an
behaviour, cognition, evolution unparalleled inter-group variation on the other. The combination is extraordi-
nary, because the variation within an unusually genetically homogeneous
species excludes a fully biological explanation. In this paper, we take the view
Keywords:
that human communication is evolutionarily stratified, composed of layers of
language evolution, intentional vocalization, abilities of different types and different antiquity. A wide range of scholars
gesture, multi-modal communication, of different perspectives seem to subscribe to such a general view (e.g. [1,2]).
deixis, iconicity But in this paper we suggest that viewing language as embedded in its full prag-
matic, interactive and multi-modal context transforms this stratificational
perspective. Unpeeling the layers can then help us see the different contributions
Author for correspondence:
of the distinct systems that underlie the peculiarities of human communication.
Stephen C. Levinson It is often hard for the literate world to remember that the core ecology for
e-mail: stephen.levinson@mpi.nl language use is in face-to-face interaction—this is the niche in which languages
are learnt and where the great bulk of language use occurs. In this niche,
language production always occurs with the involvement of not only the
vocal tract and lungs, but also the trunk, the head, the face, the eyes and, nor-
mally, the hands. Our upright posture allows the whole ventral surface of the
body to be used in communication. The speaker produces a multi-modal dis-
play, part semiotic, part entrained by the effort of vocal production. When
thinking about the origins of human language, it is essential to bear this ensem-
ble of linked systems in mind. The ease with which human language switches
the main channel or modality carrying lexical material from mouth to hands, as
in the sign languages of the deaf, should be a constant reminder that human
communication is a system of systems, where the burden of information can
be shifted from one part to another (see also [3–6]).

2. Human communication as an evolutionarily stratified system

The modern human communication system is, on a biological time-scale, a
recent innovation. The standard line is that the emergence of language

& 2014 The Author(s) Published by the Royal Society. All rights reserved.
 Downloaded from rstb.royalsocietypublishing.org on August 5, 2014

followed, perhaps with a 100 000 year lag, by the emergence other capacities. Indeed, careful observation of human inter- 2
of anatomically modern humans about 200 000 years ago. action reveals the very same process in constant operation—if

rstb.royalsocietypublishing.org
Clearly, however, it must predate the great diaspora of I reach as if to get the water, you pass it to me. Much of the
modern humans thought to date to 60 000 years ago. In a facility with which we interact and cooperate depends on
recent meta-study surveying all the genetic, anatomical and precisely this kind of nonce gesture [33].
archaeological evidence, Dediu & Levinson [7] argue that a Researchers in linguistic pragmatics have long held that
compelling case can be made for a much earlier origin of language is the tip of an iceberg riding on a deep infrastruc-
modern vocal language at over half a million years ago, ture of communicational abilities [34,35]. Simple utterances
dating back to the common ancestor (often identified as are rarely interpreted at face value—thus Are you using that
Homo heidelbergensis) of modern humans and Neandertals pencil? is likely to be read as a request, Do you want another
(see [8] for additional evidence supporting the idea of an beer? as an offer, What are you doing tonight? as a prelude
early origin of modern human language). The development to an invitation, and so on. Grice [36] emphasized the

Phil. Trans. R. Soc. B 369: 20130302

of speech required a by-pass of the involuntary breathing presumption of cooperation and the reciprocal recon-
control involved in primate calls and a direct connection to struction of communicative intentions, while Clark [3] has
the primary motor cortex, on the one hand, and to the inter- emphasized the joint activities in which communication is
costal muscles via the vertebral column, on the other [9,10]. embedded, and conversation analysts [37] have stressed the
This enlarged enervation is missing from the vertebral complex interactional conventions or practices that impose
column of Homo erectus in the one well-preserved fossil that semiotic significance onto utterances (as when ‘forty three’
has come down to us, with an age of 1.4 Ma [11]. Although is an answer to a specific question about someone’s age
pre-adaptive changes in the thoracic vertebrae may have or house number, etc.). These different viewpoints all
occurred for other reasons such as singing [12] or running contribute to our understanding of the large inferential
[13], they would have rapidly come to play a crucial role in background to human communication, and its special role
speech [14]. So, it seems probably that the origin of modern in the origin of human communication has been argued
speech, and the prevalence of the vocal tract in human com- for by Tomasello [2,38].
munication, would seem to lie somewhere between 1.4 Ma Recently, it has become clear that many aspects of this
and ca 600 000 years ago. This remains, of course, a recent pragmatic background to human communication are univer-
date, compared to, for example, the 50 Myr of evolution sal, or at least highly similar across cultures. The inferential
behind birdsong or bat echolocation. system that Grice called ‘generalized conversational implica-
ture’ seems to be universal in principle and generates small
inferences from (sometimes language-specific) contrasts in
(a) Pragmatics as the foundation of human the lexicon [39]. There are clear tendencies for a core inven-
communication tory of speech acts like questions, greetings, requests and so
But speech is just one system in the ‘system of systems’ that on, to recur across cultures, even though many actions
constitutes human communication, and given its recent ori- beyond the core vary [40]. Above all, the interactional
gins, there is every reason to expect that it was a late layer system that is essential for human communication seems to
on other systems. This would fit well with a gestural theory have a strong common foundation across all cultures. For
of language origins [2,15–24]. Call & Tomasello [25] show example, the core niche for language use in all cultures is a
that among our closest relatives in the Hominidae, intentional speech and gesture exchange system in which participants
communicative acts are signalled largely by non-vocal take short, rapidly alternating turns. It is in this context that
means, especially with the hands (but see [26,27] on the the great bulk of language production occurs—extrapolating
intentional nature of at least some great ape vocalizations). from earlier studies, we each seem to produce about 16 000
Many of these gestures have an iconic or indexical meaning, words in about 1200 turns a day in conversation [41]. Turns
for example, an offering of something presented with the on average across languages have only about 200 ms between
hand or other body parts [28], a request made with an them [42], which is extraordinary if one bears in mind that
open hand [29], or a request to be picked up by an infant the latencies involved in producing even a single word are
with raised arms [30]. Such semiotic functions can be readily of the order of 600 ms [43]. Consequently, most language
associated by ontogenetic ritualization—the use of an initial use in conversation is based on a predictive look-ahead
gesture in a sequence coming to indicate the full sequence (e.g. [44–46]). Turn-taking appears remarkably early in
of actions [31]. In the great apes, ritualized gestures emerge infancy, certainly within the first year and well before the
in the following way: A does something XYZ, which routinely first words [47]. It is not a simple reflex of the limitations of
leads B to do something else; B comes to respond on the basis the vocal–auditory channel, as it occurs equally in manual
of only the first step, X, of A’s action; subsequently, A antici- sign languages. Interestingly, vocal alternation also occurs
pates B’s response and only performs X in order to elicit the (if patchily) right across the primate order, from lemurs [48]
response [25, pp. 7–8]. In a broader sense, ritualization is to marmosets [49,50], from Campbell’s monkeys [51] to gib-
the major route by which all animal communication systems bons [52]. Although it is, curiously, not a major feature of
seem to arise [32], but in the great apes, including us, the pro- communication among the great apes, the trait is perhaps
cess takes place by learning between interacting dyads, rather persistent enough to suggest homology, though convergent
than by biological evolution—what is biologically required are evolution cannot be ruled out [53,54].
the motivation and ability to predict an action sequence As noted, many speech acts seem universal in character, and
from an initial part, and to conventionalize it as a means so do the sequences of actions they construct—for example,
of expression. By the standard phylogenetic reasoning, we pairs of questions and answers, offers and acceptances, greet-
should have inherited this system, and indeed we almost cer- ings and greetings, and so forth. These sequences, though
tainly have, but have supplemented it with speech among typically expressed in language, are also embodied in other
 Downloaded from rstb.royalsocietypublishing.org on August 5, 2014

ways: a request ( perhaps visual) may prompt a visible action, a persons without verbal reference at all [75]. Village sign 3
wave another wave back, a passing of a needed item a recipro- languages are systems that have arisen de novo over four or

rstb.royalsocietypublishing.org
cal grasping and so forth. They also appear to have a largely more generations in remote areas where there are significant
universal structure, and recent work suggests that precursors pockets of inherited deafness; these typically make extensive
of some of these very same visual, bodily action sequences use of pointing, to the extent, for example, where the
can be seen in apes [30,55]. Sequences can be embedded in language has no need for place-names [76].
other sequences, so offering recursive structures that may be While pointing has a strong symbolic element with its
the ultimate origin of recursion in language syntax [40]. arbitrary and species-specific use of the index finger, many
Human interactional ethology, despite the presence of of the details of pointing are indexical—for example, the
precursor elements in other primate species, as a whole varied hand shapes, orientations and elevations of the arm
ensemble is entirely distinctive. For example, the toleration are adapted to the relevant aspects of the surroundings, as
or indeed expectation of mutual gaze is of paramount impor- when the distance of an unseen referent is indicated with

Phil. Trans. R. Soc. B 369: 20130302

tance in humans [56] but occurs less in other primates the vertical angle of the arm, probably because more distant
(e.g. [57], see also [58]), and the white sclera of the human objects are higher in the field of view on a plane surface. Since
eye has almost certainly evolved to enhance gaze detection the kinds of gestures that derive from ritualization are also
[59]—remarkably, human infants are sensitive to the differ- largely indexical in character, indexicality may play a much
ence between direct and averted gaze from just 2– 5 days greater role in the gestural and visual modalities than is
after birth [60]. The rapid turn-taking despite indefinitely usually thought, where iconicity is often given pride of place.
varying contents of turns is again without parallel; the sus- Iconic representation is nevertheless another important
tained multi-modal deployment of vocal and visual signals element in the accumulation of communicative capacities
on hands, face and body and the sheer amount of time and [20,77]. The apparently sudden appearance of iconic rep-
effort invested in communication seem without parallel resentations like cave paintings in the archaeological record
among the other primates. The capacities that lie behind some 30 000 years ago has even led some scholars to suggest
this unique ethology have been called the interaction engine that they index the birth of the modern mind [78,79]. How-
[61], a term we use later. ever, since Saussure’s insistence on the arbitrariness of the
linguistic sign, iconicity in language has been thought to lie
largely at the borders of the linguistic system, for example
(b) Indexicality and iconicity in language evolution in sound-symbolism. Some spoken languages nevertheless
One communicational peculiarity of humans is pointing, bundle considerable expressive power in a specific gramma-
especially with the index finger [2,62,63]. It appears in tical category, traditionally called expressives or ideophones
human infancy before language, before the first year and— (words conveying, for example, the quality of a sensory
as far as we know—across cultures [64]. On the one hand, experience by exploiting not only consistent meaning–
it may arise from ritualization of a reaching gesture [65], sound relationships but also structural analogy, as in
but this will only account for the so-called imperative point- reduplication [80,81]). Despite this lexical recognition, these
ings. Although apes display begging behaviour with the words resist deep incorporation into the grammatical
open hand [29,66], offer food in the palm [28] and may per- machinery of a language. This resistance reflects a tension
form what appear to be deictic gestures with the whole between grammatical integration and richness of iconic
hand [67] or localized referential actions with deictic elements form and meaning—once integrated, expressives become
[68], they do not point with the index finger (but see [69] for subject to the normal processes of reduction and regulariz-
some possible exceptions in captive apes). Declarative point- ation and lose most of their resonance [82]. This puts limits
ing, on the other hand, whose sole function is to draw the on the extent to which spoken language can fully exploit its
other’s attention to something, is human-specific in both own iconic potential, even though this is visible throughout
form and function—it typically involves the index finger linguistic systems (see [77] for a review), as for example in
extended and has an abstract symbolic function: the recipient prosody or in the sequential reading of Caesar’s veni, vidi,
is meant not to look at the finger, but in the vector indicated vici (I came, I saw, I conquered), which relies on the parallel
beyond the finger, and find some referent that is probably between linguistic time and event time (for wide-ranging
what the pointer had in mind as worthy of attention. principles of this type operating in grammar, see [83]).
It would therefore appear to be tightly linked to social cogni- Where iconicity comes into its own is in gesture. Iconic ges-
tive development [70], including Theory of Mind [71], tures are the gestures that mimic motion, depict size, trace
and for that reason has been heralded as an ontogenetic shapes or sketch the spatial relations between things [5,84],
milestone on the way to language (and, indeed, occurs in and they have intrigued researchers. The prime role that
the majority of cases with vocalizations [72]). Pointing is iconicity plays in gesture is the special affordances of gesture
thus an extremely powerful device, not because it accurately to indicate spatial relations. Spatial thinking seems central to
denotes (as Wittgenstein ([73], paragraph 33) noted, a point human cognition [75] and plays a role in structuring many
at, say, pieces of paper could be indicating the colour, the other cognitive domains, like time, kinds of social relationship
shape or even the number) but because it invites the recipient or set relations. The relative position of entities with respect to
to locate a referent of mutual interest (a social process that each other is frequently depicted by iconic gestures [85,86],
recruits reward-related neurocircuitry in humans [74]). In because the two hands provide a natural means of conveying
some cultures, pointing plays a fundamental role in com- the spatial relations between two (or more) things. And
munication, especially when there are conventions of whereas spoken language is, through its finite lexicon, invari-
accurate orientation: Guugu Yimithirr speakers, for example, ably coarse on spatial relations, gesture affords accurate
will inspect a gesture for precise directional ‘compass’ vera- depictions of angle, orientation and shape: the two together
city, which allows the elliptical reference to places and offer the complementarity of ‘digital’ and ‘analogue’ channels.
 Downloaded from rstb.royalsocietypublishing.org on August 5, 2014

The ability of the gestural modality to depict spatial relations hypothesis is that, given its language independence, the inter- 4
has implications beyond the spatial domain, for iconic gestures action engine is phylogenetically older than language, and

rstb.royalsocietypublishing.org
and signs are well suited to depicting transitivity, and thus perhaps characterized the communication of early Homo
agents and patients [87,88]. In the evolution of language, this before complex speech evolved. This system is itself no doubt
facility to indicate space must have given gesture a special phylogenetically layered, and one can speculate that the
importance (for a detailed discussion of the role of the visual more ethological elements (e.g. mutual gaze, gesture and
modality in the emergence of complex syntax, see [2]). In turn-taking) may have partially preceded and driven the
current human communication, gestures depicting spatial underlying cognitive capacity for evaluating other minds,
relations still carry a great deal of the communicative import since there are plausible precursors in the primate lineage
of informal conversation. (e.g. gestures among the great apes, and turn-taking in other
All this suggests a long phylogenetic acquisition of layers primate clades). The development of this system may in turn
of this system. At the deepest stratum, one may assume a have been pushed by ecological changes to which increased

Phil. Trans. R. Soc. B 369: 20130302

capacity for ad hoc gestural symbols ritualized from early cooperation was a necessary response [38]. We will here treat
elements of action sequences, as found among the present the interaction engine as a unified package, which is a simpli-
great apes. Gestures of this kind may have been early fication for current purposes. What is clear however from this
enriched with simple vocalizations that, among other package is that early human communication was highly
things, would draw visual attention to the signaller. A stra- visual and thus at least partially gestural, since it is otherwise
tum layered on top of this, before complex speech evolved, hard to account for the white sclera and the unusual toleration
might have consisted of a pointing-enriched manual sign and exploitation of mutual gaze.
system (something similar to a ‘gestural protolanguage’, as Once an interactional system of gesture (including pointing)
has been proposed by others, e.g. [16,89 –91]) and may is combined with some capacity for iconic representation,
have been associated with the speechless H. erectus, the itself a natural affordance of gesture, a system emerges that
first human species to exit Africa some 1.5 Ma spreading allows for communication about events happening elsewhere
throughout Eurasia and carrying a highly developed technol- or in the past or future (Hockett’s [93] design feature of
ogy including Acheulian (Mode 2) handaxes and the control ‘displacement’) [94]. Further, once the spatial representations
of fire. These handaxes are stone tools regularly flaked on of gesture have been exploited for relational encoding,
both sides to produce symmetrical, flattened pear-shaped we have the basis for the emergence of something like a
tools with sharp cutting edges, demonstrating advanced basic sign language (similar to a homesign or village sign
tool-making skills, which can only be replicated by months system), likely accompanied by simple vocalizations.
of practice under modern tuition [92]. This technology And, finally, as already indicated, it seems that complex
indexes a level of cultural replication that is pretty incon- vocal communication can be traced back somewhere before
ceivable without a complex and expressive communication H. heidelbergensis, at over half a million years ago, when volun-
system, perhaps approximating the complexity of village tary breathing was in place. A high degree of breath control is
sign or homesign systems. We think it likely that many required for modern speech, since the depth of inhalation
aspects of human communication ethology date back this must correlate with the length of what is to be expressed, and
far. Earlier varieties of humans, like Homo habilis, who was the timing with every point of stress. This required, as men-
active over 2 Ma, had much simpler tool assemblages, but tioned, a rewiring of both the central and the peripheral
nevertheless this was, compared to the modern great apes, nervous systems. There would be over 0.8 m years between
an advanced culture-carrying species that almost certainly H. erectus and H. heidelbergensis for the evolution of this
had some relatively complex communication system. system, and it was presumably acquired gradually in tandem
with the increasing reliance on the vocal channel.
Table 1 summarizes the suggestions made in this section. We
(c) Phylogeny of a multi-modal, sequential should add that much speculation on the evolution of language
communication system has gone on to suggest accumulated layers within the spoken
As indicated above, modern human communication exhibits a language system (e.g. [2,16,22,95]), suggestions that attempt to
specialized ethology not found in other animals: its face-to-face spell out the steps between first simple vocalizations and com-
character, which affords the full deployment of multiple articu- plex modern spoken language. A review of these extensive
lators, the frequent deployment of mutual gaze and the developments lies beyond our immediate purposes here.
sustained exchange of short but complex communicative The layered acquisition of our communicative compe-
turns are characteristics. In addition, there is the cognitive tences, as summarized in table 1, suggests that systems have
background of joint attention, common ground, collaboration been added but not replaced. The reason is that each layer
and the reasoning about communicative intent, as noted has its natural affordances: ritualized bodily reaching is still
by many commentators on human communication (e.g. a current practice used, for example, to make routine requests
[2,3,36]). This whole assemblage, which we have called the at the dinner table, pointing usefully augments speech and can
interaction engine [61], is visible early in human development be dovetailed with speech through the innovation of demon-
(e.g. in the ‘proto-conversation’ of six-month-old infants), strative words, iconic gestures can capture the movements,
and, in contrast to linguistic detail, strongly universal across shape, size or spatial arrangement of entities better than a
cultures [42], and allows us to communicate without spoken thousand words, and so forth. The gestural modality is
words, as when I indicate with a gesture that you have signs especially well adapted to communicating about these sorts
of your breakfast on your chin. In modern humans, it is this of visuo-spatial dimensions, which lie close to human preoc-
system that enables the acquisition or enlargement of spoken cupations and are central to human cognition [75]. Speech
or signed languages through ‘ritualization’ or the re-use of itself has many advantages: the small articulators permit
nonce signals that have been used successfully before. Our rapid and low-effort encoding, allow wide and distant signal
 Downloaded from rstb.royalsocietypublishing.org on August 5, 2014

Table 1. Hypothetical layers of communicative competencies as they evolved. 5

rstb.royalsocietypublishing.org
voluntary modern
ritualized ‘interaction iconic gestural vocal language
time-scale taxa gestures engine’ pointing representations utterances capacities

6 mþ Hominidae þ
2m early Homo þ þ þ þ
1 – 0.6 m immediate ancestor of þ þ þ þ þ
H. heidelbergensis
0.2 H. sapiens (incl. þ þ þ þ þ þ

Phil. Trans. R. Soc. B 369: 20130302

Neandertals)

broadcast, communication without sight and so on. Thus, a account of how the communication burden may gradually
multi-modal communication system that combines both ges- have shifted from hand to mouth), what results is one inte-
ture and speech, and thus their complementary strengths grated multi-modal communication system, as suggested by
and weaknesses, seems to meet human communication many details of the whole assemblage. Evidence for this
needs rather optimally. hypothesis is plentiful. For example, hand and mouth are clo-
The proposed sequential accumulation of layers of a com- sely connected in the somatotopic organization of the human
munication system maps roughly onto the stages we observe motor cortex (e.g. [99,100]), and a very similar connection
during the development of communication in human is also evident in the monkey motor cortex [101]. The
children—proto-gestures deriving from ritualized action hand – mouth connection is further evidenced by overt
sequences such as stretching one’s arms up in the air in human behaviour such as drawing or cutting something,
order to be picked up occur very early (albeit in an already which is frequently accompanied by intricate movements of
more generalized form than with young apes, [96]) and act the tongue, lips or jaws. And although the hands are the
as a form of pre-linguistic turn-taking, followed by pointing, major articulators in sign languages, the mouth and face
followed by both kinds of brachio-manual action becoming are always also involved. Neuroanatomical asymmetries in
integrated with speech to function as co-speech deictic and the brains of non-human primates and the lateralization of
co-speech iconic (or other representational) gestures. both their vocal and their gestural communicative signals
The details of this development are telling. For example, [102 –104] further corroborate the notion of an early evol-
turn-taking before language can be quite fast, with the infant utionary link between hand and mouth (but see [105]). In
responding in under three-quarters of a second [97], approach- addition, congenitally blind individuals gesture while they
ing adult norms. But later, when children are trying to respond speak despite never having seen a single gesture [106,107].
with more complex language at say 3 years of age, the response And further evidence consists of the fact that neurons
times can be twice as slow [98], converging with adult norms coding for manual goal-directed transitive movements
only in middle childhood. This suggests that the natural occupy areas in the monkey brain that correspond to brain
rhythm of conversation is independent of spoken language, areas critical for processing language in the human brain—
and children have to gradually learn to compress complex the putative mirror neuron system [108– 110].
material into the short rapid bursts of speech that adults use. It may then be that there was pre-adaptation for an inte-
The adult turn-taking speed puts extraordinary pressure on grated multi-modal communication system based on a close
language production and comprehension: since it takes marriage between hands and mouth, which was only fully
between 600 and 1500 ms to plan a response, and the gaps exploited when the changes in cortical organization occurred
between turns are only on average 200 ms, this forces those that made voluntary breathing and intentional spoken
engaged in dialogue to be already planning responses long communication possible. Given the large time-scales we are
before the other speaker has completed his or her utterance. envisaging, the gradual co-evolution of vocal language with
Comprehension and production must thus work in parallel, a pre-existing gestural mode of communication may have
with the next speaker predicting the ongoing turn by the taken place over nearly a million years, so that the different
other, in order to achieve precisely timed behavioural alterna- modalities are deeply intertwined. This view may therefore
tion. The whole system suggests an evolution from an original not be as diametrically opposed as it seems to McNeill’s
rapid exchange of very simple gestural or vocal material, into a [111] proposal that speech and gesture coevolved from the
system where the complexity of the linguistic and gestural beginning. In the stratificational model we are proposing,
material that is crammed into these short bursts has grown to our cousins the great apes suggest an early reliance on the
the very limits that human cognition can process. gestural modality, but co-occurring simple vocalizations
may rapidly have emerged as a way of drawing attention
to the signals. Indeed, it is possible that multimodality is
3. One multi-modal communication system or actually present in our great ape cousins and thus in the
common ancestor, since the extent to which their gestural sig-
separate systems? nals co-occur with vocalizations is still a largely unexplored
The evidence is that, despite the modern human communi- domain. In any case, from those initial multi-modal seeds
cation system having evolved in layers (see [22,90] for one our fully-fledged multi-modal communication system
 Downloaded from rstb.royalsocietypublishing.org on August 5, 2014

would have evolved in the layered manner suggested. development, the emergence of gesture and speech is intri- 6
During the course of this evolution, the communicational cately connected as evidenced by the two modalities’

rstb.royalsocietypublishing.org
burden has progressively but only partially shifted from parallel developmental trajectories and patterns [5,118]. This
hand to mouth, but sign languages offer a constant reminder connection is maintained in adulthood and evident in the
that the roots of the system are ‘hand þ mouth’ and where precise temporal, semantic and pragmatic relationship of
the motivation exists the burden can be readily shifted back. speech and gesture [5], and further supported by the finding
Crucially, this perspective is at odds with the view that that semantic information conveyed by speech and gesture is
gesture was a ‘bridging modality’ that withered away processed in the same brain areas [119–121].
once conventionalized spoken language had emerged (e.g. Despite the remarkable flexibility that allows us to shift
[2,21,109], see also [111]). Such a view reduces the role of ges- between visual and verbal modes of communication as
ture to a scaffolding function for the evolution and needs require, the neglected modality is rarely fully repressed.
ontogenetic development of speech, with speech, once fully Thus, deaf signers mouth and even vocalize, while speakers

Phil. Trans. R. Soc. B 369: 20130302

developed, completely eclipsing the multi-modal origins and continue to gesture frequently even when the gestures are
leaving gesture as an almost functionless relic. This would be not visually accessible to our addressee, such as on the tele-
a fundamentally misleading picture. As other scholars have phone [122], supporting the notion of an evolutionarily tight
suggested (first and foremost [5,6,84,111 – 115]), together connection between gesture and speech. Interestingly, the
speech and gesture constitute adult human language—the use of unseen gestures is specifically associated with interac-
two modalities are different components of one and the same tional uses of language [122,123] but not with monologue-
system. Owing to their differing nature, these two components like settings [124,125]. This observation is very much in line
make perfect complements, each taking over what the other is with the idea that gestural communication is strongly
less well suited to perform. During their coevolution, speech embedded in collaborative action [2,3] and that dialogic
and gesture have mutually and maximally adapted to one social engagement with another (as in the interaction engine)
another, resulting in a default mode for modern human is foundational to human communication [61].
communication characterized by highly efficient and informa-
tionally rich visual – verbal utterances. The system remains
highly flexible, allowing us to shift the burden from words to
gestures as required by the current communicative needs, as 4. Conclusion
when hunting or on a noisy factory floor, with sign languages We have argued that despite the tight integration of the
forming just one end of the spectrum [116]. different modalities into modern human communication,
Even in linguistically fully developed individuals, ges- the whole ensemble should be seen as a system of systems
tures remain a prominent part of human communication; that has accumulated over the two and a half million years
adults accompany most clauses [5] or idea units [113] with that humans have been a cognitively advanced, tool-using
one or more gestures, and children’s use of gestures increases species. The accumulations can be thought of as strata, and
(rather than decreases) over the course of their acquisition of peeling away the strata successively can give us some insights
spoken language (with their gesture rate reaching adult levels into the probable evolution of the whole complex system.
by the age of 4–5 years [5]). Further evidence for the tight This holistic account of our communicational capacities also
speech –gesture connection in adult human communication helps to bridge the gulf between the articulate species and
is the striking flexibility of this two-pronged communication our inarticulate cousins, allowing us to see precursor adap-
system. Without speech, the gestural modality is able to tations in, for example, the turn-taking widely if sparsely
take over much of the communicative burden, as evidenced represented in current primates and the gestural skills of
by sign languages themselves, and when hearing adults are great apes.
asked not to speak but to communicate only by means of
their hands, they turn to creating sequentially structured
gestural utterances [4,117]. Conversely, when preventing Acknowledgements. We thank the editor and two anonymous
reviewers for their invaluable feedback on an earlier version of this
people from using gestures, the imagistic content of their manuscript.
messages decreases radically and speech becomes hesitant, Funding statement. The authors were supported through ERC Advanced
further corroborating the assumption of gestures being inte- grant no. 269484 INTERACT awarded to Prof. Levinson and through
gral to the human language system. Finally, during child the Max Planck Society.

References
1. Hauser MD, Chomsky N, Fitch WT. 2002 The faculty grammatical expression in the manual modality. linguistic capacities and its consequences. Front.
of language: what is it, who has it, and how did it Psychol. Rev. 103, 34 –55. (doi:10.1037/0033-295X. Psychol. 4, 397. (doi:10.3389/fpsyg.2013.00397)
evolve? Science 298, 1569– 1579. (doi:10.1126/ 103.1.34) 8. Schepartz LA. 1993 Language and modern human
science.298.5598.1569) 5. McNeill D. 1992 Hand and mind: what gestures origins. Yearb. Phys. Anthropol. 36, 91– 126.
2. Tomasello M. 2008 Origins of human reveal about thought. Chicago, IL: Chicago University (doi:10.1002/ajpa.1330360607)
communication. Cambridge, UK: MIT Press. Press. 9. Jürgens U. 2002 Neural pathways underlying vocal
3. Clark HH. 1996 Using language. Cambridge, UK: 6. Kendon A. 2004 Gesture: visible action as utterance. control. Neurosci. Biobehav. Rev. 26, 235–258.
Cambridge University Press. Cambridge, UK: Cambridge University Press. (doi:10.1016/S0149-7634(01)00068-9)
4. Goldin-Meadow S, McNeill D, Singleton J. 1996 7. Dediu D, Levinson SC. 2013 On the antiquity of 10. Simonyan K, Horwitz B. 2011 Laryngeal motor
Silence is liberating: removing the handcuffs on language: the reinterpretation of Neandertal cortex and control of speech in humans.
 Downloaded from rstb.royalsocietypublishing.org on August 5, 2014

Neuroscientist 17, 197–208. (doi:10.1177/ 30. Rossano F. 2013 Gaze in conversation. In The edwardsi)? Am. J. Phys. Anthropol. 139, 523–532. 7
1073858410386727) handbook of conversation analysis (eds T Stivers, (doi:10.1002/ajpa.21017)

rstb.royalsocietypublishing.org
11. MacLarnon A, Hewitt G. 2004 Increased breathing J Sidnell), pp. 308–29. Chichester, UK: Wiley- 49. Snowdon CT, Cleveland J. 1984 ‘Conversations’
control: another factor in the evolution of human Blackwell. among pygmy marmosets. Am. J. Primatol. 7,
language. Evol. Anthropol. 13, 181– 97. (doi:10. 31. Thorpe WH. 1966 Ritualization in ontogeny. 15– 20. (doi:10.1002/ajp.1350070104)
1002/evan.20032) I. Animal play. Phil. Trans. R. Soc. Lond. B 251, 50. Takahashi DY, Narayanan DZ, Ghazanfar AA. 2013
12. Fitch WT. 2009 Fossil cues to the evolution of 311 –319. (doi:10.1098/rstb.1966.0015) Coupled oscillator dynamics of vocal turn-taking in
speech. In The cradle of language (eds RP Botha, 32. Scott-Phillips TC, Blythe RA, Gardner A, West SA. monkeys. Curr. Biol. 23, 2162 –2168. (doi:10.1016/j.
C Knight), pp. 112– 134. Oxford, UK: Oxford 2012 How do communication systems emerge? cub.2013.09.005)
University Press. Proc. R. Soc. B 279, 1943 –1949. (doi:10.1098/rspb. 51. Lemasson A, Glas L, Barbu S, Lacroix A, Guilloux M,
13. Bramble DM, Lieberman DE. 2004 Endurance 2011.2181) Remeuf K, Koda H. 2011 Youngsters do not pay
running and the evolution of Homo. Nature 432, 33. Clark HH. 2005 Coordinating with each other in a attention to conversational rules: is this so for

Phil. Trans. R. Soc. B 369: 20130302

345–352. (doi:10.1038/nature03052) material world. Discourse Stud. 7, 507 –525. nonhuman primates? Sci. Rep. 1, 22. (doi:10.1038/
14. Draper MH, Ladefoged P, Whitteridge D. 1960 Expiratory (doi:10.1177/1461445605054404) srep00022)
pressures and air flow during speech. Br. Med. J. 1, 34. Levinson SC. 1983 Pragmatics. Cambridge, UK: 52. Inoue Y, Sinun W, Yosida S, Okanoya K. 2013
1837–1843. (doi:10.1136/bmj.1.5189.1837) Cambridge University Press. Intergroup and intragroup antiphonal songs in wild
15. Tylor E. 1865 Researches into the early history of 35. Sperber D, Wilson D. 1986 Relevance: male Muellers gibbons (Hylobates muelleri).
mankind and the development of civilization. communication and cognition. Cambridge, UK: Interact. Stud. 14, 24 –43. (doi:10.1075/is.14.1.
London, UK: Murray. Cambridge University Press. 03ino)
16. Hewes G. 1973 Primate communication and the 36. Grice HP. 1989 Studies in the way of words. 53. Geissmann T. 2002 Duet-splitting and the evolution
gestural origin of language. Curr. Anthropol. 14, Cambridge, MA: Harvard University Press. of gibbon songs. Biol. Rev. 77, 57 –76. (doi:10.
65 –84. (doi:10.1086/201401) 37. Schegloff EA. 2007 Sequence organization in 1017/S1464793101005826)
17. Kendon A. 1975 Gesticulation, speech, and the interaction: a primer in conversation analysis, vol. 1. 54. Haimoff EH. 1986 Convergence in the duetting
gesture theory of language origins. Sign Lang. Stud. Cambridge, UK: Cambridge University Press. of monogamous Old World primates. J. Hum.
9, 349–373. (doi:10.1353/sls.1975.0016) 38. Tomasello M. 2014 A natural history of human Evol. 15, 51 –59. (doi:10.1016/S0047-2484(86)
18. Kendon A. 1991 Some considerations for a theory of thinking. Cambridge, MA: Harvard University Press. 80065-3)
language origins. Man 26, 199–221. (doi:10.2307/ 39. Levinson SC. 2000 Presumptive meanings: the theory 55. Wilkinson R, Leudar I, Pika S. 2012 Requesting
2803829) of generalized conversational implicature. behaviors within episodes of active sharing. In
19. Armstrong DF, Wilcox S. 2007 The gestural origin of Cambridge, UK: MIT Press. Developments in primate gesture research, vol. 6
language. Oxford, UK: Oxford University Press. 40. Levinson SC. 2013 Recursion in pragmatics. (eds S Pika, K Liebal), pp. 199– 221. Amsterdam,
20. Sterelny K. 2012 Language, gesture, skill: the Language 89, 149–162. (doi:10.1353/lan.2013. The Netherlands: John Benjamins.
co-evolutionary foundations of language. Phil. 0005) 56. Argyle M, Cook M. 1976 Gaze and mutual gaze.
Trans. R. Soc. B 367, 2141– 2151. (doi:10.1098/rstb. 41. Mehl MR, Vazire S, Ramı́rez-Esparza N, Slatcher RB, Cambridge, UK: Cambridge University Press.
2012.0116) Pennebaker JW. 2007 Are women really more 57. Bard KA, Myowa-Yamakoshi M, Tomonaga M,
21. Corballis MC. 2002 From hand to mouth: the origins of talkative than men? Science 317, 82. (doi:10.1126/ Tanaka M, Costall A, Matsuzawa T. 2005 Group
language. Princeton, NJ: Princeton University Press. science.1139940) differences in the mutual gaze of chimpanzees
22. Arbib MA. 2005 From monkey-like action 42. Stivers T et al. 2009 Universals and cultural variation (Pan troglodytes). Dev. Psychol. 41, 616–624.
recognition to human language: an evolutionary in turn-taking in conversation. Proc. Natl Acad. Sci. (doi:10.1037/0012-1649.41.4.616)
framework for neurolinguistics. Behav. Brain Sci. 28, USA 106, 10 587 –10 592. (doi:10.1073/pnas. 58. Nettle D, Cronin KA, Bateson M. 2013 Responses of
105–124. (doi:10.1017/S0140525X05000038) 0903616106) chimpanzees to cues of conspecific observation.
23. Gentilucci M, Corballis MC. 2006 From manual 43. Indefrey P, Levelt WJ. 2004 The spatial and Anim. Behav. 86, 595 –602. (doi:10.1016/j.anbehav.
gesture to speech: a gradual transition. Neurosci. temporal signatures of word production 2013.06.015)
Biobehav. Rev. 30, 949 –960. (doi:10.1016/j. components. Cognition 92, 101–144. (doi:10.3389/ 59. Kobayashi H, Kohshima S. 2001 Unique morphology
neubiorev.2006.02.004) fpsyg.2011.00255) of the human eye and its adaptive meaning:
24. Corballis MC. 2009 The evolution of language. Ann. 44. DeLong KA, Urbach TP, Kutas M. 2005 Probabilistic comparative studies on external morphology of the
NY Acad. Sci. 1156, 19 –43. (doi:10.1111/j.1749- word pre-activation during language primate eye. J. Hum. Evol. 40, 419–435. (doi:10.
6632.2009.04423.x) comprehension inferred from electrical brain 1006/jhev.2001.0468)
25. Call J, Tomasello M. 2007 The gestural communication activity. Nat. Neurosci. 8, 1117– 1121. (doi:10.1038/ 60. Farroni T, Csibra G, Simion F, Johnson MH. 2002 Eye
of apes and monkeys. Manhaw, NJ: LEA. nn1504) contact detection in humans from birth. Proc. Natl
26. Gruber T, Zuberbühler K. 2013 Vocal recruitment for 45. Pickering MJ, Garrod S. 2007 Do people use Acad. Sci. USA 99, 9602–9605. (doi:10.1073/pnas.
joint travel in wild chimpanzees. PLoS ONE 8, language production to make predictions during 152159999)
e76073. (doi:10.1371/journal.pone.0076073) comprehension? Trends Cogn. Sci. 11, 105–110. 61. Levinson SC. 2006 On the human ‘interaction
27. Schel AM, Townsend SW, Machanda Z, Zuberbühler (doi:10.1016/j.tics.2006.12.002) engine’. In Roots of human sociality: culture,
K, Slocombe KE. 2013 Chimpanzee alarm call 46. Magyari L, de Ruiter JP. 2012 Prediction of turn- cognition and interaction (eds NJ Enfield,
production meets key criteria for intentionality. PLoS ends based on anticipation of upcoming words. SC Levinson), pp. 39 –69. Oxford, UK: Berg.
ONE 8, e76674. (doi:10.1371/journal.pone. Front. Psychol. 3, 376. (doi:10.3389/fpsyg.2012. 62. Bates E. 1976 Language and context: the acquisition
0076674) 00376) of pragmatics, vol. 13. New York, NY: Academic
28. Liebal K, Pika S, Tomasello M. 2006 Gestural 47. Snow CE. 1977 The development of conversation Press.
communication of orangutans (Pongo pygmaeus). between mothers and babies. J. Child Lang. 4, 63. Tomasello M, Carpenter M, Liszkowski U. 2007 A
Gesture 6, 1–38. (doi:10.1075/gest.6.1.02lie) 1 –22. (doi:10.1017/S0305000900000453) new look at infant pointing. Child Dev. 78,
29. Pollick AS, de Waal FB. 2007 Ape gestures and 48. Méndez-Cárdenas MG, Zimmermann E. 2009 705–722. (doi:10.1111/j.1467-8624.2007.01025.x)
language evolution. Proc. Natl Acad. Sci. USA 104, Duetting—a mechanism to strengthen pair bonds 64. Liszkowski U, Brown P, Callaghan T, Takada A, de
8184–8189. (doi:10.1073/pnas.0702624104) in a dispersed pair-living primate (Lepilemur Vos C. 2012 A prelinguistic gestural universal of
 Downloaded from rstb.royalsocietypublishing.org on August 5, 2014

human communication. Cogn. Sci. 36, 698– 713. 81. Hinton L, Nichols J, Ohala J (eds) 1994 Sound 99. Aflalo TN, Graziano MS. 2006 Possible origins of 8
(doi:10.1111/j.1551-6709.2011.01228.x) symbolism. Cambridge, UK: Cambridge University the complex topographic organization of motor

rstb.royalsocietypublishing.org
65. Carpendale JI, Carpendale AB. 2010 The Press. cortex: reduction of a multidimensional space
development of pointing: from personal 82. Dingemanse M. In press. Expressiveness and system onto a two-dimensional array. J. Neurosci. 26,
directedness to interpersonal direction. Hum. Dev. integration. On the typology of ideophones, with 6288 – 6297. (doi:10.1523/JNEUROSCI.0768-06.
53, 110–126. (doi:10.1159/000315168) special reference to Siwu. STUF—Language 2006)
66. Pika S, Liebal K, Tomasello M. 2005 Gestural Typology and Universals. 100. Meier JD, Aflalo TN, Kastner S, Graziano MS. 2008
communication in subadult bonobos (Pan paniscus): 83. Haiman J. 1985 Natural syntax: iconicity and erosion. Complex organization of human primary motor
repertoire and use. Am. J. Primatol. 65, 39 –61. Cambridge, UK: Cambridge University Press. cortex: a high-resolution fMRI study. J. Neurophysiol.
(doi:10.1002/ajp.20096) 84. McNeill D. 1985 So you think gestures are 100, 1800– 1812. (doi:10.1152/jn.90531.2008)
67. Hobaiter C, Leavens DA, Byrne RW. 2013 Deictic nonverbal? Psychol. Rev. 92, 350–371. (doi:10. 101. Graziano MS, Aflalo TN. 2007 Mapping behavioral
gesturing in wild chimpanzees (Pan troglodytes)? 1037/0033-295X.92.3.350) repertoire onto the cortex. Neuron 56, 239 – 251.

Phil. Trans. R. Soc. B 369: 20130302

Some possible cases. J. Comp. Psychol. 128, 82– 87. 85. Holler J, Beattie G. 2002 A micro-analytic (doi:10.1016/j.neuron.2007.09.013)
(doi:10.1037/a0033757) investigation of how iconic gestures and speech 102. Vauclair J. 2004 Lateralization of communicative
68. Pika S, Mitani J. 2006 Referential gestural represent core semantic features in talk. Semiotica signals in nonhuman primates and the hypothesis
communication in wild chimpanzees (Pan 142, 31 –69. (doi:10.1515/semi.2002.077) of the gestural origin of language. Interact. Stud. 5,
troglodytes). Curr. Biol. 16, R191 –R192. (doi:10. 86. Holler J, Beattie G. 2003 How iconic gestures and 363–384. (doi:10.1075/is.5.3.04vau)
1016/j.cub.2006.02.037) speech interact in the representation of meaning: 103. Meguerditchian A, Vauclair J. 2006 Baboons
69. Leavens DA, Hopkins WD, Bard KA. 1996 Indexical are both aspects really integral to the process? communicate with their right hand. Behav. Brain Res.
and referential pointing in chimpanzees Semiotica 146, 81– 116. (doi:10.1515/semi.2003. 171, 170–174. (doi:10.1016/j.bbr.2006.03.018)
(Pan troglodytes). J. Comp. Psychol. 110, 346. 083) 104. Meguerditchian A, Vauclair J, Hopkins WD. 2010
(doi:10.1037/0735-7036.110.4.346) 87. Armstrong DF, Stokoe WC, Wilcox SE. 1995 Gesture Captive chimpanzees use their right hand to
70. Matthews D, Behne T, Lieven E, Tomasello M. 2012 and the nature of language. Cambridge, UK: communicate with each other: implications for the
Origins of the human pointing gesture: a training Cambridge University Press. origin of the cerebral substrate for language.
study. Dev. Sci. 15, 817 –829. (doi:10.1111/j.1467- 88. Beattie G, Shovelton H. 2002 An experimental Cortex 46, 40 – 48. (doi:10.1016/j.cortex.2009.02.
7687.2012.01181.x) investigation of some properties of individual iconic 013)
71. Camaioni L, Perucchini P, Bellagamba F, Colonnesi C. gestures that mediate their communicative power. 105. Fitch WT, Braccini SN. 2013 Primate laterality and
2004 The role of declarative pointing in developing Br. J. Psychol. 93, 179–192. (doi:10.1348/ the biology and evolution of human handedness: a
a theory of mind. Infancy 5, 291– 308. (doi:10. 000712602162526) review and synthesis. Ann. NY Acad. Sci. 1288,
1207/s15327078in0503_3) 89. Stokoe WC. 2001 Language in hand: why sign came 70– 85. (doi:10.1111/nyas.12071)
72. Cochet H, Vauclair J. 2010 Pointing gestures before speech. Washington, DC: Gallaudet University 106. Iverson JM, Goldin-Meadow S. 1998 Why people
produced by toddlers from 15 to 30 months: Press. gesture when they speak. Nature 396, 228. (doi:10.
different functions, hand shapes and laterality 90. Arbib MA. 2012 How the brain got language: the 1038/24300)
patterns. Infant Behav. Dev. 33, 431 –441. (doi:10. mirror system hypothesis. Oxford, UK: Oxford 107. Iverson JM, Goldin-Meadow S. 2001 The resilience
1016/j.infbeh.2010.04.009) University Press. of gesture in talk: gesture in blind speakers and
73. Wittgenstein L. 1969 On certainty. Oxford, UK: Basil 91. Fitch WT. 2010 The evolution of language. listeners. Dev. Sci. 4, 416– 422. (doi:10.1111/1467-
Blackwell. Cambridge, UK: Cambridge University Press. 7687.00183)
74. Schilbach L, Wilms M, Eickhoff SB, Romanzetti S, 92. Stout D. 2011 Stone toolmaking and the evolution 108. Rizzolatti G, Fadiga L, Gallese V, Fogassi L. 1996
Tepest R, Bente G, Shah NJ, Fink GR, Vogeley K. of human culture and cognition. Phil. Trans. R. Soc. Premotor cortex and the recognition of motor
2010 Minds made for sharing: initiating joint B 366, 1050– 1059. (doi:10.1098/rstb.2010.0369) actions. Cogn. Brain Res. 3, 131–141. (doi:10.1016/
attention recruits reward-related neurocircuitry. 93. Hockett C. 1960 The origin of speech. Sci. 0926-6410(95)00038-0)
J. Cogn. Neurosci. 22, 2702 –2715. (doi:10.1162/ Am. 203, 88– 111. (doi:10.1038/ 109. Rizzolatti G, Arbib MA. 1998 Language within our
jocn.2009.21401) scientificamerican0960-88) grasp. Trends Neurosci. 21, 188 –194. (doi:10.1016/
75. Levinson SC. 2003 Space in language and cognition: 94. Perniss P, Vigliocco G. 2014 The bridge of iconicity: S0166-2236(98)01260-0)
explorations in cognitive diversity. Cambridge, UK: from a world of experience to the experience of 110. Rizzolatti G, Craighero L. 2004 The mirror-neuron
Cambridge University Press. language. Phil. Trans. R. Soc. B 369, 20130300. system. Annu. Rev. Neurosci. 27, 169 –192. (doi:10.
76. De Vos C. 2012 Sign-spatiality in Kata Kolok: how a (doi:10.1098/rstb.2013.0300) 1146/annurev.neuro.27.070203.144230)
village sign language in Bali inscribes its signing 95. Jackendoff R. 1999 Possible stages in the 111. McNeill D. 2012 How language began: gesture and
space. PhD thesis, Radboud University Nijmegen, evolution of the language capacity. Trends Cogn. speech in human evolution. Cambridge, UK:
Nijmegen, The Netherlands. Sci. 3, 272 –279. (doi:10.1016/S1364- Cambridge University Press.
77. Perniss P, Thompson RL, Vigliocco G. 2010 Iconicity 6613(99)01333-9) 112. Kendon A. 1972 Some relationships between body
as a general property of language: evidence from 96. Marentette P, Nicoladis E. 2012 Does ontogenetic motion and speech. An analysis of an example. In
spoken and signed languages. Front. Psychol. 1, ritualization explain early communicative gestures in Studies in dyadic communication (eds A Siegman,
227. (doi:10.3389/fpsyg.2010.00227) human infants? In Developments in primate B Pope), pp. 177 –210. Elmsford, NY: Pergamon
78. Mithen S. 1996 The prehistory of the mind: the gesture research, vol. 6 (eds S Pika, K Liebal), Press.
cognitive origins of art, religion and science. London, pp. 33– 54. Amsterdam, The Netherlands: John 113. Kendon A. 1980 Gesticulation and speech: two
UK: Thames and Hudson. Benjamins. aspects of the process of utterance. In The
79. Marshack A. 1976 Some implications of the paleolithic 97. Jaffe J, Beebe B, Feldstein S, Crown CL, Jasnow MD, relationship of verbal and nonverbal communication
symbolic evidence for the origin of language. Curr. Rochat P, Stern DN. 2001 Rhythms of dialogue in (ed. MR Key), pp. 207– 227. The Hague, The
Anthropol. 17, 274–282. (doi:10.1086/201716) infancy: coordinated timing in development. Netherlands: Mouton and Co.
80. Slobin DI. 1968 Antonymic phonetic symbolism in Monogr. Soc. Res. Child Dev. 66, i –viii, 1 –132. 114. Kendon A. 2000 Language and gesture: unity or
three natural languages. J. Pers. Soc. Psychol. 10, 98. Garvey C. 1984 Children‘s talk. Cambridge, MA: duality. In Language and gesture (ed. D McNeill), pp.
301–305. (doi:10.1037/h0026575) Harvard University Press. 47–63. Cambridge, UK: Cambridge University Press.
 Downloaded from rstb.royalsocietypublishing.org on August 5, 2014

115. McNeill D. 2005 Gesture and thought. Chicago, IL: and the human mirror system. Brain Lang. 101, 123. Holler J, Tutton M, Wilkin K. 2011 Co-speech 9
University of Chicago Press. 260 –277. (doi:10.1016/j.bandl.2007.02.008) gestures in the process of meaning coordination. In

rstb.royalsocietypublishing.org
116. Kendon A. 1985 Some uses of gesture. In Perspectives on 120. Willems RM, Hagoort P. 2007 Neural evidence for Proc. 2nd GESPIN—Gesture and Speech in
silence (eds D Tannen, M Saville-Troike), pp. 215–234. the interplay between language, gesture, and Interaction Conf., Bielefeld, 5–7 September 2011.
Norwood, NJ: Ablex. action: a review. Brain Lang. 101, 278–289. See http://hdl.handle.net/11858/00-001M-0000-
117. Gershkoff-Stowe L, Goldin-Meadow S. 2002 Is there (doi:10.1016/j.bandl.2007.03.004) 0012-1BB3-D.
a natural order for expressing semantic relations? 121. Willems RM, Özyürek A, Hagoort P. 2007 When 124. Alibali MW, Heath DC, Myers HJ. 2001 Effects of visibility
Cogn. Psychol. 45, 375 –412. (doi:10.1016/S0010- language meets action: the neural integration between speaker and listener on gesture production:
0285(02)00502-9) of gesture and speech. Cerebral Cortex 17, some gestures are meant to be seen. J. Mem. Lang. 44,
118. Goldin-Meadow S. 2003 Hearing gesture: how our 2322 –2333. (doi:10.1093/cercor/bhl141) 169–188. (doi:10.1006/jmla.2000.2752)
hands help us think. Cambridge, MA: Harvard 122. Bavelas JB, Gerwing J, Sutton C, Prevost D. 2008 125. Jacobs N, Garnham A. 2007 The role of
University Press. Gesturing on the telephone: independent effects of conversational hand gestures in a narrative task.

Phil. Trans. R. Soc. B 369: 20130302

119. Skipper JI, Goldin-Meadow S, Nusbaum HC, Small dialogue and visibility. J. Mem. Lang. 58, 495–520. J. Mem. Lang. 56, 291–303. (doi:10.1016/j.jml.
SL. 2007 Speech-associated gestures, Broca’s area, (doi:10.1016/j.jml.2007.02.004) 2006.07.011)

View publication stats