The Oxford Handbook of Neurolinguist (2019)

T h e Ox f o r d H a n d b o o k o f
N E U ROL I N G U I ST IC S
The Oxford Handbook of
NEUROLINGUISTICS
Edited by
GREIG I. DE ZUBICARAY
and
NIELS O. SCHILLER
1
3
Oxford University Press is a department of the University of Oxford. It furthers
the University’s objective of excellence in research, scholarship, and education
by publishing worldwide. Oxford is a registered trade mark of Oxford University
Press in the UK and certain other countries.
Published in the United States of America by Oxford University Press

198 Madison Avenue, New York, NY 10016, United States of America.
© Oxford University Press 2019
All rights reserved. No part of this publication may be reproduced, stored in

a retrieval system, or transmitted, in any form or by any means, without the
prior permission in writing of Oxford University Press, or as expressly permitted
by law, by license, or under terms agreed with the appropriate reproduction
rights organization. Inquiries concerning reproduction outside the scope of the
above should be sent to the Rights Department, Oxford University Press, at the
address above.
You must not circulate this work in any other form

and you must impose this same condition on any acquirer.
Library of Congress Cataloging-in-Publication Data

Names: De Zubicaray, Greig, editor. | Schiller, Niels Olaf, 1969– editor.
Title: The Oxford handbook of neurolinguistics /
edited by Greig de Zubicaray & Niels O. Schiller.
Description: New York, NY : Oxford University Press, 2018. |
Includes bibliographical references and index.
Identifiers: LCCN 2018013813 (print) | LCCN 2018032934 (ebook) |
ISBN 9780190672034 (updf) | ISBN 9780190672041 (online content) |
ISBN 9780190914868 (epub) | ISBN 9780190672027 (cloth : alk. paper)
Subjects: LCSH: Neurolinguistics.
Classification: LCC QP399 (ebook) | LCC QP399.O94 2018 (print) |
DDC 612.8/2336—dc23
LC record available at https://lccn.loc.gov/2018013813
1 3 5 7 9 8 6 4 2
Printed by Sheridan Books, Inc., United States of America
Contents
Preface ix
Greig I. de Zubicaray and Niels O. Schiller
List of Contributors xi
1. Neurolinguistics: A Brief Historical Perspective 1
Sheila E. Blumstein
PA RT I . T H E M E T HOD S
2. Neurolinguistic Studies of Patients with Acquired Aphasias 19
Stephen M. Wilson
3. Electrophysiological Methods in the Study
of Language Processing 42
Michelle Leckey and Kara D. Federmeier
4. Studying Language with Functional Magnetic Resonance
Imaging (fMRI) 72
Stefan Heim and Karsten Specht
5. Transcranial Magnetic Stimulation to Study the Neural Network
Account of Language 94
Teresa Schuhmann
6. Magnetoencephalography and the Cortical Dynamics
of Language Processing 115
Riitta Salmelin, Jan Kujala, and Mia Liljeström
7. Shedding Light on Language Function and Its Development
with Optical Brain Imaging 154
Yasuyo Minagawa and Alejandrina Cristia
vi Contents
8. What Has Direct Cortical and Subcortical Electrostimulation

Taught Us about Neurolinguistics? 186
Hugues Duffau
9. Diffusion Imaging Methods in Language Sciences 212
Marco Catani and Stephanie J. Forkel
PA RT I I . DE V E L OP M E N T A N D P L A S T IC I T Y
10. Neuroplasticity: Language and Emotional Development
in Children with Perinatal Stroke 231
Judy S. Reilly and Lara R. Polse
11. The Neurolinguistics of Bilingualism: Plasticity and Control 261
David W. Green and Judith F. Kroll
12. Language and Aging 295
Jonathan E. Peelle
13. Language Plasticity in Epilepsy 317
Jeffrey R. Cole and Marla J. Hamberger
14. Language Development in Deaf Children: Sign Language
and Cochlear Implants 339
Aaron J. Newman
PA RT I I I . A RT IC U L AT ION A N D P RODU C T ION

15. Neuromotor Organization of Speech Production 371
Pascale Tremblay, Isabelle Deschamps,
and Anthony Steven Dick
16. The Neural Organization of Signed Language: Aphasia and
Neuroscience Evidence 402
David P. Corina and Laurel A. Lawyer
17. Understanding How We Produce Written Words: Lessons from
the Brain 425
Brenda Rapp and Jeremy Purcell
18. Motor Speech Disorders 449
Wolfram Ziegler, Theresa Schölderle, Ingrid Aichert,
and Anja Staiger
Contents vii
19. Investigating the Spatial and Temporal Components of Speech

Production 472
Greig I. de Zubicaray and Vitória Piai
20. The Dorsal Stream Auditory-Motor Interface for Speech 498
Gregory Hickok
PA RT I V. C ON C E P T S A N D C OM P R E H E N SION
21. Neural Representations of Concept Knowledge 519
Andrew J. Bauer and Marcel A. Just
22. Finding Concepts in Brain Patterns: From Feature Lists
to Similarity Spaces 548
Elizabeth Musz and Sharon L. Thompson-Schill
23. The How and What of Object Knowledge in the Human Brain 576
Frank E. Garcea and Bradford Z. Mahon
24. Neural Basis of Monolingual and Bilingual Reading 603
Pedro M. Paz-Alonso, Myriam Oliver, Ileana Quiñones,
and Manuel Carreiras
25. Dyslexia and Its Neurobiological Basis 626
Kaja Jasińska and Nicole Landi
26. Speech Perception: A Perspective from Lateralization,
Motorization, and Oscillation 647
David Poeppel, Gregory B. Cogan, Ido Davidesco,
and Adeen Flinker
27. Sentence Processing: Toward a Neurobiological Approach 676
Ina Bornkessel-Schlesewsky and Matthias Schlesewsky
28. Comprehension of Metaphors and Idioms: An Updated
Meta-analysis of Functional Magnetic Resonance
Imaging Studies 710
Alexander Michael Rapp
29. Language Comprehension and Emotion: Where Are
the Interfaces, and Who Cares? 736
Jos J. A. van Berkum
viii Contents
PA RT V. G R A M M A R A N D C O G N I T ION
30. Grammatical Categories 769
David Kemmerer
31. Neurocognitive Mechanisms of Agrammatism 796
Cynthia K. Thompson and Jennifer E. Mack
32. Verbal Working Memory 827
Bradley R. Buchsbaum
33. Subcortical Contributions to Language 851
David A. Copland and Anthony J. Angwin
34. Lateralization of Language 877
Lise Van der Haegen and Qing Cai
35. Neural Mechanisms of Music and Language 907
Mattson Ogg and L. Robert Slevc
Index 953
Preface
Greig I. de Zubicaray and Niels O. Schiller
Neurolinguistics is a highly interdisciplinary field, with influences from psycholin-

guistics, psychology, aphasiology, (cognitive) neuroscience, and many more. A precise
definition is elusive, but often neurolinguistics is considered to cover approximately
the same range of topics as psycholinguistics, that is, all aspects of language processing,
but approached from various scientific perspectives and methodologies. Twenty years
ago, when the first Handbook of Neurolinguistics, edited by Harry Whitaker and Brigitte
Stemmer, was published, it was relatively easy to identify the contributions from in-
dividual disciplines, with the dominant evidence base and approach being clinical
aphasiology. Today, neurolinguistics has progressed such that individual researchers
tackle topics of interest using multiple methods, and share a common sense of iden-
tity and purpose, culminating in their own society and annual conference. The Society
for the Neurobiology of Language will have its tenth anniversary in 2018, and its annual
meeting now regularly exceeds 700 attendees.
When we first proposed to collate and edit this Handbook of 35 chapters, we knew we
were undertaking a challenging task given the rapid expansion of the field and pace of
progress in recent years. We envisaged a mix of chapters from established and emerging
researchers, with contributions covering the contemporary topics of interest to the field
of neurolinguistics. We wanted more than the mere acknowledgment of the multilin-
gual brain featured in previous handbooks, and to encourage varied perspectives on
how language interacts with broader aspects of cognition and emotion. Responses to
our invitations were mostly generous. By and large, we believe we have achieved much
of what we set out to accomplish.
The scope and aim of this new Oxford Handbook of Neurolinguistics is to provide
students and scholars with concise overviews of the state of the art in particular topic
areas, and to engage a broad audience with an interest in the neurobiology of lan-
guage. The chapters do not attempt to provide exhaustive coverage, but rather present
discussions of prominent questions posed by a given topic.
Following an introductory chapter providing a brief historical perspective of the field,
Part I covers the key techniques and technologies used to study the neurobiology of
language today, including lesion-symptom mapping, functional imaging, electrophys-
iology, tractography, and brain stimulation. Each chapter provides a concise overview
x Preface
of the use of each technique by leading experts, who also discuss the various challenges
that neurolinguistic researchers are likely to encounter.
Part II addresses the neurobiology of language acquisition during healthy de-
velopment and in response to challenges presented by congenital and acquired
conditions. Part III covers the many facets of our articulate brain, its capacity for lan-
guage production—written, spoken, and signed—again from both healthy and clinical
perspectives. Questions regarding how the brain organizes and represents meaning are
addressed in Part IV, ranging from word to discourse level in written and spoken lan-
guage, from perception to statistical modeling. The final Part V reaches into broader
territory, characterizing and contextualizing the neurobiology of language with respect
to more fundamental neuroanatomical mechanisms.
Our thanks go to the authors of the chapters, without whom the Handbook would not
have been possible. Their commitment, expertise, and talent in exposition are rivaled
only by their patience with the editorial process. Thanks also go to Peter Ohlin, Hannah
Doyle, and Hallie Stebbins at Oxford University Press, who encouraged and ensured the
publication of The Oxford Handbook of Neurolinguistics.
Contributors
Ingrid Aichert, PhD, is a speech-language pathologist. She works as a Research

Associate in the Clinical Neuropsychology Research Group (EKN) at the Institute of
Phonetics and Speech Processing, University of Munich, Germany. Her main areas of
research are apraxia of speech and phonological disorders.
Anthony J. Angwin is a Senior Lecturer in Speech Pathology at the University of
Queensland. His research, focused primarily within the field of Language Neuroscience,
uses neuroimaging and behavioral paradigms to advance current understanding of lan-
guage processing and language learning in healthy adults and people with neurological
impairment.
Andrew J. Bauer received his PhD at Carnegie Mellon University and is currently a
Postdoctoral Fellow at the University of Toronto. His research uses machine learning
techniques applied to fMRI data to understand where and how knowledge is neurally
represented in the brain, and how the brain changes with learning new concepts.
Sheila E. Blumstein is the Albert D. Mead Professor Emerita of Cognitive, Linguistic,
and Psychological Sciences at Brown University. Her research is concerned with
delineating the neural basis of language and the processes and mechanisms involved
in speaking and understanding, using behavioral and neural measures of persons with
aphasia and functional neuroimaging. Blumstein’s research has focused on how the con-
tinuous acoustic signal is transformed by perceptual and neural mechanisms into the
sound structure of language, how the sound structure of language maps to the lexicon
(mental dictionary), and how the mental dictionary is organized for the purposes of lan-
guage comprehension and production.
Ina Bornkessel-Schlesewsky is Professor of Cognitive Neuroscience in the School
of Psychology, Social Work and Social Policy at the University of South Australia
in Adelaide. She was previously Professor of Neurolinguistics at the University of
Marburg, Germany, and Head of the Max Planck Research Group Neurotypology at the
Max Planck Institute for Human Cognitive and Brain Sciences in Leipzig, Germany. Her
main research interest is in the neurobiology of higher-order language processing.
Bradley R. Buchsbaum is an Associate Professor in the Department of Psychology at
the University of Toronto and a scientist at the Rotman Research Institute at Baycrest.
His research focuses on the cognitive neuroscience of memory and language, with
xii Contributors
special focus on how memory emerges from neocortical representations that underlie
perceptual and motor cognition.
Qing Cai is a cognitive psychologist and Professor of Psychology at East China Normal
University. Her research focuses on the neural basis of speech and reading, their acqui-
sition in typical and atypical development, as well as their relation to learning, memory,
music, and other higher-order cognitive functions.
Manuel Carreiras, PhD, is the Scientific Director of the Basque Center on Cognition,
Brain and Language (BCBL), Ikerbasque Research Professor, Honorary Professor of the
University College of London, and Visiting Professor of the University of the Basque
Country (UPV/EHU). His research focuses on reading, bilingualism, and second-
language learning. He has published more than 200 papers in high-impact journals
in the field. His research has been funded by various research agencies, including the
European Research Council.
Marco Catani is Professor of Neuroanatomy and Psychiatry at King’s College London
and Honorary Consultant Psychiatrist at the Maudsley Hospital. He has contributed to
the development of diffusion tractography methods applied to the study of white matter
connections in the normal brain and in a wide range of neurodevelopmental and neu-
rological disorders.
Gregory B. Cogan is an Assistant Professor of Neurosurgery at Duke University. His
research focuses on the neural underpinnings of speech and auditory cognition.
Jeffrey R. Cole is an Assistant Professor of Clinical Neuropsychology in the Department
of Neurology at Columbia University Medical Center, and Adjunct Assistant Professor
of Psychology and Education at Columbia University –Teachers College. His clinical
practice and research interests focus on patients with complex and medically refractory
epilepsies, Wada testing, and cortical language mapping.
David A. Copland is a University of Queensland Vice Chancellors Fellow and speech
pathologist. He is active in the fields of psycholinguistics, language neuroscience, and
clinical aphasia management. He has particular interests in determining the neural
mechanisms underpinning aphasia recovery and treatment, in developing better
interventions for aphasia, and understanding subcortical contributions in language as
observed in stroke and in Parkinson’s disease.
David P. Corina is a Professor in the Departments of Linguistics and Psychology at the
University of California, Davis. He is the Director of the Cognitive Neurolinguistics
Laboratory at the Center for Mind and Brain. His research interests include the neural
processing of signed and spoken languages and neural plasticity as a function of lin-
guistic and altered sensory experience.
Alejandrina Cristia received her PhD in Linguistics from Purdue University in
2009 and did postdoctoral work on neuroimaging at the Max Planck Institute for
Psycholinguistics before joining the French CNRS (Centre national de la recherche
scientifique) as a Researcher in 2013.
Contributors xiii
Ido Davidesco is a Research Assistant Professor at the Teaching and Learning

Department at New York University. His research focuses on how brain oscillations be-
come synchronized in classrooms.
Greig I. de Zubicaray is Professor and Associate Dean of Research in the Faculty of
Health at Queensland University of Technology, Brisbane, Australia. His research
covers brain mechanisms involved in language and memory and their disorders, neu-
roimaging methodologies, the aging brain and cognitive decline, and most recently, the
emerging field of imaging genetics.
Isabelle Deschamps is a Professor at Georgian College, Orillia, Ontario, Canada, and

a Researcher in the Speech and Hearing Neuroscience Laboratory in Québec City. Her
research focuses on the neural correlates of phonological processes during speech per-
ception and production.
Anthony Steven Dick is Associate Professor of Developmental Science and Director

of the Cognitive Neuroscience Program in the Department of Psychology at Florida
International University, Miami. His research focus is on the developmental cognitive
neuroscience of language and executive function.
Hugues Duffau (MD, PhD) is Professor and Chairman of the Neurosurgery

Department in the Montpellier University Medical Center and Head of the INSERM
1051 Team at the Institute for Neurosciences of Montpellier (France). He is an expert
in the awake cognitive neurosurgery of slow-growing brain tumors. For his innovative
work in neurosurgery and neurosciences, he was awarded Doctor Honoris Causa five
times, and he was the youngest recipient of the prestigious Herbert Olivecrona Award
from the Karolinska Institute in Stockholm. He has written four textbooks and over 370
publications for a total of more than 25,000 citations and with an h-index of 85.
Kara D. Federmeier is a Professor in the Department of Psychology and the

Neuroscience Program at the University of Illinois and a full-time faculty member at the
Beckman Institute for Advanced Science and Technology, where she leads the Illinois
Language and Literacy Initiative and heads the Cognition and Brain Lab. Her research
examines meaning comprehension and memory using human electrophysiological
techniques, in combination with behavioral, eye-tracking, and other functional im-
aging and psychophysiological methods.
Adeen Flinker is an Assistant Professor of Neurology at the New York University

School of Medicine. He is the Director of Intracranial Neurophysiology Research at
the Comprehensive Epilepsy Center. His research focuses on the temporal dynamics of
speech production and perception.
Stephanie J. Forkel is an Honorary Lecturer at the Departments of Neuroimaging and

Forensic and Neurodevelopmental Sciences at the Sackler Institute of Translational
Neurodevelopment, Institute of Psychiatry, Psychology and Neuroscience at King’s
College London. She has a background in psychology and neurosciences, which she
xiv Contributors
currently applies to identify neuroimaging predictors of language recovery after brain

lesions using diffusion imaging.
Frank E. Garcea completed his PhD in Cognitive Neuroscience in the Department of

Brain and Cognitive Sciences at the University of Rochester in July 2017. He is now a
postdoctoral research fellow at the Moss Rehabilitation Research Institute, where he
studies language and action representation in brain-damaged individuals.
David W. Green is an Emeritus Professor in the Faculty of Brain Sciences at University
College London. Theoretical work and neuroimaging research with neurologically
normal participants from young adults to the elderly have been combined with applied
research into the neural predictors of speech recovery post-stroke in monolingual and
multilingual individuals with aphasia.
Marla J. Hamberger is a Professor of Neuropsychology in the Department of Neurology
at Columbia University Medical Center, and Director of Neuropsychology at the
Columbia Comprehensive Epilepsy Center. Her research focuses on brain organization
of cognitive mechanisms supporting word production using electrocortical stimulation
mapping and behavioral techniques in patients who require brain surgery involving el-
oquent cortex.
Stefan Heim is a cognitive neuropsychologist and neurolinguist. He did his PhD thesis at
the Max Planck Institute for Human Cognitive and Brain Sciences in Leipzig, Germany.
He is now Professor and Chair of the academic programs for Speech-Language Therapy
(BSc, MSc) at Rheinisch-Westfälische Technische Hochschule (RWTH) Aachen
University, Germany. His main research focus is on the connectivity, and plasticity of
the language network in the human brain.
Gregory Hickok is Professor of Cognitive Sciences and Language Science at the
University of California, Irvine. He is Editor-in-Chief of Psychonomic Bulletin & Review
and author of The Myth of Mirror Neurons 2014.
Kaja Jasińska is an Assistant Professor of Linguistics and Cognitive Science at the
University of Delaware. Dr. Jasińska studies the neural mechanisms that support lan-
guage, cognitive, and reading development across the lifespan using a combination of be-
havioral, genetic, and neuroimaging research methods. Her research aims to understand
how early life experiences can change the brain’s capacity for language and learning, with
particular focus on understanding development in high-risk environments.
Marcel A. Just, D. O. Hebb Professor of Cognitive Neuroscience at Carnegie Mellon and
Director of its Center for Cognitive Brain Imaging, uses fMRI to study language-related
neural processing. The research uses machine learning and other techniques to identify
the semantic components of the neural signature of individual concepts, such as concrete
objects (e.g., hammer), emotions (e.g., sadness), and quantities (e.g., three). The projects
examine normal concept representations in college students, as well as disordered
concepts in special populations, such as patients with autism or suicidal ideation.
Contributors xv
David Kemmerer’s empirical and theoretical work focuses mainly on how different
conceptual domains are mediated by different cortical systems. He is especially inter-
ested in the relationships between semantics, grammar, perception, and action, and
in cross-linguistic similarities and differences in conceptual representation. He has
published over 60 articles and chapters, and also wrote an introductory textbook called
Cognitive Neuroscience of Language 2015.
Judith F. Kroll is Distinguished Professor of Psychology at the University of California,
Riverside, and former Director of the Center for Language Science at Pennsylvania State
University. Her research takes a cognitive neuroscience approach to second-language
learning and bilingualism.
Jan Kujala is a Staff Scientist at the Department of Neuroscience and Biomedical
Engineering, Aalto University, Finland. He has introduced and actively develops
magnetoencephalography (MEG)- based methods for investigating cortico-
cortical
connectivity and applying them in the language domain.
Nicole Landi is an Associate Professor of Psychological Sciences at the University of
Connecticut and the Director of EEG Research at Haskins Laboratories. Dr. Landi’s re-
search seeks to better understand typical and atypical language and reading develop-
ment using cognitive neuroscience and genetic methodologies.
Laurel A. Lawyer is a Lecturer in Psycholinguistics at the University of Essex. Her work
has looked at the intersection of phonological theory and speech perception, as well as
aspects of deaf language processing. Her current work investigates morphological de-
composition in speech perception, and ambient language processing in children with
cochlear implants and normal hearing adults.
Michelle Leckey is a PhD candidate in the Psychology Department at the University of
Illinois, Urbana-Champaign. As a member of the Cognition and Brain Lab, her research
uses electrophysiological methods to investigate syntactic processing across the life
span, as well as individual differences that impact lateralization of language processing.
Mia Liljeström received her doctoral degree from Aalto University, Finland. She is cur-
rently working at the Department of Neuroscience and Biomedical Engineering at Aalto
University, Finland, where she combines magnetoencephalography (MEG) and func-
tional magnetic resonance imaging (fMRI) to study large-scale functional networks un-
derlying language and speech.
Jennifer E. Mack is an Assistant Professor in the Department of Communication
Disorders at the University of Massachusetts–Amherst. Her research focuses on the
neural and cognitive basis of sentence processing impairments and recovery in aphasia,
using methods such as eye-tracking and magnetic resonance imaging (MRI).
Bradford Z. Mahon is an Associate Professor in the Department of Psychology at
Carnegie Mellon University. He is Co-Editor-in-Chief of Cognitive Neuropsychology.
His research program uses structural and functional magnetic resonance imaging
xvi Contributors
(MRI) and behavioral testing in patients with acquired brain lesions to test cognitive
and neural models of normal function, and to develop prognostic indicators of long-
term recovery.
Yasuyo Minagawa is a Professor of the Department of Psychology at Keio University.
She received her PhD in medicine from the University of Tokyo in 2000. Her research
examines the development of perception and cognition, with a focus on speech percep-
tion, social cognition, and typical and atypical brain development.
Elizabeth Musz is currently a Postdoctoral Research Fellow in the Psychological and
Brain Sciences Department at Johns Hopkins University. Her research uses neuroim-
aging to study how conceptual information is represented in the brain. She received her
PhD in Psychology at the University of Pennsylvania.
Aaron J. Newman, BA (Winnipeg), MSc, PhD (Oregon), is a Professor in the
Departments of Psychology & Neuroscience, Pediatrics, Psychiatry, and Surgery at
Dalhousie University in Halifax, Canada. His research focuses on the use of behavioral,
neuropsychological, and multi-modal neuroimaging methods to study neuroplasticity
in language and related systems. He is actively involved in training and research
initiatives involving applications and commercialization of neuroscience.
Mattson Ogg is a PhD student in the Neuroscience and Cognitive Science Program at
the University of Maryland, College Park. His background as a musician and recording
engineer informs his approach to the study of music and language. His specific interests
center around how listeners recognize sound sources.
Myriam Oliver, PhD, is a cognitive neuroscientist who obtained her PhD at the Basque
Center on Cognition, Brain and Language (BCBL). Currently, she works at the Hoeft
Lab for Educational Neuroscience at University of California, San Francisco. The main
aim of her research is to understand how reading modulates structurally and function-
ally the neural networks in healthy bilinguals and monolinguals.
Pedro M. Paz-Alonso, PhD, is the Principal Investigator leading the Language and
Memory Control research group at the Basque Center on Cognition, Brain and
Language (BCBL). He received his training at the Center for Mind and Brain at the
University of California, Davis, and at the Helen Wills Neuroscience Institute at the
University of California, Berkeley. He uses functional and structural MRI to further un-
derstand the neurobiological basis of reading, language control, and memory processes
and their development over childhood. He has been recently awarded with the Ramón y
Cajal research fellowship.
Jonathan E. Peelle is in the Department of Otolaryngology at Washington University in
Saint Louis. His research investigates the neuroscience of speech comprehension, aging,
and hearing impairment using a combination of behavioral and brain imaging methods.
Vitória Piai is an Associate Principal Investigator at the Donders Institute for Brain,
Cognition and Behaviour of Radboud University and Radboud University Medical
Contributors xvii
Center. Her research focuses on language function in healthy populations as well as

populations with speech or language impairment. She works with a range of behavioral
and neuroimaging methods and pays special attention to the intersection of language
and other functions, such as executive control, (semantic) memory, and motor control
in the case of speaking.
David Poeppel is the Director of the Department of Neuroscience at the Max-Planck-
Institute (MPIEA) in Frankfurt, Germany, and a Professor of Psychology and Neural
Science at New York University. His group focuses on the brain basis of hearing, speech,
language, and music processing.
Lara R. Polse received her PhD and training in Speech Language Pathology from the
San Diego State University/University of California, San Diego, Joint Doctoral Program
in Language and Communicative Disorders. She is currently a Speech Language
Pathologist in Davis, California.
Jeremy Purcell is a cognitive neuroscience Research Scientist in the Cognitive Science
Department at Johns Hopkins University. His research uses both cognitive neuro-
psychology and neuroimaging methods to study the neural bases of orthographic
representations in both reading and spelling.
Ileana Quiñones, PhD, is a Postdoctoral Researcher at the Basque Center on Cognition,
Brain and Language (BCBL). Her main research interests focus on the character-
ization of the brain dynamics underlying language comprehension, a theoretical
problem with a direct impact on education and social policies. Her research experi-
ence includes studies with healthy participants and atypical populations, with dif-
ferent paradigms and with a varied of behavioral and neuroimaging techniques
(e.g., electroencephalographic, MRI, fMRI and DTI).
Alexander Michael Rapp , PhD, MD, is a Psychiatrist and Researcher at the University
of Tübingen, Germany. His research interests include the functional neuroanatomy of
non-literal language in healthy subjects and patients with psychiatric diseases.
Brenda Rapp is a Professor of Cognitive Science at Johns Hopkins University and
Editor- Chief of the journal Cognitive Neuropsychology. Her research focuses
in-
on understanding the nature of the cognitive and neural bases of the orthographic
representations and processes that support reading and spelling. To this end,
she applies the methods of cognitive neuropsychology, psycholinguistics, and
neuroimaging.
Judy S. Reilly, PhD, is a developmental psycholinguist who has worked on affect and
language development (spoken and written) in both typical and atypical populations.
Riitta Salmelin is Professor of Imaging Neuroscience at the Department of Neuroscience
and Biomedical Engineering, Aalto University, Finland. She has pioneered the use of
magnetoencephalography (MEG) in language research, and has a strong track record
in multidisciplinary neuroscience research and training. She has edited the handbook
xviii Contributors
MEG: An Introduction to Methods (Oxford University Press 2010) and serves as Associate
Editor of the journal Human Brain Mapping.
Niels O. Schiller is Professor of Psycho-and Neurolinguistics at Leiden University. He
is Academic Director of the Leiden University Centre for Linguistics (LUCL) and serves
on the board of the Leiden Institute for Brain and Cognition (LIBC). His research areas
include syntactic, morphological, and phonological processes in language production
and reading aloud. Furthermore, he is interested in articulatory-motor processes during
speech production, language processing in neurologically impaired patients, and fo-
rensic phonetics.
Matthias Schlesewsky is a Professor in the School of Psychology, Social Work and Social
Policy at the University of South Australia in Adelaide. He was previously Professor
of General Linguistics at the University of Mainz, Germany, and, prior to that, one of
the first “Junior Professors” in the German Academic System, with a position at the
University of Marburg. His main research interests are in the neurobiology of language
and changes in language processing over the life span.
Theresa Schölderle, PhD, is a Research Associate in the Clinical Neuropsychology
Research Group (EKN) at the Institute of Phonetics and Speech Processing, University
of Munich, Germany. Her main area of research is early-acquired dysarthria. Moreover,
she works as a speech therapist in an institution for children and adults with multiple
disabilities.
Teresa Schuhmann is an Associate Professor of Cognitive Neuroscience at Maastricht
University. Her research focuses on applying various noninvasive neuromodulation
techniques in cognitive and clinical neuroscience. Dr. Schuhmann is one of the pioneers
in the combination of neuroimaging and neuromodulation techniques for studying the
network dynamics underlying language production.
L. Robert Slevc is an Associate Professor of Psychology, part of the program in
Neuroscience and Cognitive Science, and a member of the Maryland Language
Science Center at the University of Maryland, College Park. His research focuses on
the cognitive mechanisms underlying language processing, music processing, and their
relationships in both normal and brain-damaged populations.
Karsten Specht is a cognitive neuroscientist. He is a Professor at the Department of
Biological and Medical Psychology at the University of Bergen, Norway, where he
became head of the Bergen fMRI group, and he also holds a guest professorship at
the Arctic University of Norway in Tromsø. His main research focus is on clinical
multimodal neuroimaging, auditory perception of speech and music, connectivity
and plasticity of the language network, and rehabilitation from speech and language
disorders.
Anja Staiger, PhD, is a speech therapist and neurophonetician. She works as a Research
Associate in the Clinical Neuropsychology Research Group (EKN) at the Institute of
Contributors xix
Phonetics and Speech Processing, University of Munich, Germany. Her main areas of
research are speech motor disorders (apraxia of speech and dysarthria).
Cynthia K. Thompson is a Ralph and Jean Sundin Distinguished Professor of
Communication Science, Professor of Neurology, and Director of the Center for
the Neurobiology of Language Recovery (CNLR) at Northwestern University. Her
work, supported by the National Institutes of Health throughout her academic ca-
reer, examines normal and disordered sentence processing (and recovery in aphasia),
using online (i.e., eye-tracking), multimodal neuroimaging, and other methods. She
has published her work in more than 150 papers in referred journals, numerous book
chapters, and two books.
Sharon L. Thompson-Schill is the Christopher H. Browne Distinguished Professor of
Psychology at the University of Pennsylvania, and the founding Director of mindCORE,
Penn’s hub for the integrative study of the mind. Thompson-Schill’s lab studies the bio-
logical bases of human cognitive systems. She uses a combination of psychological and
neuroscientific methods, in both healthy and brain-damaged individuals, to study the
psychological, neurological, and genetic bases of complex thought and behavior, in-
cluding topics in perception, attention, memory, language, and decision-making.
Pascale Tremblay is Associate Professor of Speech-Language Pathology at Université
Laval in Quebec City, Canada, Researcher at the CERVO Brain Research Center, and
Director of the Speech and Hearing Neuroscience Laboratory. Her research focuses on
the cognitive neuroscience of speech perception and production and on cognitive aging.
Jos J. A. van Berkum is Professor in Communication, Cognition and Emotion at
Utrecht University. His research explores the pragmatic aspects of language comprehen-
sion, with a particular focus on affective and social factors.
Lise Van der Haegen is a Postdoctoral Researcher at the Department of Experimental
Psychology (Ghent University, Belgium), funded by the Research Foundation Flanders.
Her research focuses on (a)typical lateralization of language and face processing in left-
handers and bilinguals.
Stephen M. Wilson is Associate Professor of Hearing and Speech Sciences at Vanderbilt
University Medical Center. His research interests are aphasia and neuroimaging of lan-
guage processing.
Wolfram Ziegler, PhD, is Professor of Neurophonetics and Head of the Clinical
Neuropsychology Research Group (EKN) at the Institute of Phonetics and Speech
Processing, University of Munich, Germany. His main areas of research are speech
motor control and disorders.
Chapter 1
Neuroling u i st i c s
A Brief Historical Perspective
Sheila E. Blumstein
The past 50 years have witnessed a revolution in our understanding of how the faculties
of mind intersect with the brain. A major piece of this endeavor is neurolinguistics, the
study of the neural mechanisms underlying language. The scientific field of neurolin-
guistics was originally defined by Harry Whitaker and remains the centerpiece of the
journal Brain and Language, which he founded in 1974. However, the spirit of neurolin-
guistics predates the 1970s and has been and continues to be the subject of study under
the guise of a number of other disciplines, including neuropsychology, aphasiology, psy-
cholinguistics, and the cognitive neuroscience of language. While it is beyond the scope
of this chapter to chart the complete history of the study of the brain and language, it
will provide a retrospective view, focusing on the foundations from which our current
knowledge and questions derive, the theoretical underpinnings that still guide much of
our current research, what we have learned, and what questions and challenges remain
for the future.
Neuropsychology of Language and

the Birth of Neurolinguistics
There is a long history examining the effects of brain injury on language. From the work
of Paul Broca and Carl Wernicke, it was shown that lesions to particular areas of the
brain had specific and different consequences on language behavior. Indeed, classical
aphasiology, exemplified by the works of Kurt Goldstein (1948), Henry Head (1926),
and Alexander Luria (1966), to name a few, provided detailed descriptions of the clin-
ical syndromes that emerged pursuant to lesions to particular areas of the brain. These
clinical syndromes identified a constellation of impaired and spared language abilities
2 Sheila E. Blumstein
centering on the following: speech output, its fluency and articulation; auditory com-
prehension of sounds, words, sentences; naming; repetition of words and sentences; and
secondary language skills including reading and writing. Among these syndromes, it is
the language behavior of patients who were clinically classified as Broca’s, Conduction,
and Wernicke’s aphasia that served as the primary foundation for the detailed examina-
tion of the nature of the underlying language deficits.
From this work emerged the view that language for most individuals was left-
hemisphere dominant, and that there was a direct relation between neural areas and
language function (i.e., one could “predict” the aphasia syndrome based on lesion
site and vice versa). Indeed, the seminal monograph Disconnexion Syndromes in
Animals and Man, written by Norman Geschwind (1965), built on the classical work
of the nineteenth-and twentieth-century “diagram makers.” The diagram makers,
led by Wernicke and Ludwig Lichtheim, identified brain regions that were “centers”
specialized for particular language functions, and they mapped out, in the form of
diagrams, these centers and the connections between them. In this view, damage to
these functional centers or to the connections between them gave rise to the clas-
sical aphasia syndromes (for a detailed review, see Levelt, 2013). Lesion localization
was limited by the technology of the day, and, as we will see, with increased advances,
the accuracy of lesion-symptom mapping could not be sustained in its entirety.
Nonetheless, the notion that there were functionally specialized neural centers with
white and gray matter connections between them was a major advancement in the
field, as it became the working model characterizing language deficits in aphasia and
language-brain relations more broadly.
While the aphasia syndromes provided a rich tapestry of spared and impaired lan-
guage abilities, they beg the question of what aspects of linguistic function may be
compromised. In particular, linguists have long assumed that language is broken down
into a hierarchy of structural components, including sounds (phonetics and pho-
nology), words (the lexicon), sentences (syntax), and meaning (semantics). Each of
these components has its own set of properties or representations.
Considering language deficits in aphasia from this linguistic perspective, a dif-
ferent set of questions arise, one that more directly addresses the nature of language
deficits in aphasia. Here, for example, one can ask whether the basis of the auditory
comprehension deficit in Wernicke’s aphasia is due to an impairment in processing the
sounds of speech, in mapping sounds to words, in processing the meanings of words,
or whether the nonfluent, often agrammatic production deficit of Broca’s aphasics is
due to an articulatory/phonological impairment, a syntactic impairment, or simply to
an economy of effort. And more generally, one can ask whether linguistic deficits in
aphasia reflect impairments to representations or to the processes that access them.
This is not to say that there was no attempt by the early aphasiologists to “explain”
the nature of the deficits giving rise to aphasia. For example, Broca proposed that the
third frontal convolution (i.e., Broca’s area) was the “center for articulated speech”
and Wernicke proposed that the auditory comprehension impairment of Wernicke’s
aphasics was due to a deficit in “auditory images.” However, what distinguished these
Neurolinguistics: A Brief Historical Perspective 3
classical approaches to the syndromes of aphasia and the more modern era was that the
functional centers were defined descriptively absent a linguistic theoretical framework,
and the hypothesized functions were based on clinical observations rather than being
tested experimentally.
The beginning of the modern era in neurolinguistics began with linguistic approaches
to aphasia. There were two pioneers, Roman Jakobson and Harold Goodglass, who most
clearly led this new approach to the study of the brain and language. Roman Jakobson
was perhaps the first linguist to consider the aphasias from a linguistic perspec-
tive, suggesting that the breakdown of speech in aphasia and its development in chil-
dren reflect phonological universals of language (Jakobson, 1941, translated 1972), and
proposing that the output deficits in Broca’s and Wernicke’s aphasia reflect impairments
in syntagmatic and paradigmatic axes of language, with a syntagmatic deficit giving rise
to Broca’s aphasics’ syntactic deficit and a paradigmatic deficit giving rise to Wernicke’s
aphasics’ paragrammatic deficit (Jakobson, 1956). Interestingly, in present-day parlance,
this distinction reflects a sequencing disorder for Broca’s aphasics and a word selec-
tion disorder for Wernicke’s aphasics. Harold Goodglass, who established the Boston
Aphasia Research Center along with the neurologist Norman Geschwind and led it
from the mid-1960s to 1996, was among the first to apply experimental methods drawn
from psycholinguistics and cognitive psychology to systematically examine the nature
of linguistic deficits in aphasia (Goodglass, 1993). It is this multidisciplinary approach,
focusing on the confluence of the study of the classical aphasia syndromes with theoret-
ical approaches to language and experimental methodology, that gave birth to what we
now call neurolinguistics.
Experimental Approaches to the Study

of Language Deficits in Aphasia
Neurolinguistic approaches to the study of aphasia were guided by the view that lan-
guage is hierarchically organized into structural components and that these structural
components mapped directly to functionally defined neural regions. Thus, the focus
was on conducting parametric studies of linguistic features of the classical aphasias.
For Broca’s aphasics, the emphasis was on potential syntactic impairments and pho-
netic/phonological deficits. Early results suggested that indeed Broca’s aphasics not
only had agrammatic deficits in production, but also displayed auditory compre-
hension impairments when the only cues available were syntactic in nature (Zurif &
Caramazza, 1976; Zurif, Caramazza, & Myerson, 1972). Moreover, studies of the acoustic
properties of speech production suggested that these patients had articulatory/phonetic
planning impairments (see Blumstein, 1981, for review). For Wernicke’s aphasics, the
question was whether their auditory comprehension impairments were due to phono-
logical deficits where phonemic misperceptions might give rise to selecting incorrect
words (e.g., hearing bear and pointing to pear) or were due to semantic impairments
reflecting deficits in the underlying meaning representations of words or in accessing
the meaning of words. Early results suggested that although these patients do have
deficits in perceiving phonological contrasts, it did not predict the severity of their au-
ditory comprehension impairments (Basso, Casati, & Vignolo, 1977; Blumstein, Baker,
& Goodglass, 1977; but see the following section, “The Modern Era,” in this chapter).
Those studies examining word meaning indicated that the underlying representations
of words appeared to be relatively spared in aphasia, while access to meaning and the
time course of mapping sounds to word meanings was impaired (Milberg & Blumstein,
1981; Swinney, Zurif, & Nicol, 1989; but see the following section, “The Modern Era,” in
this chapter). A broad range of behavioral paradigms have been used in these studies,
including discrimination, identification, and psychophysical experiments of speech,
word and/or picture matching, lexical decision, hierarchical clustering, and grammat-
icality judgments, to name a few.
Nonetheless, this approach using aphasia syndromes as a basis for neurolinguistic
investigations was not without its critics. They came from two directions. One reflected a
challenge to group studies using the classical aphasia syndromes, and the other reflected
a challenge to one-to-one mapping between aphasia syndrome and lesion localization.
With regard to the former, it was proposed that clinical syndromes do not necessarily
cut across linguistic domains, and hence the study of patients grouped by aphasia syn-
drome may not provide a window into the particular linguistic deficit (Caramazza, 1984,
1986). Moreover, classification is a “messy” thing; there is variability in severity across
patients within the defining properties of the syndrome, and some patients do not have
all of the symptoms included in the classification schema. Hence, in this view, group
studies using the classical aphasia syndromes are by their very nature fundamentally
flawed (Schwartz, 1984).
To mitigate this concern, these researchers took a single case study approach where
detailed analyses were conducted of particular linguistic deficits. The overarching goal
was to use the effects of brain damage as a window into current linguistic theories and
theories of language processing (Caramazza, 1986). However, in contrast to classical
neuropsychological case studies that described unique behavioral patterns in relation to
lesion localization, this approach was typically agnostic with respect to the lesion status
of the patient. Hence, while it provided interesting insights into the linguistic nature of
deficits in aphasia (see Rapp & Goldrick, 2006, for a review), this case study approach
did not consider the neural systems underlying such deficits, and hence provided lim-
ited insight into the relation between brain and language.
Turning to lesion localization of the aphasia syndromes, technological advances in
neuroimaging over the past 20 years have provided a cautionary tale for the view that
there is a one-to-one mapping between aphasia syndrome and lesion. First, the lesions
associated with the classical aphasias were typically described incorrectly as only cor-
tical. However, in fact, lesions of patients commonly include both cortical and subcor-
tical structures (see Copland & Angwin, Chapter 33 in this volume). Second, lesions
of patients with aphasia are rarely focal; rather, they tend to be large, encompassing a
number of neural regions. If the lesions are focal, patients typically present with tran-
sient aphasias, not the chronic syndrome profile of the classical aphasias. Third, there is
variability across lesion profiles. As one might expect, no aphasic has exactly the same
lesion profile. As a result, there are differences across aphasics in the degree of damage
in a particular area, as well as in the extent to which a lesion extends to other areas of
the brain. Finally, research has shown that there is not a one-to-one relation between
aphasia syndrome and lesion localization. For example, not all Broca’s aphasics have
lesions in Broca’s area (BA45), nor do all patients with damage in Broca’s area present
with Broca’s aphasia (see Dronkers, 2000).
These issues notwithstanding, neurolinguistic investigations using the aphasia
syndromes have provided the basis for much of the modern era spanning the last
20 years. The aphasia syndromes have provided a theoretical framework for examining
linguistic deficits, suggesting that the temporal lobe is involved in accessing the
meanings of words, that the superior temporal gyrus is involved in auditory processing
of speech, that posterior temporal areas are involved in integrating auditory and artic-
ulatory processes, and that the inferior frontal gyrus (IFG) is involved in processing
syntax and articulatory planning as well as in selection processes for words. Critically,
these studies have shown, across patient groups and lesion sites, similar patterns of per-
formance as a function of linguistic structural complexity. For example, for all patients,
structurally more complex sentences are more difficult to comprehend and also to
produce, and the perception and production of phonologically similar words result
in increased errors compared to words that share few phonological attributes. Despite
claims that deficits are “selective” to a particular component of the grammar (e.g.,
syntax), nearly all aphasics, regardless of syndrome, display impairments that affect
multiple linguistic components of language, although the severity of the impairment
and the underlying functional impairment may differ.
Taken together, these studies were among the first to suggest that neural systems un-
derlying different components of the grammar are broadly tuned, recruiting neural
systems rather than functionally encapsulated neural regions (see the following sec-
tion, “The Modern Era,” for further discussion). Nonetheless, although patients across
syndromes may appear to share a common impairment to a particular aspect of lan-
guage, the patterns of performance of the patients may differ, suggesting that different
functional impairments emerge as a function of lesion site. For example, both Broca’s
and Wernicke’s aphasics show lexical processing impairments. However, examining ac-
cess to words that share phonological attributes and hence are lexical competitors (e.g.
hammer vs. hammock), lexical candidates appear to stay active longer than normal for
Wernicke’s aphasics. In contrast, Broca’s aphasics fail to select the target from among
competing lexical alternatives (Janse, 2006; Utman, Blumstein, & Sullivan, 2001; Yee,
Blumstein, & Sedivy, 2008). These findings support the structural integrity of the lexical
system in these two groups of patients since lexical access is influenced by whether lex-
ical items are competitors or not, but suggest different processing impairments in each
group, presumably reflecting the distinct functional roles of temporal and frontal lobe
structures in lexical access.
The Modern Era
It is difficult to demarcate when the “modern era” began, as science typically progresses
incrementally. Nonetheless, there are a number of factors that contributed to what
characterizes today’s approach to neurolinguistics. One has to do with the increased in-
fluence of computational models that focus on the nature of information flow in the
language system (cf. Dell, 1986; Marslen-Wilson, 1987; McClelland, 1988). These models
enriched earlier structural models in which components of the grammar were separate,
encapsulated modules (cf. Fodor, 1983). Additionally, computational models have been
used to characterize language impairments in aphasia in terms of processing deficits
(Dell, Schwartz, Martin, Saffran, & Gagnon, 1997; McNellis & Blumstein, 2001; Mirman,
Yee, Blumstein, & Magnuson, 2011; Rapp & Goldrick, 2000).
Neurobiologically inspired computational models, beginning with the Parallel
Distributed Processing (PDP) models of the 1980s (Rumelhart & McClelland, 1986) and
continuing through contemporary models (Bornkessel- Schlesewsky, Schlesewsky,
Small, & Rauschecker, 2015; Horwitz, Friston, & Taylor, 2000; Wennekers, Garagnani,
& Pulvermuller, 2006), are a major advance as they seek to develop neurally plausible
models of how the brain processes speech and language. For example, the use of neuron-
like elements allow for graded responses to stimulus input, and Hebbian learning
mechanisms can characterize word learning (Garagnani, Wennekers, & Pulvermuller,
2007). Hebbian learning has also been used to simulate lexically guided tuning effects
on speech perception (Mirman, McClelland, & Holt, 2006).
Without question, the single most important factor that has influenced the cur-
rent state of the art in neurolinguistics research is technological advances that have
revolutionized our ability to map structural and functional properties of the brain.
It is beyond the scope of this introductory chapter to review all of the methods, the
advantages and disadvantages of each, and their contributions to our understanding of
the neural basis of language, but suffice to say they provide a broad spectrum of tools
that afford different sources of information about neural activity. To briefly consider a
few, functional magnetic resonance imaging (fMRI) has had perhaps the greatest in-
fluence because it is a noninvasive procedure that allows for examining neural ac-
tivation patterns with parametric manipulations of linguistic constructs (see Heim
& Specht, Chapter 4 in this volume). Other important methods that are now a part of
the toolbox of the neurolinguist include transcranial magnetic stimulation (TMS) that
allows for the creation of “virtual lesions” (see Schuhmann, Chapter 5 in this volume),
and electrophysiological measures such as event-related potentials (ERPs; see Leckey
& Federmeier, Chapter 3 in this volume) and magnetoencephalography (MEG; see
Salmelin, Kujala, & Liljeström, Chapter 6 in this volume) that provide crucial infor-
mation about the time course (in ms) of processing. Enhanced structural information
comes from magnetic resonance imaging (MRI), which has provided advances in un-
derstanding the structural properties of the normal brain and in mapping lesions using
voxel-based lesion mapping (Bates, Wilson, Saygin, Dick, Sereno, Knight, & Dronkers,
2003; see Wilson, Chapter 2 in this volume). Additional insights come from diffusion
tensor imaging (DTI), which measures white matter fiber tracts and connections be-
tween different parts of the brain (see Catani & Forkel, Chapter 9 in this volume).
These technological advances have enriched and deepened the ability to study
the neural processing of language and have opened up new avenues of research, in-
cluding examining neural systems underlying language processing in the normal brain,
comparing the relation between language processing in the normal brain with that of
the diseased brain, charting potential changes in the neural systems underlying lan-
guage across the life span, and investigating the plasticity of the neural system in per-
sons under conditions of natural recovery or as a result of rehabilitation techniques.
These research areas are by no means all inclusive, but they underscore how technolog-
ical advances can open up new and critically important avenues of research that hold the
promise of providing deeper insights into our understanding of language and the brain.
Let us consider a few of these as a window into the neurolinguistics of today.
There is a vast literature that has examined the neural substrates of language using
fMRI. Among the many contributions of functional neuroimaging studies, there are two
sets of findings that have challenged earlier theoretical assumptions. The first concerns
the extent to which the organization of language is strictly modular, with components of
the grammar having a distinct neural locus, and the second concerns the nature of in-
formation flow between and among components of the grammar.
Turning to the modularity of components of language, fMRI studies investigating the
neural areas involved in processing different components of the grammar have shown
that the neural substrates of phonetics/phonology, the lexicon, syntax, and meaning
do not recruit a single neural area dedicated to its processing, but rather each recruits
a broadly distributed network or processing stream (for reviews, see Binder, Desai,
Graves, & Conant, 2009; Hickok, 2009; Hickok & Poeppel, 2007; Kaan & Swaab, 2002;
Blumstein & Myers, 2013; Price, 2012). Thus, the brain is not divided into functionally
dedicated “linguistic” modules. For example, although the superior temporal gyrus
and superior temporal sulcus appear to be recruited in the early auditory processing
of the speech signal, neural activation occurs in other areas, including the middle tem-
poral gyrus (MTG), parietal areas including the angular gyrus (AG) and supramarginal
gyrus (SMG), and frontal areas including the IFG (e.g., Hickok & Poeppel, 2007; Scott
& Wise, 2004). Although they appear to have different functional properties, these
neural areas work together as part of a single system involved in the perception of
speech. Similar findings have emerged in examining neural activation patterns of the
lexicon, syntax, and semantics (Vigneau, Beaucousin, Herve, Duffau, et al., 2006).
Interestingly, precursors to these fMRI findings are found in the behavioral studies of
the aphasia syndromes. Here, Broca’s and Wernicke’s aphasics display impairments in
multiple components of the grammar, including speech, lexical, syntactic, and semantic
processing, although the severity and the functional basis of the deficit appear to differ
across these groups (Blumstein, 1995).
The fMRI findings have also shown that information flow between components of
the grammar is interactive such that activation in one component of the grammar di-
rectly influences activation in other components of the grammar. Thus, for example, the
lexical structure of words has a modulatory effect not only on neural areas involved in
lexical access (SMG), but also in areas involved in phonological and phonetic processing
(IFG and pre-central gyrus) (Peramunage, Blumstein, Myers, Goldrick, & Baese-Berk,
2011). Such evidence is consistent with cascading models of language processing in
which information at one level of processing influences processing downstream from
it. Such findings challenge those models in which processing within one component of
the grammar is separate from and independent of processing in other components (e.g.,
Levelt, Roelofs, & Meyer, 1999).
An important, relatively new direction that the field has taken is to examine the neural
systems underlying language using more than one neuroimaging and/or electrophysio-
logical method. As indicated earlier, the lesion method has limitations because lesions
tend to be very large, making it difficult to assess what neural areas contribute to, or are
“responsible” for, the particular language impairment. Related issues arise from fMRI
studies (see Heim & Specht, Chapter 4 in this volume). Typically, multiple neural areas
show activation, and researchers infer the functional role of these areas, often drawing
from current models of the neural architecture of language. Thus, it is not clear what
actual role these areas may play, or if they are even involved in the linguistic “function”
under investigation (Price, Mummery, Moore, Frackowiak, & Friston, 1999; Rorden
& Karnath, 2004). Moreover, areas other than the classical language areas may be ac-
tivated. However, it is not known what role, if any, these play in language processing.
Two methods have provided a critically important window into this potential challenge.
The first is to couple fMRI results with MRI analyses of lesion data, and the second is to
couple fMRI with TMS. In both cases, it would be expected that either a real or “virtual”
lesion to a particular neural region that was activated under normal conditions using
fMRI would result in a behavioral deficit or a decrement in processing.
Finally, the modern era has witnessed a renewed interest in localization using voxel-
based lesion mapping to examine the effects of brain injury on language processing
(Bates, Wilson, Saygin, Dick, Sereno, Knight, & Dronkers, 2003; see Wilson, Chapter 2
in this volume). With greater precision in quantitatively identifying and measuring the
location, size, and extent of lesions, new results extend and clarify past findings. In par-
ticular, in contrast to earlier findings, it has been shown that phonological perception
impairments predict auditory language comprehension deficits in Wernicke’s aphasics,
although these patients may also display deficits in accessing word meaning (Robson,
Keidel, Lambon Ralph, & Sage, 2012). Furthermore, contrary to the view that the lexical-
semantic network is preserved in aphasia, degradation of semantic structure does occur
in aphasic patients with lesions extending to the anterior temporal lobe (Jeffries &
Lambon Ralph, 2006; Walker, Schwartz, Kimberg, Faseyitan, et al., 2011).
Neurolinguistics in the modern era has also begun to play a central role in the adju-
dication and development of a number of competing theories of language. Here, the
notion is that activation of particular neural areas will distinguish between/among
the claims made by these theories. For example, there are current opposing views as
to whether lexical representations are “embodied” or symbolic (amodal) (for reviews,
see Jirak, Menz, Buccino, Borghi, & Binkofski, 2010; Kiefer & Pulvermuller, 2012). If
embodied, reflecting our experience with words and the sensorimotor systems that
are activated in the use of these words, then there should be activation in the particular
sensorimotor system that is represented in that concept when hearing or using a word
(for example, to “pick up” a paper requires hand motor activity, whereas to “kick” a ball
requires foot motor activity; see Pulvermüller, Shtyrov, & Ilmoniemi, 2005). If symbolic,
there would not be a systematic relation between a word’s meaning and activation of the
sensorimotor systems it relates to, and amodal neural areas would be activated.
There are a number of other questions raised by competing theories that are currently
being “tested” using neuroimaging techniques. These include the following:
• Are speech perception representations motor (articulatorily or gesturally based)

consistent with claims of the motor theory of speech perception (Liberman,
Cooper, Shankweiler, & Studdert-Kennedy, 1967; for neural evidence, see Fadiga,
Craighero, Buccino, & Rizzolatti, 2002; Wilson, Saygin, Sereno, & Iacoboni, 2004),
or acoustic (auditorily based) consistent with early distinctive feature theories and
the acoustic theory of speech perception (Jakobson, Fant, & Halle, 1961; Stevens
& Blumstein, 1981; for neural evidence, see Chang, Rieger, Johnson, et al., 2010;
Cheung, Hamilton, Johnson, & Chang, 2016; Lotto, Hickok, & Holt, 2009)?
• Is the influence of contextual factors on degraded speech perceptual (Marslen-
Wilson & Tyler, 1980) or post-perceptual (Norris, McQueen, & Cutler, 2000)?
• Is language modular (Federenko & Thomspon-Schill, 2014), or is it built on domain
general functions, processes, and/or computations (Blumstein & Amso, 2013; Kelly
& Martin, 1994)?
While the jury is still out, and the debates continue, further evidence from neurolinguis-
tics using state-of-the-art imaging and analysis techniques holds the promise of not only
distinguishing among the claims made by these theories, but potentially developing new
and unique theories.
Future Directions and Challenges
While it is clear that the processing of language recruits a broad network of areas,
the functional role of these areas needs to be more clearly delineated, as typically the
functional role of these regions is inferred, based on traditional models of the neural
organization of language. A focus on networks examining functional and/or effective
connectivity (Friston, 1994; Horwitz, Friston, & Taylor, 2000), rather than individual
neural regions, is a critical next step for understanding the complexities of language
processing. Moreover, while it is clear that components of the grammar activate
a neural processing stream, the direction of information flow has not been fully
mapped out (for overviews of grammatical processing, see, in this volume, Kemmerer,
Chapter 30; Thompson & Mack, Chapter 31). Indeed, as described earlier, there is an
ongoing debate in the literature about the extent to which information flow is bottom-
up, with contextual influences on sensory processes reflecting executive decision-
related processes, or whether information flow is also top-down, with higher-level
information sources influencing lower levels of sensory processing (Guediche,
Blumstein, Fiez, & Holt, 2013). Here, functional connectivity analyses and methods
sensitive to the time course of processing, such as MEG and ERP, coupled with the
good spatial resolution afforded by fMRI, hold the promise of providing useful
insights. Further questions along these lines are to understand how linguistic infor-
mation across modalities is integrated. How do production (articulatory) processes
influence perception (auditory) processes (see, in this volume, Hickok, Chapter 20;
Poeppel, Cogan, Davidesco, & Flinker, Chapter 26)? How does visual information in
speech production integrate with auditory information in speech perception? (see,
in this volume, Newman, Chapter 14; Corina & Lawyer, Chapter 16; Rapp & Purcell,
Chapter 17)? How does literacy (reading) affect the neural organization of lan-
guage (access to speech, words, meaning) (see, in this volume, Paz-Alonso, Oliver,
Quiñones, & Carreiras, Chapter 24; Jasinska & Landi, Chapter 25)?
A yet unanswered question is what role the right hemisphere plays in linguistic pro-
cessing. Contradictory information comes from fMRI and lesion studies. In particular,
fMRI studies commonly show activation not only in the left hemisphere, but also in ho-
mologous areas of the right hemisphere during linguistic tasks. And yet, patients with
right hemisphere lesions do not typically display linguistic deficits (although they do
show impairments in the pragmatics of language and discourse processing). Current
findings using fMRI to examine neural changes during recovery in aphasia suggest that
initially aphasics with unilateral left hemisphere lesions show increased activation of
right hemisphere structures. However, those who recover the best show progressively
less right hemisphere activation with increased activation in perilesional areas of the left
hemisphere (Pizzamiglio, Galati, & Committeri, 2001). How then can fMRI findings be
reconciled with lesion results to determine the role of the right hemisphere in linguistic
processing (see Van der Haegen & Cai, Chapter 34 in this volume)?
Finally, a major challenge for the field is to resolve what appear to be contradictory
claims about the functional role of certain neural areas, as different linguistic and/or
cognitive “functions” have been attributed to the same neural areas. For example, the
IFG has been implicated in speech production and perception (Hickok & Poeppel,
2007; Price, 2012), phonetic categorization (Myers, Blumstein, Walsh, & Eliassen, 2009),
syntactic processing (Grodzinsky & Friederici, 2006), semantic processing (Binder,
Desai, Graves, & Conant, 2009), and executive decision-making (Badre & Wagner,
2007; Thompson-Schill, D’Esposito, Aguirre, & Farah, 1997). Similarly, the superior
temporal sulcus has been implicated in speech perception and in a broad range of cogni-
tive domains, including face processing, biological motion, social perception, theory of
mind, and audiovisual integration (Hein & Knight, 2008).
One yet unanswered question is whether the field of neurolinguistics will ultimately
lead the way to the development of new models and theories of language processing.
The intersection of mind (language) and brain is uniquely the purview of neurolin-
guistics, and understanding brain organization, function, and processes provides
constraints on models and theories of language. Here, there is room for cautious opti-
mism. If, for example, evidence shows that language is built on more general domain
computations, then it may provide the impetus to develop new models of language pro-
cessing that do not rely on “specialized” vocabularies or representations at each level of
the grammar.
Neurobiologically plausible computational models will undoubtedly play an even
greater role than they do currently. As described earlier, such efforts have already begun.
However, to provide new insights into the neural basis of language, these models will
also need to reflect the complexities inherent in language processing. That is, they will
need to be able to not only reflect the functional architecture inherent within a linguistic
level (e.g., the phonological and statistical properties giving rise to lexical density), but
also model how such information affects processing throughout the system. Further,
such models will also need to consider how new words are added throughout the life
span without destabilizing the network. For example, they will need to differentiate
a production of a word that is idiosyncratic (i.e., produced in a nonstandard way by a
speaker) from one that is a new entry, and they will have to allow for creating a new rep-
resentation of a word as it is being introduced into the lexicon. At early stages, its fre-
quency of occurrence would be greater than zero but still of a very low frequency, and
thus the word would be potentially incorrectly classified as a nonword. Finally, compu-
tational models will need to be able to scale up, including a large number of stimuli and a
broad range of linguistic parameters that not only respect the functional architecture of
the language system, but also cut across levels of linguistic processing.
One area where contributions from neurolinguistics may be unique and critically
important is in the area of language rehabilitation. Basic understanding of the theo-
retical foundations of language and its neural instantiation provides a critical bridge to
rehabilitation. As we understand the functional role that particular areas play within
the neural system, targeted rehabilitation treatments may be developed that more likely
tap the underlying deficit. That is, rather than treating a particular language impair-
ment in aphasia as a unitary deficit across patients who have varying lesion profiles and
symptom complexes, different rehabilitation programs should be developed that reflect
the functional role of the neural areas that are lesioned. Thus, as described earlier, al-
though damage to either temporal or frontal structures results in deficits in lexical se-
lection, damage to temporal structures appears to result in a failure to inhibit lexical
competitors, whereas damage to frontal structures appears to result in an inability to se-
lect among competing alternatives. Although both types of aphasics then display lexical
access impairments, the deficits are not the same, and the nature of therapy developed
for these patients should presumably differ.
Additionally, as it becomes clearer that language shares more general processes with
other cognitive domains, rehabilitation programs can apply general cognitive principles
to language rehabilitation strategies. Some recent results suggest that such approaches
have potential to enhance language recovery. For example, variability of linguistic input
is the norm in real-world environments and appears to enhance language communi-
cation. Experimental studies and rehabilitation programs, however, typically remove
such variability; they present stimuli controlling for variability in order to home in on
the particular linguistic parameter being targeted for rehab. There are some hints in
the literature that aphasic participants’ performance is facilitated when presented with
stimuli that are not typical or that may vary from the norm. As an example, Kiran and
Thompson (2003) showed that exposure to atypical exemplars of words in a semantic
category generalized to naming of both atypical and typical members of the category
in aphasic participants; exposure to typical members did not generalize to atypical
members or other members of the category.
Finally, theoretical approaches to language generally consider it to be static—never
changing either within the environment, within the individual, or across the life span.
We know that this is not correct. Language is processed under multiple conditions
of variability—the environment is noisy, there is variability in speech production
of individuals as well as in their perceptual “acumen,” and there are changes in lan-
guage processing over time. Yet, both speakers and listeners maintain stability in pro-
cessing under such “adverse” conditions. We know little about how the neural system
accommodates these multiple sources of variability, and what conditions give rise
to changes to the neural substrate in development, aging, or pursuant to brain injury.
Ultimately, theories of language will have to account for such variation, and neuro-
linguistics is positioned to provide the foundation for the development of these new
models.
In summary, the history of research in neurolinguistics has provided an evolving pic-
ture of the neural basis of language. We have witnessed how the field has progressed
from a singular focus on the aphasias to the application of a broader palette of
methodologies. These methods, in conjunction with the lesion method, have enriched
and revolutionized our understanding of language and its neural basis and have pro-
vided both new insights and new challenges. No doubt new technological advances
will be developed in the next decade, as will richer computational models of language
processing. Together, they will enhance neurolinguistics, leading us to a deeper under-
standing of language—its representations, organization, and processes—and the neural
systems underlying it—their structure, connections, computations, and intersection
with other cognitive domains.
Acknowledgments
This research was supported in part by National Institutes of Health (NIH) grants R01
DC006220 and R21DC013100. The content is solely the responsibility of the author and does
not necessarily represent the official views of the NIH or the National Institute on Deafness and
Other Communication Disorders.
References
Badre, D., & Wagner, A. D. (2007). Left ventrolateral prefrontal cortex and the cognitive con-
trol of memory. Neuropsychologia, 45, 2883–2901.
Basso, A., Casati, G., & Vignolo, L. A. (1977). Phonemic identification defect in aphasia. Cortex,
13,  85–95.
Bates, E., Wilson, S. M., Saygin, A. P., Dick, F., Sereno, M. I., Knight, R. T., & Dronkers, N. F.
(2003). Voxel-based lesion–symptom mapping. Nature Neuroscience, 6, 448–450.
Binder, J. R., Desai, R. H., Graves, W. W., & Conant, L. L. (2009). Where is the semantic system?
A critical review and meta-analysis of 120 functional neuroimaging studies. Cerebral Cortex,
19, 2767–2796.
Blumstein, S. (1981). Phonological aspects of aphasia. In M. T. Sarno (Ed.), Acquired aphasia
(pp. 129–155). New York: Academic Press.
Blumstein, S. (1995). The neurobiology of language. In J. Miller & P. Eimas (Eds.), Speech, lan-
guage, and communication (pp. 339–370). New York: Academic Press.
Blumstein, S. E., & Amso, D. (2013). Dynamic functional organization of language: Insights
from functional neuroimaging. Perspectives on Psychological Science, 8(1),  44–48.
Blumstein, S. E., Baker, E., & Goodglass, H. (1977). Phonological factors in auditory compre-
hension in aphasia. Neuropsychologia, 15,  19–30.
Blumstein, S. E., & Myers, E. B. (2013). Neural systems underlying speech perception.
In K. Ochsner & S. Kosslyn (Eds.), Oxford handbook of cognitive neuroscience (Vol. 1,
pp. 507–523). New York: Oxford University Press.
Bornkessel- Schlesewsky, I., Schlesewsky, M., Small, S. L., & Rauschecker, J. P. (2015).
Neurobiological roots of language in primate audition: Common computational properties.
Trends in Cognitive Sciences, 19(3), 142–150.
Caramazza, A. (1984). The logic of neuropsychological research and the problem of patient
classifications in aphasia. Brain and Language, 21,  9–20.
Caramazza, A. (1986). On drawing inferences about the structure of normal cognitive systems
from the analysis of impaired performance: The case for single patient studies. Brain and
Cognition, 5,  41–66.
Chang, E. F., Rieger, J. W., Johnson, K., Berger, M. S., Barbaro, N. M., & Knight, R. T. (2010).
Categorical speech representation in human superior temporal gyrus. Nature Neuroscience,
13(11), 1428–1432.
Cheung, C., Hamilton, L. S., Johnson, K., & Chang, E. F. (2016). The auditory representation of
speech sounds in human motor cortex. Elife, 5, e12577.
Dell, G. S. (1986). A spreading activation theory of retrieval in sentence production.
Psychological Review, 93, 283–321.
Dell, G. S., Schwartz, M. F., Martin, N, Saffran, E. M., & Gagnon, D. A. (1997). Lexical access in
aphasic and nonaphasic speakers. Psychological Review, 104(4), 801–838.
Dronkers, N. F. (2000). The pursuit of brain–language relationships. Brain and Language,
71,  59–61.
Fadiga, L., Craighero, L., Buccino, G., & Rizzolatti, G. (2002). Speech listening specifi-
cally modulates the excitability of tongue muscles: A TMS study. European Journal of
Neuroscience, 15(2), 399–402.
Fedorenko, E., & Thompson-Schill, S. L. (2014). Reworking the language network. Trends in
Cognitive Sciences, 18(3), 120–126.
Fodor, J. (1983). The modularity of mind. Cambridge, MA: MIT Press.
Friston, K. J. (1994). Functional and effective connectivity in neuroimaging: A synthesis.

Human Brain Mapping, 2(1–2),  56–78.
Garagnani, M., Wennekers, T., & Pulvermüller, F. (2007). A neuronal model of the language
cortex. Neurocomputing, 70(10), 1914–1919.
Geschwind, N. (1965). Disconnexion syndromes in animals and man. Brain, 88, 237–294,
585–644.
Goldstein, K. (1948). Language and language disturbances. New York: Grune and Stratton.
Goodglass, H. (1993). Understanding aphasia. New York: Academic Press.
Grodzinsky, Y., & Friederici, A. D. (2006). Neuroimaging of syntax and syntactic processing.
Current Opinions in Neurobiology, 15, 240–246.
Guediche S., Blumstein S. E., Fiez, J., & Holt, L. L. (2013). Speech perception under adverse
conditions: Insights from behavioral, computational and neuroscience research. Frontiers in
Systems Neuroscience, 7.
Head, H. (1926). Aphasia and kindred disorders of speech. Vols. 1–2. Cambridge: Cambridge
University Press.
Hein, G., & Knight, R. T. (2008). Superior temporal sulcus—it’s my area: Or is it?. Journal of
cognitive neuroscience, 20(12), 2125–2136.
Hickok, G. (2009). The functional neuroanatomy of language. Physics of Life Reviews, 6,
121–143.
Hickok, G., & Poeppel, D. (2007). The cortical organization of speech processing. Nature
Reviews Neuroscience, 8, 393–402.
Horwitz, B., Friston, K. J., & Taylor, J. G. (2000). Neural modeling and functional brain im-
aging: An overview. Neural Networks, 13(8), 829–846.
Jakobson, R. (1941). Kindersprache, Aphasie, und Allgemeine Lautgesetze. Uppsala: Universitet
Arsskrift.
Jakobson, R. (1956). Two aspects of language and two types of aphasic disturbances. In R.
Jakobson & M. Halle (Eds.), Fundamentals of language (pp. 55–82). The Hague: Mouton.
Jakobson, R. (1972). Child language, aphasia, and phonological universals. The Hague: Mouton.
Jakobson, R., Fant, G., & Halle, M. (1961). Preliminaries to speech analysis: The distinctive
features and their correlates. Cambridge, MA: MIT Press.
Janse, E. (2006). Lexical competition effects in aphasia: Deactivation of lexical candidates in
spoken word processing. Brain and Language, 97,  1–11.
Jefferies, E., & Lambon Ralph, M. A. (2006). Semantic impairment in stroke aphasia versus se-
mantic dementia: A case-series comparison. Brain, 129, 2132–2147.
Jirak, D., Menz, M. M., Buccino, G., Borghi, A. M., & Binkofski, F. (2010). Grasping language: A
short story on embodiment. Consciousness and Cognition, 19(3), 711–720.
Kaan, E., & Swaab, T. Y. (2002). The brain circuitry of syntactic comprehension. Trends in
Cognitive Science, 6, 350–356.
Kelly, M. H., & Martin, S. (1994). Domain-general abilities applied to domain-specific
tasks: Sensitivity to probabilities in perception, cognition, and language. Lingua, 92, 105–140.
Kiefer, M., & Pulvermüller, F. (2012). Conceptual representations in mind and brain: Theoretical
developments, current evidence and future directions. Cortex, 48(7), 805–825.
Kiran, S., & Thompson, C. K. (2003). The role of semantic complexity in treatment of naming
deficits: Training semantic categories in fluent aphasia by controlling exemplar typicality.
Journal of Speech, Language, and Hearing Research, 46(3), 608–622.
Levelt, W. J. M. (2013). A history of psycholinguistics: The pre-Chomskyan era. Oxford: Oxford
University Press.
Levelt, W. J. M., Roelofs, A., & Meyer, A. S. (1999). A theory of lexical access in speech produc-
tion. Behavioral and Brain Sciences, 22,  1–75.
Liberman, A. M., Cooper, F. S., Shankweiler, D. P., & Studdert-Kennedy, M. (1967). Perception
of the speech code. Psychological Review, 74(6), 431.
Lotto, A. J., Hickok, G. S., & Holt, L. L. (2009). Reflections on mirror neurons and speech per-
ception. Trends in Cognitive Sciences, 13(3), 110–114.
Luria, A. R. (1966). Higher cortical functions in man. New York: Basic Books.
Marslen-Wilson, W. D. (1987). Functional parallelism in spoken word recognition. Cognition,
25(1–2), 71–102.
Marslen-Wilson, W., & Tyler, L. K. (1980). The temporal structure of spoken language under-
standing. Cognition, 8(1),  1–7 1.
McClelland, J. L. (1988). Connectionist models and psychological evidence. Journal of Memory
and Language, 27, 107–123.
McNellis, M., & Blumstein, S. E. 2001. Self-organizing dynamics of lexical access in normals
and aphasics. Journal of Cognitive Neuroscience, 13, 151–170.
Milberg, W., & Blumstein, S. E. (1981). Lexical decision and aphasia: Evidence for semantic pro-
cessing. Brain and Language, 14, 371–385.
Mirman, D., McClelland, J. L., & Holt, L. L. (2006). An interactive Hebbian account of
lexically guided tuning of speech perception. Psychonomic Bulletin & Review, 13(6),
958–965.
Mirman, D., Yee, E., Blumstein, S. E., & Magnuson, J. (2011). Theories of spoken word recogni-
tion deficits in aphasia: Evidence from eye-tracking and computational modeling. Brain and
Language, 117(2),  53–68.
Myers, E. B., Blumstein, S. E., Walsh, E., & Eliassen, J. (2009). Inferior frontal regions underlie
the perception of phonetic category invariance. Psychological Science, 20, 895–903.
Norris, D., McQueen, J. M, & Cutler, A. (2000). Merging information in speech recogni-
tion: Feedback is never necessary. The Behavioral and Brain Sciences, 23(3), 299–325; discus-
sion 325–270.
Peramunage, D., Blumstein, S. E., Myers, E. B., Goldrick, M., & Baese-Berk, M. (2011).
Phonological neighborhood effects in spoken word production: An fMRI study. Journal of
Cognitive Neuroscience, 23(3), 593–603.
Pizzamiglio, L., Galati, G., & Giorgia Committeri, G. (2001). The contribution of functional
neuroimaging to recovery after brain damage: A review. Cortex, 37(1),  11–31.
Price, C. (2012). A review and synthesis of the first 20 years of PET and fMRI studies of heard
speech, spoken language and reading. NeuroImage, 62(2), 816–847.
Price, C. J., Mummery, C. J., Moore, C. J., Frackowiak, R. S. J., & Friston, K. J. (1999). Delineating
necessary and sufficient neural systems with functional imaging studies of neuropsycholog-
ical patients. Journal of Cognitive Neuroscience, 11(4), 371–382.
Pulvermüller, F., Shtyrov, Y., & Ilmoniemi, R. (2005). Brain signatures of meaning access in ac-
tion word recognition. Journal of Cognitive Neuroscience, 17(6), 884–892.
Rapp, B., & Goldrick, M. (2000). Discreteness and interactivity in spoken word production.
Rapp, B., & Goldrick, M. (2006). Speaking words: Contributions of cognitive neuropsycholog-
ical research. Cognitive Neuropsychology, 23,  39–73.
Robson, H., Keidel, J., Lambon Ralph, M. A., & Sage, K. (2012). Revealing and quantifying
the impaired phonological analysis underpinning impaired comprehension in Wernicke’s
aphasia. Neuropsychologia, 50, 276–288.
Rorden, C., & Karnath, H. O. (2004). Using human brain lesions to infer function: A relic from
a past era in the fMRI age? Nature Reviews Neuroscience, 5(10), 813–819.
Rumelhart, D., & McClelland, J. (1986). PDP models and general issues in cognitive science.
In D. Rumelhart & J. McClelland (Eds.), Parallel distributed processing: Explorations in the
microstructure of cognition (Vol. 1, pp. 110–146). Foundations. Cambridge, MA: MIT Press.
Schwartz, M. F. (1984). What the classical aphasia categories can’t do for us and why. Brain and
Language, 21,  3–8.
Scott, S. K., & Wise, R. J. S. (2004). The functional neuroanatomy of prelexical processing in
speech perception. Cognition, 92,  13–45.
Stevens, K. N., & Blumstein, S. E. (1981). The search for invariant acoustic correlates of phonetic
features. In P. D. Eimas & J. L. Miller (Eds.), Perspectives on the study of speech (pp. 1–38).
New York: Lawrence Erlbaum.
Swinney, D., Zurif, E. B., & Nicol, J. (1989). The effects of focal brain damage on sentence pro-
cessing: An examination of the neurological organization of a mental module. Journal of
Cognitive Neuroscience, 1,  25–37.
Thompson-Schill, S. L., D’Esposito, M., Aguirre, G. K., & Farah, M. J. (1997). Role of left infe-
rior prefrontal cortex in retrieval of semantic knowledge: A reevaluation. Proceedings of the
National Academy of Sciences, 94, 14792–14797.
Utman, J. A., Blumstein, S. E., & Sullivan, K. (2001). Mapping from sound to meaning: Reduced
lexical activation in Broca’s aphasics. Brain and Language, 79, 444–472
Vigneau, M., Beaucousin, V., Herve, P. Y., Duffau, H., Crivello, F., Houde, O., Mazoyer, B. &
Tzourio-Mazoyer, N. (2006). Meta-analyzing left hemisphere language areas: Phonology,
semantics, and sentence processing. NeuroImage, 30(4), 1414–1432.
Walker, G. M., Schwartz, M. F., Kimberg, D. Y., Faseyitan, O., Brecher, A., Dell, G. S., & Coslett,
H. B. (2011). Support for anterior temporal involvement in semantic error production in
aphasia: New evidence from VLSM. Brain and Language, 117, 110–122.
Wennekers, T., Garagnani, M., & Pulvermüller, F. (2006). Language models based on Hebbian
cell assemblies. Journal of Physiology-Paris, 100(1),  16–30.
Wilson, S. M., Saygin, A. P., Sereno, M. I., & Iacoboni, M. (2004). Listening to speech activates
motor areas involved in speech production. Nature Neuroscience, 7(7), 701–702.
Yee, E., Blumstein, S. E., & Sedivy, J. C. (2008). Lexical-semantic activation in Broca’s and
Wernicke’s aphasia: Evidence from eye movements. Journal of Cognitive Neuroscience, 20,
592–612.
Zurif, E. B., & Caramazza, A. (1976). Dissociation of algorithmic and heuristic processes in lan-
guage comprehension: Evidence from aphasia. Brain and Language, 3, 572–582.
Zurif, E. B., Caramazza, A., & Myerson, R. (1972). Grammatical judgments of agrammatic
aphasics. Neuropsychologia, 10, 405–417.
Pa rt   I
T H E M E T HOD S
Chapter 2
Neurolinguisti c St u di e s
of Patients w i t h
Ac quired Aphasias
Stephen M. Wilson
Introduction
Our earliest insights into the neural underpinnings of language came from studies of
patients with acquired aphasia, that is, deficits in producing and/or comprehending
language, due to brain damage. Descriptions of aphasia can be found in the medical
literature dating back to about 400 b.c.e. (Benton & Joynt, 1960), and there is even a
possible reference to aphasia with right hemiplegia in the Bible, dating back to a similar
time: “If I forget you, Jerusalem, may my right hand forget its skill. May my tongue cling
to the roof of my mouth if I do not remember you” (Psalm 137:5–6, New International
Version). The association of aphasia with a motor deficit in the right hand could con-
ceivably reflect an understanding that language is localized to the left hemisphere
(Benton, 1971). In the early nineteenth century, physicians of the phrenological school,
including Gall, Spurzheim, and Bouillaud, postulated that language was a function of
the frontal lobes, based in part on observations of patients with frontal lobe damage and
deficits in speaking (Bouillaud, 1825; Gall, 1825; Spurzheim, 1815). In 1836, Marc Dax, a
French neurologist, even prepared a paper in which he observed an association between
aphasia and damage to the left hemisphere (Dax, 1836/1865); however, there is no evi-
dence that the paper was actually presented at the conference for which it was prepared
(Levelt, 2013).
The modern field of neurolinguistics began in earnest with a series of papers by
Paul Broca in the first half of the 1860s. Broca’s first paper was a case report of a pa-
tient who could not produce spoken language, and who came to autopsy just a few days
after Broca examined him (Broca, 1861). Broca found that although the patient’s brain
20 Stephen M. Wilson
was damaged quite extensively, the epicenter of the damage appeared to be the infe-
rior frontal gyrus of the left frontal lobe, leading him to hypothesize that this brain re-
gion is responsible for our ability to speak aloud. Broca’s claim generated great interest,
and before long, numerous similar cases had been described. A few years after his first
case, Broca observed the striking fact that virtually all reported cases of aphasia in-
volved damage to the left hemisphere, almost all involved concomitant paralysis of the
right hand, and aphasia rarely occurred after damage to the right hemisphere. From
these observations, he concluded that language must be localized in the left hemisphere
(Broca, 1865).
Ten years later, Carl Wernicke, a young German physician, reported two patients with
damage to the posterior superior temporal lobe and deficits in language comprehension
(Wernicke, 1874), suggesting for the first time that there was not a single language area,
but multiple language areas with distinct functions. Wernicke developed an extraor-
dinarily prescient theory of the neural organization of language. Ludwig Lichtheim,
a German neurologist, refined and expanded on Wernicke’s model (Lichtheim, 1885),
laying a firm foundation for the field of neurolinguistics as we know it today (Caplan,
1987; Levelt, 2013).
Three main approaches to patient studies can be identified in the literature. In the
first, which we will call the cognitive neuropsychological approach, researchers under-
take comprehensive investigations of single patients in order to document patterns of
impaired and spared functions in one or more components of the language system,
and make inferences about the normal functional architecture of the language system.
In the second approach, which we call the syndromic approach, researchers study
groups of patients defined by sharing a clinical syndrome (e.g., Broca’s aphasia) or
a clinical feature (e.g., agrammatism), in order to characterize the precise nature of
linguistic deficits, co-occurrence of symptoms, patterns of recovery, and so on. The
third approach we will call the lesion-deficit correlation approach. Here, researchers
investigate the relationships between damage to different brain regions and different
kinds of language deficits, so as to uncover the functional neuroanatomy of the lan-
guage system. Some studies blend aspects of more than one of these three approaches.
All three approaches assume that language functions as a modular system, the
components of which can be damaged independently of one another. The rich body of
literature that has followed from this fundamental assumption is evidence of its basic
soundness. The lesion-deficit approach further assumes that the functionally modular
components of the language system are to some extent physically localized in spe-
cific brain regions or networks, usually but not always taken to be relatively consistent
across individuals.
We will discuss these three approaches by focusing on representative studies that ex-
emplify their strengths and weaknesses. But first, we need to consider the individuals
with acquired aphasias from whom we hope to learn something about the neural basis
of language. Who are they? What kinds of brain damage lead to acquired aphasia? Does
it matter how the damage came about?
Neurolinguistic Studies of Patients with Acquired Aphasias 21
Patients with Acquired Aphasias
There are numerous ways that language regions of the brain can become damaged, such
as stroke, neurodegenerative disease, neurosurgical resections, or trauma. In general,
damage to brain regions involved in language processing can be expected to result in
similar deficits regardless of how the damage came about. However, there are impor-
tant differences between patients with acquired aphasias due to different etiologies,
such as typical distributions of damage, the time course of damage and/or recovery, and
whether damage is complete or gradated.
The single most common cause of acquired aphasia is stroke. There are two reasons
for this. First, strokes are very common. Approximately 800,000 people in the United
States experience a stroke each year; rates are similar in other economically developed
countries, and even higher in less economically developed countries. Second, the inci-
dence of aphasia is very high after stroke due to the anatomy of the vasculature, which
dictates which brain regions are typically damaged. The most common type of stroke
is ischemic stroke, which occurs when arteries supplying the brain are blocked by
blood clots. The reduction or cessation of blood flow causes infarction (cell death) in
the region(s) supplied by the artery. A large artery named the middle cerebral artery
runs through the Sylvian fissure, giving off branches that supply the majority of the lat-
eral cerebral hemispheres. Key language regions, including Broca’s area and Wernicke’s
area, are located in the immediate vicinity of the Sylvian fissure, so blockages of different
branches of the middle cerebral artery can readily cause damage that is localized to spe-
cific language regions. Moreover, blockage in the trunk of the middle cerebral artery can
cause damage to multiple language regions.
While ischemic strokes in the left middle cerebral artery almost invariably cause
aphasia, other types of strokes can also lead to aphasia. Two other arteries supply the
remainder of the cerebral hemispheres: the posterior cerebral artery and the anterior
cerebral artery. Ischemic strokes in either of these arteries, but especially the posterior
cerebral artery, often result in aphasia. Brain regions at the peripheries of vascular terri-
tories are susceptible to global reductions in blood flow, usually from a blockage in the
neck. These so-called watershed strokes often affect frontal or parietal regions and often
result in aphasia. Finally, in addition to ischemic strokes, which make up about 85% of
strokes, there are also hemorrhagic strokes, which involve leakage or rupture of a blood
vessel in the brain, causing cell damage or cell death. Hemorrhagic strokes have a dif-
ferent distribution of affected areas than ischemic strokes, but common sites include the
basal ganglia, the thalamus, and the cerebral hemispheres, damage to all of which can
potentially lead to aphasia. While many of the regions that are often impacted by stroke
appear to be involved in language function, the regions damaged by stroke are by no
means evenly distributed: regions adjacent to the middle cerebral artery, such as the in-
sula, are very frequently damaged, whereas other regions such as the anterior temporal
lobes are rarely damaged, and practically never damaged in isolation.
The onset of language deficits after a stroke is usually sudden, or at least rapid (over
the course of hours). However, language deficits are by no means static. Rather, almost
all patients with aphasia experience some degree of recovery of language function after
a stroke. The greatest gains are often made in the first day or two, and largely reflect the
resolution of blood flow deficits in penumbral regions beyond the irreversibly damaged
region (Hillis et al., 2002). After this initial period, much less is known regarding the
mechanisms underlying recovery, which may include resolution of edema, continued
perfusion changes, and neuroplasticity (Heiss, Karbe, et al., 1997; Heiss, Kessler, Thiel,
Ghaemi, & Karbe, 1999; Saur et al., 2006; Saur & Hartwigsen, 2012). While the greatest
gains take place within the first three months (Kertesz & McCabe, 1977; Pedersen,
Jørgensen, Nakayama, Raaschou, & Olsen, 1995), many patients continue to improve
substantially after that time period (Naeser et al., 1998; Smania et al., 2010; Swinburn,
Porter, & Howard, 2004).
In sum, patients who survive strokes provide a rich source of data on the neural basis
of language, due to the high incidence of stroke, the high incidence of aphasia after
stroke, and the anatomy of the middle cerebral artery with respect to key language re-
gions. Major challenges of studying stroke patients include the uneven distribution of
lesion locations, which is dictated by the vascular anatomy, and the substantial recovery
that typically takes place relatively rapidly, such that most studies are carried out after
significant functional reorganization has probably already taken place.
A second relatively common cause of aphasia is neurodegenerative disease. There are
many different neurodegenerative diseases, which impact different brain regions, and
progress in different ways. When language regions of the brain are affected before other
regions, then language deficits will be the first symptoms of the disease. This is termed
primary progressive aphasia (PPA) (Mesulam, 1982, 2001). Mesulam (2001) defines PPA
as a slow, insidious onset and gradual progression of speech and/or language deficits,
with no other significant deficits (e.g., memory problems, apathy, disinhibition) for at
least two years, and with language remaining the most impaired function when other
deficits do emerge.
Language deficits can be remarkably focal early in the course of the disease. Recent
consensus guidelines for the diagnosis of PPA recognize three variants: nonfluent/
agrammatic PPA; semantic PPA (also known as semantic dementia); and logopenic
PPA (Gorno-Tempini et al., 2011). The three variants have different underlying patho-
logical causes (Grossman, 2010; Snowden et al., 2011), different distributions of atrophy
(Gorno-Tempini et al., 2004), and following from that, differential impacts on various
speech and language domains (Wilson, Henry, et al., 2010).
There are three important differences in the nature of brain damage between PPA and
stroke. First, whereas strokes typically completely destroy any affected brain regions, in
PPA the damage is graded, not absolute. Atrophy begins gradually and progresses over
time. Affected regions still contain neural tissue, and the extent to which it retains its
functionality is an empirical question, which can be addressed with methodologies such
as functional magnetic resonance imaging (fMRI) (Wilson, Dronkers, et al., 2010).
Second, neurodegenerative diseases tend to impact functional networks of regions,

which may or may not be spatially contiguous (Seeley, Crawford, Zhou, Miller, &
Greicius, 2009). For instance, semantic dementia impacts anterior and ventral tem-
poral regions bilaterally, even though the left and right temporal regions that are af-
fected in each hemisphere are not adjacent. Degeneration of functionally connected
networks may result in more discrete linguistic deficits than damage to regions that
happen to be spatially adjacent, but which may or may not have anything in common
functionally.
Third, the regions that are impacted in PPA are not the same as those typically dam-
aged in stroke. Most important, the anterior temporal lobes are rarely damaged in iso-
lation in stroke, due to vascular anatomy (Holland & Lambon Ralph, 2010), whereas
they are damaged in quite a focal manner in semantic dementia (Hodges, Patterson,
Oxbury, & Funnell, 1992). Another example is that it is relatively common in PPA to find
patients with significantly disproportionate damage to dorsal or ventral white matter
tracts connecting frontal and temporal language areas (Wilson, Galantucci, et al., 2011).
This contrasts with the situation in stroke, in which middle cerebrary artery strokes are
more likely to damage either both tracts or neither tract, making their functions hard to
dissociate (Griffiths, Marslen-Wilson, Stamatakis, & Tyler, 2013).
In sum, then, some of the advantages to the study of PPA are that language deficits
can be remarkably isolated early in the course of disease, that damage tends to follow
functionally meaningful networks, and that a wide range of regions can be damaged
that are not constrained by vascular anatomy. Some of the major challenges include
determining the functional status of atrophic tissue, and accounting for functional re-
organization that takes place as core language regions degenerate (Wilson, Brambati,
et al., 2009).
There are many other etiologies that can result in acquired aphasia, which we will not
discuss in detail here. Neurosurgical resection of tumors or epileptogenic foci results
in small discrete lesions and focal language deficits that quickly resolve (Penfield &
Roberts, 1959; Wilson et al., 2015). However, neurosurgery is, of course, only performed
in patients with intractable neurological problems, which may result in reorganiza-
tion of language prior to surgery (Haglund, Berger, Shamseldin, Lettich, & Ojemann,
1994). This complicates the interpretation of conclusions drawn from these patients.
Gunshot wounds can be focal, and obviously the regions damaged are not constrained
by vascular anatomy. Pioneers of aphasiology, including Goldstein, Kleist, and Luria,
based their work substantially on war-related injuries. However, the changing nature of
modern warfare has made gunshot wounds relatively rare.
There are also many clinical syndromes in which language deficits co-occur along
with significant other symptoms in other cognitive or behavioral domains. Examples in-
clude Alzheimer’s disease, Parkinson’s disease, traumatic brain injury, and autism spec-
trum disorder. While studies of each of these syndromes have yielded insights into the
neural basis of language, the concomitant occurrence of other deficits makes it chal-
lenging to infer brain-language relationships.
The Cognitive Neuropsychological

Approach
One of the most fruitful approaches to the study of language in patients with acquired
aphasia is to investigate language function in individual patients in great detail, de-
termining which functions are impaired and which are spared. We can then make
inferences from the pattern of impaired and spared functions to the normal functional
architecture of the language system. We refer to this as the cognitive neuropsychological
approach.
The brain itself need not be a central focus in these kinds of studies. Descriptions
of the nature and location of damage are typically included, and possibly computer-
ized tomography (CT) or magnetic resonance imaging (MRI) findings, but these are
not usually of central importance. Rather, the critical idea of this approach is that cer-
tain patterns of deficits have implications for the functional architecture of the normal
language system. The logic is that if the language system can possibly break down in a
certain manner (as evidenced by the patient under investigation), then it must be struc-
tured in such and such a way. We will illustrate this approach with two examples.
Shapiro, Shelton, and Caramazza (2000) reported a patient named JR who had a se-
lective difficulty in inflecting nouns, but was relatively spared in inflecting verbs. JR had
other related deficits, too, but we focus here on the morphological deficit. To document
the nature of JR’s deficit, the authors designed a set of experiments specifically for this
patient. The tasks involved adding and removing inflectional suffixes from nouns and
verbs; we focus here only on the tasks that involved adding suffixes.
In order to carefully contrast nominal (noun-related) and verbal (verb-related)
morphology, the authors took advantage of two facts about English. First, English has
large numbers of words that can function either as nouns or verbs (e.g., guide and sail).
Second, the plural suffix, which applies to nouns, and the third person singular present
tense agreement suffix, which applies to verbs, have identical phonological forms, iden-
tical allomorphs ([-s], [-z], [-əz]), and identical morphophonemic rules governing al-
lomorph selection. Therefore, any differences observed between performance on noun
and verbs could not be attributed to phonological factors or morphology in general.
JR was presented with frames such as the following:
“This is a guide; these are ___” (guides)

“These people sail; this person ___” (sails). (Shapiro, Shelton, & Caramazza,
2000, p. 674)
Similar frames were also presented using made-up pseudowords instead of real nouns
and verbs, in order to avoid any semantic differences between nouns and verbs.
There were striking differences in JR’s ability to inflect nouns and verbs. With real
words, he inflected 94% of verbs correctly, but only 73% of nouns. With pseudowords,
he inflected 87% of verbs correctly, but only 38% of nouns. By carefully constructing an
experiment to probe a specific aspect of JR’s language function, the authors were able to
demonstrate a single dissociation: inflection of nouns can be impaired while inflection of
verbs is spared, suggesting that these processes are independent of one another in some
respect.
In a follow-up study, the authors built upon this finding, reporting on another pa-
tient named RC (Shapiro & Caramazza, 2003). RC was tested with a very similar set of
materials to JR, yet he showed a different pattern. With real words, he inflected 76% of
nouns correctly, but only 22% of verbs. With pseudowords, he inflected 50% of nouns
correctly, but only 21% of verbs. In other words, RC showed the opposite pattern to
JR: he showed a selective deficit in inflecting verbs, but was relatively spared in his ability
to inflect nouns.
Taken together, these two patients form a double dissociation: JR could inflect verbs
but not nouns, whereas RC could inflect nouns but not verbs. This is often considered
to be more informative than a single dissociation, since it shows that neither process
is contained within the other. The authors identify two possible explanations for their
results. First, there could be separate morphological processing components for nouns
and verbs. Second, there could be separate mechanisms for retrieving syntactic infor-
mation related to nouns and verbs, which feed into a unitary morphological processing
component. In either case, the detailed case studies of the two patients reveal some-
thing about the cognitive architecture of the language system. The fact that inflectional
morphology can break down for nouns but not verbs, or vice versa, shows that mor-
phological processing related to these major word classes is at least partially segregated
(for further discussion of grammatical categories and agrammatism, see Kemmerer,
Chapter 30, and Thompson & Mack, Chapter 31, in this volume).
Our second example of the cognitive neuropsychological approach comes from the
domain of single word reading. It is well established that reading involves at least two
mechanisms: a sublexical mechanism based on regular orthographic-to-phonological
mappings, and a whole-word mechanism based on retrieval of (potentially idiosyn-
cratic) item-specific information about the pronunciation of particular words (Marshall
& Newcombe, 1973). The former mechanism is required for reading pseudowords
(e.g., hance), the latter is required for reading exception words (e.g., choir), and double
dissociations have been robustly documented, showing that the two processes are sep-
arable. A question that continues to be debated is the extent to which the whole-word
mechanism is tied to semantics. On one view, the whole-word mechanism is always se-
mantically mediated: a word like choir is read by mapping the orthographic form C-
H-O-I-R onto the semantic representation of “a group of singers,” and then mapping
that semantic representation onto its phonological form [kwaɪə˞] (Plaut, McClelland,
Seidenberg, & Patterson, 1996). This view predicts that semantic deficits and deficits in
exception word reading should be highly correlated, and indeed this is the case. In par-
ticular, patients with semantic dementia, characterized by semantic impairment, almost
always present with surface dyslexia, a disorder in which exception words are sounded
out (Patterson & Hodges, 1992), and the degree of semantic impairment is correlated
with the severity of surface dyslexia (Graham, Hodges, & Patterson, 1994; Woollams,
Ralph, Plaut, & Patterson, 2007). An alternative view is that while semantic mediation
is always possible, there are also direct links between orthographic representations
(i.e., C-H-O-I-R) and phonological representations (i.e., [kwaɪə˞]) that do not require
semantic mediation (Coltheart, Rastle, Perry, Langdon, & Ziegler, 2001). We will dis-
cuss a paper that provides evidence for this second view.
Blazely, Coltheart, and Casey (2005) investigated two patients with semantic de-
mentia, PC and EM, on a battery of semantic and orthographic tasks. Both patients
showed similar deficits in semantic processing: they were similarly impaired in picture
naming, and spoken and written word-to-picture matching tasks, with disproportionate
deficits on lower frequency items, as is typical in semantic dementia.
However, the two patients differed strikingly in their performance on the ortho-
graphic tasks. PC showed a pattern of performance typical of semantic dementia: he
made numerous errors when reading exception words, especially lower frequency
items, yet he read regular words well, regardless of frequency. On a visual lexical de-
cision task, he also performed poorly, disproportionately so for low-frequency items.
In contrast, EM performed nearly flawlessly on reading aloud both regular and ex-
ception words, and nearly flawlessly on visual lexical decision. Her pattern of perfor-
mance is difficult to reconcile with the view that exception word reading is necessarily
mediated by semantics, since she showed a similar if not greater semantic impairment
than PC, yet her exception word reading was spared. Blazely and colleagues argue that
EM demonstrates that there must be a lexical non-semantic route for reading aloud
that can be spared even when semantics is impaired. In other words, a single case is
sufficient to demonstrate something important about the functional architecture of the
reading system (though actually, several other similar patients have been reported; see
Blazely et al.. 2005, for references; see also Paz-Alonso, Oliver, Quiñones, & Carreiras,
Chapter 24 in this volume).
How then is the typical association of semantic deficits with surface dyslexia to be
explained? Blazely et al. (2005) argue that the association is not due to the functional
dependence of the two systems, but rather is due to “brain geography”: inferior tem-
poral brain regions are thought to be important for semantic processing, and left
posterior middle temporal and/or occipito-temporal regions are thought to be im-
portant for the orthographic lexicon (Noble, Glosser, & Grossman, 2000). These brain
regions are in close proximity and therefore are typically impacted in parallel by the
neurodegeneration that underlies semantic dementia.
The two examples we have discussed show how dissociations of language skills in in-
dividual patients can provide compelling evidence regarding the functional architecture
of the language system. While the studies we discussed do describe the brain regions
damaged in the patients investigated, the primary concern is the functional architecture
of the language system itself, rather than its neuroanatomy. This is the primary limita-
tion of the cognitive neuropsychological approach. While a carefully documented pat-
tern of performance in a single patient can be sufficient to draw inferences about the
necessary architecture of the normal system, the same logic cannot be used to interpret
the effect of a lesion in any single patient, due to an unknown degree of individual varia-
bility in localization of language function in the brain.
The Syndromic Approach
A second major approach to patient-based studies of the neural basis of language is to

define one or more groups of patients, and then to investigate some aspect of language,
co-occurrence of symptoms, patterns of recovery, or such, across that group. We refer to
this as the syndromic approach. There are many different ways that patient groups can be
defined, with some of the most common being clinical syndromes (e.g., Broca’s aphasia,
semantic dementia), clinical features (e.g., agrammatism, apraxia of speech), or under-
lying pathological causes in PPA patients (e.g., patients who show a specific type of cel-
lular pathology at autopsy). Like the approach discussed in the previous section, the
syndromic approach does not necessarily address neural correlates of language deficits
directly. To the extent that a clinical syndrome reflects damage to a particular brain re-
gion or network (e.g., Broca’s area for Broca’s aphasia, the anterior temporal lobes for
semantic dementia), investigating patterns of performance in patients with that clin-
ical syndrome may shed light on the function of the brain region(s) in question. But
as we will discuss in the next section, the relationship between clinical syndromes and
damaged brain regions is often complicated. The most important contribution of the
syndromic approach is to gain a better understanding of the nature of deficits in clinical
populations, which ultimately informs studies that explicitly explore brain-language
relationships. We will discuss two studies that exemplify the syndromic approach.
Grodzinsky (1984) investigated the manifestation of agrammatism in languages with
different relationships between bound inflectional morphology and lexical categories
(noun, verb, adjective). This is an example of a study that shed light on the nature of
the clinical syndrome of agrammatism, without explicit consideration of its neural
correlates.
Grodzinsky noted that the then- current predominant understanding of
agrammatism was centered on the omission of closed-class function words and inflec-
tional morphology (e.g., Goodglass & Berko, 1960). This is a satisfactory basic descrip-
tion of agrammatism in English. However, in English, lexical items have uninflected
forms that are well formed both morphologically and phonologically (e.g., boy, walk).
This is not the case in all languages. In languages like Italian and Russian, uninflected
forms are typically well formed phonologically, but not morphologically. For instance,
the Italian word for “red” is rosso (masculine singular), rossa (feminine singular),
rossi (masculine plural), or rosse (feminine plural). The uninflected form appears to
be ross, and although this would be a phonotactically legal word in Italian, it is not a
morphologically well-formed word, and it would never be produced. In Hebrew and
other Semitic languages, uninflected forms are neither phonotactically nor morpho-
logically well formed. For instance, the word for “dress” is simla (singular) or smalot
(plural); the root consists of only the three consonants s-m-l, and so the root needs to
be inflected and cannot be produced alone without violating phonotactic constraints.
These observations lead to the following question: What would agrammatism look like
in these languages, where omission of bound inflectional morphemes would result in
morphologically impossible (Italian, Russian), or phonotactically and morphologi-
cally impossible (Hebrew) forms?
Drawing on his own observations of agrammatic aphasia in Hebrew, and data from
Miceli, Mazzucchi, Menn, and Goodglass (1983) on Italian and Tsvjetkova and Glozman
(1978) on Russian, Grodzinsky found that agrammatic speakers of these languages do
not, in fact, produce phonologically or even morphologically impossible forms, as one
might expect if agrammatism simply involved the omission of inflectional morphology.
Rather, in languages where uninflected forms are not well formed, agrammatic speakers
substitute incorrect bound morphemes, rather than omitting them. For instance, a
Hebrew-speaking patient produced nasʔu baʔali (“drove-3pl my husband”), in which
the verb root n-s-ʔ (“to drive”) is incorrectly inflected for agreement, rather than being
produced in its phonotactically impossible root form.
While Grodzinsky’s study was based on an extremely limited data set, subsequent sys-
tematic studies confirmed that bound inflectional morphemes are substituted, rather
than omitted, in richly inflected languages (Bates, Friederici, & Wulfeck, 1987). This line
of research is an excellent example of how careful, linguistically informed investigation
of language processing in a clinically defined group of patients can shed light on the
nature of the clinical syndrome shared by the patients. While these studies do not di-
rectly address the neural correlates of agrammatism, they provide crucial information
that bears on the interpretation of other studies that do investigate the neural correlates
of agrammatism (e.g., Mohr, 1976, discussed in the next section; see also Thompson &
Mack, Chapter 31 in this volume).
A second example of the syndromic approach is a study by Bozeat et al. (Bozeat,
Lambon Ralph, Patterson, Garrard, & Hodges, 2000) on nonverbal semantic
impairments in semantic dementia. At the time this study was published, the most sa-
lient features of semantic dementia were understood to be anomia and verbal compre-
hension deficits at the single-word level. The researchers suspected that individuals with
semantic dementia have conceptual semantic impairments that go beyond language,
but the only clear evidence to that effect came from case studies of single patients.
Bozeat and colleagues tested 10 patients with semantic dementia on several semantic
tasks that did not involve language in any way. The first was the pictures subtest of the
Pyramids and Palm Trees test, in which patients are required to decide which of two
pictures (e.g., a palm tree and a pine tree) goes with a target picture (e.g., a pyramid)
(Howard & Patterson, 1992). Successful performance requires access to conceptual se-
mantic knowledge (i.e., palm trees and pyramids are both associated with deserts). Two
new semantic assessments were also designed for the study: a four-alternative forced-
choice semantic association task similar to the Pyramids and Palm Trees test, and a test
in which patients were asked to match environmental sounds to their corresponding
pictures. Finally, patients’ language was assessed with a battery of language tasks.
The authors found that most patients were impaired on all three of the nonverbal
semantic tasks, and that the degree of semantic impairment on verbal and nonverbal
material was correlated. They concluded that the anomia and word comprehension
deficits that are so salient in semantic dementia are in fact just one manifestation of
a generalized impairment of conceptual semantic knowledge. The study includes no
neuroimaging or discussion of the brain regions that were damaged in the 10 patients
studied, but given the well-established finding that semantic dementia follows from
atrophy of anterior temporal and ventral temporal regions (e.g., Hodges et al., 1992), it
is reasonable to infer that these regions (or some subset of these regions) are critical for
conceptual semantic memory independent of any role in language per se.
The studies we have discussed by Grodzinsky (1984) and Bozeat et al. (2000) are
excellent examples of the syndromic approach to studies of patients with acquired
aphasia. They each reveal new information about the nature of a clinical syndrome.
Studies like these are most effective when the patient group under investigation is
a “natural kind” (i.e., a grouping reflecting the structure of the world and the phe-
nomena in question), rather than an artificial grouping. Semantic dementia, the de-
fining clinical syndrome in the study by Bozeat et al. (2000), is clearly a “natural kind,”
with characteristic clinical features, neuropsychological findings, specific patterns
of brain atrophy, and, in most cases, a common pathological substrate (Hodges &
Patterson, 2007). In contrast, it is less clear that agrammatism is a “natural kind”; in-
deed, Badecker and Caramazza (1985) argued forcefully that it is not, because few if
any patients produce “canonical” agrammatic utterances, there is considerably indi-
vidual variability, and there is no way to define agrammatism in any way that it is
not arbitrary. These issues have been debated vigorously (see Caramazza & Badecker,
1991), but the limitations emphasized by Caramazza, Badecker, and others have not
put a stop to syndromic studies, which continue to be a productive line of neurolin-
guistic research.
The Lesion-Deficit Approach
The two approaches we have discussed so far are focused more on the characterization
of language behavior than directly on brain-language relationships. We turn now to
studies in which the neural correlates of language deficits are investigated explicitly. In
the lesion-deficit approach, brain damage is characterized through neuroimaging, au-
topsy, or other methods, and language deficits are characterized through the kinds of
tasks and analyses we have discussed already. In this approach, researchers attempt to
infer the normal functions of particular brain regions from the patterns of deficits that
result when they are damaged.
Single cases can be informative with respect to brain-language relationships, but this
is the exception rather than the rule. Broca’s first and most famous case, “Tan,” may have
had a relatively small lesion and an isolated speech production deficit at some point in
the course of his neurological history, but by the time Broca examined him, he had been
mute for over 20 years, and there had been very substantial progression of not only his
brain damage, but also his symptoms (Broca, 1861). When Broca examined the brain at
autopsy, he found that besides the damage to the inferior frontal gyrus, there was also
significant damage to the middle frontal gyrus, precentral gyrus, anterior parietal re-
gions, superior temporal gyrus, insula, and underlying white matter. “Tan” was not just
unable to speak; he also had motor symptoms, including paralysis of the right hand and
right leg. Moreover, substantial comprehension and cognitive deficits were evident,
even based on Broca’s rather cursory examination. Broca’s inference that the inferior
frontal gyrus specifically was responsible for an isolated speech production deficit was
largely speculative and was not supported by any such specific relationship in the patient
as Broca observed him. And indeed, this finding has not stood the test of time, as we will
discuss shortly.
In contrast, some single cases have yielded novel and enduring findings. A good ex-
ample is Liepmann’s (1898) description of a patient with pure word deafness (i.e., au-
ditory agnosia for words) due to a unilateral lesion (see also Geschwind, 1965). Pure
word deafness is a clinical syndrome in which words cannot be understood, despite
hearing being otherwise intact. Liepmann showed that the patient’s lesion, which was
located subcortically in the left temporal lobe, had destroyed fibers from both left and
right auditory cortices, preventing auditory input from reaching the posterior temporal
language region (i.e., Wernicke’s area). Patients with pure word deafness have relatively
normal speech, in contrast to the jargon aphasia that results when Wernicke’s area itself
is damaged. Liepmann’s single case study suggests then that (1) associations between au-
ditory word forms and meanings are carried out only in the left hemisphere; and (2) au-
ditory input from primary auditory cortex in either hemisphere is sufficient to associate
word forms with their meanings (because pure word deafness arises only when auditory
input from not only the left but also the right hemisphere is unavailable, due either to a
subcortical lesion or to bilateral lesions). Liepmann’s case provides an example of how
informative a single patient can be with respect to brain-behavior relationships. What
makes this possible is that the language deficit is circumscribed and well characterized,
the lesion is small, and the effects of the lesion are understood in terms of the anatomical
regions and functional pathways involved. It is rare for all of these criteria to be met in a
study of a single case (see also Poeppel, Cogan, Davidesco, & Flinker, Chapter 26 in this
volume).
More often, patients with acquired aphasia have larger lesions or degenerative
processes that affect multiple brain regions and pathways. Therefore, the majority of
studies of brain-language relationships have investigated lesion locations in groups of
patients sharing common clinical or linguistic features. The most straightforward ap-
proach to a lesion-deficit study is to create an overlay of the lesions of patients with
a common clinical syndrome or a common pattern of performance on some task, in
order to determine which brain region(s) are damaged in all (or at least most) of the
patients. The commonly destroyed region can then be associated with the syndrome
or linguistic behavior in question, and it can potentially be inferred that the region in
question is critical for the language process(es) that is (are) impaired in the syndrome
under investigation.
These studies began to appear with the advent of CT imaging in the 1970s and then
structural MRI in the 1980s. Some of the early studies looked at lesion overlays of
classical aphasic syndromes. For example, Naeser and Hayward (1978) created lesion
overlays of patients diagnosed with Broca’s aphasia, Wernicke’s aphasia, conduction
aphasia, transcortical motor aphasia, and global aphasia. There were three to four cases
of each type. The lesion patterns were broadly in accordance with expectations based
on the classical model (Wernicke, 1874; Lichtheim, 1885), but the lesions were typically
large in extent. Basso, Lecours, Moraschini, and Vanier (1985) looked at CT scans of 267
patients with left hemisphere lesions, of whom 207 had cortical lesions. Of these 207
patients, they found that 171 had aphasia syndromes that were in accord with expec-
tations based on classical theory, whereas 36 patients did not: they had aphasia despite
damage that spared language regions, or no aphasia despite damage to language regions,
or a different type of aphasia than would be expected from the location of the damage. It
is encouraging how many patients had language disorders that were consistent with the
classical model, yet the 36 patients with unexpected findings raise challenging questions
about individual variability in language localization, cortical plasticity, possible func-
tional deficits in addition to structural deficits, and so on, that remain the subject of
active research.
Probably the most important study from this period is Mohr’s (1976) seminal paper
entitled “Broca’s Area and Broca’s Aphasia,” which comprises two complementary le-
sion overlay studies. Mohr’s clinical experience as a stroke neurologist had led him to
question the widely accepted relationship between damage to Broca’s area (which he
defined quite narrowly as the pars opercularis of the left inferior frontal gyrus), and the
clinical syndrome of Broca’s aphasia, defined in the way the term is generally used in the
literature: by halting, effortful speech characterized by agrammatism, short words and
phrases, disordered motor planning and/or execution of speech sounds, stereotypies,
impaired written language, and relatively spared comprehension except for more syn-
tactically complex constructions. Mohr took a simple and elegant approach to showing
that the assumed relationship between damage to Broca’s area and Broca’s aphasia, does
not hold. First, he documented the clinical syndromes resulting from lesions restricted
to Broca’s area. Then, he overlaid the lesions of patients diagnosed with the clinical syn-
drome of Broca’s aphasia. If the traditional view were correct, these two approaches
should have involved the same patients and the same lesion locations, but this did not
turn out to be the case.
Mohr examined 12 patients who had lesions that were relatively restricted to Broca’s
area (i.e., the posterior part of the left inferior frontal gyrus), as shown by neuroim-
aging or autopsy. The patients varied considerably in their clinical deficits immedi-
ately after stroke, ranging from barely detectable disturbances of speech to complete
mutism. But all recovered quickly, with most passing for normal within a month or two,
and agrammatism was rarely seen even acutely. Not one of these patients experienced a
persisting Broca’s aphasia.
Then, Mohr identified 10 patients who satisfied the criteria for the clinical syndrome
of Broca’s aphasia some months after their stroke. Using neuroimaging or autopsy, he
showed that in each patient, there was always infarction not just of Broca’s area, but of
nearly all of the territory of the upper division of the middle cerebral artery, that is, pos-
terior inferior frontal regions, the insula, the precentral and postcentral gyri, anterior
parietal regions, and underlying white matter. In other words, damage needed to extend
well beyond Broca’s area to cause the clinical syndrome of Broca’s aphasia.
This study is an excellent example of the power of the lesion overlay approach to doc-
ument the language deficits that result from lesions to a given brain area, and the brain
regions that must be damaged for a given pattern of deficits to result. It is worth noting
that this study was made possible by two things: the availability of CT scans, and Mohr’s
role on the stroke service of a large hospital. The two dozen patients discussed in the
paper were selected from hundreds, if not thousands, of stroke patients (see also Mohr
et al., 1978).
Another important lesion overlay study employed two groups of patients in a dif-
ferent way. Dronkers (1996) sought to identify the brain region(s) where damage was as-
sociated with the clinical syndrome of apraxia of speech, an impairment in planning and
coordinating speech movements (see, in this volume, Tremblay, Deschamps, & Dick,
Chapter 15; Ziegler, Schölderle, Aichert, & Staiger, Chapter 18). She identified one group
of 25 chronic stroke patients diagnosed with apraxia of speech, and a second group of 19
patients who were similar in some respects, but did not present with apraxia of speech.
She overlaid the lesions of the 25 patients with apraxia of speech, and found that all 25
lesions included a region in the left anterior insula (specifically, the superior part of the
precentral gyrus of the insula). She then overlaid the lesions of the 19 patients without
apraxia of speech, and found that none of their lesions involved that region of the insula.
Because none of the lesions in the control group without apraxia of speech overlapped
with the area of damage common to the 25 individuals with apraxia of speech, she
concluded that damage to the left anterior insula is necessary and sufficient to cause
apraxia of speech, and that this region is therefore essential for coordinating speech
articulation.
The inclusion of the control group without apraxia of speech is critical in this study,
especially since the anterior insula is adjacent to the middle cerebral artery, and is com-
monly damaged in stroke. However, Dronkers’s findings have been questioned by other
researchers, most notably Hillis et al. (2004), who studied 80 acute stroke patients
and found that apraxia of speech was more robustly associated with infarction or
hypoperfusion of Broca’s area than of the anterior insula. Hillis and colleagues argued
that it is important to study patients in the acute phase, because deficits due to small
lesions may resolve quickly. Bonilha and Fridriksson (2009) questioned Dronkers’s
findings from another point of view: they argued that the association reported by
Dronkers is not due to damage to the anterior insula, but rather is due to damage to
the immediately medially adjacent white matter fiber pathway connecting frontal and
temporal language areas. The role of the insula in speech production, and in the clinical
syndrome of apraxia of speech, continues to be debated. Our purpose here is simply
to examine and explain the logic of Dronkers’s study (in particular, the control group
without apraxia of speech) and the kinds of objections that may still arise.
Many language deficits are graded, not discrete. So while apraxia of speech is typi-
cally conceived as a clinical feature that is either present or absent, other deficits such
as impaired lexical access occur in essentially all patients with aphasia, but to different
degrees. In these cases, it is not possible to visually compare two groups of overlaid
lesions, such as in the Dronkers study, where one group presented with the symptom
and the other group did not present with the symptom. When the behavioral deficit is
present over a range of severity, a study can still utilize the lesion-deficit approach, but it
requires modern computerized procedures. Specifically, the related techniques of voxel-
based morphometry (VBM) (Ashburner & Friston, 2000), and voxel-based lesion-
symptom mapping (VLSM) (Bates et al., 2003) are used to investigate brain-language
relationships with continuous behavioral variables. Both approaches involve computing
relationships between damage and continuous behavioral variables at each voxel (three-
dimensional pixel) in a brain image. VBM is used for degenerative cohorts in which at-
rophy is graded, so at each voxel a correlation coefficient is calculated between degree of
atrophy and degree of behavioral impairment. VLSM applies to cohorts in which lesions
are modeled as discrete, such as stroke, so at each voxel a statistical comparison is made
between patients whose lesions do and do not include that voxel. The development of
VLSM and VBM has advanced the lesion-deficit approach in other ways, too, such as
enabling confounding variables (e.g., age, gender, time post-onset) to be factored out as
covariates when examining language-brain relationships. Bates et al. (2003) provided a
basic proof of principle, showing that reduced fluency and comprehension deficits were
associated with damage to distinct regions, specifically the anterior insula or the dorsal
white matter tract connecting anterior and posterior language areas for fluency, and the
posterior middle temporal gyrus for comprehension (Figure 2.1).
VBM and VLSM are capable of revealing distinct neural correlates, not only of dis-
tinct language processes such as fluency and comprehension, but also of closely related
parts of the same language process. For instance, Wilson, Henry, et al. (2010) showed in
a neurodegenerative cohort distinct brain regions where atrophy was correlated with
prevalence of phonemic paraphasias and with prevalence of distortions, which reflect
two different stages of speech production. In another study using VLSM, Schwartz et al.
(2009) showed that semantic errors in picture naming (e.g., horse for goat) are associated
with damage to the anterior and middle portions of the middle temporal gyrus. A par-
ticularly interesting feature of this study was that the authors included a covariate of per-
formance on nonverbal semantic tasks (the same two picture-association tasks that were
employed in the study by Bozeat and colleagues [2000] described earlier). By showing
that damage to the anterior to mid-middle temporal gyrus predicts semantic errors
above and beyond any effect it may have on conceptual semantic processing, Schwartz
and colleagues were able to argue that this brain region has a role in a specific stage of
lexical access, namely lemma retrieval, where an abstract pre-phonological word form
is retrieved. Several recent studies have shown that VLSM can also provide informative
analysis of measures of underlying language-processing constructs derived from factor
(a) (b) (c)

8.5
1.7
(d) (e) (f)

7.6
1.8
c, f
b, e
a, d
Figure 2.1. VLSM maps computed for fluency and auditory comprehension performance of
101 individuals with post-stroke aphasia. These maps are colorized depictions of t-test results
evaluating patient performance on a voxel-by-voxel basis. Patients with lesions in a given voxel
were compared to those without lesions to that voxel on measures of fluency (a, b, c) or auditory
comprehension (d, e, f). Lesions to voxels shown in hot colors (red, orange) had a highly signifi-
cant impact on fluency (top panels) or auditory comprehension (bottom panels).
Source: Reproduced from Bates et al. (2003).
analysis (Mirman et al., 2015), principal components analysis (Butler, Lambon Ralph, &
Woollams, 2014), or parameterization of computational models (Dell, Schwartz, Nozari,
Faseyitan, & Coslett, 2013).
In their basic form, VBM and VLSM are mass univariate approaches, meaning that the
statistical computation at each voxel does not take into account damage to the rest of the
brain. This is a significant limitation, since in reality there is always damage extending
beyond any single voxel under consideration, and the damage beyond any given single
voxel presumably also contributes to any behavioral deficits. Moreover, the patterns of
involvement of other voxels are nonrandom, since they reflect underlying factors re-
lated to the etiology, such as vascular anatomy in the case of stroke (Inoue, Madhyastha,
Rudrauf, Mehta, & Grabowski, 2014; Mah, Husain, Rees, & Nachev, 2014). Therefore,
regardless of how strongly damage to any voxel is associated with the behavioral def-
icit in question, it cannot be concluded that the voxel is really critical for the behavior.
To account for the contribution of voxels other than the one under consideration, it is
necessary to carry out multiple regression, in which damage to multiple brain regions
is entered into a predictive model. However, the number of voxels will always be much
larger than the number of patients, so only a subset of regions can be considered, typi-
cally motivated by a priori expectations. For instance, Wilson et al. (2015) showed in a
neurosurgical cohort that damage to the temporal pole does not contribute to naming
deficits once damage to the basal temporal language area is taken into account. Studies
in which larger numbers of regions have been entered into statistical models have gener-
ally not yielded clear results (e.g., Caplan et al., 2007), because the more regions that are
entered, the more patients are required to distinguish between the contributions of the
different regions.
Another approach to the mass univariate limitation is to think differently about how
to interpret the meaning of significant findings in VBM and VLSM analyses. Rather
than interpreting them as showing that any given voxel is involved or not involved in
a language process, we can instead think of these as methods for determining whether
there is a nonrandom relationship between the location of brain damage and a par-
ticular type of language deficit. Permutation testing, in which the available behavioral
data are scrambled among the patients under investigation and so randomly associated
with lesions, can be used to empirically generate the distribution of voxelwise lesion-
symptom associations under the null hypothesis that there is no relation between lo-
cation of damage and the language variable of interest (Kimberg, Coslett, & Schwartz,
2007; Wilson, Henry, et al., 2010). Then, the real data can be compared to this null
distribution to determine whether there are particular regions in which association
is stronger than would be expected based on chance. VBM or VLSM statistical maps
should therefore properly be interpreted not as showing whether any given voxel is crit-
ical for a language function, but as showing that lesion location is predictive of deficits.
Recently, several studies have moved beyond the mass univariate approach by
using machine- learning methods to investigate relationships between distributed
patterns of damage and aphasic syndromes or symptoms (Wilson, Ogar, et al., 2009;
Xing et al., 2016; Yourganov, Smith, Fridriksson, & Rorden, 2015; Zhang, Kimberg,
Coslett, Schwartz, & Wang, 2014). Another major focus of current research is on better
characterizing the nature of brain damage, as well as potential reorganization, in patients
with acquired aphasias. CT and structural MRI provide relatively clear delineation of
core lesions; however, it is becoming increasingly apparent that damage to any one re-
gion often has implications for brain regions that appear to be structurally intact. These
changes may include hypoperfusion (reduced blood flow and/or metabolic function)
(Hillis et al., 2002; Metter et al., 1989), degeneration of white matter tracts (Griffiths,
Marslen-Wilson, Stamatakis, & Tyler, 2013; Wilson, Galantucci, et al., 2011), diaschisis
(dysfunction of adjacent or functionally connected regions due to lack of normal inputs
from the damaged region) (Mummery et al., 1999), and other functional abnormalities
(Warren, Crinion, Lambon Ralph, & Wise, 2009). Better understanding the complex set
of changes that take place in the brains of individuals with acquired aphasia requires a
multimodal neuroimaging approach, using techniques such as structural and functional
MRI, perfusion MRI, diffusion tensor imaging, and positron emission tomography.
Conclusion
Patients with acquired aphasias due to stroke, neurodegenerative disease, neurosur-

gery, and other etiologies have yielded a great deal of information about the cogni-
tive and neural architecture of the language system. Cognitive neuropsychological
investigations of individual patients can lead to strong inferences about the functional
architecture of the language system. Studies of patients defined by a clinical syndrome
can also be informative in understanding the nature of the syndrome, to the extent
that the syndrome is a “natural kind.” Lesion-deficit studies examine the relationships
between damage to different brain regions and resulting language deficits.
It is always challenging to make inferences about the functional roles of spe-
cific brain regions. Only rarely is brain damage confined to a single brain region.
Similarly, only rarely do patients present with a deficit in a single aspect of language
processing. More often, multiple regions are damaged, and there are consequences
for multiple aspects of language processing, making it challenging to relate regions
and functions on a one-to-one basis. Patients recover to varying extents from strokes
and from neurosurgical lesions, implying that there is considerable cortical plas-
ticity, so when we examine a patient with a lesion we are generally dealing with a lan-
guage system that has been reorganized to some extent. Finally, while it is clear that
brain regions are not equipotent when it comes to language, it is also surely not the
case that each region simply performs a single function. While all researchers intu-
itively understand the limits of modularity, all three of the approaches that we have
discussed are predicated on functional and/or neural modularity. A major challenge
in the coming decades will be to develop methods and approaches that are better able
to reveal the more complex reality of brain-language relationships. Despite these and
the other challenges discussed in this chapter, patient studies continue to offer some
of the clearest and most persuasive data on the cognitive architecture and neural
basis of language.
Acknowledgments
I thank Andrew DeMarco, Sarah Schneck, Greig de Zubicaray, and Niels Schiller for construc-
tive feedback on this chapter.
References
Ashburner, J., & Friston, K. J. (2000). Voxel-based morphometry: The methods. NeuroImage,
11, 805–821.
Badecker, W., & Caramazza, A. (1985). On considerations of method and theory governing
the use of clinical categories in neurolinguistics and cognitive neuropsychology: The case
against agrammatism. Cognition, 20, 97–125.
Basso, A., Lecours, A. R., Moraschini, S., & Vanier, M. (1985). Anatomoclinical correlations
of the aphasias as defined through computerized tomography: Exceptions. Brain and
Language, 26, 201–229.
Bates, E., Friederici, A., & Wulfeck, B. (1987). Grammatical morphology in aphasia: Evidence
from three languages. Cortex, 23, 545–574.
(2003). Voxel-based lesion-symptom mapping. Nature Neuroscience, 6, 448–450.
Benton, A. L. (1971). A biblical description of motor aphasia and right hemiplegia. Journal of
the History of Medicine and Allied Sciences, 26, 442–444.
Benton, A. L., & Joynt, R. J. (1960). Early descriptions of aphasia. Archives of Neurology, 3,
205–222.
Blazely, A. M., Coltheart, M., & Casey, B. J. (2005). Semantic impairment with and without
surface dyslexia: Implications for models of reading. Cognitive Neuropsychology, 22, 695–7 17.
Bonilha, L., & Fridriksson, J. (2009). Subcortical damage and white matter disconnection asso-
ciated with non-fluent speech. Brain, 132, e108.
Bouillaud, J. (1825). Recherches cliniques propres a demontrer que la perte de la parole corre-
spond a la lesion des lobules anterieurs du cerveau, et a confirmer l’opinion de M. Gall, sur le
siege de l’organe du language article. Archives Générales de Médecine, 8,  25–45.
Bozeat, S., Lambon Ralph, M. A., Patterson, K., Garrard, P., & Hodges, J. R. (2000). Non-verbal
semantic impairment in semantic dementia. Neuropsychologia, 38, 1207–1215.
Broca, P. (1861). Remarques sur le siège de la faculté du langage articulé, suivies d’une obser-
vation d’aphémie (perte de la parole). Bulletins de la Société Anatomique de Paris, 2e Serie, 6,
330–357.
Broca, P. (1865). Sur le siège de la faculté du langage articulé. Bulletins de la Société d’Anthropologie
de Paris, 6, 377–393.
Butler, R. A., Lambon Ralph, M. A., & Woollams, A. M. (2014). Capturing multidimensionality
in stroke aphasia: Mapping principal behavioural components to neural structures. Brain,
137, 3248–3266.
Caplan, D. (1987). Neurolinguistics and linguistic aphasiology: An introduction. Cambridge:
Cambridge University Press.
Caplan, D., Waters, G., Kennedy, D., Alpert, N., Makris, N., DeDe, G., . . . Reddy, A. (2007).
A study of syntactic processing in aphasia II: Neurological aspects. Brain and Language,
101, 151–177.
Caramazza, A., & Badecker, W. (1991). Clinical syndromes are not God’s gift to cognitive neu-
ropsychology: A reply to a rebuttal to an answer to a response to the case against syndrome-
based research. Brain and Cognition, 16, 211–227.
Coltheart, M., Rastle, K., Perry, C., Langdon, R., & Ziegler, J. (2001). DRC: A dual route
cascaded model of visual word recognition and reading aloud. Psychological Review, 108,
204–256.
Dax, M. (1836/1865). Lésions de la moitié gauche de l’encéphale coïncident avec l’oublie des
signes de la pensée. Gazette Hebdomadaire de Médecine et de Chirurgie, 17, 259–260.
Dell, G. S., Schwartz, M. F., Nozari, N., Faseyitan, O., & Coslett, H. B. (2013). Voxel-based
lesion-parameter mapping: Identifying the neural correlates of a computational model of
word production. Cognition, 128, 380–396.
Dronkers, N. F. (1996). A new brain region for coordinating speech articulation. Nature, 384,
159–161.
Gall, F. (1825). Sur les fonctions du cerveau et sur celles de chacune de ses parties. Paris: Boucher.
Geschwind, N. (1965). Disconnexion syndromes in animals and man. I. Brain, 88, 237–294.
Goodglass, H., & Berko, J. (1960). Agrammatism and inflectional morphology in English.
Journal of Speech and Hearing Research, 3, 257–267.
Gorno-Tempini, M. L., Dronkers, N. F., Rankin, K. P., Ogar, J. M., Phengrasamy, L., Rosen, H.
J., . . . Miller, B. L. (2004). Cognition and anatomy in three variants of primary progressive
aphasia. Annals of Neurology, 55, 335–346.
Gorno-Tempini, M. L., Hillis, A. E., Weintraub, S., Kertesz, A., Mendez, M., Cappa, S.
F., . . . Grossman, M. (2011). Classification of primary progressive aphasia and its variants.
Neurology, 76, 1006–1014.
Graham, K. S., Hodges, J. R., & Patterson, K. (1994). The relationship between comprehension
and oral reading in progressive fluent aphasia. Neuropsychologia, 32, 299–316.
Griffiths, J. D., Marslen-Wilson, W. D., Stamatakis, E. A., & Tyler, L. K. (2013). Functional orga-
nization of the neural language system: Dorsal and ventral pathways are critical for syntax.
Cerebral Cortex, 23, 139–147.
Grodzinsky, Y. (1984). The syntactic characterization of agrammatism. Cognition, 16, 99–120.
Grossman, M. (2010). Primary progressive aphasia: Clinicopathological correlations. Nature
Reviews Neurology, 6,  88–97.
Haglund, M. M., Berger, M. S., Shamseldin, M., Lettich, E., & Ojemann, G. A. (1994). Cortical
localization of temporal lobe language sites in patients with gliomas. Neurosurgery, 34,
567–576.
Heiss, W. D., Karbe, H., Weber-Luxenburger, G., Herholz, K., Kessler, J., Pietrzyk, U., & Pawlik,
G. (1997). Speech-induced cerebral metabolic activation reflects recovery from aphasia.
Journal of the Neurological Sciences, 145, 213–217.
Heiss, W. D., Kessler, J., Thiel, A., Ghaemi, M., & Karbe, H. (1999). Differential capacity of left
and right hemispheric areas for compensation of poststroke aphasia. Annals of Neurology,
45, 430–438.
Hillis, A. E., Wityk, R. J., Barker, P. B., Beauchamp, N. J., Gailloud, P., Murphy, K., . . . Metter, E.
J. (2002). Subcortical aphasia and neglect in acute stroke: The role of cortical hypoperfusion.
Brain, 125, 1094–1104.
Hillis, A. E., Work, M., Barker, P. B., Jacobs, M. A., Breese, E. L., & Maurer, K. (2004). Re-
examining the brain regions crucial for orchestrating speech articulation. Brain, 127,
1479–1487.
Hodges, J. R., & Patterson, K. (2007). Semantic dementia: A unique clinicopathological syn-
drome. Lancet Neurology, 6, 1004–1014.
Hodges, J. R., Patterson, K., Oxbury, S., & Funnell, E. (1992). Semantic dementia: Progressive
fluent aphasia with temporal lobe atrophy. Brain, 115, 1783–1806.
Holland, R., & Lambon Ralph, M. A. (2010). The anterior temporal lobe semantic hub is a part
of the language neural network: Selective disruption of irregular past tense verbs by rTMS.
Howard, D., & Patterson, K. (1992). Pyramids and palm trees: A test of semantic access from
pictures and words. Bury St. Edmunds: Thames Valley.
Inoue, K., Madhyastha, T., Rudrauf, D., Mehta, S., & Grabowski, T. (2014). What affects
detectability of lesion- deficit relationships in lesion studies? NeuroImage: Clinical, 6,
388–397.
Kertesz, A., & McCabe, P. (1977). Recovery patterns and prognosis in aphasia. Brain, 100,  1–18.
Kimberg, D. Y., Coslett, H. B., & Schwartz, M. F. (2007). Power in voxel-based lesion-symptom
mapping. Journal of Cognitive Neuroscience, 19, 1067–1080.
Levelt, W. J. M. (2013). A history of psycholinguistics: The pre-Chomskyan era. Oxford: Oxford

University Press.
Lichtheim, L. (1885). On aphasia. Brain, 7, 433–484.
Liepmann, H. (1898). Ein Fall von reiner Sprachtaubheit. Breslau: Schletter’sche Buchhandlung.
Mah, Y.-H., Husain, M., Rees, G., & Nachev, P. (2014). Human brain lesion-deficit inference
remapped. Brain, 137, 2522–2531.
Marshall, J. C., & Newcombe, F. (1973). Patterns of paralexia: A psycholinguistic approach.
Journal of Psycholinguistic Research, 2, 175–199.
Mesulam, M. (1982). Slowly progressive aphasia without generalized dementia. Annals of
Neurology, 11, 592–598.
Mesulam, M. (2001). Primary progressive aphasia. Annals of Neurology, 49, 425–432.
Metter, E. J., Kempler, D., Jackson, C., Hanson, W. R., Mazziotta, J. C., & Phelps, M. E. (1989).
Cerebral glucose metabolism in Wernicke’s, Broca’s, and conduction aphasia. Archives of
Neurology, 46,  27–34.
Miceli, G., Mazzucchi, A., Menn, L., & Goodglass, H. (1983). Contrasting cases of Italian
agrammatic aphasia without comprehension disorder. Brain and Language, 19,  65–97.
Mirman, D., Chen, Q., Zhang, Y., Wang, Z., Faseyitan, O. K., Coslett, H. B., & Schwartz, M.
F. (2015). Neural organization of spoken language revealed by lesion-symptom mapping.
Nature Communications, 6, 6762.
Mohr, J. P. (1976). Broca’s area and Broca’s aphasia. In H. Whitaker & H. Whitaker (Eds.),
Studies in neurolinguistics (Vol. 1, pp. 201–233). New York: Academic Press.
Mohr, J. P., Pessin, M. S., Finkelstein, S., Funkenstein, H. H., Duncan, G. W., & Davis, K. R.
(1978). Broca aphasia: Pathologic and clinical. Neurology, 28, 311–324.
Mummery, C. J., Patterson, K., Wise, R. J., Vandenberghe, R., Vandenbergh, R., Price, C. J.,
& Hodges, J. R. (1999). Disrupted temporal lobe connections in semantic dementia. Brain,
122,  61–73.
Naeser, M. A., & Hayward, R. W. (1978). Lesion localization in aphasia with cranial computed
tomography and the Boston Diagnostic Aphasia Exam. Neurology, 28, 545–551.
Naeser, M. A., Palumbo, C. L., Prete, M. N., Fitzpatrick, P. M., Mimura, M., Samaraweera, R., &
Albert, M. L. (1998). Visible changes in lesion borders on CT scan after five years poststroke,
and long-term recovery in aphasia. Brain and Language, 62,  1–28.
Noble, K., Glosser, G., & Grossman, M. (2000). Oral reading in dementia. Brain and Language,
74,  48–69.
Patterson, K., & Hodges, J. R. (1992). Deterioration of word meaning: Implications for reading.
Neuropsychologia, 30, 1025–1040.
Pedersen, P. M., Jørgensen, H. S., Nakayama, H., Raaschou, H. O., & Olsen, T. S. (1995). Aphasia
in acute stroke: Incidence, determinants, and recovery. Annals of Neurology, 38, 659–666.
Penfield, W., & Roberts, L. (1959). Speech and brain-mechanisms. Princeton, NJ: Princeton
University Press.
Plaut, D. C., McClelland, J. L., Seidenberg, M. S., & Patterson, K. (1996). Understanding
normal and impaired word reading: Computational principles in quasi-regular domains.
Saur, D., & Hartwigsen, G. (2012). Neurobiology of language recovery after stroke:
Lessons from neuroimaging studies. Archives of Physical Medicine and Rehabilitation,
93, S15–S25.
Saur, D., Lange, R., Baumgaertner, A., Schraknepper, V., Willmes, K., Rijntjes, M., & Weiller, C.
(2006). Dynamics of language reorganization after stroke. Brain, 129, 1371–1384.
Schwartz, M. F., Kimberg, D. Y., Walker, G. M., Faseyitan, O., Brecher, A., Dell, G. S., &
Coslett, H. (2009). Anterior temporal involvement in semantic word retrieval: Voxel-based
lesion-symptom mapping evidence from aphasia. Brain, 132, 3411–3427.
Seeley, W. W., Crawford, R. K., Zhou, J., Miller, B. L., & Greicius, M. D. (2009). Neurodegenerative
diseases target large-scale human brain networks. Neuron, 62,  42–52.
Shapiro, K., & Caramazza, A. (2003). Grammatical processing of nouns and verbs in left frontal
cortex? Neuropsychologia, 41, 1189–1198.
Shapiro, K., Shelton, J., & Caramazza, A. (2000). Grammatical class in lexical produc-
tion and morphological processing: Evidence from a case of fluent aphasia. Cognitive
Neuropsychology, 17, 665–682.
Smania, N., Gandolfi, M., Aglioti, S. M., Girardi, P., Fiaschi, A., & Girardi, F. (2010). How long
is the recovery of global aphasia? Twenty-five years of follow-up in a patient with left hemi-
sphere stroke. Neurorehabilitation and Neural Repair, 24, 871–875.
Snowden, J. S., Thompson, J. C., Stopford, C. L., Richardson, A. M. T., Gerhard, A., Neary, D., &
Mann, D. M. A. (2011). The clinical diagnosis of early-onset dementias: Diagnostic accuracy
and clinicopathological relationships. Brain, 134, 2478–2492.
Spurzheim, J. G. (1815). The physiognomical system of Drs. Gall and Spurzheim. London:
Baldwin, Cradock, and Joy.
Swinburn, K., Porter, G., & Howard, D. (2004). Comprehensive aphasia test. Hove: Psychology
Press.
Tsvjetkova, L. S., & Glozman, Z. M. (1978). Agrammatism pri afasji. Moscow: University of
Moscow Press.
Warren, J. E., Crinion, J. T., Lambon Ralph, M. A., & Wise, R. J. S. (2009). Anterior tem-
poral lobe connectivity correlates with functional outcome after aphasic stroke. Brain, 132,
3428–3442.
Wernicke, C. (1874). Der aphasische symptomencomplex. Breslau: Cohn and Weigert.
Wilson, S. M., Brambati, S. M., Henry, R. G., Handwerker, D. A., Agosta, F., Miller, B.
L., . . . Gorno-Tempini, M. L. (2009). The neural basis of surface dyslexia in semantic de-
mentia. Brain, 132,  71–86.
Wilson, S. M., Dronkers, N. F., Ogar, J. M., Jang, J., Growdon, M. E., Agosta, F., . . . Gorno-
Tempini, M. L. (2010). Neural correlates of syntactic processing in the nonfluent variant of
primary progressive aphasia. Journal of Neuroscience, 30, 16845–16854.
Wilson, S. M., Galantucci, S., Tartaglia, M. C., Rising, K., Patterson, D. K., Henry, M.
L., . . . Gorno-Tempini, M. L. (2011). Syntactic processing depends on dorsal language tracts.
Neuron, 72, 397–403.
Wilson, S. M., Henry, M. L., Besbris, M., Ogar, J. M., Dronkers, N. F., Jarrold, W., . . . Gorno-
Tempini, M. L. (2010). Connected speech production in three variants of primary progres-
sive aphasia. Brain, 133, 2069–2088.
Wilson, S. M., Lam, D., Babiak, M., Perry, D., Shih, T., Hess, C. P., Berger, M. S., & Chang, E. F.
(2015). Transient aphasias after left hemisphere resective surgery. Journal of Neurosurgery,
123, 581–593.
Wilson, S. M., Ogar, J. M., Laluz, V., Growdon, M., Jang, J., Glenn, S., . . . Gorno-Tempini, M.
L. (2009). Automated MRI-based classification of primary progressive aphasia variants.
NeuroImage, 47, 1558–1567.
Woollams, A. M., Ralph, M. A. ., Plaut, D. C., & Patterson, K. (2007). SD-squared: On the
association between semantic dementia and surface dyslexia. Psychological Review, 114,
316–339.
Xing, S., Lacey, E. H., Skipper-Kallal, L. M., Jiang, X., Harris-Love, M. L., Zeng, J., & Turkeltaub,
P. E. (2016). Right hemisphere grey matter structure and language outcomes in chronic left
hemisphere stroke. Brain, 139, 227–241.
Yourganov, G., Smith, K. G., Fridriksson, J., & Rorden, C. (2015). Predicting aphasia type from
brain damage measured with structural MRI. Cortex, 73, 203–215.
Zhang, Y., Kimberg, D. Y., Coslett, H. B., Schwartz, M. F., & Wang, Z. (2014). Multivariate
lesion-symptom mapping using support vector regression. Human Brain Mapping, 35,
5861–5876.
Chapter 3
E l ectrophysi ol o g i c a l
Methods in t h e St u dy
of L anguage Pro c e s si ng
Michelle Leckey and Kara D. Federmeier
Introduction and History
An interest in how language processing unfolds and how it is implemented in the brain is
long-standing, yet the complexity of language and its absence (at least in complete form)
in nonhuman animals has rendered the study of language especially difficult. Language
comprehension, in particular, has no necessary behavioral consequence, so cognitive
studies of language have resorted to developing language-related tasks (e.g., naming,
lexical decision) in order to obtain behavioral measures. This approach has yielded val-
uable results, giving insights into the nature of underlying cognitive mechanisms, but
comes with unavoidable limitations, as it reduces ecological validity and ultimately
provides no direct information about the processes occurring in the period between
stimulus presentation and the behavioral response. Measurements of electrical brain ac-
tivity (as well as other types of neural measures) have played a critical role in bridging
this gap, as they can trace processing over time with a high degree of accuracy without
requiring an overt response (e.g., Kutas & Van Petten, 1994). Accordingly, they have
been used to add weight to or to falsify behaviorally posited arguments and also to pro-
vide completely novel insights into the cognitive and neural bases of language.
The origins of human electrophysiology can be traced back to the 1920s, when the
first electroencephalogram (EEG) was recorded from a human participant. Prior to this,
recordings of electrical activity had been taken from the brains of nonhuman mammals,
but it wasn’t until German neurologist Hans Berger (1873–1941) took an interest in con-
necting higher mental processes to their underlying physiological mechanisms that
these methods were extended to human testing (Millett, 2001). The first successful
human recording came in 1924, when electrodes were placed under the scalp of a patient
Electrophysiological Methods 43
who had had part of his skull removed as part of a medical procedure, but the findings of
this and other early studies (including those from participants with intact skulls) were
not published until 1929 (Berger, 1929), and it was not until the mid-1930s that the EEG
was recognized as a valuable diagnostic tool in clinical neurology. In more recent years,
this method, along with the derivation of event-related potentials (ERPs), has been used
to great effect in cognitive neuroscience research.
Human electrophysiological measures are noninvasive, recording brain signals
through the use of electrodes placed over the head and on the face and extremities.
These electrodes pick up the electrical potentials that are an inherent part of neural
communication. Neural transmission involves the flow of ions, which carry a charge,
across the cell membrane, resulting in changes in the electrical potential, the attractive
force that dictates the extent to which charged particles feel the urge to move. In a ca-
nonical neural transmission event, an electrical signal, originating in the cell body of a
transmitting neuron, travels rapidly down the axon via saltatory conduction. This ac-
tion potential causes the release of neurotransmitters, which bind to the dendrites of
a receiving neuron, causing a change in its electrical potential. Depending on the neu-
rotransmitter that is released, this change in potential can make the receiving cell more
likely to fire (an excitatory postsynaptic potential, or EPSP) or less likely to fire (an in-
hibitory postsynaptic potential, or IPSP). These potential changes can be detected at a
distance, forming the signal captured by the EEG recording (for a detailed discussion,
see Nunez & Srinivasan, 2006).
Because the strength of the electrical potential falls off with distance, the ability to
pick up potential changes depends on the sensitivity of the recording equipment and
the size of the potential change. The generation of signals of sufficient strength to be de-
tectable at the scalp using modern recording equipment requires that large numbers of
neurons be subject to EPSPs or IPSPs in relative synchrony, so that their potentials can
add. Moreover, in order for the potentials to sum to create a large “equivalent dipole”
that is measurable at a distance, the neurons must be arranged in what is known as an
open field, wherein they are all pointing in a similar direction. If the neurons are instead
arranged in a closed field, as is true for a number of subcortical structures, the potentials
generated by individual neurons cancel one another so that there is no detectable ac-
tivity at the scalp (see Allison, Wood, & McCarthy, 1986).
Luckily for the study of human cognition, the cortex provides good conditions for
electrophysiological recordings. Cortical pyramidal cells are generally arranged in
an open field, tend to become active in relative synchrony, and are close to the scalp,
meaning they are more easily picked up by scalp electrodes (see Luck, 2014, Chapter 1, for
a more detailed explanation of this). Although in some circumstances, action potentials
can be picked up, it is mostly postsynaptic currents that are measured, as they are di-
polar, so their electromagnetic field falls off less rapidly than that of action potentials,
which are quadrupolar (Ilmoniemi, 1993). Thus, the EEG provides a direct measure of
neural activity, much of which originates from postsynaptic changes in cortical pyram-
idal cells. The signals that are detected from the cortex are very small, and therefore must
be amplified. Moreover, along with brain activity, other biological signals are picked up,
44 Michelle Leckey and Kara D. Federmeier
such as muscle activity, eye movements, and the heartbeat (as, even though these signals
are produced farther away from the sensors, they are much larger than brain electrical
activity). Thus, as will be discussed in more detail later, these potential artifacts must be
avoided or removed in order to allow uncontaminated extraction of the neural signals
that are of interest for most experimental designs.
Event-R elated Potentials:

Recording Considerations
The EEG signal is continuous and is characterized by large, rhythmic fluctuations across
different frequency bands. Although these signals have clinical utility and are sometimes
measured to address cognitive questions (see, e.g., Bastiaansen, Mazaheri, & Jensen,
2012; Mathewson et al., 2011), most cognitive studies are interested in the neural events
that immediately follow a stimulus or that lead up to a behavioral response. These time-
locked (and phase-locked) neural responses—or event-related potentials (ERPs)—are
therefore extracted from the continuous EEG. Traditionally, these signals are separated
from the background “noise” of electrical activity that is not temporally aligned with
events of interest via averaging (but see, e.g., Makeig, Debener, Onton, & Delorme,
2004, for alternative approaches). By taking the mean of a number of segments of the
EEG aligned in time with the presentation of a stimulus (to create a stimulus-locked
ERP) or the execution of a response (a response-locked ERP), it is possible to measure
brain activity that is yoked to that event, with an improvement of signal to noise that
increases with the square root of the number of trials that are averaged together (as-
suming consistent signal and random noise).
In designing an ERP study, the primary goal is generally to obtain stable within-
subject responses that can be readily interpreted with respect to experimental
conditions of interest. In the service of this goal, it is critical to plan ahead to avoid
confounds and/or to have strategies for dealing with confounds and issues that cannot
be avoided. One of the primary considerations is the number of participants and
the number of items per condition, and these factors trade off with one another. In a
typical language experiment in which a fairly large effect is expected, trial numbers
would typically be around 25–60 items in each critical condition of interest, and ap-
proximately 20–30 participants would be run. However, this number of trials may be
unrealistic in designs that use long texts (where large numbers of trials may induce par-
ticipant fatigue) or that target language phenomena with inherently limited numbers
of items (where increasing trial numbers would entail repetition, a manipulation that
is known to affect processing—and the resultant ERP waveform—in multiple ways).
More limited numbers of trials, including even single-item designs, can be used if more
participants are run and/or appropriate analytical approaches are used in order to keep
power high (e.g., Laszlo & Federmeier, 2011, 2014).
Another critical consideration for designing an ERP experiment is the choice of the
reference electrode. The electrical potential is the force that compels a charged particle
to move from one place to another; it thus inherently involves a comparison across two
locations. In practice, to reduce contributions of external noise sources to recordings,
EEG techniques generally make use of a minimum of three electrodes and differen-
tial amplification, amplifying the difference between an active electrode and an elec-
trical ground and a reference electrode and that ground (see discussion in Luck, 2014).
This double subtraction serves to remove noise that is commonly experienced by all
electrodes, leaving just the signal of interest, as a difference between each active location
and a common reference electrode (or set of electrodes). Obviously, the signal recorded
from an active electrode is therefore importantly shaped by the choice of reference. For
example, the fact that the recorded signal is a subtraction is part of the reason that the
absolute polarity (positive or negative) of the measured voltage does not provide any
information about the nature of the underlying cognitive or neural processes involved.
Ideally, the reference would be placed somewhere “neutral” that picked up neither
brain signals nor artifactual activity from the body. In practice, no such sites exist. Thus,
the choice of reference electrode is dictated by locations that are practical and that are
likely to provide the least distortion to signals of most critical interest. Common ref-
erence sites include the nose and the mastoid processes (bony sites behind the ear) or
earlobes. When mastoid/earlobe references are used, the reference is the average of the
activity across the two electrodes, to avoid creating lateralized biases in the recording.
Instead of picking a particular reference site, some researchers create what is known as
an average reference, in which the average of activity across all electrodes is subtracted
from each electrode. This approach tends to highlight activity that is localized to only a
few electrode sites and to reduce effects with a widespread distribution (including sev-
eral of the most commonly studied language-related components). This approach is ob-
viously also very sensitive to the overall number and distribution of electrode sites used.
All of these reference types pick up brain activity, and if the neural activity of interest
is near the reference electrode, it may be subtracted out. Thus, although the use of the
average mastoid/earlobe reference is perhaps most common, especially for studies of
language, different subfields have come to use different typical reference sites to allow
the best characterization of the responses of most interest in that literature. When de-
signing an ERP experiment, therefore, it is critical to note what reference is used in the
literature that forms the background for the work, and generally it is preferable to use
that reference. Comparisons of effects are difficult, if not impossible, across different ref-
erence configurations (although in some cases, data can be re-referenced).
Other choices that need to be made include the type, number, and distribution of
electrode sites. Electrodes have traditionally been passive sensors, made of electrically
reactive metals like silver/silver chloride or tin, which carry signals to an amplifier some
distance away. However, some modern systems build the amplifier into the electrode
(in what has been termed active electrode systems), as the shorter distance between the
sensor and the amplifier can reduce noise in electrically compromised environments
and/or under conditions of high impedance. These two types of electrodes have
trade-offs in terms of cost, ease of use, and performance (for a direct comparison of the
performance of passive and active electrodes under different recording conditions, see
Laszlo, Ruiz-Blondet, Khalifian, Chu, & Jin, 2014).
Up to 256 electrodes can be used (beyond this, there is no further gain in spatial preci-
sion), but large arrays come with costs in terms of preparation time, chances of bad data
from one or more sensors, bridging between electrodes, and processing complexity. In
language studies, arrays of 20–60 electrodes are most common, and even studies that
use larger arrays often select or group electrodes to form coarser regions of analysis
(raising questions about the utility of having initially obtained high-density recordings).
Better than having large numbers of electrodes is yoking the number of electrodes to the
design and question of interest. For example, broadly distributed and well-characterized
responses can be measured with a small, well-chosen array of electrodes, and the sav-
ings in preparation time for each participant can thereby allow for designs with much
higher numbers of participants and enhanced ability to look at responses to even indi-
vidual items (see, e.g., Laszlo & Federmeier, 2011).
Finally, because the brain signals are small and attenuated by passing through the me-
ninges, skull, skin, and so on, EEG data must be amplified. The process of amplifying
and then sampling the data changes that data in ways that are critical to understand
when carrying out EEG recording. All amplifiers have a dynamic range, and it is impor-
tant that the amplifier gain be set so that responses are likely to stay within that range, to
prevent “blocking” (in which the amplifier becomes insensitive to changes in potentials
and registers just a maximum or minimum measurement), and that, if blocking occurs,
those trials be removed from the data. As well as amplifying the neural signals, the am-
plifier also acts as a filter, meaning that it removes signals with certain frequency charac-
teristics (i.e., low-pass filters that attenuate high-frequency signals, and high-pass filters
that attenuate low-frequency signals). Filtering is a complex topic, and it is critical for
researchers using EEG/ERP methods to familiarize themselves with how filtering works
and what types of signal distortions it can create (see, e.g., Chapter 5 in Luck, 2014). As
general guidelines, language researchers should be aware that commonly studied lan-
guage responses contain frequencies between about 0.1 and 20 Hz, so filters that ex-
tend into this range run the risk of attenuating or distorting effects of interest (see, e.g.,
Tanner, Morgan-Short, & Luck, 2015), and that the analog filtering implemented in an
amplifier is different from digital filtering of the sampled data. The greater complexity
and reduced frequency precision of analog filters generally make it preferable to record
the data with limited filtering and to apply digital filters later, as necessary. However,
some filtering is necessary at the stage of amplification, in order to avoid the problem of
aliasing, as described next.
The amplifier provides a continuous, analog signal. To obtain data from this signal,
therefore, it must be sampled—that is, transformed into a digital signal with individual
pairings of time points and voltages (at some level of precision). The sampling rate
describes how often such samples are taken. The minimum sampling rate that should
be used is given by the Nyquist criterion, which states that the sampling rate must be at
least twice as fast as the highest expected frequency in the signal to avoid the possibility
of artificially creating a low-frequency signal in the data by undersampling a high-

frequency one (i.e., aliasing) (Luck, 2014). Filtering is therefore used at the stage of am-
plification to limit the high-frequency content of the sampled signal (and also to prevent
artifacts from low-frequency drift). Because the cutoffs of analog filters are gradual, in
practice the sampling rate must be more than twice the low-pass frequency set on the
amplifier. Thus, with sampling rates of 250 Hz or more, frequencies up to 100 Hz can be
safely sampled.
Event-R elated Potentials:

Experimental Design and Analyses
To collect interpretable ERPs, several important design considerations should be kept

in mind. First, the exquisite temporal sensitivity of ERPs means that timing must be a
critical consideration in the experimental design. Sensory ERPs are elicited in response
not only to stimulus onsets, but also to offsets and, indeed, any perceptible change in
the stimulus; thus, it may be desirable to try to time stimulus presentation so that offset
potentials, for example, fall outside the range of other components of interest. Along the
same lines, it should be noted that forward and backward masks both elicit their own
ERP responses, which are likely to overlap substantially with those of the critical stim-
ulus. Since masking is sometimes used in behavioral paradigms to try to target “early”
aspects of processing, which ERPs naturally reveal, it may not be necessary in many ERP
adaptations of traditional designs—and, when masking is used, the resulting ERPs must
be analyzed and interpreted with care. Responses, including the preparation to make a
response, also elicit ERPs. Because ERPs can be used to measure cognition without the
need for an overt response, it is often desirable to design ERP experiments so that be-
havioral responses are not elicited to critical stimuli—or, if needed, are delayed (see Van
Vliet et al., 2014, for a discussion of the difficulties created by response-related activity
in cognitive ERP experiments). ERP responses are also elicited by anticipation of a stim-
ulus or response. Many ERP designs therefore introduce temporal jitter into parts of the
experimental design, such as between a fixation point and the first word of a sentence, to
reduce the contribution of activity related to this kind of anticipation to the average ERP.
The temporal sensitivity of ERPs allows for the study of processes unfolding at many
different time scales. Often in the study of language comprehension, researchers will
time-lock to the onset of a critical word; however, one can also time-lock to the onset of
a critical phoneme or morpheme, to a “gap” in a syntactic structure, to a phrase, clause,
or sentence boundary, to the presentation of a probe stimulus, or to a response—an op-
tion that is often used in production research (Ganushchak, Christoffels, & Schiller,
2011). At the same time, the need for a time-locking point makes it more difficult to
study processes whose onset is unknown, variable, or extended in time (e.g., inference-
drawing, prediction). Often researchers can only look at downstream consequences of
these processes with ERPs (e.g., the apprehension of an unpredictable word). However,
the measurement of slow potentials and examination of non-time-locked activity in the
time-frequency domain promise to open new avenues of exploration for these types of
processes as well (e.g., King & Kutas, 1995; Rommers et al., 2017).
Second, one of the advantages of ERPs is their sensitivity to many, sometimes subtle,
variables. However, the fact that ERPs are a sensitive measure and one that picks up
activity across the entire time course of processing makes them also good at revealing
confounds in experimental designs. Therefore, when designing an ERP experiment, it is
critical to control for the full range of perceptual variables that could differ across stimuli,
for the probability of stimulus categories, for all aspects of response-related activity (e.g.,
timing of response preparation, hand used to respond, type and probability of errors,
etc.), and for various aspects of what participants anticipate, what they are attending to,
and what strategies they are employing. When possible, the use of perceptually iden-
tical critical stimuli and/or of identical contexts is ideal. When not possible, extraneous
factors should be matched as closely as possible, or the study should be designed so
that the effects of potentially confounding variables can be directly examined. For lan-
guage experiments in particular, it is important to consider not only word length, part of
speech, and frequency (as is also common in behavioral designs), but also concreteness
and neighborhood density, which often have larger effects on language-related brain
responses than do frequency and length (see, e.g., Kounios & Holcomb, 1994; Laszlo &
Federmeier, 2009). For sentences, length, structure, and complexity are important, and
the position and predictability (e.g., cloze probability; Taylor, 1953) of critical words in
the sentence must be controlled. When words are being presented in a series, one must
consider the possibility of influences from words that immediately follow critical words,
as ERP responses elicited to those subsequent words are likely to overlap in time with
the ERPs to the word of interest. Finally, many aspects of the brain response are highly
sensitive to repetition, at all levels; thus, when necessary or desirable, repetition must be
used with care in experimental designs.
As a general guideline, caution should be used when ERPs are being directly
compared across different participant groups (e.g., young adults and older adults), dif-
ferent sentence positions (e.g., sentence-initial, sentence-intermediate, and sentence-
final words), or different screen positions (e.g., left and right visual field). Across such
comparisons, the basic morphological characteristics of the waveform are likely to
differ, meaning it is difficult to align the waveforms appropriately to interpret effects.
In these cases, it is better to compare the patterns of effects that are measured within
a group or position. Similar problems arise when directly comparing conditions with
different configurations of componentry (e.g., a condition that elicits just an N400
versus one that elicits both an N400 and a P600) or that are associated with response
differences. Comparisons across average ERPs based on very different numbers of trials
can also be problematic, due to differences in noise.
ERP study designs must also keep in mind the need to avoid contamination of the
data by various types of artifacts. Perhaps the most common artifacts are those that
come from eye movements and blinks. The eye is an electrical dipole, which means that
any movement of the eyes creates large electrical signals that can overpower the smaller
brain signals, especially at sites over the front of the head, where eye movements have
more of an effect. The most common method of minimizing these artifacts is through
prevention: by asking participants to limit eye movements and blinks and by designing
stimuli and paradigms to make these less likely. ERP experiments using visual words,
for example, generally present them one at a time in the center of the screen, to mitigate
the need to move the eyes (although some recent work has been successful at recording
ERPs during natural reading; see Hutzler et al., 2007). Similarly, ERP experiments using
pictures or scenes generally size these to allow them to be apprehended without scan-
ning. However, some participants have difficulty controlling these responses, and some
researchers worry that inhibiting blinks and eye movements can be a type of task in and
of itself and therefore may affect the resulting data. When blinks and eye movements do
not invalidate the experimental manipulation (e.g., a blink during a critical time window
that means a participant likely didn’t see the stimulus, or a saccade that indicates that a
participant likely foveated a lateralized stimulus), corrections can sometimes be applied
that aim to selectively remove the artifactual activity, yielding usable trials (see Croft,
Chandler, Barry, Cooper, & Clarke, 2005, for a discussion). As these methods vary in
any given circumstance as to how well they can remove the artifacts successfully while
not affecting the data in any other way, they should be used with care and their success
assessed empirically whenever they are applied. Other sources of artifact include muscle
activity, which shows up as high-frequency bursts and should be prevented when pos-
sible but can often be filtered out during post-processing, and heartbeat (EKG), which
can sometimes be prevented by keeping electrodes away from locations with a palpable
pulse, but which, when not preventable, will often be reduced by averaging.
ERPs are commonly visualized as waveforms—plots of voltage over time, one for each
recording site. Each waveform is made up of a series of positive-and negative-going
voltage deflections (when well studied, sometimes called components), relative to a
baseline. Baselining consists of setting the mean voltage in each condition to zero in
some time window, generally 100 milliseconds (ms) or more immediately prior to the
time-locking point. It is done to remove artifactual slow activity (e.g., DC offsets from
skin potentials) that might differ across conditions; because these can vary by electrode,
baselines are generally computed separately for each. ERPs are a highly multidimen-
sional measure, which can yield information about when activity patterns or effects
occur, about the size of those responses, and about their scalp topography (which is
not the same as the spatial location of their neural sources). Analyzing ERP data thus
requires selecting measures to address the question(s) of interest and then making spe-
cific choices about how those measures will be obtained.
One of the most basic questions that can be asked with ERPs is whether there is a
difference in waveform amplitude across conditions. Such questions are answered by
measuring either mean voltage or peak (maximum/minimum) voltage in one or more
time windows at one or more active electrode sites. Mean amplitude measures are often
preferred to peak amplitudes because peak amplitude measures are more dependent on
waveform morphology (e.g., some effects don’t have a clearly defined peak) and more
susceptible to noise. However, when multiple, differentiable responses occur in rapid

succession (as, for example, is often the case for early sensory responses), peak ampli-
tude measures may provide a means of separating specific responses of interest. In the
absence of any a priori knowledge of when and where a difference might arise, compre-
hensive, exploratory analyses may be used that test for amplitude differences across the
whole waveform using a “moving window” kind of approach (e.g., tests across succes-
sive 50-ms time windows). Such analyses might be run over all electrodes, over regions
created by grouping electrodes spatially, or over a targeted region or set of electrodes. In
these cases it is absolutely critical that appropriate corrections for multiple comparisons
be applied (see Groppe, Urbach, & Kutas, 2011) and that effects be interpreted with cau-
tion until they have been replicated.
In other cases, mean amplitudes are measured within an a priori time window of in-
terest and/or in a time window centered on the measured peak latency of a target com-
ponent or effect. In the latter case, that time window might be determined on the basis of
the peak latency in the “grand average” data (i.e., averaged across participants) or calcu-
lated independently for individual participants (and there are different issues with each
of these approaches). These measures might be made over all electrode sites or in an a
priori region of interest. In the former case, it is often informative to include aspects of
electrode position as factors in a multivariate analysis, by, for example, organizing elec-
trode positions along dimensions such as front-to-back and left-to-right. This type of dis-
tributional analysis can also be used to establish a region of interest for post hoc analyses.
If a reliable amplitude difference between conditions is obtained, one can definitively
conclude that the brain is sensitive to the variable that was manipulated (assuming, of
course, that one has adequately taken care of confounds and artifacts). Moreover, one
can say something about the timing of that difference. The time at which the ERPs can
be shown to reliably diverge can be taken as the upper limit on the time that the brain
must have appreciated the difference between the two conditions; of course, there could
have been earlier differences that were not detectable in the ERP signal. More gener-
ally, differences in amplitude on well-studied components may allow more specific
inferences, which often take the form of assumptions that a particular cognitive process
or neural system is engaged to a greater or lesser degree in one condition than in another.
However, it is important to remember that amplitude differences based on averages can
arise either because there is a smaller/larger signal on most trials in one condition than
in another or because there is more temporal variability (latency jitter) in one condition
than another. If there is more latency jitter in one condition, the response will tend to be
distributed over a larger time window, and one will end up with a smaller, broader peak.
Of course, there are ways to get around this problem (see, e.g., Puce, Berkovic, Cadusch,
& Bladin, 1996)—by examining the variance in peak latency to determine if jitter is a
problem, for example—but it is important to be mindful of it. It is also worth noting
that there are a number of types of neural changes that could result in a scalp amplitude
difference (e.g., smaller postsynaptic potentials in the same set of neurons, activity in a
smaller set of neurons, activity that is less temporally synchronous, etc.), which may en-
gender different interpretations.
In addition to amplitude differences and their timing, the latency of well-defined

electrophysiological responses can be assessed. This is typically done using peak latency
measures, wherein the time of the maximum or minimum value of the waveform in a
time window is measured and compared across conditions, often leading to the infer-
ence that the “same” neural/cognitive processes are occurring in both but are delayed
in one condition relative to another. Care should be taken when measuring peaks: time
windows should be chosen that are large enough to encompass the peak of interest in
all participants (avoiding the possibility of a spurious maximum or minimum at the
boundary of the measurement time window), but small enough to avoid the possibility
of measuring a different response, and low-pass filtering is usually desirable to avoid
spurious peaks due to noise.
The topography of ERP responses can also be measured and used to make inferences.
Rather than looking at voltages through time at an individual location on the head, one
can plot the distribution of voltages over the head in a specific window of time. Statistical
analyses that include scalp location as factors (e.g., dividing the scalp into regions in the
anterior-to-posterior and left/right lateral-to-medial directions) can be used to deter-
mine where amplitudes are largest and whether that distribution over the scalp differs
across conditions. There is often confusion about what kinds of inferences are licensed
when scalp topographic differences are observed. If the brain activity in one condition
has a different distribution over the scalp from that in another condition, then it must
be the case that the neural source of the activity in the two conditions differs in some
way (note that the converse does not hold: different populations of active neurons could
add up to produce the same pattern of activity across the scalp). However, it cannot be
directly inferred that those sources differ in location, as differences in the strength of
(the same set of) underlying sources can also be a cause of topographic differences at
the scalp (and normalization cannot compensate for that; see Urbach & Kutas, 2002).
Moreover, one cannot make direct inferences about the spatial location of neural
sources from scalp topography alone, even at a gross level. For example, a source in the
back of the head can create an electrical dipole that has a maximum/minimum picked
up by electrode sites positioned over the front of the head (because the closer end of the
dipole extends into the neck, where it cannot be picked up) and, similarly, a source in
the left hemisphere can create an electrical dipole with a maximum/minimum detected
over the right hemisphere (so-called paradoxical lateralization; see Van Petten & Luka,
2006). Inferences about neural sources of ERP responses require constraints from other
methods and/or mathematical modeling (see discussion in Luck, 2014).
In addition to these standard techniques for analyzing ERP data, there are some newer
techniques that are becoming more common and that take advantage of the combined
spatial and temporal properties of the signal (these techniques are also increasingly used
to help remove artifacts from EEG data; see, e.g., Jung et al., 2000). One example is in-
dependent component analysis (ICA); note that here the word “component” refers to
a part of the data that is separated mathematically, whereas the term “component” as
used more generally in the ERP literature refers to a scalp-recorded effect that is taken
to reflect a specific neural or psychological process. ICA treats the observed EEG data
as a sum of contributions from a set of fixed, spatially defined sources, and separates
the signal into subcomponents by assuming that temporally correlated but spatially
separable parts of the signal are coming from separable—independent—sources. The
technique makes a number of assumptions, some of which are likely true for EEG data,
and others that are almost certainly not (e.g., that the number of sources is the same
as the number of sensors). Other spatiotemporal separation techniques, such as prin-
cipal component analysis (PCA), make different assumptions about the nature of the
underlying signal (for a comparison of methods, see Dien, Khoe, & Mangun, 2007). As
a general rule, researchers using spatiotemporal analysis techniques should (1) make
sure they fully understand the assumptions and limitations of the techniques they are
applying; (2) always present raw waveform data as well as the output of these techniques,
to allow comparison with the wider literature; and (3) bear in mind that inferences from
the output of these techniques require the same process of establishing a linking hypo-
thesis (as discussed next) as do inferences from raw waveform features. Indeed, part of
what makes using newer analysis techniques difficult is that they require—and currently
lack—the kind of extensive replication and validation that is available from decades of
work analyzing the raw waveform with traditional approaches.
In sum, because they are direct measures of neural activity, ERPs allow direct
inferences about the upper limit of the timing with which the brain appreciates
differences between two experimental conditions and, to some extent, the nature of that
processing difference (e.g., whether it manifests as a change in amplitude, latency, or
scalp topography of the response). Inferences about some aspect of psychology, such
as language processing, then require a linking hypothesis: a hypothesis that associates
a particular psychological process or state with a particular aspect of the measured re-
sponse. The quality of the inference drawn depends on the quality of this linking hypo-
thesis, and the hypothesis itself is subject to updating based on new data. The most basic
linking hypothesis is that a difference in neural activity means a difference in psycholog-
ical processing. More specific hypotheses link well-studied aspects of the ERP response
(i.e., components) to aspects of psychological processing, including processes that are
important for language. In the next section, therefore, we introduce a number of ERP
components that have been used to study various aspects of language processing.
Known ERP Components Used

in Language Research
In principle, any part of the ERP signal can be informative about the brain’s sensitivity
to various experimental manipulations, revealing when, how, and how much a given
variable influences brain electrical activity. However, having found a reliable effect, it
is important to replicate it, and useful for the field to then begin to learn more about
that response: What factors modulate it? Which do not? What are the downstream
consequences, on other ERP responses or behavior, of this activity? When activity can
be reliably elicited and identified and when its functional properties have begun to
be well characterized, the field begins to talk about that response as an ERP “compo-
nent.” Despite the fact that we may not fully understand the cognitive or neural bases of
each component, accumulated knowledge about ERP components allows researchers
to make inferences about the type of processing that is occurring during a particular
task. Early use of ERPs was mostly dedicated to developing good linking hypotheses
for these components, allowing them to now serve as indices of particular psycho-
logical constructs or neural mechanisms. In the following, we highlight some of the
well-known, as well as some more newly characterized, ERP components that play an
important role in language research. Note, however, that many (perhaps all) of these
components are not “language responses” as such, but rather indices of more general
processes that are nonetheless informative when used in studies of language. Indeed,
language both depends on and influences other cognitive processes, including sensory
perception, attention, memory, emotion, reasoning, and decision-making, and behav-
ioral responses of all types. ERPs are sensitive to all of these factors, thereby providing a
very rich set of dependent measures.
Language Comprehension
ERPs provide measures that can be obtained without a secondary task, simply while
people listen to or view stimuli for comprehension, allowing a picture of language com-
prehension in its more natural state. However, there are challenges for both visual and
auditory studies of language comprehension. In the case of reading, the biggest chal-
lenge comes from the fact that, as already discussed, eye movements can create prob-
lematic artifacts in ERP data. Therefore, studies with visual words usually present them
one at a time in the center of the screen, at a rate of about two to three words per second.
Although this rate is slower than eye-movement patterns seen during normal reading, it
is important to remember that, different from normal reading, serial visual presentation
methods provide no information about sentence length, no preview of upcoming words,
and no opportunities to revisit words that have already been read. In fact, when people
are allowed to self-select their pace for serial visual presentation, they tend to read at a
rate of about two to three words per second (Ditman, Holcomb, & Kuperberg, 2007),
suggesting that this is a comfortable speed for the average reader. Furthermore, effects
obtained with word-by-word reading often pattern closely with effects seen in natural
speech (for example, compare Federmeier & Kutas, 1999, and Federmeier, McLennan,
Ochoa, & Kutas, 2002), despite the fact that natural speech unfolds more quickly.
For auditory language comprehension, the major challenge involves time-locking, as
it is difficult to pinpoint word onsets (or other points of interest) in continuous speech.
Some experimenters insert pauses before critical words in order to have a discrete au-
ditory onset for time-locking (and also then generally make recordings without that
critical word, to avoid contamination by co-articulatory information). However, this,

of course, reduces the naturalness of the input. More generally, because auditory infor-
mation accrues over time, even with precise time-locking, ERP responses to auditory
language input tend to be more distributed over time (less peaked) than corresponding
effects with visual words.
Despite these challenges, ERPs have played a particularly important role in the
study of comprehension. Language unfolds very rapidly and on multiple time scales.
Some processes, such as the differentiation of consonant sounds, can take place in
milliseconds, whereas discourse processing can unfold over the course of minutes.
Timing is therefore arguably one of the most important considerations for measures of
comprehension; a method like ERPs that can be used across these different time scales
is thus very valuable. Moreover, given that processing across different levels of language
is interdependent, the continuous nature of ERPs is also advantageous, allowing the si-
multaneous recording and analysis of a discourse, the sentences that make up the dis-
course, the words that make up the sentences, and the phonemes or letters that make
up the words. As described next, because different subcomponents of language pro-
cessing have been associated with different ERP components, ERPs provide a means of
tapping into specific aspects of processing. Indeed, it is possible to measure multiple
subprocesses at once, although experimental designs that focus on a single component
are generally easier to interpret.
N400
The N400 (Figure 3.1 A) is probably the most widely used ERP measure in language re-
search. It was discovered by researchers looking at the effects of sentential context on
word processing (Kutas & Hillyard, 1980a). For example, although the words “dog” and
“sugar” have a similar out-of-context (lexical) frequency, when presented as endings to
a sentence like “He takes his coffee with milk and . . . ,” the probability of the two words
changes. It was expected that the improbable appearance of “dog” in this sentence would
elicit a P300 (a component sensitive to probability, as described later). Instead, however,
around 400 ms post-word-onset there was a larger negativity to “dog” than to “sugar”—
a pattern that is now referred to as an “N400 anomaly effect.” At the same time, words
that had an unexpected physical feature (e.g., changed size) did elicit the expected P300,
and words that were both semantically anomalous and physically improbable elicited
both effects (Kutas & Hillyard, 1980b), highlighting the difference in the brain’s response
to these two factors.
Subsequent studies designed to replicate this effect and uncover the neural and
functional properties of the N400 have now established that the N400 is not a specific
response to semantic anomalies, but rather part of the brain’s normal processing of sen-
sory inputs (see review by Kutas & Federmeier, 2011). In particular, the N400 appears to
reflect activity involved in linking sensory stimuli with long-term memory—in other
(A) N400
µV
5µV 3.72
3.35
2.98
2.60
2.23
1.86
400 800 1.49
1.12
0.74
0.37
0.00
–0.37
–0.74
–1.12
–1.49
–1.86
–2.23
–2.60
–2.98
350–450ms –3.35
–3.72
(B) Frontal Positivity

µV
1.33
1.19
5µV 1.06
0.93
0.80
0.66
0.53
0.40
400 800 0.27
0.13
0.00
–0.13
–0.27
–0.40
–0.53
–0.66
–0.80
–0.93
–1.06
–1.19
600–800ms –1.33
µV
(C) P600 7.18
6.47
5.75
5.03
4.31
3.59
5µV 2.87
2.16
1.44
400 800 0.72
0.00
–0.72
–1.44
–2.16
–2.87
–3.59
–4.31
–5.03
–5.75
–6.47
–7.18
600–800ms
Figure 3.1. (A) The N400 effect can be seen in the comparison between sentences with unexpected
endings (dashed line), which elicit a large negative peak around 400 ms after stimulus onset, compared
to sentences with expected endings (solid line). The topographic map shows the centroparietal distri-
bution of the N400 seen to unexpected endings. (B) A larger frontal positivity can be seen in response
to plausible low cloze probability completions of high-constraint sentences (dashed line) compared
to the same words completing low-constraint sentences (solid line). The topographic map shows
the frontal distribution of the effect. (C) The P600 effect can be seen to syntactically incongruous
sentences (dashed line) when compared to syntactically congruous sentences (solid line). The topo-
graphic map shows the posterior distribution typical of the P600 effect. In all cases the waveform is
taken from the electrode denoted by a solid black circle on the corresponding topographic map.
words, accessing the meaning of those stimuli. N400 activity is observed to words (and
word-like strings) in all languages, modalities, and forms (Holcomb & Neville, 1991;
Kutas, Neville & Holcomb, 1987), as well as to pictures, scenes, videos, and environ-
mental sounds (Ganis, Kutas & Sereno, 1996; Sitnikova, Kuperberg & Holcomb, 2003;
Van Petten, & Rheinfelder, 1995). As such, although the N400 is an ERP component im-
portant in the study of language, it is not a language component (i.e., it is not specific to
linguistic processing).
The N400 manifests as a negative-going component that, in healthy young adults,
peaks just before 400 ms after the onset of a stimulus. It has a widespread distribution
(when an average mastoid reference is used), which, for written words tends to be largest
at centroparietal scalp sites. An important source of scalp-recorded N400 activity seems
to come from the temporal lobe, but it is clear that, more generally, the N400 reflects
concurrent activity in a highly distributed neural network. A distinguishing character-
istic of the N400 is its temporal stability: although participant-level characteristics such
as age, language proficiency (e.g., in multilingual speakers), and some neurological or
psychiatric conditions may affect N400 latency, within-participant manipulations mod-
ulate the size of the N400 but not its latency (Federmeier & Laszlo, 2009).
N400 amplitudes are modulated by many factors that affect semantic access. Out of
context, N400 amplitudes to words are importantly affected by orthographic neighbor-
hood size, as one of several factors (also including neighbor frequency and number of
lexical associates) linked to the size of the lexico-semantic network that a given stim-
ulus normally activates (Laszlo & Federmeier, 2009, 2011; Laszlo & Plaut, 2012). Thus,
words with larger neighborhoods elicit larger N400s (i.e., engender more activity in the
semantic network). When context information, from individual words (or pictures)
or built up across larger language (or nonlinguistic) structures, renders words more
predictable, N400 amplitudes are reduced, presumably because some of the informa-
tion normally associated with that word was preactivated in the course of processing
the context. In this sense, N400 amplitudes can sometimes be interpreted as reflecting
the “ease” of processing the current word. In sentences, N400 amplitudes are an inverse
function of cloze probability (a measure of the percentage of people who would finish
a sentence with a particular word), such that words with a high cloze probability elicit
a smaller N400 than those with a low cloze probability (Kutas & Hillyard, 1984). The
N400 also shows effects of repetition, wherein repeated items elicit a reduced ampli-
tude compared to first presentations (Rugg, Doyle, & Melan, 1993; for a computational
account of this, see Laszlo & Armstrong, 2014). These factors, therefore, must be taken
into account in experiments in which an N400 is expected.
Because the N400 can be recorded to every word in a list, sentence, or larger lan-
guage context, and can be measured without the need for an extraneous task (i.e., while
participants read or listen for comprehension), studies using the N400 have provided
critical insights into a wide range of language-related questions and, in some cases, have
challenged ideas and theories that had been built primarily from behavioral data. As
just a few examples, N400 data have been important for revealing the role of prediction
in language comprehension (Federmeier, 2007), the ability of the RH to comprehend
language and differences between the hemisphere’s comprehension biases (Federmeier,

Wlotko, & Meyer, 2008), and the accrual of word-related knowledge in second-
language learners before that knowledge can be tapped into for behavioral responding
(McLaughlin, Osterhout, & Kim, 2004).
Frontal Positivity
A well-established property of the N400 is its insensitivity to contextual constraint

(Kutas & Hillyard, 1984). That is, although the N400 varies with the cloze probability
of the current word in its context, it does not vary as a function of how much a different
word might have been expected. A low cloze probably word elicits the same size N400
response when embedded in a weakly constraining context (where no single word is
particularly expected) as when embedded in a strongly constraining context (i.e., where
a different word was highly predicted). The N400 thus reflects the amount of prior acti-
vation for the current stimulus, and not something about the global level of prediction
or of information accrued across the context. Instead, processing differences associated
with disconfirmed predictions manifest in the form of a (pre-)frontal positivity, which
follows the N400 (Figure 3.1 B).
For example, Federmeier, Wlotko, De Ochoa-Dewald, and Kutas (2007) observed a
frontal positivity in response to low cloze probability completions of high-constraint
sentences (e.g., “He bought her a pearl necklace for her collection” [expected = “birthday”])
compared to the same words used as completions of low-constraint sentences. This re-
sponse has now been replicated in a number of studies, and can be distinguished from
other positivities to unexpected words (i.e., the Late Positive Complex or P600, as
reviewed later in the chapter) both by its frontal (as opposed to posterior) topography and
the fact that it is elicited by unexpected words that are, nevertheless, congruent in their
contexts (see Van Petten & Luka, 2012, for review). The presence of the frontal positivity
can serve as an index of the use of predictive processing mechanisms during comprehen-
sion (Wlotko, Federmeier, & Kutas, 2012). In characterizing this effect, however, it is im-
portant to contrast two low cloze probability items, rather than examine the difference
between an unexpected and an expected item. This avoids difficulties caused by earlier
differences on the N400 and also avoids confusing the frontal positivity to the unex-
pected word with frontal negativity that is sometimes elicited by words of moderate to
high cloze probability (see Wlotko & Federmeier, 2012a, 2012b).
P600 and Other Posterior Positivities
Another well-known ERP component used in studies of language processing is

the P600 (Figure 3.1 C). The P600 is a positive-going component typically observed
between 500 and 1,000 ms after stimulus onset; as is fairly typical of later ERP
components, the P600 manifests as a response that usually lasts for several hundred
milliseconds and, as such, often has no clear, discrete peak. Unlike the N400, the P600
does not have a characteristic latency; both the amplitude and the latency of this re-
sponse change, depending on the nature of the experimental manipulation. The distri-
bution also can change (for example, with normal aging the distribution becomes more
frontal; Kemmer, Coulson, De Ochoa, & Kutas, 2004), but in young adults it is usually
largest over posterior scalp sites.
The P600 is most commonly associated with syntactic anomalies, as it is observed to
words that violate the morphosyntactic or phrase structure regularities of language (at
least temporarily, as in “garden path” sentences). For example, the P600 was originally
described by Osterhout and Holcomb (1992), who gave participants sentences such as
“the broker hoped to sell the stock” and “the broker persuaded to sell the stock (was
tall).” A P600 was observed to the word “to” in the latter sentence, as participants ini-
tially interpreted the word “persuaded” as the main verb of the sentence and therefore
expected that a direct object (e.g., the person being persuaded) will follow. However, the
P600 is not specific to violations. P600s have also been seen, for instance, to words in
object-relative sentences when compared to subject-relative ones, as the latter are easier
to understand in English. The P600 can also be “primed,” showing a smaller amplitude
to words in less preferred sentence structures when these are preceded by a sentence
with the same structure (Tooley, Traxler, & Swaab, 2009).
Although the P600 has been most closely linked with syntactic processing, and, on
some models, has been specifically associated with efforts to repair and/or reanalyze
syntactic information (e.g., Friederici, 1995; Osterhout, Holcomb, & Swinney, 1994) and/
or with the difficulty of syntactic integration (e.g., Kaan, Harris, Gibson, & Holcomb,
2000), P600-like responses also have been seen in response to manipulations that, on the
surface, seem more related to semantics than to syntax. Kuperberg, Sitnikova, Caplan,
and Holcomb (2003) presented participants with three types of critical sentences: the
first was a standard, “control” sentence (e.g., “For breakfast the boys would only eat toast
and jam), whereas the other two types contained semantic/pragmatic violations (e.g.,
pragmatic violation: “For breakfast the boys would only bury toast and jam”; and the-
matic role violation: “For breakfast, the eggs would only eat toast and jam”). Relative
to the control sentence, the first type of (pragmatic) violation elicited a larger N400 at
the critical word (italicized in the examples). However, the response to the thematic
role violation (at “eat” following “eggs”) was a large, posterior positivity. Such semantic
P600s have now been replicated in a range of studies, with varying theories about the
type of processing being indexed by the P600 in this case (Brouwer, Fitz, & Hoeks, 2012;
Kuperberg, 2007).
P600-like responses (posterior late positivities, which may sometimes also be
termed Late Positive Complex, or LPC responses) have also been seen to spelling errors
(e.g., Vissers, Chwilla, & Kolk, 2006), leading some to posit that the P600 reflects a
general error-monitoring process (Kolk & Chwilla, 2007). Indeed, there are impor-
tant similarities between the P600 and the more general P300 (specifically, the P3b).
The P3b is a domain-general response that is sensitive to stimulus probability, among

other factors. Its latency varies (from around 300 ms to much later) with the time
needed to evaluate a stimulus in the context of the task, and its amplitude is sensitive
to subjective probability, task relevance, and salience, with larger P3b responses to
less probable and more relevant/salient stimuli (see Polich, 2007, for a review). Like
the P600, it has a posterior distribution. The P3b has been used in its own right to
look at the role of attention and working memory in language processing—for ex-
ample, in special populations (e.g., Evans, Selinger, & Pollak, 2011). Similarities be-
tween the response properties of the P3b and the P600 have raised questions about
the language specificity of the P600. Coulson, King, and Kutas (1998), for example,
showed that the P600 response was affected by the probability of the grammatical
violation. This, coupled with the lack of difference in topographical distribution
between the two components, led them to conclude that the components were not
distinct. Similarly, Sassenhagen, Schlesewsky, and Bornkessel-Schlesewsky (2014)
showed that the P600 aligns in time with response-related processes—again, like the
P3b, and arguably against the predictions of many prominent theories of the P600
as a language-related component. Other researchers, however, maintain that the
two components are separable, as evidenced by experimental manipulations (e.g.,
Osterhout, McKinnon, Bersick, & Corey, 1996), studies with clinical populations
(Frisch, Kotz, von Cramon, & Friederici, 2003; Hagoort, Wassenaar, & Brown, 2003),
and differences in oscillatory signatures (Davidson & Indefrey, 2007; Ford, Roach,
Hoffman, & Mathalon, 2008).
Left Anterior Negativity
The left anterior negativity (LAN) is another component that has been linked to syn-
tactic processing (Friederici, 1995; Osterhout & Holcomb, 1992). The distribution and
timing of responses labeled as LANs in the literature has been variable, although it is
unclear whether this reflects actual malleability of a similar type of brain response or, in-
stead, because different types of brain activity have been (mis)labeled as LANs. The LAN
tends to be described as appearing in a similar time window as the N400—between
about 300 and 500 ms post-stimulus-onset—but with a more frontal and (sometimes)
left-lateralized distribution. The similarity in timing with the N400 (which will be
observed to every word) contributes to the difficulty of identifying the LAN, especially
since component overlap (e.g., with posterior positivities like the P600/LPC) can shift
the apparent distribution of the N400.
The two components have been dissociated experimentally, with an N400 seen to
semantic anomalies and a LAN seen to syntactic ones (Münte, Heinze, & Mangun,
1993). Some link the LAN specifically to morphosyntactic agreement (Friederici, 2002).
However, there has also been evidence for the component in cases in which the syntactic
structure is correct (garden paths: Kaan & Swaab, 2003). Based on this type of evidence,
some have argued that the LAN reflects more general, working- memory- related
processes, and have associated the more transient LAN responses seen to individual
words with sustained negative activity seen for various types of language structures that
are working-memory intensive (King & Kutas, 1995; Kluender & Kutas, 1993).
Finally, an “early LAN” (eLAN) has been described in some cases, especially to
word-category violations. Friederici (2002) links the eLAN with “first-pass” syn-
tactic processes involved in phrase structure building. However, several recent studies
have suggested instead that the eLAN may reflect a domain-general sensory mis-
match detection process (similar to the MMN, described later) that registers the dif-
ference between the perceptual input and a sensory expectation derived from strong
contextual constraints (Dikker, Rabagliati, & Pylkkänen, 2009; Lau, Stroud, Plesch, &
Phillips, 2006).
NRef and Other

Ambiguity-R elated Effects
In addition to frontal negativities that have been linked to syntactic processing and/or
working memory, there is a growing literature on negativities that have been linked to
the resolution of various kinds of ambiguity. For example, Van Berkum, Brown, and
Hagoort (1999) examined the resolution of referential ambiguity, comparing responses
to a noun or pronoun (e.g., “the man” or “he”) in the context of a discourse that had pre-
viously introduced only one possible referent (e.g., “John and Mary . . .”) or more than
one possible referent (e.g., “John and Bill . . .”). The presence of referential ambiguity
elicited a sustained negativity beginning around 200 ms post-stimulus-onset, which
had a widespread distribution, but was larger over frontal scalp sites. This difference has
been labeled the “NRef effect.” Its link to ambiguity-resolution processes is strengthened
by the fact that if the referential ambiguity is eliminated prior to the critical noun/pro-
noun (e.g., because, although two possible referents were originally mentioned, one
left the scene before the need to establish reference), the NRef effect is also eliminated
(Nieuwland, Otten, & Van Berkum, 2007).
An effect very similar in time course and distribution also has been seen in the con-
text of ambiguity resolution for noun/verb homographs (“duck”) presented in syn-
tactically well-specified but semantically neutral contexts (e.g., “John wanted to/the
duck . . .”; Lee & Federmeier, 2006, 2009). Again, this effect begins around 200 ms into
processing of the homographs and is sustained, even across multiple words (e.g., Lee
& Federmeier, 2012). Experiments examining the downstream consequences of this
ERP effect (Lee & Federmeier, 2012), as well as its correlation with eye-movement
patterns (Stites, Federmeier, & Stine-Morrow, 2013), have suggested that it reflects
brain activity associated with suppressing the context-inappropriate meaning of the
ambiguous word.
Mismatch Negativity
The mismatch negativity (MMN) is seen in response to an auditory change, with

or without attention being paid to the stimuli (Näätänen & Michie, 1979). Often in
experiments measuring the MMN, participants are exposed to a series of auditory
stimuli while they are reading a book, playing a game or are otherwise engaged in a
cognitive task. These stimuli consist of “standard” sounds that are mixed with less fre-
quent “deviant” sounds (different from the standards in intensity, pitch, duration, or
some other sensory feature). The ERP signal to the deviant sound is characterized by a
negative-going deflection around 150–250 ms after the deviance point. The MMN has a
fronto-central distribution and has been linked to sources in primary auditory cortex,
as well as the inferior frontal gyrus.
Importantly, the MMN is sensitive not only to basic sensory features, but also to ab-
stract patterns, including learned regularities. This makes the MMN a valuable tool for
examining speech processing and language acquisition (see review by Pulvermüller &
Shtyrov, 2006). MMNs can be recorded even from infants and can be used to assess,
for example, the ability to differentiate between phonemes. It is established that in early
life infants are capable of differentiating between almost all known phonemes, but that
adults only retain the ability to differentiate between those in their native language.
With this in mind, Choeur and colleagues (1998) used the MMN to show how memory
traces for language information develop. The amplitude of the MMN gets larger in re-
sponse to a deviant phoneme as the difference between the deviant and the standard
becomes greater. Choeur et al. (1998) presented children raised in a Finnish-speaking
environment with a standard phoneme from their language mixed with two types of
deviants, one that had a different vowel that was a phonemic contrast in Finnish and
one with a different vowel that was not a phonemic contrast in Finnish (and which was
more acoustically deviant from the standard). At six months of age, children’s MMN
responses varied with the amount of acoustic difference between the standards and the
deviants, independent of phonemic status. However, by one year, the children showed
larger MMN responses to the deviant that formed a phonemic contrast in their lan-
guage, providing neurological evidence for the development of speech memory traces
and their emergence between the ages of 6 and 12 months.
Language Production
Historically, ERPs have been less widely used in studies of language production, in part
because it was thought that the motor activity created by overt speech would cause
too much contamination of the brain data (although, as discussed in the following,
recent studies have actually been successful in collecting clean data even with overt
productions). Obtaining enough trials can also be an issue. Many behavioral studies of
production have used speech errors as data; however, these errors are generally rare and,
in most cases, experimental designs may not elicit a sufficient number of errors to allow
traditional analyses of average ERPs. The lack of control over what participants produce
in many naming studies can also make it difficult to obtain enough trials of the same
type for averaging. Moreover, time-locking to participants’ overt utterances is a labor-
intensive process (that must be done for each participant individually) and, as in studies
of speech comprehension, can be difficult. However, despite these difficulties, ERP
experiments in the production domain are becoming more prevalent (e.g., Ganushchak,
Christoffels, & Schiller, 2011) and are yielding important insights into the time course
and nature of the processes involved in planning speech.
Some research has used overt production paradigms, working to deal with the associ-
ated artifacts and other issues, and, in many cases, measuring the same components used
in comprehension research. For example, Koester and Schiller (2008) showed Dutch
participants words and images that were either morphologically related (e.g., jaszak [coat
pocket] with an image of a coat [jas]), in a semantically transparent or nontransparent
way, or were form related but not orthographically related (jasmijn [jasmine] with an
image of a coat [jas]). Participants were asked to read the word aloud, as well as to name
the image. N400 responses were attenuated to the morphologically related trails, irre-
spective of semantic transparency, but not to those trials that were merely form-related.
Investigators interpreted this result as evidence that morphological information is dis-
tinct from semantic and phonological representations within the mental lexicon. More
generally, this study and others like it show that ERPs can be successfully obtained during
word-reading and picture-naming tasks, allowing measurement of perceptual, atten-
tional, and language-related components, such as the N400 and the P2 (described next).
In other cases, researchers have used clever experimental designs to assess production-
related processing without contamination from overt speech (e.g., Van Turennout,
Hagoort, & Brown, 1998). As described in more detail in the following, much of this work
measures domain-general components, clearly illustrating how the full range of types of
brain responses can be harnessed to answer questions about language.
P2
The P2 is part of the visual evoked potential, the normal sequence of electrophysiological
responses observed in response to a visual onset, offset, or change. It is a positive-going
component that peaks between about 150 and 250 ms. The P2 is sensitive to stimulus
features (such as the complexity of a visual stimulus) and is modulated by repetition and
attention. In the context of language comprehension, P2 amplitudes have been found to
vary with expectancy, as indexed by sentential constraint (e.g., Wlotko & Federmeier,
2007). Recently, P2 response modulations have been observed in the context of simple pic-
ture naming with speeded, overt responses. Strijkers, Costa, and Thierry (2010) observed
more positive responses to pictures that elicit low-frequency names than those that elicit
high-frequency names beginning on the P2 response (around 180 ms). Such effects could
be due to visual/conceptual aspects of the picture, but a similar effect was seen in bilinguals
for words that are not cognates, compared to those that are. Based on this, and the finding
that this effect pattern seems to depend on the intention to name the picture (Strijkers,
Holcomb, & Costa, 2011), Strijkers and colleagues have argued that their P2 results index
the onset of lexical access occurring around 200 ms into picture processing.
Lateralized Readiness Potential

and N200
The lateralized readiness potential (LRP) is a domain-general index of motor-response

preparation that has been used to address questions about language production. The
LRP is derived, via subtraction, from the readiness potential (RP), and reflects activity
primarily from the primary motor cortex (Coles, 1989). To isolate activity specific to the
preparation of a particular hand, which generates activity in contralateral cortical areas,
the EEG signal obtained from an electrode positioned over the ipsilateral motor cortex
is subtracted from that obtained from an electrode over the contralateral motor cortex,
thereby removing activity associated with general motor preparation. This component
can thus be used to make inferences about whether a particular response is prepared,
and, if so, when and to what extent.
One example of how the LRP can be used to answer questions about language makes
use of a Go/No-Go task (Van Turennout, Hagoort, & Brown, 1998). In this task two factors
are mapped onto response behaviors (which hand to prepare and whether or not to ex-
ecute a response) in order to provide information about how and when these sources of
information become available. Schmitt, Münte, and Kutas (2000) used this paradigm to
investigate whether semantic information or phonological information is available earlier
in an implicit picture-naming task. They began by mapping the semantic content to the
response hand (e.g., respond with the right hand if the stimulus is an animal and the left
hand if not) and the phonological content to the Go/No-Go decision (e.g., if the word
begins with a vowel, respond, but do not respond if the word begins with a consonant).
They then switched this mapping in a second condition, so that semantics was paired
with the Go/No-Go task and phonology was mapped to the response hand. An LRP was
seen to the No-Go trials when the semantic task was mapped to hand of response but not
when the phonology task was, indicating that semantic information must become avail-
able before phonological information (i.e., when semantics was mapped to response hand,
participants could determine which hand to respond with before being able to determine
whether or not to respond, leading to motor preparation, and hence LRP activity, in the
No-Go case). This example illustrates how the LRP can be used in any case wherein there
is a question about which information becomes available to the response system first.
Another component that can be assessed in these type of designs is the N200 (Schmitt,
Münte, & Kutas, 2000), which is generally associated with inhibition. The N200 is a
negative-going potential with a fronto-central scalp distribution, which tends to peak
between about 200 and 350 ms post-stimulus-onset. The timing of the N200 on No-Go
trials (which require response inhibition) can be used to ask questions similar to those
assessed with the LRP, but the more peaked N200 may be easier to measure than the
slowly ramping LRP (Jansma, Rodriguez-Fornells, Möller, & Münte, 2004). The N200
also has been used to look for evidence of inhibitory processes in the context of bilingual
language processing (Rodriguez-Fornells et al., 2005).
Error-R elated Negativity
The error-related negativity (ERN, sometimes called the Ne) is a component that
indexes errors when people are making a rapid response, and is generally measured as a
difference between trials in which an error has been made and those that were answered
correctly (Gehring, Coles, Meyer, & Donchin, 1990). The onset of the negative-going
fronto-centrally distributed effect is seen just before the behavioral error, and it peaks
around 100 ms later when time-locked to the response. The component is thought to
have a source in the anterior cingulate cortex, which is an area of the brain associated
with cognitive control functions (Dehaene, Posner, & Tucker, 1994). The ERN is often
followed by an error-related positivity, and, whereas the ERN is seen to error trials re-
gardless of whether or not the error is noticed, the positivity only follows those trials in
which the participant is aware of his or her error (Nieuwenhuis, Ridderinkhof, Blom,
Band, & Kok, 2001).
One use of the ERN in language studies has been to investigate differences in the pro-
cessing of native versus second languages. For example, Ganushchak and Schiller (2009)
showed that native German speakers who were fluent in Dutch as a second language
had more interference from their native language, indexed by larger ERNs to errors,
when they were carrying out a Dutch phoneme-monitoring task under time pressure.
Another study has seen similar differences in bilingual speakers of Spanish and Catalan
(Sebastian-Gallés, Rodríguez-Fornells, de Diego-Balaguer, & Díaz, 2006). These studies
highlight differences between processing in one’s native and second languages that are
often not apparent in behavioral studies.
Conclusions
ERPs provide information about when and how processing unfolds in the brain,
and, having now been used to gain insights into the cognitive and neurobiological
mechanisms of language (and other aspects of cognitive processing) for more than
30 years, they provide a rich set of functionally well-characterized measures. ERPs are
sometimes criticized for their relative weakness as a tool for functional localization.
However, when used in combination with other methods that have better spatial resolu-
tion (e.g., MRI, TMS, optical imaging) or when examined alongside a priori knowledge
of component sources (gained from animal studies, clinical studies, source modeling,
etc.), ERPs can provide critical information about the brain areas involved in particular
types of processing. Moreover, and perhaps especially in complex domains like language,
the neural mechanisms of cognitive functions are often less a product of activation in
specific brain areas and more about the dynamics of activation across a large-scale brain
network. Hence, ERPs are used in clinical populations, not to find out which brain areas
are damaged, but to understand how the brain works when damaged (although it is im-
portant to note that structural changes to the brain and skull due to damage or medical
interventions can significantly change how electrical activity is propagated to the scalp,
raising issues for component identification). Moreover, ERPs have been important for re-
vealing how similar brain structures (e.g., the right and left hemispheres) can instantiate
importantly different functions via differences in processing dynamics.
In conclusion, ERPs are a method particularly well-suited to the study of language.
They provide a continuous measure, and their excellent temporal resolution allows for
the assessment of all aspects of linguistic information down to the millisecond level.
The lack of need for an overt task allows measures to be taken in a more ecologically
valid way, and also allows researchers to extend their work to populations, including
infants and some clinical populations, who cannot follow instructions or give behav-
ioral responses. The ability to make measurements without an overt task also means that
ERPs can shed light on aspects of processing that cannot be tapped using behavioral
methods—for example, revealing evidence for implicit learning of language regularities
in participants acquiring a second language (McLaughlin et al., 2004; Tokowicz &
MacWhinney, 2005). ERPs can also reveal cases in which similar outcomes are achieved
by different underlying processing dynamics, such as shifts away from predictive pro-
cessing mechanisms in healthy older adults (Federmeier et al., 2002). With contin-
uing advances in our understanding of good methodological practices and of ERP
components, it seems clear that electrophysiology will remain an invaluable tool for
elucidating linguistic processing for many more years to come.
References
Allison, T., Wood, C. C., & McCarthy, G. (1986). The central nervous system. In M. G. H. Coles,
S. W. Porges, & E. Donchin (Eds.), Psychophysiology: Systems, processes, and applications (pp.
5–25). New York: Guilford Press.
Bastiaansen, M., Mazaheri, A., & Jensen, O. (2012). Beyond ERPs: Oscillatory neuronal dy-
namics. In S. J. Luck & E. S. Kappenman (Eds.), The Oxford handbook of event-related
potentials (pp. 31–50). New York: Oxford University Press.
Berger, H. (1929). Über das Elektrenkephalogramm des Menschen (On the human electroen-
cephalogram). Archiv f. Psychiatrie u. Nervenkrankheiten, 87, 527–570.
Brouwer, H., Fitz, H., & Hoeks, J. (2012). Getting real about semantic illusions: Rethinking the
functional role of the P600 in language comprehension. Brain Research, 1446, 127–143.
Cheour, M., Ceponiene, R., Lehtokoski, A., Luuk, A., Allik, J., Alho, K. & Näätänen, R. (1998).
Development of language-specific phoneme representations in the infant brain. Nature
Coles, M. G. (1989). Modern mind-brain reading: Psychophysiology, physiology, and cogni-
tion. Psychophysiology, 26(3), 251–269.
Coulson, S., King, J. W., & Kutas, M. (1998). Expect the unexpected: Event-related brain re-
sponse to morphosyntactic violations. Language and Cognitive Processes, 13(1),  21–58.
Croft, R. J., Chandler, J. S., Barry, R. J., Cooper, N. R., & Clarke, A. R. (2005). EOG correction: A
comparison of four methods. Psychophysiology, 42(1),  16–24.
Davidson, D. J. & Indefrey, P. (2007). An inverse relation between event-related and time-
frequency violation responses in sentence processing. Brain Research, 1158,  81–92.
Dehaene, S., Posner, M. I., & Tucker, D. M. (1994). Localization of a neural system for error de-
tection and compensation. Psychological Science, 5, 303–305.
Dien, J., Khoe, W., & Mangun, G. R. (2007). Evaluation of PCA and ICA of simulated
ERPs: Promax vs. Infomax rotations. Human Brain Mapping, 28(8), 742–763.
Dikker, S., Rabagliati, H., & Pylkkänen, L. (2009). Sensitivity to syntax in visual cortex.
Cognition, 110(3), 293–321.
Ditman, T., Holcomb, P. J., & Kuperberg, G. R. (2007). An investigation of concurrent ERP and
self‐paced reading methodologies. Psychophysiology, 44(6), 927–935.
Evans, J. L., Selinger, C., & Pollak, S. D. (2011). P300 as a measure of processing capacity in audi-
tory and visual domains in specific language impairment. Brain Research, 1389, 93–102.
Federmeier, K. D. (2007). Thinking ahead: The role and roots of prediction in language com-
prehension. Psychophysiology, 44, 491–505.
Federmeier, K. D., & Kutas, M. (1999). A rose by any other name: Long-term memory structure
and sentence processing. Journal of Memory and Language, 41, 469–495.
Federmeier, K. D., & Laszlo, S. (2009). Time for meaning: Electrophysiology provides insights
into the dynamics of representation and processing in semantic memory. In B. H. Ross (Ed.),
Psychology of learning and motivation (Vol. 51, pp. 1–44). Burlington: Academic Press.
Federmeier, K. D., McLennan, D. B., Ochoa, E., & Kutas, M. (2002). The impact of semantic
memory organization and sentence context information on spoken language processing by
younger and older adults: An ERP study. Psychophysiology, 39(2), 133–146.
Federmeier, K. D., Wlotko, E. W., De Ochoa-Dewald, E., & Kutas, M. (2007). Multiple effects of
sentential constraint on word processing. Brain Research, 1146,  75–84.
Federmeier, K. D., Wlotko, E., & Meyer, A. M. (2008). What’s “right” in language comprehen-
sion: ERPs reveal right hemisphere language capabilities. Language and Linguistics Compass,
2,  1–17.
Ford, J. M., Roach, B. J., Hoffman, R. S. & Mathalon, D. H. (2008). The dependence of P300 am-
plitude on gamma synchrony breaks down in schizophrenia. Brain Research, 1235, 133–142.
Friederici, A. D. (1995). The time course of syntactic activation during language processing: A
model based on neuropsychological and neurophysiological data. Brain and Language,
50(3), 259–281.
Friederici, A. D. (2002). Towards a neural basis of auditory sentence processing. Trends in
Cognitive Sciences, 6(2),  78–84.
Frisch, S., Kotz, S. A., von Cramon, D. Y., & Friederici, A. D. (2003). Why the P600 is not just a
P300: The role of the basal ganglia. Clinical Neurophysiology, 114(2), 336–340.
Ganis, G., Kutas, M., & Sereno, M. I. (1996). The search for “common sense”: An electrophys-
iological study of the comprehension of words and pictures in reading. Journal of Cognitive
Ganushchak, L. Y., Christoffels, I. K., & Schiller, N. O. (2011). The use of electroencephalog-
raphy in language production research: A review. Frontiers in Psychology, 2(208),  1–6.
Ganushchak, L. Y., & Schiller, N. O. (2009). Speaking one’s second language under time
pressure: An ERP study on verbal self- monitoring in German- Dutch bilinguals.
Psychophysiology, 46(2), 410–419.
Gehring, W. J., Coles, M. G. H., Meyer, D. E., & Donchin, E. (1990). The error-related nega-
tivity: An event-related brain potential accompanying errors. Psychophysiology, 27, S34.
Groppe, D. M., Urbach, T. P., & Kutas, M. (2011) Mass univariate analysis of event-related brain
potentials/fields I: A critical tutorial review. Psychophysiology, 48, 1711–1725.
Hagoort, P., Wassenaar, M., & Brown, C. (2003). Real-time semantic compensation in patients
with agrammatic comprehension: Electrophysiological evidence for multiple-route plas-
ticity. Proceedings of the National Academy of Sciences, 100(7), 4340–4345.
Holcomb, P. J., & Neville, H. J. (1991). Natural speech processing: An analysis using event-
related potentials. Psychobiology, 19(4), 286–300.
Hutzler, F., Braun, M., Võ, M. L. H., Engl, V., Hofmann, M., Dambacher, M., Leder, H., &
Jacobs, A. M. (2007). Welcome to the real world: Validating fixation-related brain potentials
for ecologically valid settings. Brain Research, 1172, 124–129.
Ilmoniemi, R. J. (1993). Models of source currents in the brain. Brain Topography, 5(4), 331–336.
Jansma, B. M., Rodriguez-Fornells, A., Möller, J., & Münte, T. F. (2004). Electrophysiological
studies of speech production. Trends in Linguistics Studies and Monographs, 157, 361–396.
Jung, T. P., Makeig, S., Humphries, C., Lee, T. W., McKeon, M. J., Iragui, V., & Sejnowski,
T. J. (2000). Removing electroencephalographic artifacts by blind source separation.
Kaan, E., Harris, A., Gibson, E., & Holcomb, P. (2000). The P600 as an index of syntactic inte-
gration difficulty. Language and Cognitive Processes, 15(2), 159–201.
Kaan, E., & Swaab, T. Y. (2003). Electrophysiological evidence for serial sentence processing: A
comparison between non-preferred and ungrammatical continuations. Cognitive Brain
Research, 17(3), 621–635.
Kemmer, L., Coulson, S., De Ochoa, E., & Kutas, M. (2004). Syntactic processing with
aging: An event‐related potential study. Psychophysiology, 41(3), 372–384.
King, J., & Kutas, M. (1995). Who did what and when? Using word-and clause-level ERPs
to monitor working memory usage in reading. Journal of Cognitive Neuroscience, 7(3),
376–395.
Kluender, R., & Kutas, M. (1993). Bridging the gap: Evidence from ERPs on the processing of
unbounded dependencies. Journal of Cognitive Neuroscience, 5(2), 196–214.
Koester, D., & Schiller, N. O. (2008). Morphological priming in overt language produc-
tion: Electrophysiological evidence from Dutch. NeuroImage, 42(4), 1622–1630.
Kolk, H., & Chwilla, D. (2007). Late positivities in unusual situations. Brain and Language,
100(3), 257–261.
Kounios, J., & Holcomb, P. J. (1994). Concreteness effects in semantic processing: ERP evidence
supporting dual-coding theory. Journal of Experimental Psychology: Learning, Memory, and
Cognition, 20(4), 804.
Kuperberg, G. R. (2007). Neural mechanisms of language comprehension: Challenges to
syntax. Brain Research, 1146,  23–49.
Kuperberg, G. R., Sitnikova, T., Caplan, D., & Holcomb, P. J. (2003). Electrophysiological
distinctions in processing conceptual relationships within simple sentences. Cognitive Brain
Research, 17(1), 117–129.
Kutas, M., & Federmeier, K. D. (2011). Thirty years and counting: Finding meaning in the N400
component of the event-related brain potential (ERP). Annual Review of Psychology, 62,
621–647.
Kutas, M., & Hillyard, S. A. (1980a). Reading senseless sentences: Brain potentials reflect se-
mantic incongruity. Science, 207(4427), 203–205.
Kutas, M., & Hillyard, S. A. (1980b). Event-related brain potentials to semantically inappro-
priate and surprisingly large words. Biological Psychology, 11, 99–116.
Kutas, M., & Hillyard, S. A. (1984). Brain potentials reflect word expectancy and semantic asso-
ciation during reading. Nature, 307, 161–163.
Kutas, M., Neville, H. J., & Holcomb, P. J. (1987). A preliminary comparison of the N400 re-
sponse to semantic anomalies during reading, listening and signing. Electroencephalography
and Clinical Neurophysiology, Supplement, 39, 325–330.
Kutas, M., & Van Petten, C. K. (1994). Psycholinguistics electrified: Event-related brain poten-
tial investigations. In M. A. Gernsbacher (Ed.) Handbook of psycholinguistics (pp. 83–143).
San Diego, CA: Academic Press.
Laszlo, S., & Armstrong, B. C. (2014). PSPs and ERPs: Applying the dynamics of post-synaptic
potentials to individual units in simulation of ERP reading data. Brain and Language, 132,  22–27.
Laszlo, S., & Federmeier, K. D. (2009). A beautiful day in the neighborhood: An event-related
potential study of lexical relationships and prediction in context. Journal of Memory and
Language, 61(3), 326–338.
Laszlo, S., & Federmeier, K. D. (2011). The N400 as a snapshot of interactive pro-
cessing: Evidence from regression analyses of orthographic neighbor and lexical associate
effects. Psychophysiology, 48(2), 176–186.
Laszlo, S., & Federmeier, K. D. (2014). Never seem to find the time: Evaluating the physiolog-
ical time course of visual word recognition with regression analysis of single-item event-
related potentials. Language, Cognition and Neuroscience, 29(5), 642–661.
Laszlo, S., & Plaut, D. C. (2012). A neurally plausible parallel distributed processing model of
event-related potential word reading data. Brain and Language, 120(3), 271–281.
Laszlo, S., Ruiz-Blondet, M., Khalifian, N., Chu, F., & Jin, Z. (2014). A direct comparison
of active and passive amplification electrodes in the same amplifier system. Journal of
Neuroscience Methods, 235, 298–307.
Lau, E., Stroud, C., Plesch, S., & Phillips, C. (2006). The role of structural prediction in rapid
syntactic analysis. Brain and Language, 98(1),  74–88.
Lee, C. L., & Federmeier, K. D. (2006). To mind the mind: An event-related potential study of
word class and semantic ambiguity. Brain Research, 1081(1), 191–202.
Lee, C. L., & Federmeier, K. D. (2009). Wave-ering: An ERP study of syntactic and semantic
context effects on ambiguity resolution for noun/verb homographs. Journal of Memory and
Language, 61(4), 538–555.
Lee, C. L., & Federmeier, K. D. (2012). Ambiguity’s aftermath: How age differences in resolving
lexical ambiguity affect subsequent comprehension. Neuropsychologia, 50(5), 869–879.
Luck, S. J. (2014). An introduction to the event-related potential technique (2nd ed.). Cambridge,
MA: MIT Press.
Makeig, S., Debener, S., Onton, J., & Delorme, A. (2004). Mining event-related brain dynamics.
Mathewson, K. E., Lleras, A., Beck, D. M., Fabiani, M., Ro, T., & Gratton, G. (2011). Pulsed out
of awareness: EEG alpha oscillations represent a pulsed-inhibition of ongoing cortical pro-
cessing. Frontiers in Psychology, 2(99),  1–15.
McLaughlin, J., Osterhout, L., & Kim, A. (2004). Neural correlates of second-language word
learning: Minimal instruction produces rapid change. Nature Neuroscience, 7, 703–704.
Millett, D. (2001). Hans Berger: From psychic energy to the EEG. Perspectives in Biology and
Medicine, 44(4), 522–542.
Münte, T. F., Heinze, H. J., & Mangun, G. R. (1993). Dissociation of brain activity related to
syntactic and semantic aspects of language. Journal of Cognitive Neuroscience, 5(3), 335–344.
Näätänen, R., & Michie, P. T. (1979). Early selective attention effects on the evoked potential: A
critical review and reinterpretation. Biological Psychology, 8, 81–136.
Nieuwenhuis, S., Ridderinkhof, K. R., Blom, J., Band, G. P., & Kok, A. (2001). Error‐related
brain potentials are differentially related to awareness of response errors: Evidence from an
antisaccade task. Psychophysiology, 38(5), 752–760.
Nieuwland, M. S., Otten, M., & Van Berkum, J. J. (2007). Who are you talking about? Tracking
discourse- level referential processing with event- related brain potentials. Journal of
Nunez, P. L., & Srinivasan, R. (2006). The electric fields of the brain: The neurophysics of EEG.
Oxford: Oxford University Press.
Osterhout, L., & Holcomb, P. J. (1992). Event-related brain potentials elicited by syntactic
anomaly. Journal of Memory and Language, 31(6), 785–806.
Osterhout, L., Holcomb, P. J., & Swinney, D. A. (1994). Brain potentials elicited by garden-
path sentences: Evidence of the application of verb information during parsing. Journal of
Experimental Psychology: Learning, Memory, and Cognition, 20(4), 786.
Osterhout, L., McKinnon, R., Bersick, M. & Corey, V. (1996). On the language specificity of the
brain response to syntactic anomalies: Is the syntactic positive shift a member of the P300
family? Journal of Cognitive Neuroscience, 8(6), 507–526.
Polich, J. (2007). Updating P300: An integrative theory of P3a and P3b. Clinical Neurophysiology,
118(10), 2128–2148.
Puce, A. Berkovic, S. F., Cadusch, P. J., & Bladin, P. F. (1996). P3 latency jitter assessed
using 2 techniques: I. Simulated data and surface recordings in normal subjects.
Electroencephalography and Clinical Neurophysiology, 92, 352–364.
Pulvermuller, F., & Shtyrov, Y. (2006). Language outside the focus of attention: The mis-
match negativity as a tool for studying higher cognitive processes. Progress in Neurobiology,
79,  49–7 1.
Rodriguez-Fornells, A., Van Der Lugt, A., Rotte, M., Britti, B., Heinze, H. J., & Münte, T. F.
(2005). Second language interferes with word production in fluent bilinguals: Brain poten-
tial and functional imaging evidence. Journal of Cognitive Neuroscience, 17(3), 422–433.
Rommers, J., Dickson, D. S., Norton, J. J. S., Wlotko, E. W., & Federmeier, K. D. (2017). Alpha
and theta band dynamics related to sentential constraint and word expectancy. Language,
Cognition, and Neuroscience, 32(5), 576–589.
Rugg, M. D., Doyle, M. C., & Melan, C. (1993). An event-related potential study of the effects
of within-and across-modality word repetition. Language and Cognitive Processes, 8(4),
357–377.
Sassenhagen, J., Schlesewsky, M., & Bornkessel-Schlesewsky, I. (2014). The P600-as-P3 hypo-
thesis revisited: Single-trial analyses reveal that the late EEG positivity following linguisti-
cally deviant material is reaction time aligned. Brain and Language, 137,  29–39.
Schmitt, B. M., Münte, T. F., & Kutas, M. (2000). Electrophysiological estimates of the
time course of semantic and phonological encoding during implicit picture naming.
Sebastian-Gallés, N., Rodríguez-Fornells, A., de Diego-Balaguer, R., & Díaz, B. (2006).
First-and second-language phonological representations in the mental lexicon. Journal of
Sitnikova, T., Kuperberg, G., & Holcomb, P. J. (2003). Semantic integration in videos of real-
world events: An electrophysiological investigation. Psychophysiology, 40(1), 160–164.
Stites, M. C., Federmeier, K. D., & Stine-Morrow, E. A. (2013). Cross-age comparisons re-
veal multiple strategies for lexical ambiguity resolution during natural reading. Journal of
Experimental Psychology: Learning, Memory, and Cognition, 39(6), 1823.
Strijkers, K., Costa, A., & Thierry, G. (2010). Tracking lexical access in speech produc-
tion: Electrophysiological correlates of word frequency and cognate effects. Cerebral Cortex,
20(4), 912–928.
Strijkers, K., Holcomb, P. J., & Costa, A. (2011). Conscious intention to speak proactively
facilitates lexical access during overt object naming. Journal of Memory and Language, 65(4),
345–362.
Tanner, D., Morgan-Short, K, & Luck, S. J. (2015). How inappropriate high-pass filters can pro-
duce artifactual effects and incorrect conclusions in ERP studies of language and cognition.
Psychophysiology, 52, 997–1009.
Taylor, W. L. (1953). “Cloze procedure”: A new tool for measuring readability. Journalism
Quarterly, 30, 415–433.
Tokowicz, N., & MacWhinney, B. (2005). Implicit and explicit measures of sensitivity to
violations in second language grammar: An event-related potential investigation. Studies in
Second Language Acquisition, 27(2), 173–204.
Tooley, K. M., Traxler, M. J., & Swaab, T. Y. (2009). Electrophysiological and behavioral ev-
idence of syntactic priming in sentence comprehension. Journal of Experimental
Psychology: Learning, Memory, and Cognition, 35(1), 19.
Urbach, T. P., & Kutas, M. (2002). The intractability of scaling scalp distributions to infer
neuroelectric sources. Psychophysiology, 39(6), 791–808.
Van Berkum, J. J. A., Brown, C. M., & Hagoort, P. (1999). Early referential context effects in
sentence processing: Evidence from event-related brain potentials. Journal of Memory and
Language, 41, 147–182.
Van Petten, C., & Luka, B. J. (2006). Neural localization of semantic context effects in electro-
magnetic and hemodynamic studies. Brain and Language, 97, 279–293.
Van Petten, C., & Luka, B. J. (2012). Prediction during language comprehension: Benefits, costs,
and ERP components. International Journal of Psychophysiology, 83(2), 176–190.
Van Petten, C., & Rheinfelder, H. (1995). Conceptual relationships between spoken words and
environmental sounds: Event-related brain potential measures. Neuropsychologia, 33(4),
485–508.
Van Turennout, M., Hagoort, P., & Brown, C. M. (1998). Brain activity during speaking: From
syntax to phonology in 40 milliseconds. Science, 280(5363), 572–574.
Van Vliet, M., Manyakov, N. V., Storms, G., Flas, W., Wiersema, J. R., & Van Hulle, M. M.
(2014). Response-related potentials during semantic priming: The effect of a speeded button
response task on ERPs. PloS One, 9(2), e87650.
Vissers, C. T. W., Chwilla, D. J., & Kolk, H. H. (2006). Monitoring in language perception: The
effect of misspellings of words in highly constrained sentences. Brain Research, 1106(1), 150–163.
Wlotko, E. W., & Federmeier, K. D. (2007). Finding the right word: Hemispheric asymmetries
in the use of sentence context information. Neuropsychologia, 45(13), 3001–3014.
Wlotko, E. W., & Federmeier, K. D. (2012a). So that’s what you meant! Event-related potentials
reveal multiple aspects of context use during construction of message-level meaning.
NeuroImage, 62(1), 356–366.
Wlotko, E. W., & Federmeier, K. D. (2012b). Age‐related changes in the impact of contextual
strength on multiple aspects of sentence comprehension. Psychophysiology, 49(6), 770–785.
Wlotko, E. W., Federmeier, K. D., & Kutas, M. (2012). To predict or not to predict: Age-related
differences in the use of sentential context. Psychology and Aging, 27(4), 975.
Chapter 4
St udy ing L ang uag e w i t h

F unctional Mag net i c
Resonance Im ag i ng   ( fMRI )
Stefan Heim and Karsten Specht
Why Study Language with fMRI?
Functional magnetic resonance imaging (fMRI) provides a noninvasive window

to the responses of the living human brain under well-defined circumstances. It
thus provides an opportunity to study the unimpaired neurobiology of language, in-
stead of relying on inferences from the localization, size, and shape of one or mul-
tiple lesions in patient populations. The latter historical approach is certainly
powerful. The careful application of symptom-lesion mapping by the forefathers of
modern neurolinguistics—Broca, Wernicke, Lichtheim, Déjerine, Geschwind, and
others—led not only to the idea that cognitive functions have their own loci in the
brain, but even more astonishingly to the notion of a network architecture with nodes
and connections (see, in this volume, Blumstein, Chapter 1, and Wilson, Chapter 2).
But fMRI, at present the most widely used neuroimaging technique, can do more.
It provides insights into the varying degrees to which a region like Broca’s area
contributes, for example, to varying degrees of syntactic complexity; it permits the
quantifiable assessment of the role of contralateral homologue areas despite a lesion
in language-sensitive cortex; it represents a wealth of oscillatory bio-signals over time,
which can be submitted to elaborate algorithms to detect correlations, causalities, and
even predictive coding of upcoming events. An additional advantage is that patients
with lesions are rare, whereas university students, the most frequently used “typical
healthy subjects,” are available in abundance. Recruiting samples from a healthy pop-
ulation also has the advantage that potential nuisance variables can be controlled,
stratified, or matched (e.g., age, gender, handedness, IQ, or even prior experience with
language experiments).
Studying Language with fMRI 73
There are very few limits to the creativity of the fMRI research approach (one being
the maximum amount of time participants can spend inside the scanner)—but there
are substantial physical limits to what is possible. The MRI scanner makes noise that may
obscure subtle manipulations in the auditory speech signal. This noise likewise makes it
difficult to monitor subjects’ vocal responses. The MRI scanner bore is narrow, so visu-
ally presented materials have limited extension in space and/or complexity due to typ-
ical presentation methods involving projection screens and mirrors. The MRI scanner
takes “snapshots” of the brain, so any movement may blur these pictures—and speaking
requires and causes head motion.
In this chapter, we dig a bit deeper into these problems and illustrate potential
solutions that can be implemented in experimental design. We also discuss technical
solutions that have been developed to address these problems from the hardware side.
Before doing so, however, it is necessary to understand what it is we do with fMRI. What
is the signal we measure? How do we measure it? And why does that cause the various
problems outlined in the preceding?
fMRI: BOLD Images for Bold

Experiments
As noted earlier, fMRI is a noninvasive neuroimaging technique for investigating

functional-structural relationships in the brain. Compared to other methods from the
field of neuroscience, fMRI is a relatively new method, and the neuroimaging commu-
nity recently celebrated the twentieth anniversary of the introduction of fMRI, after
the pioneering work in the late 1980s and the early 1990s (Kwong, 2012; Ogawa, Lee,
Kay, & Tank, 1990; Turner, 2012). The first decade of fMRI was predominantly marked
by methods, developments, and investigations of the neuronal and physiological
underpinnings of the BOLD signal (blood oxygenation level dependent). The second
decade was characterized by the application of fMRI to study brain functions in healthy
participants as well as groups of patients (brain mapping). Now in its third decade, fMRI
is not only becoming a standard clinical application; the methods also have become suf-
ficiently advanced that fMRI is used for creating network models, investigating dynamic
and causal effects within a detected network of brain areas.
The BOLD Signal
When fMRI first appeared on the scene, positron emission tomography (PET) was an es-
tablished and well-understood tool for functional neuroimaging, although not broadly
available and limited in its applicability. Even though the first studies on human fMRI
were published in the early 1990s, the idea of using the MRI technique for investigating
74 Stefan Heim and Karsten Specht
brain function predates this by many years, as nicely described by two of the pioneers in
this field (Kwong, 2012; Turner, 2012). The first fMRI studies were predominantly ded-
icated to the investigation of the BOLD effect that may in the first instance appear as
a “paradoxical” phenomenon: Although neuronal activity is known to consume blood
oxygen, increasing neuronal activity is accompanied by regional cerebral blood flow
(rCBF) increases such that more oxygenated blood flows into the activated brain area.
Since the extraction rate of oxygen (ΔCMRO2) does not increase to the same extent as
the blood flow, the relative concentration of oxygenated blood that passes “unused”
through an activated area actually increases. Since oxygenated blood (oxy-Hb) is dia-
magnetic, and deoxygenated blood (deoxy-Hb) is paramagnetic, a change in the ratio
of oxy-Hb to deoxy-Hb causes the local magnetic environment to change. This change
to the local magnetization (or “susceptibility”) can be detected as a differential BOLD
signal when a series of appropriate MRI images is acquired.
Underlying the BOLD effect is a multicomponent process that consists of both extra-
vascular and intravascular components, where changes in blood flow, blood volume, and
oxygen consumption are the main factors determining the extension and temporal dy-
namics of the signal (Bandettini, 2009; Buxton, 2012; Duong et al., 2003; Villringer, 2012).
However, the exact mechanisms and coupling between the different factors contributing
to the BOLD response are still not fully understood. To measure the BOLD signal, one
needs an imaging sequence that is both fast and sensitive enough to detect small changes
in brain tissue magnetization. Although a range of imaging sequences, including the fast-
low-angle-shot (FLASH; Frahm, Merboldt, & Hänicke, 1993; Frahm, Merboldt, Hänicke,
Kleinschmidt, & Boecker, 1994; Fransson, Krüger, Merboldt, & Frahm, 1997), are sen-
sitive to BOLD signal fluctuations, today a gradient-echo echo-planar-imaging (EPI)
sequence is used almost exclusively (Bandettini, Wong, Hinks, Tikofsky, & Hyde, 1992;
Bandettini, Wong, Jesmanowicz, Hinks, & Hyde, 1994; Brüning et al., 1995; Kwong, 2012;
Turner, 2012). The advantage of EPI is that it can acquire an entire brain slice in less than
100 milliseconds (ms). Hence the entire brain can be covered in less than 3 seconds by
conventional MRI systems. An fMRI study that lasts, for example, 15 minutes can easily
collect more than 300 whole brain scans. However, EPI sequences are not only sensitive
to the BOLD effect, but also to other susceptibility “artifacts” (i.e., signal changes unre-
lated to brain activity). Consequently, whole-brain fMRI raw data are geometrically dis-
torted to some extent. In addition, there are brain areas that suffer from relative signal
loss due to magnetic susceptibility artifacts at borders to air-filled cavities, such as ear
canals or above the vocal tract. Because of these artifacts, parts of the inferior temporal
gyrus and orbitofrontal cortex typically do not produce reliable fMRI signals. However,
a variety of methods have been developed and are now routinely applied to correct the
distortions and partially recover the MRI signal in these areas.
Although the physiological mechanisms contributing to the BOLD effect are still
not fully understood, the overall time course of the signal is well characterized. Due
to its hemodynamic origins, the BOLD signal is a very smooth signal, not properly re-
flecting the sharpness of neuronal activation. Immediately after the onset of neuronal
activation, a small undershoot in the BOLD signal may occur—although more often
observed in animal than in human studies—followed by a strong signal increase. The
latter reaches its maximum typically 4–6 seconds after the onset of the neuronal ac-
tivation. Subsequently, the signal decays over a period of 15–20 seconds, including a
prominent post-stimulus undershoot, before it returns to baseline (see Figure 4.1 A).
Several different mechanisms have been proposed as responsible for this character-
istic time course. Although a consideration of the mechanisms is beyond the scope
of this chapter, the interested reader is referred to recent papers using biophysical
(A)
Stimulus presentation
Hemodynamic (BOLD) response

Undershoot
Initial dip
Peak
4–6 sec
Stimulus
Onset
(B)
Continuous acquisition
TR
Volume Volume Volume Volume Volume Volume Volume Volume Volume Volume
1 2 3 4 5 6 7 8 9 10
TR
Sparse sampling
TA
Volume Volume Volume Volume Volume

1 2 3 4 5
Figure 4.1. (A) Schematic display of the BOLD response in relation to the stimulus onset.
(B) Two examples of data acquisition schemes, with (top) continuous acquisition, where one
volume is immediately followed by the next with no detectable silent gap for the participant,
and (bottom) sparse sampling with a silent gap that might be used for stimulus presentation or
recording of a verbal response.
Abbreviations: TR = repetition time; TA = acquisition time of one volume.
models (e.g., Buxton, 2012; Buxton, Wong, & Frank, 1998; Hua, Stevens, Huang, Pekar,
& van Zijl, 2011).
The recent years have also shown an increasing interest in combining fMRI data
with metabolic data, such as rCBF. Until recently, this was accessible only with either
a contrast agent–based perfusion MRI or with PET, with the disadvantage of injecting
a tracer. With the advent of the arterial-spin labeling (ASL) technique, one is able to
perform perfusion measurements within the MR scanner and even within the same
session as the fMRI examination (Krieger, Huber, Poser, Turner, & Egan, 2015; Viviani,
Messina, & Walter, 2011; Wolf & Detre, 2007). In fact, various options are available, since
ASL could be performed as just a single measurement, as a dynamic perfusion examina-
tion, similar to BOLD-based fMRI (Hocking, McMahon, & de Zubicaray, 2009a), or as
a dynamic supplement to fMRI-based BOLD data with additional information on task-
related perfusion changes or during resting state (Viviani et al., 2011). With respect to
studies on speech production, it also has been demonstrated that ASL is less susceptible
to movement and image artifacts in an overt speech production paradigm than BOLD
fMRI (Kemeny, Ye, Birn, & Braun, 2005), making this an interesting option for studying
speech production.
Variability and Reliability

Group fMRI studies reflect inter-and intra-individual variability in BOLD signal
responses, even in healthy subjects. Some sources of variability under experimenter
control can be differences in cardiovascular fitness (affecting the efficiency of blood ox-
ygen uptake/use), and use of caffeine and nicotine. The latter are vasoactive substances,
with caffeine in particular being a BOLD signal enhancer (Bartsch, Homola, Biller,
Solymosi, & Bendszus, 2006; Honey & Bullmore, 2004). In general, one has to bear in
mind that the fMRI signal reflects changes in the ratio of deoxy-Hb to oxy-Hb. This
is of special importance in clinical cases, where the cerebrovasculature is typically
compromised. Several different methods for assessing the reliability of fMRI have been
put forward, ranging from global measures, such as overlap estimates of activations
across different occasions (Rombouts et al., 1997; Rombouts, Barkhof, Hoogenraad,
Sprenger, & Scheltens, 1998), receiver operating characteristic (ROC) curves (Chen &
Small, 2007), to the intra-class correlation coefficient (ICC) (Specht, Willmes, Shah, &
Jäncke, 2003). In particular, the ICC has been widely used in assessing test-retest re-
liability of fMRI for various paradigms and groups of participants (Aron Gluck, &
Poldrack, 2006; Fernandez et al., 2003; Hjelmervik, Hausmann, Osnes, Westerhausen, &
Specht, 2014; Plichta et al., 2012; Specht, Willmes, et al., 2003; van den Noort, Specht,
Rimol, Ersland, & Hugdahl, 2008). In general, the results demonstrate that the highest
reliability is achieved by employing paradigms with focused attention, while more pas-
sive (i.e., resting) paradigms demonstrate lower reliability. This is of importance for
clinical applications where it might be difficult to instruct and motivate patients appro-
priately and therefore passive paradigms are often used. However, with the advent of
“online” analyses on the MR scanner console, quality control is possible, and activation
strength/patterns can be monitored during the ongoing fMRI examination (Fernández
et al., 2001; Friedman et al., 2008; Specht, Ersland, et al., 2003; Specht, Scheffler, Reinartz,
& Reul, 2003).
Experimental Designs
Study Design and Implementation

The presentation of stimuli in an fMRI study is typically controlled by trigger signals
from the MR scanner to synchronize them with data acquisition. Most commonly used
are block designs or event-related designs. A block design describes an experimental
setting with trains of stimuli that belong to the same condition and are presented in
succession (e.g., over a period of 30 seconds), alternated with periods of no stimula-
tion or presentation of a control condition, respectively (Matthews & Jezzard, 2004).
An event-related design, by contrast, is based on the presentation of single trials of
one or more conditions in an arbitrary sequence (Dale, 1999; Rosen, Buckner, & Dale,
1998). Event-related designs often have a lower statistical efficiency than a block design,
due to the smaller signal changes generated, but this depends also on the timing of the
different trials (Mechelli, Price, Henson, & Friston, 2003). But there are strategies for
increasing the efficiency of event-related designs to reliably generate a detectable BOLD
signal. One approach to increasing the efficiency of an event-related design with rapid
presentation of trials is to use a variable inter-stimulus interval (ISI), called jittering
(Dale, 1999). In such a design, trials are presented with randomly varied intervals of 2
seconds or more, which causes predictable fluctuations of the BOLD signal that can be
modeled in the statistical analysis. This increases the detectability of the BOLD signal as
compared to a steady-state situation with a short, but constant, ISI. Ultimately, the selec-
tion of an appropriate experimental design depends not only on the research question,
but also on whether the task may be simply inappropriate for a block-or event-related
design (Matthews & Jezzard, 2004).
The most common fMRI study design employs a method called cognitive subtraction.
Although it has often been criticized (Friston et al., 1996; Sartori & Umiltà, 2000), it is
still the most established and supported method. The core design aspect is to generate
two conditions/tasks that differ only in the cognitive component of interest, so that the
components of no interest can be “subtracted.” It involves the assumption of “pure inser-
tion” (i.e., that each cognitive component evokes its own response and that the measured
signal is just a linear sum of the differences). For example, if the goal is to study the brain
regions responsible for processing phonetic information, one could create a task con-
dition with phonetic stimuli and another with acoustically matched but nonphonetic
stimuli, and so explore the differences between the two—an approach that results in re-
liable activation within left superior temporal sulcus (Specht, Osnes, & Hugdahl, 2009;
Specht & Reul, 2003). Alternatively, one might also combine a block or event-related
design with a parametric modulation (Sternberg, 1969). Here, a stimulus category is fac-
torially varied in one dimension, such as loudness, complexity, or task difficulty. One ex-
ample that falls under this category is the so-called sound morphing paradigm in which a
stimulus gradually changes its sound quality so that it “morphs” from white noise into a
recognizable speech sound (Osnes, Hugdahl, & Specht, 2011; Specht et al., 2009; Specht,
Rimol, Reul, & Hugdahl, 2005).
A pragmatic limitation of fMRI is the duration of a study. Since an experiment needs
a certain number of trials per condition to reliably measure the evoked BOLD signal
(e.g., 5 blocks of 5 trials per condition for a block design, or 30 trials per condition for an
event-related design), one is forced to limit the research questions that should and could
be addressed by a single study. Therefore, fMRI studies should be limited to only one or
two research questions that could be addressed within one study setup. The main reason
for this is that participants become fatigued in this unnatural situation of lying in a
scanner and performing cognitive tasks. Consequently, cognitive and task performance
at the beginning of the study might be different from that at the end. Another important
and often overlooked factor is the instruction given to the participants, as demonstrated
in a couple of studies that used manipulated speech-like sounds where specific informa-
tion about the stimuli were given or not, thus creating different expectancies that can
alter the activation pattern (Dufor, Serniclaes, Sprenger-Charolles, & Demonet, 2007;
Osnes, Hugdahl, Hjelmervik, & Specht, 2012). Therefore, standardized instructions are
a particular prerequisite for limiting the inter-individual variability.
With respect to language studies, especially those with auditory stimuli and/or overt
responses, fMRI has additional limitations. The most prominent limitation is the am-
bient scanner noise that limits audibility of acoustic stimuli. New hardware solutions
are under development to provide active noise cancellation during scanning, and it is
reasonable to assume that it is just a matter of time until reliable solutions are avail-
able. Alternatively, research studies on speech perception can overcome the problem
of scanner noise by using a special scanning technique involving inserting brief silent
gaps in the data acquisition, also called a sparse sampling approach (Hall et al., 1999;
Perrachione & Ghosh, 2013; van den Noort et al., 2008) (see Figure 4.1 B). While this has
obvious advantages with respect to the audibility of stimuli, it introduces several new
limitations. In event-related designs, options for “jittering” of the stimulus presentations
are limited, and as one collects a discontinuous time series, analysis approaches that try
to fit a hemodynamic response function to the data are inappropriate due to missing
time points. In the case of longer silent gaps, one has to optimize the stimulus presen-
tation to hit the peak of the BOLD signal, taking into account the next data acquisition
(Perrachione & Ghosh, 2013; van den Noort et al., 2008).
A critical aspect in fMRI design of neurolinguistic studies involves the choice of re-
sponse modality, since different types of responses, such as a simple button press,
overt verbal responses, covert/withheld verbal responses, or passive listening, will
add their specific signature to the underlying cognitive processes that are targeted by
the study. Examples are given by Indefrey and Levelt (2004) in their meta-analysis of
neuroimaging studies of speech production (see, e.g., their Figure 1). Continuous im-
aging acquisitions involving designs that require an overt verbal response suffer not
only in terms of recording during scanner noise, making responses often unintelligible;
overt responses also create massive magnetic susceptibility-related artifacts in the fMRI
images through the movement of the head, the articulators, and the varying volume of
the air-filled cavities (Birn, Bandettini, Cox, Jesmanowicz, & Shaker, 1998; Birn, Cox, &
Bandettini, 2004). Here, a sparse sampling design is recommended because it ensures
that the overt response happens only during the silent gap in imaging (cf. de Zubicaray,
Wilson, McMahon, & Muthiah, 2001, Hocking, McMahon, & de Zubicaray, 2009b, for
an early implementation; van den Noort et al., 2008; or the seminal paper by Gracco,
Tremblay, & Pike, 2005). Processing tools that can correct for deformations of EPI
images caused by movements, a so-called unwarp procedure, may also help in reducing
the extent of those artifacts once the data are acquired (Andersson, Hutton, Ashburner,
Turner, & Friston, 2001).
Data Analysis
Once a data set is collected, which might consist of several hundred EPI scans, the data
must be processed and analyzed statistically. For this analysis, various software packages are
available (e.g., SPM: http://www.fil.ion.ucl.ac.uk/spm; FSL: https://fsl.fmrib.ox.ac.uk/fsl/
fslwiki; Free Surfer: https://surfer.nmr.mgh.harvard.edu; AFNI: https://afni.nimh.nih.gov;
Brain Voyager: www.brainvoyager.com; and LIPSIA: http://www.cbs.mpg.de/institute/
software/lipsia). Processing usually starts with a quality check of the data and correction
of artifacts. First of all, it must be assured that no obvious imaging artifacts are present in
the data, which can occur as “stripes” in the images, distorted or blurred images, and so on.
Those artifacts can have many different sources, like technical problems of the MR scanner,
sudden and strong movements of the participant, or radio frequency (RF) leakage into the
scanner room. Although most artifacts are not able to be corrected and require the data set
to be discarded, some can be. Routinely, all data sets are corrected for head movements,
since they always occur but should not exceed more than 1–2 millimeters (mm). Other
artifacts (e.g., spikes) can be subjected to special tools to reduce their influence (Mazaika,
Whitfield, & Cooper, 2005). Processing steps that are especially relevant for group studies
are the normalization to a standard reference brain, and Gaussian smoothing of the data
to improve signal-to-noise ratio and to reduce persistent anatomical differences between
participants and to guarantee a certain amount of smoothness, assumed by, for example,
the Gaussian-Random Field theory that estimates corrected p-values (Worsley et al., 1996).
From Univariate to Multivariate Analyses

The most common way of analyzing fMRI data is to specify a general linear model
(GLM) for each subject’s data, where a predicted time course is created that is based on
the individual stimulus onset times, stimulus duration, and/or response data. By fitting
this hypothesized time course voxel-wise to the fMRI data, one obtains an estimate for
each voxel that reflects how well this model fits the real data (Friston, Frith, Frackowiak,
& Turner, 1995; Worsley, 1997). Next, these estimates are subjected to analysis at the
group level to identify areas that show a significant relationship between predicted and
measured time courses across the population.
However, one limitation of this approach is that the analysis is performed for each
voxel separately and independently. Thus, the results reflect only differences in activa-
tion “strength” on a voxel-by-voxel level. This is a limitation, as task effects might be
reflected more in specific activation patterns. In contrast, multivariate analysis strategies
are more sensitive, as they analyze the entire data space—across voxels and participants.
One multivariate method of increasing importance is the independent component anal-
ysis (ICA). Here, the entire subject-voxel-time data space can be subjected to the ICA
analysis (Beckmann, 2012; Beckmann & Smith, 2005; Calhoun, Liu, & Adali, 2009). An
ICA applies higher order statistics to dynamic data and relies on the assumption that the
fMRI signal is a linear mixture of hidden sources. However, the true number of hidden
sources is typically not known and has to be approximated by using algorithms, such as
the minimum description length (MDL) (Calhoun, Adali, Pearlson, & Pekar, 2001). The
aim of the ICA algorithm is to estimate the mixing matrix, given the number of expected
sources within the data set. Hence, the number of expected sources has to be predefined,
but no a priori hypotheses about possible spatiotemporal patterns or time courses are re-
quired at the initial stage. Further, ICA can be applied on the individual as well as group
level, using tensor-or probabilistic-based ICA (Beckmann, 2012; Beckmann & Smith,
2004, 2005) or concatenated spatial ICA (Calhoun, Adali, & Pekar, 2004; Calhoun et al.,
2009). The critical selection of relevant components is then based on sorting criteria,
using either spatial or temporal criteria. Note that an a priori hypothesis is required for
this final step; thus ICA is not a hypothesis-or model-free method, as is often claimed.
Further, an ICA analysis is often performed in conjunction with a general linear model
analysis as a complimentary approach to detect additional effects. For example, an anal-
ysis that is based on an ICA is able to separate different, but overlapping networks from
each other, like the bilateral network for auditory and phonological processing from that
of semantic and syntactic processing (Specht, Huber, Willmes, Shah, & Jäncke, 2008).
Going Meta: Data versus Database
Designing an fMRI study is challenging and rewarding: One has to control all known
nuisance variables and manipulate those that help to isolate the linguistic processes of
interest. Doing so, it is possible to address very specific research questions with highly
specialized designs. The flip side, however, is that one must scan enough participants
to have sufficient power to detect meaningful effects. Whereas in the year 2000 a
published block-design study could easily involve eight participants (e.g., Burton,
Small, & Blumstein, 2000), there is growing consensus that one needs much larger
sample sizes that match, or even exceed, those of behavioral studies (Button et al., 2013).
Underpowered fMRI studies are likely to produce erratic findings, in particular if a con-
sistency check (e.g., percent of participants showing the group brain activation pattern
at the individual level) is missing. Moreover, due to publication pressure, minor results
may get published, but fail to replicate. As the meta-analyses of neuroimaging studies
of language by Vigneau et al. (2006, 2011) demonstrated, there is hardly a single bit of
cortex that has not been activated in at least a handful of papers.
This potentially skewed picture can be addressed in two ways. One option is to run
studies with large samples and define the necessary sample size by a priori power anal-
ysis. The advantages have been discussed earlier, and the disadvantages are obvious: high
expense and a relatively low reward in terms of impact for a study with a long duration
that binds resources necessary for other research. The second option is not to acquire
data, but rather go to existing databases. Databases can either contain original raw data
from published experiments open for use and reanalysis by peers (for neuroimaging
data, e.g., OASIS, INDI, or OpenfMRI; cf. Button et al., 2013) or contain reported findings
from published studies in terms of peak coordinates and activation statistics per reported
contrast (e.g., BrainMap: www.brainmap.org; Neurosynth: http://neurosynth.org). Both
options are attractive as complementary ways to increase sample size, signal-to-noise
ratio, and thus statistical power in comparison to a single activation study, at negligible
expense. Good examples of database studies in the field of language imaging are the
meta-analysis of the semantic system by Binder and colleagues (Binder, Desai, Graves,
& Conant, 2009) and that of the speech production network by Eickhoff, Heim, Zilles,
& Amunts (2009). In the Binder study, 120 original studies were included, yielding data
from 1,642 individual volunteers. A potential problem with a meta-analysis of the “se-
mantic system” was that each group of authors might have a slightly different notion of
“semantics,” so Binder et al. had to define clear inclusion criteria. They were even able to
distinguish between “verbal” versus “perceptual” semantics based on the contrasts de-
fined in the original papers. Activation foci reported for these contrasts were then entered
into the Activation Likelihood Estimation (ALE) algorithm (Turkeltaub, Eden, Jones, &
Zeffiro, 2002) by which the probabilities and extents of activations can be assessed and
plotted in an image comparable to that of an original study. By this procedure, individual
clusters from individual studies only have a weighted contribution to the overall pattern,
which then reflects the relatively consistent involvement of regions in “semantic” tasks.
The validity of this approach was nicely demonstrated by Eickhoff et al. (2009). In
their study, ALE was applied to studies listed in the BrainMap data base. Those regions
surviving ALE could then be compared to newly acquired original data from their
own lab, which provided the advantage that subsequent connectivity analyses could
be run on these data. Most important for the purpose of this chapter, there was a con-
vincing match between the original data and the ALE results, representing reassuring
support for the application of meta-analyses. Pursuing this approach further, Clos,
Amunts, Laird, Fox, & Eickhoff (2013) used the same database to analyze co-activation
patterns of left area 44 in Broca’s region, finding a stable and functionally heterogeneous
parcellation. Such an analysis would scarcely have been possible if all data had to be
collected in one lab.
One Region, One Function?

Modules versus Networks
Broca’s (1861a, 1861b, 1863) finding that a circumscribed damage to the brain resulted in
a distinguishable deficit gave rise to the idea of the “siège du langage articulé,” the local-
ization of spoken language. Thirteen years later, Wernicke’s (1874) report of a different
lesion producing different language symptoms extended this idea, which soon led to the
notion of a language network in the brain (cf. Lichtheim, 1885): This model architecture
featured well-characterized regions in the brain, forming one large functional ensemble.
Subsequent neurological research investigated the functional characterization of
different parts of this network, rather than their interaction. The map of Karl Kleist
(1934) already had a remarkable precision in this respect, even including functional
subdivisions of the cytoarchitectonic areas identified by Korbinian Brodmann (1909)—
the more remarkable considering it was based purely on lesion studies. The advent of
neuroimaging techniques like PET and fMRI first only continued this tradition, now
being able to localize language and other functions in the healthy brain as well as in
patients. The Wernicke-Lichtheim idea of a functional network, which had its renais-
sance in Norman Geschwind’s works (e.g., 1970), was rather implicit (see Blumstein,
However, it attracted novel attention with both technical and mathematical advances.
The development of diffusion tensor imaging (DTI; see Catani & Forkel, Chapter 9 in
this volume), which allowed the investigation of fiber tracts in the living human brain,
led to the investigation of the hard-wired connections of the classic language areas
(Saur et al., 2008), the anatomical, white-matter based parcellation of Broca’s region
(Anwander, Tittgemeyer, von Cramon, Friederici, & Knösche, 2007), and the notion
of dorsal versus ventral processing streams (Saur et al., 2008) that develop differently
during ontogenesis (Brauer, Anwander, & Friederici, 2011; Friederici, Oberecker, &
Brauer, 2012). Perhaps even more intriguing was the introduction of algorithms such as
Psycho-Physiological Interaction (PPI) in the SPM software package and later Dynamic
Causal Modeling (DCM; Friston, Harrison, & Penny, 2003). With these algorithms,
novel insights could be gained into the communication of brain areas. With PPI, which
assesses partial correlations of BOLD time courses in different regions under certain
task contexts, Stephan et al. (2003) were able to distinguish a left hemisphere network
for letter processing from one in the right hemisphere supporting visuospatial orien-
tation. Most interestingly, the same stimuli were processed by both networks; the left
network processed the identity of the letter in a short word, whereas the right network
coded its position in the left or right hemifield.
DCM, developed again by the makers of SPM, proved even more powerful: Instead
of assessing mere functional connectivity (i.e. the spatiotemporal correlation of the
BOLD signal), it allows modeling of “effective” connectivity (i.e., the causal influence
that activation in region A exerts over that in regions B, C, D, etc., and vice versa). DCM
distinguishes three types of variables: inputs into the network that drive the energy
(which can be either sensory or internal, e.g., in a word generation task); the intrinsic
connections between regions (which can be uni-or bi-directional and may represent
facilitatory or inhibitory influences); and the modulation of their strength by contex-
tual factors such as the presence versus absence of a task, the differential complexity
of stimuli, and so on. With DCM, it becomes possible to understand the differen-
tial coupling of brain regions (e.g., Eickhoff et al., 2009; Mechelli et al., 2005; Specht,
Baumgartner, Stadler, Hugdahl, & Pollmann, 2014), why brain regions are up-regulated
under certain conditions (Osnes et al., 2011), or whether different brain regions have
different functions in a language task, even in the absence of any BOLD amplitude
differences in the classical GLM analysis (e.g., Abel et al., 2011). DCM is based on
Bayesian statistics, calculating dependent probabilities of the network parameters given
the present set of empirical data. These data are usually BOLD time courses extracted
from coordinates, spheres, or anatomically defined regions of interest (ROIs) based
on findings in the literature or the preceding GLM analysis. Usually, a set of plausible
DCM models is tested in parallel, and Bayesian model selection is used to pick the one
model with the best fit. There are various ways to define the model space, which can
be done either on the basis of plausibility considerations, in different model “families”
each featuring, for example, a unique connectivity pattern but different modulations,
or a-theoretically by simply permuting all possible parameters. The current approach
is typically limited to a maximum of eight, because each node features a self-inhibitory
connection meant to prevent the overall energy in the model from increasing infinitely.
Including more than eight regions is theoretically possible, but one buys these addi-
tional degrees of freedom with an increasing amount of inhibition in the model, which
at some point may simply drain all its energy. These technicalities being explained (and
the reader being referred, e.g., to Stephan et al.’s “Ten Simple Rules for Dynamic Causal
Modeling,” 2010), we will now give some examples of the usefulness of DCM in the
realm of language processing.
For instance, Osnes et al. (2011) investigated the hypothesis that the left premotor
cortex plays a crucial role in the identification of speech sounds. They morphed audi-
tory music or speech sounds with white noise, thus parametrically varying the amount
of music or speech in each auditory stimulus. Participants found performing decisions
with respect to whether they heard a “music” or a “speech sound” increasingly easy, but
performance at one proportion of signal-to-white noise stuck out: here, participants
were already pretty confident about hearing a speech sound when it was in the stimulus,
but less confident when they judged the presence of music. Remarkably, it was at that
point that the left premotor cortex was involved in, and presumably assisting, speech
sound processing. The analysis with DCM now revealed the mechanism underlying this
effect. For all auditory stimuli, a network of left Heschl’s gyrus, planum temporale, and
the superior temporal sulcus (STS) closely interacted. In the presence of auditory speech
sounds, however, there was an additional connection from the planum temporale to the
premotor cortex, representing the propagation of the signal from the sensory into the
motor domain. In addition, the premotor cortex then established a bi-directional con-
nection to the STS for further analysis of the signal.
This example from speech perception illustrates how connectivity analysis adds to the
understanding of the functional roles of individual regions within a network when there
is already evidence that region A may be involved in process X but not Y. However, con-
nectivity analysis can even reveal latent differences hidden under seemingly comparable
activation patterns in different conditions. One example from the domain of spoken
language is the study by Abel et al. (2011), who investigated group differences between
right-handed, left-handed, or ambidextrous participants during picture naming. In
their original analysis of the data, the GLM revealed no group differences whatsoever—
a strange and unexpected finding given ample reports of different laterality in left-
handers. Therefore, DCM was used in order to test whether laterality differences for
handedness could be found in the connectivity patterns, rather than in the amplitude
of the BOLD signal. The model space included three models, each featuring posterior
(i.e., the left and right fusiform gyrus) and anterior (i.e., area 44 in Broca’s region and
its right homologue) language regions. The models differed with respect to the nature
of the interhemispheric connections, which could be present or absent between the
posterior regions, anterior regions, or from posterior to anterior. Bayesian model se-
lection revealed one solution for the entire group of participants, which was replicated
for each of the three groups of handedness, implying that the basic underlying con-
nectivity structure was comparable. Most interestingly, however, for the right-handers
there was a strong intra-hemispheric coupling in the left hemisphere, whereas a mir-
rored pattern with additional inclusion of Broca’s region was found for the left-handers.
These findings again demonstrate the usefulness of connectivity analysis for detecting
differences in activation patterns beyond mere variations in BOLD amplitude.
The Resting Brain: Low-L evel Baseline

or Novel Information?
In the light of the potential problems facing language studies with patients, their
number is nonetheless increasing. A closer look reveals that many of these studies do
not actually run fMRI tasks, but rather are resting-state studies. In the early PET and
fMRI studies, “rest” was used as a low-level baseline condition that could come in
the form of null events (i.e., empty trials) or longer blocks with no stimulation or re-
sponse. Marcus Raichle discovered at one point that the statistical maps for these resting
conditions bore surprising resemblances to one another independent of the actual acti-
vation task (Raichle et al., 2001). This systematicity was taken to reflect a default mode
network (i.e., the co-activation of regions functionally connected in the brain at rest).
Soon, differences in the default mode network were observed for certain disorders, and
resting-state connectivity emerged as a novel variable to assess brain networks in the ab-
sence of an activation task (Hugdahl, Raichle, Mitra, Specht, 2015).
With resting-state fMRI, one was not limited to investigations of the default mode
network. One development involved taking a well-known “language region” previously
identified in a classical fMRI language study (e.g., Broca’s area) as a seed region and
analyzing which other parts of the brain show temporal correlations with the resting-
state fMRI time series in it (e.g., Muller & Meyer, 2014). This procedure was termed
seed correlation analysis. In the Muller and Meyer (2014) study, it revealed a connec-
tivity pattern that nicely resembled activation patterns found in language-task studies.
The authors took this overall connectivity pattern as the starting point for subsequent
in-depth analyses as to, for example, the cluster structure (identified by ICA) and de-
rived novel insights into the functional nature of Broca’s homologue in the right infe-
rior frontal cortex—which might then be verified again with language-task fMRI. The
validity of this approach was confirmed by Zhu et al. (2014), who explicitly looked at its
reliability.
The advantages of this approach are obvious. One may investigate language-related
questions or networks even if the participants cannot perform a task to a sufficient de-
gree inside the magnet. Different groups of participants with distinct language profiles
can be compared (e.g., Weiler et al., 2014; Zhang et al., 2014), and therapy-induced
effects may be traced (Ferguson, Nielsen, & Anderson, 2014).
Yet, the resting-state approach probably cannot exist without task-related fMRI. The
fact that, for example, area 44 is involved in language processing does not mean there
is an isomorphism, a one-to-one mapping of structure and function. Meta-analyses
have revealed the various functions that area 44 can support (cf. Clos et al., 2013). In
addition, task-related connectivity analysis has taught us that the functional interac-
tion of brain regions may vary drastically in different contexts (cf. the DCM studies
by Heim, Eickhoff, & Amunts, 2009, vs. Heim, Eickhoff, Ischebeck, et al., 2009). Thus,
looking at the resting-state connectivity of a single region does provide insight into the
network of other regions it communicates with—at rest. Whether these connections
are maintained, how they are intensified or blocked, and whether there is competi-
tion or cooperation between connected regions are matters that need to be addressed
in well-conducted task-related fMRI studies of language processing, ideally with clear
hypotheses and a direct reliance on linguistic theory.
Multimodality Studies
In recent years, fMRI has been combined with other neuroimaging methods, in-
cluding electroencephalography (EEG) or MR spectroscopy. Simultaneous recording
of EEG and fMRI combines the high spatial resolution of fMRI with the high temporal
resolution of EEG (see Leckey & Federmeier, Chapter 3 in this volume). Although this
is a technically challenging approach, MR-compatible devices and analysis tools are
available for performing multimodal integration studies (Debener, Ullsperger, Siegel,
& Engel, 2006; Eichele, Calhoun, & Debener, 2009; Eichele et al., 2008; Herrmann &
Debener, 2008; Moosmann, Eichele, Nordby, Hugdahl, & Calhoun, 2008; Moosmann
et al., 2009; Mullinger, Yan, & Bowtell, 2011).
Functional MRI and MR spectroscopy (MRS) have coexisted as MR applications
for many years, but only in recent years have researchers started combining these two
sources of information in humans (Falkenberg et al., 2014; Falkenberg, Westerhausen,
Specht, & Hugdahl, 2012; Horn et al., 2010; van Wageningen, Jørgensen, Specht, &
Hugdahl, 2010; Yücel et al., 2007), despite their application in animal studies for some
time (Hyder et al., 2001). Scanners with a field strength of 3 Tesla or above are capable of
reliably measuring the concentration of neurotransmitters, like glutamate, which is one
of the most important excitatory neurotransmitters in the brain. There is increasing evi-
dence that the regional concentration of glutamate not only is related to inter-individual
behavioral differences, but also may influence the fMRI signal in remote brain areas
(Falkenberg et al., 2012; Falkenberg et al., 2014; van Wageningen et al., 2010). To assess
the concentration of the most important inhibitory neurotransmitter, namely gamma-
aminobutyric acid (GABA), one needs a special sequence, like MEGA-PRESS (Henry,
Lauriat, Shanahan, Renshaw, & Jensen, 2011). GABA is of particular interest, since in-
hibitory mechanisms are related to many central functions, and its influence on the
BOLD signal has been recently demonstrated (Muthukumaraswamy, Edden, Jones,
Swettenham, & Singh, 2009; Muthukumaraswamy, Evans, Edden, Wise, & Singh, 2012).
Conclusion
Neuroimaging techniques such as fMRI have expanded our insight into the com-
plex neuronal architecture and mechanics by which language is processed in brain
regions that form networks. Still, many technological issues need to be solved before
we reach spatial and temporal resolutions sufficient to allow modeling of the neuronal
interactions supporting speech and language use, or develop protective and therapeutic
strategies for more efficient recovery from language-relevant brain damage.
References
Abel, S., Huber, W., Weiller, C., Amunts, K., Eickhoff, S., & Heim, S. (2011). The influence of
handedness on hemispheric interaction during word production: Insights from effective
connectivity analysis. Brain Connectivity, 1, 219–231.
Andersson, J. L., Hutton, C., Ashburner, J., Turner, R., & Friston, K. J. (2001). Modeling geo-
metric deformations in EPI time series. NeuroImage, 13, 903–919.
Anwander, A., Tittgemeyer, M., von Cramon, D. Y., Friederici, A. D., & Knösche, T. R. (2007).
Connectivity-based parcellation of Broca’s area. Cerebral Cortex, 17, 816–825.
Aron, A. R., Gluck, M. A., & Poldrack, R. A. (2006). Long-term test-retest reliability of func-
tional MRI in a classification learning task. NeuroImage, 29, 1000–1006.
Bandettini, P. A. (2009). What’s new in neuroimaging methods? Annals of the New York
Academy of Sciences, 1156, 260–293.
Bandettini, P. A., Wong, E. C., Hinks, R. S., Tikofsky, R. S., & Hyde, J. S. (1992). Time course
EPI of human brain function during task activation. Magnetic Resonance in Medicine, 25,
390–397.
Bandettini, P. A., Wong, E. C., Jesmanowicz, A., Hinks, R. S., & Hyde, J. S. (1994). Spin-echo and
gradient-echo EPI of human brain activation using BOLD contrast: A comparative study at
1.5 T. NMR Biomedical, 7,  12–20.
Bartsch, A. J., Homola, G., Biller, A., Solymosi, L., & Bendszus, M. (2006). Diagnostic func-
tional MRI: Illustrated clinical applications and decision-making. Journal of Magnetic
Resonance Imaging, 23, 921–932.
Beckmann, C. F. (2012). Modelling with independent components. NeuroImage, 62, 891–901.
Beckmann, C. F., & Smith, S. M. (2004). Probabilistic independent component analysis for
functional magnetic resonance imaging. IEEE Transactions in Medical Imaging, 23, 137–152.
Beckmann, C. F., & Smith, S. M. (2005). Tensorial extensions of independent component anal-
ysis for multisubject FMRI analysis. NeuroImage, 25, 294–311.
19, 2767–2796.
Birn, R. M., Bandettini, P. A., Cox, R. W., Jesmanowicz, A., & Shaker, R. (1998). Magnetic field
changes in the human brain due to swallowing or speaking. Magnetic Resonance in Medicine,
40,  55–60.
Birn, R. M., Cox, R. W., & Bandettini, P. A. (2004). Experimental designs and processing
strategies for fMRI studies involving overt verbal responses. NeuroImage, 23, 1046–1058.
Brauer, J., Anwander, A., & Friederici, A. D. (2011). Neuroanatomical prerequisites for lan-
guage functions in the maturing brain. Cerebral Cortex, 21, 459–466.
Broca, P. (1861a). Nouvelle observation aphémie produite par un lésion de la moité postérieure
des deuxième et troisième circonvolutions frontales. Bulletins de la Socie´te´ Anatomique, 6,
398–407.
Broca, P. (1861b). Remarques sur le siége de la faculté du langage articulé, suivies d’une
observatoin d’aphémie (Perte de la Parole). Bulletin de la Société Anatomique de Paris, 6,
330–357.
Broca, P. (1863). Localisation des fonctions cérébrales: Siège du langage articulé. Bulletins de
la Société d’Anthropologie, 4, 200–204.
Brodmann, K. (1909). Beiträge zur histologischen Lokalisation der Grosshirnrinde. VI. Die
Cortexgliederung des Menschen. Journal für Psychologie und Neurologie, 10, 231–246.
Brüning, R., Weber, J., Wu, R. H., Kwong, K. K., Hennig, J., & Reiser, M. (1995). Echo-planar
imaging des gehirns [Echo-planar imaging of the brain]. Radiologe, 35, 902–910.
Burton, M. W., Small, S. L., & Blumstein, S. E. (2000). The role of segmentation in phonological
processing: An fMRI investigation. Journal of Cognitive Neuroscience, 12, 679–690.
Button, K. S., Ioannidis, J. P., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S., & Munafò, M.
R. (2013). Power failure: Why small sample size undermines the reliability of neuroscience.
Nature Reviews Neuroscience, 14, 365–376.
Buxton, R. B. (2012). Dynamic models of BOLD contrast. NeuroImage, 62, 953–961.

Buxton, R. B., Wong, E. C., & Frank, L. R. (1998). Dynamics of blood flow and oxygenation
changes during brain activation: The balloon model. Magnetic Resonance in Medicine, 39,
855–864.
Calhoun, V. D., Adali, T., Pearlson, G. D., & Pekar, J. J. (2001). A method for making group
inferences from functional MRI data using independent component analysis. Human Brain
Mapping, 14, 140–151.
Calhoun, V. D., Adali, T., & Pekar, J. J. (2004). A method for comparing group fMRI data
using independent component analysis: Application to visual, motor and visuomotor tasks.
Magnetic Resonance Imaging, 22, 1181–1191.
Calhoun, V. D., Liu, J., & Adali, T. (2009). A review of group ICA for fMRI data and ICA for
joint inference of imaging, genetic, and ERP data. NeuroImage, 45, S163–S172.
Chen, E. E., & Small, S. L. (2007). Test-retest reliability in fMRI of language: Group and task
effects. Brain and Language, 102, 176–185.
Clos, M., Amunts, K., Laird, A. R., Fox, P. T., & Eickhoff, S. B. (2013). Tackling the multifunc-
tional nature of Broca’s region meta-analytically: Co-activation-based parcellation of area
44. NeuroImage, 83, 174–188.
Dale, A. M. (1999). Optimal experimental design for event-related fMRI. Human Brain
Mapping, 8, 109–114.
Debener, S., Ullsperger, M., Siegel, M., & Engel, A. K. (2006). Single-trial EEG-fMRI reveals
the dynamics of cognitive function. Trends in Cognitive Sciences, 10, 558–563.
de Zubicaray, G. I., Wilson, S. J., McMahon, K. L., & Muthiah, S. (2001). The semantic inter-
ference effect in the picture-word paradigm: An event-related fMRI study employing overt
responses. Human Brain Mapping, 14, 218–227.
Dufor, O., Serniclaes, W., Sprenger-Charolles, L., & Demonet, J. F. (2007). Top-down processes
during auditory phoneme categorization in dyslexia: A PET study. NeuroImage, 34,
1692–1707.
Duong, T. Q., Yacoub, E., Adriany, G., Hu, X., Uğurbil, K., & Kim, S.-G. (2003). Microvascular
BOLD contribution at 4 and 7 T in the human brain: Gradient-echo and spin-echo fMRI
with suppression of blood effects. Magnetic Resonance in Medicine, 49, 1019–1027.
Eichele, T., Calhoun, V. D., & Debener, S. (2009). Mining EEG-fMRI using independent com-
ponent analysis. International Journal of Psychophysiology, 73,  53–61.
Eichele, T., Calhoun, V. D., Moosmann, M., Specht, K., Jongsma, M. L. A., Quiroga, R. Q.,
Nordby, H., & Hugdahl, K. (2008). Unmixing concurrent EEG-fMRI with parallel inde-
pendent component analysis. International Journal of Psychophysiology, 67, 222–234.
Eickhoff, S. B., Heim, S., Zilles, K., & Amunts, K. (2009). A systems perspective on the effective
connectivity of overt speech production. Philosophical Transactions of the Royal Society A,
367, 2399–2421.
Falkenberg, L. E., Westerhausen, R., Craven, A. R., Johnsen, E., Kroken, R. A., Løberg, E.-M.,
Specht, K., & Hugdahl, K. (2014). Impact of glutamate levels on neuronal response and cog-
nitive abilities in schizophrenia. NeuroImage Clinical, 4, 576–584.
Falkenberg, L. E., Westerhausen, R., Specht, K., & Hugdahl, K. (2012). Resting-state glutamate
level in the anterior cingulate predicts blood-oxygen level-dependent response to cognitive
control. Proceedings of the National Acadademy of Sciences of the USA, 109, 5069–5073.
Ferguson, M. A., Nielsen, J. A., & Anderson, J. S. (2014). Altered resting functional connec-
tivity of expressive language regions after speed reading training. Journal of Clinical and
Experimental Neuropsychology, 36, 482–493.
Fernandez, G., Specht, K., Weis, S., Tendolkar, I., Reuber, M., Fell, J., Klaver, P., Ruhlmann, J.,
Reul, J., & Elger, C. E. (2003). Intrasubject reproducibility of presurgical language lateraliza-
tion and mapping using fMRI. Neurology, 60, 969–975.
Fernández, G., de Greiff, A., Oertzen, von, J., Reuber, M., Lun, S., Klaver, P., Ruhlmann, J., Reul,
J., & Elger, C. E. (2001). Language mapping in less than 15 minutes: Real-time functional
MRI during routine clinical investigation. NeuroImage, 14, 585–594.
Frahm, J., Merboldt, K. D., & Hänicke, W. (1993). Functional MRI of human brain activation at
high spatial resolution. Magnetic Resonance in Medicine, 29, 139–144.
Frahm, J., Merboldt, K. D., Hänicke, W., Kleinschmidt, A., & Boecker, H. (1994). Brain or
vein—oxygenation or flow? On signal physiology in functional MRI of human brain activa-
tion. NMR Biomedical, 7,  45–53.
Fransson, P., Krüger, G., Merboldt, K. D., & Frahm, J. (1997). A comparative FLASH and EPI
study of repetitive and sustained visual activation. NMR Biomedical, 10, 204–207.
Friederici, A. D. (2012). Language development and the ontogeny of the dorsal pathway.
Frontiers in Evolutionary Neuroscience, 4, 3.
Friederici, A. D., Oberecker, R., & Brauer, J. (2012). Neurophysiological preconditions of syntax
acquisition. Psychological Research, 76, 204–211.
Friedman, L., Stern, H., Brown, G. G., Mathalon, D. H., Turner, J., Glover, G. H., Gollub, R.
L., Lauriello, J., Lim, K. O., Cannon, T., Greve, D. N., Bockholt, H. J., Belger, A., Mueller, B.,
Doty, M. J., He, J., Wells, W., Smyth, P., Pieper, S., Kim, S., Kubicki, M., Vangel, M., & Potkin,
S. G. (2008). Test-retest and between-site reliability in a multicenter fMRI study. Human
Brain Mapping, 29, 958–972.
Friston, K. J., Frith, C. D., Frackowiak, R. S., & Turner, R. (1995). Characterizing dynamic brain
responses with fMRI: A multivariate approach. NeuroImage, 2, 166–172.
Friston, K. J., Harrison, L., & Penny, W. (2003). Dynamic causal modelling. NeuroImage, 19,
1273–302.
Friston, K. J., Price, C. J., Fletcher, P., Moore, C., Frackowiak, R. S., & Dolan, R. J. (1996). The
trouble with cognitive subtraction. NeuroImage, 4, 97–104.
Geschwind, N. (1970). Organization of language in the brain. Science, 170, 940–944.
Gracco, V. L., Tremblay, P., & Pike, B. (2005). Imaging speech production using fMRI.
NeuroImage, 26, 294–301.
Hall, D. A., Haggard, M. P., Akeroyd, M. A., Palmer, A. R., Summerfield, A. Q., Elliott, M. R.,
Gurney, E. M., & Bowtell, R. W. (1999). “Sparse” temporal sampling in auditory fMRI.
Human Brain Mapping, 7, 213–223.
Heim, S., Eickhoff, S. B., & Amunts, K. (2009). Different roles of cytoarchitectonic BA 44 and
BA 45 in phonological and semantic verbal fluency as revealed by dynamic causal model-
ling. NeuroImage, 48, 616–624.
Heim, S., Eickhoff, S. B., Ischebeck, A. K., Friederici, A. D., Stephan, K. E., & Amunts, K. (2009).
Effective connectivity of the left BA 44, BA 45, and inferior temporal gyrus during lexical
and phonological decisions identified with DCM. Human Brain Mapping, 30, 392–402.
Henry, M. E., Lauriat, T. L., Shanahan, M., Renshaw, P. F., & Jensen, J. E. (2011). Accuracy
and stability of measuring GABA, glutamate, and glutamine by proton magnetic reso-
nance spectroscopy: A phantom study at 4 Tesla. Journal of Magnetic Resonance, 208,
210–218.
Herrmann, C. S., & Debener, S. (2008). Simultaneous recording of EEG and BOLD
responses: A historical perspective. International Journal of Psychophysiology, 67,
161–168.
Hjelmervik, H., Hausmann, M., Osnes, B., Westerhausen, R., & Specht, K. (2014). Resting
States are resting traits: An FMRI study of sex differences and menstrual cycle effects in
resting state cognitive control networks. PLoS ONE, 9, e103492.
Hocking, J., McMahon, K. L., & de Zubicaray, G. I. (2009a). Semantic context and visual fea-
ture effects in object naming: An fMRI study using arterial spin labeling. Journal of Cognitive
Neuroscience, 21, 1571–1583.
Hocking, J., McMahon, K. L., & de Zubicaray, G. I. (2009b). Semantic interference in object
naming: An fMRI study of the postcue naming paradigm. NeuroImage, 50, 796–801.
Honey, G., & Bullmore, E. (2004). Human pharmacological MRI. Trends in Pharmacological
Sciences, 25, 366–374.
Horn, D. I., Yu, C., Steiner, J., Buchmann, J., Kaufmann, J., Osoba, A., Eckert, U., Zierhut, K. C.,
Schiltz, K., He, H., Biswal, B., Bogerts, B., & Walter, M. (2010). Glutamatergic and resting-
state functional connectivity correlates of severity in major depression: The role of pregenual
anterior cingulate cortex and anterior insula. Frontiers in Systems Neuroscience, 4, 33.
Hua, J., Stevens, R. D., Huang, A. J., Pekar, J. J., & van Zijl, P. C. M. (2011). Physiological origin
for the BOLD poststimulus undershoot in human brain: Vascular compliance versus oxygen
metabolism. Journal of Cerebral Blood Flow and Metabolism, 31, 1599–1611.
Hugdahl, K., Raichle, M. E., Mitra, A., Specht, K. (2015). On the existence of a generalized non-
specific task-dependent network. Frontiers in Human Neuroscience, 9,  1–15.
Hyder, F., Kida, I., Behar, K. L., Kennan, R. P., Maciejewski, P. K., & Rothman, D. L. (2001).
Quantitative functional imaging of the brain: Towards mapping neuronal activity by BOLD
fMRI. NMR Biomedical, 14, 413–431.
Indefrey, P., & Levelt, W. J. M. (2004). The spatial and temporal signatures of word production
components. Cognition, 92, 101–144.
Kemeny, S., Ye, F. Q., Birn, R., & Braun, A. R. (2005). Comparison of continuous overt speech
fMRI using BOLD and arterial spin labeling. Human Brain Mapping, 24, 173–183.
Kleist, K. (1934). Gehirnpathologie. Leipzig: Johann Ambrosius Barth.
Krieger, S. N., Huber, L., Poser, B. A., Turner, R., & Egan, G. F. (2015). Simultaneous acquisi-
tion of cerebral blood volume-, blood flow-, and blood oxygenation-weighted MRI signals
at ultra-high magnetic field. Magnetic Resonance in Medicine, 7, 513–517.
Kwong, K. K. (2012). Record of a single fMRI experiment in May of 1991. NeuroImage, 62,
610–612.
Lichtheim, L. (1885). Ueber Aphasie: Aus der medicinischen Klinik in Bern. Deutsches Archiv
für klinische Medicin, Leipzig, 36, 204–268.
Matthews, P. M., & Jezzard, P. (2004). Functional magnetic resonance imaging. Journal of
Neurology, Neurosurgery and Psychiatry, 75,  6–12.
Mazaika, P. K., Whitfield, S., & Cooper, J. C. (2005). Detection and repair of transient artifacts
in fMRI data. NeuroImage, 26 (OHBM Abstract).
Mechelli, A., Price, C. J., Henson, R. N. A., & Friston, K. J. (2003). Estimating efficiency a
priori: A comparison of blocked and randomized designs. NeuroImage, 18, 798–805.
Mechelli, A., Crinion, J. T., Long, S., Friston, K. J., Lambon Ralph, M. A., Patterson, K.,
McClelland, J. L., & Price, C. J. (2005). Dissociating reading processes on the basis of neu-
ronal interactions. Journal of Cognitive Neuroscience, 17, 1753–1765.
Moosmann, M., Eichele, T., Nordby, H., Hugdahl, K., & Calhoun, V. D. (2008). Joint inde-
pendent component analysis for simultaneous EEG- fMRI: Principle and simulation.
International Journal of Psychophysiology, 67, 212–221.
Moosmann, M., Schönfelder, V. H., Specht, K., Scheeringa, R., Nordby, H., & Hugdahl, K.
(2009). Realignment parameter-informed artefact correction for simultaneous EEG-fMRI
recordings. NeuroImage, 45, 1144–1150.
Muller, A. M., & Meyer, M. (2014). Language in the brain at rest: New insights from resting
state data and graph theoretical analysis. Frontiers in Human Neuroscience, 8, 228.
Mullinger, K. J., Yan, W. X., & Bowtell, R. (2011). Reducing the gradient artefact in simulta-
neous EEG-fMRI by adjusting the subject’s axial position. NeuroImage, 54, 1942–1950.
Muthukumaraswamy, S. D., Edden, R. A. E., Jones, D. K., Swettenham, J. B., & Singh, K. D.
(2009). Resting GABA concentration predicts peak gamma frequency and fMRI amplitude
in response to visual stimulation in humans. Proceedings of the National Academy of Sciences
of the USA, 106, 8356–8361.
Muthukumaraswamy, S. D., Evans, C. J., Edden, R. A. E., Wise, R. G., & Singh, K. D. (2012).
Individual variability in the shape and amplitude of the BOLD-HRF correlates with endoge-
nous GABAergic inhibition. Human Brain Mapping, 33, 455–465.
Ogawa, S., Lee, T. M., Kay, A. R., & Tank, D. W. (1990). Brain magnetic resonance imaging with
contrast dependent on blood oxygenation. Proceedings of the National Academy of Sciences
of the USA, 87, 9868–9872.
Osnes, B., Hugdahl, K., Hjelmervik, H., & Specht, K. (2012). Stimulus expectancy modulates
inferior frontal gyrus and premotor cortex activity in auditory perception. Brain and
Language, 121,  65–69.
Osnes, B., Hugdahl, K., & Specht, K. (2011). Effective connectivity analysis demonstrates in-
volvement of premotor cortex during speech perception. NeuroImage, 54, 2437–2445.
Perrachione, T. K., & Ghosh, S. S. (2013). Optimized design and analysis of sparse-sampling
FMRI experiments. Frontiers in Neuroscience, 7, 55.
Plichta, M. M., Schwarz, A. J., Grimm, O., Morgen, K., Mier, D., Haddad, L., Gerdes, A. B. M.,
Sauer, C., Tost, H., Esslinger, C., Colman, P., Wilson, F., Kirsch, P., & Meyer-Lindenberg, A.
(2012). Test-retest reliability of evoked BOLD signals from a cognitive-emotive fMRI test
battery. NeuroImage, 60, 1746–1758.
Raichle, M. E., MacLeod, A. M., Snyder, A. Z., Powers, W. J., Gusnard, D. A., & Shulman, G. L.
(2001). A default mode of brain function. Proceedings of the National Academy of Sciences of
the USA, 98, 676–682.
Rombouts, S. A., Barkhof, F., Hoogenraad, F. G., Sprenger, M., Valk, J., & Scheltens, P. (1997).
Test-retest analysis with functional MR of the activated area in the human visual cortex.
American Journal of Neuroradiology, 18, 1317–1322.
Rombouts, S. A. R. B., Barkhof, F., Hoogenraad, F. G. C., Sprenger, M., & Scheltens, P. (1998).
Within-subject reproducibility of visual activation patterns with functional magnetic res-
onance imaging using multislice echo planar imaging. Magnetic Resonance Imaging, 16,
105–113.
Rosen, B. R., Buckner, R. L., & Dale, A. M. (1998). Event-related functional MRI: Past, present,
and future. Proceedings of the National Academy of Sciences of the USA, 95, 773–780.
Sartori, G., & Umiltà, C. (2000). How to avoid the fallacies of cognitive subtraction in brain im-
aging. Brain and Language, 74, 191–212.
Saur, D., Kreher, B. W., Schnell, S., Kümmerer, D., Kellmeyer, P., Vry, M. S., Umarova, R.,
Musso, M., Glauche, V., Abel, S., Huber, W., Rijntjes, M., Hennig, J., & Weiller, C. (2008).
Ventral and dorsal pathways for language. Proceedings of the National Academy of Sciences of
the USA, 105, 18035–18040.
Specht, K., Baumgartner, F., Stadler, J., Hugdahl, K. & Pollmann, S. (2014). Functional
asymmetry and effective connectivity of the auditory system during speech perception
is modulated by the place of articulation of the consonant: A 7T fMRI study. Frontiers in
Psychology, 5, 549.
Specht, K., Ersland, L., Andersen, E., Reul, J., Thomsen, T., & Hugdahl, K. (2003). Plug & play
fMRI. Rivista di Neuroradiologia, 16, 965–968.
Specht, K., Huber, W., Willmes, K., Shah, N. J., & Jäncke, L. (2008). Tracing the ventral stream
for auditory speech processing in the temporal lobe by using a combined time series and in-
dependent component analysis. Neuroscience Letters, 442, 180–185.
Specht, K., Osnes, B., & Hugdahl, K. (2009). Detection of differential speech-specific processes
in the temporal lobe using fMRI and a dynamic “sound morphing” technique. Human Brain
Mapping, 30, 3436–3444.
Specht, K., & Reul, J. (2003). Functional segregation of the temporal lobes into highly
differentiated subsystems for auditory perception: An auditory rapid event-related
fMRI-task. NeuroImage, 20, 1944–1954.
Specht, K., Rimol, L. M., Reul, J., & Hugdahl, K. (2005). “Soundmorphing”: A new approach to
studying speech perception in humans. Neuroscience Letters, 384,  60–65.
Specht, K., Scheffler, M., Reinartz, J., & Reul, J. (2003). Experiences and applicability of
presurgical real-time fMRI. Rivista di Neuroradiologia, 16, 1092–1096.
Specht, K., Willmes, K., Shah, N. J., & Jäncke, L. (2003). Assessment of reliability in functional
imaging studies. Journal of Magnetic Resonance Imaging, 17, 463–471.
Stephan, K. E., Marshall, J. C., Friston, K. J., Rowe, J. B., Ritzl, A., Zilles, K., & Fink, G. R. (2003).
Lateralized cognitive processes and lateralized task control in the human brain. Science, 301,
384–386.
Stephan, K. E., Penny, W. D., Moran, R. J., den Ouden, H. E. M., Daunizeau, J., & Friston, K. J.
(2010). Ten simple rules for dynamic causal modeling. NeuroImage, 49, 3099–3109.
Sternberg, S. (1969). Memory- scanning: Mental processes revealed by reaction- time
experiments. American Scientist, 57, 421–457
Turkeltaub, P. E., Eden, G. F., Jones, K. M., & Zeffiro, T. A. (2002). Meta-analysis of the func-
tional neuroanatomy of single-word reading: Methods and validation. NeuroImage, 16,
765–780.
Turner, R. (2012). The NIH experience in first advancing fMRI. NeuroImage, 62, 632–636.
van den Noort, M., Specht, K., Rimol, L.M., Ersland, L., & Hugdahl, K. (2008). A new
verbal reports fMRI dichotic listening paradigm for studies of hemispheric asymmetry.
van Wageningen, H., Jørgensen, H. A., Specht, K., & Hugdahl, K. (2010). A 1H-MR spectros-
copy study of changes in glutamate and glutamine (Glx) concentrations in frontal spectra
after administration of memantine. Cerebral Cortex, 20, 798–803.
Vigneau, M., Beaucousin, V., Hervé, P. Y., et al. (2006). Meta-analyzing left hemisphere
language areas: Phonology, semantics, and sentence processing. NeuroImage, 30,
1414–1 432.
Vigneau, M., Beaucousin, V., Hervé, P. Y., et al. (2011). What is right-hemisphere contribution
to phonological, lexico-semantic, and sentence processing? Insights from a meta-analysis.
Villringer, A. (2012). The intravascular susceptibility effect and the underlying physiology of
fMRI. NeuroImage, 62, 995–999.
Viviani, R., Messina, I., & Walter, M. (2011). Resting state functional connectivity in perfusion
imaging: Correlation maps with BOLD connectivity and resting state perfusion. PLoS ONE,
6, e27050.
Weiler, M., Fukuda, A., Massabki, L. H., Lopes, T. M., Franco, A. R., Damasceno, B. P., Cendes,
F., & Balthazar, M. L. (2014). Default mode, executive function, and languagefunctional
connectivity networks are compromised in mild Alzheimer’s disease. Current Alzheimer
Research, 11, 274–282.
Wernicke, C. (1874). Der Aphasische Symptomenkomplex. Breslau: Cohn & Weigart.
Wolf, R. L., & Detre, J. A. (2007). Clinical neuroimaging using arterial spin-labeled perfusion
magnetic resonance imaging. Neurotherapeutics, 4, 346–359.
Worsley, K. J. (1997). An overview and some new developments in the statistical analysis of
PET and fMRI data. Human Brain Mapping, 5, 254–258.
Worsley, K. J., Marrett, S., Neelin, P., Vandal, A. C., Friston, K. J., & Evans, A. C. (1996). A uni-
fied statistical approach for determining significant signals in images of cerebral activation.
Human Brain Mapping, 4,  58–73.
Yücel, M., Lubman, D. I., Harrison, B. J., Fornito, A., Allen, N. B., Wellard, R. M., Roffel, K.,
Clarke, K., Wood, S. J., Forman, S. D., & Pantelis, C. (2007). A combined spectroscopic and
functional MRI investigation of the dorsal anterior cingulate region in opiate addiction.
Molecular Psychiatry, 12, 691–702.
Zhang, M., Li, J., Chen, C., Xue, G., Lu, Z., Mei, L., Xue, H., Xue, F., He, Q., Chen, C., Wie, M.,
& Dong, Q. (2014). Resting-state functional connectivity and reading abilities in first and
second languages. NeuroImage, 84, 546–553.
Zhu, L., Fan, Y., Zou, Q., Wang, J., Gao, J. H., & Niu, Z. (2014). Temporal reliability and laterali-
zation of the resting-state language network. PLoS One, 9, e85880.
Chapter 5
T r anscrania l Mag net i c

St imul ation to St u dy
t he Neural Net work
Ac c ou nt of L a ng uag e
Teresa Schuhmann
Transcranial Magnetic Stimulation
Transcranial magnetic stimulation (TMS) is based on the physical principles

of mutual and electromagnetic induction. These refer, respectively, to the phys-
ical phenomenon that electrical and magnetic fields can be reciprocally converted,
and the possibility that a current can be produced in a conductive medium by ei-
ther moving the conductive object through a static magnetic field, or by placing the
conductive object into a time-varying magnetic field (Faraday’s law). Based on these
two principles, with TMS one is able to noninvasively induce a secondary current
in the brain, acting as the conductive medium, by applying a time-varying mag-
netic field using an electromagnetic coil. The beginning of modern TMS was the de-
velopment of the first TMS device by Barker in 1985 (Barker, Jalinous, & Freeston,
1985). By stimulating the human motor cortex and inducing the respective motor
contractions, Barker and colleagues measured the connectivity and excitability of
the motor cortex in healthy participants. Moreover, Barker performed the first clin-
ical examinations by comparing conduction times of motor responses in healthy
participants and patients with different neurological diseases (Barker, Freeston,
Jabinous, & Jarratt, 1986; Barker, Jalinous, & Freeston, 1985). Subsequently, Pascual-
Leone and colleagues (Pascual-Leone, Gates, & Dhuna, 1991) were the first to study
language with TMS.
Transcranial Magnetic Stimulation 95
Basic Principles of TMS
Transcranial magnetic stimulation is based on the aforementioned principles of electro-

magnetic induction. A large and brief pulse of current is discharged into an electromag-
netic coil held above a participant’s head. This current produces a time-varying magnetic
field perpendicular to the current, lasting for approximately 100–200 microseconds
(μs). The magnetic field passes transcranially through the intact skull into the tissue and
induces a perpendicular electric field. The strength of the induced electric field mainly
depends on the rate of change of the magnetic field. Due to the electrical conductivity of
the living tissue, the electric field leads to an electrical current in the cortex parallel but
opposite in direction to the current in the coil (Lenz’s law), and subsequently to a depo-
larization of the underlying neurons (Hallett, 2000).
Physically, the rate of change of the electric current in the coil determines the strength
of the induced electrical field. Therefore, the electrical field has its maximum at the in-
stant of the switching of the electrical current since this point marks the maximum rate
of change of the current. With increasing current strength, this rate of change decreases,
and consequently the induced electrical field strength decreases, reaching zero at the
current’s maximum value.
TMS Hardware
Any TMS device consists of a bank of capacitors capable of producing high discharge
currents, and an electromagnetic stimulating coil to apply magnetic pulses of up to
4 Tesla. The high and rapidly changing currents are discharged into the coil, thereby
creating a strong and time-varying magnetic field (pulse) that can reach its peak in
less than 250 μs. This time-varying magnetic field induces an electric current in the
neuronal tissue underneath the coil, and thus induces action potentials when applied
at proper intensities. Different geometric shapes of the electromagnetic coils allow
for different stimulation characteristics, including focality, depth of penetration, and
strength of the induced activity changes. For example, in the so-called circular coil,
low-resistance copper is wound in one or several turns into a ring-shaped configura-
tion. This coil type generally has no single magnetic field focus but a maximum cur-
rent in the entire outer winding, forming a ring-shaped magnetic field around the coil.
Therefore, the site of stimulation in a standard circular coil is not well defined. Inside
a figure-of-eight coil there are two ring-shaped coils, mounted next to each other and
coated by the characteristic butterfly-shaped coil mantle. Importantly, the current in-
side each of these two coil windings circulates in opposite directions, causing the two
96 Teresa Schuhmann
respective magnetic fields of these two coil loops to summate at the coils’ intersec-
tion. This enables a more focal magnetic brain stimulation as compared to other coil
geometries, making it, generally speaking, more suitable for research in cognitive neu-
roscience (e.g., in the context of functional mapping of the brain or identifying causal
brain-behavior relationships).
Chronometric versus Repetitive TMS
TMS can be applied with one pulse at a time (single-pulse TMS), in pairs of pulses
separated by a variable interval (paired-pulse TMS), or in multiple pairs of pulses
(ranging from triple-pulse up to quintuple-pulse TMS). Importantly, all of these ways
of application are usually linked to the onset of an external event such as task onset,
and therefore are capable of revealing information about the chronometry of a cogni-
tive process. We thus can refer to these approaches as chronometric or event-related
TMS, or in other words, protocols that are used to study causal chronometry in brain-
behavior relations.
In contrast, TMS can also be applied in a repetitive manner (repetitive (r)TMS).
Today, we distinguish between conventional and patterned protocols of repetitive stim-
ulation (Rossi, Hallett, Rossini, & Pascual-Leone, 2009). In conventional protocols,
single TMS pulses are repeated and applied in a regular rhythm. Here, one can once
again make a distinction between low-frequency rTMS (stimulation frequency of 1 Hz
or less) and high-frequency rTMS (stimulation frequency of more than 1 Hz). With
newly developed TMS stimulators, the maximum stimulation frequency that can be
reached lies at 250 Hz. Patterned rTMS refers to repetitive application of short rTMS
bursts at a high inner frequency, interleaved by short pauses of no stimulation. In re-
cent years, a new protocol was introduced called theta burst stimulation (TBS). In TBS
protocols, short bursts of 50 Hz rTMS are repeated at a rate in the theta range (5 Hz) as
a continuous (cTBS) or intermittent (iTBS) train (Di Lazzaro, 2008; Huang, Edwards,
Rounis, Bhatia, & Rothwell, 2005).
The important feature of both conventional and patterned TMS is that it is capable
of modulating the excitability of the stimulated area even beyond the duration of the
TMS application itself. The after-effects of TBS were found to be significantly longer
lasting compared to conventional rTMS (Huang, Edwards, Rounis, Bhatia, & Rothwell,
2005) with shorter stimulation time needed. Therefore, patterned rTMS protocols are
starting to play a larger role in brain stimulation. Both 1 Hz rTMS and cTBS are con-
sistently found to produce lasting inhibitory after-effects. High-frequency rTMS and
iTBS, on the other hand, induced lasting facilitatory after-effects on motor corticospinal
output in healthy participants. In sum, the number of repetitive stimuli per second, the
stimulation frequency, as well as the stimulus intensity, the duration of the stimulation
train, the inter-train interval, and finally the total number of trains and total number of
stimuli represent the stimulation parameters of TMS, which may all be combined in dif-
ferent ways to produce the desired neuronal effects at the targeted brain site.
TMS can thus be applied in a chronometric, event-related manner, as well as in a
repetitive manner. Both options provide excellent tools for the study of cognition.
Applying TMS in a chronometric manner, appropriately delivered in time and space,
can disturb the function of a cognitive process online. Applying TMS repetitively results
in an offline, and longer lasting effect. Those two effects enable the study of two aspects
of the contribution of a given cortical region to a specific behavior: What does it do?
And when does it do it?
Critical Issues of TMS

in Language Research
Pioneers in the stimulation of language-related areas were not cognitive neuroscientists,

but neurosurgeons, namely Wilder Penfield and George Ojemann. They used di-
rect cortical stimulation in awake neurosurgical patients to probe language pro-
cessing (Ojemann, 1979; Penfield & Jasper, 1954; see Duffau, Chapter 8 in this volume).
Ojemann was able to show that applying a direct current to a focal brain region can
selectively disrupt linguistic processes. As exciting as these findings were, the invasive
nature of Ojemann’s research did not allow for extensive testing of language functions
in healthy individuals. The introduction of TMS led to new possibilities as it indisput-
ably represents a unique research tool within the spectrum of techniques used to inves-
tigate how the human brain works and how neural mechanisms relate to behavior and
cognition. TMS allows researchers to noninvasively induce local neural activity changes
in the healthy human brain. It thereby directly probes the behavioral and cognitive
consequences of these experimentally induced neural perturbations under controlled
experimental and repeatable conditions.
Nevertheless, TMS also faces general challenges, as well as challenges that are specific
to its application in the domain of language. Advances in technological developments
of TMS, as well as TMS protocols and designs, have been able to address some of these
challenges. Yet, several problems inherent to the method and rationale of TMS, as well
as the neuroanatomy of language, still pose critical limitations. It is important to be
aware of these limitations when conducting or interpreting TMS language studies.
First, and most important, the neural correlates of language include a widely distrib-
uted cortical circuitry of which several regions are not easily identifiable or reachable
by TMS. The challenges thus already start with the crucial question of optimal TMS coil
placement, or how to best determine the optimal stimulation site. In fact, knowing ex-
actly where and how to place the TMS coil has been shown to be decisive for inducing
the intended neural and behavioral consequences (Sack et al., 2009). Yet, in practice
this crucial starting point of any TMS investigation often lacks clear criteria of which
98 Teresa Schuhmann
exact “hot spot” within a given brain region should be targeted. Moreover, determining
such an exact “hot spot” would require the availability of individual anatomical or func-
tional brain imaging data, depending on whether the intended stimulation target site
has to be anatomically or functionally defined on each individual brain reconstruction.
Often, such individual brain-imaging data are not available, and consequently alterna-
tive coil-placement strategies are employed, using, for example, anatomical landmarks
on the scalp or the 10–20 electroencephalography (EEG) system to place the TMS coil.
This falsely assumes a perfect and consistent correspondence between anatomical scalp
landmarks and the underlying neural structures across participants.
What further complicates the matter is the fact that even a perfect coil placement
above the exact “hot spot,” for example as determined by preceding functional imaging
measurements, still leaves the question of how to best rotate the TMS coil. It has been
shown that even when stimulating the exact same stimulation site, rotating the coil such
that the induced electric field crosses the targeted neural structures in either a parallel or
perpendicular configuration clearly affects the outcome of the stimulation. While it has
been suggested that a perpendicular crossing of the target gyrus might be most effective
in the visual cortex (Kammer, Beck, Erb, & Grodd, 2001), it remains largely unclear what
rotation of the coil is optimal for any other brain region or experimental question.
Beside coil placement and rotation, the largest factual limitations of TMS are its
very limited depth of penetration and its rather limited spatial resolution. While both
parameters depend on TMS coil geometry and coil size, it must be realized that the mag-
netic field strength of the TMS coil decreases logarithmically with increasing distance,
limiting its direct effectiveness to a few centimeters (cm) from the coil. It thus must be
stated very clearly that TMS is limited to the superficial areas of the cortex with a max-
imum distance of approximately 2–3 cm from the coil on the scalp (Sandrini, Umilta,
& Rusconi, 2011; Thielscher & Kammer, 2004; Weyh, Wendicke, Mentschel, Zantow, &
Siebner, 2005). Consequently, it is not possible to directly stimulate any deeper-lying
brain structures with TMS. Since several language-related areas are located in the depth
of the Sylvian fissure, direct stimulation effects of these areas are questionable. However,
the penetration depth is sufficient to effectively stimulate the fronto-parieto-temporal
language areas at the cortical surface.
The focality of the magnetic field distribution also decreases with increasing distance,
limiting the spatial resolution of TMS to roughly 1–2 cm, depending on the geometry
and size of the TMS coil used, as well as the distance from the coil. In addition, language-
related areas often lack a clear region-specific separation in space, as is the case in other
cognitive functions such as memory or attention, which again makes selective targeting
of these regions with TMS problematic. Finally, language, as one of the most complex
cognitive skills, is almost never exclusively processed within single isolated brain re-
gions, but requires the organized neural activity within widely distributed and dynam-
ically interacting neural networks. It is therefore often difficult to attribute a behavioral
effect following the stimulation of a single area to changes in the entire network.
What complicates the matter is the fact that a real validation of TMS-induced neural
effects is usually not possible in classical behavioral TMS studies. Here, the circular
assumption often is made that a successful TMS-induced behavioral change is indica-

tive of a successful neural efficiency of the used TMS protocol at the site of stimulation.
However, such behavioral changes are potentially confounded by many psychological
(e.g., placebo) or indirect (e.g., remote network effects, sensory side effects of stimu-
lation, etc.) effects of TMS. This general limitation of TMS, namely interpreting a be-
havioral effect of stimulation as a validation of successful neural efficiency of TMS,
represents an invalid circular argument. This is particularly critical in studies where the
intended stimulation site is physically at the border or outside the reach of the induced
magnetic field of TMS, as often is the case in language studies.
In directly addressing this critical question of neural versus behavioral effects of TMS,
few studies have been able to apply TMS while simultaneously measuring the neural
and metabolic consequences using concurrent functional imaging techniques such as
EEG, positron emission tomography (PET), or functional magnetic resonance imaging
(fMRI) (for an exhaustive review, see Reithler, Peters, & Sack, 2011). These studies have
consistently revealed that locally applied TMS has both local and remote neural network
effects. This means that the TMS-induced local neural activity changes are not restricted
to the directly targeted brain region, but rather spread along anatomical and functional
connections to widely distributed remote regions within a specific neural network.
Interestingly, these remote effects are state-dependent, indicating that TMS specifi-
cally co-activates those remote regions of a given network that are functionally relevant
during cognitive performances. It has been shown that these remote network effects of
TMS are also functionally relevant for successful performance of the task under inves-
tigation. This may seem to be a limitation of TMS, as it complicates the simple brain-
behavior relationship studies in classical behavioral TMS experiments. However, this
also represents a great advantage of TMS, as TMS not only affects single isolated brain
regions, but dynamically interacts with widely distributed brain networks. It thereby in-
directly opens the door to deeper-lying brain regions outside the reach of direct stim-
ulation and in that way allows the possibility of studying the functional relevance of
distributed neural networks for behavior and cognition.
Apart from the technical limitations of TMS, researchers often experience more
practical issues during the application of TMS in language studies. For example,
TMS causes strong side effects, such as muscle twitches, when inferior frontal areas
are stimulated. In language-production studies, this is problematic, since participants
are occasionally unable to speak correctly as a consequence of the muscle stimulation.
Moreover, in online TMS studies, the clicking sound of the TMS coil can be disturbing
and interfering with auditory stimulus presentation. Finally, when participants have to
complete a task in which a vocal reaction is required, the standard setup, in which the
participant is often resting her head in a chin rest with the TMS coil being attached to a
TMS coil holder, cannot be used, as in this setup movements of the mouth are not pos-
sible. Therefore, the coil and the participant’s head ideally are handheld during online
task performance (see Figure 5.1).
Despite these serious challenges in using TMS to study such a complex cognitive ability
as language, several well-conducted studies have successfully demonstrated the value of
100 Teresa Schuhmann
Figure 5.1. Optimal coil placement setup during online speech production experiments.
The participant’s chins is not placed in a chin rest so that he or she can freely move his or her
mouth. Therefore, the stimulation coil has to be handheld.
Figure courtesy of Minye Zhan.
TMS in investigating the functional relationship between a given set of brain regions and
various aspects of language processing. Today, over 25 years after the publication of the
first pioneering study on TMS and language by Pascual-Leone and colleagues (1991), it
is safe to conclude that TMS has been demonstrated to be a valuable tool to study the
neural network account of language. Following the described logic and capability of dif-
ferent TMS protocols, three levels of investigations using TMS to study language will be
distinguished in this chapter, demonstrating that TMS is able to (1) test the functional rel-
evance of brain regions for language processing, (2) chart the exact time period of func-
tional relevance of these regions for successful language processing, and (3) unravel the
dynamics within distributed neural networks underlying language processing.
Using TMS to Test and Segregate

the Functional Relevance of a Brain
Region for Language Processing
Pascual-Leone and colleagues (Pascual-Leone, Gates, & Dhuna, 1991) were the first to
study language with TMS. They induced speech arrest in pre-surgical epilepsy patients
in order to determine whether TMS could be used as a noninvasive alternative to
intracarotid amorbarbital testing. Speech arrest is an effect that is obtained when rTMS
is applied to the left inferior frontal gyrus. Participants undergoing such stimulation ex-
perience everything ranging from a mild hesitation to say something up to complete
mutism. One of the patients of Pascual-Leone and colleagues noted, “I could move my
mouth and I knew what I wanted to say, but I could not get the numbers to my mouth”
(Pascual-Leone et al., 1991, p. 699).
It has been shown that frequencies as low as 4 Hz are sufficient to induce a speech
arrest when using a stimulation intensity of 140% of the resting motor threshold or
higher (Epstein et al., 1996). Higher frequencies can lead to prominent facial and la-
ryngeal muscle contractions, and can significantly increase the discomfort or pain as-
sociated with stimulation, making speech arrest more difficult to determine. Other
research groups were also able to show that rTMS can produce transient speech arrests
(Epstein, 1999; Epstein et al., 1996; Jennum, Friberg, Fuglsang-Frederiksen, & Dam,
1994; Michelucci et al., 1994). Epstein and colleagues (1999) revealed that rTMS to the
left inferior frontal area led to an arrest of spontaneous speech and reading aloud, while
other forms of language processing, such as writing, comprehension, singing, and so on,
remained unaffected.
Following these pioneering findings, several studies have aimed at using TMS in order
to bring to light the functional role of the left inferior frontal gyrus (IFG), Broca’s area,
during language processing and production (Andoh et al., 2006; Devlin, Matthews,
& Rushworth, 2003; Flitman et al., 1998; Hartwigsen, Siebner, Deuschl, Jansen, &
Ulmer, 2010; Kohler, Paus, Buckner, & Milner, 2004; Mottaghy et al., 1999; Mottaghy,
Sparing, & Topper, 2006; Naeser et al., 2005; Nixon, Lazarova, Hodinott-Hill, Gough,
& Passingham, 2004; Pobric, Jefferies, & Ralph, 2007; Romero, Walsh, & Papagno,
2006; Sakai, Noguchi, Takeuchi, & Watanabe, 2002; Shapiro, Pascual-Leone, Mottaghy,
Gangitano, & Caramazza, 2001). We have seen from functional brain-imaging studies
that within the left IFG, there seems to be an anterior-posterior division of labor for se-
mantic and phonological processing (Buckner, Raichle, & Petersen, 1995; Fiez, 1997).
TMS studies have been able to support this claim and have confirmed this division
of labor, while further clarifying the specific regional contributions to semantic and
phonological processing. Devlin and colleagues (2003), for example, were interested
in the role of the left inferior prefrontal cortex (LIPC) in phonological processing. In
their fMRI study, they demonstrated that both semantic and phonological processing
activated a common set of areas within this region. In a following TMS experiment, they
stimulated the anterior portion of the LIPC to determine whether this region was es-
sential for normal semantic performance. Both repetitive and single-pulse TMS signifi-
cantly slowed participants’ reactions for the semantic but not for the perceptual control
task. They thus demonstrated that the anterior and posterior regions of the LIPC con-
tribute to both semantic and phonological processing, although to different extents.
Likewise, Kohler and colleagues (2004) stimulated anterior left IFG with fMRI-guided
rTMS and showed that semantic decisions were slower after stimulation, which again
shows that anterior left IFG is necessary for semantic processing. Nixon and colleagues
(2004) examined the role of the left IFG in phonological processing and verbal working
memory. They applied rTMS to the posterior left IFG while participants performed a
delayed phonological matching task. They then compared the effects of disrupting this
area either during the delay, or memory, phase or at the response, or decision, phase of
the task. When the stimulation was applied during the memory phase when participants
were required to remember the sound of a visually presented word, rTMS impaired their
accuracy. When delivered later in the trial, during the decision phase, rTMS had no effect.
They therefore concluded that the posterior region of the IFG is necessary for the normal
operation of phonologically based working memory mechanisms. Similarly, Aziz-Zadeh
and colleagues (Aziz-Zadeh, Cattaneo, Rochat, & Rizzolatti, 2005) were interested in
whether blocking internal speech can be achieved by rTMS and included a phonolog-
ical task in their study. They found that rTMS over posterior left IFG leads to higher re-
action times compared to trials in which no stimulation was applied. This study again
supported the role of posterior left IFG in phonological processing. Taken together these
TMS studies significantly broaden the previous neuroimaging results by demonstrating
that anterior left IFG is necessary for semantic processing, while posterior left IFG is nec-
essary for phonological processing (Devlin & Watkins, 2007), as suggested by earlier
functional imaging studies (Costafreda et al., 2006; Vigneau et al., 2006). Other studies
showed that not only the left IFG but also the inferior parietal cortex (IP) contributes
to phonological processing (Kirschen, Davis-Ratner, Jerde, Schraedley-Desmond, &
Desmond, 2006; Pattamadilok, Knierim, Kawabata Duncan, & Devlin, 2010; Romero
et al., 2006; Stoeckel, Gough, Watkins, & Devlin, 2009).
The left posterior superior temporal gyrus (PSTG), Wernicke’s area, has also been
targeted by TMS (Andoh et al., 2006; Mottaghy et al., 1999; Mottaghy et al., 2006) in
order to unravel its functional contribution to language processing in healthy human
volunteers. Andoh and colleagues (2006) explored the role of the left PSTG in semantic
and phonological processing with fMRI-guided rTMS. While listening to sentences in
their native language and in languages unknown to them, they were required to per-
form a fragment-detection task, during which they also received rTMS. Low-frequency
rTMS applied over Wernicke’s area resulted in a decreased reaction-time response and
was stronger for native than non-native languages. This facilitatory effect was specific
to stimulation of Wernicke’s area. The authors therefore concluded that the left PSTG
is involved in lexical processing. Other studies have also reported facilitatory effects of
rTMS on language areas (Mottaghy et al., 1999; Mottaghy et al., 2006; Wassermann et al.,
1999). In these studies, rTMS to Wernicke’s area influenced picture-naming latencies,

resulting in a shortening of naming latencies without affecting the accuracy of the re-
sponse; rTMS over the visual cortex, Broca’s area, or over the corresponding sites in the
nondominant hemisphere had no effect. Repetitive transcranial magnetic stimulation
over Wernicke’s area also led to a brief facilitation of picture naming, possibly by short-
ening linguistic processing time.
Finally, using the approach of employing TMS to test and segregate the functional
relevance of a brain region for language processing, the role of primary and secondary
motor cortex in speech (Aglioti & Pazzaglia, 2010; Aziz-Zadeh et al., 2005; Cattaneo,
Devlin, Salvini, Vecchi, & Silvanto, 2010; Fadiga, Craighero, Buccino, & Rizzolatti, 2002;
Mottonen & Watkins, 2009; Tremblay & Gracco, 2009; Watkins & Paus, 2004; Watkins,
Strafella, & Paus, 2003) has also been successfully shown (for a complete overview,
see Cattaneo, 2013; Devlin & Watkins, 2007; Hartwigsen, 2015). In sum, these studies
show how TMS can probe the functional relevance of a given language-related brain
region for a particular language process. In particular, the demonstrated ability of this
approach to segregate functionally distinct anatomical subregions and a respective
double-dissociation with distinct subfunctions is highly valuable.
Using TMS to Chart the Exact Time

Period of Functional Relevance of
Single Brain Regions for Successful
Language Processing
TMS is not only able to give information about the functional relevance of a brain re-
gion, but it can also provide information about when this area is needed for successful
task execution. One can thus chart the exact time period of functional relevance for suc-
cessful language processing in a brain region. To do so online, event-related protocols
are used, during which trains of pulses are given at different points in time. Töpper and
colleagues (Töpper, Mottaghy, Brugmann, Noth, & Huber, 1998) belong to the pioneers
in chronometric TMS designs in language research. They were interested in the role of
Wernicke’s area and the motor cortex during picture naming and applied single TMS
pulses to these areas at different points in time, ranging from 5,000 milliseconds (ms)
before picture presentation up to 300 ms after picture presentation. Interestingly, they
did not find an effect on naming accuracy by TMS over any of the stimulated areas, or
any effects of motor cortex stimulation on naming latencies. However, single-pulse TMS
over Wernicke’s area resulted in a reduction in reaction times when the pulses were
given 1,000 ms and 500 ms before picture presentation. The authors concluded that
TMS is able to facilitate lexical processes due to a general pre-activation of language-
related neuronal networks when delivered over Wernicke’s area.
Schuhmann, Schiller, Goebel, and Sack (2009) also performed a study to investi-
gate the temporal dynamics of the posterior IFG in picture naming. In their study,
participants were shown pictures of monosyllabic nouns and asked to name them aloud.
Using an online event-related triple-pulse MRI-guided TMS paradigm, these authors
applied real and sham TMS to Broca’s area at various time points between 150 and 575
ms following picture presentation. They observed that vocal reaction times were signif-
icantly slowed down when real TMS pulses were delivered around 300 ms after picture
presentation, but not when TMS pulses were applied earlier or later in time. The authors
concluded that Broca’s area was functionally relevant at 300 ms after picture presenta-
tion and could successfully link this neural timing information to the model of speech
production presented by Indefrey and Levelt (2004). They interpreted the slowing down
of the vocal reaction-time data as a disruption of the syllabification process through
TMS. Syllabification is defined by Indefrey and Levelt as an abstract segmental repre-
sentation that takes place prior to the activation of articulatory motor representations
that are required for speech output, and occurs at around 330 ms after the presentation
of a picture, according to their neurolinguistic model.
Following up on this study, Wheat and colleagues (2013) used fMRI-guided chron-
ometric TMS to investigate whether the effects that were found during picture naming
when stimulating Broca’s area can also be seen when reading words. They delivered
pulses to individually predefined functional maps in the left IFG 70–500 ms after stim-
ulus onset. They found that both reading and picture-naming reaction times were signif-
icantly slower when TMS pulses were applied to Broca’s area at 225–300 ms. This study
replicated the findings reported by Schuhmann and colleagues (2009) and expanded
the revealed functional chronometry of Broca’s area for picture naming to reading.
To conclude, these studies clearly demonstrate the feasiblity of using event-related
chronometric TMS protocols in the domain of language research. These approaches
are of great potential value, as they allow charting the exact time point at which certain
language-related brain regions are functionally needed for successfully performing spe-
cific language (sub)processes. Considering the complexity of the language network and
the highly distributed neural processing underlying the various subfunctions of language,
it seems necessary, however, to apply short event-related TMS pulse bursts consisting of
three or more high-frequency pulses time-locked to the onset of the task, and temporally
separated by an appropriate inter-trial interval to avoid carryover effects, as has success-
fully been done in some studies (Schuhmann et al., 2009; Wheat et al., 2013).
From Single Brain Regions

to Unraveling the Dynamics of the
Neural Network Account of Language
So far, we have discussed how TMS can be used to test and segregate the functional
relevance of a brain region and how the exact time point of functional relevance of
single brain regions can be charted for successful language processing. However, re-
cently, language studies have started to look at the neural network accounts of lan-
guage. These studies not only looked at one brain region at a time, but also looked at
various brain regions together. This can be done by stimulating various brain regions
separately, receiving timing information of each of the regions, and thereby receiving
indirect evidence about information flow. However, stimulating two or more areas si-
multaneously is also possible. Thereby, inter-hemispheric interaction and compensa-
tion can be studied, if they are compared to unifocal stimulation. Another very elegant
and recent approach to studying how neural network processes are modulated requires
direct monitoring of the TMS-induced changes in brain activity. By combining TMS
with other measuring techniques, both the instant and longer-lasting neurophysiolog-
ical consequences of applying TMS can be assessed (Reithler et al., 2011; Sack & Linden,
2003; Siebner et al., 2009). To be able to monitor what is happening in the rest of the
brain while a particular cortical site is targeted can reveal the full network dynamics
reflecting the impact of TMS, instead of only assuming certain corollary activation
changes or ignoring potential remote effects of TMS altogether (Reithler et al., 2011).
Mottaghy and colleagues (2003) were interested in the temporal dynamics of pari-
etal and prefrontal cortex involvement in verbal working memory (see Buchsbaum,
Chapter 32 in this volume). They applied MRI-guided single-pulse TMS to the middle
frontal gyrus (MFG) and the IP on each hemisphere. TMS was applied at 10 different
time points, 140–500 ms into the delay period of a two-back verbal working-memory
task. As a control task, the authors used a choice-reaction task. They discovered different
patterns in reactivity when TMS interfered with task accuracy, depending on the brain
area that they were stimulating. To be more precise, interference with task accuracy was
induced by TMS in the parietal cortex earlier as compared to the prefrontal cortex and
earlier over the right than the left hemisphere. They thus showed a clear time-locked
main effect in specific brain areas on the accuracy of task performance and concluded
that this reveals a propagation of the information flow from parietal to prefrontal cor-
tical sites, advancing faster over the right as compared to the left hemisphere.
Similarly, Schuhmann and colleagues (Schuhmann, Schiller, Goebel, & Sack,
2012) tried to dissect the neural network of language production during picture
naming. To do so, they stimulated the left superior temporal gyrus (STG; Wernicke’s
area), left IFG (Broca’s area), and the midsection of the left middle temporal gyrus
(MTG) with MRI-navigated, event-related triple-pulse TMS. They stimulated each
site, in separate sessions, with pulses applied between 150 ms and 575 ms after picture
presentation. Stimulation of all three regions led to a decrease in reaction times; how-
ever, the pattern of when the reaction times decreased differed between brain regions.
While stimulation of the left MTG led to a decrease in reaction time at 225 ms and
again at 400 ms after picture presentation, stimulation of the left IFG had an effect at
300 ms and stimulation of STG at 400 ms, indicating that all areas are functionally rel-
evant for language production, but at different points in time. Thus, by applying TMS
over several nodes of the same widely distributed network underlying language pro-
cessing and production, these studies allow for charting of the relative time points of
functional necessity in each of these network nodes, and thereby allow documentation
of a certain temporal order of functional relevance between distinct brain regions.

This finding clearly shows a specific spatiotemporal organization within the language
production network in terms of relative time course in which every area contributes at
a different stage during the production process, suggesting distinct underlying func-
tional roles within this complex multicomponential skill (see de Zubicaray & Piai,
Hartwigsen and colleagues (Hartwigsen, Baumgaertner, et al., 2010; Hartwigsen,
Price, et al., 2010) took a different approach, and employed a rather novel approach
in which they applied rTMS either unilaterally over left-and right-hemispheric re-
gions, or they applied rTMS simultaneously to these regions. By doing so, they wanted
to study the inter-hemispheric interaction and compensation mechanisms of phono-
logical decisions. Specifically, Hartwigsen and colleagues (Hartwigsen, Price, et al.,
2010) investigated the role of left and right supramarginal gyri during phonological
decisions. They applied high-frequency bursts of TMS to either right, left, or bilateral
supramarginal gyrus (SMG). The results showed that accuracy and reaction times of
phonological decisions were selectively disrupted relative to semantic and percep-
tual decisions when real TMS was applied over the left, right, and bilateral SMG. This
shows that both left and right SMG are needed for accurate and efficient phonolog-
ical decisions. Moreover, since there was no difference between unilateral and bilateral
stimulation, the authors concluded that left and right SMG could not acutely compen-
sate for one another.
To date there are very few studies that have used the advantages of multimodal
TMS studies in the field of language research. Fuggetta, Rizzo, Pobric, Lavidor, and
Walsh (2009) combined rTMS with event-related potentials (ERPs) to gain insight
into the neural basis of the semantic system and in particular to study the temporal
and functional organization of object categorization processing. The authors applied
short trains of high-frequency TMS over Wernicke’s area, the homologous area in the
right hemisphere, and a control site while participants were performing a picture-word
verification task in which line drawings of natural (e.g., animal) and artifactual (e.g.,
tool) categories were associated with a word. When Wernicke’s area was stimulated im-
mediately before the stimulus onset, the authors observed a delay in reaction times
to artifactual items, and thus, an increased dissociation between natural and artifac-
tual domains. This effect was not found after right hemisphere Wernicke’s homologous
stimulation. Interestingly, the behavioral effect they found had a direct ERP correlate.
In the response period, the stimuli from the natural domain elicited a significantly
larger late positivity complex than those from the artifactual domain. The authors
suggested that this amplitude increase reflects a compensatory transfer of language
function from the left to the right hemisphere. These findings demonstrate that rTMS
interferes with post-perceptual categorization processing of natural and artifactual
stimuli that involves separate subsystems in distinct cortical areas. They also support
the view that the representation of semantic knowledge associated with different con-
ceptual domains is based on a network of partially segregated neural systems of func-
tionally interconnected cortical regions. From a cognitive neuroscience perspective,
the combination of TMS and ERP is ideal for gaining new insights into the neural basis
of a variety of cognitive processes that could not be obtained on the basis of one of these
methods alone.
The experimental combination of TMS and fMRI in language research is very rare.
Some studies, however, have tried to combine offline TMS with subsequent fMRI. This
means that participants first received rTMS outside of the MRI scanner, and were then
quickly put into the scanner to see the neural effects outlasting the stimulation duration
itself. Hartwigsen and colleagues (2013) did such an offline rTMS study, combining it
with immediate fMRI following applied TMS, to research the contribution of the right
hemisphere to language production after either the left posterior or anterior IFG was
stimulated with an inhibitory TMS protocol. Participants subsequently had to perform
an overt repetition task on word and pseudoword stimuli while they were lying in the
MR scanner. The authors found that compared to TMS of the anterior IFG, the stimu-
lation of the left posterior IFG decreased activity in the targeted area and increased ac-
tivity in the contralateral homologous area during a simple pseudoword repetition task
(Hartwigsen et al., 2013). They also found an up-regulation of the contralateral homolo-
gous area after stimulating the posterior IFG compared to the anterior IFG. Interestingly,
when comparing response times, the authors noted that reaction times became faster as
the influence of the right posterior IFG on the left posterior IFG increased. This finding
shows that homologous areas in the right hemisphere can actively contribute to lan-
guage production after focal inhibitory stimulation to the left hemisphere. This multi-
modal TMS study thus revealed the underlying neural effects of TMS and also showed
how the brain compensates for a TMS perturbation. These findings also lend further
support to the notion that increased activation of homologous right hemisphere areas
supports aphasia recovery after left hemisphere damage.
In sum, all of these studies applied innovative and cutting-edge TMS methodology to
unravel the dynamic neural network effects underlying language processing. This is par-
ticularly valuable, as the complexity of language requires us to look at both the spatial
and temporal aspects of distributed neural processing, including task-dependent dy-
namic changes, direction of information flow, recurrent peaks of functional relevance,
as well as remote and compensatory effects following various network perturbations.
All of these complex questions can only be addressed using multi-site chronometry
TMS and/or TMS in combination with other modalities to directly assess the neural and
metabolic consequences of brain stimulation.
TMS as Treatment in Clinical

Populations
Aphasia is a common consequence of stroke that typically results from injury to cor-
tical and subcortical structures perfused by the left middle cerebral artery (Hamilton,
Chrysikou, & Coslett, 2011; McNeill & Pratt, 2001). While most of the aphasic patients
show some degree of spontaneous recovery during the first two to three months post-
stroke onset (Laska, Hellblom, Murray, Kahan, & Von Arbin, 2001; Lendrem & Lincoln,
1985; Nicholas, Helm-Estabrooks, Ward-Lonergan, & Morgan, 1993), the majority of
patients with post-stroke aphasia are left with some degree of chronic deficit. There is
growing evidence that noninvasive brain stimulation can have beneficial effects in the
treatment of aphasia (Hamilton et al., 2011).
Several TMS studies have used low-frequency (thus inhibitory) stimulation of the
right hemisphere, trying to focally reduce neural activity in the intact hemisphere,
contralateral to the lesion (Martin et al., 2004; Naeser et al., 2005). For example, in a
study by Martin and colleagues (2004), low-frequency rTMS was applied to four dif-
ferent points on right-hemisphere perisylvian regions in chronic nonfluent aphasics.
After stimulation of the right pars triangularis (the right-hemisphere homologue of
Broca’s area), an improvement in reaction time and accuracy in a picture-naming
task was observed. When extending this experiment to a TMS treatment of right pars
triangularis stimulation for 20 minutes a day, five days a week for two weeks, signifi-
cant improvement of picture naming was observed that lasted for up to eight months
(Martin et al., 2004). These findings were also replicated by other research teams
(Hamilton et al., 2010). Recent studies thus provide the first evidence that TMS might
be beneficial in promoting recovery from aphasia after stroke. Future investigations
should determine whether combining noninvasive brain stimulation with standard
behavioral rehabilitation could lead to even better therapeutic results.
Noninvasive brain stimulation is also starting to play a role in pre-surgical brain
mapping. The purpose of pre-surgical brain mapping is to facilitate surgical pla-
nning, prevent or reduce morbidity, and optimize the therapeutic effects of surgery
(Papanicolaou et al., 2014). Currently, one can identify which hemisphere is the dom-
inant hemisphere for language and memory through the Wada procedure, and which
areas underlie somatosensation, motor functions, and language functions with di-
rect cortical stimulation (DCS) mapping. Even though these procedures are the gold
standard in pre-surgical brain mapping, research has recently shown that transcranial
magnetic stimulation provides equally trustworthy results, and therefore may replace
the Wada test and DCS in many, if not most cases. Picht and colleagues (Picht et al.,
2013) compared the safety and effectiveness of preoperative TMS with DCS mapping
during awake surgery for the identification of language areas in patients with left-
sided cerebral lesions. They examined 20 patients with tumors in or close to left-sided
language-eloquent regions rTMS before patients received surgery. During awake sur-
gery, language-eloquent cortex was identified by DCS. They observed very good overall
correlation between rTMS and DCS, and therefore concluded that noninvasive inhi-
bition mapping with TMS is evolving as a valuable tool for preoperative mapping of
language areas.
TMS thus shows promise as a valuable tool in aphasia rehabilitation, and as an alter-
native to invasive preoperative brain mapping.
Outlook and Future Perspectives
In the last 25 years of language research using TMS, our knowledge of the functional
role of different brain regions in language processing has increased enormously. As
outlined in this chapter, TMS is a very promising tool to help further understand the
processes of language processing. As discussed earlier, TMS also has several limitations.
However, new methods that are constantly appearing in the field of cognitive neurosci-
ence might help overcome many of these limitations. Among these recently developed
techniques, the association of TMS with whole-brain measures of neural functions such
as fMRI or EEG (Siebner et al., 2009) has the potential to provide valuable informa-
tion on the neural network dynamics within language-related brain regions, including
deeper cortical areas not directly reachable by TMS. The future of TMS research in lan-
guage processing therefore lies in employing multimodal approaches to combine non-
invasive brain stimulation with neuroimaging techniques. This can be used to validate
the assumed neural network changes and to unravel the functional dependencies within
these networks, including their relation to various language (sub-)processes. Similarly,
computational modeling should be added to the prediction and interpretation of the be-
havioral consequences of different TMS approaches. This can help to better understand
the induced behavioral effects of TMS, to provide concrete models of the underlying
mechanism, and to use concurrent TMS and imaging data to further inform, restrict,
and validate these models.
Another important future development concerns the application of multi-site TMS
protocols, using several smaller focal TMS coils that are still capable of reaching the in-
tended multiple brain regions either simultaneously or with a pre-set inter-coil stimu-
lation delay. This way, several relevant brain regions within the language network can
be stimulated at either the same time or in short temporal succession. Consequently,
one can directly probe their effective connectivity and directly investigate the complex
chronometry of linguistic processes within the millisecond temporal resolution of TMS.
Used within clever experimental designs, such multi-site TMS approaches will take ad-
vantage of both the neural network dynamics underlying language processing, as well as
the excellent temporal resolution of chronometric TMS. Although smaller coils allow
more focal and thus spatially isolatable stimulation, in principle, the described problem
of spatial overlap within and across language-related brain regions remains. A possible
future way of at least partly addressing this issue could come from those experimental
TMS approaches that look into the role of state dependency of TMS effects on cognition,
using adaptation TMS to specifically target functionally distinct neuronal populations
independent of their spatial proximity (Silvanto & Pascual-Leone, 2008).
When it comes to the clinical potential of TMS in the domain of language-related
deficits, it seems most promising to conceptualize TMS as an additional treatment to
be used in combination with other rehabilitation approaches. TMS can then be used
to either prepare the neural network optimally for receiving a given treatment, or to
support and guide the adaptive plastic reorganization during rehabilitation. TMS has
been shown to enhance and support such recovery processes after brain injury, espe-
cially when combined with other therapies to stimulate adaptive reorganization to re-
gain lost functions.
References
Aglioti, S. M., & Pazzaglia, M. (2010). Representing actions through their sound. Experimental
Brain Research, 206(2), 141–151. doi: 10.1007/s00221-010-2344-x
Andoh, J., Artiges, E., Pallier, C., Rivière, D., Mangin, J. F., Cachia, A., . . . Martinot, J. L. (2006).
Modulation of language areas with functional MR image-guided magnetic stimulation.
NeuroImage, 29(2), 619–627.
Aziz-Zadeh, L., Cattaneo, L., Rochat, M., & Rizzolatti, G. (2005). Covert speech arrest induced
by rTMS over both motor and nonmotor left hemisphere frontal sites. Journal of Cognitive
Neuroscience, 17(6), 928–938. doi: 10.1162/0898929054021157
Barker, A. T., Freeston, I. L., Jabinous, R., & Jarratt, J. A. (1986). Clinical evaluation of conduc-
tion time measurements in central motor pathways using magnetic stimulation of human
brain. The Lancet, 1(8493), 1325–1326.
Barker, A. T., Jalinous, R., & Freeston, I. L. (1985). Non-invasive magnetic stimulation of human
motor cortex. Lancet, 1(8437), 1106–1107.
Buckner, R. L., Raichle, M. E., & Petersen, S. E. (1995). Dissociation of human prefrontal
cortical areas across different speech production tasks and gender groups. Journal of
Neurophysiology, 74(5), 2163–2173.
Cattaneo, Z. (2013). Language. Handbook of Clinical Neurology, 116, 681–691. doi: 10.1016/
B978-0-444-53497-2.00054-1
Cattaneo, Z., Devlin, J. T., Salvini, F., Vecchi, T., & Silvanto, J. (2010). The causal role of category-
specific neuronal representations in the left ventral premotor cortex (PMv) in semantic pro-
cessing. NeuroImage, 49(3), 2728–2734. doi: 10.1016/j.neuroimage.2009.10.048
Costafreda, S. G., Fu, C. H., Lee, L., Everitt, B., Brammer, M. J., & David, A. S. (2006). A system-
atic review and quantitative appraisal of fMRI studies of verbal fluency: Role of the left infe-
rior frontal gyrus. Human Brain Mapping, 27(10), 799–810. doi: 10.1002/hbm.20221
Devlin, J. T., Matthews, P. M., & Rushworth, M. F. (2003). Semantic processing in the left infe-
rior prefrontal cortex: A combined functional magnetic resonance imaging and transcranial
magnetic stimulation study. Journal of Cognitive Neuroscience, 15(1),  71–84.
Devlin, J. T., & Watkins, K. E. (2007). Stimulating language: Insights from TMS. Brain,
130(Pt 3), 610–622.
Di Lazzaro, V. (2008). The physiological basis of the effects of intermittent theta burst stimula-
tion of the human motor cortex. The Journal of Physiology, 586(16), 3871–3871.
Epstein, C. M. (1999). Language and TMS/ rTMS. Electroencephalography and Clinical
Neurophysiology. Supplement, 51, 325–333.
Epstein, C. M., Lah, J. J., Meador, K., Weissman, J. D., Gaitan, L. E., & Dihenia, B. (1996).
Optimum stimulus parameters for lateralized suppression of speech with magnetic brain
stimulation. Neurology, 47(6), 1590–1593.
Epstein, C. M., Meador, K. J., Loring, D. W., Wright, R. J., Weissman, J. D., Sheppard,
S., . . . Davey, K. R. (1999). Localization and characterization of speech arrest during
transcranial magnetic stimulation. Clinical Neurophysiology, 110(6), 1073–1079.
Fiez, J. A. (1997). Phonology, semantics, and the role of the left inferior prefrontal cortex.
Human Brain Mapping, 5(2),  79–83.
Flitman, S. S., Grafman, J., Wassermann, E. M., Cooper, V., O’Grady, J., Pascual-Leone, A., &
Hallett, M. (1998). Linguistic processing during repetitive transcranial magnetic stimula-
tion. Neurology, 50(1), 175–181.
Fuggetta, G., Rizzo, S., Pobric, G., Lavidor, M., & Walsh, V. (2009). Functional representation
of living and nonliving domains across the cerebral hemispheres: A combined event-related
potential/transcranial magnetic stimulation study. Journal of Cognitive Neuroscience, 21(2),
403–414. doi: 10.1162/jocn.2008.21030
Hallett, M. (2000). Transcranial magnetic stimulation and the human brain. Nature, 406(6792),
147–150.
Hamilton, R. H., Chrysikou, E. G., & Coslett, B. (2011). Mechanisms of aphasia recovery after
stroke and the role of noninvasive brain stimulation. Brain and Language, 118(1–2), 40–50.
doi: 10.1016/j.bandl.2011.02.005
Hamilton, R. H., Sanders, L., Benson, J., Faseyitan, O., Norise, C., Naeser, M., . . . Coslett, H. B.
(2010). Stimulating conversation: Enhancement of elicited propositional speech in a patient
with chronic non-fluent aphasia following transcranial magnetic stimulation. Brain and
Language, 113(1),  45–50.
Hartwigsen, G. (2015). The neurophysiology of language: Insights from non-invasive brain stimula-
tion in the healthy human brain. Brain and Language, 148, 81–94. doi: 10.1016/j.bandl.2014.10.007
Hartwigsen, G., Baumgaertner, A., Price, C. J., Koehnke, M., Ulmer, S., & Siebner, H. R. (2010).
Phonological decisions require both the left and right supramarginal gyri. Proceedings of the
National Academy of Sciences of the USA, 107(38), 16494–16499. doi: 10.1073/pnas.1008121107
Hartwigsen, G., Price, C. J., Baumgaertner, A., Geiss, G., Koehnke, M., Ulmer, S., & Siebner,
H. R. (2010). The right posterior inferior frontal gyrus contributes to phonological word
decisions in the healthy brain: Evidence from dual-site TMS. Neuropsychologia, 48(10),
3155–3163. doi: 10.1016/j.neuropsychologia.2010.06.032
Hartwigsen, G., Saur, D., Price, C. J., Ulmer, S., Baumgaertner, A., & Siebner, H. R. (2013).
Perturbation of the left inferior frontal gyrus triggers adaptive plasticity in the right homol-
ogous area during speech production. Proceedings of the National Academy of Sciences of the
USA, 110(41), 16402–16407. doi: 10.1073/pnas.1310190110
Hartwigsen, G., Siebner, H. R., Deuschl, G., Jansen, O., & Ulmer, S. (2010). Incidental findings
are frequent in young healthy individuals undergoing magnetic resonance imaging in brain
research imaging studies: A prospective single-center study. Journal of Computer Assisted
Tomography, 34(4), 596–600. doi: 10.1097/RCT.0b013e3181d9c2bb
Huang, Y.-Z., Edwards, M. J., Rounis, E., Bhatia, K. P., & Rothwell, J. C. (2005). Theta burst
stimulation of the human motor cortex. Neuron, 45(2), 201–206.
Indefrey, P., & Levelt, W. J. (2004). The spatial and temporal signatures of word production
components. Cognition, 92(1–2), 101–144.
Jennum, P., Friberg, L., Fuglsang-Frederiksen, A., & Dam, M. (1994). Speech localization using
repetitive transcranial magnetic stimulation. Neurology, 44(2), 269–273.
Kammer, T., Beck, S., Erb, M., & Grodd, W. (2001). The influence of current direction on phos-
phene thresholds evoked by transcranial magnetic stimulation. Clinical Neurophysiology,
112(11), 2015–2021.
Kirschen, M. P., Davis-Ratner, M. S., Jerde, T. E., Schraedley-Desmond, P., & Desmond, J. E.
(2006). Enhancement of phonological memory following transcranial magnetic stimula-
tion (TMS). Behavioral Neurology, 17(3–4), 187–194.
Kohler, S., Paus, T., Buckner, R. L., & Milner, B. (2004). Effects of left inferior prefrontal stimu-
lation on episodic memory formation: A two-stage fMRI-rTMS study. Journal of Cognitive
Laska, A. C., Hellblom, A., Murray, V., Kahan, T., & Von Arbin, M. (2001). Aphasia in acute
stroke and relation to outcome. Journal of Internal Medicine, 249(5), 413–422.
Lendrem, W., & Lincoln, N. B. (1985). Spontaneous recovery of language in patients with aphasia
between 4 and 34 weeks after stroke. Journal of Neurology, Neurosurgery and Psychiatry, 48(8),
743–748.
Martin, P. I., Naeser, M. A., Theoret, H., Tormos, J. M., Nicholas, M., Kurland, J., . . . Pascual-
Leone, A. (2004). Transcranial magnetic stimulation as a complementary treatment for
aphasia. Seminars in Speech and Language, 25(2), 181–191.
McNeill, M. R., & Pratt, S. R. (2001). Defining aphasia: Some theoretical and clinical
implications of operating from a formal definition. Aphasiology, 15, 900–911.
Michelucci, R., Valzania, F., Passarelli, D., Santangelo, M., Rizzi, R., Buzzi, A. M., . . . Tassinari,
C. A. (1994). Rapid-rate transcranial magnetic stimulation and hemispheric language domi-
nance: Usefulness and safety in epilepsy. Neurology, 44(9), 1697–1700.
Mottaghy, F. M., Gangitano, M., Krause, B. J., & Pascual-Leone, A. (2003). Chronometry of
parietal and prefrontal activations in verbal working memory revealed by transcranial mag-
netic stimulation. NeuroImage, 18(3), 565–575.
Mottaghy, F. M., Hungs, M., Brugmann, M., Sparing, R., Boroojerdi, B., Foltys, H., . . . Topper,
R. (1999). Facilitation of picture naming after repetitive transcranial magnetic stimulation.
Neurology, 53(8), 1806–1812.
Mottaghy, F. M., Sparing, R., & Topper, R. (2006). Enhancing picture naming with transcranial
magnetic stimulation. Behavioral Neurology, 17(3–4), 177–186.
Mottonen, R., & Watkins, K. E. (2009). Motor representations of articulators contribute
to categorical perception of speech sounds. Journal of Neuroscience, 29(31), 9819–9825.
doi: 10.1523/JNEUROSCI.6018-08.2009
Naeser, M. A., Martin, P. I., Nicholas, M., Baker, E. H., Seekins, H., Helm-Estabrooks,
N., . . . Pascual-Leone, A. (2005). Improved naming after TMS treatments in a chronic,
global aphasia patient: Case report. Neurocase, 11(3), 182–193.
Nicholas, M. L., Helm-Estabrooks, N., Ward-Lonergan, J., & Morgan, A. R. (1993). Evolution
of severe aphasia in the first two years post onset. Archives of Physical Medicine and
Rehabilitation, 74(8), 830–836.
Nixon, P., Lazarova, J., Hodinott-Hill, I., Gough, P., & Passingham, R. (2004). The infe-
rior frontal gyrus and phonological processing: An investigation using rTMS. Journal of
Ojemann, G. A. (1979). Individual variability in cortical localization of language. Journal of
Neurosurgery, 50(2), 164–169.
Papanicolaou, A. C., Rezaie, R., Narayana, S., Choudhri, A. F., Wheless, J. W., Castillo, E. M., . . .
Boop, F. A. (2014). Is it time to replace the Wada test and put awake craniotomy to sleep?
Epilepsia, 55(5), 629–632. doi: 10.1111/epi.12569
Pascual-Leone, A., Gates, J. R., & Dhuna, A. (1991). Induction of speech arrest and
counting errors with rapid-rate transcranial magnetic stimulation. Neurology, 41(5),
697–702.
Pattamadilok, C., Knierim, I. N., Kawabata Duncan, K. J., & Devlin, J. T. (2010). How does
learning to read affect speech perception? Journal of Neuroscience, 30(25), 8435–8444.
doi: 10.1523/JNEUROSCI.5791-09.2010
Penfield, W., & Jasper, H. H. (1954). Epilepsy and the functional anatomy of the human brain.
Boston: Little, Brown.
Picht, T., Krieg, S. M., Sollmann, N., Rosler, J., Niraula, B., Neuvonen, T., . . . Ringel, F. (2013).
A comparison of language mapping by preoperative navigated transcranial magnetic stim-
ulation and direct cortical stimulation during awake surgery. Neurosurgery, 72(5), 808–819.
doi: 10.1227/NEU.0b013e3182889e01
Pobric, G., Jefferies, E., & Ralph, M. A. (2007). Anterior temporal lobes mediate semantic
representation: Mimicking semantic dementia by using rTMS in normal participants.
Proceedings of the National Academy of Sciences of the USA, 104(50), 20137–20141.
doi: 10.1073/pnas.0707383104
Reithler, J., Peters, J. C., & Sack, A. T. (2011). Multimodal transcranial magnetic stimula-
tion: Using concurrent neuroimaging to reveal the neural network dynamics of non-
invasive brain stimulation. Progress in Neurobiology, 94(2), 149– 165. doi: 10.1016/
j.pneurobio.2011.04.004
Romero, L., Walsh, V., & Papagno, C. (2006). The neural correlates of phonological short-
term memory: A repetitive transcranial magnetic stimulation study. Journal of Cognitive
Neuroscience, 18(7), 1147–1155. doi: 10.1162/jocn.2006.18.7.1147
Rossi, S., Hallett, M., Rossini, P. M., & Pascual-Leone, A. (2009). Safety, ethical considerations,
and application guidelines for the use of transcranial magnetic stimulation in clinical prac-
tice and research. Clinical Neurophysiology, 120(12), 2008–2039.
Sack, A. T., Cohen Kadosh, R., Schuhmann, T., Moerel, M., Walsh, V., & Goebel, R. (2009).
Optimizing functional accuracy of TMS in cognitive studies: A comparison of methods.
Journal of Cognitive Neuroscience, 21(2), 207–221. doi: 10.1162/jocn.2009.21126
Sack, A. T., & Linden, D. E. (2003). Combining transcranial magnetic stimulation and func-
tional imaging in cognitive brain research: Possibilities and limitations. Cognitive Brain
Research, 43(1),  41–56.
Sakai, K. L., Noguchi, Y., Takeuchi, T., & Watanabe, E. (2002). Selective priming of syntactic
processing by event-related transcranial magnetic stimulation of Broca’s area. Neuron, 35(6),
1177–1182.
Sandrini, M., Umilta, C., & Rusconi, E. (2011). The use of transcranial magnetic stimulation
in cognitive neuroscience: A new synthesis of methodological issues. Neuroscience and
Biobehavioral Reviews, 35(3), 516–536. doi: 10.1016/j.neubiorev.2010.06.005
Schuhmann, T., Schiller, N. O., Goebel, R., & Sack, A. T. (2009). The temporal characteristics
of functional activation in Broca’s area during overt picture naming. Cortex, 45(9), 1111–1116.
doi: 10.1016/j.cortex.2008.10.013
Schuhmann, T., Schiller, N. O., Goebel, R., & Sack, A. T. (2012). Speaking of which: Dissecting
the neurocognitive network of language production in picture naming. Cerebral Cortex,
22(3), 701–709. doi: 10.1093/cercor/bhr155
Shapiro, K. A., Pascual-Leone, A., Mottaghy, F. M., Gangitano, M., & Caramazza, A. (2001).
Grammatical distinctions in the left frontal cortex. Journal of Cognitive Neuroscience, 13(6),
713–720.
Siebner, H. R., Bergmann, T. O., Bestmann, S., Massimini, M., Johansen- Berg, H.,
Mochizuki, H., . . . Rossini, P. M. (2009). Consensus paper: Combining transcranial stim-
ulation with neuroimaging. Brain Stimulation, 2(2), 58–80. doi: 10.1016/j.brs.2008.11.002
Silvanto, J., & Pascual-Leone, A. (2008). State-dependency of transcranial magnetic stimula-

tion. Brain Topography, 21(1), 1–10. doi: 10.1007/s10548-008-0067-0
Stoeckel, C., Gough, P. M., Watkins, K. E., & Devlin, J. T. (2009). Supramarginal gyrus involve-
ment in visual word recognition. Cortex, 45(9), 1091–1096. doi: 10.1016/j.cortex.2008.12.004
Thielscher, A., & Kammer, T. (2004). Electric field properties of two commercial figure-8 coils
in TMS: Calculation of focality and efficiency. Clinical Neurophysiology, 115(7), 1697–1708.
Töpper, R., Mottaghy, F. M., Brugmann, M., Noth, J., & Huber, W. (1998). Facilitation of picture
naming by focal transcranial magnetic stimulation of Wernicke’s area. Experimental Brain
Research, 121(4), 371–378.
Tremblay, P., & Gracco, V. L. (2009). Contribution of the pre-SMA to the production of words
and non-speech oral motor gestures, as revealed by repetitive transcranial magnetic stimu-
lation (rTMS). Brain Research, 1268, 112–124. doi: 10.1016/j.brainres.2009.02.076
Vigneau, M., Beaucousin, V., Herve, P. Y., Duffau, H., Crivello, F., Houde, O., . . . Tzourio-
Mazoyer, N. (2006). Meta-analyzing left hemisphere language areas: Phonology, semantics,
and sentence processing. NeuroImage, 30(4), 1414–1432. doi: 10.1016/j.neuroimage.2005.11.002
Wassermann, E. M., Blaxton, T. A., Hoffman, E. A., Berry, C. D., Oletsky, H., Pascual-Leone,
A., & Theodore, W. H. (1999). Repetitive transcranial magnetic stimulation of the dominant
hemisphere can disrupt visual naming in temporal lobe epilepsy patients. Neuropsychologia,
37(5), 537–544.
Watkins, K., & Paus, T. (2004). Modulation of motor excitability during speech percep-
tion: The role of Broca’s area. Journal of Cognitive Neuroscience, 16(6), 978–987. doi: 10.1162/
0898929041502616
Watkins, K. E., Strafella, A. P., & Paus, T. (2003). Seeing and hearing speech excites the motor
system involved in speech production. Neuropsychologia, 41(8), 989–994.
Weyh, T., Wendicke, K., Mentschel, C., Zantow, H., & Siebner, H. R. (2005). Marked
differences in the thermal characteristics of figure-of-eight shaped coils used for repetitive
transcranial magnetic stimulation. Clinical Neurophysiology, 116(6), 1477–1486. doi: 10.1016/
j.clinph.2005.02.002
Wheat, K. L., Cornelissen, P. L., Sack, A. T., Schuhmann, T., Goebel, R., & Blomert, L. (2013).
Charting the functional relevance of Broca’s area for visual word recognition and pic-
ture naming in Dutch using fMRI-guided TMS. Brain and Language, 125(2), 223–230.
doi: 10.1016/j.bandl.2012.04.016
Chapter 6
M agnetoe nc e -
phal o graph y a nd t h e
C ortical Dy na mi c s of
L anguage Pro c e s si ng
Riitta Salmelin, Jan Kujala,

and Mia Liljeström
Introduction
Magnetoencephalography (MEG) has become an increasingly popular method to

study language function in the brain. MEG measures the magnetic field produced by
electrical activity generated by cortical neurons. As thousands of neurons become syn-
chronously activated in a cortical patch, the magnetic fields sum up to create a small,
but measurable, signal outside the head. The resulting changes in the magnetic field can
be tracked with millisecond resolution. Magnetic fields are relatively unaffected by the
tissues surrounding the brain (i.e., skull and scalp), thus providing a spatially accurate
view of the underlying current sources. Efficient source-modeling techniques enable the
reconstruction of cortical sources from the measured signal.
Owing to its combined temporal and spatial sensitivity, MEG can identify different
processing stages of cortical processing as they unfold. MEG is thus ideally suited to
studying the complex cortical activation sequences that occur during language pro-
cessing in the brain. Knowledge about timing and location of activation in the brain
is important since the same general brain area can participate in multiple stages of
language processing, and since the timing of activation can tell us about the specific
processing stage at which a particular brain region participates.
MEG provides a versatile set of neural markers for studying language. Evoked
responses that are averaged with respect to an external event provide a consistent
116 Riitta Salmelin, Jan Kujala, and Mia Liljeström
view of cortical dynamics in language tasks. Cortical rhythmic activity helps to extend
and complement the view provided by evoked responses, and connectivity measures
can reveal networks that support language processing. Using these neural markers,
experiments can be designed to make good use of the strengths of MEG.
Continuous development of source modeling and data analysis techniques allows
the study of increasingly complex aspects of language and cognition using MEG.
Today, distributed patterns underlying semantic processing (Simanova, van Gerven,
Oostenveld, & Hagoort, 2015; Sudre et al., 2012), and global functional networks un-
derlying language (Fonteneau, Bozic, & Marslen-Wilson, 2015; Kujala et al., 2007;
Liljeström, Kujala, Stevenson, & Salmelin, 2015) can be studied with a temporal resolu-
tion that permits the separation of different processing stages.
In this chapter, we review how neural signaling is picked up by MEG, how source
modeling can be used to estimate loci of neural activation, and how MEG has been
used to study language (for a more comprehensive description of the MEG method,
see, e.g., Baillet, Mosher, & Leahy, 2001; Hämäläinen, Hari, Ilmoniemi, Knuutila, &
Lounasmaa, 1993; Hansen, Kringelbach, & Salmelin, 2010). The great advantage of MEG
is that it provides the temporal information not available in functional magnetic res-
onance imaging (fMRI) (see Heim & Specht, Chapter 4 in this volume), and the spa-
tial information not readily available in EEG (see Leckey & Federmeier, Chapter 3 in
this volume). However, each measurement technique has its own strengths and limi-
tations. Understanding the limitations and restrictions related to a particular method
helps in avoiding common pitfalls and dealing with difficulties and, thus, in producing
the best, most reliable brain data. Therefore, we also review problems and common
misunderstandings.
How Brain Activity Can Be Detected

and Quantified with MEG
Postsynaptic Currents in Populations of Cortical

Pyramidal Cells Can Be Detected with SQUIDs
Neural electric currents are extremely weak, and, generally, summation of electric cur-
rent in tens of thousands of nearby neurons is necessary to render the signal detectable
from outside the head. However, not all neural currents are equally well suited for effi-
cient summation. Postsynaptic electric currents in the apical dendrites of cortical py-
ramidal cells are the main source of the magnetic field recorded with MEG (and the
electric potential recorded with electroencephalography, EEG). The parallel orientation
of the neighboring apical dendrites and the relatively long duration of the postsynaptic
potential (~10 ms) can raise the local population-level electric current to a detectable
MEG and the Cortical Dynamics of Language Processing 117
level. Further, the postsynaptic electric current is dipolar in nature, and the resulting
dipolar magnetic field persists furthest in distance. In contrast, for example, an action
potential lasts only about 1 millisecond (ms), and its quadrupolar field dies away rapidly
with distance.
Even the strongest magnetic fields generated by regular cortical activity are extremely
weak in absolute terms, on the order of 100 femtotesla (but can be markedly stronger
for epileptic spikes). While noninvasive recording of the brain’s magnetic fields was first
demonstrated with coils of copper wire (Cohen, 1968), much more sensitive devices
were needed for MEG to have any practical use. The necessary sensitivity came in the
form of superconducting quantum interference devices (SQUIDs). By exploiting weak
links in a superconducting ring (Josephson junction), SQUIDs serve as highly sensitive
transformers of magnetic field into measurable voltage. Optimal sensitivity is achieved
by making SQUIDs rather small. As a result, they couple only weakly to magnetic fields.
Thus, viable MEG recordings require the use of superconducting flux transformers,
composed of a pickup coil and a signal coil that bring the magnetic field to the SQUIDs.
The Pickup Coil Geometry Influences

the Sensor-Level Appearance
The geometry of the pickup coils determines the component of the magnetic field that
is measured and the appearance of the recorded sensor-level MEG signals. Pickup coils
are typically formed of just one wire loop (magnetometer, measuring the absolute field
component) or by two oppositely wound loops (gradiometer, measuring the spatial
gradient of the field). Magnetometers have the best depth sensitivity but, for the same
reason, they also readily pick up signals from sources other than the brain (noise from
the brain-signal recording point of view). Gradiometers cancel out magnetic fields from
faraway sources and are most sensitive to nearby cortical sources. In an axial gradiom-
eter, two oppositely wound loops are placed one above the other, and the distance be-
tween the loops determines the depth sensitivity; with an increasing distance between
the two loops, an axial gradiometer approaches a magnetometer. In a planar gradiom-
eter, the oppositely wound loops are placed on a plane, side by side, forming a figure
eight (cf. Figure 6.1).
The sensor-level appearance of the different types of pickup coils is remarkably dis-
tinct. Magnetometers and axial gradiometers detect the maximum value over the field
maximum, whereas planar gradiometers detect the maximum value over areas where
the magnetic field shows the most rapid spatial change. Thus, as illustrated in Figure 6.1
for a dipolar magnetic field, magnetometers and axial gradiometers detect two maxima,
a positive and a negative one on both sides of the source of electric current (neural
activation), and no signal directly above the source where the field strength is zero. The
planar gradiometers, instead, detect one maximum directly above the source, where
the magnetic field derivative is strongest, and the signal dies away with distance from
(B) (A)
Field
strength
Distance
from
300 fT source
–50...250 ms
(C)
Current
flow
100 fT/cm
–50...250 ms
Figure 6.1. Influence of pickup coil geometry on the MEG sensor signals. (A) Magnetic field
pattern recorded over the left hemisphere at ~100 ms after auditory stimulus onset. The black
arrow indicates current flow in the auditory cortex (direction perpendicular to the course of
the Sylvian fissure). The red area indicates magnetic field emerging from the brain, and the blue
area indicates the re-entering field. These data were recorded with the Elekta Neuromag device,
which has both planar gradiometers (two orthogonal sensors) and magnetometers at the same
102 locations. (B) Magnetometers, or axial gradiometers, detect two maxima, one negative and
the other positive, on both sides of the current. The signal is zero directly above the source. The
measurement helmet is flattened onto a plane and viewed from above, with the nose pointing up-
ward. Each curve denotes the change of magnetic field as a function of time, from 50 ms before
to 250 ms after stimulus onset. (C) The planar gradiometers detect one maximum, directly above
the source current, where the field gradient is largest. Thus, the sensor array now displays only a
single area of strong deflections in each hemisphere, above the auditory cortex.
the source. When there are only a couple of salient, spatially distinct sources, such as in
passive listening to short tones, it is easy to discern the rough source configuration di-
rectly at the sensor level, regardless of the type of pickup coil.
MEG Signals Pass Through the Skull and

the Scalp Unaffected
The closely spherical geometry of the head, with conductivity changes occurring pri-
marily along the radius (brain–skull–scalp), has important consequences for electro-
magnetic fields. Theoretically, it can be shown that in the special case of a sphere where
the conductivity changes occur only along the radius, a dipolar current source generates
a (component of) magnetic field that reflects the intracellular current flow (com-
monly referred to as primary current), with no influence of the volume/return currents
(Hämäläinen et al., 1993). Although the human brain is obviously not fully spherical,
nor does it have homogeneous conductivity, the conductivity changes tend to be lo-
cally quite radial, and the contribution of volume currents is markedly lower than in
EEG. Thus, the spatial profiles of EEG and MEG differ considerably. The passage from
the highly conducting brain tissue through the poorly conducting skull to the highly
conducting scalp strongly affects the volume currents, resulting in considerable spatial
blurring of EEG recordings (where the cortical electric field is sampled at the scalp).
The MEG signals recorded from outside the head are, however, essentially the same that
would be recorded on top of the cortex, with the skull removed (for empirical evidence,
see Okada, Wu, & Kyuhou, 1997).
The approximate spherical symmetry has further consequences for the recorded
MEG signals. Dipolar currents that are oriented fully along the radius do not generate a
measurable magnetic field. The pyramidal cells are oriented normal to the cortical sur-
face, which means that activation strictly limited to the gyral crown or sulcal base might
remain undetected with MEG. All the other signals that are even slightly tilted from the
radial orientation are readily observed. The tangential current flow in the sulcal walls is
optimally detectable with MEG. Notably, in cognitive processing, activation is also un-
likely to be limited to the very narrow cortical areas of fully radial current flow.
Real-Time MEG Signals Facilitate Several

Complementary Views into Brain Function
In a sense, electromagnetic recordings are like a blood sample: when the data are col-
lected with a fairly liberal bandwidth (e.g., 0.01–200 Hz) and stored in their full form,
one can later extract many types of measures from them. An evoked response is the
(presumably) invariable neural activation that always occurs at the same time with re-
spect to a certain event. Since the brain is constantly involved in many operations, the
electrophysiological brain signals show plenty of modulation that is unrelated to the
task of interest. This unwanted “noise” is diminished by averaging responses across

multiple trials of the same type (e.g., presentation of real words or pseudowords).
Averaging the wide-band signal with respect to a certain trigger point, such as a stim-
ulus or movement onset or offset, should highlight the evoked responses tightly linked
to those events. The evoked response is strongest to stimulus changes, and the signal
typically starts to diminish after ~1 second (s) from the trigger. The highly reproducible
and informative evoked response continues to be the most commonly used descriptor
of neural engagement.
By performing a different type of analysis, the background “noise” may reveal other
valuable information about brain processes. Using spectral analysis, one can show that
the cortical cell populations display spontaneous oscillatory activity, most saliently at
around 10 Hz (often referred to as “alpha”) and 20 Hz (“beta”). This activity is strongest
when the individual is resting, with eyes closed (Hari & Salmelin, 1997). Event-related
time-frequency (TFR) analysis has further demonstrated that induced oscillations
(i.e., stimulus-and task-related modulations) may appear at a much wider range
of frequencies, from around 5 Hz (“theta”) to 90 Hz (“low gamma” 30–60 Hz, “high
gamma” 60–90 Hz). Frequencies up to about 30 Hz reflect actual oscillatory processes
with fairly well-defined spectral peaks, whereas the gamma range activity may denote
also more arrhythmic processes, possibly related to spiking activity (He, Zempel, Snyder,
& Raichle, 2010). While there are many suggestions of the possible functional roles of
the different frequency ranges (Bastos et al., 2015; Donner & Siegel, 2011), a unified inter-
pretation is still lacking. In TFR analysis, the signal power per frequency bin is averaged
across multiple trials, with information of signal phase omitted; thus, the resulting
event-related (or induced) modulations reflect time-locked but not necessarily phase-
locked activity. Accordingly, the event-related modulations of band-limited power are
less sensitive than evoked responses to inter-trial jitter in task timing. Generally, the
modulations of rhythmic activity tend to be quite slow, peaking and lasting from hun-
dreds of milliseconds to seconds after the event. The evoked responses and event-related
power modulations in various frequency bands provide markedly distinct windows into
brain processes (Laaksonen, Kujala, Hultén, Liljeström, & Salmelin, 2012) (Figure 6.2).
Source Models Allow Estimation of the Loci

of Neural Activation
MEG sensors record electromagnetic fields of all ongoing neural activity at once. This
is different from, for example, fMRI where the full image is generated by probing one
area at a time. Even though the coupling between a neural source and an MEG sensor
varies over space, each source is seen by all sensors (field spread) and thus the signal
at each sensor is a weighted sum of signals from all sources. Estimation of the spatial
loci of the neural currents from electromagnetic field patterns requires modeling of the
source currents and the conductivity profile of the human head (spherical or realistic
head model). However, there is no single optimal way to model the source structure
speech
word ?
0 300 800 ms
Current
flow
(A) Evoked
–200...1800 ms
(B) 10 Hz
–1...5 s
(C) 20 Hz
–1...5 s
Figure 6.2. Evoked responses and cortical rhythms provide different views into brain func-
tion. In this experiment, a word was presented at time 0, for 300 ms. After an interval of 500
ms, a question mark appeared at 800 ms, prompting the subject to read the word out loud. Each
trial lasted for about 5 seconds. (A) Evoked responses extending over a period of 2 s, including
a 200 ms pre-stimulus baseline interval. There is plenty of activation all over the cortex, espe-
cially within about 1 s after stimulus onset. (B) Event-related modulation of the mean amplitude
of 10-Hz oscillations from the same data. The time window extends from 1 s before word onset to
5 s after it. Modulation, primarily suppression, occurs over occipital and posterior parietal areas.
(C) Event-related modulation of the mean amplitude of 20-Hz oscillations from the same data.
The time window extends from 1 s before word onset to 5 s after it. The effect is, again, mainly sup-
pression, but now occurs along the central sulcus.
for reaching from the measured distribution of magnetic field at the sensors to cortical
distribution of neural current flow. While the forward problem from neural sources to
magnetic field is neurophysiologically, physically, and mathematically well described
and understood (Hämäläinen et al., 1993; Lopes da Silva, 2010; Okada et al., 1997), the
inverse problem (i.e., going in the opposite direction to determine the active brain areas
and the time courses of activation from the measured magnetic fields) is more chal-
lenging. A single current dipole is a reasonable equivalent descriptor of focal cortical
current flow at the typical measurement distance from the cortex (>3 cm). The com-
monly used MEG source estimation approaches, all employing a current dipole as the
source model, can be divided into discrete current-dipole models, distributed source
estimates, and beamformers.
The equivalent current dipole (ECD) approach assumes that the measurements are
generated by one or more active sites in the brain, with each site small enough to ex-
hibit a salient dominant direction of neural current flow that can be well described
with a current dipole as an equivalent source. One identifies the ECD locations,
orientations, and amplitudes that best match the measured MEG data in the least-
squares sense. Typically, a relatively small number (up to 10) of spatially and/or tempo-
rally separable ECD components suffice to account for the MEG data, even in higher
cognitive tasks such as language processing (Salmelin, 2010). The distributed source
estimates consider a continuous spatial distribution of current dipoles, often re-
stricted to a cortical grid determined from the participant’s individual MRI data. The
commonly employed minimum norm estimate (MNE; Gramfort et al., 2014) assumes
that the distribution of current with the minimum overall power among those ca-
pable of explaining the data is the best source-level solution. Although there is no
neurophysiological information to argue for minimization of overall power or ampli-
tude as a relevant criterion, the approach is mathematically appealing. Both ECD and
MNE have proven to be efficient tools for characterizing cortical dynamics of sensory
and motor as well as language and other higher cognitive processing, as evidenced by
evoked responses.
Beamforming is a somewhat different approach, in which the brain volume is scanned
by sequential application of a spatial filter that is optimized to pass activity from a spe-
cific brain area at unit gain while suppressing activity from other areas. To achieve such
suppression, one has to assume that none of the other sources is temporally correlated
with the target location (Van Veen & Buckley, 1988); while brain regions commonly
do display correlated activity, the correlations between them are normally lower than
would be needed to markedly affect the beamforming suppression (Gross et al., 2001).
A current dipole again serves as the elementary model of neural activity at each tested
grid point. The implementation of beamformers entails the use of a data covariance
estimate to construct the spatial filter. Beamformers have been successfully applied to
the power mapping of MEG data in the time (Robinson & Vrba, 1997) and frequency
domains (Gross et al., 2001; Laaksonen, Kujala, & Salmelin, 2008; Liljeström, Kujala,
Jensen, & Salmelin, 2005) and to the estimation of cortico-cortical coherence (Gross
et al., 2002; Kujala et al., 2007) in various tasks.
While source modeling of electromagnetic fields may require some thought,

the relatively straightforward access to the time courses of source-level estimates of
activation is the great strength of MEG. The magnetic field is not markedly blurred
by conductivity changes in different parts of the head, and the specific sensitivity to
the tangential component of the neural currents greatly simplifies the calculations
and interpretations. Indeed, brain-level description of activation, from individual
participants to the group level, has always been at the core of the MEG method. Most
important, MEG facilitates the use of reasonably accurate spatial information as a pow-
erful means of decomposing the sensor-level time courses to separable, functionally
meaningful neural-level time courses.
MEG Facilitates Analysis of Real-Time Connectivity

MEG is well suited for exploring task-related brain networks as it facilitates direct es-
timation of neurophysiological connectivity measures that can be linked to specific
cortical areas. There are two main approaches for studying cortical interactions with
MEG. First, one may describe connectivity within an experimental condition using
connectivity measures that minimize sensitivity to field spread (Nolte et al., 2004; Stam,
Nolte, & Daffertshofer, 2007). Second, it is possible to evaluate connectivity differences
between experimental conditions when their power levels, and thus field-spread
properties, are comparable (Kujala, Vartiainen, Laaksonen, & Salmelin, 2012; Schoffelen
& Gross, 2009).
The rich temporo-spectral content of the MEG signal allows the use of various met-
rics and approaches for characterizing interactions between brain regions. As regards
frequency-specific interactions, coherence among brain regions is thought to consti-
tute a mechanism through which information is transferred within cortical networks
(e.g., Fries, 2005). Using coherence as the measure enables direct whole-cortex map-
ping of connectivity at different frequencies without the need to estimate time series
at the level of cortical sources (Kujala, Gross, & Salmelin, 2008). Coherence reflects
both amplitude and phase co-modulation between areas. One may reduce the effects
of amplitude fluctuations by focusing on the phase dynamics of the signals (Palva &
Palva, 2012; Simões, Jensen, Parkkonen, & Hari, 2003). Moreover, the temporal speci-
ficity of MEG signals facilitates the evaluation of directed interactions between regions,
for example, using Granger causality (Barnett & Seth, 2014; Geweke, 1982; Kujala et al.,
2012; Michalareas, Schoffelen, Paterson, & Gross, 2013) or dynamic causal modeling
(Daunizeau, Kiebel, & Friston, 2009; Woodhead et al., 2014), typically among prede-
fined seed regions (activation or coherence maxima).
Although the field-spread effects combined with the rich spectro-temporal content of
the MEG signals make reliable evaluation of neural connectivity with MEG technically
challenging, the recent years have seen important technical and conceptual advances
(Kujala et al., 2008; Nolte et al., 2004; Palva & Palva, 2012; Schoffelen & Gross, 2010) that
have enabled the successful quantification of cortico-cortical interactions in the resting
state and in various task modalities, with both MNE and beamforming (Gross et al.,
2002; Jerbi et al., 2007; Kujala et al., 2007; Palva, Monto, Kulashekhar, & Palva, 2010).
Connectivity analysis facilitates the study of continuous, increasingly naturalistic tasks
(Kujala et al., 2007), but can equally well be applied to event-related paradigms (Palva
et al., 2010), as long as there are enough data to reach reliable estimates. Interconnected
functional networks may be formed by multiple brain areas (Gross et al., 2002) or by
loops formed by muscles (electromyography, EMG) and the brain (Butz et al., 2006).
Methodological Considerations
Artifactual Signals Should Be Identified and Excluded

as They Can Confuse the MEG Patterns or
Be Misconstrued as Neural Activation
Any remaining magnetic material in the body (e.g., from accidents or surgery) or on
the body (e.g., watch, jewelry, underwire bra, permanent hair or eyelash coloring) may
move with breathing or body movements, generating an electromagnetic field. These
signals are far stronger than brain signals, but their sources can usually be identified
and often removed (Hansen et al., 2010). The heartbeat generates a strong field, which
can be notably picked up by the MEG sensors, especially in children. Because of the
very distinct spatiotemporal pattern of heartbeat that differentiates it from brain signals,
the heartbeat artifact can usually be efficiently removed using principal or independent
component analysis (PCA, ICA; Gross, Baillet, et al., 2013) or temporal signal space sep-
aration (tSSS; Taulu & Simola, 2006) methods.
Electromagnetic signals from eye blinks and movements, as well as face, tongue, and
neck muscle activity also far exceed the strength of brain signals. Methods like PCA,
ICA, and tSSS can help to reduce those artifacts, but as their spatiotemporal patterns
are relatively similar to those of brain signals, it is practically impossible to completely
remove these types of artifacts without risking concomitant removal of relevant brain
activity. Accordingly, whenever possible, it is best to make every effort to minimize
those sources of artifacts in the experiments, or to focus on time windows where they
are minimally present (as evaluated by electro-oculogram, eye tracking, electromyo-
gram, etc.). Averaging across multiple trials can further reduce the (remaining) effect
of artifacts unless the task is such that, for example, participants tend to make inad-
vertent, small eye movements more consistently and time-locked to one stimulus cate-
gory than another.
A remaining eye movement artifact can be problematic for functional interpretations.
Figure 6.3 illustrates the source level appearance of an eye blink. The field has a charac-
teristic pattern with negative values in one hemisphere and positive values in the other,
with strong maxima in the front. The ECD solution does not have to be restricted to a
(A)
(B) (C)
MNE
ECD
Figure 6.3. Eye blink artifact appearing at the source level. (A) Field pattern of MEG signals
associated with eye blinks. (B) ECD analysis finds the source between the eyes, well outside the
brain. (C) Anatomically constrained MNE analysis restricts the solution to the cortex. The blink
artifact is mapped to the cortex as well. The signal seems to be generated, incorrectly, in the left
inferior frontal and anterior temporal cortex.
specific volume, and it can usually pinpoint artifactual sources as they tend to localize
outside the brain. Here, the ECD analysis localizes the source between the eyeballs.
Distributed methods, in order to attain reasonable spatial specificity, confine all source
estimates to the cortical surface. Thus, regardless of whether the signals are actually
generated in the cortex or elsewhere, they are mapped to the cortex. Here, the MNE esti-
mate of cortical activation clusters in the inferior prefrontal and anterior temporal lobes
(Figure 6.3 C). It is worth noting that the most anterior and inferior parts of the frontal
and temporal cortices are also the areas that are usually least optimally covered by the
MEG helmet and, therefore, typically have the lowest signal-to-noise-ratio (SNR).
These general areas are often associated with high-level cognitive processes, including
language processing (Vigneau et al., 2006). Careful removal of eye movement artifacts is
thus of utmost importance in MEG studies of language.
In studies of language production, artifact signals from the face, mouth, and tongue
muscles are unavoidable. Depending on the research question, one may try to mini-
mize their effect on the brain signal analysis by focusing on the interval when the overt
response is being prepared but not yet executed, or by using an experimental design
where the overt response is delayed. An alternative approach is to try to isolate the mag-
netic field pattern related to the mouth movement artifact as cleanly as possible. This
may be successful in situations where salient trigger points can be identified separately
for stimulus presentation, mouth movement (from electromyography recording), and
speech onset (from microphone recording). When the relative onsets of these three
stages jitter across trials by a few hundred milliseconds, it is possible to highlight the
speech production artifact by averaging the MEG signals with respect to the micro-
phone onset without picking up too much of the cortical activation time-locked to
stimulus or speech onset, as those signals mostly average out (Salmelin, Schnitzler,
Schmitz, & Freund. 2000; Salmelin, 2010). The artifact related to speech production
typically shows signals that are strongest at the rim of the sensor array, centering above
the cheek area (Figure 6.4 A). When this field pattern is removed from the evoked
response time-locked to mouth movement onset, salient focal activation is revealed
over the frontal lobes that localizes to the face representation area in the sensorimotor
cortex bilaterally (Figure 6.4 B) (Salmelin, 2010). Even when there are no triggers to
guide the artifact extraction, it may be possible to sufficiently suppress the artifact (e.g.,
using PCA or ICA). However, the success of this exercise should be carefully evaluated,
as it depends heavily on the experimental design and the type and amount of artifacts
in the experimental conditions to be contrasted.
When the actual experimental response is, or could be, mixed with non-brain arti-
fact signals, caution is warranted in analysis and interpretation. It is possible that the
observed source-level patterns are partly artifact driven, especially in brain areas where
the sensitivity of the MEG method (or measurement helmet) is not very high. In such re-
gions (e.g., basal temporal cortex and anterior and medial frontal cortex), any remnants
of major external artifacts or small but systematic movements (muscle activity) of the
eyes, face, neck, tongue, and so on, that tend to occur more regularly with one stimulus
category than another can falsely be interpreted as neural activation, especially when
using cortically constrained distributed source models.
(A) (B)
Figure 6.4. Artifact signal associated with speech production. (A) Field pattern of MEG sig-
nals reflecting artifactual muscle signals during spoken word production (time 0 with respect to
microphone signal onset). (B) Field pattern of MEG signals time-locked to mouth movement
in spoken word production (20 ms after the onset of the electromyogram signal recorded from
mouth muscles), where the artifact field in (A) has been projected out. The white arrow represents
the ECD that best accounts for the local magnetic field pattern. The left side of the MEG sensor
array is shown above, the right side below.
Source: Modified from Salmelin (2010).
Focal and Distributed Source Models Each Have Their

Strengths and Limitations but Neither of Them Yields
the Actual Spatial Extent of the Active Areas
Focal and distributed source models each have their strengths and limitations. Efficient
high-quality multi-ECD modeling requires experienced user intervention. However, it
also simultaneously encourages a thorough familiarization with the data, which impor-
tantly helps to avoid misinterpretations. MNE provides a fairly automatic approach to
source-level mapping, but when the solution is forced to the cortical surface any pos-
sible artifact signals are also mapped to the cortex. It is not possible to define an absolute
threshold of meaningful activation strength in distributed maps; in the focal model, this
translates to the difficulty of deciding how weak a source should still be included in the
model. The most salient source-level activity never goes undetected in either distributed
or focal analysis. MNE provides an estimate of current strength at all cortical locations,
whereas ECD gives non-zero values only at the hot spots of activation, but then also
provides an estimate of the dominant direction of current flow in that locus. The ECD
and MNE methods are thus partly complementary. The choice of the primary approach
is best determined by the research question. Ideally, one would use both focal and dis-
tributed modeling to reach a valid source-level description and interpretation.
Source modeling yields estimated sites of local maxima of activation, but neither
focal nor distributed estimates reflect the true extent or shape of the cortical activation.
Regardless of whether the actual source in the brain is spatially highly focal (as is prob-
ably the case for early sensory activations) or more extended (as might occur for high-
level cognitive processes), it will appear focal in ECD and distributed in MNE: this is
a direct manifestation of the ill-posed nature of the electromagnetic inverse problem.
Independent of the true extent of the sources, both focal and extended source-level
models can be equally good in explaining the measured magnetic fields. While it seems
that many experimenters may be able to accept that the distributed MNE map could
actually reflect more focal neural activity, it may be less obvious that, owing to the con-
siderable distance of the sensors from the sources, an ECD may equally well serve as
an excellent equivalent representation of an extended source (“regional source”)
(Scherg, 1990).
Smoothness of the Electromagnetic Field Is Both

an Asset and a Nuisance
The electromagnetic field is inherently smooth: small dislocations of the source currents
can have a fairly small effect on the spatial configuration of the field. The magnetic field
generated by a current dipole (see Figure 6.1) is modulated more markedly when the
source location changes in the direction perpendicular to the source current, rather
than along the current flow. Similarly, the field pattern is not very sensitive to changes
of source depth. However, any change of the orientation of the current flow, while the
location remains unchanged, has a strong effect on the field pattern. Although the in-
herent smoothness of the magnetic field often hampers accurate localization of neural
currents, it simultaneously has the benefit that small differences in source localization
have very little influence on the estimated time courses of activity in that general source
area. Thus, the MEG-derived regional time courses can be highly accurate even when
there may be some uncertainty in the source locations. Indeed, an important beneficial
consequence of the inherent spatial smoothness is that the exact choice of the areal ex-
tent and shape of activation in a distributed model is not critical for an accurate estima-
tion of activation time courses, as long as the selection includes the local maximum. Nor
does some variation in the focal ECD parameters compromise the accuracy of the time
course of the neural activation it describes. MEG-based “regions of interest” may thus
be most appropriately referred to as “regional labels.”
Because of the inverse problem and the smoothness of the field pattern, MEG cannot
boast high absolute accuracy of source localization, except in the rare cases of early
sensory responses when essentially only one cortical locus shows activation. When
that happens, for example, for somatosensory stimulation of the hand, the hand repre-
sentation area in the primary somatosensory cortex can be localized with an accuracy
of a few millimeters. This type of mapping has great clinical value in preparation for
surgical treatment of tumors, stroke, and epilepsy. Systematic relative differences be-
tween sources can often be determined with very good accuracy (e.g., representations
of different digits in the hand area; Forss, Jousmäki, & Hari, 1995). In language tasks,
multiple brain areas are active simultaneously, and the “noise” from other active
sources can hamper the localization accuracy of individual sources; in practice, the lo-
calization accuracy is in the centimeter rather than millimeter range. As regards spa-
tial resolution, two dipolar current sources located next to each other, with the same
direction of current flow, can usually be differentiated if they are at least 2 centimeters
apart (Liljeström, Kujala, Jensen, & Salmelin, 2005). However, the more dissimilar the
directions of current flow, the better dissociable are the sources. Moreover, distinct
time courses of activation can help immensely in differentiating between spatially close
sources.
Connectivity Estimates Can Be Severely Confounded

by Field Spread
Because all MEG sensors see the signals from all neural sources, to varying degrees,
sensor-level estimates of connectivity are deeply confounded (Schoffelen & Gross,
2009). Indeed, source-level estimates are particularly critical in connectivity analysis.
However, the estimated cortical time courses are not entirely free from the problem-
atic effects of field spread, either. While it is formally possible to compute connectivity
measures between time series estimated for any brain areas, caution is necessary.
If the field-spread effects are not properly taken into account, one may obtain spu-
rious estimates of connectivity. It would be particularly easy to misinterpret changes
in the level of activity between conditions as changes in the level of interaction across
brain regions; such effects are especially prominent when considering nearby sources.
Moreover, as connectivity measures are generally very sensitive and can pick up co-
modulation in low-amplitude signals, they also readily detect spurious effects from
small remnants of external artifacts. Thus, in MEG connectivity studies, it is even more
imperative than in activation studies to collect high-quality data and to ensure that the
detected effects do indeed have a cortical origin.
Different measures of connectivity are optimally suited for different types of analyses.
For example, resting-state connectivity is frequently assessed using the imaginary com-
ponent of coherence as the measure, which helps to reduce the effect of field spread
(Marzetti et al., 2013). As imaginary coherence is sensitive only to time-lagged synchro-
nization, the instantaneous field spread from a single neural generator does not lead
to spurious connectivity between separate brain areas (Nolte et al., 2004). However, as
information about the absolute amount of coherence is simultaneously lost, imaginary
coherence is best suited for the study of isolated experimental conditions, such as the
resting state, where the interest is primarily in whether there might be a link with an-
other area. This measure is less optimal for estimating functionally relevant changes of
connectivity between experimental conditions, which is typically the focus of cognitive
studies. In those cases, it is better to consider the full coherence and to reduce the effect
of field spread by limiting the functional interpretation to long-distance connections.
When contrasting connectivity between two conditions, it is essential to ensure that
their power levels do not differ markedly in the time and frequency ranges of interest
(Kujala et al., 2012; Schoffelen & Gross, 2009, 2010).
Spatial Dependence of Distributed Source-Level

Estimates Must Be Taken into Account
in Statistical Testing
While it is formally possible to calculate fMRI-type whole-head contrasts between dis-
tributed source-level MEG estimates, such estimates do not reflect source size or shape,
and the interpretation of any resulting differences can thus be problematic. The values
at the neighboring voxels or grid points are spatially highly dependent, and the maps
should be considered in their entirety, taking properly into account the effects of the
point-spread functions of the source estimates and crosstalk between cortical regions;
cluster-based statistics (Maris & Oostenveld, 2007) may be developing toward this goal.
A regional label approach helps to circumvent the issue of field spread, to a reason-
able degree, as functional differences are then compared between conditions per source
areas whose shape and size are kept constant. One reasonable approach is an omnibus
search in which regional labels are selected from activation maps pooled across all
experimental conditions and participants, and the signal values within those regions
are then evaluated per condition, per participant (Hultén, Karvonen, Laine, & Salmelin,
2014; Lee, Hämäläinen, Dyckman, Barton, & Manoach, 2011). In ECD modeling, the
corresponding omnibus approach identifies ECDs that account for the data across
conditions, per participant, and then cluster them across participants into groups by
similarity of location, direction of current flow, and time course of activation (Hultén
et al., 2014; Salmelin, 2010). It is also possible to use a priori brain atlas divisions, as long
as the entire cortex is considered at once. However, the local activation maxima may not
fall neatly onto a single atlas division, as existing brain atlases are mostly based on ana-
tomical segregation. Furthermore, using sulci as dividing lines is problematic for MEG,
which is most sensitive to activity within the sulci but cannot readily tell apart the two
opposing sides of a sulcus.
How Language Function May

Be Studied with MEG
Evoked Responses Provide a Consistent View of Cortical

Dynamics in Language Tasks
Evoked responses are the trustworthy workhorse of electromagnetic brain mapping.
Two decades of research using evoked MEG responses have resulted in a solid view of
cortical dynamics in written and spoken word processing, as well as in picture naming
(Salmelin, 2007), that has been further exploited to address more specific questions: for
example, the neural correlates of semantic representation and composition (Halgren
et al., 2002; Helenius, Salmelin, Service, & Connolly, 1998; Pylkkänen, Bemis, & Blanco
Elorrieta, 2014; Pylkkänen & Marantz, 2003; Vartiainen, Parviainen, & Salmelin, 2009);
morphosyntax (Friederici, Wang, Herrmann, Maess, & Oertel, 2000; Pylkkänen,
Feintuch, Hopkins, & Marantz, 2004; Service, Helenius, Maury, & Salmelin, 2007;
Vartiainen, Aggujaro, et al., 2009); language learning (Dobel et al., 2010; Hultén et al.,
2014; Nora et al., 2012); and developmental language disorders (Helenius, Tarkiainen,
Cornelissen, Hansen, & Salmelin, 1999; Parviainen, Helenius, & Salmelin, 2005;
Salmelin, Schnitzler, et al., 2000).
MEG is well suited for the study of speech perception. Auditory signals, including
speech, evoke a typical sequence of responses in the Heschl’s gyrus and the adjacent
temporal sulci that is readily picked up by MEG. The auditory cortex is first activated at
~15 ms (Liegeois-Chauvel, Musolino, Badier, Marquis, & Chauvel, 1994) after an audi-
tory stimulus, but salient detection of the earliest responses (<50 ms) usually requires
hundreds of repetitions of the same stimulus, which is typically not an ideal paradigm
for language studies. However, a strong transient response at ~100 ms after stimulus
onset (“N100m”) may in some cases stand out from the background activity, even in
a single trial. Although the time window of the N100m response has been associated
with acoustic-phonetic processing, a fairly low-level process in speech perception
(Parviainen et al., 2005; Tiitinen, Mäkelä, Mäkinen, May, & Alku, 2005), in neurophysi-
ological terms N100m is a rather late cortical response. Studies using an oddball design
(i.e., repeated presentation of a certain stimulus, randomly interrupted by a different,
rare stimulus) have indicated that phonemes are distinguished by ~150 ms (Näätänen
et al., 1997; Vihla, Lounasmaa, & Salmelin, 2000).
The time course of lexical-semantic processing in spoken word perception has been
probed with various types of stimulus and task manipulations and has been associated
most consistently with the onset phase (~200–400 ms) of a strong sustained activation
that typically lasts from about 200 to 800 ms after word onset (Helenius et al., 2002).
The phenomenon of sustained auditory cortical responses, as such, is not limited to
linguistic stimuli. Sustained responses are evoked by environmental sounds (Renvall,
Formisano, et al., 2012) and even by long-duration meaningless auditory stimuli (Eulitz,
Diesch, Pantev, Hampson, & Elbert, 1995). Furthermore, it is important to realize that
none of the responses and time windows of activation can be associated solely with one
function; rather, some degree of modulation of the evoked response strength or timing
typically can be linked with a specific processing stage.
From the MEG point of view, when advancing in time, the cortical locus of the
evoked response to spoken words in and around the auditory cortex remains essentially
the same (Figure 6.5). However, it may be possible to demonstrate, at the group level,
that the center of activation shifts between different time windows or experimental
conditions (e.g., between ipsi-and contralateral responses to tones (Parviainen et al.,
2005)). Furthermore, within the source region it is possible to identify substructures
that display unique time courses of activation. Figure 6.5 illustrates group-level distrib-
uted MNE and focal ECD estimates when participants were listening to spoken words,
knowing that they would be expected to subsequently reproduce each word (Nora
et al., 2012). The MNE view illustrates how activation remains in the same superior
temporal locus (Figure 6.5 A). The ECD analysis can further exploit MEG’s sensitivity
to the orientation of neural current flow. Across individuals, ECD analysis identifies
two consistent clusters within this same general region, one with superior-inferior
current flow, indicating neural activation in the lower lip of the Sylvian fissure, and
the other with a posterior-anterior current flow. The two types of sources have very
different time courses of activation (Figure 6.5 B) and different reactivity to experi-
mental manipulation, and thus different functions (Nora et al., 2012; Nora, Renvall,
Kim, Service, & Salmelin, 2015). Here, MNE and ECD analyses, together, confirmed
the spatiotemporal pattern of cortical activation, and the substructure obtained with
the ECD analysis allowed to extract remarkably clean time courses of activation for de-
tailed functional analysis.
The accurate temporal information linked with brain areas provides a unique view to
brain function. For example, the MNE sequence in Figure 6.5 also suggests a weak acti-
vation in the frontal cortex, which was identified with ECD analysis as well (not shown).
Motor or premotor cortical activation during the phase of spoken-word perception may
(A)
100–200 ms 200–300 ms 300–600 ms 600–900 ms 900–1200 ms
0.25
0.15
(B)
nAm nAm
20 20
0 1200 ms 0
Figure 6.5. Cortical processing of spoken words. (A) Source localization using distrib-
uted modeling (minimum norm, MNE), left hemisphere displayed. The distribution of activa-
tion, plotted as a function of time from 100 to 1200 ms after stimulus onset, is an average over 10
subjects; noise-normalized MNEs of individual participants were morphed to a standard brain
and normalized. A salient activation maximum persists in the superior temporal cortex. (B) ECD
analysis reveals a functional substructure within the superior temporal maximum. Each dot
represents the center of an active cortical patch in one subject and the line attached to it the mean
direction of current flow in that area. There are two clusters of sources with practically orthogonal
current flow and markedly different time courses of activation. Source-level activation strength is
given in nanoamperemeters (nAm).
Source: Modified from Nora et al. (2012); Nora et al. (2015).
not be that surprising when participants are prepared to reproduce the heard word im-
mediately afterward. However, the relevance of the motor cortex in speech perception
has received much interest, and the possibility of directly testing whether the motor
cortex might be active at an early or late stage of speech processing makes MEG an at-
tractive tool for this type of research (Möttönen, van de Ven, & Watkins, 2014; Shtyrov,
Butorina, Nikolaeva, & Stroganova, 2014). The robust N100m response, in particular,
allows for fine distinctions in timing, on the order of milliseconds. In the later time win-
dows, behaviorally relevant differences in timing may typically be identified in the range
of tens to hundreds of milliseconds.
Word reading and picture naming activate the cortex much more widely than speech
perception. Systematic stimulus manipulations have allowed associating part of these
dynamics with specific processing stages. In word reading, the functionally identified
sequence starts from stimulus-non-specific visual activation in the occipital cortex at
~100 ms, proceeds to letter-string analysis ~150 ms that is strongly lateralized to the left
inferior occipitotemporal cortex, and then continues particularly to the left superior
temporal cortex from ~200 ms onward, with the signal modulations associated with
lexical-semantic, phonological, and even syntactic processing (Figure 6.6) (reviewed in
Salmelin, 2007). This cortical representation of lexical-semantic processing is spatio-
temporally remarkably similar for spoken and written words (Vartiainen, Parviainen,
& Salmelin, 2009). The timing of the sustained left superior temporal response has been
(A) Visual Letter-string Word

features analysis meaning
Activation
strength
0 400 800 0 400 800 0 400 800

Time (ms)
(B) S1 Sentences: S2 Word lists: S3 Single words:
Expected ending Related meaning Short words
Plausible meaning Unrel meaning Long words
Wrong meaning Related sound form Short pseudow
Plausible letters Unrel sound form Long pseudow
0 400 800
Time (ms)
S4 Single words: S5 Sentences:

High-freq base form Correct ending
High-freq inflected Wrong meaning
Low-freq base form Wrong word class
Low-freq inflected Wrong morphosyntax
Figure 6.6. Cortical dynamics of word reading. (A) Among the extensive bilateral activa-
tion in reading, it is possible to identify three spatiotemporally and functionally highly con-
sistent responses. Basic visual feature analysis is reflected in early, ~100-ms transient activation
around the occipital midline. Letter-string activation occurs in the left inferior occipitotemporal
cortex at ~150 ms. This stage reflects pre-lexical analysis, as the response is the same to words
and nonwords, and even to consonant strings. The subsequent sustained activation in the left
superior temporal cortex at 200 to 600 ms does differentiate between words and nonwords and
reflects analysis of word meaning. (B) The left temporal sustained response seems to play a role in
multiple types of processing, ranging from semantic and phonological analysis to morphosyntax.
Examples from studies that used Finnish stimuli (agglutinative language with transparent or-
thography). Study 1 (S1; Helenius et al., 1998) used sentences that created a very high expectation
for a specific final word. The expected word suppressed the sustained response, whereas words
linked, for example, with the behavioral measure of reading speed (Helenius et al., 1998).
An early ~100 ms activation of the left inferior frontal cortex (Broca’s area) has addition-
ally been reported in processing of English words and interpreted as fast phonological
access in visual word recognition, possibly related to the contrived mapping between
letters and sounds in the English language (Wheat, Cornelissen, Frost, & Hansen, 2010;
Woodhead et al., 2014).
In picture naming, there is initial widespread activation in the occipital cortex within
the first ~200 ms, and from about ~300 ms onward the activation advances to posterior
temporal and frontal cortices (Levelt, Praamstra, Meyer, Helenius, & Salmelin, 1998;
Liljeström, Hultén, Parkkonen, & Salmelin, 2009; Salmelin, Hari, Lounasmaa, & Sams,
1994; Vihla, Laine, & Salmelin, 2006). It has been fairly hard to associate specific cor-
tical areas and time windows of cortical activation in picture naming with processing
stages postulated by behaviorally based models. However, the left posterior temporal
activation has been linked with both phonological and semantic processing, left frontal
activation with output phonology, and the bilateral frontal activations with articulation
(Hultén, Vihla, Laine, & Salmelin, 2009; Vihla et al., 2006).
Most often, MEG evoked responses are analyzed until the point of actual speech
production and the nearly inevitable artifact signal from face and tongue muscles. It
may be possible, although challenging, to markedly reduce the artifact and to track
Figure 6.6. Continued
with a wrong meaning and words with a wrong meaning but sharing the initial letters with the ex-
pected words (plausible letters) resulted in a strong response (“N400m”). The semantic gradation
was manifested as the medium strong response to words that were not the expected words but
had a plausible meaning in the sentence context. Study 2 (S2; Vartiainen, Parviainen, & Salmelin,
2009) used lists of four words that were related either semantically or phonologically, and the
final word either fulfilled the expectation set by the preceding three words (Related meaning/
sound form) or not (Unrel meaning/sound form). Both modulations affected the response, but
the semantic influence was markedly stronger. Study 3 (S3; Wydell et al., 2003) examined cortical
responses to short and long real words and pronounceable pseudowords (pseudow). The dual-
route model of reading (Coltheart et al., 1993) predicts that the word length would have little influ-
ence on real word (lexical route) but a strong influence on pseudoword (grapheme-to-phoneme
transformation) reading. Indeed, such an effect was found in the sustained response, suggesting a
link to phonological analysis. Study 4 (S4; Vartiainen, Aggujaro, et al., 2009) investigated reading
of monomorphemic (base form) and inflected high-and low-frequency (high/low-freq) words.
Only the low-frequency inflected words showed an increased processing cost (stronger activa-
tion), indicating that the sustained response also reflects morphosyntactic processing and that
only very high-frequency inflected words are not parsed. Study 5 (S5; Service et al., 2007) used
sentences to explore morphosyntax. The strongest response was recorded to final words with a
wrong meaning, while a correct ending resulted in a suppressed response. Notably, a medium
strong response was elicited by endings where the correct word had the wrong inflectional form
(wrong morphosyntax) or the expected noun was replaced by a verb (wrong word class).
Source: Modified from Helenius et al. (1998); Salmelin, Helenius, & Service (2000); Service et al. (2007);
Vartiainen, Aggujaro, et al. (2009); Vartiainen, Parviainen, & Salmelin, (2009); Wydell et al. (2003).
the response all the way through the speech act (Salmelin, Schnitzler, et al., 2000; see
also the discussion of artifactual signals earlier in this chapter), but the source-level
model is certainly less reliable at that point. Small head movements due to speech
are, however, less problematic than in fMRI: the spatially smooth electromagnetic
field (discussed earlier in this chapter) can tolerate some movement without com-
pletely distorting the source-level model. MEG has helped to assess cortical correlates
of phonetic, phonological, and word-level learning (Dobel et al., 2010; Hultén et al.,
2014; Nora et al., 2012; Zhang et al., 2009). These studies have focused on the relative
changes of the cortical responses to various types of stimuli (familiar, newly learned,
unfamiliar).
Advancing from vocabulary learning to acquisition of syntax requires assessment of
entire sentence structures. Figure 6.7 illustrates cortical areas involved in a task where
participants, prompted by images of two animals, produced descriptive sentences of
the type “bear begs mouse” using a newly learned artificial language (nouns, verbs,
morphosyntactic rules) as well as their native Finnish language (Hultén et al., 2014).
The MEG data highlighted the role of the right temporal cortex in a semi-naturalistic
sentence-level task. This type of complex paradigm activates the cortex widely, with
considerable inter-individual variation in the activation patterns. Applying both dis-
tributed, cortically constrained MNE and focal, spatially unconstrained ECD models
can be very helpful in determining the regions that reflect neural activation, as well as in
obtaining an estimate of how consistently a salient activation in or near a certain cortical
region was observed in individual participants (Figure 6.7).
Evoked responses are an efficient tool when the experiment can be designed such that
there are well-defined trigger points for averaging, and long enough intervals between
triggers for reliable estimation of baseline. It is also possible to evaluate the effects of more
continuous stimulation on specific cortical areas indirectly, using probe stimuli. For ex-
ample, the stage of the auditory cortex has been probed by recording evoked responses
to simple tone stimuli while the participant was reading silently or aloud (Numminen,
Salmelin, & Hari, 1999). One may choose to extract additional information by evaluating
the modulation of rhythmic activity (Laaksonen, Kujala, Hultén, Liljeström, & Salmelin,
2012; Salmelin, Helenius, & Service, 2000) or connectivity changes (Kujala et al., 2012).
When the stimuli or tasks are temporally less precisely defined or continuous, rhythmic
activity and connectivity are likely to be the tools of choice.
Cortical Rhythmic Activity Helps to Extend and

Complement the View Provided by Evoked Responses
In neurolinguistics, most MEG studies have used evoked responses as a measure of
neural engagement, with much fewer studies applying rhythmic activity. Generally, it
seems that the two types of measures pick up rather different types of neural activity.
Indeed, when both measures have been evaluated from the same cognitive data set
(e.g., Laaksonen et al., 2012; Salmelin, Schnitzler, et al., 2000), they have provided quite
(A)
Sentence
?
dosuda nosepy ??????
Preparation
Cloze task
phase
Word sequence
garomi xxxxxx ?????? ?
0 1.5 3 4.5 6 7s
(B) (C) Along the word list Inflected noun >

activation ↓ activation ↑ noun in base form
Figure 6.7. Sentence-level learning of vocabulary and grammar. A miniature language was
created, called Anigram. It contains 20 nouns (animal names) and 10 transitive verbs, and has a
subject-verb-object structure. The sentence object is indicated by a suffix, which is determined
by the subject gender. (A) Example of the stimulus images. The image was first presented for 1.5 s,
allowing the participants to plan the sentence. The more controlled cloze test part then started,
where the words were printed on top of the picture, one by one, for 1.5 s each. Generation of
the sentence object form was the most relevant test for grammar and word learning. A string of
question marks prompted the participant to covertly produce the word form and then produce
it overtly when the picture was replaced by a single question mark. As a control, Word sequence
trials showed two animals standing next to each other, without a depicted action. The task was
to retrieve the two animal names in their base form, without a syntactically controlled sentence
structure. (B) Combining MNE and ECD modeling allowed the identification of MNE-derived
group-level regional labels (black blobs) that represented real neural activity as they agreed with
ECD (white dots) clustering across subjects. (C) The left and right temporal cortices displayed
markedly different functions. In the left temporal cortex, activation was overall reduced when
advancing along the word list (sentence or word sequence), but the response to the final word was
stronger when it was inflected (sentence) than in the base form (word sequence). In contrast, in
the right temporal cortex, activation was increased when advancing along the word list.
Source: Modified from Hultén et al. (2014).
complementary views of cortical implementation of language function. For example,

when tested on multiple variants of a picture-naming paradigm, evoked responses
and rhythmic modulation in the salient 10-Hz and 20-Hz bands yielded largely sepa-
rate networks, with spatial overlap mainly in the primary sensorimotor and visual
areas. Moreover, in the cortical regions that were identified with both measures, the
experimental effects that each measure conveyed differed in terms of timing and func-
tion (Laaksonen et al., 2012).
Similar to evoked responses as the workhorse of noninvasive electromagnetic map-
ping, modulation in the gamma-band activity has assumed the corresponding role in
intracranial recordings (electrocorticography, ECoG). It is thought to pinpoint the func-
tionally most interesting cortical phenomena (Lachaux, Axmacher, Mormann, Halgren,
& Crone, 2012) and has been proposed as the most direct counterpart of the blood ox-
ygenation level dependent (BOLD) fMRI signal (McDonald et al., 2010; see Heim &
Specht, Chapter 4 in this volume). With MEG, gamma activity can be quite salient in the
visual cortex (Hoogenboom, Schoffelen, Oostenveld, Parkes, & Fries, 2006) but often
is less conspicuous in other areas. Nevertheless, gamma activity has also been reported
with MEG in areas other than the visual cortex (Lehongre, Ramus, Villiermet, Schwartz,
& Giraud, 2011; van Ackeren, Schneider, Musch, & Rueschemeyer, 2014), especially
when its SNR has been enhanced through correlation between frequencies (Gross,
Hoogenboom, et al., 2013; Holz, Glennon, Prendergast, & Sauseng, 2010), areas (Betti
et al., 2013; Kujala et al., 2012) or different imaging measures (Kujala et al., 2014).
Event-related modulations of the lower, more salient frequency bands have been infor-
mative in MEG studies of language (Arnal, Wyart, & Giraud, 2011; Bastiaansen, Magyari,
& Hagoort, 2010; Longcamp, Tanskanen, & Hari, 2006). For example, motor cortex in-
volvement in speech production has been tracked with the help of 20-Hz modulation, as
the muscle artifacts can be less problematic in that frequency band than at frequencies
below 10 Hz or in the gamma range or when using evoked responses. Such studies have
shown, for example, an active role for the motor cortex in cognitive control of visually
triggered mouth movements beyond mere movement execution, left-hemisphere prec-
edence for control of mouth movements, and clear focus to the face/mouth area in lin-
guistic mouth movements but involvement of the neighboring hand representation area
in nonlinguistic mouth movements (Saarinen, Laaksonen, Parviainen, & Salmelin, 2006;
Salmelin & Sams, 2002). In speech perception, cortical rhythmic activity is often utilized
in a rather different manner, by focusing on coupling between the speech input and the
time course of auditory cortical activation, as well as cross-frequency coupling within the
auditory cortex. Such studies have found, for example, that the listener’s auditory cortex
tracks the speaker’s prosodic rhythm (Bourguignon et al., 2013), the theta band (~5 Hz)
tracks and discriminates spoken sentences (Luo & Poeppel, 2007), the cerebro-acoustic
phase locking is strongest for intelligible speech (Peelle, Gross, & Davis, 2013), and cou-
pling between theta and gamma oscillations plays an important role in the segmentation
and coding of speech (Gross, Hoogenboom, et al., 2013).
Connectivity Measures Can Reveal Networks

That Support Language Processing
Connectivity analysis is being increasingly applied to language function, both in form
of EMG–MEG coherence estimation and as full-fledged all-to-all cortico-cortical
connectivity computation. For example, it has been demonstrated that during articu-
lation, coherent oscillatory coupling between the mouth sensorimotor cortex and the
mouth muscles is strongest at the frequency of the individual spontaneous rhythmicity
of speech at 2–3 Hz, which is also the typical rate of word production (Ruspantini et al.,
2012). During writing, EMG–MEG coherence was found to appear primarily in the fre-
quency of writing movements (3–7 Hz) while MEG–MEG coherence between cerebral
sources occurred at ~10 Hz (Butz et al., 2006). Direct cortico-cortical coherence estima-
tion applied to reading (rapid serial visual presentation of connected text) also identified
~10 Hz as the dominant frequency of cortico-cortical interaction. When Granger cau-
sality was evaluated among the most densely connected cortical areas, the left inferior
occipitotemporal cortex, involved in early letter-string processing, and the cerebellum
turned out to be the main forward driving nodes of the network (Kujala et al., 2007). It is
also possible to track the evolution of connectivity in an event-related design, with rea-
sonable temporal specificity (~300 ms), as illustrated by comparison of coherence maps
over the course of silent versus overt picture naming (Figure 6.8) (Liljeström et al., 2015).
Language Processing Can Be Studied in

the Developing Brain
The spatiotemporal dynamics accessible with MEG can provide interesting insights
to the development of the cortical correlates of language. MEG recordings are safe
and completely noninvasive. In addition, the noiseless environment is more pleasant
than the noisy and confined MR environment, which is an important advantage when
studying young children. As long as the participants can stay reasonably well in place,
which tends to be the case for small babies and children older than about 7 years,
MEG recordings are technically feasible (Huotilainen, Shestakova, & Hukki, 2008;
Kuhl, Ramirez, Bosseler, Lin, & Imada, 2014; Parviainen, Helenius, Poskiparta, Niemi,
& Salmelin, 2011), particularly when the head position within the helmet can be re-
corded continuously. MEG has been used to follow the development of the auditory
system in discriminating speech sounds in newborns and infants from 6 months of age
(Huotilainen et al., 2008; Kuhl et al., 2014), permitting the identification and separation
of cortical sources in auditory versus motor regions (Kuhl et al., 2014).
MEG can also be utilized to compare both spatial and temporal characteris-
tics of neural activation between children and adults (Kuhl et al., 2014; Parviainen
et al., 2011). Figure 6.9 illustrates the time course of activation in word reading in
7-to 8-year-old children, compared with the adult time course (Parviainen, Helenius,
Poskiparta, Niemi, & Salmelin, 2006). Although these children were still fairly fragile
readers, the overall sequence of activation was qualitatively quite similar to that in
adults: stimulus-non-specific visual activation in the occipital cortex followed by a tran-
sient letter-string activation in the left inferior occipitotemporal cortex and sustained
activation in the left superior temporal cortex. However, the activation was significantly
delayed in children, by ~50 ms in the first stage of low-level visual analysis, by ~100 ms
0–300 ms 300–600 ms
SFG
SFGO
SFG
IF
MFG
SFGO
LH RH
MFG
O
MF
MFG G
IFG GO
O
IF
IFG rC
LH RH LH RH
P GO
P C
R
Fr Fr Fr Fr
P rC
Po O
RO oC
Cen Cen Cen Cen SF SFM
M M
SM O SF F M O
S MA
PC A S L
Med Med Med Me d L PC
IN IN
Ins Ins Ins Ins HES HES
STG STG
TPS TPS
MTG MTG
Tpl Tpl Tpl Tpl T PM TPM
ITG ITG
ACIN
A CIN
N MC
Med Med Med Med MCI IN PCI IN
PC Q PQ N
P
S
L IP PL
Ptl Ptl Ptl Ptl SP PL
I L
A
G
SM G
SM
Occ Occ Occ Occ AG G
SI
FU
FU G
MO
Q
C
C
S O SI
Q
IOG
SO G
IOG
MO G
LING
LING
SFG Superior frontal gyrus, dorsolateral PCL Paracentral lobule SPL Superior parietal lobule
SFGO Sup. frontal gyrus, orbital part IN Insula IPL Inferior parietal lobule
MFG Middle frontal gyrus HES Heschl gyrus SMG Supramarginal gyrus
MFGO Mid. frontal gyrus, orb. part STG Superior temporal gyrus AG Angular gyrus
IFG Inferior frontal gyrus TPS Temporal pole, sup. temp. gyrus FUSI Fusiform gyrus
IFGO Inf. frontal gyrus, orb. part MTG Middle temporal gyrus SOG Superior occipital gyrus
PrC Precentral gyrus TPM Temporal pole, mid. temp. gyrus MOG Middle occipital gyrus
PoC Postcentral gyrus ITG Inferior temporal gyrus IOG Inferior occipital gyrus
RO Rolandic operculum ACIN Anterior cingulate gyrus LING Lingual gyrus
SFM Sup. frontal gyrus, medial MCIN Middle cingulate gyrus Q Cuneus
SFMO Sup. frontal gyrus, medial orbital PCIN Posterior cingulate gyrus C Calcarine fissure and
SMA Supplementary motor area PQ Precuneus surrounding cortex
Figure 6.8. Task-dependent modulation of coherence in cortical networks in picture naming. Connectivity circles at 0–300 ms (left) and 300–600 ms
(middle), with respect to the onset of the pictured item, indicate which areas of the parcellated cortex (right) were more coherently active when preparing
for overt naming than for naming silently. Pooled across frequencies 3–90 Hz.
Fr = frontal, Cen = central, Med = medial, Ins = insular, Tpl = temporal, Ptl = parietal, Occ = occipital.
Source: Modified from Liljeström et al. (2015).
(A) Visual Letter-string Word (B)

features analysis meaning
60 r = –0.8
p < 0.01
Letter-string 40
activation
Activation strength
strength 20 Adult
(nAm)
level
0
0 400 800 0 800 0 400 800 –1 0 1
Time (ms) Phonological awareness
Figure 6.9. Development of language function. (A) Cortical dynamics of silent reading in

seven-to eight-year-old children (colored curves) and in fluently reading adults (black curves).
In children, the sequence of activation was qualitatively similar to that in adults but delayed in
time. (B) A letter-string-sensitive response was detected in about half of the children. Those
children showed a strong correlation between phonological awareness, estimated with a set of
behavioral tests, and the cortical activation strength. The response strength was decreased with
better phonological skills, approaching the adult level of activation, which is generally much
lower than in children.
Source: Modified from Parviainen et al. (2006).
at the stage of letter-string processing, and even more by the sustained superior tem-
poral response reflecting lexical-semantic processing. The somewhat larger variability
of response timing in children than adults further allowed the demonstration of signif-
icant correlation between the timing of the stages of visual and letter-string processing,
on the one hand, and the letter-string and semantic processing, on the other hand. The
strength of the letter-string response in children correlated with their behaviorally
estimated phonological abilities.
Language Lateralization and Disorders May Be Assessed

with MEG
MEG serves as a clinical tool, and it is widely used to identify brain sources of epi-
leptic spikes, to map sensorimotor cortex prior to brain surgery, and to assess the brain
effects of stroke. As regards language processing in the clinical context, the aspect that
is being most fervently addressed is that of language lateralization, with MEG as a
means to replace the highly invasive Wada test, possibly combined with other imaging
measures (Kamada et al., 2007). Proposed approaches include counting the number of
localizable ECDs in the left and right hemisphere when participants see or hear words
(Papanicolaou et al., 2004; Tanaka et al., 2013), evaluation of the spatial sensitivity
of activation maps (D’Arcy et al., 2013), estimation of beta-band power modulations
(Findlay et al., 2012), and comparison of auditory-cortex N100m responses to vowels
versus tones (Kirveskari, Salmelin, & Hari, 2006).
When characterizing aphasia, it can be particularly important to collect data from

two separate sessions, as there may be far more day-to-day variation in a damaged
brain than in a healthy one, partly due to emergence of low-frequency activity (Laine,
Salmelin, Helenius, & Marttila, 2000). MEG studies have shown, for example, that the
generally remarkably similar cortical dynamics of action versus object naming from the
same images in healthy controls were markedly dissociated in an aphasic patient with
specific difficulties with noun naming (Sörös, Cornelissen, Laine, & Salmelin, 2003);
that effortful reprocessing of perceived sentences in short-term memory can support
improved comprehension in aphasia (Meltzer, Wagage, Ryder, Solomon, & Braun, 2013);
and that successful treatment of word-finding difficulties in aphasia can strengthen
activation related to picture naming in the damaged left hemisphere (Cornelissen
et al., 2003).
The excellent time resolution of MEG has proven valuable particularly when studying
how individuals with developmental language disorders perform in linguistic tasks.
When reading-impaired adults are compared with non-reading-impaired controls, the
cortical responses within the first ~200 ms after spoken or written word presentation
differ in activation strength, but from ~200 ms onward, in timing: in reading-impaired
individuals, the cortical response reflecting lexical-semantic processing is delayed both
in speech perception (~50 ms) and reading (~100 ms) (Helenius, Salmelin, Service, &
Connolly, 1999; Helenius et al., 2002). In developmental stuttering, functionally im-
portant timing differences appear in speech preparation, outside of the core language
processes (Figure 6.10): in fluent speakers, the left inferior frontal cortex was found to
be activated first, followed by left motor/premotor cortex, but in the stutterers the order
was reversed, thus suggesting that motor preparation started prior to articulatory plan
ning (Salmelin, Schnitzler, et al., 2000).
Timing Strength
Fluent speakers
Stutterers
0 800 ms 0 800 ms 0 800 ms

Time (ms)
Figure 6.10. Altered cortical dynamics in stuttering. Source areas in which differences were
found between groups of fluent speakers and stutterers, with the time courses of activation
depicted below. Differences in timing were found within 400 ms after word presentation and
differences in activation strength at the time of overt speech production.
Source: Reproduced from Salmelin (2010).
New Winds: How MEG May Further

Contribute to a Better Understanding
of Language Function
New Devices May Increase the Spatial Accuracy of MEG

In the current MEG systems, the sensors are located at least 3 cm away from even the
most superficial neural current sources. This is because the need to keep the supercon-
ducting sensors in a low-temperature bath of liquid helium necessitates a rigid structure
and insulated outmost layer of the MEG helmet. The helmet, with its fixed sensor array,
provides an excellent coverage for much of the cortex, but less optimal coverage espe-
cially for the most anterior and inferior parts of the frontal and temporal cortices. The
considerable gap between sensors and sources is unfortunate, as the magnetic field dies
away rapidly with distance.
In the future, novel room-temperature sensor designs, such as atomic magnetometers
(Dang, Maloof, & Romalis, 2010; Mhaskar, Knappe, & Kitching, 2012), will likely allow
bringing the sensors closer to the head and setting up a more extensive coverage of the
cortex. Thus, they propose to markedly improve the spatial resolution of MEG and allow
better dissociation of the contribution from nearby cortical regions. In fact, it is pos-
sible that devices based on these principles would eventually facilitate whole-head MEG
recordings with comparable sensitivity and specificity (at least in cortical regions) to
those obtained with intracranial EEG recordings. For example, gamma activity, thought
to reflect highly local signaling (Buzsaki & Wang, 2012) and not readily picked up at
the distance available with the current MEG systems, might become better detectable
with the new sensors. Such new MEG devices would allow the tracking of language pro-
cessing with a combined spatial and temporal specificity that is not currently possible,
even with joint use of different neuroimaging modalities.
Task-Related Connectivity Can Be Tracked as

a Function of Time
During the last decade, the vast majority of interaction studies have focused on resting-
state connectivity. Whether such studies have used MEG or fMRI, they generally have
ignored the modulation of connectivity across time. Recently, new interest has arisen in
the nonstationarity of connectivity during rest, as well as during and between different
task conditions. These are questions that can be addressed best with MEG and its supe-
rior temporal resolution compared with, for example, fMRI. So far, the temporal resolu-
tion of MEG has been utilized implicitly, by focusing on oscillatory neurophysiological
mechanisms of interactions (Palva & Palva, 2012).
It is now also possible to explicitly evaluate the specific times at which connectivity
modulations manifest or experimental conditions differ from each other. For example,
it has been shown that network-level modulations occur at different times in semantic
and phonological priming of written words (Kujala et al., 2012). Moreover, it has been
demonstrated in picture naming that the additional cortical connectivity in overt versus
silent naming is not stationary but changes markedly from the early to late time window
preceding speech (cf. Figure 6.8) (Liljeström et al., 2015). MEG thus allows researchers
to follow the dynamic reconfiguration of the interacting cortical networks at the scale of
hundreds of milliseconds as the tasks unfold. Such a capacity offers the possibility not
only to determine the relevant network structures but also to dissociate between early
and late network interactions and thus, presumably, lower-and higher-level distributed
processing in language perception and production.
Multiple Distinct Imaging Views and Genetic

and Behavioral Measures Will Be Combined for a More
Complete Picture of Human Language Function
Currently, most MEG studies focus on one measure of choice. However, as evoked
responses, modulation of cortical rhythms, and connectivity measures each provide
their own angles to the study of brain function, it would seem worthwhile to exploit such
complementary information in the future. Some recent studies have sought to do just
that. A study of semantic and phonological priming of written words found that while
activation was reduced in the same area and time window (superior temporal cortex at
250–650 ms) for both types of priming, the spatiospectral connectivity patterns differed
between the two processes (Kujala et al., 2012). Other studies have utilized both evoked
responses and modulation of rhythmic activity to characterize language comprehension
(Wang et al., 2012) and cortical effects of the age of acquisition on object recognition
(Urooj et al., 2014).
Even more angles to the cortical correlates of language function can be obtained
by combining different imaging modalities. Electrophysiological imaging with MEG
and hemodynamic imaging with fMRI form an interesting combination, as func-
tional localization can be done readily and independently with each method, and
the results may then be considered together. Such multimodal imaging has been ap-
plied to speech processing (Renvall, Formisano, et al., 2012), reading (Lau, Gramfort,
Hämäläinen, & Kuperberg, 2013; McDonald et al., 2010; Vartiainen, Liljeström,
Koskinen, Renvall, & Salmelin, 2011) and picture naming (Liljeström et al., 2009).
While the methods have converged on a number of functional interpretations, they
have also indicated some salient differences, particularly in word reading (Vartiainen
et al., 2011): Semantic processing was associated with the left inferior frontal cortex
in fMRI, as usual, but with the left superior temporal cortex in MEG, again as usual.
Perhaps even more intriguingly, the MEG-fMRI data, combined with a host of earlier,
separate MEG and fMRI literature, indicated that the MEG evoked response to letter-
strings in the left occipitotemporal cortex reflects largely a feed-forward process,
as it is not influenced by task. The corresponding BOLD fMRI signal, however, was
strongly affected by task, suggesting considerable feedback influence. The possibility
that MEG and fMRI might be differentially sensitive to feed-forward versus feedback
processes should be worth exploring, as it represents an essential step toward an in-
formed use of multimodal imaging that would reach beyond the mere combination of
location and timing of neural activation.
In striving for a more complete picture of language processing, behavioral and other
non-brain measures, such as eye tracking, should obviously be incorporated into the
analysis as much as possible, while retaining the MEG signals as noise-free as possible.
Research into the influence of genes on the MEG responses, let alone on the complex
linguistic activations, is only starting but will likely become an important component
of understanding human language processing, its cortical correlates, and their indi-
vidual variation. There are two main approaches: choose a gene and test the effect of
its haplotypes on cortical measures (Lamminmäki, Massinen, Nopola-Hemmi, Kere,
& Hari, 2012), or find a cortical signal with suitable properties (robust within an indi-
vidual, comparable between siblings, and highly variable across non-siblings) and per-
form a genome-wide search for genes that influence that response (Renvall, Salmela,
et al., 2012).
Multivariate Analysis and Predictive Computational

Models Promise a New Angle to Assessing Cortical
Implementation of Language
To date, most MEG studies have focused on univariate analysis and have compared
different stimulus categories (e.g., words vs. nonwords, verbs vs. nouns, speech vs.
nonspeech sounds, related vs. unrelated items), tasks (e.g., overt vs. covert naming,
semantic vs. perceptual decisions), or participant groups (e.g., adults vs. children,
reading-impaired vs. control participants). Over the recent years, multivariate analyses
and computational techniques derived from machine learning have also found their
way into MEG studies of language (Chan, Halgren, Marinkovic, & Cash, 2011; Clarke,
Devereux, Randall, & Tyler, 2015; Koskinen et al., 2013; Miozzo, Pulvermuller, & Hauk,
2015; Simanova, van Gerven, Oostenveld, & Hagoort, 2015; Sudre et al., 2012).
The cortical representation of individual word meanings and the time course of word
processing from primarily perceptual to essentially semantic features have been of par-
ticular interest (Chan et al., 2011; Clarke et al., 2015; Sudre et al., 2012); word categories
have been classified even in a free word-generation task (Simanova et al., 2015).
Generative computational models have further allowed researchers not merely to dis-
tinguish between the individual item maps, but also to estimate the cortical dynamics of
the possible underlying features, ranging from low-level visual properties such as aspect
ratio (visual cortex, 0–200 ms) to high-level cognitive properties such as whether one
can hold the item (superior parietal cortex, 200–400 ms) or whether the item is bigger
than a car (sensorimotor cortex, 400–600 ms) (Sudre et al., 2012). While the new tools
make it possible to work on the item level instead of stimulus categories, multiple trials
per item (or multiple items per category when classifying by category) are still needed to
ensure a reasonable SNR. However, whether the reasonable minimum number will turn
out to be 5, 10, or 20, it seems that a markedly lower number of trials per item may suffice
than in the more traditional univariate analysis.
In conclusion, MEG combines millisecond-level timing with reasonable spatial ac-
curacy and is, therefore, an excellent method for tracking the cortical dynamics of lan-
guage processing. Language, as a higher cognitive function, seems to rely largely on
cortical activation, and that is what MEG detects best. After 20 years of research with
helmet-like sensor systems that, by covering the whole head at once, truly facilitated
MEG study of higher cognitive functions, we have a good basic understanding of the
cortical dynamics of speech perception, reading, and picture naming (speech produc-
tion) as measured by evoked responses. We can now build on this solid basis when
learning to make the best, most informative use of the various neuroimaging measures
in the study of cognition and aiming to deepen our understanding of the organization of
language and knowledge in the human brain.
Acknowledgments
This work was supported by the Academy of Finland (LASTU Programme 2012–2016, per-
sonal grants to JK and RS), the Sigrid Jusélius Foundation, the Finnish Cultural Foundation,
and the Brain Research at Aalto University and University of Helsinki (BRAHE) consortium.
References
Arnal, L. H., Wyart, V., & Giraud, A. L. (2011). Transitions in neural oscillations reflect predic-
tion errors generated in audiovisual speech. Nature Neuroscience, 14, 797–801.
Baillet, S., Mosher, C., & Leahy, M. (2001. Electromagnetic brain mapping. IEEE Signal
Processing Magazine, 18,  14–30.
Barnett, L., & Seth, A. K. (2014). The MVGC multivariate Granger causality toolbox: A new ap-
proach to Granger-causal inference. Journal of Neuroscience Methods, 223,  50–68.
Bastiaansen, M., Magyari, L., & Hagoort, P. (2010). Syntactic unification operations are re-
flected in oscillatory dynamics during on-line sentence comprehension. Journal of Cognitive
Bastos, A. M., Vezoli, J., Bosman, C. A., Schoffelen, J. M., Oostenveld, R., Dowdall, J. R., De
Weerd, P., Kennedy, H., & Fries, P. (2015). Visual areas exert feedforward and feedback
influences through distinct frequency channels. Neuron, 85, 390–401.
Betti, V., Della Penna, S., de Pasquale, F., Mantini, D., Marzetti, L., Romani, G. L., & Corbetta,
M. (2013). Natural scenes viewing alters the dynamics of functional connectivity in the
human brain. Neuron, 79, 782–797.
Bourguignon, M., De Tiege, X., de Beeck, M. O., Ligot, N., Paquier, P., Van Bogaert, P.,
Goldman, S., Hari, R., & Jousmäki, V. (2013). The pace of prosodic phrasing couples the
listener’s cortex to the reader’s voice. Human Brain Mapping, 34, 314–326.
Butz, M., Timmermann, L., Gross, J., Pollok, B., Dirks, M., Hefter, H., & Schnitzler, A. (2006).
Oscillatory coupling in writing and writer’s cramp. Journal of Physiology, Paris, 99,  14–20.
Buzsaki, G., & Wang, X. J. (2012). Mechanisms of gamma oscillations. Annual Review of
Chan, A. M., Halgren, E., Marinkovic, K., & Cash, S.S. (2011). Decoding word and category-
specific spatiotemporal representations from MEG and EEG. NeuroImage, 54, 3028–3039.
Clarke, A., Devereux, B. J., Randall, B., & Tyler, L. K. (2015). Predicting the time course of indi-
vidual objects with MEG. Cerebral Cortex, 25, 3602–3612.
Cohen, D. (1968). Magnetoencephalography: Evidence of magnetic fields produced by alpha-
rhythm currents. Science, 161, 784–786.
Coltheart, M., Curtis, B., Atkins, P., & Haller, M. (1993). Models of reading aloud: Dual-route
and parallel-distributed-processing approaches. Psychological Review, 100, 589–608.
Cornelissen, K., Laine, M., Tarkiainen, A., Jarvensivu, T., Martin, N., & Salmelin, R. (2003).
Adult brain plasticity elicited by anomia treatment. Journal of Cognitive Neuroscience, 15,
444–461.
D’Arcy, R. C., Bardouille, T., Newman, A. J., McWhinney, S. R., Debay, D., Sadler, R. M., Clarke,
D. B., & Esser, M. J. (2013). Spatial MEG laterality maps for language: Clinical applications in
epilepsy. Human Brain Mapping, 34, 1749–1760.
Dang, H. B., Maloof, A. C., & Romalis, M. V. (2010). Ultrahigh sensitivity magnetic field and
magnetization measurements with an atomic magnetometer. Applied Physics Letters, 97,
151110.
Daunizeau, J., Kiebel, S. J., & Friston, K. J. (2009). Dynamic causal modelling of distributed
electromagnetic responses. NeuroImage, 47, 590–601.
Dobel, C., Junghofer, M., Breitenstein, C., Klauke, B., Knecht, S., Pantev, C., & Zwitserlood, P.
(2010). New names for known things: On the association of novel word forms with existing
semantic information. Journal of Cognitive Neuroscience, 22, 1251–1261.
Donner, T. H., & Siegel, M. (2011). A framework for local cortical oscillation patterns. Trends in
Cognitive Sciences, 15, 191–199.
Eulitz, C., Diesch, E., Pantev, C., Hampson, S., & Elbert, T. (1995). Magnetic and electric brain
activity evoked by the processing of tone and vowel stimuli. Journal of Neuroscience, 15,
2748–2755.
Findlay, A. M., Ambrose, J. B., Cahn-Weiner, D. A., Houde, J. F., Honma, S., Hinkley, L. B.,
Berger, M. S., Nagarajan, S. S., & Kirsch, H. E. (2012). Dynamics of hemispheric domi-
nance for language assessed by magnetoencephalographic imaging. Annals of Neurology, 71,
668–686.
Fonteneau, E., Bozic, M., & Marslen-Wilson, W. D. (2015). Brain network connectivity during
language comprehension: Interacting linguistic and perceptual subsystems. Cerebral Cortex,
25, 3962–3976.
Forss, N., Jousmäki, V., & Hari, R. (1995). Interaction between afferent input from fingers in
human somatosensory cortex. Brain Research, 685,  68–76.
Friederici, A. D., Wang, Y., Herrmann, C. S., Maess, B., & Oertel, U. (2000). Localization of
early syntactic processes in frontal and temporal cortical areas: A magnetoencephalographic
study. Human Brain Mapping, 11,  1–11.
Fries, P. (2005). A mechanism for cognitive dynamics: Neuronal communication through neu-

ronal coherence. Trends in Cognitive Sciences, 9, 474–480.
Geweke, J. (1982). Measurement of linear dependence and feedback between multiple time se-
ries. Journal of the American Statistical Association, 77, 304–313.
Gramfort, A., Luessi, M., Larson, E., Engemann, D. A., Strohmeier, D., Brodbeck, C.,
Parkkonen, L., & Hämäläinen, M. S. (2014). MNE software for processing MEG and EEG
data. NeuroImage, 86, 446–460.
Gross, J., Baillet, S., Barnes, G. R., Henson, R. N., Hillebrand, A., Jensen, O., Jerbi, K., Litvak,
V., Maess, B., Oostenveld, R., Parkkonen, L., Taylor, J. R., van Wassenhove, V., Wibral, M.,
& Schoffelen, J. M. (2013). Good practice for conducting and reporting MEG research.
Gross, J., Hoogenboom, N., Thut, G., Schyns, P., Panzeri, S., Belin, P., & Garrod, S. (2013).
Speech rhythms and multiplexed oscillatory sensory coding in the human brain. PLoS
Biology, 11, e1001752.
Gross, J., Kujala, J., Hämäläinen, M., Timmermann, L., Schnitzler, A., & Salmelin, R. (2001).
Dynamic imaging of coherent sources: Studying neural interactions in the human brain.
Proceedings of the National Academy of Sciences USA, 98, 694–699.
Gross, J., Timmermann, L., Kujala, J., Dirks, M., Schmitz, F., Salmelin, R., & Schnitzler, A.
(2002). The neural basis of intermittent motor control in humans. Proceedings of the National
Academy of Sciences USA, 99, 2299–2302.
Halgren, E., Dhond, R. P., Christensen, N., Van Petten, C., Marinkovic, K., Lewine, J. D., &
Dale, A. M. (2002). N400-like magnetoencephalography responses modulated by semantic
context, word frequency, and lexical class in sentences. NeuroImage, 17, 1101–1116.
Hämäläinen, M., Hari, R., Ilmoniemi, R., Knuutila, J., & Lounasmaa, O. V. (1993).
Magnetoencephalography: Theory, instrumentation, and applications to noninvasive
studies of the working human brain. Reviews of Modern Physics, 65, 413–497.
Hansen, P. C., Kringelbach, M. L., & Salmelin, R. (2010). MEG: An introduction to methods.
New York: Oxford University Press.
Hari, R., & Salmelin, R. (1997). Human cortical oscillations: A neuromagnetic view through
the skull. Trends in Neuroscience, 20,  44–49.
He, B. J., Zempel, J. M., Snyder, A. Z., & Raichle, M. E. (2010). The temporal structures and
functional significance of scale-free brain activity. Neuron, 66, 353–369.
Helenius, P., Salmelin, R., Service, E., & Connolly, J. F. (1998). Distinct time courses of word and
context comprehension in the left temporal cortex. Brain, 121, 1133–1142.
Helenius, P., Salmelin, R., Service, E., & Connolly, J. F. (1999). Semantic cortical activation in
dyslexic readers. Journal of Cognitive Neuroscience, 11, 535–550.
Helenius, P., Salmelin, R., Service, E., Connolly, J. F., Leinonen, S., & Lyytinen, H. (2002).
Cortical activation during spoken-word segmentation in nonreading-impaired and dyslexic
adults. Journal of Neuroscience, 22, 2936–2944.
Helenius, P., Tarkiainen, A., Cornelissen, P., Hansen, P. C., & Salmelin, R. (1999). Dissociation
of normal feature analysis and deficient processing of letter-strings in dyslexic adults.
Holz, E. M., Glennon, M., Prendergast, K., & Sauseng, P. (2010). Theta-gamma phase synchro-
nization during memory matching in visual working memory. NeuroImage, 52, 326–335.
Hoogenboom, N., Schoffelen, J. M., Oostenveld, R., Parkes, L. M., & Fries, P. (2006). Localizing
human visual gamma-band activity in frequency, time and space. NeuroImage, 29, 764–773.
Hultén, A., Karvonen, L., Laine, M., & Salmelin, R. (2014). Producing speech with a newly
learned morphosyntax and vocabulary: An magnetoencephalography study. Journal of
Cognitive Neuroscience, 26, 1721–1735.
Hultén, A., Vihla, M., Laine, M., & Salmelin, R. (2009). Accessing newly learned names and
meanings in the native language. Human Brain Mapping, 30, 976–989.
Huotilainen, M., Shestakova, A., & Hukki, J. (2008). Using magnetoencephalography in
assessing auditory skills in infants and children. International Journal of Psychophysiology,
68, 123–129.
Jerbi, K., Lachaux, J. P., N’Diaye, K., Pantazis, D., Leahy, R. M., Garnero, L., & Baillet, S. (2007).
Coherent neural representation of hand speed in humans revealed by MEG imaging.
Kamada, K., Sawamura, Y., Takeuchi, F., Kuriki, S., Kawai, K., Morita, A., & Todo, T. (2007).
Expressive and receptive language areas determined by a non-invasive reliable method
using functional magnetic resonance imaging and magnetoencephalography. Neurosurgery,
60, 296–305; discussion 305–296.
Kirveskari, E., Salmelin, R., & Hari, R. (2006). Neuromagnetic responses to vowels vs. tones re-
veal hemispheric lateralization. Clinical Neurophysiology, 117, 643–648.
Koskinen, M., Viinikanoja, J., Kurimo, M., Klami, A., Kaski, S., & Hari, R. (2013). Identifying
fragments of natural speech from the listener’s MEG signals. Human Brain Mapping, 34,
1477–1489.
Kuhl, P. K., Ramirez, R. R., Bosseler, A., Lin, J. F., & Imada, T. (2014). Infants’ brain responses to
speech suggest analysis by synthesis. Proceedings of the National Academy of Sciences USA,
111, 11238–11245.
Kujala, J., Gross, J., & Salmelin, R. (2008). Localization of correlated network activity at the
cortical level with MEG. NeuroImage, 39, 1706–1720.
Kujala, J., Pammer, K., Cornelissen, P., Roebroeck, A., Formisano, E., & Salmelin, R. (2007).
Phase coupling in a cerebro-cerebellar network at 8–13 Hz during reading. Cerebral Cortex,
17, 1476–1485.
Kujala, J., Sudre, G., Vartiainen, J., Liljeström, M., Mitchell, T., & Salmelin, R. (2014).
Multivariate analysis of correlation between electrophysiological and hemodynamic
responses during cognitive processing. NeuroImage, 92, 207–216.
Kujala, J., Vartiainen, J., Laaksonen, H., & Salmelin, R. (2012). Neural interactions at the core of
phonological and semantic priming of written words. Cerebral Cortex, 22, 2305–2312.
Laaksonen, H., Kujala, J., Hultén, A., Liljeström, M., & Salmelin, R. (2012). MEG evoked
responses and rhythmic activity provide spatiotemporally complementary measures of
neural activity in language production. NeuroImage, 60,  29–36.
Laaksonen, H., Kujala, J., & Salmelin, R. (2008). A method for spatiotemporal mapping of
event-related modulation of cortical rhythmic activity. NeuroImage, 42, 207–217.
Lachaux, J. P., Axmacher, N., Mormann, F., Halgren, E., & Crone, N. E. (2012). High-frequency
neural activity and human cognition: Past, present and possible future of intracranial EEG
research. Progress in Neurobiology, 98, 279–301.
Laine, M., Salmelin, R., Helenius, P., & Marttila, R. (2000). Brain activation during reading in
deep dyslexia: An MEG study. Journal of Cognitive Neuroscience, 12, 622–634.
Lamminmäki, S., Massinen, S., Nopola-Hemmi, J., Kere, J., & Hari, R. (2012). Human
ROBO1 regulates interaural interaction in auditory pathways. Journal of Neuroscience,
32, 966–971.
Lau, E. F., Gramfort, A., Hämäläinen, M. S., & Kuperberg, G. R. (2013). Automatic semantic fa-
cilitation in anterior temporal cortex revealed through multimodal neuroimaging. Journal
of Neuroscience, 33, 17174–17181.
Lee, A. K., Hämäläinen, M. S., Dyckman, K. A., Barton, J. J., & Manoach, D. S. (2011). Saccadic
preparation in the frontal eye field is modulated by distinct trial history effects as revealed by
magnetoencephalography. Cerebral Cortex, 21, 245–253.
Lehongre, K., Ramus, F., Villiermet, N., Schwartz, D., & Giraud, A. L. (2011). Altered low-
gamma sampling in auditory cortex accounts for the three main facets of dyslexia. Neuron,
72, 1080–1090.
Levelt, W. J., Praamstra, P., Meyer, A. S., Helenius, P., & Salmelin, R. (1998). An MEG study of
picture naming. Journal of Cognitive Neuroscience, 10, 553–567.
Liegeois-Chauvel, C., Musolino, A., Badier, J. M., Marquis, P., & Chauvel, P. (1994). Evoked
potentials recorded from the auditory cortex in man: Evaluation and topography of the
middle latency components. Electroencephalography and Clinical Neurophysiology, 92,
204–214.
Liljeström, M., Hultén, A., Parkkonen, L., & Salmelin, R. (2009). Comparing MEG and fMRI
views to naming actions and objects. Human Brain Mapping, 30, 1845–1856.
Liljeström, M., Kujala, J., Jensen, O., & Salmelin, R. (2005). Neuromagnetic localization of
rhythmic activity in the human brain: A comparison of three methods. NeuroImage, 25,
734–745.
Liljeström, M., Kujala, J., Stevenson, C., & Salmelin, R. (2015). Dynamic reconfiguration of the
language network preceding onset of speech in picture naming. Human Brain Mapping, 36,
1202–1216.
Longcamp, M., Tanskanen, T., & Hari, R. (2006). The imprint of action: Motor cortex involve-
ment in visual perception of handwritten letters. NeuroImage, 33, 681–688.
Lopes da Silva, F. H. (2010). Electrophysiological basis of MEG signals. In P. C. Hansen,
M. L. Kringelbach, & R. Salmelin (Eds.), MEG: An introduction to methods (pp. 1–23).
New York: Oxford University Press.
Luo, H., & Poeppel, D. (2007). Phase patterns of neuronal responses reliably discriminate
speech in human auditory cortex. Neuron, 54, 1001–1010.
Maris, E., & Oostenveld, R. (2007). Nonparametric statistical testing of EEG-and MEG-data.
Journal of Neuroscience Methods, 164, 177–190.
Marzetti, L., Della Penna, S., Snyder, A. Z., Pizzella, V., Nolte, G., de Pasquale, F., Romani, G. L.,
& Corbetta, M. (2013). Frequency specific interactions of MEG resting state activity within
and across brain networks as revealed by the multivariate interaction measure. NeuroImage,
79, 172–183.
McDonald, C. R., Thesen, T., Carlson, C., Blumberg, M., Girard, H. M., Trongnetrpunya, A.,
Sherfey, J. S., Devinsky, O., Kuzniecky, R., Dolye, W. K., Cash, S. S., Leonard, M. K., Hagler,
D. J., Jr., Dale, A. M., & Halgren, E. (2010). Multimodal imaging of repetition priming: Using
fMRI, MEG, and intracranial EEG to reveal spatiotemporal profiles of word processing.
NeuroImage, 53, 707–7 17.
Meltzer, J. A., Wagage, S., Ryder, J., Solomon, B., & Braun, A. R. (2013). Adaptive significance
of right hemisphere activation in aphasic language comprehension. Neuropsychologia, 51,
1248–1259.
Mhaskar, R., Knappe, S., & Kitching, J. (2012). A low-power, high-sensitivity micromachined
optical magnetometer. Applied Physics Letters, 101, 241105
Michalareas, G., Schoffelen, J. M., Paterson, G., & Gross, J. (2013). Investigating causality be-
tween interacting brain areas with multivariate autoregressive models of MEG sensor data.
Miozzo, M., Pulvermuller, F., & Hauk, O. (2015). Early parallel activation of semantics and pho-
nology in picture naming: Evidence from a multiple linear regression MEG study. Cerebral
Cortex, 25, 3343–3355.
Möttönen, R., van de Ven, G. M., & Watkins, K. E. (2014). Attention fine-tunes auditory-motor
processing of speech sounds. Journal of Neuroscience, 34, 4064–4069.
Näätänen, R., Lehtokoski, A., Lennes, M., Cheour, M., Huotilainen, M., Iivonen, A., Vainio,
M., Alku, P., Ilmoniemi, R. J., Luuk, A., Allik, J., Sinkkonen, J., & Alho, K. (1997). Language-
specific phoneme representations revealed by electric and magnetic brain responses. Nature,
385, 432–434.
Nolte, G., Bai, O., Wheaton, L., Mari, Z., Vorbach, S., & Hallett, M. (2004). Identifying
true brain interaction from EEG data using the imaginary part of coherency. Clinical
Neurophysiology, 115, 2292–2307.
Nora, A., Hultén, A., Karvonen, L., Kim, J. Y., Lehtonen, M., Yli-Kaitala, H., Service, E., &
Salmelin, R. (2012). Long-term phonological learning begins at the level of word form.
Nora, A., Renvall, H., Kim, J. Y., Service, E., & Salmelin, R. (2015). Distinct effects of memory
retrieval and articulatory preparation when learning and accessing new word forms. PLoS
ONE, 10, e0126652.
Numminen, J., Salmelin, R., & Hari, R. (1999). Subject’s own speech reduces reactivity of the
human auditory cortex. Neuroscience Letters, 265, 119–122.
Okada, Y. C., Wu, J., & Kyuhou, S. (1997). Genesis of MEG signals in a mammalian CNS struc-
ture. Electroencephalograpy and Clinical Neurophysiology, 103, 474–485.
Palva, J. M., Monto, S., Kulashekhar, S., & Palva, S. (2010). Neuronal synchrony reveals working
memory networks and predicts individual memory capacity. Proceedings of the National
Palva, S., & Palva, J.M. (2012). Discovering oscillatory interaction networks with M/
EEG: Challenges and breakthroughs. Trends in Cognitive Sciences, 16, 219–230.
Papanicolaou, A. C., Simos, P. G., Castillo, E. M., Breier, J. I., Sarkari, S., Pataraia, E.,
Billingsley, R. L., Buchanan, S., Wheless, J., Maggio, V., & Maggio, W. W. (2004).
Magnetocephalography: A noninvasive alternative to the Wada procedure. Journal of
Neurosurgery, 100, 867–876.
Parviainen, T., Helenius, P., Poskiparta, E., Niemi, P., & Salmelin, R. (2006). Cortical sequence
of word perception in beginning readers. Journal of Neuroscience, 26, 6052–6061.
Parviainen, T., Helenius, P., Poskiparta, E., Niemi, P., & Salmelin, R. (2011). Speech perception
in the child brain: Cortical timing and its relevance to literacy acquisition. Human Brain
Mapping, 32, 2193–2206.
Parviainen, T., Helenius, P., & Salmelin, R. (2005). Cortical differentiation of speech and
nonspeech sounds at 100 ms: Implications for dyslexia. Cerebral Cortex, 15, 1054–1063.
Peelle, J. E., Gross, J., & Davis, M. H. (2013). Phase-locked responses to speech in human audi-
tory cortex are enhanced during comprehension. Cerebral Cortex, 23, 1378–1387.
Pylkkänen, L., Bemis, D. K., & Blanco Elorrieta, E. (2014). Building phrases in language pro-
duction: An MEG study of simple composition. Cognition, 133, 371–384.
Pylkkänen, L., Feintuch, S., Hopkins, E., & Marantz, A. (2004). Neural correlates of the effects
of morphological family frequency and family size: An MEG study. Cognition, 91, B35–B45.
Pylkkänen, L., & Marantz, A. (2003). Tracking the time course of word recognition with MEG.
Trends in Cognitive Sciences, 7, 187–189.
Renvall, H., Formisano, E., Parviainen, T., Bonte, M., Vihla, M., & Salmelin, R. (2012).
Parametric merging of MEG and fMRI reveals spatiotemporal differences in cortical pro-
cessing of spoken words and environmental sounds in background noise. Cerebral Cortex,
22, 132–143.
Renvall, H., Salmela, E., Vihla, M., Illman, M., Leinonen, E., Kere, J., & Salmelin, R. (2012).
Genome-wide linkage analysis of human auditory cortical activation suggests distinct loci
on chromosomes 2, 3, and 8. Journal of Neuroscience, 32, 14511–14518.
Robinson, S. E., & Vrba, J. (1997). Functional neuroimaging by synthetic aperture
magnetometry (SAM). In T. Yoshimoto, M. Kotani, S. Kuriki, H. Karibe, & B. Nakasato
(Eds.), Recent advances in biomagnetism (pp. 302–305). Sendai, Japan: Tohoku University
Press.
Ruspantini, I., Saarinen, T., Belardinelli, P., Jalava, A., Parviainen, T., Kujala, J., & Salmelin, R.
(2012). Corticomuscular coherence is tuned to the spontaneous rhythmicity of speech at
2–3 Hz. Journal of Neuroscience, 32, 3786–3790.
Saarinen, T., Laaksonen, H., Parviainen, T., & Salmelin, R. (2006). Motor cortex dynamics in
visuomotor production of speech and non-speech mouth movements. Cerebral Cortex, 16,
212–222.
Salmelin, R. (2007). Clinical neurophysiology of language: The MEG approach. Clinical
Salmelin, R. (2010). Multi-dipole modeling in MEG. In P. C. Hansen, M. L. Kringelbach, &
R. Salmelin (Eds.), MEG: An introduction to methods (pp. 124–155). New York: Oxford
University Press.
Salmelin, R., Hari, R., Lounasmaa, O. V., & Sams, M. (1994). Dynamics of brain activation
during picture naming. Nature, 368, 463–465.
Salmelin, R., Helenius, P., & Service, E. (2000). Neurophysiology of fluent and impaired
reading: A magnetoencephalographic approach. Journal of Clinical Neurophysiology, 17,
163–174.
Salmelin, R., & Sams, M. (2002). Motor cortex involvement during verbal versus non-verbal lip
and tongue movements. Human Brain Mapping, 16,  81–91.
Salmelin, R., Schnitzler, A., Schmitz, F., & Freund, H. J. (2000). Single word reading in develop-
mental stutterers and fluent speakers. Brain, 123, 1184–1202.
Scherg, M. (1990). Fundamentals of dipole source potential analysis. In F. Grandori, M. Hoke,
& G. L. Romani (Eds.), Auditory evoked magnetic fields and potentials (pp. 40–69). Basel:
Karger.
Schoffelen, J. M., & Gross, J. (2009). Source connectivity analysis with MEG and EEG. Human
Schoffelen, J. M., & Gross, J. (2010). Improving the interpretability of all-to-all pairwise source
connectivity analysis in MEG with nonhomogeneous smoothing. Human Brain Mapping,
32, 426–437.
Service, E., Helenius, P., Maury, S., & Salmelin, R. (2007). Localization of syntactic and se-
mantic brain responses using magnetoencephalography. Journal of Cognitive Neuroscience,
19, 1193–1205.
Shtyrov, Y., Butorina, A., Nikolaeva, A., & Stroganova, T. (2014). Automatic ultrarapid activa-
tion and inhibition of cortical motor systems in spoken word comprehension. Proceedings of
the National Academy of Sciences USA, 111, E1918–1923.
Simanova, I., van Gerven, M. A., Oostenveld, R., & Hagoort, P. (2015). Predicting the semantic
category of internally generated words from neuromagnetic recordings. Journal of Cognitive
Neuroscience, 27,  35–45.
Simões, C., Jensen, O., Parkkonen, L., & Hari, R. (2003). Phase locking between human pri-
mary and secondary somatosensory cortices. Proceedings of the National Academy of
Sciences USA, 100, 2691–2694.
Sörös, P., Cornelissen, K., Laine, M., & Salmelin, R. (2003). Naming actions and objects:
Cortical dynamics in healthy adults and in an anomic patient with a dissociation in action/
object naming. NeuroImage, 19, 1787–1801.
Stam, C. J., Nolte, G., & Daffertshofer, A. (2007). Phase lag index: Assessment of functional
connectivity from multi channel EEG and MEG with diminished bias from common
sources. Human Brain Mapping, 28, 1178–1193.
Sudre, G., Pomerleau, D., Palatucci, M., Wehbe, L., Fyshe, A., Salmelin, R., & Mitchell, T. (2012).
Tracking neural coding of perceptual and semantic features of concrete nouns. NeuroImage,
62, 451–463.
Tanaka, N., Liu, H., Reinsberger, C., Madsen, J. R., Bourgeois, B. F., Dworetzky, B. A.,
Hamalainen, M. S., & Stufflebeam, S. M. (2013). Language lateralization represented by spa-
tiotemporal mapping of magnetoencephalography. American Journal of Neuroradiology, 34,
558–563.
Taulu, S., & Simola, J. (2006). Spatiotemporal signal space separation method for rejecting
nearby interference in MEG measurements. Physics in Medicine and Biology, 51,
1759–1768.
Tiitinen, H., Mäkelä, A. M., Mäkinen, V., May, P. J., & Alku, P. (2005). Disentangling the effects
of phonation and articulation: Hemispheric asymmetries in the auditory N1m response of
the human brain. BMC Neuroscience, 6, 62.
Urooj, U., Cornelissen, P. L., Simpson, M. I., Wheat, K. L., Woods, W., Barca, L., & Ellis, A. W.
(2014). Interactions between visual and semantic processing during object recognition re-
vealed by modulatory effects of age of acquisition. NeuroImage, 87, 252–264.
van Ackeren, M. J., Schneider, T. R., Musch, K., & Rueschemeyer, S. A. (2014). Oscillatory
neuronal activity reflects lexical-semantic feature integration within and across sensory
modalities in distributed cortical networks. Journal of Neuroscience, 34, 14318–14323.
Van Veen, B., & Buckley, K. (1988). Beamforming: A versatile approach to spatial filtering. IEEE
ASSP Magazine, 5,  4–24.
Vartiainen, J., Aggujaro, S., Lehtonen, M., Hulten, A., Laine, M., & Salmelin, R. (2009). Neural
dynamics of reading morphologically complex words. NeuroImage, 47, 2064–2072.
Vartiainen, J., Liljeström, M., Koskinen, M., Renvall, H., & Salmelin, R. (2011). Functional magnetic
resonance imaging blood oxygenation level-dependent signal and magnetoencephalography
evoked responses yield different neural functionality in reading. Journal of Neuroscience, 31,
1048–1058.
Vartiainen, J., Parviainen, T., & Salmelin, R. (2009). Spatiotemporal convergence of semantic
processing in reading and speech perception. Journal of Neuroscience, 29, 9271–9280.
Vigneau, M., Beaucousin, V., Herve, P. Y., Duffau, H., Crivello, F., Houde, O., Mazoyer, B., &
Tzourio-Mazoyer, N. (2006). Meta-analyzing left hemisphere language areas: Phonology,
semantics, and sentence processing. NeuroImage, 30, 1414–1432.
Vihla, M., Laine, M., & Salmelin, R. (2006). Cortical dynamics of visual/semantic vs. phono-
logical analysis in picture confrontation. NeuroImage, 33, 732–738.
Vihla, M., Lounasmaa, O. V., & Salmelin, R. (2000). Cortical processing of change detec-
tion: Dissociation between natural vowels and two-frequency complex tones. Proceedings of
the National Academy of Sciences USA, 97, 10590–10594.
Wang, L., Jensen, O., van den Brink, D., Weder, N., Schoffelen, J. M., Magyari, L., Hagoort, P., &
Bastiaansen, M. (2012). Beta oscillations relate to the N400m during language comprehen-
sion. Human Brain Mapping, 33, 2898–2912.
Wheat, K. L., Cornelissen, P. L., Frost, S. J., & Hansen, P. C. (2010). During visual word recog-
nition, phonology is accessed within 100 ms and may be mediated by a speech production
code: Evidence from magnetoencephalography. Journal of Neuroscience, 30, 5229–5233.
Woodhead, Z. V., Barnes, G. R., Penny, W., Moran, R., Teki, S., Price, C. J., & Leff, A. P. (2014).
Reading front to back: MEG evidence for early feedback effects during word recognition.
Wydell, T. N., Vuorinen, T., Helenius, P., & Salmelin, R. (2003). Neural correlates of letter-
string length and lexicality during reading in a regular orthography. Journal of Cognitive
Zhang, Y., Kuhl, P. K., Imada, T., Iverson, P., Pruitt, J., Stevens, E. B., Kawakatsu, M.,
Tohkura, Y., & Nemoto, I. (2009). Neural signatures of phonetic learning in adulthood: A
magnetoencephalography study. NeuroImage, 46, 226–240.
Chapter 7
Shedding L i g h t on
L anguage Func t i on a nd
Its Devel op me nt w i t h
Op tical Bra i n I mag i ng
Yasuyo Minagawa and Alejandrina Cristia
Introduction
Near-infrared spectroscopy (NIRS) measures local changes in the concentrations

of oxygenated (oxy-) and deoxygenated (deoxy-) hemoglobin (Hb) associated with re-
gional activation in the cerebral cortex, similar to magnetic resonance imaging (MRI)
(see Heim & Specht, Chapter 4 in this volume). NIRS is portable, innocuous, nonin-
vasive, and relatively resistant to movement artifacts, which makes it ideal for use with
vulnerable and/or mobile populations (e.g., awake infants and children; Cristia et al.,
2013). Although electroencephalography (EEG) (see Leckey & Federmeier, Chapter 3
in this volume) shares several of these advantages, the superior spatial resolution of
NIRS invites examination of localization questions. The introduction of multi-channel
NIRS in the early 1990s led to the development of the technique as a means of meas-
uring human cognitive functions, sometimes referred to as functional NIRS (fNIRS)
(Hoshi & Tamura, 1993; Kato, Kamei, Takashima, & Ozaki, 1993; Villringer, Planck,
Hock, Schleinkofer, & Dirnagl, 1993). Since then, more than 1,200 papers with a use of
NIRS, including language studies, have been published (Boas, Elwell, Ferrari, & Taga,
2014). These language studies include some with clinical populations such as cochlear-
implanted individuals, people diagnosed with schizophrenia, people with a diagnosis of
a developmental disorder (including autism spectrum disorder and stuttering, among
others), and others investigating general language function in normal adults.
This chapter first illustrates the general principles of NIRS and outlines its pros and
cons by comparing it with other brain-imaging methods with a view of the optimal
Optical Brain Imaging 155
investigation of language. We then summarize the general methodological aspects of

NIRS, including experimental paradigms, procedures, and data analyses. These sections
are partly based on previous articles (Minagawa-Kawai, Mori, Hebden, & Dupoux,
2008; Minagawa-Kawai et al., 2009). Functional neuroimaging is an interdisciplinary
area where knowledge of physiology, engineering, psychology, and information pro-
cessing are required. These sections describe these components in a comprehensible
fashion. NIRS has many aspects in common with functional magnetic resonance im-
aging (fMRI). Thus, these method sections would be helpful to students or beginners of
fNIRS as well as fMRI. Finally, we review some of the neurolinguistic discoveries made
so far in this emerging research field.
Fundamentals of the NIRS System:

How NIRS Measures Brain Activity
General Principles
Electrophysiological and hemodynamic measures are the two brain signals that
have mainly been utilized in neuroscience. NIRS measures the latter type of signal as
changes in oxy-and deoxy-Hb provide an indirect indication of cerebral activities.
Brain activation is closely related to regional changes in blood flow and oxygenation
(Fox & Raichle, 1986). Despite the unclear relationship between neuronal activity
and the vascular response (Logothetis, Pauls, Augath, Trinath, & Oeltermann, 2001),
studies on hemodynamic behavior in the brain are based on the assumption that an
increase in blood flow, which in turn causes an increase in the mean local oxygena-
tion, signals an increase in brain activity. Blood flow and blood volume are correlated
and are mostly interchangeable when assessing cerebral activity with some reasonable
assumptions (K. Villringer et al., 1997; Hoshi, Kobayashi, & Tamura, 2001; Strangman,
Culver, Thompson, & Boas, 2002).
NIRS evaluates brain function by identifying changes in the concentrations of
oxy-and deoxy-Hb in the circulating red blood cells by measuring changes in the dif-
fuse transmittance of NIR light at a suitable combination of wavelengths. Their spe-
cific absorption coefficients are shown in Figure 7.1. Notice that at around 800 nm the
coefficients of the two types of Hb are equal. Usually researchers measure a few wave-
length points (e.g., one under 800 nm and the other above 800 nm), so as to accurately
estimate changes in oxy-Hb and deoxy-Hb concentrations separately, and additional
ones for other chromophores such as cytochrome. A chromophore is a functional
group of molecules responsible for color, namely wavelength. Chromophores have
different light absorption characteristics, which can therefore be measured using op-
tical measurements. Further details of the principles of NIRS are described by Ferrari,
Mottola, and Quaresima (2004).
156 Yasuyo Minagawa and Alejandrina Cristia
Absorption Coefficients (mM–1·cm–1)

3 [deoxy-Hb]
2
[oxy-Hb]
0
700 800 900 1,000
Wavelength (nm)
Figure 7.1. NIR absorption spectra of oxygenated hemoglobin [oxy-Hb] and deoxygenated
hemoglobin [deoxy-Hb].
The Continuous-Wave System among Various

NIRS Systems
There is a variety of methods and instruments proposed for noninvasive NIRS,
including time-domain systems, frequency-domain systems, and continuous-wave
(CW) systems. Further, using particular techniques, these systems can enable us to
map the activation in either a two-dimensional or a three-dimensional fashion, as il-
lustrated in Figure 7.2. The most common and simplest method involves measuring
the intensity of diffusely reflected light using continuously emitting light sources.
Instruments capable of performing such measurements are referred to as CW
(continuous-wave) systems, and multichannel CW NIRS is now largely used for func-
tional neuroimaging, namely fNIRS. Many of the cognitive studies, including language
studies, have been carried out with CW fNIRS. Henceforth, when we speak of fNIRS,
we will refer to CW unless otherwise noted. As for other systems, readers can find a
summary of each system in the next section.
In CW systems, at least once source is placed in a given location on the scalp. One or
multiple detectors are then placed at a certain distance, and the intensity of the light as
it has traveled from the source to the detector(s) is measured (see Figure 7.2). Notice
that, in this flight through not only the cerebral cortex (of most interest) but also the
skull and skin and so on, the light is highly scattered and attenuated. With sufficient
separation of source and detector fibers, a significant proportion of the detected signal
penetrates the cortical regions of the brain (Okada & Delpy, 2003). Changes in the inten-
sity at two or more wavelengths can then be converted into changes in concentrations
of oxy-Hb and deoxy-Hb using the modified Beer-Lambert law (Reynolds et al., 1988;
Delpy & Cope, 1997). The modified Beer-Lambert law is expressed as
A = log e ( I0 /I) = ε × c × d × DPF + G (7.1)

where A is the attenuation, I0 and I are the initial and final intensities, ε is the spe-
cific extinction coefficient of the chromophore (measured, for instance, in μmolar−1.
cm−1), c is the concentration of the chromophore (which may be expressed in μmolar
units), d is the distance between the source and detector fibers, DPF is a differential
path length factor, and G is an unknown term due to scattering losses. DPF is the ratio
of the mean path traveled in the tissue by detected photons (elementary particles of
light) and the minimum path d. Although path length does not vary by more than
~10% due to intra-individual variability across different brain areas and about ~20%
due to inter-individual variability (Ferrari, Wei, Carraresi, De Blasi, & Zaccanti, 1992;
Duncan et al. 1996), the exact value of DPF remains unknown because light does not
travel in a straight light. Since neither G nor DPF is known, the absolute change in
the chromophore concentration cannot be derived. However, if we consider changes
in attenuation due to changes in the concentrations of oxy-Hb and deoxy-Hb, we can
eliminate G as follows:
∆A = { ∆c[oxy Hb] × ε[oxy Hb] + ∆c[deoxy Hb] × ε[deoxy Hb]} × d × DPF (7.2)
If measurements of ΔA are determined at two wavelengths (e.g., 780 nm and 830 nm),

simultaneous equations can be obtained and solved for Δc[oxy-Hb] and Δc[deoxy-
Hb], estimating an appropriate value for DPF. These values can then be used to calcu-
late changes in blood volume and oxygenation. Although the absolute Hb concentration
changes cannot be derived with the CW system as explained, some other systems are
capable of this, as described in the next section. Other limitations of CW NIRS are illus-
trated elsewhere (Obrig & Villringer, 2003; Scholkmann et al., 2014).
The technique known as optical topography (Watanabe, Yamashita, Maki, Ito, &
Koizumi, 1996), or multi-channel NIRS, uses arrays of multiple NIR sources (e.g., laser
diodes) and detectors (Figures 7.2, 7.3). This method enables a broad area of the
cortex to be sampled with minimal numbers of optical fibers, and a two-dimensional
Source-Detector
Separation Length
Detector
Near-Infrared
(e.g., 780 & 830 nm)
Scalp
BRAIN
Skull
Figure 7.2. Optical topography using an array of sources and detectors. Near-infrared light
illuminates the brain through the skin and skull, and is diffusely reflected back toward the
detectors.
Source
Detector
Shallow
Deep
Figure 7.3. A NIRS measurements of (left) frontal cortex in a 3-month-old infant using a
Hitachi CW optical topography system; (right) temporal cortex in an infant using the UCL op-
tical topography system.
map of brain activity to be generated. In order to identify the source associated with
a given detected signal, it is necessary to either illuminate each source sequentially
or modulate each source at a unique frequency, following which the signals can be
distinguished using lock-in amplifiers or software to calculate the Fourier trans-
form. Several CW optical topography systems have been developed and employed to
study the human cognitive function (Chance et al., 1998; Hintz et al., 2001; Koizumi
et al., 2003).
Other Systems
Time-Domain Topography
The obstacle that prevents CW NIRS from obtaining absolute chromophore concen
trations is the unknown DPF (DPF in equations (7.1) and (7.2)) due to the fact that
NIR light does not travel through tissue in a straight line. In order to evaluate changes
in chromophore concentration in nonarbitrary units, we should determine the av-
erage path length of the diffusely reflected light (i.e., to determine an average value of
d × DPF in equation (7.2)). A time-domain system can directly record the path length
using a short laser pulse and a fast time-resolved detector. The average flight time
multiplied by the speed of light through the tissue provides the mean path length.
A time-domain system utilizes this principle (Delpy et al., 1988), usually using time-
correlated single-photon counting hardware. These systems necessitate longer acqui-
sition times (as single photons need to be tracked accurately) but have the advantage
of providing better resolution, both in terms of localization and in terms of the sep-
arate estimation of absorption and scatter (Arridge & Lionheart, 1998). Although
time-domain devices are not normally used for fNIRS studies at present, this has
a large potential to be a first choice of fNIRS with further development (Torricelli
et al., 2014).
Frequency-Domain  System
The average path length can also be calculated using an intensity-modulated source and
a device that measures the phase delay of the transmitted signal (Chance, Maris, Sorge,
& Zhang, 1990). This method is employed by NIRS with frequency-domain systems.
Such systems do not rely on photon counting, and therefore provide greater signal-to-
noise ratio (SNR) and dynamic range than time-domain systems. However, they pro-
vide less adequate temporal information unless multiple frequencies are employed
(Nissila et al., 2006).
Apart from identifying hemodynamic changes in the brain, optical techniques can
detect signals considered to be of neuronal origin that vary over much smaller time
intervals. The neuronal signal exhibits a latency of around 10–100 milliseconds (ms),
and changes in neuronal tissue scattering have been proposed as a possible cause of this
signal (Franceschini & Boas, 2004; Gratton & Fabiani, 2001; Gratton, Sarno, Maclin,
Corballis, & Fabiani, 2000). Therefore, this may provide much more direct measures of
cellular activity. This method is sometimes called event-related optical signal (EROS)
or fast NIRS. At present, this technique is also limited to measurements of the cortical
surface area. Although this technique is well established for invasive NIRS with exposed
brains, noninvasive fast NIRS (e.g., Gratton & Fabiani, 2001) has not been sufficiently
studied (Steinbrink et al., 2000; Steinbrink, Kempf, Villringer, & Obrig, 2005) and may
require further examination for effective general use.
Tomography System
If there is a sufficient number of sources and detectors within a probe pad, it is feasible to
use NIRS measurements to generate three-dimensional images of the optical properties
within the brain (for a review, see Gibson, Hebden, & Arridge, 2005). Although this
type of system is still being developed to improve its resolution and accuracy, diffuse op-
tical tomography (DOT) has already shown a functional mapping of visual cortex with
millimeter-order spatial resolution for both activation and functional connectivity during
visuo-motor coordination (Zeff, White, Dehghani, Schlaggar, & Culver, 2007; White et al.,
2009). This is a promising tool to study human language function in the future.
General Characteristics of NIRS

with a View to Neurolinguistic
Research
Hemodynamic Physiology
One of the strong advantages of NIRS is its ability to measure both oxy-Hb and deoxy-
Hb (and consequently also total Hb), whereas blood oxygenation level dependent
(BOLD) fMRI is sensitive only to changes in the ratio of oxy-to deoxy-Hb. Thus, NIRS
provides rich signal information on neural activation. Indeed, studies using NIRS have
shown that an absence of a BOLD signal does not always indicate a lack of brain activa-
tion (Fujiwara et al., 2004; Seiyama et al., 2004). An increase in local arterial blood flow
produces an increase in oxy-Hb and a decrease in deoxy-Hb, whereas an increase in ox-
ygen consumption causes a decrease in oxy-Hb and an increase in deoxy-Hb. Previous
NIRS studies have reported that the overall effect of these opposing mechanisms is
different in infants than in adults, possibly due to the immaturity of vascular regula-
tion. Whereas brain activation in adults usually causes a localized increase in oxy-Hb
and a decrease in deoxy-Hb, known as the hemodynamic response function (HRF)
(A. Villringer et al., 1993; also see Sakatani, Xie, Lichty, Li, & Zuo, 1998), some infant
data are consistent with a net increase in both oxy-Hb and deoxy-Hb (e.g., Meek et al.,
1998; Sakatani, Chen, Lichty, Zuo, & Wang, 1999). Another atypical variation involves
a decrease in oxy-Hb and increase in deoxy-Hb for 4-month-olds (Csibra et al.,
2004) and 1-to 3-month-olds (Kusaka et al., 2004). These atypical HRF patterns have
been chiefly reported for the occipital cortex in response to visual stimuli, and only
rarely for the sensorimotor area and temporal area in young infants. This seems to be
because of region-dependent brain maturation. Myelination overall progresses slowly
in most of the occipital area (Flechsig, 1901), in contrast to the sensorimotor area and
the temporal area. Recent infant NIRS studies are consistent with this neurophysio-
logical development, and neonates typically show normal HRF for temporal areas in
response to speech (e.g., Gervain, Macagno, Cogoi, Peña, & Mehler, 2008; Peña et al.,
2003). In the case of premature infants, our recent data with speech stimuli (Arimitsu
et al., 2018) revealed an inverted HRF in many preterm infants. Further examination
showed that HRF typicality is not significantly correlated with the duration of language
exposure (i.e., age after birth), but with corrected gestational age (i.e., maturational
age), suggesting that more typical forms of brain responses to speech parallel neuro-
physiological development.
Apart from physiological maturation, such HRF variation in response has been
attributed to differences in attention and the level of consciousness (Taga, Asakawa,
Maki, Konishi, & Koizumi, 2003). Inverted HRFs do not always mean “deactivations,”
despite a functionally related decrease being the dominant explanation for an inverted
HRF. In fact, data gathered in our lab from full-term neonates revealed an inverted HRF
response to mother’s speech as opposed to a normal HRF response to non-mother’s
speech (Uchida-Ota et al., 2014). This difference cannot be explained by immature brain
development because these neonates showed normal patterns to a non-mother’s voice
stimuli in the same experiment; additionally, behavioral studies have revealed a special
role of mother’s speech even in fetuses (e.g., Kisilevsky et al., 2003). As the decrease was
observed strongly in the dorsomedial prefrontal area, external attention to the stimuli is
one of the explanations (Gusnard & Raichle, 2001). As similar task-induced decreases
in brain activity have been frequently discussed in fMRI literature (e.g., Binder, 2012),
readers will find more detailed descriptions there. Although both physiological and
functional mechanisms of HRF patterns are yet to be fully understood, NIRS’s sensitivity
to oxy-and deoxy-Hb will further clarify interpretations of hemodynamic signals in

NIRS, positron emission tomography (PET), and fMRI experiments.
Temporal and Spatial Resolutions

The temporal resolution of fast NIRS is 10–100 Hz and that of CW NIRS is around
5–10 Hz (Figure 7.4). However, because of the slow hemodynamic response to neural
activation, temporal activation is effectively around 0.3–0.5 Hz. The temporal resolu-
tion of CW NIRS is comparable or better than fMRI and PET, but inferior to MEG (see
Salmelin, Kujala, & Liljeström, Chapter 6 in this volume) and EEG. Thus, it is difficult
to obtain a detailed time course of each cognitive stage in language processing. In ge-
neral, brain activations during cognitive tasks within a block lasting 10–30 s or an event
lasting 3–5 s are examined (see later discussion in this chapter for more detail). Yet if two
distinctive cognitive processes (e.g., encoding and retrieving) are well separated in an
experimental task, NIRS can capture each response in a form of two peaks within one
curve (Miyata, Watanabe, & Minagawa-Kawai, 2011). Furthermore, peak latency can
serve as a good parameter to examine the effectiveness of language processing. A NIRS
study that investigated 4- month- old infants listening to various auditory stimuli
(Minagawa-Kawai et al., 2008) indicated about a 2-s delay in peak latency for the he-
modynamic response in the auditory area compared to the canonical HRF. Another ex-
ample involves learning of phonemic contrasts that frequently and infrequently occur
within one language (Tsuji et al., 2017). In 5-to 8-month-old Dutch infants, a frequent
contrast evoked earlier response peaks than an infrequent one, suggesting a neural effi-
ciency attuned by the frequent phonemic exposure.
Spatial resolution can be characterized as lateral (parallel to the brain surface) or
depth resolution, both strongly dependent on the arrangement of source and detector
fibers on the scalp. Indeed, one can improve lateral resolution by increasing density
0
Brain
MEG PET
–1 ERP NIRS
Log size (m)
Map
–2 Lesions
fMRI
–3
Column
Layer
–4
–4 –3 –2 –1 0 1 2 3 4 5 6 7 8
Millisecond Second Minute Hour Day
Log Time (sec)
Figure 7.4. Spatial and temporal resolution of a variety of techniques for the study of brain
function.
(bearing in mind limitations related to light traveling in tissue; Yamamoto et al.,
2002), and depth resolution by using multiple source-detector separations. In adults,
diffuse light reflection between white and gray matter limits depth resolution even
if source-detector distance is increased, whereas information from deeper regions
can be obtained in neonates due to the lower density of white matter (Fukui, Ajichi,
& Okada, 2003). Overall, the spatial resolution of CW NIRS is superior to EEG and
MEG but inferior to fMRI at the cortical surface. Unlike fMRI, fNIRS experiments
usually measure targeted brain areas by attaching some probe pads onto regions of in-
terest. We will mention how to place the pads to examine language-related areas later
in this chapter.
Comparison with Other Methods: Pros and Cons

Advantages
One of the strong advantages of NIRS is its flexibility and innocuousness. In addition
to the fact that multi-channel NIRS systems are portable, NIRS with flexible optical
fibers can accommodate almost any head position and posture, and can tolerate a de-
gree of head movement. These advantages allow its use in various types of experimental
settings and tasks, including interpersonal communication and language-production
tasks. Above all, these advantages enable examination of young infants, children with
developmental disabilities, and people of all ages with cochlear implants. In contrast,
fMRI requires the infant’s head to be held stationary for long periods so that infants have
to be asleep (naturally or sedated). Consequently, fMRI is not normally the primary
choice for studies on the developing brain, although infant fMRI studies, particularly
studies on anatomical structures and resting state, are rapidly multiplying.
Another advantage of fNIRS over fMRI related to the study of language is that op-
tical techniques can be conducted in silence, whereas an MRI scanner is a very noisy
environment. Some additional advantages of NIRS over fMRI include its lower cost and
compatibility with other electrical or magnetic monitoring systems (e.g., eye-tracking
system, electrocardiograph, and EEG) and therapeutic devices.
Although EEG shares some of the same advantages as NIRS methods compared
to fMRI, such as portability, silent operation, and tolerance of posture, its ability to
localize the focus of activity is generally poorer than that of fNIRS. Because the elec-
trophysiological signal is so weak, many repetitions of stimulus events are required
to be able to average out noise, which results in a longer experimental session with
EEG than with fNIRS. In addition, most EEG systems require more careful electrode
placement than fNIRS, which takes a relatively long time. MEG is far superior to EEG
in these respects. However, MEG requires a strict fixation of the participant’s head
relative to the sensor assembly, which often requires sedation for young participants.
Both EEG and MEG are excellent at resolving millisecond neural events, which
escape fNIRS.
Limitations
While fNIRS represents a potentially powerful tool for cognitive studies, it has several
crucial limitations. NIRS offers poorer spatial resolution and depth sensitivity than
fMRI, which prevents us from measuring deep regions such as the basal ganglia in re-
lation to language/grammar, or the amygdala and hippocampus in relation to emotion
and memory processing. However, many other language-related areas, namely superior
temporal gyrus (STG), supramarginal gyrus (SMG), angular gyrus, and inferior frontal
gyrus (IFG), can be recorded.
Another difficult issue is that NIRS signals measured from the scalp surface include
systemic vascular effects not only from inside, but also from outside the brain. These
changes originate from cardiac and respiratory pulsations, physiological vasomotion
(Mayhew et al., 1999), and low-frequency oscillations (Katura, Tanaka, Obata, Sato, &
Maki, 2006). Therefore, it is important to use experimental tasks that do not evoke large
systemic vascular changes, unless they are independently monitored (e.g., heart rate,
blood pressure, and skin conductance). This may not be a big issue in typical language
tasks, where no strong emotional factors are evolved.
Another potential problem is the change in blood volume in the scalp and within the
muscles beneath the optical probes. There are some crucial points to consider in de-
signing experimental tasks with NIRS. Although a language-production task or oral re-
sponse to a question is feasible with fNIRS, one must first make sure that the probes do
not sample from tissue where blood flows toward the specific muscles, including the
so-called temporal muscle. Similarly, when performing studies of interpersonal com-
munication, one must be careful that signals picked up on probes placed on the fore-
head are not contaminated by local movement due to the participant producing facial
expressions involving this area. Head position may also affect blood volume. For in-
stance, if a participant tilts his or her neck to the right, the blood volume will increase on
the right side, resulting in Hb signal changes. Such false-positive brain activation caused
by neck tilting has been examined in relation to the area volume of the common carotid
artery and internal jugular vein (Takeda, Gunji, Watanabe, & Kato, 2008).
A standard method to exclude systemic vascular effects from external or noise signals is
lacking for NIRS, but a variety of methods have been investigated in this respect (see dis-
cussion of data analysis later in this chapter). Further clarification of the systemic vascular
changes and hemodynamic physiology of the brain are necessary to establish a standard
framework for data analysis. Discussions of further technological or methodological is-
sues related to NIRS can be found elsewhere (Aslin & Mehler, 2005; Hebden, 2003).
Experimental Design and Analysis
Designing an optimal experimental paradigm is crucial to perform a successful NIRS

experiment, just as in other neurocognitive techniques. Because experimental design is
critically related to the analysis methods that can be applied, this section describes some
key aspects of the analysis methods, and provides readers with elementary knowledge
concerning study design, including task selection and experimental time intervals.
Experimental design and analysis methods of NIRS are roughly comparable to those
for fMRI because both methods are based on hemodynamic parameters. Thus, analysis
methods for NIRS can utilize standard methods in fMRI, such as general linear models
(GLM) and connectivity analyses. However, the fact that measurements are derived
from sensors placed on the scalp leads to other aspects being more similar to analyses
for EEG, as exemplified by the questions regarding sensor attachment.
How to Schedule Time Intervals
Block Design
As in BOLD fMRI, the block paradigm and the event-related (trial-based) paradigm are
the two major designs used to measure the hemodynamic responses in fNIRS studies.
In the block design, individual stimulus trials or events are tightly clustered into “on”
periods of activation, alternated with “off ” control periods (e.g., alternating 20 s “off/
rest” and “on/task” periods). Because this method generally employs long “on” periods
ranging from 10 to 40 s, relatively large brain activations are usually obtained. Therefore,
a large number of blocks (a set of “on” and “off ”) repetitions is not usually necessary, and
one can perform several conditions for “on” periods within one session. In our studies
(Minagawa-Kawai, Mori, Sato, & Koizumi, 2004; Arimitsu et al., 2011), for example, a
phonemic condition and a prosodic condition are pseudo-randomly presented as “on”
periods against a control condition as “off ” period within a single session. This kind of
block design has been widely used in language studies with NIRS for both children and
adults. To determine the language laterality for patients requiring surgery for seizure
relief, a word fluency test is employed with NIRS (Watanabe et al., 1998). Patients are
instructed to write words starting with a certain letter of the alphabet presented for each
“on” period, while during the “off ” period they draw a picture by copying a prepared
stimulus as a control task.
Event-Related  Design
Neural activity in response to specific events can be time-locked using an event-
related design (Friston et al., 1998; Friston, Zarahn, Josephs, Henson, & Dale, 1999;
Zarahn, Aguirre, & D’Esposito, 1997). Hemodynamic changes can be measured in re-
sponse to single events, such as performing one trial or the onset of a single movement.
After the single event, a baseline period which usually has various time intervals are
inserted. In general, Hb changes to any one trial are too small to detect; accordingly,
averaging numerous trials is necessary to obtain a clear signal. Nevertheless, there are
various advantages favoring an event-related paradigm over a block design. For ex-
ample, while participants can sometimes anticipate the next stimuli in a block design,
the event-related design keeps the participant’s attention uniform without anticipation.
Furthermore, the random time intervals prevent Hb data from being contaminated by
systemic vascular signals such as heart beats. Variable time intervals increase statistical
power (Birn, Cox, & Bandettini, 2002). The event-related design significantly broadens
the types of neural processes that can be investigated.
HRF and Time Schedule

Performing pilot studies to fix the experimental time schedule is important, because the
optimal time schedule must provide a large enough HRF in response to the stimuli/task,
and such optimal timing varies depending on the stimuli/task. In a study on phono-
logical grammar, Minagawa-Kawai et al. (2013) showed that a certain combination of
experimental conditions worked better, and led to obtaining an HRF that was more than
five times larger than other conditions. Specifically, a longer stimulus period with more
salient auditory stimuli evoked larger responses than “on” blocks that were shorter and
with more similar stimuli. Another issue of interest pertains to the length of the rest
period. Ideally, one must have a good idea of the expected time course of the hemody-
namic response elicited by an intended task, so as to set up a long enough rest period.
Neural activation causes changes in Hb, an increase in blood flow, and increases in local
oxygenation (Fox & Raichle, 1986). Typically, hemodynamic responses should return
to baseline during the rest period to facilitate the separation of Hb signals between the
task and rest periods. As shown in the fMRI literature, the time latency and amplitude
of hemodynamic responses vary depending on the region and the task. Thus, the timing
of Hb return should be checked in a pilot study. We note that this is particularly desir-
able when doing averaging analyses, but it is not relevant when applying GLM analysis,
because one can use a de-convolution process with GLM. If we assume that the cerebral
responses to different and successive cognitive events are linearly accumulated, then
each response curve can be separated by the de-convolution procedure, provided dif-
ferent conditions are presented (Wobst, Wenzel, Kohl, Obrig, & Villringer, 2001).
Where to Attach the NIRS Probe
According to the study hypothesis, one should first decide the regions of interest, which
should be on the cortical surface. Usually, the NIRS probe pad is attached such that the
most focused area can be precisely covered by some pairs of emitter and detector. In
targeting the STG as posterior language area, for example, T3 and T4 positions (from
the international 10–20 system) are typically used as landmarks in attaching the probes
T3
Figure 7.5. An example of probe attachment to measure language areas. 3 x 5 optode array was
fitted to temporal and frontal areas using landmarks of the international 10–20 method (left).
Estimation of brain regions and channel positions (right).
by employing a procedure called spatial registration (Tsuzuki et al., 2007). In the case
of a 3 x 5 optode (optical sensor) array, if we place one probe onto T3 and position the
bottom probe line to be roughly fitted on T3-Fp1 line (Figure 7.5 left), channels positions
are estimated to cover left temporal brain regions as indicated in Figure 7.5 (right). This
positioning allows us to measure the SMG and IFG and is thus suited to examine the
language network in the perisylvian area. This registration method has been devel-
oped to probabilistically register NIRS data onto the Montreal Neurological Institute
(MNI) coordinate space or Tarailach coordinates without using anatomical MR images
(Okamoto et al., 2004; Tsuzuki et al., 2007). Although this method is currently gen-
erally applicable to adult brains, it can be applied to infants and children under some
conditions of probe alignment. As for school-age children, this method is applicable
without any additional procedures because brain sizes are almost identical between
school-age children and adults (Sugiura et al., 2011). Spatial registration methods are
still in development, and various new methods have been proposed (Tsuzuki & Dan,
2014). (Spatial estimations of various probe arrays are available at http://brain.job.affrc.
go.jp/resources/index.php.) Therefore, one way to decide the attachment location is to
refer to these estimation results. However, in employing this method, it is necessary to
precisely measure head sizes, following the international EEG 10–20 system.
Language Tasks
Almost all language tasks performed in fMRI studies are applicable to fNIRS. Several
other tasks are additionally possible with fNIRS because of its tolerance for partici-
pant motion. Participants are able to conduct movement tasks, as exemplified by a se-
ries of fNIRS studies on motor skills and rehabilitation (Hatakenaka, Miyai, Mihara,
Sakoda, & Kubota, 2007; Miyai et al., 2001; Miyai et al., 2003.). Thus, writing, reading,
speaking, and listening tasks are all appropriate. If adequate space and channels are
available, it is feasible to simultaneously measure and examine the interactive com-
municative processes of two people. Wireless NIRS will further expand its task ap-
plicability. Nonetheless, some considerations unique to fNIRS should be taken into
account when designing tasks. First, it is necessary to preclude tasks that trigger large
facial movements or other specific motion-related artifacts. As stated previously, the
experimenter should carefully check for movement in the temporal muscle when
employing an oral pronunciation task. Emotional stimuli affect the facial expressions
of participants, causing artifacts in the probes placed on the forehead. Emotional
stimuli also affect the systemic vascular change, which in turn influences NIRS signals.
It is therefore recommended that biological signals such as blood pressure and skin
circulation be monitored while performing emotional studies (see discussion of co-
registration later in this chapter).
In performing an fNIRS experiment, additional behavioral testing prior to or after
the fNIRS testing is sometimes very useful for interpreting fNIRS data. In a study where
we examined infants’ brain responses to their own and other names (Imafuku, Hakuno,
Uchida-Ota, Yamamoto, & Minagawa, 2014), behavioral attention while hearing the
names measured in a separate behavioral test correlated well with the amplitude of brain
responses from dorsomedial prefrontal cortex, suggesting a functional role for that re-
gion. Apart from the task, other general assessments, including handedness, develop-
ment, and intelligence, are also useful indices to verify participants’ general cognitive
abilities.
Combining NIRS with Other Modalities
NIRS can be easily combined with other physiological measurements. To compensate

for the low temporal resolution of NIRS, combining it with EEG or MEG is advised
(e.g., Telkemeyer et al., 2009). Since NIRS data are affected by systemic vascular signals,
measuring such external signals outside the brain (e.g., blood pressure and respiration)
is also useful (Katura et al., 2006). Because mothers’ infant-directed speech increases or
decreases infants’ heart rate (DeCasper & Fifer, 1980), it is possible that such a vascular
change affects NIRS signals. By taking this issue into account, we are currently per-
forming a study of NIRS in combination with EEG, electrocardiograph, and respiratory
measurements. This type of experiment can provide insights into a possible relationship
between brain function and the autonomic nervous system. As is frequently employed
in fMRI experiments, recording eye movements at the same time as the NIRS meas-
urement is also useful to interpret the Hb data. By combining NIRS with eye-tracking
measurements on fast-reading specialists and normal readers, Miyata et al. (2011) found
that cerebral activity around the angular gyrus was related to reading speed. If metal
free NIRS is available, fMRI-NIRS is feasible. This combination would be beneficial to
study the physiological bases of BOLD fMRI because absence of a BOLD response does
not always mean an absence of brain activation, as revealed by NIRS measurements

(Seiyama et al., 2004).
Data Analysis
NIRS lacks a standard method or software for data analysis, unlike fMRI studies, which
have some powerful parametric mapping tools such as SPM (see Heim & Specht,
Chapter 4 in this volume). However, there are some basic processes for data analysis
that have been employed in many NIRS studies. Here, we provide a quick summary
of such fundamentals of NIRS signal processing. As we have already described how
attenuation (absorbance) data are converted to Hb data, we will start with the pre-
processing of Hb data, which usually consists of artifact rejection, filtering, and de-
trending (Figure 7.6). Then several methods to evaluate the data, including averaging
and GLM, are illustrated, followed by information about analysis software that is avail-
able for free.
Artifact Rejection
NIRS data contain a variety of irregular signals, primarily due to body movement
and loose probe attachment. Artifacted blocks and/or channels are usually removed
as a whole, but depending on the case, artifacted data can be dealt with using certain
mathematical methods (e.g., linearly interpolating a short missing stretch, or in GLM
giving that region of the signal a weight of zero in the regression). Although artifacts
are characterized by a sharp change that can be detected easily by the human eye, objec-
tive methods such as setting a threshold using the standard deviation of the Hb data are
Conversion Artifact De-

Attenuation Filtering
to rejection trending
data
Hb data
0.5 0.5 0.5

∆[Oxy-Hb]
∆[Oxy-Hb]
∆[Oxy-Hb]
0 0 0
–0.5 –0.5 –0.5
0 50 100 150 200 0 50 100 150 200 0 50 100 150 200

Time [s] Time [s] Time [s]
Figure 7.6. Several stages of pre-processing the NIRS data.

Source: Three figures at the bottom are taken from Takeda (2007).
generally employed. Several additional methods are employed to exclude external noise,
such as wavelet-based algorithms (Sato et al., 2006) and principal component analysis
(Zhang, Brooks, Franceschini, & Boas, 2005).
Filtering
NIRS data contain vascular signals due to cardiac and respiratory pulsations and phys-
iological vasomotion (Mayhew et al., 1999). Raw data consist of various frequency
components, some of which may be noise. The activation signals must be reliably
separated from those of the noise components. Filtering removes this type of base-
line frequency. The choice of filter types (high-pass, low-pass, band-pass, or band-
elimination) depends on the type of noise component to be excluded. A moving average
is another method to smooth data. A wide range of high-frequency components can be
eliminated with a longer duration moving average. This means that the moving average
functions as a low-pass filter.
De-trending
Concentration changes measured with NIRS tend to show constant increments or
decrements, mainly due to natural physiological fluctuations (Figure 7.6). The de-
trending process works to remove such linear drift. With a GLM approach, which will
be mentioned in the last part of this section, the filtering and de-trending processes can
be achieved by employing multiple regressors in the design matrix.
Hb Parameters
NIRS studies generally employ oxy-Hb as the dependent measure when examining
brain activation. Although it is generally accepted that oxy-Hb and total-Hb are
correlated with BOLD (Strangman et al., 2002), the relationships between Hb
parameters and BOLD obtained in prior studies are sometimes conflicting (Steinbrink
et al., 2006). Huppert, Hoge, Diamond, Franceschini, and Boas (2006), for example,
reported that oxy-Hb has a higher SNR, whereas deoxy-Hb better reflects region-
specific activation.
Averaging Method
Averaging is widely and routinely used to analyze EEG data. Typically, after integrating
information across all the blocks to derive a grand-averaged Hb time-course (Figure 7.7,
top right panel), oxy-Hb changes during the baseline period and the task (stimulation)
0.4
0.3
∆[Oxy-Hb]
(B) 0.2
(A) 0.1
Pre-processed data Average
0
0.5 –0.1
∆[Oxy-Hb]
0 10 20 30 40
0
Time [s]
(C)
–0.5 1
0.8
∆[Oxy-Hb]
GLM
approach 0.6
0 50 100 150 200 0.4
0.2
Time [s] 0
0 100 200
Time [s]
Box-car function convolved with HRF
Estimation of the goodness of fit
between the data and model
Figure 7.7. Analyzing pre-processed data (A) with averaging (B) and GLM (C) methods.

Source: These are modified figures, each of which is taken from Takeda (2007).
period are compared. As the number of blocks increases, random background noise
diminishes, and more function-related brain signals can be extracted. Similar to EEG,
time windows for baseline and task periods are set, and the averaged values for each
participant are subjected to a second-order analysis with a t-test or an analysis of vari-
ance (ANOVA).
General Linear Model (GLM)

The GLM approach is typically employed in the analysis of BOLD-fMRI data (e.g.,
Friston, 1994). Although this approach is very popular in the field of fMRI, it is em-
ployed more rarely in NIRS. In plain terms, GLM estimates the association between
brain-scanner data and an ideal brain-activation model, which is a linear combina-
tion of a set of modeled HRFs plus random noise. If the response data fit well with the
model response, which indicates a high correlation, then one can say that the data show
strong brain activation. Pre-processed raw data (Figure 7.7, left panel), for instance,
can be evaluated with the model indicated in Figure 7.7 C (bottom right panel). This
model is simple in that it only contains a prediction of the expected HRF response, but
other variables can be added to the analysis matrix so as to model certain aspects of
the “noise” (e.g., sine and cosine functions at different periods to filter out vascular sig-
nals). It is difficult to identify an optimal analysis matrix a priori (i.e., how many and
which type of models to include to best represent the study outcomes). Using more
model functions than necessary lowers the sensitivity by taking up degrees of freedom,
whereas an insufficient model may enhance the risk for systematic errors, resulting in
lower statistical power. Although selecting the optimal model is critical, it may vary
depending on the brain region of interest, experimental time setting and task, and age of
the participants (e.g., adults vs. infants).
Connectivity Analysis
Functional connectivity has been increasingly popular in fMRI studies in the last
decade, and fNIRS has also employed some techniques borrowed from fMRI, as
exemplified in the following review section. Due to differences in measurement prin-
ciples and limited measurement of brain areas in NIRS, not all the methods from fMRI
can be used for fNIRS analysis. At present, cross-correlation and phase-locking analyses
are some representative methods to assess connectivity. To assess the connectivity be-
tween two persons, namely, synchronicity of the brain activities in two persons, wavelet
coherence is typically used.
Available Software for NIRS Analysis

A variety of software for fNIRS analysis is available. These include HomER, functional
optical signal analysis (foSA) (Koh et al., 2007), NIRS-SPM (Ye, Tak, Jang, Jung, &
Jang, 2009), and Platform of Optical Topography Analytic Tool (POTATo) offered by
Hitachi. Most are custom Matlab programs, which are available as freeware. A package
for spatial estimation is available at http://brain.job.affrc.go.jp/resources/index.php.
Language Studies with fNIRS:

A Brief Review
In this section, we review a handful of NIRS studies bearing on language acquisi-

tion. Our goal is not to provide an exhaustive review, but rather to introduce a few
examples illustrating the strengths and limitations of the technique for the study
of language acquisition. Readers interested in more extensive discussions can read
Quaresima, Bisconti, and Ferrari, (2012) for an overview of fNIRS studies on lan-
guage processing across the life span; and Aslin (2012) for a recent summary and
critique of infant fNIRS studies, including language. Here, we will cover only
three salient topics: the biases leading to left dominance; the development of brain
networks engaged during language processing; and some clinical applications. Our
closing words highlight the unique advantages of fNIRS research for the study of lan-
guage acquisition.
Biases Leading to Left Dominance

It is clear that linguistic knowledge and neural implementation go hand in hand, such
that localized damage to the brain results in specific linguistic impairments. Moreover,
differences in speech perception due to diverse linguistic experiences are expressed
through differences in neural commitment (e.g., Lenneberg, 1966). This work has
highlighted the engagement of perisylvian cortices in the left hemisphere for speech
processing (e.g., Binder et al., 2000; Zatorre & Gandour, 2008). But what are the biases
driving this left-lateralization in the engagement of cortices in the STG and supe-
rior temporal sulcus (STS)? Previous work, primarily with adults, has suggested that
speech may be associated with left dominance due to (a) its acoustic characteristics;
and/or (b) its intrinsic linguistic nature. Signal-driven hypotheses propose that the
rapid, changing acoustic nature of spoken stimuli is better captured by the left hem-
isphere neural structures (e.g., Zatorre & Belin, 2001), while domain-driven theories
argue for an innate language module located in the left perisylvian area, which detects
and processes language already at birth (Dehaene-Lambertz & Gliga, 2004).
Adult evidence is equivocal on the question of what shapes such language networks,
since all the many factors that could be playing a role (e.g., acoustic and language experi-
ence) are confounded. Moreover, although it is possible to test infants with EEG, MEG,
and fMRI, the first is not ideally suited to answer questions of strict localization, and the
latter two are both challenging to implement for infants and relatively expensive. In this
context, cheap, infant-friendly NIRS has provided the field with a much-needed tool
to investigate the differential engagement of left and right perisylvian cortices in two
ways. First, it has allowed researchers to measure the effects of the careful manipulation
of specific factors, associated with the two families of hypotheses noted earlier (signal-
and domain-driven), in healthy newborn infants, a population that is challenging to
neuroimage otherwise. The ease with which newborn fNIRS studies are carried out
effectively means that many more research groups can gather independent, comple-
mentary evidence on the same question. Second, NIRS also facilitates tracking devel-
opmental curves because the technique is applicable to a range of ages, including awake
mobile toddlers. As we will see, both of these features have been critical to shedding
light on the ontogenetic origins of left-dominant language processing.
We begin with a specific example. Telkemeyer et al. (2009) investigated a specific
hypothesis within the signal-driven class, according to which left auditory cortices re-
spond more than the right when presented with short sounds, whereas the opposite
lateralization should be observed for longer sounds. When newborns were presented
with these two types of auditory stimulation, one channel tapping temporo-parietal cor-
tices showed the predicted crossover, although this asymmetry was not strong enough
to survive correction for multiple comparisons. Using a different set of stimuli that
manipulated spectral entropy (fast versus slow changes), Minagawa-Kawai, Cristia,
Vendelin, Cabrol, and Dupoux (2011) also failed to uncover any modulation of the asym-
metry of the activation (although they found a modulation of the size of the activation).
Together, these studies suggest that these two specific instantiations of the signal-driven
hypothesis might not apply to the newborn brain, and that others should be entertained
(although see Poeppel, Idsardi, & van Wassenhove, 2008, for an argument that signal
factors explain right-hemispheric engagement rather than left-dominance per se).
As mentioned earlier, signal-driven hypotheses are one of the two mainstream
families of explanations attempting to account for left-dominant cortical engagement
during language processing. The second family proposes that in addition to, or instead
of, any signal-driven biases, left perisylvian cortices are innately prepared to process lin-
guistic material. We have reviewed infant neuroimaging evidence from birth through
adulthood by evaluating results against the predictions made by each family of explana-
tion (Minagawa-Kawai et al., 2011). We concluded that neither the signal-driven nor the
domain-driven explanations, on their own, sufficed to cover extant findings. Instead,
we proposed that these two and a third explanation must be weaved together, as follows.
Early on, slow and/or prosodically rich stimuli modulate right-hemispheric STG and
STS engagement, whereas to a certain extent left-dominant activations must await
infants’ retrieval of certain abstract properties in their input, such that the domain-
driven biases are established at least partially via learning.
Evidence corroborating this proposal comes from three newborn NIRS studies. Peña
et al. (2003) presented Italian newborns with infant-directed sentences in Italian played
forward or backward, and found that the strongest discrimination between these two
types was found in left channels tapping STG and/or STS. Two groups carried out in-
dependent extensions of that work, crossing the direction factor (forward versus back-
ward) with a familiarity factor (native versus foreign speech). Sato et al. (2012) presented
Japanese newborns with infant-directed Japanese and English, both forward and back-
ward, and they replicated Peña’s findings in the Japanese forward-backward compar-
ison, but not with the English stimuli. For their part, May, Byers-Heinlein, Gervain,
and Werker (2011) tested Canadian-English neonates with English and Tagalog for-
ward and backward sentences that had been low-pass filtered, such that the intonation
and rhythm were perceptible, but their segmental properties were otherwise removed.
No asymmetry in processing was apparent in the latter study. Taking all three studies
together, we can see that the activation of left STG/STS is not modulated by direction
when only the slow, prosodically rich components of speech are presented. In contrast,
when speech is presented in all its complexity, left-dominance occurs for familiar for-
ward stimuli, fitting with the notion that domain-driven biases benefit from learning. In
the present case, learning will have occurred in the womb, as much previous behavioral
evidence has shown tuning to the native language’s broad characteristics by birth (e.g.,
Mehler et al., 1988).
As mentioned earlier, NIRS also provides a window on development by virtue of
being applicable to a range of ages. The prime example comes from studies on processing
of sound contrasts. In our review (Minagawa-Kawai et al., 2011), we identified nine
such experiments on infants 0–20 months of age and adults, covering the full range of
contrasts employed in the world’s languages (consonants, vowels, duration, and tones)
in addition to non-segmental contrasts (prosody). Some of the contrasts were present
in the listener’s native language, and others were not. In general terms, one observes
that discrimination of sound contrasts, regardless of the type and the native status, are
processed bilaterally early on, with increasing lateralization (left-dominant for native
contrasts, right-dominant for prosodic contrasts) with age, which is already clear at 10–
20 months. We will return to this age range later, but for now we note that this evidence
fits well with the proposal of lateralization for linguistic processing being reinforced as a
function of both age and experience.
In sum, this section has summarized a few fNIRS studies bearing on the very early
sources of leftward asymmetries in STS/STG during language processing. We have seen
that this research profits from the technique of NIRS through independent conceptual
replications as well as longitudinal comparisons, which together suggest that there is a
developmental and experiential component to asymmetric language processing.
The Development of Brain Networks

The previous subsection dealt with a question for which NIRS was useful because of the
possibility of assessing asymmetries in localized activation. In the present subsection,
we deal with another problem related to localization, namely the emergence of larger
networks, involving multiple cortical regions. Indeed, it is widely agreed upon that to
explain language processing we must not only describe localized activations, but also
how activation is correlated across different regions. There are only a handful of studies
assessing global networks with EEG and fMRI in infancy and childhood (see Chu-
Shore, Kramer, Bianchi, Caviness, & Cash, 2011, for a review), and none of them bears
directly on language acquisition.
This is, for the time being, a question on which there is evidence from only one lab
(as of 2014). Using a unique high-density NIRS system, Homae, Taga, Watanabe, and
colleagues have investigated correlations in low-frequency oscillations across 94 meas-
urement channels, distributed around the head to tap frontal, temporal, parietal, and
occipital regions.
In one study (Homae, Watanabe, Nakano, & Taga, 2011), this pad was used to col-
lect data from 3-month-olds during three 3-minute periods: quiet, awake resting
state; stimulation with recorded spoken passages; and another silent period. The
correlations found during the initial resting state replicate those found in a previous
study (Homae et al., 2010): They were stronger within regions and within hemispheres
than between regions or hemispheres. During speech stimulation, this pattern
changed somewhat, with strengthening of fronto-temporal correlations, particularly
in the left hemisphere. This altered pattern was maintained during the third, quiet, pe-
riod, suggesting that strengthening of fronto-temporal correlations was not solely due
to the direct effect of speech stimulation, but indicated instead the involvement of an
emergent functional network.
Interestingly, the same effects were observed when the stimulation consisted of pure
tones (Taga, Watanabe, & Homae, 2011), suggesting that this precise functional network
may not be specific to language processing. Our data (Uchida-Ota et al., 2014) add some
insights into this. We measured 37 full-term neonates to examine connectivity during

listening to infant-directed speech, spoken by either their mother or a stranger, using
phase-locking analysis. Mother’s speech exclusively evoked a strong and dense fronto-
temporal connectivity. Connectivity between STG and inferior frontal areas was spe-
cific to mother’s speech. We would look forward to additional studies that may shed
light on networks that are specifically involved in linguistic processing, either by testing
older infants and children (who would presumably be proficient enough to carry out
such operations) and/or by using stimulation paradigms that help infants focus on lin-
guistic aspects of speech, rather than solely the acoustic ones.
Some Clinical Applications

The prime example of clinical application of NIRS comes from the study of language
processing by individuals with cochlear implants. Since the implant is typically an elec-
trical device with metal parts, it is not possible to use MRI with implanted individuals,
and the EEG/MEG signal has to be carefully sifted due to the interferences from the
device itself (when it is turned on). Thus, fNIRS research is in a unique position to
shed light on language processing by infants and children who have received cochlear
implants.
We report on an example of this literature in some detail both because it illustrates
this unique vantage point, and because it allows us to bring out some strengths and
weaknesses of the application of NIRS to study language acquisition among clinical
populations. Sevy et al. (2010) measured responses to speech in left and right temporal
cortices in four populations: a group of normal-hearing adults; another of normal-
hearing children (aged 4–15 years; 9 years old on average); a group composed of deaf chil-
dren with at least 4 months of use of a cochlear implant (aged 2–19 years; 8 on average);
and a group of deaf children at the moment of activation of the cochlear implant (aged
2–8 years; 5 on average). Several aspects of their data are of keen interest to us, as follows.
The first relates to the proportion of participants whose data could be included be-
cause they accepted the cap and were quiet enough during testing. This percentage was
92% among normal-hearing children, 93% among experienced cochlear-implant chil-
dren, and nearly 70% among the younger children whose implants had just been turned
on. These acceptance rates are encouraging because they suggest that NIRS may be a
technique that could be used with a majority of participants.
Second, and also on the methodological side, they found significant activations to
speech (in oxyHb and/or deoxyHb) for 100% of adults; 82% of normal-hearing chil-
dren; 76% of experienced cochlear users; and 78% of the children tested at activation.
Notice that percentages are very similar across all child groups, and differences across
the groups do not correlate with differences in perceptibility that are likely found be-
tween (inexperienced) cochlear users and normally hearing children. The fact that
percentages are lower among all child groups than the adults suggests a limitation of the
technique, since they were all stimulated with the same sounds but activations were not
registered in all cases. What may be the reasons for such variability? We know that chil-
dren have thinner skulls, such that their optical data are both stronger and less affected
by surface artifacts than adults. Thus, weaker sensitivity is not a convincing explana-
tion. In contrast, three explanations related to weaker signals seem more convincing.
It is possible that children payed less attention to the stimuli, and narrower cortical re-
gions were therefore activated (potentially limited to primary auditory areas, which,
located in sulci, are less accessible with NIRS). Furthermore, individual differences
in functional engagement of a given cortical region might be greater among children
than among adults. Finally, there may be more variability in scalp-to-cortex mapping
among children than adults, leading to greater cap placement errors (even when surface
landmarks are carefully used) in the former than the latter population.
Regardless of these limitations, a further aspect of their data illustrates the potential of
such studies to inform psycholinguistic research. We have explained earlier the interest
in studying lateralization for speech processing. We might then wonder, comparing the
three groups of children, whether lateralization patterns may differ among the groups,
given that two of them have lacked exposure to speech during the critical period of in-
fancy. In fact, responses in the two clinical groups were stronger and more stable (at least
in terms of oxyHb) in the right than the left hemisphere, whereas the normally hearing
children exhibited a clear left-dominant response. One aspect of these results is, how-
ever, intriguing: Adults, like the two clinical groups, showed clearer responses in the
channels located on the right hemisphere. Given the stimuli samples were not chosen to
isolate specific characteristics (e.g., processing of segmental contrasts), we cannot easily
explain either the right bias observed in adults, or the observed differences among the
child populations. More research is needed to understand whether these diverse later-
alization patterns were specific to speech (by comparison to, e.g., music), and to what
extent they were affected by age and amount of auditory experience.
Before closing this section, we mention two additional strands of literature
illustrating clinical applications of NIRS. By virtue of its ease of use, innocuousness, and
portability, NIRS is a good tool for bedside testing of a number of clinical populations.
Many studies have been carried out on preterm infants and infants that are hospitalized
(due to low birth weight, respiratory distress syndrome, etc.). A number of these studies
are closely related to the hospitalization experience (e.g., response to pain; Slater,
Cantarella, Franck, Meek, & Fitzgerald, 2008) or on the correlates of simple perception
(odor, auditory, visual, etc.; e.g., Kusaka et al., 2004). An interest in language is rarer,
and the evidence is too scarce, at present, to reveal patterns that are of specific interest
to psycholinguists. For example, Nishida et al. (2008) report that temporoparietal
channels respond to speech faster in infants born preterm tested around their due date
compared to full-term neonates tested shortly after birth. It is unlikely, however, that
this reflects differential linguistic processing of the speech signal, since a similar pat-
tern (faster responses in preterms than full-terms, matched in gestational age) had been
found with sinewave tones (Kotilahti et al., 2005). Moreover, one cannot, at present, be
certain whether such differences in processing should be attributed to the infants’ dif-
ferential life experience and/or differences in neurological development caused by their
perinatal conditions (see Bosch, 2011, for a recent discussion of literature on preterm
speech processing and language acquisition).
Additionally, NIRS can also be used with any other clinical population. For instance,
a recent strand of work focuses on lateralization patterns in children with a diagnosis of
autistic spectrum disorder (ASD) (Minagawa-Kawai et al., 2009), and in children who
stutter (Sato et al., 2011). Minagawa-Kawai et al. (2009), for instance, presented ASD
children (aged 9 years) and typically developing children (aged 7 years) with blocks of
words such that, within a block, the same word was repeated over and over (e.g., itta
itta itta . . . ; these are called non-alternating blocks) or alternated with a minimal pair
(e.g., itta itte itta . . . ; these are called alternating blocks). Previous work had shown that,
when presented with an alternation of vowels, as in the example just given, adults and
normally-developing toddlers show left-dominant responses in STS/STG (Furuya
& Mori, 2003; Sato et al., 2003). Minagawa and colleagues replicated these results in
their sample of normally developing children, whereas responses in a group of autistic
individuals were perfectly bilateral.
Final Remarks
Before moving on, we would like to point out some populations and topics in which
NIRS does not hold a particular advantage. In this section, we have focused more on
infants than young children, and very little on older children, in part because this
reflects the amount of research that is done on each age. It is widely believed that the
early years make a greater contribution to first language acquisition than later ones (e.g.,
Dupoux, Peperkamp, & Sebastian-Galles, 2010), which may in part explain the bimodal
distribution of ages tested. Nonetheless, children of some ages (saliently between 1.5 and
2.5 years of age) are simply more difficult to test, and NIRS caps, albeit generally well
accepted (as we saw in the numbers from Sevy et al., 2010), still need to be placed and
tolerated for long periods of time.
We have not reviewed research on second-language acquisition, despite the fact that
it is a topic that has attracted some attention in the NIRS literature (Minagawa-Kawai
et al., 2004). This is simply because NIRS advantages and disadvantages for this topic
are roughly the same as those we have mentioned elsewhere—there may be more labs
using it because it is inexpensive, but nothing specifically recommends NIRS for the
study of this topic.
Future Prospects
NIRS has a huge potential for various applications in relation to language because of
its innocuousness and mobility. Applications for brain-machine interfaces is one ex-
ample, as NIRS could conceivably be used to pick up brain signals and convert them
into speech production commands for patients of amyotrophic lateral sclerosis or

patients without vocal tract. In fact, a very primitive level of such a brain-machine in-
terface was implemented and marketed in 2005, although it was limited to “yes” or “no”
responses at that time. Furthermore, there is one area of research that does not exist
but would be ideally studied using NIRS, and that is the development of social interac-
tion and conversational skills. One recent study reports on correlations in NIRS data
collected from two people playing a joint computer game (Cui, Bryant, & Reiss, 2012),
using similar analysis techniques to the ones we evoked for the development of global
networks. NIRS researchers are in a unique position to apply such methods to the study
of natural communications, carried out in an ecological setting, whereby two or more
people simply talk to each other face to face (while each wears a NIRS cap). Beyond the
two-person hyper-scanning, NIRS would enable us to perform multi-person simulta-
neous recording, which may further reveal brain system for social communicative be-
havior in the real world. Apart from such pioneering use, traditional studies of human
brain imaging will also move forward greatly with the advance of data analysis methods
such as those for brain connectivity, and technical advances including the general use of
high-density NIRS. In particular, further details of the neural substrates of language ac-
quisition in infants and children will be unveiled in the future. Currently, fNIRS meth-
odology is still developing with several limitations, as described in this chapter. We
believe many of these problems will be overcome in the near future, and many more
linguists and clinicians will start to use fNIRS for a broad range of applications, leading
fNIRS to grow into a major technique in the cognitive neurosciences.
References
Arimitsu, T., Minagawa, Y., Yagihashi, T., Uchida-Ota, M., Matsuzaki, A., Ikeda, K., &
Takahashi, T. (2018). The cerebral hemodynamic response to speech in preterm and term
infants: The impact of postmenstrual age. Neuroimage Clinical, 19, 599–606.
Arimitsu, T., Uchida-Ota, M., Yagihashi, T., Kojima, S., Watanabe, S., Hokuto, I., Ikeda, K.,
Takahashi, T., & Minagawa-Kawai, Y. (2011). Functional hemispheric specialization in pro-
cessing phonemic and prosodic auditory changes in neonates. Frontiers in Psychology, 2, 202.
Arridge, S. R., & Lionheart, W. R. (1998). Nonuniqueness in diffusion-based optical tomog-
raphy. Optics Letters, 23(11), 882–884.
Aslin, R. N. (2012). Questioning the questions that have been asked about the infant brain
using near-infrared spectroscopy. Cognitive Neuropsychology, 29(1–2),  7–33.
Aslin, R. N., & Mehler, J. (2005). Near-infrared spectroscopy for functional studies of brain
activity in human infants: Promise, prospects, and challenges. Journal of Biomedical Optics,
10(1), 11009.
Binder, J. R. (2012). Task-induced deactivation and the “resting” state. NeuroImage, 62(2),
1086–1091.
Binder, J. R., Frost, J. A., Hammeke, T. A., Bellgowan, P. S., Springer, J. A., Kaufman, J. N., &
Possing, E. T. (2000). Human temporal lobe activation by speech and nonspeech sounds.
Cerebral Cortex, 10(5), 512–528.
Birn, R. M., Cox, R. W., & Bandettini, P. A. (2002). Detection versus estimation in event-related
fMRI: Choosing the optimal stimulus timing. NeuroImage, 15(1), 252–264.
Boas, D. A., Elwell, C. E., Ferrari, M., & Taga, G. (2014). Twenty years of functional near-
infrared spectroscopy: Introduction for the special issue. NeuroImage, 85,  1–5.
Bosch, L. (2011). Precursors to language in preterm infants: Speech perception abilities in the
first year of life. Progress in Brain Research, 189, 239–257.
Chance, B., Anday, E., Nioka, S., Zhou, S., Hong, L., Worden, K., Li, C., Murray, T., Ovetsky, Y.,
Pidikiti, D., & Thomas, R. (1998). A novel method for fast imaging of brain function, non-
invasively, with light. Optics Express, 2(10), 411–423.
Chance, B., Maris, M. B., Sorge, J., & Zhang, M. Z. (1990). A Phase modulation system for dual
wavelength difference spectroscopy of hemoglobin deoxygenation in tissues. Proceedings of
SPIE (Proc Soc Photo Optical Instrum Engr), 1204, 481–491.
Chu-Shore, C. J., Kramer, M. A., Bianchi, M. T., Caviness, V. S., & Cash, S. S. (2011). Network
analysis: Applications for the developing brain. Journal of Child Neurology, 26(4), 488–500.
Cristia, A., Dupoux, E., Hakuno, Y., Lloyd-Fox, S., Schuetze, M., Kivits, J., Bergvelt, T., van
Gelder, M., Filippin, L., Charron, S., & Minagawa-Kawai, Y. (2013). An online database of
infant functional near infrared spectroscopy studies: A community-augmented systematic
review. PLoS One, 8(3), e58906.
Csibra, G., Henty, J., Volein, A., Elwell, C., Tucker, L., Meel, J., & Johnson, M. (2004). Near in-
frared spectroscopy reveals neural activation during face perception in infants and adults.
Journal of Pediatric Neurology, 2(2), 85–89.
Cui, X., Bryant, D. M., & Reiss, A. L. (2012). NIRS-based hyperscanning reveals increased in-
terpersonal coherence in superior frontal cortex during cooperation. NeuroImage, 59(3),
2430–2437.
DeCasper, A. J., & Fifer, W. P. (1980). Of human bonding: Newborns prefer their mothers’
voices. Science, 208(4448), 1174–1176.
Dehaene-Lambertz, G., & Gliga, T. (2004). Common neural basis for phoneme processing in
infants and adults. Journal of Cognitive Neuroscience, 16(8), 1375–1387.
Delpy, D. T., & Cope, M. (1997). Quantification in tissue near– infrared spectroscopy.
Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences,
352(1354), 649–659.
Delpy, D. T., Cope, M., van der Zee, P., Arridge, S., Wray, S., & Wyatt, J. (1988). Estimation of op-
tical pathlength through tissue from direct time of flight measurement. Physics in Medicine
and Biology, 33(12), 1433–1442.
Duncan, A., Meek, J. H., Clemence, M., Elwell, C. E., Fallon, P., Tyszczuk, L., Cope, M., & Delpy,
D. T. (1996). Measurement of cranial optical path length as a function of age using phase re-
solved near infrared spectroscopy. Pediatric Research, 39(5), 889–894.
Dupoux, E., Peperkamp, S., & Sebastian- Galles, N. (2010). Limits on bilingualism
revisited: Stress ‘deafness’ in simultaneous French-Spanish bilinguals. Cognition, 114(2),
266–275.
Ferrari, M., Mottola, L., & Quaresima, V. (2004). Principles, techniques, and limitations of near
infrared spectroscopy. Canadian Journal of Applied Physiology, 29(4), 463–487.
Ferrari, M., Wei, Q., Carraresi, L., De Blasi, R. A., & Zaccanti, G. (1992). Time-resolved spec-
troscopy of the human forearm. Journal of Photochemistry and Photobiology B, 16(2), 141–153.
Flechsig, P. (1901). Developmental (myelogenetic) localisation of the cerebral cortex in the
human subject. The Lancet, 158(4077), 1027–1030.
Fox, P. T., & Raichle, M. E. (1986). Focal physiological uncoupling of cerebral blood flow and
oxidative metabolism during somatosensory stimulation in human subjects. Proceedings of
the National Academy of Sciences USA, 83(4), 1140–1144.
Franceschini, M. A., & Boas, D. A. (2004). Noninvasive measurement of neuronal activity with
near-infrared optical imaging. NeuroImage, 21(1), 372–386.
Friston, K. J. (1995). Commentary and opinion: II. Statistical parametric mapping: Ontology
and current issue. Journal of Cerebral Blood Flow & Metabolism, 15(3), 361–370.
Friston, K. J., Fletcher, P., Josephs, O., Holmes, A., Rugg, M. D., & Turner, R. (1998). Event-
related fMRI: Characterizing differential responses. NeuroImage, 7(1),  30–40.
Friston, K. J., Zarahn, E., Josephs, O., Henson, R. N., & Dale, A. M. (1999). Stochastic designs in
event-related fMRI. NeuroImage, 10(5), 607–619.
Fujiwara, N., Sakatani, K., Katayama, Y., Murata, Y., Hoshino, T., Fukaya, C., & Yamamoto, T.
(2004). Evoked-cerebral blood oxygenation changes in false-negative activations in BOLD
contrast functional MRI of patients with brain tumors. NeuroImage, 21(4), 1464–1471.
Fukui, Y., Ajichi, Y., & Okada, E. (2003). Monte Carlo prediction of near-infrared light propa-
gation in realistic adult and neonatal head models. Applied Optics, 42(16), 2881–2887.
Furuya, I., & Mori, K. (2003). Cerebral lateralization in spoken language processing measured
by multi-channel near-infrared spectroscopy (NIRS). No To Shinkei, 55(3), 226–231.
Gervain, J., Macagno, F., Cogoi, S., Peña, M., & Mehler, J. (2008). The neonate brain
detects speech structure. Proceedings of the National Academy of Sciences USA, 105(37),
14222–14227.
Gibson, A. P., Hebden, J. C., & Arridge, S. R. (2005). Recent advances in diffuse optical im-
aging. Physics in Medicine and Biology, 50(4),  R1–43.
Gratton, G., & Fabiani, M. (2001). Shedding light on brain function: The event-related optical
signal. Trends in Cognitive Sciences, 5(8), 357–363.
Gratton, G., Sarno, A., Maclin, E., Corballis, P. M., & Fabiani, M. (2000). Toward noninvasive
3-D imaging of the time course of cortical activity: Investigation of the depth of the event-
related optical signal. NeuroImage, 11(5 Pt 1), 491–504.
Gusnard, D. A., & Raichle, M. E. (2001). Searching for a baseline: Functional imaging and the
resting human brain. Nature Reviews Neuroscience, 2(10), 685–694.
Hatakenaka, M., Miyai, I., Mihara, M., Sakoda, S., & Kubota, K. (2007). Frontal regions in-
volved in learning of motor skill: A functional NIRS study. NeuroImage, 34(1), 109–116.
Hebden, J. C. (2003). Advances in optical imaging of the newborn infant brain.
Hintz, S. R., Benaron, D. A., Siegel, A. M., Zourabian, A., Stevenson, D. K., & Boas, D. A. (2001).
Bedside functional imaging of the premature infant brain during passive motor activation.
Journal of Perinatal Medicine, 29(4), 335–343.
Homae, F., Watanabe, H., Nakano, T., & Taga, G. (2011). Large-scale brain networks underlying
language acquisition in early infancy. Frontiers in Psychology, 2, 93.
Homae, F., Watanabe, H., Otobe, T., Nakano, T., Go, T., Konishi, Y., & Taga, G. (2010).
Development of global cortical networks in early infancy. Journal of Neuroscience, 30(14),
4877–4882.
Hoshi, Y., Kobayashi, N., & Tamura, M. (2001). Interpretation of near-infrared spectros-
copy signals: A study with a newly developed perfused rat brain model. Journal of Applied
Physiology, 90(5), 1657–1662.
Hoshi, Y., & Tamura, M. (1993). Dynamic multichannel near-infrared optical imaging of
human brain activity. Journal of Applied Physiology, 75(4), 1842–1846.
Huppert, T. J., Hoge, R. D., Diamond, S. G., Franceschini, M. A., & Boas, D. A. (2006). A tem-
poral comparison of BOLD, ASL, and NIRS hemodynamic responses to motor stimuli in
adult humans. NeuroImage, 29(2), 368–382.
Imafuku, M., Hakuno, Y., Uchida-Ota, M., Yamamoto, J. I., & Minagawa, Y. (2014). “Mom
called me!” Behavioral and prefrontal responses of infants to self-names spoken by their
mothers. NeuroImage, 103, 476–484.
Kato, T., Kamei, A., Takashima, S., & Ozaki, T. (1993). Human visual cortical function during
photic stimulation monitoring by means of near-infrared spectroscopy. Journal of Cerebral
Blood Flow and Metabolism, 13(3), 516–520.
Katura, T., Tanaka, N., Obata, A., Sato, H., & Maki, A. (2006). Quantitative evaluation of
interrelations between spontaneous low-frequency oscillations in cerebral hemodynamics
and systemic cardiovascular dynamics. NeuroImage, 31(4), 1592–1600.
Kisilevsky, B. S., Hains, S. M., Lee, K., Xie, X., Huang, H., Ye, H. H., Zhang, K., & Wang, Z.
(2003). Effects of experience on fetal voice recognition. Psychological Science, 14(3), 220–224.
Koh, P. H., Glaser, D. E., Flandin, G., Kiebel, S., Butterworth, B., Maki, A., Delpy, D. T., & Elwell,
C. E. (2007). Functional optical signal analysis: A software tool for near-infrared spectros-
copy data processing incorporating statistical parametric mapping. Journal of Biomedical
Optics, 12(6), 064010.
Koizumi, H., Yamamoto, T., Maki, A., Yamashita, Y., Sato, H., Kawaguchi, H., & Ichikawa,
N. (2003). Optical topography: Practical problems and new applications. Applied Optics,
42(16), 3054–3062.
Kotilahti, K., Nissila, I., Huotilainen, M., Makela, R., Gavrielides, N., Noponen, T., Bjorkman,
P., Fellman, V., & Katila, T. (2005). Bilateral hemodynamic responses to auditory stimulation
in newborn infants. Neuroreport, 16(12), 1373–1377.
Kusaka, T., Kawada, K., Okubo, K., Nagano, K., Namba, M., Okada, H., Imai, T., Isobe, K., &
Itoh, S. (2004). Noninvasive optical imaging in the visual cortex in young infants. Human
Brain Mapping, 22(2), 122–132.
Lenneberg, E. H. (1966). Speech development: Its anatomical and physiological concomitants.
Brain Function, 3, 37–66.
Logothetis, N. K., Pauls, J., Augath, M., Trinath, T., & Oeltermann, A. (2001).
Neurophysiological investigation of the basis of the fMRI signal. Nature, 412(6843), 150–157.
May, L., Byers-Heinlein, K., Gervain, J., & Werker, J. F. (2011). Language and the newborn
brain: Does prenatal language experience shape the neonate neural response to speech?
Frontiers in Psychology, 2, 222.
Mayhew, J., Zheng, Y., Hou, Y., Vuksanovic, B., Berwick, J., Askew, S., & Coffey, P. (1999).
Spectroscopic analysis of changes in remitted illumination: The response to increased neural
activity in brain. NeuroImage, 10(3), 304–326.
Meek, J. H., Firbank, M., Elwell, C. E., Atkinson, J., Braddick, O., & Wyatt, J. S. (1998). Regional
hemodynamic responses to visual stimulation in awake infants. Pediatric Research, 43(6),
840–843.
Mehler, J., Jusczyk, P., Lambertz, G., Halsted, N., Bertoncini, J., & Amiel-Tison, C. (1988). A
precursor of language acquisition in young infants. Cognition, 29(2), 143–178.
Minagawa-Kawai, Y., Cristia, A., & Dupoux, E. (2011). Cerebral lateralization and early speech
acquisition: A developmental scenario. Developmental Cognitive Neuroscience, 1(3), 217–232.
Minagawa-Kawai, Y., Cristia, A., Long, B., Vendelin, I., Hakuno, Y., Dutat, M., Filippin, L.,
Cabrol, D., & Dupoux, E. (2013). Insights on NIRS sensitivity from a cross-linguistic study
on the emergence of phonological grammar. Frontiers in Psychology, 4, 170.
Minagawa-Kawai, Y., Cristia, A., Vendelin, I., Cabrol, D., & Dupoux, E. (2011). Assessing
signal-driven mechanisms in neonates: Brain responses to temporally and spectrally dif-
ferent sounds. Frontiers in Psychology, 2, 135.
Minagawa-Kawai, Y., Mori, K., Hebden, J. C., & Dupoux, E. (2008). Optical imaging of
infants’ neurocognitive development: Recent advances and perspectives. Developmental
Neurobiology, 68(6), 712–728.
Minagawa-Kawai, Y., Mori, K., Sato, Y., & Koizumi, T. (2004). Differential cortical responses in
second language learners to different vowel contrasts. Neuroreport, 15(5), 899–903.
Minagawa-Kawai, Y., Naoi, N., Kikuchi, N., Yamamoto, J., Nakamura, K., & Kojima, S. (2009).
Cerebral laterality for phonemic and prosodic cue decoding in children with autism.
Neuroreport, 20(13), 1219–1224.
Miyai, I., Tanabe, H. C., Sase, I., Eda, H., Oda, I., Konishi, I., Tsunazawa, Y., Suzuki, T., Yanagida,
T., & Kubota, K. (2001). Cortical mapping of gait in humans: A near-infrared spectroscopic
topography study. NeuroImage, 14(5), 1186–1192.
Miyai, I., Yagura, H., Hatakenaka, M., Oda, I., Konishi, I., & Kubota, K. (2003). Longitudinal
optical imaging study for locomotor recovery after stroke. Stroke, 34(12), 2866–2870.
Miyata, H., Watanabe, S., & Minagawa-Kawai, Y. (2011). Two successive neurocognitive
processes captured by near-infrared spectroscopy: Prefrontal activation during a computer-
ized plus-shaped maze task. Brain Research, 1374,  90–99.
Nishida, T., Kusaka, T., Isobe, K., Ijichi, S., Okubo, K., Iwase, T., Kawada, K., Namba, M., Imai,
T., & Itoh, S. (2008). Extrauterine environment affects the cortical responses to verbal stimu-
lation in preterm infants. Neuroscience Letters, 443(1),  23–26.
Nissila, I., Hebden, J. C., Jennions, D., Heino, J., Schweiger, M., Kotilahti, K., Noponen, T.,
Gibson, A., Jarvenpaa, S., Lipiainen, L., & Katila, T. (2006). Comparison between a time-
domain and a frequency-domain system for optical tomography. Journal of Biomedical
Optics, 11(6), 064015.
Obrig, H., & Villringer, A. (2003). Beyond the visible: Imaging the human brain with light.
Journal of Cerebral Blood Flow and Metabolism, 23(1),  1–18.
Okada, E., & Delpy, D. T. (2003). Near-infrared light propagation in an adult head model. II.
Effect of superficial tissue thickness on the sensitivity of the near-infrared spectroscopy
signal. Applied Optics, 42(16), 2915–2922.
Okamoto, M., Dan, H., Sakamoto, K., Takeo, K., Shimizu, K., Kohno, S., Oda, I., Isobe, S.,
Suzuki, T., Kohyama, K., & Dan, I. (2004). Three-dimensional probabilistic anatomical
cranio-cerebral correlation via the international 10–20 system oriented for transcranial
functional brain mapping. NeuroImage, 21(1), 99–111.
Peña, M., Maki, A., Kovacic, D., Dehaene-Lambertz, G., Koizumi, H., Bouquet, F., & Mehler,
J. (2003). Sounds and silence: An optical topography study of language recognition at birth.
Proceedings of the National Academy of Sciences USA, 100(20), 11702–11705.
Poeppel, D., Idsardi, W. J., & van Wassenhove, V. (2008). Speech perception at the interface
of neurobiology and linguistics. Philosophical Transactions of the Royal Society of London.
Series B: Biological Sciences, 363(1493), 1071–1086.
Quaresima, V., Bisconti, S., & Ferrari, M. (2012). A brief review on the use of functional near-
infrared spectroscopy (fNIRS) for language imaging studies in human newborns and adults.
Brain and Language, 121(2),  79–89.
Reynolds, E. O., Wyatt, J. S., Azzopardi, D., Delpy, D. T., Cady, E. B., Cope, M., & Wray, S.
(1988). New non-invasive methods for assessing brain oxygenation and haemodynamics.
British Medical Bulletin, 44(4), 1052–1075.
Sakatani, K., Chen, S., Lichty, W., Zuo, H., & Wang, Y. P. (1999). Cerebral blood oxygenation
changes induced by auditory stimulation in newborn infants measured by near infrared
spectroscopy. Early Human Development, 55(3), 229–236.
Sakatani, K., Xie, Y., Lichty, W., Li, S., & Zuo, H. (1998). Language-activated cerebral blood
oxygenation and hemodynamic changes of the left prefrontal cortex in poststroke aphasic
patients: A near-infrared spectroscopy study. Stroke, 29(7), 1299–1304.
Sato, H., Hirabayashi, Y., Tsubokura, H., Kanai, M., Ashida, T., Konishi, I., Uchida-Ota,
M., Konishi, Y., & Maki, A. (2012). Cerebral hemodynamics in newborn infants exposed
to speech sounds: A whole-head optical topography study. Human Brain Mapping, 33(9),
2092–2103.
Sato, H., Tanaka, N., Uchida, M., Hirabayashi, Y., Kanai, M., Ashida, T., Konishi, I., & Maki, A.
(2006). Wavelet analysis for detecting body-movement artifacts in optical topography sig-
nals. NeuroImage, 33(2), 580–587.
Sato, Y., Mori, K., Furuya, I., Hayashi, R., Minagawa-Kawai, Y., & Koizumi, T. (2003).
Developmental changes in cerebral lateralization during speech processing measured by
near infrared spectroscopye. Japan Journal of Logopedics and Phoniatrics, 44(3), 165–171.
Sato, Y., Mori, K., Koizumi, T., Minagawa-Kawai, Y., Tanaka, A., Ozawa, E., Wakaba, Y., &
Mazuka, R. (2011). Functional lateralization of speech processing in adults and children who
stutter. Frontiers in Psychology, 2, 70.
Scholkmann, F., Kleiser, S., Metz, A. J., Zimmermann, R., Mata Pavia, J., Wolf, U., & Wolf, M.
(2014). A review on continuous wave functional near-infrared spectroscopy and imaging in-
strumentation and methodology. NeuroImage, 85(Pt 1), 6–27.
Seiyama, A., Seki, J., Tanabe, H. C., Sase, I., Takatsuki, A., Miyauchi, S., Eda, H., Hayashi,
S., Imaruoka, T., Iwakura, T., & Yanagida, T. (2004). Circulatory basis of fMRI sig-
nals: Relationship between changes in the hemodynamic parameters and BOLD signal in-
tensity. NeuroImage, 21(4), 1204–1214.
Sevy, A. B., Bortfeld, H., Huppert, T. J., Beauchamp, M. S., Tonini, R. E., & Oghalai, J. S. (2010).
Neuroimaging with near-infrared spectroscopy demonstrates speech-evoked activity in
the auditory cortex of deaf children following cochlear implantation. Hearing Research,
270(1–2),  39–47.
Slater, R., Cantarella, A., Franck, L., Meek, J., & Fitzgerald, M. (2008). How well do clinical pain
assessment tools reflect pain in infants? PLoS Medicine, 5(6), e129.
Steinbrink, J., Kempf, F. C., Villringer, A., & Obrig, H. (2005). The fast optical signal: Robust or
elusive when non-invasively measured in the human adult? NeuroImage, 26(4), 996–1008.
Steinbrink, J., Kohl, M., Obrig, H., Curio, G., Syre, F., Thomas, F., Wabnitz, H., Rinneberg, H.,
& Villringer, A. (2000). Somatosensory evoked fast optical intensity changes detected non-
invasively in the adult human head. Neuroscience Letters, 291(2), 105–108.
Steinbrink, J., Villringer, A., Kempf, F., Haux, D., Boden, S., & Obrig, H. (2006). Illuminating
the BOLD signal: Combined fMRI-fNIRS studies. Magnetic Resonance Imaging, 24(4),
495–505.
Strangman, G., Culver, J. P., Thompson, J. H., & Boas, D. A. (2002). A quantitative compar-
ison of simultaneous BOLD fMRI and NIRS recordings during functional brain activation.
NeuroImage, 17(2), 719–731.
Sugiura, L., Ojima, S., Matsuba-Kurita, H., Dan, I., Tsuzuki, D., Katura, T., & Hagiwara, H.
(2011). Sound to language: Different cortical processing for first and second languages in
elementary school children as revealed by a large-scale study using fNIRS. Cerebral Cortex,
21(10), 2374–2393.
Taga, G., Asakawa, K., Maki, A., Konishi, Y., & Koizumi, H. (2003). Brain imaging in awake
infants by near-infrared optical topography. Proceedings of the National Academy of Sciences
USA, 100(19), 10722–10727.
Taga, G., Watanabe, H., & Homae, F. (2011). Spatiotemporal properties of cortical
haemodynamic response to auditory stimuli in sleeping infants revealed by multi-channel
near-infrared spectroscopy. Philosophical Transactions A: Mathematical, Physical and
Engineering Sciences, 369(1955), 4495–4511.
Takeda, K., Gunji, Y., Watanabe, M., & Kato, H. (2008). Effect of neck tilting on NIRS data: An
investigation of false positive activation. Poster session presented at the 31st annual meeting of
the Japan Neuroscience Society.
Telkemeyer, S., Rossi, S., Koch, S. P., Nierhaus, T., Steinbrink, J., Poeppel, D., Obrig, H., &
Wartenburger, I. (2009). Sensitivity of newborn auditory cortex to the temporal structure of
sounds. Journal of Neuroscience, 29(47), 14726–14733.
Torricelli, A., Contini, D., Pifferi, A., Caffini, M., Re, R., Zucchelli, L., & Spinelli, L. (2014). Time
domain functional NIRS imaging for human brain mapping. NeuroImage, 85(1),  28–50.
Tsuji, S., Fikkert, P., Minagawa, Y., Dupoux, E., Filippin, L., Versteegh, M., Hagoort, P., &
Cristia, A. (2017). The more, the better? Behavioral and neural correlates of frequent and in-
frequent vowel exposure. Developmental Psychobiology, 59(5), 603–612.
Tsuzuki, D., & Dan, I. (2014). Spatial registration for functional near-infrared spectros-
copy: From channel position on the scalp to cortical location in individual and group
analyses. NeuroImage, 85(1), 92–103.
Tsuzuki, D., Jurcak, V., Singh, A. K., Okamoto, M., Watanabe, E., & Dan, I. (2007). Virtual spa-
tial registration of stand-alone fNIRS data to MNI space. NeuroImage, 34(4), 1506–1518.
Uchida-Ota, M., Arimitsu, T., Yatabe, K., Ikeda, K., Takahashi, T., & Minagawa, Y. (2014).
Persistent functional connectivity between frontal and temporal cortices while neonates
hear their mothers’ voice. Faculty of Integrated Arts and Social Sciences Journal, 25, 93–101.
Villringer, A., Planck, J., Hock, C., Schleinkofer, L., & Dirnagl, U. (1993). Near infrared spec-
troscopy (NIRS): A new tool to study hemodynamic changes during activation of brain
function in human adults. Neuroscience Letters, 154, 101–104.
Villringer, K., Minoshima, S., Hock, C., Obrig, H., Ziegler, S., Dirnagl, U., Schwaiger, M., &
Villringer, A. (1997). Assessment of local brain activation: A simultaneous PET and near-
infrared spectroscopy study. Advances in Experimental Medicine and Biology, 413, 149–153.
Watanabe, E., Maki, A., Kawaguchi, F., Takashiro, K., Yamashita, Y., Koizumi, H., & Mayanagi, Y.
(1998). Non-invasive assessment of language dominance with near-infrared spectroscopic
mapping. Neuroscience Letters, 256(1),  49–52.
Watanabe, E., Yamashita, Y., Maki, A., Ito, Y., & Koizumi, H. (1996). Non-invasive func-
tional mapping with multi-channel near infra-red spectroscopic topography in humans.
Neuroscience Letters, 205(1),  41–44.
White, B. R., Snyder, A. Z., Cohen, A. L., Petersen, S. E., Raichle, M. E., Schlaggar, B. L., &
Culver, J. P. (2009). Resting-state functional connectivity in the human brain revealed with
diffuse optical tomography. NeuroImage, 47(1), 148–156.
Wobst, P., Wenzel, R., Kohl, M., Obrig, H., & Villringer, A. (2001). Linear aspects of changes in
deoxygenated hemoglobin concentration and cytochrome oxidase oxidation during brain
activation. NeuroImage, 13(3), 520–530.
Yamamoto, T., Maki, A., Kadoya, T., Tanikawa, Y., Yamad, Y., Okada, E., & Koizumi, H. (2002).
Arranging optical fibres for the spatial resolution improvement of topographical images.
Physics in Medicine and Biology, 47(18), 3429–3440.
Ye, J. C., Tak, S., Jang, K. E., Jung, J., & Jang, J. (2009). NIRS-SPM: Statistical parametric map-
ping for near-infrared spectroscopy. NeuroImage, 44(2), 428–447.
Zarahn, E., Aguirre, G., & D’Esposito, M. (1997). A trial-based experimental design for fMRI.
NeuroImage, 6(2), 122–138.
Zatorre, R. J., & Belin, P. (2001). Spectral and temporal processing in human auditory cortex.
Zatorre, R. J., & Gandour, J. T. (2008). Neural specializations for speech and pitch: Moving
beyond the dichotomies. Philosophical Transactions of the Royal Society of London. Series
B: Biological Sciences, 363(1493), 1087–1104.
Zeff, B. W., White, B. R., Dehghani, H., Schlaggar, B. L., & Culver, J. P. (2007). Retinotopic map-
ping of adult human visual cortex with high-density diffuse optical tomography. Proceedings
of the National Academy of Sciences USA, 104(29), 12169–12174.
Zhang, Y., Brooks, D. H., Franceschini, M. A., & Boas, D. A. (2005). Eigenvector-based spa-
tial filtering for reduction of physiological interference in diffuse optical imaging. Journal of
Biomedical Optics, 10(1), 11014.
Chapter 8
What Has Dire c t C ort i c a l

and Su b c ort i c a l
Electrosti mu l at i on
Tau ght Us a b ou t
Neuroling u i st i c s ?
Hugues Duffau
Introduction
Investigating the neural basis of language is one of the most important challenges in
neuroscience. The development of functional neuroimaging has allowed a noninvasive
study of the organization of critical brain structures. Nonetheless, this technique still
lacks reliability at the individual level, especially concerning language mapping in brain
tumor patients, due to neurovascular uncoupling (Agarwal et al., 2016).
As a consequence, invasive electrophysiological methods are currently considered
the “gold standard” for brain mapping. Extraoperative mapping using a subdural
grid can also be achieved. Although this technique has been used extensively in ep-
ilepsy surgery because it also enables detection of seizure foci, only the cortex can
be mapped: it provides no information about subcortical connectivity. Thus, in the
past decade, a growing number of authors have advocated the use of direct elec-
trical stimulation (DES) intraoperatively, especially in neuro-oncology, since glioma
invades both cortical and subcortical structures (Almairac, Herbet, Moritz-Gasser,
de Champfleur, & Duffau, 2015; Mandonnet, Capelle, & Duffau, 2006). Indeed,
during brain surgery, it has become common clinical practice to awaken patients
in order to assess the functional role of restricted cerebral regions. The surgeon can
maximize the extent of resection, and thereby improve the overall survival, without
Direct Cortical and Subcortical Electrostimulation 187
eliciting permanent neurological impairment, owing to an individual mapping and

preservation of critical structures. This means that the resection is achieved according
to functional boundaries (Duffau, Gatignol, Mandonnet, Capelle, & Taillandier, 2008;
Duffau, 2012a).
In practice, patients perform several sensorimotor, visuospatial, language, cogni-
tive, or even emotional tasks while the surgeon temporarily interacts with discrete
areas within the gray and white matters around the tumor, using DES (Figure 8.1).
If the patient stops moving or speaking, or produces the wrong response, the sur-
geon avoids removing the stimulated site (Duffau, 2011a). Indeed, DES transiently
Figure 8.1. Left: Preoperative sagittal T1-weighted MRI showing a low-grade glioma involving

the whole left dominant temporal lobe in a right-handed patient who experienced seizures, with
a normal neurological examination.
Middle: Intraoperative photograph of the left hemisphere after surgical resection under local
anesthesia, with the use of DES to map language both at cortical and subcortical levels. The
number tags correspond to the critical areas detected by DES mapping, as follows: (1) ventral
premotor cortex, eliciting anarthria when stimulated; (39) posterior portion of the inferior
longitudinal fascicle, eliciting alexia when stimulated; (45) anterior part of the vertical (tem-
poral) portion of the arcuate fascicle, eliciting phonemic paraphasia and repetition disorders
during stimulation; (27) horizontal (temporal) part of the inferior fronto-occipital fascicle,
eliciting semantic paraphasia during DES. All these eloquent structures have been preserved,
due to representation of the functional boundaries of the resection.
Straight arrow: Labbe’s vein; curved arrow: Sylvian fissure; A = anterior; P = posterior.
Right: Postoperative sagittal T1-weighted MRI showing an extensive resection of the left dom-
inant temporal lobe, including the “Wernicke’s area” (posterior part of the superior temporal
gyrus beyond the Labbe’s vein). Due to neuroplasticity mechanisms, and thanks to the preserva-
tion of the subcortical language connectivity, the neurological examination was normal after sur-
gery, in particular with no language disturbances. The patient resumed a normal familial, social,
and professional life.
188 Hugues Duffau
interacts locally with a small cortical or axonal site, but also nonlocally, as the focal
perturbation will indeed disrupt the entire subnetwork sustaining a given function
(Mandonnet, Winkler, & Duffau, 2010). Thus, in contrast to functional neuroim-
aging, DES is able to detect the structures essential for brain functions, by inducing a
transient virtual lesion based on the inhibition of a subcircuit during approximately
4 seconds—with the possibility to check whether the same functional disorders are
reproduced when repeated stimulations are applied over the same area. Interestingly,
by collating all cortical and axonal sites where the same type of errors are observed
when stimulated, one can assemble the subnetwork of the disrupted subfunction.
Consequently, DES represents a unique opportunity to identify with great accuracy
(about 5 mm) and reproducibility, in vivo in humans, the structures that are crucial for
cognitive functions, both at cortical and subcortical (white matter and deep gray nu-
clei) levels (Duffau, 2015). Combining transient disturbances elicited by DES with the
anatomical data provided by pre-and postoperative MRI enables reliable anatomo-
functional correlations, supporting a network organization of the brain (Duffau,
2014b), and leading to the reappraisal of cognitive models—notably those regarding
language representation (Duffau, Moritz-Gasser, & Mandonnet, 2014).
In this chapter, the goal is to critically review the basic principles of DES, its advantages
and limitations, and what DES can tell us about neural foundations of language, that is,
the large-scale distribution of language areas in the brain, their connectivity, and their
ability to reorganize—the so-called neuroplasticity.
Basic Aspects of DES
Principles of DES
The membrane potential (MP) of the neuron at rest varies between −60 and −100
millivolts (mV). The principle of electrical stimulation is to generate membrane ex-
citability via an initial phase of passive modification of the local MP at the level of the
cathode (i.e., the negative electrode). Here the inner side of the membrane becomes
progressively less negative than the outer side (the membrane becomes inversely
hyperpolarized with respect to the anode). The intensity of this phenomenon depends
on the parameters of the stimulations and the characteristics of the membrane (Jayakar,
1993). The outer membrane can be more easily stimulated at the level of the initial seg-
ment of the axon, and at the level of the fibers that are myelinized and have a larger di-
ameter (Ranck, 1981). If the MP reaches the liminar depolarization (i.e., threshold), a
second phase occurs that begins with the opening of voltage-dependent ionic canals,
which allows entry of Na+ ions, and therefore inverts the MP between +20 mV and +30
mV. A secondary output of K+ ions, associated with an inhibition of the entering flux of
Na+ ions, brings the MP back to its resting state. Once generated, this rapid sequence
of MP fluctuation—the action potential—is still the same, whatever the stimulation

parameters are (law of “all or nothing”).
Stimulation Parameters: Theoretical Approach

The first requirement for DES is that it must be entirely safe for the cerebral parenchyma.
Nonetheless, it may generate a lesion by accumulating negative charge at the level of
the cathode, or by producing metal ions at the level of the anode (Agnew & McCreery,
1987). The use of biphasic impulses eliminates these risks because the second stimulus
phase inverts the effects of the first. Tissue damage can also be induced, on the one hand,
by excessive heat, produced specifically by hydrolysis, which could cause vacuolization
and chromatolysis; or, on the other hand, by “leakage” of the intracellular current, which
goes from the anode to the cathode through the cytoplasm, posing a risk of lesions to
the mitochondria and endoplasmic reticulum; or even by alteration of the homeostasis
when the neurons are activated in a manner that is too repetitive and synchronous
(Yeomans, 1990). These risks are directly linked to the density of the charge, and studies
based on animal experiments have led to an upper threshold limit of 55 microcoulombs/
cm2/phase, at which no lesions were observed (Gordon et al., 1990). In theory, DES can
also generate seizures, even though the rate of intraoperative epilepsy is basically nil
in teams with experience (Deras et al., 2012). Nonetheless, in case of seizures, irriga-
tion of the cortex with cold serum allows an interruption of the crisis in a few seconds
(Sartorius & Berger, 1998).
The second requirement for DES is that it should induce a reproducible response
when applied to neural structures. The relationship between stimulation parameters
and tissue-response characteristics can be summed up by the intensity/duration curve,
in which the intensity of the current is recorded as a function of the duration of the im-
pulse (Jayakar, Alvarez, Duchowny, & Resnick, 1992). The rheobasis is thus defined as
the minimal intensity required to generate an action potential at elevated values of im-
pulse duration. Chronaxie is the impulse duration required to elicit a response when the
stimulus intensity is twice the rheobasis. The chronaxie corresponds to the point on the
intensity/duration curve where the energy allowing an action potential to be generated
is minimal, for a charge twice the minimum charge: this is the optimal stimulation point
(best benefit/risk ratio).
The chronaxie point depends on the characteristics of the tissue being stimulated,
specifically on its impedance. The few studies on this topic (Geddes & Baker, 1967) have
produced resistance values of 250 Ohms for gray matter, 500 Ohms for white matter,
and 65 Ohms for cerebrospinal fluid (Nathan, Lesser, & Gordon, 1993). Moreover,
chronaxie can be significantly modified by the degree of cerebral maturation, notably
by myelinization. In this way, chronaxie of non-myelinized nerve fibers is consider-
ably longer (0.4–3.5 ms) than that of the myelinized axons (0.05–0.4 ms) (Jayakar et al.,
1992). The size of the fiber also seems to be a factor, in that the axons of greater diameter
190 Hugues Duffau
are more readily excited (Ranck, 1981). Moreover, impedances can be modified by the
state of the patient (awake or under general anesthesia). Finally, any pathological pro-
cess, whether lesional (tumor) or non-lesional (epilepsy, postictal status), can interfere
directly with the tissue’s excitability (Jayakar, 1993).
As regards the frequency of electrical impulses, myelinized axons produce a single
response for each stimulation delivered between 50 and 100 Hz (Jayakar, 1993).
A neural membrane is, in fact, refractory to all stimuli lasting 0.6 milliseconds (ms) to
2 ms following an action potential. This status precedes a second phase of transitory
hyperexcitability, during which the tissue could be stimulated by less intense currents
than during the initial stimulus, but the risk of seizure is increased.
Moreover, when the neural structures are kept in a state of infra-liminar depolari-
zation, the threshold required to generate the impulse increases; this phenomenon is
known as accommodation. Accommodation occurs if the MP changes quite progres-
sively, which may be observed when sinusoidal impulses are used. This is the reason why
rectangular impulses are recommended for stimulations.
Although two electrodes are still required to produce a current, the stimulations are
considered to be “monopolar” if only one of the electrodes is “active” (in general, the
cathode), that is, localized in relation to the target tissue, while the reference electrode
(in general, the anode) is located at a distance. Whereas the current density is distributed
in a relatively uniform manner around the electrode, each tissue located in the current’s
pathway can nevertheless be stimulated, especially if its depolarization threshold is less
than the target threshold. To reduce this risk of a false positive, it is preferable to use
a bipolar stimulation, that is, where the cathode and the anode are both “active,” or in
other words, both are located at the level of the target tissue (Nathan et al., 1993). Only
structures located between the two electrodes are stimulated in this way, so there is less
risk of diffusion and therefore a greater precision (Haglund, Ojemann, & Blasdel, 1993).
Nevertheless, because current distribution is more complex with bipolar stimulation
than with monopolar stimulation, it is more difficult to build a model in order to choose
optimal parameters for achieving the most reproducible responses (Nathan et al., 1993),
especially in view of the overlapping of effects at the cathode (depolarization state of
the level of peri-cathodic tissues) and the anode (hyperpolarization state at the level of
peri-anodic tissues). It is recommended to use biphasic impulses to compensate for this
limitation (Jayakar, 1993).
Finally, a recent computational study has suggested that directing the bipolar
electrodes orthogonal (and not parallel) to the axis of a white matter fascicle should fa-
cilitate its identification, as the chances are higher in this configuration that at least one
of the electrode tips will be in contact with the tract (Mandonnet & Pantz, 2011).
Stimulation Parameters: Practical Approach

In practice, bipolar electrode tips that are spaced 5 millimeters (mm) apart and that de-
liver a biphasic current (pulse frequency 60 Hz, single-pulse phase duration 1 ms, and
intensity from 6 to 18 mA under general anesthesia or from 2 to 6 mA under local anes-

thesia) are applied to the brain (Duffau, 2004, 2007). DES allows the mapping of motor
function (by inducing involuntary motor response if the stimulation is applied at the
level of a motor site, or by eliciting arrest of movement when stimulating the negative
motor network), somatosensory function (by eliciting dysesthesia described by the pa-
tient himself intraoperatively), and cognitive functions such as language (spontaneous
speech, counting, picture naming, comprehension, reading, writing, bilingualism,
etc.), calculation, memory, visuospatial processing, judgment, or even mentalizing
(Fernandez-Coello et al., 2013). Patients are awake, and transient disturbances are
generated by applying DES at the level of a functional “epicenter” (Ojemann, Ojemann,
Lettich, & Berger, 1989). A speech therapist or neuropsychologist must be present in
the operating room, in order to accurately interpret the kind of disorders induced by
DES, for instance, speech arrest, anarthria, speech apraxia, phonological disturbances,
semantic paraphasia, perseveration, anomia, dyscalculia, alexia, and so on. Thus, DES is
able to identify in real time the cortical sites essential for the function before beginning
the resection, in order to both select the best surgical approach and define the cortical
limits of the resection (Duffau, 2005a, 2011a) (Figure 8.1).
In addition, DES also allows the identification and preservation of subcortical
pathways crucial for sensorimotor, visuospatial, language, and other cognitive functions
(Duffau et al., 2002; Duffau et al., 2003a; Duffau, 2015; Thiebaut de Schotten et al., 2005).
Indeed, it allows the study of the anatomo-functional connectivity by directly and reg-
ularly stimulating the white matter tracts throughout the resection, and by eliciting a
functional response when in contact with the essential eloquent fibers, according to the
same principle as that described at the cortical level. Thus, it is important that the pa-
tient remains awake during the entire removal of the tumor, and not only for the initial
stage of cortical stimulation. This is why intraoperative cortico-subcortical stimulation
is time-consuming, and why the number of tasks during surgery are limited by the pro-
gressive tiredness of the patient.
Strengths of DES: A Door

into Neural Networks
One of the major advantages of DES is that it intrinsically does not cause any false
negatives. Indeed, each critical structure, whatever its actual role in brain function,
will be electrically disturbed by DES, which necessarily will induce a functional con-
sequence. This optimal sensitivity explains why DES is currently considered the gold
standard in brain mapping. First, intraoperative DES enables neurosurgeons to tailor
the resection according to functional boundaries, thus minimizing the risk of perma-
nent deficit while preserving the patient’s quality of life (Duffau, 2012a). Second, DES
is used to validate the noninvasive methods of functional neuroimaging (functional
192 Hugues Duffau
magnetic resonance imaging [fMRI] and magnetoencephalography [MEG]) (Giussani

et al., 2010; Korvenoja et al., 2006; Roux et al., 2003) as well as diffusion tensor imaging
(DTI) (Kinoshita et al., 2005; Leclerq et al., 2010). Indeed, the reliability of fMRI has
been estimated at only 60%–70%, both in healthy subjects (Havel et al., 2006) and in
patients with cerebral tumors (Giussani et al., 2010; Roux et al., 2003). Recently, by
comparing preoperative language (fMRI; see Heim & Specht, Chapter 4 in this volume)
with intraoperative electrical stimulation mapping, Kuchsinski et al. (2015) calculated
the sensitivity of fMRI at 37.1% and the specificity at 83.4%. In addition, functional neu-
roimaging is not able to differentiate those areas essential for function from those that
can be compensated while activated during a task. For example, glioma involving the
supplementary motor area can be surgically excised without generating any perma-
nent neurological deficit due to the recruitment of the contra-lateral homologous re-
gion (Krainik et al., 2004), even though this supplementary motor area was detected on
preoperative fMRI in patients performing language tasks (Krainik et al., 2003). Finally,
functional neuroimaging is able to map the gray matter but not the white matter tracts.
The recent development of DTI, enabling a tractography of white matter tracts in vivo,
has provided new insights into cerebral connectivity (Catani, Jones, & Ffytche, 2005; see
Catani & Forkel, Chapter 9 in this volume). However, it is noteworthy that recent review
articles by DTI experts have underlined the need for a rigorous anatomical validation
(Hubbard & Parker, 2009; Jbabdi & Johansen-Berg, 2011; Le Bihan, Poupon, Amadon,
& Lethimonnier, 2006). Indeed, DTI does not directly reflect cerebral functional reality,
but provides a very indirect approximation based on biomathematic reconstructions—
explaining why results may vary depending on the model used to analyze the MRI data
(Duffau, 2014a; Kinoshita et al., 2005; Pujol et al., 2015). Correlation studies between
tractography and intraoperative direct subcortical electrostimulation have shown con-
cordance in only 82% of cases (Leclerq et al., 2010). Moreover, DTI provides anatom-
ical but not functional information about the white matter pathways. As a consequence,
DES is a unique tool to better understand the organization of networks underpinning
brain processing (Duffau, 2014)—especially language (see later discussion).
With regard to the spatial resolution, although it has been argued that the currents
delivered by the tips of the bipolar electrodes can diffuse over the cortex, thus limiting
the spatial accuracy of the tested areas, it is necessary to distinguish two types of diffu-
sion. The first one involves the spread of the depolarizing electric field over the cortical
surface. It was demonstrated using optical imaging that the spatial area affected by this
spreading diffusion is about 5 mm in diameter (Haglund et al., 1993). The diffusion has
not been investigated experimentally for axonal stimulations, but theoretically, it should
be of the same order, or even smaller when the bipolar electrode is parallel to the direc-
tion of the fascicle (Mandonnet & Pantz, 2011). Thus, the spatial resolution of DES is
about 5 mm, namely less than the 1 centimeter (cm) spacing of subdural strip and grid
electrodes used for recording epileptiform activity and for cortical mapping before epi-
lepsy surgery.
The second type of diffusion refers to the biological and physiological propagation of
the stimulation. Indeed, when stimulating the cortical or the axonal part of the neuron,
the action potential is further transmitted by the physiological conduction along the
axon, and consequently, neurons at the end of the axons can be stimulated by these pre-
synaptic currents. So, in essence, DES is highly nonlocal: it interacts with the entire net-
work that sustains a function. The stimulated point (axonal or cortical) is only an input
gate to this entire network (Mandonnet et al., 2010).
Since DES allows the performance of online, anatomo-functional correlations at
both cortical and subcortical levels, and since each eloquent discrete site identified is
only an input gate to a wider functional network, DES is consequently a perfect tool
for the study of spatiotemporal brain connectivity. For a long time, however, DES was
considered a method that could only detect cortical “epicenters,” and thus useful for
mapping only gray matter (Ojemann et al., 1989). In fact, intraoperative DES offers the
unique opportunity to directly access not only the functional cortical organization, but
also the effective connectivity—that is, the influence that a cerebral region may have
on another region (Lee, Friston, & Horwitz, 2006). Indeed, it is now well known that to
generate a function, it is necessary that several areas work together. Specific temporal
dynamics allow synchronization within the distributed networks (Bartels & Zeki, 2005;
McClelland & Rogers, 2003). This explains why lesions involving the white matter very
frequently induce severe and permanent deficits, as demonstrated in stroke studies
(Catani & Ffytche, 2005). It is thus likely that subcortical DES induces the same effect
as a transient virtual lesion, which disrupts the electrical connectivity between brain re-
gions that normally communicate to generate the function. This hypothesis is supported
by the fact that the DES of main pathways has the same functional consequence as a
disconnection syndrome, for instance, conduction aphasia during stimulation of the
left arcuate fascicle (Duffau et al., 2002; Maldonado, Moritz-Gasser, & Duffau, 2011), or
semantic disturbances during stimulation of the left inferior fronto-occipital fascicle
(Duffau et al., 2005; Moritz-Gasser, Herbet, & Duffau, 2013). Furthermore, beyond such
an inhibition elicited within long-range cortico-cortical networks, DES may also gen-
erate a virtual disconnection within the cortico-subcortical loops, as demonstrated by
the induction of transcortical motor aphasia during stimulation of the left dominant
frontostriatal tract—which connects the supplementary motor area and the cingulum
with the head of the caudate nucleus (Duffau et al., 2002; Kinoshita et al., 2015). It is
worth noting that the possibility of obtaining these very precise anatomo-functional
correlations supports the fact that DES propagates in a “subcircuit” rather than in the
entire language network, since stimulation can inhibit only a specific component of lan-
guage (e.g., phonology, semantics, or syntax). Finally, this unifying vision of network
stimulation can contribute to understanding why conduction aphasia can be observed
for subcortical (Duffau et al., 2002) as well as cortical stimulation (Quigg & Fountain,
1999; Quigg, Geldmacher, & Elias, 2006): the same network is disrupted, with an input
either subcortical or cortical.
This view also agrees with works using implanted grids for pre-surgical planning
of medication-resistant epileptic patients. It has been shown that the stimulation of a
cortical site induces a signal in a distant (but axonally linked) area (i.e., the so-called
cortico-cortical evoked potential; Yamao et al., 2014). This concept is also very similar to
194 Hugues Duffau
deep brain stimulation of the basal ganglia. Indeed, it is now widely accepted that stim-
ulation of the subthalamic nucleus also interacts with the entire network of the basal
ganglia (McIntyre, Savasta, Walter, & Vitek, 2004; Montgomery, 2004), including the
primary motor cortex and its feedback loops. From a practical point of view, this under-
standing suggested new targets in neuromodulation for Parkinson’s disease, such as the
primary motor area (Cilia et al., 2007; Pagni et al., 2005). Theoretically, these networks
are modeled as interacting nonlinear dynamic systems, and pathological states can be
associated with abnormal locking in a strongly synchronized state, leading to proposals
for new schemes of stimulation (Popovych, Hauptmann, & Tass, 2006; Rosenblum &
Pikovsky, 2004). Such bio-mathematical modeling may also explain how DES actually
induces functional disturbance.
On the other hand, diffusion within a large-scale network may also represent a limita-
tion of DES, as discussed in the next section.
Weaknesses of DES
False Negatives
It is important to stress that the slightest technical mishap can result in false negatives.
Indeed, it was previously explained that an intensity of stimulation that is too low, that
is of too short duration, or that is performed during a transient post-epileptic refrac-
tory phase, may lead to an erroneous “negative mapping” (Taylor & Bernstein, 1999).
Nevertheless, such failure can be avoided by strictly following the theoretical and prac-
tical rules of stimulation.
A subtler limitation would be an inappropriate functional task selection. For in-
stance, if intraoperative testing is based only on a visual-naming task, word-finding
difficulties can arise after resection of the posterior superior temporal gyrus, due to
the anatomic dissociation of visual and auditory naming (Hamberger, McClelland,
McKhann, Williams, & Goodman, 2007). Repetition tasks should be added for lesions
involving this same region (Quigg & Fountain, 1999; Quigg et al., 2006). In addition,
patients who undergo operations for a tumor within the left dominant hemisphere may
exhibit postoperative working memory deficits, despite extensive intraoperative lan-
guage mapping (du Boisgueheneuc et al., 2006; Teixidor et al., 2007). This is a result
of the nonspecific engagement of working memory by tasks adopted during surgery.
Thus, the erroneous conclusion is drawn that the tissue is “not functional”—which may
have been true for language only. Specific testing of spatial awareness is also manda-
tory during surgery within the right “non-dominant” hemisphere in order to avoid
any postoperative hemineglect (Thiebaut de Schotten et al., 2005). Thus, the limited
number of feasible intraoperative tasks can limit function detection. Thus it is impor-
tant to optimize the selection of the intrasurgical tests for each patient on the basis of the
preoperative functional assessment from both extensive neurological and neuropsycho-

logical examinations (Fernandez-Coello et al., 2013).
False Positives
Although the sensitivity of intraoperative electrical mapping is very high, its specificity
remains a matter of debate. First, the patient’s tiredness after approximately one to two
hours of continuous functional assessment during an awake procedure may induce a de-
cline of accuracy and rapidity of response. It could become difficult to differentiate the
immediate proximity of the eloquent structures (transient deficit during DES) from the
tiredness of the patient. Second, DES may induce partial seizures that can look like a “pos-
itive effect” of the stimulation. For example, a partial seizure elicited by DES may gen-
erate a transient language disorder after stimulation within the dominant hemisphere,
creating the wrong impression that this area is crucial for the function. A rigorous meth-
odology of stimulation, as detailed in the preceding, will avoid such false positives. Above
all, the reproducibility of the symptoms elicited by DES during initial cortical mapping
needs to be confirmed by subcortical mapping. The coherency of the two maps will en-
sure that the organization of the network (and not only isolated epicenters) has been
correctly understood. Third, while there is no cortical spreading of DES, there is a propa-
gation of the stimulation along the axon within a network wider than the sole area tested.
Although this property allows the study of connectivity with a very high sensitivity, it
can also lead to false positives. Indeed, it has been reported that when electrical mapping
was performed using subdural grids for preoperative planning in chronic epilepsy, DES
of a cortical site elicited a signal in a remote but axonally linked region (Ishitobi et al.,
2000). Such mechanisms, especially if the stimulation generates a backward propagation,
might explain observations that resections of “positive” areas did not cause postoperative
deficits—such as the left basal temporal area (Lüders et al., 1991).
What DES Can Tell Us about

the Neural Basis of Language:
Rethinking Models of Picture Naming
Bilateral Probabilistic Map of Eloquent Cortical Areas

Detected by DES: Broca’s Area Revisited
One of the best examples of a language circuit that should be well understood is the
network underlying picture naming. Indeed, DES has allowed a re-examination of
the classical Broca-Wernicke’s model, which took precedence over all others for many
196 Hugues Duffau
Motor
Sensory
Dysarthria
Anarthria/Arrest
Anomia
Semantic
Phonemic
Syntax
Spatial Cognition
Figure 8.2. Probabilistic map of critical functional regions of the human cortex in the left and
right hemispheres, issued from DES in awake patients performing sensorimotor, language, and
line bisection tasks.
Source: Modified from Tate et al. (2014).
decades. A recent study performed in 165 consecutive patients who underwent awake
surgery with the use of intraoperative DES provided the first bilateral probabilistic
map for essential cortical functions in the left and right hemispheres of humans (Tate,
Herbet, Moritz-Gasser, Tate, & Duffau, 2014). Stimulation sites eliciting positive (sen-
sory/motor) or negative (speech arrest, dysarthria, anomia, phonological or semantic
paraphasias, syntactic disorders, hemineglect) findings in patients who performed
counting, picture naming, and line bisection tasks were recorded and mapped onto a
standard Montreal Neurologic Institute (MNI) brain atlas. Compilation of all stimu-
lation data (n = 771 stimulation sites) demonstrated the wide bilateral distribution of
cortical representation within and between critical functions (Figure 8.2). In partic-
ular, it was demonstrated that speech arrest during DES was localized to ventral pre-
motor cortex, not the classical Broca’s area (Tate, Herbet, Moritz-Gasser, Tate, & Duffau,
2015). In addition, anomia/paraphasia data demonstrated foci not only within classical
Wernicke’s area, but also within the middle and inferior frontal gyri. Therefore, these
data challenge classical theories of brain organization (e.g., Broca’s area as speech output
region) and provide a distributed framework for future studies of language networks
(Tate et al., 2014).
The Network Underpinning Picture Naming:

Lessons from DES
According to the revisited model of picture naming issued for DES (Duffau et al.,
2014) (Figure 8.3), in the naming process, the first step is visual perception and recog-
nition. DES of the optic pathways may elicit reversible phosphenes or visual loss in the
contralateral visual field (described by the awake patients) due to an inhibition of visual
Caudate nucleus Supplementary Primary motor Speech output and
motor area cortex movement
Control
executive functions
Superior Parietal
Lobule
Dorsolateral Articulatory loop
prefrontal Ventral premotor
working memory
cortex cortex/anterior insula
Supremarginalis
Middle frontal Gyrus
Inferior frontal Dorsal phonological stream
Gyrus
Gyrus (pars Postero-superior Angular
Orbito-frontal opercularis and temporal area Gyrus
area triangularis)
Postero-middle Superior and

temporal area middle
Occipital Gyri
Inferior
Ventral Stream
occipital
verbal and nonverbal semantics
Postero-inferior Gyrus
Temporal pole temporal cortex/Fusa Visual
recognition
Superfical layer of the IFOF Arcuate fascicle (deep part of the SLF)
Deep layer of IFOF Lateral portion of SLF (anterior segment)
Inferior longitudinal fascicle Lateral portion of SLF (posterior segment) Visual input
Uncinate fascicle Frontal aslant tract (optic radiations)
Middle longitudinal fascicle Fronto-striatal tract (subcallosal fascicle)
U-fibers
Figure 8.3. Proposal of a new model of connectivity underlying language processing and its relationships with cognitive functions, with incorpo-
ration of anatomic subcortical constraints, elaborated on the basis of structural-functional correlations provided by intraoperative DES combined
with perioperative neuroimaging.
Source: Modified from Duffau et al. (2014).
198 Hugues Duffau
perception (Gras-Combes, Moritz-Gasser, Herbet, & Duffau, 2012). Visual formal par-
aphasia has also been evoked by electrical interferences with a second stage of visual
processing (i.e., visual recognition). These disorders have been generated by axonal
DES of a subpart of the posterior inferior longitudinal fascicle (ILF), which links the
visual cortex with the “visual object form area” (Mandonnet, Gatignol, & Duffau, 2009;
Zemmoura, Herbet, Moritz-Gasser, & Duffau, 2015). This area, participating in object
recognition, is close to the visual word form area, which receives another subpart of the
ILF as afference, a subpathway involved in reading—thus generating alexia when dam-
aged (Gaillard et al., 2006; Zemmoura et al., 2015). Furthermore, by performing sub-
cortical DES of distinct subcomponents within the left occipito-temporal white matter,
a double dissociation between alexia (lower fibers) and anomia (upper fibers) has been
generated in the same patients (Chan-Seng, Moritz-Gasser, & Duffau, 2014; Gil Robles
et al., 2013). Thus, these data support the existence of parallel pathways coming from the
occipital cortex, specifically involved in word versus object recognition.
Following this first step of visual recognition, a dual model for visual language pro-
cessing was proposed on the basis of DES findings, with a ventral stream involved in
mapping visual information to meaning (the “what” pathway) and a dorsal stream
dedicated to mapping visual information to articulation through visuo-phonological
conversion—completing the model proposed by Hickok and Poeppel (2004; see Hickok,
Chapter 20 in this volume), which nonetheless did not take into account anatomic
constraints, especially with regard to white matter tracts. Because double dissociations
between phonemic and semantic errors have been demonstrated by DES (Maldonado
et al., 2011), both processes seem to be performed in parallel, and not serially. Indeed,
concerning the ventral semantic stream, cortically, semantic paraphasias have been in-
duced by DES along the posterior part of the superior temporal sulcus, in the dorsolat-
eral prefrontal cortex, and in the pars orbitaris of the inferior frontal gyrus (Duffau et al.,
2005; Tate et al., 2014). Axonally, such disturbances were generated by stimulation of
the left inferior fronto-occipital fascicle (IFOF), a pathway that connects the posterior
occipital lobe and visual object form area to anterior cortical frontal areas, including the
inferior frontal gyrus and dorsolateral prefrontal cortex (Martino, Brogna, Gil Robles,
Vergani, & Duffau, 2010; Sarubbo, De Benedictis, Maldonado, Basso, & Duffau, 2013)—
known to be involved in language semantics and higher cognitive functions as multi-
modal integration or judgment (Duffau et al., 2005; Moritz-Gasser et al., 2013; Plaza,
Gatignol, Cohen, Berger, & Duffau, 2008). Therefore, pretreated information by the
visual recognition system is subsequently processed by the semantic system (in parallel
to the dorsal phonological stream, see later discussion) before being processed by the
executive system. In addition to this direct ventral route subserved by the IFOF, there is
an indirect ventral semantic pathway with a relay at the level of the temporal pole, which
represents a semantic “hub” allowing integration of the multimodal data coming from
the unimodal systems (Holland & Lambon-Ralph, 2010). This indirect ventral stream is
constituted by the anterior part of the ILF, connecting the visual object form area with
the temporal pole (Mandonnet, Nouet, Gatignol, Capelle, & Duffau, 2007), and then
relayed by the uncinate fascicle, which links the temporal pole with the pars orbitaris
of the inferior frontal gyrus (Duffau, Gatignol, Moritz-Gasser, & Mandonnet, 2009;
Duffau, Herbet, & Moritz-Gasser, 2013).
Regarding the dorsal phonological stream, cortically, phonemic paraphasia can be
elicited by DES of the inferior parietal lobule and inferior frontal gyrus (Maldonado
et al., 2011; Tate et al., 2014). Axonally speaking, phonemic paraphasias and repetition
disorders are elicited when stimulating the arcuate fascicle (Maldonado et al., 2011),
which is a fiber tract stemming from the caudal part of the temporal lobe, mainly the
inferior and middle temporal gyri, that arches around the insula and advances for-
ward to end within the frontal lobe, essentially within the ventral premotor cortex
and pars opercularis of the inferior frontal gyrus (Duffau et al., 2002; Martino et al.,
2013). Geschwind (1970) has previously postulated that lesions of this tract would
produce conduction aphasia, including phonemic paraphasia, supporting the role
of the subpart of the dorsal stream mediated by the arcuate fascicle in phonological
processing. Interestingly, the posterior cortical origin of this tract within the poste-
rior part of the inferior temporal gyrus corresponds to the visual object form area
(Martino et al., 2013). Indeed, this region represents a functional hub, involved both in
semantic (see earlier discussion) and phonological processing dedicated to visual ma-
terial (Vigneau et al., 2006). Thus, phonological processing subserved by the arcuate
fascicle is performed in parallel to the semantic processing undertaken by the ventral
route. In addition to this direct dorsal route, tractography studies show the existence
of an indirect dorsal stream, running more superficially, and undertaken by the lateral
superior longitudinal fascicle (Catani et al., 2005). This pathway is implicated in artic-
ulation and phonological working memory, as demonstrated by DES. Cortical areas
eliciting articulatory disorders are located essentially in the ventral premotor cortex,
and also in the supramarginal gyrus and posterior part of the superior temporal gyrus
(Duffau et al., 2003b; Tate et al., 2014). Axonally, stimulation of the white matter under
the frontoparietal operculum and supramarginal gyrus, laterally and ventrally to the
arcuate fascicle, induces anarthria as well (van Geemen, Herbet, Moritz-Gasser, &
Duffau, 2014). Indeed, this lateral operculo-opercular component of the superior lon-
gitudinal fascicle constitutes the articulatory loop, by connecting the supramarginal
gyrus/posterior portion of the superior temporal gyrus (which receives feedback in-
formation from somatosensory and auditory areas) with the frontal operculum (which
receives afferences bringing the phonological/phonetic information to be translated
into articulatory motor programs and efferences toward the primary motor area;
Duffau et al., 2003b).
To sum up, DES has resulted in the elaboration of a new model of picture naming,
based on multiple direct and indirect cortico-subcortical interacting subnetworks in-
volved in semantic, phonological, and articulatory processes. This model offers sev-
eral advantages in comparison with previous ones: (1) it explains double dissociations
during DES (e.g., semantic versus phonemic paraphasias); (2) it takes into account
the cortical and subcortical anatomic constraints; (3) it explains the possible recovery
of aphasia following a lesion within the “classical” language areas (see the following
discussion).
200 Hugues Duffau
Other Language Aspects: New Insights Provided by DES

Using the same paradigm, DES has allowed a reappraisal of cognitive models under-
lying other language aspects. For instance, DES has demonstrated that syntactic pro-
cessing was subserved by de-localized cortical regions (left inferior frontal gyrus and
posterior middle temporal gyrus) connected by a subpart of the left inferior superior
longitudinal fascicle. Interestingly, this subcircuit is interacting but independent of
the subnetwork involved in naming, as demonstrating by double dissociation between
syntactic (especially grammatical gender) and naming processing during DES. These
findings support parallel rather than serial theory, calling into question the principle of
the “lemma” (Vidorreta, Garcia, Moritz-Gasser, & Duffau, 2011).
In connectionist views of brain organization, interactions between different systems
have also been described. For instance, a recent DES study demonstrated that the dorsal
phonological pathway, supported by the left arcuate fascicle/superior longitudinal
fascicle complex subcortically, is crucial to accurate word repetition (Moritz-Gasser
& Duffau, 2013). It enables the conversion of auditory input, processed in the verbal
working memory system, into phonological and articulatory-based representations.
Although this contribution is essential, the ventral semantic pathway, connected by the
left IFOF, also contributes to word repetition of real words or pseudo-words. Indeed,
perioperative and intraoperative language evaluations of glioma patients undergoing
awake surgery highlight the strong interaction of both pathways in word repetition in
order to convert auditory input into articulatory output efficiently (Moritz-Gasser &
Duffau, 2013).
DES also has showed the existence of an executive system (including prefrontal
cortex, anterior cingulum, and caudate nucleus) involved in the cognitive control of
more dedicated subcircuit as language switching—itself constituted by a wide cortico-
subcortical network comprising posterior temporal areas, supramarginal and an-
gular gyri, inferior frontal gyrus, and a subpart of the superior longitudinal fascicle
(Moritz-Gasser & Duffau, 2009). In addition, it seems that the frontal aslant tract,
which connects the pre-supplementary motor area and anterior cingulate with the in-
ferior frontal gyrus (Thiebaut de Schotten, Dell’Acqua, Valabregue, & Catani, 2012),
might play a role in the control of language, especially with regard to planning of
speech articulation (Kinoshita et al., 2015). Indeed, stimulation of this pathway may
induce stuttering (Kemerdere et al., 2016). In the same vein, a cortico-subcortical loop
involving the deep grey nuclei, especially the caudate nucleus, was also demonstrated
as participating in the control of language (selection/inhibition), since DES of the head
of the caudate nucleus in the left hemisphere generated perseverations with a high level
of reliability (Gil Robles, Gatignol, Capelle, Mitchell, & Duffau, 2005). This cortico-
striatal loop could be anatomically supported by the fronto-striatal tract (Kinoshita
et al., 2015).
Furthermore, by using a semantic association task (e.g., Pyramids and Palm Tree
Test; Howard & Patterson, 1992), it was reported that DES of a wide subnetwork
comprising the left postero-superior temporal area, dorsolateral prefrontal cortex, and
IFOF generated disturbances of comprehension, including verbal and nonverbal se-
mantic processing as well as cross-modal judgment (Gatignol, Capelle, Le Bihan, &
Duffau, 2004; Moritz-Gasser et al., 2013; Plaza et al., 2008). Interestingly, a double dis-
sociation between picture naming and comprehension has been observed during DES,
again supporting parallel subnetworks rather than serial processing (Duffau et al.,
2013; Duffau et al., 2014). Of note, in this framework, the exact role of the middle lon-
gitudinal fascicle remains unclear (de Witt Hamer, Moritz-Gasser, Gatignol, & Duffau,
2011). The next step is to use DES to explore cortico-subcortical circuits involved in
verbal and nonverbal working memory and attention, with a likely implication of a
frontoparietal loop subserved by the lateral part of the superior longitudinal fascicle
(Moritz-Gasser & Duffau, 2013), in emotional processing such as theory of mind and
mentalizing (Herbet et al., 2014a; Herbet, Lafargue, Moritz-Gasser, Bonnetblanc, &
Duffau, 2015), and even in consciousness (Herbet et al., 2014b; Herbet, Lafargue, &
Duffau, 2016; Moritz-Gasser et al., 2013).
To sum up, our vision of the neural basis of language has begun to shift. For a long
time, language was conceived in associationist terms of centers and pathways, the ge-
neral assumption being that information is processed in localized cortical regions with
the serial passage of information between regions accomplished through white matter
tracts. Currently, an alternative hodotopical account is proposed, in which brain func-
tion is conceived as resulting from parallel distributed processing performed by dis-
tributed groups of connected neurons rather than individual centers (Duffau, 2008,
2014b). Conversely to serial models in which one process must be finished before the
information proceeds to another level of processing, these new models of “independent
networks” state that different processing can be performed simultaneously with inter-
active feedback. A multimodality approach seems now mandatory, with a more accu-
rate analysis of the interactions between the subnetworks underlying distinct cognitive
functions (e.g., language, visuospatial components, executive functions, as well as be-
havioral aspects) in order to open new avenues to better understand brain connectivity
or “connectomics” (Duffau, 2015) (Figure 8.3).
DES as a Tool to Investigate

the Reorganization
of Language Circuits
As mentioned, in this hodotopical framework, brain functions are supported by exten-

sive circuits comprising both the cortical epicenters (topos, i.e., sites) and connections
between these “nodes,” created by associating bundles of white matter (hodos, i.e.,
pathways) (De Benedictis & Duffau, 2011). Neurological function comes from the
202 Hugues Duffau
synchronization between different epicenters, working in phase during a given task,

and explaining why the same node may take part in several functions depending on
the other cortical areas with which it is temporarily connected at any one time. In
this connectionist model that challenges the traditional localizationist philosophy,
the central nervous system is organized into parallel networks that are dynamic, in-
teractive, and able to compensate for each other—at least to a certain extent (Duffau,
2014b). In other words, language maps may be reorganized within remote networks,
making neuroplasticity mechanisms possible, both ontogenetically (developmental
learning) and after brain injury (Duffau, 2005b, 2006). Interestingly, in patients with
slow-growing tumors such as diffuse low-grade gliomas, DES has demonstrated
massive language reshaping, which allowed wide surgical resection of brain regions
conventionally deemed to be inoperable on the basis of a rigid view of cerebral pro-
cessing (Desmurget, Bonnetblanc, & Duffau, 2007). For example, extensive excisions
of tumors infiltrating Broca’s area in the left hemisphere have been carried out without
causing permanent neurological deficit (Duffau, 2012b; Benzagmout, Gatignol, &
Duffau, 2007). Indeed, resection of the pars opercularis and/or pars triangularis of
the left inferior frontal gyrus is possible without risking aphasia, because, as already
mentioned, Broca’s area is not the final common pathway for speech (Tate et al., 2014;
Tate et al., 2015). In addition, this area can be compensated by recruitment of adja-
cent regions, primarily the ventral premotor cortex (the crucial epicenter for speech
articulation), and the pars orbitalis of the inferior frontal gyrus, the dorsal lateral pre-
frontal cortex, or the insula (Benzagmout et al., 2007; Duffau, 2012b). In the same
vein, removal of gliomas infiltrating Wernicke’s area also can be considered without
eliciting permanent language disorders (Figure 8.1). Once again, the language com-
pensation following resection of the postero-superior part of the left temporal lobe
(and its junction with the inferior parietal lobule) can be explained by the organi-
zation of this complex function into remote networks. As a result, in addition to
recruiting areas immediately around the lesion, reorganization may also involve re-
mote lesions in the left hemisphere (particularly the supramarginal gyrus and also the
pars triangularis of the inferior frontal gyrus), as well as contralateral sites in the right
hemisphere because of transcallosal disinhibition—as demonstrated by combining
intraoperative DES with perioperative functional neuroimaging (Sarubbo, Le Bars,
Moritz-Gasser, & Duffau, 2012). Above all, subcortical connectivity should be obliga-
torily preserved to avoid aphasia.
Indeed, although a huge plastic potential has been demonstrated at the cortical level,
subcortical plasticity is low, implying that axonal connectivity should be preserved
to allow post-lesional compensation. Lessons from stroke studies have taught us that
damage to the white matter pathways generate a more severe neurological impairment
than lesions of the cortex. Recently, the elaboration of a probabilistic atlas computed on
a series of patients who underwent resection for a glioma on the basis of intraoperative
DES was proposed (Herbet, Maheu, Costi, Lafargue, & Duffau, 2016; Ius, Angelini, de
Schotten, Mandonnet, & Duffau, 2011). The anatomo-functional correlations obtained
by combining the intrasurgical functional data with postoperative anatomical MRI
findings provided new insights into the potentials and limitations of cerebral plasticity.
This probabilistic atlas highlighted the crucial role of the axonal pathways, namely,
the connectome, in the reorganization of the brain after a lesion. It provided a general
framework to establish anatomo-functional correlations by computing, for each brain
voxel, its probability to remain intact—due to its functional role—on the postoperative
MRI. Its overlap with the cortical MNI template and a DTI atlas offered a unique tool
to analyze the potentialities and the limitations of inter-individual variability and plas-
ticity, both for cortical areas and axonal pathways. As a rule, a low probability of residual
tumors was observed on the cortical surface, whereas most of the regions with high
probability of residual tumor were located in the deep white matter. Thus, projection
and association axonal pathways seem to play a critical role in the proper functioning
of the brain, especially the arcuate fascicle, superior longitudinal fascicle, and IFOF. In
other words, language function subserved by long-range axonal pathways seems to be
less subject to inter-individual variability and reorganization than cortical sites (Duffau,
2009; Herbet, Maheu, et al., 2016; Ius et al., 2011). The reproducibility of these results
may suggest the existence of a “minimal common brain” necessary for basic cognitive
functions such as language—even if insufficient for more complex functions such as
multi-processing.
Because of this limitation of neuroplasticity due to the connectome, DES has great
value for investigating brain processing, even though stimulation is by definition
performed in patients with cerebral lesions. Indeed, if we admit that plasticity enabled
the compensation of all areas involved by the tumor, it would not be possible to iden-
tify crucial epicenters in a probabilistic map based on glioma patients—as nonetheless
done by Tate et al. (Tate et al., 2014; Tate et al., 2015). Thus, the ability to detect crit-
ical epicenters with a high rate of probability by inducing a reproducible deficit during
intraoperative DES (such as anarthria during stimulation of the vPMC), despite cere-
bral reshaping, has significant implications for understanding the normal functional
anatomy of the brain (Duffau, 2011b). In fact, the crucial cortical language (phonemic,
semantic) epicenters detected by intraoperative electrical mapping in the study by Tate
et al. (2014), involving glioma patients, correlate well with results provided by fMRI,
extensively described in a meta-analysis extracted from 129 scientific reports (with 730
activation peaks) that investigated language using functional neuroimaging in healthy
volunteers (Vigneau et al., 2006). Of note, these good correlations between DES and
fMRI are only observed when comparing statistical maps, while only poor correlations
are found at the individual level (see earlier discussion), explaining why DES remains
the gold standard for a given patient.
Conclusion
DES represents a unique opportunity to identify with great accuracy, reproducibility,

and reliability, in vivo in humans, the cortical and subcortical structures that are
204 Hugues Duffau
indispensable to the function, especially language, by inducing a transient virtual le-

sion based on the inhibition of a subcircuit over a few seconds. Currently, this is the
only technique able to directly investigate the functional role of white matter tracts
in humans. In addition, combining serial perioperative functional neuroimaging and
online intraoperative DES enables the study of mechanisms underlying cerebral plas-
ticity. Therefore, DES led our group to elaborate new individual and integrative models
of language. These networking models have resulted in a better knowledge of the dy-
namic spatiotemporal reorganization of parallel and interactive circuits in adults, and
a better knowledge of the limitations of neuroplasticity, mainly represented by the
connectome. Understanding axonal connectivity is crucial to optimizing neurolin-
guistic models, which should integrate the anatomic constraints represented by sub-
cortical pathways. Finally, it is worth noting that this improved knowledge of language
networks is a prerequisite for restoring brain functions in aphasic patients, in partic-
ular by opening new approaches in the field of brain-computer interfaces (Mandonnet
& Duffau, 2014).
References
Agarwal, S., Sair, H. I., Airan, R., Hua, J., Jones, C. K., Heo, H. Y., et al. (2016). Demonstration
of brain tumor-induced neurovascular uncoupling in resting-state fMRI at ultrahigh field.
Brain Connect, 6, 267–272.
Agnew, W. F., & McCreery, D. B. (1987). Considerations for safety in the use of extracranial
stimulation for motor evoked potentials. Neurosurgery, 20, 143–147.
Almairac, F., Herbet, G., Moritz-Gasser, S., de Champfleur, N. M., & Duffau, H. (2015). The left
inferior fronto-occipital fasciculus subserves language semantics: A multilevel lesion study.
Brain Structure and Function, 220, 1983–1995.
Bartels, A., & Zeki, S. (2005). The chronoarchitecture of the cerebral cortex. Philosophical
Transactions of the Royal Society B: Biological Sciences, 360, 733–750.
Benzagmout, M., Gatignol, P., & Duffau, H. (2007). Resection of World Health Organization
Grade II gliomas involving Broca’s area: Methodological and functional considerations.
Catani, M., & Ffytche, D. H. (2005). The rises and falls of disconnection syndromes. Brain, 128,
2224–2239.
Catani, M., Jones, D. K., & Ffytche, D. H. (2005). Perisylvian language networks of the human
brain. Annals of Neurology, 57,  8–16.
Chan-Seng, E., Moritz-Gasser, S., & Duffau, H. (2014). Awake mapping for low-grade gliomas
involving the left sagittal stratum: Anatomofunctional and surgical considerations. Journal
of Neurosurgery, 120, 1069–1077.
Cilia, R., Landi, A., Vergani, F., Sganzerla, E., Pezzoli, G., & Antonini, A. (2007). Extradural
motor cortex stimulation in Parkinson’s disease. Movement Disorders, 22, 111–114.
de Benedictis, A., & Duffau, H. (2011). Brain hodotopy: From esoteric concept to practical sur-
gical applications. Neurosurgery, 68, 1709–1723.
Deras, P., Moulinié, G., Maldonado, I. L., Moritz-Gasser, S., Duffau, H., & Bertram, L.
(2012). Intermittent general anesthesia with controlled ventilation for asleep-awake-asleep
brain surgery: A prospective series of 140 gliomas in eloquent areas. Neurosurgery, 71,
764–7 71.
Desmurget, M., Bonnetblanc, F., & Duffau, H. (2007) Contrasting acute and slow growing
lesions: A new door to brain plasticity. Brain, 130, 898–914.
De Witt Hamer, P., Moritz-Gasser, S., Gatignol, P., & Duffau, H. (2011). Is the human left middle
longitudinal fascicle essential for language? A brain electrostimulation study. Human Brain
Mapping, 32, 962–973.
du Boisgueheneuc, F., Levy, R., Volle, E., Seassau, M., Duffau, H., Kinkingnehun, S., et al.
(2006). Functions of the left superior frontal gyrus in humans: A lesion study. Brain, 129,
3315–3328.
Duffau, H. (2004). Cartographie fonctionnelle per-opératoire par stimulations électriques
directes. Aspects méthodologiques [Intraoperative functional mapping using direct elec-
trical stimulations. Methodological considerations]. Neurochirurgie, 50, 474–483.
Duffau, H. (2005a). Intraoperative cortico-subcortical stimulations in surgery of low-grade
gliomas. Expert Review in Neurotherapeutics, 5, 473–485.
Duffau, H. (2005b). Lessons from brain mapping in surgery for low-grade glioma: Insights into
associations between tumour and brain plasticity. Lancet Neurology, 4, 476–486.
Duffau, H. (2006). Brain plasticity: From pathophysiological mechanisms to therapeutic
applications. Journal of Clinical Neuroscience, 13, 885–897.
Duffau, H. (2007). Contribution of cortical and subcortical electrostimulation in brain glioma
surgery: Methodological and functional considerations. Clinical Neurophysiology, 37, 373–382.
Duffau, H. (2008). The anatomo-functional connectivity of language revisited: New insights
provided by electrostimulation and tractography. Neuropsychologia, 46, 927–934.
Duffau, H. (2009). Does post- lesional subcortical plasticity exist in the human brain?
Neuroscience Research, 65, 131–135.
Duffau, H. (Ed.) (2011a). Brain mapping: From neural basis of cognition to surgical applications.
New York: Springer.
Duffau, H. (2011b). Do brain tumours allow valid conclusions on the localisation of human
brain functions? Cortex, 47, 1016–1017.
Duffau, H. (2012a). The challenge to remove diffuse low-grade gliomas while preserving brain
functions. Acta Neurochirurgica (Wien), 154, 569–574.
Duffau, H. (2012b). The “frontal syndrome” revisited: Lessons from electrostimulation map-
ping studies. Cortex, 48, 120–131.
Duffau, H. (2014a). The dangers of magnetic resonance imaging diffusion tensor tractography
in brain surgery. World Neurosurgery, 81,  56–58.
Duffau, H. (2014b). The huge plastic potential of adult brain and the role of connectomics: New
insights provided by serial mappings in glioma surgery. Cortex, 58, 325–337.
Duffau, H. (2015). Stimulation mapping of white matter tracts to study brain functional con-
nectivity. Nature Reviews Neurology, 11, 255–265.
Duffau, H., Capelle, L., Sichez, N., Denvil, D., Bitar, A., Sichez, J. P., & Fohanno, D. (2002).
Intraoperative mapping of the subcortical language pathways using direct stimulations: An
anatomo-functional study. Brain, 125, 199–214.
Duffau, H., Capelle, L., Denvil, D., Sichez, N., Gatignol, P., Taillandier, L., et al. (2003a).
Usefulness of intraoperative electrical subcortical mapping during surgery for low-grade
gliomas located within eloquent brain regions: Functional results in a consecutive series of
103 patients. Journal of Neurosurgery, 98, 764–778.
206 Hugues Duffau
Duffau, H., Capelle, L., Denvil, D., Gatignol, P., Sichez, N., Lopes, M., Sichez, J.P., & van
Effenterre, R. (2003b). The role of dominant premotor cortex in language: A study using
intraoperative functional mapping in awake patients. NeuroImage, 20, 1903–1914.
Duffau, H., Gatignol, P., Mandonnet, E., Capelle, L., & Taillandier, L. (2008). Intraoperative
subcortical stimulation mapping of language pathways in a consecutive series of 115
patients with Grade II glioma in the left dominant hemisphere. Journal of Neurosurgery,
109, 461–471.
Duffau, H., Gatignol, P., Mandonnet, E., Peruzzi, P., Tzourio-Mazoyer, N., & Capelle, L. (2005).
New insights into the anatomo-functional connectivity of the semantic system: A study
using cortico-subcortical electrostimulations. Brain, 128, 797–810.
Duffau, H., Gatignol, P., Moritz-Gasser, S., & Mandonnet, E. (2009). Is the left uncinate
fasciculus essential for language? A cerebral stimulation study. Journal of Neurology, 256,
382–389.
Duffau, H., Herbet, G., & Moritz-Gasser, S. (2013). Toward a pluri-component, multimodal,
and dynamic organization of the ventral semantic stream in humans: Lessons from stimula-
tion mapping in awake patients. Frontiers System Neuroscience, 7, 44.
Duffau, H., Moritz-Gasser, S., & Mandonnet, E. (2014). A re-examination of neural basis of
language processing: Proposal of a dynamic hodotopical model from data provided by brain
stimulation mapping during picture naming. Brain and Language, 131,  1–10.
Fernández Coello, A., Moritz-Gasser, S., Martino, J., Martinoni, M., Matsuda, R., & Duffau, H.
(2013). Selection of intraoperative tasks for awake mapping based on relationships between
tumor location and functional networks. Journal of Neurosurgery, 119, 1380–1394.
Gaillard, R., Naccache, L., Pinel, P., Clémenceau, S., Volle, E., Hasboun, D., et al. (2006). Direct
intracranial, FMRI, and lesion evidence for the causal role of left inferotemporal cortex in
reading. Neuron, 50, 191–204.
Gatignol, P., Capelle, L., Le Bihan, R., & Duffau, H. (2004). Double dissociation between
picture naming and comprehension: An electrostimulation study. Neuroreport, 15,
191–195.
Geddes, L. A., & Baker, L. E. (1967). The specific resistance of biological material: A compen-
dium of data for the biomedical engineer and physiologist. Medical Biological Engineering,
5, 271–293.
Geschwind, N. (1970). The organization of language and the brain. Science, 170, 940–944.
Gil Robles, S., Carvallo, A., Jimenez Mdel, M., Gomez Caicoya, A., Martinez, R., Ruiz-Ocaña,
C., & Duffau H. (2013). Double dissociation between visual recognition and picture
naming: A study of the visual language connectivity using tractography and brain stimula-
tion. Neurosurgery, 72, 678–686.
Gil Robles, S., Gatignol, P., Capelle, L., Mitchell, M. C., & Duffau, H. (2005). The role of dom-
inant striatum in language: A study using intraoperative electrical stimulations. Journal of
Neurology, Neurosurgery and Psychiatry, 76, 940–946.
Giussani, C., Roux, F. E., Ojemann, J., Sganzerla, E. P., Pirillo, D., & Papagno, C. (2010). Is pre-
operative functional magnetic resonance imaging reliable for language areas mapping in
brain tumor surgery? Review of language functional magnetic resonance imaging and direct
cortical stimulation correlation studies. Neurosurgery, 66, 113–120.
Gordon, B., Lesser, R. P., Rance, N. E., Hart, J. Jr., Webber, R., Uematsu, S., et al. (1990).
Parameters for direct cortical electrical stimulation in the human: Histopathologic confir-
mation. Electroencephalography Clinical Neurophysiology, 75, 371–377.
Gras-Combes, G., Moritz-Gasser, S., Herbet, G., & Duffau, H. (2012). Intraoperative subcor-
tical electrical mapping of optic radiations in awake surgery for glioma involving visual
pathways. Journal of Neurosurgery, 117, 466–473.
Haglund, M. M., Ojemann, G. A., & Blasdel, G. G. (1993). Optical imaging of bipolar cortical
stimulation. Journal of Neurosurgery, 78, 785–793.
Hamberger, M. J., McClelland, S., 3rd, McKhann, G. M., 2nd, Williams, A. C., & Goodman,
R. R. (2007). Distribution of auditory and visual naming sites in nonlesional temporal lobe
epilepsy patients and patients with space-occupying temporal lobe lesions. Epilepsia, 48,
531–538.
Havel, P., Braun, B., Rau, S., Tonn, J. C., Fesl, G., Bruckmann, H., et al. (2006). Reproducibility
of activation in four motor paradigms: An fMRI study. Journal of Neurology, 253, 471–476.
Herbet, G., Lafargue, G., Bonnetblanc, F., Moritz-Gasser, S., Menjot de Champfleur, N., &
Duffau, H. (2014a). Inferring a dual-stream model of mentalizing from associative white
matter fibres disconnection. Brain, 137, 944–959.
Herbet, G., Lafargue, G., de Champfleur, N. M., Moritz-Gasser, S., le Bars, E., Bonnetblanc, F.,
& Duffau, H. (2014b). Disrupting posterior cingulate connectivity disconnects conscious-
ness from the external environment. Neuropsychologia, 56, 239–244.
Herbet, G., Lafargue, G., & Duffau, H. (2016). The dorsal cingulate cortex as a critical gateway
in the network supporting conscious awareness. Brain, 139, e23.
Herbet, G., Lafargue, G., Moritz-Gasser, S., Bonnetblanc, F., & Duffau, H. (2015). Interfering
with the neural activity of mirror-related frontal areas impairs mentalistic inferences. Brain
Structure and Function, 220, 2159–2169.
Herbet, G., Maheu, M., Costi, E., Lafargue, G., & Duffau, H. (2016). Mapping the neuroplastic
potential in brain-damaged patients. Brain, 139, 829–844.
Hickok, G., & Poeppel, D. (2004). Dorsal and ventral streams: A framework for understanding
aspects of the functional anatomy of language. Cognition, 92,  67–99.
Holland, R., & Lambon-Ralph, M. A. (2010). The anterior temporal lobe semantic hub is a part
of the language neural network: Selective disruption of irregular past tense verb by rTMS.
Howard, D., & Patterson, K. (1992). The pyramid and palm trees test. Bury St Edmunds: Thames
Valley Test.
Hubbard, P. L., & Parker, G. J. M. (2009). Validation of tractography. In H. Johansen-Berg & T.
Behrens (Eds.), Diffusion MRI (pp. 353–376). Burlington: Academic Press.
Ishitobi, M., Nakasato, N., Suzuki, K., Nagamatsu, K., Shamoto, H., & Yoshimoto, T. (2000).
Remote discharges in the posterior language area during basal temporal stimulation.
Neuroreport, 11, 2997–3000.
Ius, T., Angelini, E., de Schotten, M. T., Mandonnet, E., & Duffau, H. (2011). Evidence for
potentials and limitations of brain plasticity using an atlas of functional respectability of
WHO grade II gliomas: Towards a “minimal common brain.” NeuroImage, 56, 992–1000.
Jayakar, P., Alvarez, L. A., Duchowny, M. S., & Resnick, T. J. (1992). A safe and effective paradigm
to functionally map the cortex in childhood. Journal of Clinical Neurophysiology, 9, 288–293.
Jayakar, P. (1993). Physiological principles of electrical stimulation. In O. Devinsky, A.
Beric, & M. Dogali (Eds.), Electrical and magnetic stimulation of the brain and spinal cord.
New York: Raven Press. 63: 17–27.
Jbabdi, S., & Johansen-Berg, H. (2011). Tractography: Where do we go from here? Brain
Connection, 1, 169–183.
208 Hugues Duffau
Kemerdere, R., de Champfleur, N. M., Deverdun, J., Cochereau, J., Moritz-Gasser, S., Herbet,
G., et al. (2016). Role of the left frontal aslant tract in stuttering: A brain stimulation and
tractographic study. Journal of Neurology, 263, 157–167.
Kinoshita, M., Yamada, K., Hashimoto, N., Kato, A., Izumoto, S., Baba, T., et al. (2005). Fiber-
tracking does not accurately estimate size of fiber bundle in pathological condition: Initial
neurosurgical experience using neuronavigation and subcortical white matter stimulation.
Kinoshita, M., Menjot de Champfleur, N., Deverdun, J., Moritz-Gasser, S., Herbet, G., &
Duffau, H. (2015). Role of fronto-striatal tract and frontal aslant tract in movement and
speech: An axonal mapping study. Brain Structure and Function, 220, 3399–412.
Korvenoja, A., Kirveskari, E., Aronen, H. J., Avikainen, S., Brander, A., Huttunen, J., et al.
(2006). Sensorimotor cortex localization: Comparison of magnetoencephalography, func-
tional MR imaging, and intraoperative cortical mapping. Radiology, 241, 213–222.
Krainik, A., Duffau, H., Capelle, L., Cornu, P., Boch, A. L., Mangin, J. F., et al. (2004). Role of the
healthy hemisphere in recovery after resection of the supplementary motor area. Neurology,
62, 1323–1332.
Krainik, A., Lehéricy, S., Duffau, H., Capelle, L., Chainay, H., Cornu, P., et al. (2003).
Postoperative speech disorder after medial frontal surgery: Role of the supplementary
motor area. Neurology, 60, 587–594.
Kuchcinski, G., Mellerio, C., Pallud, J., Dezamis, E., Turc, G., Rigaux-Viodé, O., et al. (2015).
Three-tesla functional MR language mapping: Comparison with direct cortical stimulation
in gliomas. Neurology, 84, 560–568.
Le Bihan, D., Poupon, C., Amadon, A., & Lethimonnier, F. (2006). Artifacts and pitfalls in dif-
fusion MRI. Journal of Magnetic Resonance Imaging, 24, 478–488.
Leclercq, D., Duffau, H., Delmaire, C., Capelle, L., Gatignol, P., Ducros, M., et al. (2010).
Comparison of diffusion tensor imaging tractography of language tracts and intraoperative
subcortical stimulations. Journal of Neurosurgery, 112, 503–511.
Lee, L., Friston, K., & Horwitz, B. (2006). Large-scale neural models and dynamic causal mod-
elling. NeuroImage, 30, 1243–2154.
Lüders, H., Lesser, R. P., Hahn, J., Dinner, D. S., Morris, H. H., Wyllie, E., et al. (1991). Basal tem-
poral language area. Brain, 114, 743–754.
Maldonado, I. L., Moritz-Gasser, S., & Duffau, H. (2011). Does the left superior longitudinal
fascicle subserve language semantics? A brain electrostimulation study. Brain Structure and
Function, 216, 263–264.
Mandonnet, E., Capelle, L., & Duffau, H. (2006). Extension of paralimbic low grade
gliomas: Toward an anatomical classification based on white matter invasion patterns.
Journal of Neurooncology, 78, 179–185.
Mandonnet, E., & Duffau, H. (2014). Understanding entangled cerebral networks: A pre-
requisite for restoring brain function with brain-computer interfaces. Frontiers System
Neuroscience, 8, 82.
Mandonnet, E., Gatignol, P., & Duffau, H. (2009). Evidence for an occipito-temporal tract under-
lying visual recognition in picture naming. Clinical Neurology and Neurosurgery, 111, 601–605.
Mandonnet, E., Nouet, A., Gatignol, P., Capelle, L., & Duffau, H. (2007). Does the left inferior
longitudinal fasciculus play a role in language? A brain stimulation study. Brain, 130, 623–629.
Mandonnet, E., & Pantz, O. (2011). The role of electrode direction during axonal bipolar elec-
trical stimulation: A bidomain computational model study. Acta Neurochirurgica (Wien),
153, 2351–2355.
Mandonnet, E., Winkler, P. A., & Duffau, H. (2010). Direct electrical stimulation as an
input gate into brain functional networks: Principles, advantages and limitations. Acta
Neurochirurgica (Wien), 152, 185–193.
Martino, J., Brogna, C., Gil Robles, S., Vergani, F., & Duffau, H. (2010). Anatomic dissection
of the inferior fronto-occipital fasciculus revisited in the lights of brain stimulation data.
Cortex, 46, 691–699.
Martino, J., De Witt Hamer, P. C., Berger, M. S., Lawton, M. T., Arnold, C. M., de Lucas, E.
M., & Duffau, H. (2013). Analysis of the subcomponents and cortical terminations of the
perisylvian superior longitudinal fasciculus: A fiber dissection and DTI tractography study.
Brain Structure and Function, 218, 105–121.
McClelland, J. L., & Rogers, T. T. (2003). The parallel distributed processing approach to se-
mantic cognition. Nature Review Neuroscience, 4, 310–322.
McIntyre, C. C., Savasta, M., Walter, B. L., & Vitek, J. L. (2004). How does deep brain stimula-
tion work? Present understanding and future questions. Journal of Clinical Neurophysiology,
21,  40–50.
Montgomery, E. B., Jr. (2004). Dynamically coupled, high-frequency reentrant, non-linear
oscillators embedded in scale-free ganglia-thalamic-cortical networks mediating function
and deep brain stimulation effects. Nonlinear Studies, 11, 385–421.
Moritz-Gasser, S., & Duffau, H. (2009). Cognitive processes and neural basis of language
switching: Proposal of a new model. Neuroreport, 20, 1577–1580.
Moritz-Gasser, S., & Duffau, H. (2013). The anatomo-functional connectivity of word repe-
tition: Insights provided by awake brain tumor surgery. Frontiers in Human Neuroscience,
7, 405.
Moritz-Gasser, S., Herbet, G., & Duffau, H. (2013). Mapping the connectivity underlying mul-
timodal (verbal and non-verbal) semantic processing: A brain electrostimulation study.
Nathan, S. S., Lesser, R. P., & Gordon, B. (1993). Electrical stimulation of the human cerebral
cortex: Theoretical approach. In O. Devinsky, A. Beric, & M. Dogali (Eds.), Electrical and
magnetic stimulation of the brain and spinal cord (pp. 183–192). New York: Raven Press.
Ojemann, G., Ojemann, J., Lettich, E., & Berger, M. S. (1989). Cortical language localization in
left, dominant hemisphere: An electrical stimulation mapping investigation in 117 patients.
Journal of Neurosurgery, 71, 316–326.
Pagni, C. A., Altibrandi, M. G., Bentivoglio, A., Caruso, G., Cioni, B., Fiorella, C., et al.
(2005). Extradural motor cortex stimulation (EMCS) for Parkinson’s disease: History and
first results by the study group of the Italian neurosurgical society. Acta Neurochirurgica
Supplement, 93, 113–119.
Plaza, M., Gatignol, P., Cohen, H., Berger, B., & Duffau, H. (2008). A discrete area within
the left dorsolateral prefrontal cortex involved in visual-verbal incongruence judgment.
Popovych, O. V., Hauptmann, C., & Tass, P. A. (2006) Control of neuronal synchrony by non-
linear delayed feedback. Biological Cybernetics, 95,  69–85.
Pujol, S., Wells, W., Pierpaoli, C., Brun, C., Gee, J., Cheng, G., et al. (2015). The DTI
Challenge: Toward standardized evaluation of Diffusion Tensor Imaging tractography for
neurosurgery. Journal of Neuroimaging, 25, 875–882
Quigg, M., & Fountain, N. B. (1999). Conduction aphasia elicited by stimulation of the left
posterior superior temporal gyrus. Journal of Neurology, Neurosurgery and Psychiatry, 66,
393–396.
210 Hugues Duffau
Quigg, M., Geldmacher, D. S., & Elias, W. J. (2006). Conduction aphasia as a function of the
dominant posterior perisylvian cortex: Report of two cases. Journal of Neurosurgery, 104,
845–848.
Ranck, J. B. (1981). Extracellular stimulation. In M. M. Patterson & R. P. Kesner (Eds.), Electrical
stimulation research methods (pp. 2–36). New York: Academic Press.
Rosenblum, M., & Pikovsky, A. (2004). Delayed feedback control of collective synchrony: An
approach to suppression of pathological brain rhythms. Physical Review E: Statistical,
Nonlinear, and Soft Matter Physics, 70, 041904.
Roux, F. E., Boulanouar, K., Lotterie, J. A., Mejdoubi, M., LeSage, J. P., & Berry, I. (2003).
Language functional magnetic resonance imaging in preoperative assessment of language
areas: Correlation with direct cortical stimulation. Neurosurgery, 52, 1335–1345.
Sartorius, C. J., & Berger, M. S. (1998). Rapid termination of intraoperative stimulation-evoked
seizures with application of cold Ringer’s lactate to the cortex: Technical note. Journal of
Sarubbo, S., De Benedictis, A., Maldonado, I. L., Basso, G., & Duffau, H. (2013). Frontal
terminations for the inferior fronto-occipital fascicle: Anatomical dissection, DTI study
and functional considerations on a multi-component bundle. Brain Structure and Function,
218,  21–37.
Sarubbo, S., Le Bars, E., Moritz-Gasser, S., & Duffau, H. (2012). Complete recovery after sur-
gical resection of left Wernicke’s area in awake patient: A brain stimulation and functional
MRI study. Neurosurgical Review, 35, 287–292.
Tate, M., Herbet, G., Moritz-Gasser, S., Tate, J. E., & Duffau, H. (2014). Probabilistic map of
critical functional regions of the human cerebral cortex: Broca’s area revisited. Brain, 137,
2773–2782.
Tate, M., Herbet, G., Moritz-Gasser, S., Tate, J. E., & Duffau, H. (2015) Broca’s area is not the
speech output region. Brain, 138, e338.
Taylor, M. D., & Bernstein, M. (1999). Awake craniotomy with brain mapping as the routine
surgical approach to treating patients with supratentorial intraaxial tumors: A prospective
trial of 200 cases. Journal of Neurosurgery, 90,  35–41.
Teixidor, P., Gatignol, P., Leroy, M., Masuet-Aumatell, C., Capelle, L., & Duffau, H. (2007).
Assessment of verbal working memory before and after surgery for low-grade glioma.
Journal of Neurooncology, 81, 305–313.
Thiebaut de Schotten, M., Dell’Acqua, F., Valabregue, R., & Catani, M. (2012). Monkey to
human comparative anatomy of the frontal lobe association tracts. Cortex, 48,  82–96.
Thiebaut de Schotten, M., Urbanski, M., Duffau, H., Volle, E., Levy, R., Dubois, B., &
Bartolomeo, P. (2005). Direct evidence for a parietal-frontal pathway subserving spatial
awareness in humans. Science, 309, 2226–2228.
van Geemen, K., Herbet, G., Moritz-Gasser, S., & Duffau, H. (2014). Limited plastic potential of
the left ventral premotor cortex in speech articulation: Evidence from intraoperative awake
mapping in glioma patients. Human Brain Mapping, 35, 1487–1596.
Vidorreta, J. G., Garcia, R., Moritz-Gasser, S., & Duffau, H (2011). Double dissociation be-
tween syntactic gender and picture naming processing: A brain stimulation mapping study.
Vigneau, M., Beaucousin, V., Herve, P. Y., Duffau, H., Crivello, F., Houdé, O., et al. (2006).
Meta-analyzing left hemisphere language areas: Phonology, semantics, and sentence pro-
cessing. NeuroImage, 30, 1414–1432.
Yamao, Y., Matsumoto, R., Kunieda, T., Arakawa, Y., Kobayashi, K., Usami, K., et al. (2014).
Intraoperative dorsal language network mapping by using single-pulse electrical stimula-
tion. Human Brain Mapping, 35, 4345–4361.
Yeomans, J. S. (1990). Principles of brain stimulation. New York: Oxford University Press.
Zemmoura, I., Herbet, G., Moritz-Gasser, S., & Duffau, H. (2015). New insights into the neural
network mediating reading processes provided by cortico-subcortical electrical mapping.
Chapter 9
Diffusion I mag i ng
M ethod s i n
L anguage S c i e nc e s
Marco Catani and Stephanie J. Forkel
Introduction
Over the last two centuries, neuronal correlates of language have been investigated
employing various structural and functional methods. Structural methods include
traditional post-mortem dissections (Broca, 1865; Wernicke, 1874), computer to-
mography (Naeser & Hayward, 1978; Yarnell, Monroe, & Sobel, 1976) and magnetic
resonance imaging (MRI) to visualize damage to cortical and subcortical anatomy
(DeWitt, Grek, Buonanno, Levine, & Kistler, 1985; see Wilson, Chapter 2 in this
volume). Functional methods include electro-and magnetoencephalography
(Friederici, von Cramon, & Kotz, 1999; Hari & Lounasmaa, 1989; Salmelin, 2007;
Tikofsky, Kooi, & Thomas, 1960; see Salmelin, Kujala, & Liljeström, Chapter 6 in
this volume), functional MRI (McCarthy, Blamire, Rothman, Gruetter, & Shulman,
1993; see Heim & Specht, Chapter 4 in this volume), direct electrical stimulation (see
Duffau, Chapter 8 in this volume), and tomography methods for brain hemody-
namics and metabolism, such as single photon emission computerized tomography
(Perani, Vallar, Cappa, Messa, & Fazio, 1987) and positron emission tomography
(Cappa et al., 1997; Wise, Hadar, Howard, & Patterson, 1991). These methods, albeit
complementary to each other in terms of spatial and temporal resolution, are in-
sufficient to investigate the structural connections supporting distributed language
processing in the human brain. Additionally, they are unable to provide quantitative
measures of tract anatomy to study, for example, structural asymmetry between the
two hemispheres in the same subject or across groups.
Understanding the anatomy of language networks and its variability in both the
healthy population and patients represents one of the most significant contributions
Diffusion Imaging Methods in Language Sciences 213
of diffusion tractography to modern neurolinguistics. First, by looking at tracts we

can reconcile data on patients who present with aphasia and “atypical” lesion loca-
tion (Catani et al., 2012a). These patients often have lesions that are distant from re-
gions dedicated to specific language functions. The classical example is represented
by patients presenting with Broca’s aphasia with retro- rolandic lesions sparing
Broca’s area but affecting tracts connected to this area (Basso, Lecours, Moraschini, &
Vanier, 1985; Catani et al., 2012a). Second, and most important, a network approach
emphasizes the parallel and integrated nature of the cognitive processes underlying
language. This network approach is a fundamental evolution away from static and rigid
localizationist models of brain function and helps to broaden the disconnectionist ap-
proach to a wider range of language disorders in neurology and psychiatry (Catani &
ffytche, 2005).
In the last 15 years, diffusion tractography has become an established noninvasive quan-
titative method to study connectional anatomy in the living human brain. When applied
to language, tractography has proven to be a powerful tool to gain new insights into the
functional anatomy of language. For example, tractography has indicated that networks
for language and social communication are more complex than previously anticipated
(Catani & Bambini, 2014; Catani, Jones, & ffytche, 2005; Turken & Dronkers, 2011) and
extend to areas and connections that were not included in the classical Broca-Wernicke-
Geschwind model (Geschwind, 1965). This complexity is heterogeneous among the
healthy population, with striking differences between the male and the female brain
(Catani et al., 2007) and across ages (Budisavljevic et al., 2015; Lebel, Walker, Leemans,
Phillips, & Beaulieu, 2008). These inter-individual differences are beginning to explain
the observed variance in language performances among healthy controls (Catani et al.,
2007; Lopez-Barroso et al., 2013) and in the recovery of language after stroke (Forkel
et al., 2014).
In this chapter, we will discuss the principles of diffusion imaging and tractography
alongside examples of how this method has informed our understanding of language
development, variance in language performances, and language disorders.
Principles of Diffusion Tractography
The history of diffusion imaging and tractography spans about 30 years and can
be broadly split into two halves. The first 15 years extend from 1985, the year that the
first diffusion-weighted images of the brain were acquired, to 1999, when a handful of
researchers proposed diffusion tractography as a method to study trajectories of white
matter pathways in the living brain (Le Bihan et al., 1986; Conturo et al., 1999; Jones,
Simmons, Williams, & Horsfield, 1999; Mori, Crain, Chacko, & van Zijl, 1999). The
second half (i.e., 2000–2015) has been characterized by a systematic application of dif-
fusion methods, including tractography, to study the anatomy of connections in the
healthy population and the impact of disorders on white matter organization in patient
214 Marco Catani and Stephanie J. Forkel
cohorts. While novel methods for diffusion imaging are continuously proposed, here we
focus on current mainstream methods that are widely employed.
Diffusion Weighted Imaging

Diffusion weighted imaging (DWI) is a noninvasive, in vivo MRI technique that
quantifies water diffusion in biological tissues. In neuronal tissue, the displacement of
water molecules is not random due to the presence of biological structures such as cell
membranes, filaments, and nuclei. These structures hinder water diffusion in the three-
dimensional space. In the white matter, the overall displacement is reduced unevenly due
to the presence of axonal membranes and myelin sheets, which hinder water diffusion in
a direction perpendicular to the axonal fibers. Within a single voxel, water diffusion can
be described geometrically as an ellipsoid (the tensor) calculated from the diffusion coef-
ficient values (eigenvalues, λ1–3) and orientations (eigenvectors, ν1–3) of its three principal
axes (Figure 9.1) (Basser, Mattiello & Le Bihan, 1994). A detailed analysis of the tensor can
provide precise information about not only the average water molecular displacement
within a voxel (e.g., mean diffusivity), but also the degree of tissue anisotropy (e.g., frac-
tional anisotropy), and the main orientation of the underlying white matter pathways
(e.g., principal eigenvector or color-coded maps). These indices provide complementary
information about the microstructural composition and architecture of brain tissue.
Mean diffusivity (MD) is a rotational invariant quantitative index that describes the
average mobility of water molecules and is calculated from the three eigenvalues of the
tensor according to the formula MD = [(λ1 + λ2 + λ3)/3]. Voxels containing gray and
white matter tissue show similar MD values (Pierpaoli, Jezzard, Basser, Barnett, & Di
Chiro, 1996). MD reduces with age within the first years of life and increases in those
disorders characterized by demyelination, axonal injury, and edema (Beaulieu, 2009).
The fractional anisotropy (FA) index ranges from 0 to 1 and represents a quantita-
tive measure of the degree of anisotropy in biological tissue (Figure 9.1). In the healthy
adult brain, FA varies typically between 0.2 (e.g., in gray matter) and ≥0.8 in some white
matter regions with highly myelinated parallel fibers. In the white matter FA provides
information about the local organization of the fibers (e.g., parallel, crossing, kissing
fibers) and their biological features (e.g., degree of myelination, axonal membrane per-
meability, etc.). FA changes in pathological tissue (e.g., demyelination, edema, ischemia)
and is therefore commonly used as an indirect index of microstructural disorganization
or integrity in brain disorders.
Perpendicular [(λ2 + λ3)/2] and parallel diffusivity (λ1) describe the diffusivity along the
principal directions of the water diffusion. The perpendicular diffusivity, also indicated by
the term radial diffusivity (RD), is generally considered a more sensitive index of axonal or
myelin damage, although interpretation of their changes in regions with crossing fibers is
not always straightforward (Dell’Acqua & Catani, 2012). The principal eigenvector and the
red-green-blue (RGB) color-coded maps are particularly useful to visualize the principal
orientation of the tensor within each voxel (Pajevic & Pierpaoli, 1999; see Figure 9.1).
(A)
z λ1 = λ2 = λ3 λ1 = λ2 >> λ3 λ1 >> λ2 = λ3
λ1
v1
v2
λ2
λ3
x v3
Isotropy Planar Anisotropy Axial Anisotropy

(B)
Mean Diffusivity (MD) Fractional Anisotropy (FA) Principal eigenvector Color-coded FA
0 [10–3 mm2/s] 3 0 1
(C)
v1
Figure 9.1. Principles of diffusion tensor imaging (DTI) tractography. (A) Visualization

of the diffusion tensor as an ellipsoid. The size and the shape of the tensor are defined by the
three eigenvalues (λ1, λ2, λ3 in red), while the spatial orientation is described by the three
eigenvectors (v1, v2, v3 in blue). In biological tissues, the tensor can vary between three pos-
sible configurations: (1) isotropy or weak anisotropy (equal or similar diffusivity along the three
eigenvalues) is commonly observed, for example, inside the ventricles (isotropy) or in the gray
matter (weak anisotropy); (2) planar anisotropy (unequal diffusivity between of one eigenvalue
and the other two) is common in voxels containing, for example, two groups of crossing or
diverging fibers; and (3) axial anisotropy is typical of voxels containing parallel fibers (unequal
diffusivity between the axial eigenvalue and the two perpendicular eigenvalues). (B) Exemplified
indices extracted from diffusion data, such as mean diffusivity (MD), fractional anisotropy (FA),
principle eigenvector, and color-coded FA maps. (C) Streamline tractography is based on the as-
sumption that in each white matter voxel the principal eigenvector (black arrows) is tangent to
the main trajectory of the underlying fibers (black lines). Starting from a region of interest (red
circle), the tractography algorithm propagates, voxel by voxel, a streamline (red) by piecing to-
gether neighboring principal eigenvectors. An example is shown in the neighboring panel where
the streamlines are tracked on a principal eigenvector map and the tractography reconstruction
of the arcuate fasciculus is visualized as 3D streamtubes.
Tractography-Based Reconstruction
of White Matter Pathways
Compared to previously established methods used for studying tract anatomy, such
as axonal tracing in animals or human post-mortem blunt dissection, diffusion tensor
tractography offers the unique advantage of being a completely noninvasive technique,
and therefore its use is not restricted to nonhuman primates but can be applied to the
living human brain. Furthermore, the data required to obtain tract reconstructions with
tractography can be readily acquired on standard clinical MRI systems with scanning
times typically ranging between 5 and 20 minutes. Recently, methodological advances
have enabled shorter acquisition times, which makes tractography a suitable tool for clin-
ical populations, including children with developmental language disorders (Rimrodt,
Peterson, Denckla, Kaufmann, & Cutting, 2010) and adults with stroke (Forkel et al.,
2014) or neurodegenerative disorders (Catani et al., 2013; D’Anna et al., 2016).
The main assumption underpinning diffusion tensor tractography is that the diffu-
sion of water molecules can be described mathematically by a diffusion tensor whose
principal axis aligns with the predominant orientation of the fibers contained within
each voxel (Basser, Mattiello, & Le Bihan, 1994). Based on this assumption, tractography
algorithms track white matter trajectories by inferring axonal continuity from voxel
to voxel (Figure 9.1). In simple terms, this process is achieved by following the direc-
tion of maximum diffusion from a given voxel into a neighboring voxel (Basser, Pajevic,
Pierpaoli, Duda, & Aldroubi, 2000; Conturo et al., 1999; Jones et al., 1999; Mori et al.,
1999; Poupon et al., 2000). How to piece together discrete estimates of water diffusion
between contiguous voxels depends on the algorithm used and the choice of some
tracking and stopping parameters. Most tractography algorithms adopt angular and FA
thresholds to avoid unrealistic fiber bending or tracking outside of white matter regions.
Diffusion tractography can be used to generate indirect measures of tract volume
and microstructural properties of fibers. Common measures of tract volume are
the overall number of streamlines that compose a single tract, or the total volume of
voxels intersected by a tract (Dell’Acqua & Catani, 2012). Tractography-derived inter-
hemispheric differences in tract volume are widely reported in the literature, especially
for language pathways (Catani et al., 2007), although their exact interpretation is not
straightforward. Histological properties that are likely to determine larger tract volumes
include axonal diameter and myelination of fibers, high axonal density and fiber disper-
sion, and the presence of fiber crossing and branching.
In addition to tract volume, for each voxel intersected by streamlines, other diffu-
sion indices can be extracted and the total average can be extrapolated. Examples in-
clude the fractional anisotropy, mean diffusivity, and parallel and radial diffusivity.
These can provide important information on the microstructural properties of fibers
and their organization. Asymmetry in fractional anisotropy, for example, could indi-
cate differences in the axonal anatomy (intraxonal composition, axon diameter, and
membrane permeability), fiber myelination (myelin density, internodal distance, and
myelin distribution), or fiber arrangement and morphology (axonal dispersion, ax-

onal crossing, and axonal branching) (Beaulieu, 2002; Concha, 2014). Other diffusion
measurements may reveal more specific fiber properties. Changes in axial diffusivity
measurements, for example, could be related to intraxonal composition, while radial
diffusivity may be more sensitive to changes in membrane permeability and myelin
density (Song et al., 2002). These in vivo diffusion-based measurements allow connec-
tional anatomy to be defined at different scales from microstructure to gross anatomy
for individual tracts and across the lifespan.
Tractography-Based Models
of Language Networks
In the past 15 years, tractography has greatly contributed to contemporary revisions of

anatomical models of language networks. New pathways, which were not previously
described in animal models, have been identified and their anatomy characterized in
the human brain by tractography (Figure 9.2) (Catani et al., 2005; Catani et al., 2012;
Lawes et al., 2008).
In general, tractography has identified two sets of white matter connections for lan-
guage and social communication: a core network encompassing classical perisylvian
language regions, and an extended network connecting perisylvian regions to other
cortical and subcortical hubs (Catani & Bambini, 2014; Catani & Mesulam, 2008). The
paragraphs that follow do not represent a comprehensive account of the literature on the
tracts forming the core and extended language networks, but are intended to provide
examples of how tractography has been used to address specific questions in healthy
cohorts and in patients with language disorders.
The Core Language Network

The arcuate fasciculus is a dorsal perisylvian tract connecting Wernicke’s region in the
posterior temporal lobe to Broca’s region in the inferior frontal lobe (Figure 9.2 A). Early
tractography studies showed that the classical Wernicke-Broca-Geschwind model was
an oversimplification of the real anatomy (Catani et al., 2005, Figure 9.2 B). Indeed, two
parallel pathways within the arcuate fasciculus have been identified: the medial, direct
pathway connecting Wernicke’s region with Broca’s region (i.e., the arcuate fasciculus
sensu strictu or long segment), and the indirect pathway consisting of an anterior seg-
ment linking Broca’s region to Geschwind’s region (encompassing the angular and
supramarginal gyrus in the inferior parietal lobe) and a posterior segment between
Geschwind’s region and Wernicke’s region. There is evidence that this anatomical divi-
sion has important functional implications (Catani & Mesulam, 2008).
(A)
Arcuate Fasciculus
Anterior segment
Long segment
(B)
Posterior segment
Frontal Aslant Tract
Uncinate fasciculus
Temporal longitudinal fasciculus (TLF)

(C)
Inferior fronto-occipital fasciculus (IFOF)
Inferior longitudinal fasciculus (ILF)
Figure 9.2. Tractography-based reconstruction of the language networks from classical

models to contemporary neurolinguistics. (A) Classical language model with the arcuate
fasciculus connecting Broca’s region in the inferior frontal gyrus to Wernicke’s region in the su-
perior temporal gyrus. (B) Extension of the classical arcuate fasciculus sense strictu to include the
anterior segment, connecting inferior frontal to inferior parietal lobe, and the posterior segment
linking the inferior parietal to the temporal lobe. (C) Current model of an extended language net-
work beyond the three segments of the arcuate fasciculus. The frontal aslant tract (FAT) connects
the inferior frontal gyrus to the pre-supplementary motor cortex. The ventral network includes
the uncinate fasciculus between the anterior temporal lobe and the orbital frontal and inferior
frontal cortex, the temporal longitudinal fasciculus (TLF) between the posterior temporal lobe
and the temporal pole, the inferior fronto-occipital fasciculus (IFOF) connecting the ventral
frontal cortex to the occipital cortex, and the inferior longitudinal fasciculus (ILF) between the
occipital and anterior temporal cortex.
Lopez- Barroso et al. (2013) demonstrated that performance in word learning
correlates with microstructural properties (as measured with DTI) and strength of
functional connectivity (as measured with fMRI) of the left direct segment. In addition,
the long segment tends to myelinate later in childhood, and its maturation is associated
with the acquisition of syntactic abilities (Brauer, Anwander, Perani, & Friederici, 2013).
These studies indicate that our ability to learn new words and develop syntax relies on an
efficient and fast communication between auditory temporal and motor frontal regions.
The presence of a less prominent long segment in nonhuman primates might explain
human linguistic specialization based on an exceptional auditory memory unique to
our species (López-Barroso et al., 2013; Thiebaut de Schotten, Dell’Acqua, Valabregue, &
Catani, 2012).
While the direct pathway may support auditory-motor integration, which is cru-
cial during early stages of language acquisition, the role of the indirect pathway
and Geschwind’s region could be more complex and related to linking semantic and
phonological processes (Newhart et al., 2012) for tasks that require verbal working
memory to understand complex sentences (Jacquemot & Scott, 2006). In addition,
the temporoparietal regions connected by the posterior segment activate in tasks
for the comprehension of ambiguous sentences (e.g., garden-path paradigms) (den
Ouden, Walsh Dickey, Anderson, & Christianson, 2016), metaphors (Bambini, Gentili,
Ricciardi, Bertinetto, & Pietrini, 2011), and indirect speech acts (Bašnáková, Weber,
Petersson, van Berkum, & Hagoort, 2014), as well as for tasks that involve the represen-
tation of discourse and the protagonist’s perspective in narratives (Mason & Just, 2009).
A recent model for social communication and language evolution and development
(SCALED) suggests that the posterior network supports complex integration and infer-
ential mechanisms that reach several layers of meta-representations for the attribution
of beliefs and emotional states to conversational partners (Catani & Bambini, 2014).
Among all tracts of the human brain, the arcuate fasciculus displays the greatest de-
gree of inter-hemispheric and inter-individual asymmetry. By extracting volumetric
measurements of the three segments of the arcuate fasciculus, it has been demonstrated that
the long segment is strongly left lateralized in 60% of the population, whereas the remaining
40% show a bilateral pattern. The bilateral pattern seems to be more prevalent among the fe-
male population as compared to males. In the extremely left lateralized group, males repre-
sent 68% while females account for 32%, in contrast to the bilateral group with 80% females
but only 20% males. Moreover, the pattern of asymmetry correlated with performances on
the California Verbal Learning Test (CVLT), a verbal memory task that relies on semantic
clustering for word learning and retrieval; the correlation indicated that a more bilateral
representation was advantageous for the retrieval of word lists (Catani et al., 2007).
A better understanding of the pattern of asymmetry of the long segment has im-
portant implications. First, it offers a neuroanatomical explanation for the observed
performances of females over males on verbal learning tasks (Kramer, Delis, & Daniel,
1988). Second, the high variability of asymmetry in the general population can help to
identify different trajectories to language recovery in patients with aphasia after left
hemisphere stroke. This was demonstrated employing tractography in a longitudinal
study of aphasia recovery in which the volumetric measurements of the long segment
in the right hemisphere were predictive of aphasia recovery six months after stroke
(Forkel et al., 2014). Tractography measurements of the volume of the right long seg-
ment improved the predictive value for recovery above and beyond models accounting
for demographics alone by explaining an additional 30% of the observed variance in re-
covery at six-months post stroke (Forkel et al., 2014).
The Extended Language Network

The extended language network is composed of several tracts, including the uncinate
fasciculus, the frontal aslant tract, the inferior fronto-occipital fasciculus, the inferior
longitudinal fasciculus, the temporal longitudinal fasciculus and the fronto-insular
tracts (Figure 9.2 C).
The uncinate fasciculus, which connects the anterior temporal lobe to the orbito-
frontal region and part of the inferior frontal gyrus (Catani, Howard, Pajevic, & Jones,
2002), is classically considered to be a major pathway of the limbic system (Catani,
Dell’Acqua, & Thiebaut de Schotten, 2013). In addition to its role in emotion processing
and behavior, the uncinate fasciculus has been associated with tasks involving lexical re-
trieval, semantic association, and naming (Heide, Skipper, Klobusicky, & Olson, 2013).
The uncinate fasciculus is severely damaged in patients with the semantic variant of pri-
mary progressive aphasia, and the severity of its degeneration correlates with scores on
tests for naming (Catani et al., 2013).
The frontal aslant tract is a recently described pathway connecting Broca’s region with
dorsal medial frontal areas, including the pre-supplementary motor area and cingulate
cortex (Catani et al., 2012; Ford, McGregor, Case, Crosson, & White, 2010; Lawes et al.,
2008). Medial regions of the frontal lobe facilitate speech initiation through direct con-
nection to the precentral gyrus and the pars opercularis and triangularis of the inferior
frontal gyrus. Patients with lesions to these areas present with various degrees of speech
impairment, from a total inability to initiate speech (i.e., mutism) to mildly altered flu-
ency. Tractography studies have demonstrated damage to the frontal aslant tract in
patients with the nonfluent/agrammatic form of primary progressive aphasia (Catani
et al., 2013; Mandelli et al., 2014).
The inferior fronto- occipital fasciculus and the inferior longitudinal fasciculus
convey visual inputs to the frontal and anterior temporal lobe, respectively. The pro-
posal of a prominent role of these two tracts in semantic cognition has been originally
suggested by intraoperative stimulation studies reporting the observation of anomia
for stimulation of the white matter fibers running within the external/extreme cap-
sule or the anterior temporal regions (see Duffau, Chapter 8 in this volume). This pro-
posal is challenged by the prominent role of the anterior temporal cortex in semantic
cognition as demonstrated in patients with the semantic variant of primary progres-
sive aphasia (Catani et al., 2013) and the lack of cases of anomic deficits in patients with
stimulation of the most posterior portion of the inferior fronto-occipital fasciculus and
inferior longitudinal fasciculus. Also the inability of determining the exact fiber lo-
calization during intraoperative stimulation raises the question of whether other
tracts running in parallel with the inferior fronto-occipital fasciculus or the inferior
longitudinal fasciculus have been stimulated instead. Among these tracts the recently
described temporal longitudinal fasciculus connecting Wernicke’s region to the ante-
rior temporal pole is likely to be involved in single word comprehension and retrieval
(Maffei et al., 2017).
The frontal operculum is connected to the insula through a system of short U-shaped
fronto-insular tracts (Catani et al., 2012). Direct insular inputs to Broca’s region from
the insula provide visceral and emotional information for speech output modulation
according to internal states. Lesions to these insular connections may result in motor
aprosodia (e.g., flat intonation) in the right hemisphere (Witteman et al., 2014), while in
the left hemisphere it may be associated with apraxia of speech (Dronkers, 1996).
Limitations and Future Directions

The advent of diffusion tractography signified a fundamental advancement of our
understanding of networks underpinning language functions. The ability to track
connections in the living human brain offers the possibility to move beyond network
models based on axonal tracing in monkeys and perform direct clinical-anatomical
correlations in the living human brain. Despite the obvious advantages, tractography
has several limitations, some of which will be briefly discussed in the following.
Compared to classical axonal tracing studies, tractography is unable to differentiate
anterograde and retrograde connections, detect the presence of synapses, or determine
whether a pathway is functional. In addition, while injected tracers follow the termina-
tion of single axons, tractography follows the principal axis of the water diffusion, which
is obtained by averaging the MRI signal within a voxel. Typically, the voxel resolution is
too low to identify small fiber bundles. The level of noise in the diffusion data and the in-
trinsic MRI artifacts constitute important factors that affect the precision and accuracy of
the diffusion measurements and, as a consequence, the quality of the tractography recon-
struction (Basser et al., 2000; Le Bihan et al., 2006). Finally, diffusion tensor tractography
assumes that fibers in each voxel are well described by a single orientation estimate,
which is a valid assumption for voxels containing only one population of fibers with a
similar orientation. The majority of white matter voxels, however, contain populations
of fibers with multiple orientations; in these regions fibers cross, kiss, merge, or di-
verge, and the tensor model is inadequate to capture this anatomical complexity (Basser
et al., 2000). More recent tractography developments based on HARDI (high angular
resolution diffusion imaging) methods (Dell’Acqua et al, 2010; Dell’Acqua & Tournier,
2018; Frank, 2001) and appropriate processing techniques are able to partially resolve
fiber crossings (Tournier, Calamante, Gadian, & Connelly, 2004; Tuch, 2004; Wedeen
et al., 2005; Alexander, 2005; Behrens et al., 2007). Among the HARDI methods, spher-
ical deconvolution algorithms (Figure 9.3) have been used to reconstruct white matter
Diffusion Tensor Imaging Spherical Deconvolution
(A)
FA HMOA
(B)
1.00 1.18
0.75 0.13
0.50 0.09
0.25 0.05
0.00 0.01
(C)
Figure 9.3. Comparison of diffusion tensor imaging (DTI) and advanced diffusion

tractography based on spherical deconvolution. (A) Visualization of voxel-wise fibre orientation
in a coronal slice passing through the corpus callosum using diffusion tensor imaging (left) and
spherical deconvolution (right). The shape and orientation of the tensor model indicates the av-
erage diffusion properties and orientation of all fibre populations contained within a voxel. The
spherical deconvolution model differentiates between different fiber populations within each
voxel and gives fiber specific information . (B) Eigenvector map visualization of the white matter
organization of the corpus callosum and the corona radiata based on the tensor model (left) and
on spherical deconvolution (right). The arcuate fasciculus and the lateral projections of the corpus
callosum are distinct tracts that cross at the level of the corona radiata where their fibers occupy
the same voxels. Fractional anisotropy values extracted for each voxel that contain crossing fibers
indicate low values for both tracts. The low FA values are due to the weak anisotropic properties of
the voxels containing crossing fibers. Spherical deconvolution permits to calculate the hindrance
modulated orientational anisotropy (HMOA) index, which is a measure of tissue anisotropy spe-
cific of each fiber population occupying a voxel. In this example the fibers of the arcuate fasciculus
and corpus callosum that cross at the level of the corona radiata show different HMOA values
even when they occupy the same voxel. (C) Virtual dissections of the corpus callosum (red) and
corticospinal tracts (yellow) based on diffusion tensor tractography (left) and spherical convolu-
tion (right). The diffusion tensor reconstruction of the corpus callosum is limited only to its most
central part while tractography based on spherical deconvolution (right) shows streamlines of the
corpus callosum that cross the corticospinal tracts and reach the lateral cortex.
pathways in regions with multiple fiber orientations, such as in the triangle between
the corpus callosum, the superior longitudinal fasciculus, and the cortico-spinal tract
(Thiebaut de Schotten et al., 2011; Dell'Acqua, Simmons, Williams, & Catani, 2013).
All these limitations may lead to tracking pathways that do not exist (false posi-
tive) or fail to track those that do exist (false negative). A few studies have so far dealt
with the issue of validating the tractography results with neuronal tracers (Dauguet
et al., 2007; Dyrby et al., 2007) or performing reproducibility analysis on human
subjects using post-mortem blunt dissections or diffusion data sets acquired at high
spatial resolution (Heiervang et al., 2006; Lawes et al., 2008; Wakana et al., 2007;
Catani et al., 2012b).
It is evident from all the considerations in the preceding that interpretation of
tractography results requires experience and a priori anatomical knowledge. This
is particularly true in the diseased brain, where alteration and anatomic distor-
tion due to the presence of pathology, such as brain edema, hemorrhage, and com-
pression, generates tissue changes likely to lead to a greater number of artifactual
reconstructions (Catani, 2006; Ciccarelli, Catani, Johansen-Berg, Clark, & Thompson,
2008; Dell’Acqua & Catani, 2012).
The recent development of MRI scanners with stronger gradients and multi-band
acquisition sequences represents one of many steps toward a significant amelioration
of the diffusion tractography approach. The possibility of combining tractography
with other imaging modalities or direct electrical stimulation methods will
contribute to a more complete picture of the functional anatomy of human language
pathways.
References
Alexander, D. C. (2005). Multiple-fiber reconstruction algorithms for diffusion MRI. Annals
of the New York Academy of Sciences, 1064, 113–133.
Bambini, V., Gentili, C., Ricciardi, E., Bertinetto, P. M., & Pietrini, P. (2011). Decomposing
metaphor processing at the cognitive and neural level through functional magnetic reso-
nance imaging. Brain Research Bulletin, 86(3–4), 203–216.
Basser, P. J., Mattiello, J., & Le Bihan, D. (1994). MR diffusion tensor spectroscopy and imaging.
Biophysical Journal, 66(1), 259–267.
Basser, P. J., Pajevic, S., Pierpaoli, C., Duda, J., & Aldroubi, A. (2000). In vivo fiber tractography
using DT-MRI data. Magnetic Resonance in Medicine, 44(4), 625–632.
Basso, A., Lecours, A. R., Moraschini, S., & Vanier, M. (1985). Anatomoclinical correlations
of the aphasias as defined through computerized tomography: Exceptions. Brain and
Language, 26(2), 201–229.
Bašnáková, J., Weber, K., Petersson, K. M., van Berkum, J., & Hagoort, P. (2014). Beyond the
language given: The neural correlates of inferring speaker meaning. Cerebral Cortex, 24(10),
2572–2578.
Beaulieu, C. (2009). The biological basis of diffusion anisotropy. In Johansen-Berg, H. &
Behrens, T. E. (Eds.), Diffusion MRI: From quantitative measurement to in vivo neuroa-
natomy (pp. 105–126). London: Elsevier.
Beaulieu, C. (2002). The basis of anisotropic water diffusion in the nervous system: A technical
review. NMR in Biomedicine, 15(7–8), 435–455.
Behrens, T. E., Berg, H. J., Jbabdi, S., Rushworth, M. F., & Woolrich, M. W. (2007). Probabilistic
diffusion tractography with multiple fibre orientations: What can we gain? NeuroImage,
34(1):144–155.
Le Bihan, D., Breton, E., Lallemand, D., Grenier, P., Cabanis, E., & Laval-Jeantet, M. (1986).
MR imaging of intravoxel incoherent motions: Application to diffusion and perfusion in
neurologic disorders. Radiology, 161(2), 401–407.
Le Bihan, D., Poupon, C., Amadon, A., & Lethimonnier, F. (2006). Artifacts and pitfalls in dif-
fusion MRI. Journal of Magnetic Resonance Imaging, 24(3), 478–488.
Brauer, J., Anwander, A., Perani, D., & Friederici, A. D. (2013). Dorsal and ventral pathways in
language development. Brain and Language, 127(2), 289–295.
Broca, P. (1865). Sur le siege de la faculte du langage articule. Bulletin de la Société d’Anthropologie
de Paris, 6, 377–393.
Budisavljevic, S., Dell’Acqua, F., Rijsdijk, F. V., Kane, F., Picchioni, M., McGuire, P., et al. (2015).
Age-related differences and heritability of the perisylvian language networks. The Journal of
Neuroscience, 35(37), 12625–12634.
Cappa, S. F., Perani, D., Grassi, F., Bressi, S., Alberoni, M., Franceschi, M., et al. (1997). A PET
follow-up study of recovery after stroke in acute aphasics. Brain and Language, 56(1),  55–67.
Catani, M. (2006). Diffusion tensor magnetic resonance imaging tractography in cognitive
disorders. Current Opinion in Neurology, 19(6), 599–606.
Catani, M., Allin, M. P., Husain, M., Pugliese, L., Mesulam, M. M., Murray, R. M., & Jones,
D. K. (2007). Symmetries in human brain language pathways correlate with verbal recall.
Catani, M., & Bambini, V. (2014). A model for Social Communication And Language Evolution
and Development (SCALED). Current Opinion in Neurobiology, 28, 165–171.
Catani M, Dell’Acqua F, Thiebaut de Schotten M. 2013. A revised limbic system model for
memory, emotion and behaviour.pdf Neurosci Biobehav Rev 37:1724–1737.
Catani, M., Dell’Acqua, F., Bizzi, A., Forkel, S. J., Williams, S. C., Simmons, A., Murphy, D. G., &
Thiebaut de Schotten, M. (2012a). Beyond cortical localization in clinico-anatomical corre-
lation. Cortex, 48(10), 1262–1287.
Catani, M., Dell’Acqua, F., Vergani, F., Malik, F., Hodge, H., Roy, P., et al. (2012b). Short frontal
lobe connections of the human brain. Cortex, 48(2), 273–291.
Catani, M., & ffytche, D. H. (2005). The rises and falls of disconnection syndromes. Brain,
128(10), 2224–2239.
Catani, M., Howard, R. J., Pajevic, S., & Jones, D. K. (2002). Virtual in vivo interactive dissec-
tion of white matter fasciculi in the human brain. NeuroImage, 17(1),  77–94.
Catani, M., Jones, D. K., & ffytche, D. H. (2005). Perisylvian language networks of the human
brain. Annals of Neurolology, 57(1), 8–16.
Catani, M., & Mesulam, M. (2008). The arcuate fasciculus and the disconnection theme in lan-
guage and aphasia: History and current state. Cortex, 44(8), 953–961.
Catani, M., Mesulam, M. M., Jakobsen, E., Malik, F., Martersteck, A., Wieneke, C., et al. (2013).
A novel frontal pathway underlies verbal fluency in primary progressive aphasia. Brain,
136(8), 2619–2628.
Ciccarelli, O., Catani, M., Johansen-B erg, H., Clark, C. A., & Thompson, A. (2008).
Diffusion-based tractography in neurological disorders: Concepts, applications, and
future developments. Lancet Neurology, 7(8), 715–727.
Concha, L. (2014). A macroscopic view of microstructure: Using diffusion-weighted images to

infer damage, repair, and plasticity of white matter. Neuroscience, 276, 14–28.
Conturo, T. E., Lori, N. F., Cull, T. S., Akbudak, E., Snyder, A. Z., Shimony, J. S., et al. (1999).
Tracking neuronal fiber pathways in the living human brain. Proceedings of the National
Academy of Sciences USA, 96(18), 10422–10427.
Dauguet, J., Peled, S., Berezowski, V., Delzescaux, T., Warfield, S. K., Born, R., & Westin, C. F.
(2007). Comparison of fibre tracts derived from in-vivo DTI tractography with 3D histolog-
ical neural tract tracer reconstruction on a macaque brain. NeuroImage, 37(2), 530–538.
D’Anna, L., Mesulam, M. M., Thiebaut de Schotten, M., Dell’Acqua, F., Murphy, D., Wieneke,
C., et al. (2016). Frontotemporal networks and behavioral symptoms in primary progressive
aphasia. Neurology, 86(15), 1393–1399.
Dell’Acqua, F., & Catani, M. (2012). Structural human brain networks: Hot topics in diffusion
tractography. Current Opinion in Neurology, 25(4), 375–383.
Dell’Acqua, F., Scifo, P., Rizzo, G., Catani, M., Simmons, A., Scotti, G., & Fazio, F. (2010). A
modified damped Richardson-Lucy algorithm to reduce isotropic background effects in
spherical deconvolution. NeuroImage, 49(2), 1446–1458.
Dell’Acqua, F., Simmons, A., Williams, S. C., & Catani, M. (2013). Can spherical deconvolu-
tion provide more information than fiber orientations? Hindrance modulated orientational
anisotropy, a true-tract specific index to characterize white matter diffusion. Human Brain
Mapping, 34(10), 2464–2683.
Dell’Acqua, F., Tournier, J. D. (2018). Modelling white matter with spherical deconvolu-
tion: How and why? NMR Biomedicine, e3945. doi: 10.1002/nbm.3945
Derby, T., Søgaard, L., Parker, G. (2007). Validation of in vitro probabilistic tractography.
NeuroImage, 37(4), 1267–1277.
DeWitt, L. D., Grek, A. J., Buonanno, F. S., Levine, D. N., & Kistler, J. P. (1985). MRI and the
study of aphasia. Neurology, 35(6), 861–865.
Dronkers, N. F. (1996). A new brain region for coordinating speech articulation. Nature,
384(6605), 159–161.
Ford, A., McGregor, K. M., Case, K., Crosson, B., & White, K. D. (2010). Structural connectivity
of Broca’s area and medial frontal cortex. NeuroImage, 52(4), 1230–1237.
Forkel, S. J., Thiebaut de Schotten, M., Dell’Acqua, F., Kalra, L., Murphy, D. G., Williams, S. C.,
& Catani, M. (2014). Anatomical predictors of aphasia recovery: A tractography study of bi-
lateral perisylvian language networks. Brain, 137(7), 2027–2039.
Frank L. Anisotropy in high angular resolution diffusion-tensor MRI. (2001). Magnetic
Resonance in Medicine, 45, 935–939.
Friederici, A. D., Cramon, von, D. Y., & Kotz, S. A. (1999). Language related brain potentials in
patients with cortical and subcortical left hemisphere lesions. Brain, 122(6), 1033–1047.
Geschwind, N. (1965). Disconnexion syndromes in animals and man. I. Brain, 88(2), 237–294.
Hari, R., & Lounasmaa, O. V. (1989). Recording and interpretation of cerebral magnetic fields.
Science, 244(4903), 432–436.
Heiervang, E., Behrens, T. E., Mackay, C. E., Robson, M. D., & Johansen-Berg, H. (2006).
Between session reproducibility and between subject variability of diffusion MR and
tractography measures. NeuroImage, 33, 867–877.
Heide, Von Der, R. J., Skipper, L. M., Klobusicky, E., & Olson, I. R. (2013). Dissecting the unci-
nate fasciculus: Disorders, controversies and a hypothesis. Brain, 136(6), 1692–1707.
Jacquemot, C., & Scott, S. K. (2006). What is the relationship between phonological short-term
memory and speech processing? Trends in Cognitive Sciences, 10(11), 480–486.
Jones, D. K., Simmons, A., Williams, S. C., & Horsfield, M. A. (1999). Non-invasive assess-
ment of axonal fiber connectivity in the human brain via diffusion tensor MRI. Magnetic
Resonance in Medicine, 42(1),  37–41.
Kramer, J. H., Delis, D. C., & Daniel, M. (1988). Sex differences in verbal learning. Journal of
Clinical Psychology, 44(6), 907–915.
Lawes, I. N. C., Barrick, T. R., Murugam, V., Spierings, N., Evans, D. R., Song, M., & Clark, C. A.
(2008). Atlas-based segmentation of white matter tracts of the human brain using diffusion
tensor tractography and comparison with classical dissection. NeuroImage, 39(1), 62–79.
Lebel, C., Walker, L., Leemans, A., Phillips, L., & Beaulieu, C. (2008). Microstructural matura-
tion of the human brain from childhood to adulthood. NeuroImage, 40(3), 1044–1055.
López-Barroso, D., Catani, M., Ripollés, P., Dell’Acqua, F., Rodríguez-Fornells, A., & de Diego-
Balaguer, R. (2013). Word learning is mediated by the left arcuate fasciculus. Proceedings of
Maffei, C., Capasso, R., Cazzolli, G., Colosimo, C., Dell’Acqua, F., Piludu, F., Catani, M., &
Miceli, G. (2017). Pure word deafness following left temporal damage: Behavioral and neu-
roanatomical evidence from a new case. Cortex, 97, 240–254.
Mandelli, M. L., Caverzasi, E., Binney, R. J., Henry, M. L., Lobach, I., Block, N., et al. (2014).
Frontal white matter tracts sustaining speech production in primary progressive aphasia.
Journal of Neuroscience, 34(29), 9754–9767.
Mason, R. A., & Just, M. A. (2009). The role of the theory-of-mind cortical network in the com-
prehension of narratives. Language and Linguistics Compass, 3(1), 157–174.
McCarthy, G., Blamire, A. M., Rothman, D. L., Gruetter, R., & Shulman, R. G. (1993).
Echo-planar magnetic resonance imaging studies of frontal cortex activation during
word generation in humans. Proceedings of the National Academy of Sciences USA, 90(11),
4952–4956.
Mori, S., Crain, B. J., Chacko, V. P., & van Zijl, P. C. (1999). Three-dimensional tracking of ax-
onal projections in the brain by magnetic resonance imaging. Annals of Neurology, 45(2),
265–269.
Naeser, M. A., & Hayward, R. W. (1978). Lesion localization in aphasia with cranial computed
tomography and the Boston Diagnostic Aphasia Exam. Neurology, 28(6), 545–551.
Newhart, M., Trupe, L. A., Gomez, Y., Cloutman, L., Molitoris, J. J., Davis, C., et al. (2012).
Asyntactic comprehension, working memory, and acute ischemia in Broca’s area versus an-
gular gyrus. Cortex, 48(10), 1288–1297.
den Ouden, D. B., Walsh Dickey, M., Anderson, C., & Christianson, K. (2016). Neural correlates
of early-closure garden-path processing: Effects of prosody and plausibility. Quarterly
Journal of Experimental Psychology, 69, 926–949.
Pajevic, S., & Pierpaoli, C. (1999). Color schemes to represent the orientation of anisotropic
tissues from diffusion tensor data: Application to white matter fiber tract mapping in the
human brain. Magnetic Resonance in Medicine, 42(3), 526–540.
Perani, D., Vallar, G., Cappa, S., Messa, C., & Fazio, F. (1987). Aphasia and neglect after sub-
cortical stroke. A clinical/cerebral perfusion correlation study. Brain, 110(5), 1211–1229.
Pierpaoli, C., Jezzard, P., Basser, P. J., Barnett, A., & Di Chiro, G. (1996). Diffusion tensor MR
imaging of the human brain. Radiology, 201(3), 637–648.
Poupon, C., Clark, C. A., Frouin, V., Regis, J., Bloch, I., Le Bihan, D., & Mangin, J. (2000).
Regularization of diffusion-based direction maps for the tracking of brain white matter
fascicles. NeuroImage, 12(2), 184–195.
Rimrodt, S. L., Peterson, D. J., Denckla, M. B., Kaufmann, W. E., & Cutting, L. E. (2010).
White matter microstructural differences linked to left perisylvian language network
in children with dyslexia. Cortex, 46(6), 739–749.
Salmelin, R. (2007). Clinical neurophysiology of language: The MEG approach. Clinical
Song, S.-K., Sun, S.-W., Ramsbottom, M. J., Chang, C., Russell, J., & Cross, A. H. (2002).
Dysmyelination revealed through MRI as increased radial (but unchanged axial) diffusion
of water. NeuroImage, 17(3), 1429–1436.
Thiebaut de Schotten, M., Dell’Acqua, F., Forkel, S. J., Simmons, A., Vergani, F., Murphy, D.
G., & Catani, M. (2011). A lateralized brain network for visuospatial attention. Nature
Neuroscience, 14(10), 1245–1246.
human comparative anatomy of the frontal lobe association tracts. Cortex, 48(1), 82–96.
Tikofsky, R. S., Kooi, K. A., & Thomas, M. H. (1960). Electroencephalographic findings and re-
covery from aphasia. Neurology, 10, 154–156.
Tournier, J. D., Calamante, F., Gadian, D. G., & Connelly, A. (2004). Direct estimation of the
fiber orientation density function from diffusion-weighted MRI data using spherical de-
convolution. NeuroImage, 23(3), 1176–1185.
Turken, A. U., & Dronkers, N. F. (2011). The neural architecture of the language comprehension
network: Converging evidence from lesion and connectivity analyses. Frontiers in Systems
Neuroscience, 5, 1.
Wakana, S, Caprihan, A, Panzenboeck, M. M., Fallon J. H., Perry, M., Gollub, R. L., Hua, K.,
Zhang, J., Jiang, H., Dubey, P., Blitz, A., van Zijl, P., & Mori, S. (2007). Reproducibility of
quantitative tractography methods applied to cerebral white matter. NeuroImage, 36(3),
630–644.
Wedeen, V. J., Hagmann, P., Tseng, W. Y., Reese, T. G., & Weisskoff, R. M. (2005). Mapping
complex tissue architecture with diffusion spectrum magnetic resonance imaging. Magnetic
Resonance in Medicine, 54(6), 1377–1386.
Wernicke, C. (1874). Der Aphasische Symptomencomplex: Eine psychologische Studie auf
anatomischer Basis. Breslau: Cohn & Weigert.
Wise, R., Hadar, U., Howard, D., & Patterson, K. (1991). Language activation studies with
positron emission tomography. Ciba Foundation Symposium, 163, 218–28–discussion
228–234.
Witteman, J., Goerlich-Dobre, K. S., Martens S., Aleman A., Van Heuven, V. J., & Schiller, N. O.
(2014). The nature of hemispheric specialization for prosody perception. Cognitive Affective
& Behavioral Neuroscience, 14(3), 1104–1114.
Yarnell, P., Monroe, P., & Sobel, L. (1976). Aphasia outcome in stroke: A clinical neuro
radiological correlation. Stroke, 7(5), 516–522.
Pa rt   I I
DE V E L OP M E N T
A N D P L A ST IC I T Y
Chapter 10
Neu ropl ast i c i t y

Language and Emotional Development
in Children with Perinatal Stroke
Judy S. Reilly and Lara R. Polse
The term “plasticity” derives from the ancient Greek plasso (platho in modern Greek,
πλάθω) and is often employed in reference to brain adaptation and reorganization in re-
sponse to a cerebral injury. Plasticity in reference to the nervous system first appeared
when William James (1890) proposed that the human brain has the capacity to change
and reorganize, noting that “[o]‌rganic matter, especially nervous tissue, seems endowed
with a very extraordinary degree of plasticity.” In the early twentieth century, Ramon
y Cajal (1904) extended this notion by suggesting that new behavioral learning was
supported by neuronal change and development. Such statements suggest that plas-
ticity is not only a response to insult, but rather that plasticity is characteristic of, and
functions as, the neural mechanism supporting learning and development throughout
the life span (Pascual-Leone, Amedi, Fregni, & Merabet, 2005; Stiles, Reilly, Levine,
Trauner, & Nass, 2012). In the young child, as tissue is presumably less specified and
neurons less committed to specific functions, neuroplasticity has a wider potential in
the developing brain than in an adult brain that has already committed neural resources
to various cognitive functions. Children who have suffered a unilateral focal lesion be-
fore their first month of postnatal life represent an extraordinary case in which the brain
must adapt and (re)organize in the face of this early lesion. As such, the development of
these children with perinatal stroke (PS) provides a unique context to elucidate the na-
ture, extent, and limits of neuroplasticity.
When we began our studies on children with PS about 30 years ago, we looked to
the adult model as a means to understand brain organization for language, as the adult
model represents the steady or end state of development. As the majority of chapters
in this volume attest, we currently know a great deal concerning the neural substrates
that participate in language processing in adults (e.g., see, in this volume, Peelle,
Chapter 12; Rapp & Purcell, Chapter 17; de Zubicaray & Piai, Chapter 19; Paz-Alonso,
232 Judy S. Reilly and Lara R. Polse
Oliver, Quiñones, & Carreiras, Chapter 24; Bornkessel-Schlesewsky & Schlesewsky,

Chapter 27; Catani, Jones, & ffytche, 2005; Price, 2010; Vigneau et al., 2006). An exten-
sive literature of imaging and neuropsychological studies on neurotypical and stroke
patients over the last 30 years has broadly confirmed and refined the original findings of
Broca and Wernicke: that there exists a predominantly left hemisphere fronto-temporal
neural network for language processing. In contrast to this broad and extensive liter-
ature on the organization of language in the adult brain, we are just beginning to un-
derstand how the developing brain achieves this mature state. An early comment from
Jules Cotard, a colleague of Broca, suggested that strokes early in development do not
have the same consequences as later strokes: “Intelligence may be normal when a hem-
isphere is destroyed during infancy . . . in these cases one never encounters aphasia”
(translated and quoted in Pearn & Gardner-Thorpe, 2002, p. 1401). Almost a century
later, Basser (1962) and Lenneberg (1967) presented empirical evidence in support of
this notion, leading Lenneberg to hypothesize that the two hemispheres of the brain
were initially equipotential for language. More than twenty-five years of prospective
studies chronicling the linguistic development of children with PS (Bates & Roe, 2001;
Bates et al., 1997; Feldman, 1994; Feldman, Holland, Kemp, & Janosky, 1992; Reilly, Bates,
& Marchman, 1998; Reilly, Levine, Nass, & Stiles, 2008; Reilly, Losh, Bellugi, & Wulfeck,
2004; Reilly, Wasserman, & Appelbaum, 2013; Rowe, Levine, Fisher, & Goldin-Meadow,
2009; Stiles, Reilly, Levine, Trauner, & Nass, 2012; Thal et al., 1991; Thal, Reilly, Seibert,
Jeffries, & Fenson, 2004) have demonstrated that, unlike adults with homologous stroke,
children exhibit remarkable progress in language acquisition after PS. In contrast, the
developmental trajectories for other cognitive systems (e.g., spatial cognition and affec-
tive expression) in these same children have shown subtle, but persistent deficits remi-
niscent of adults with late-onset strokes (Stiles et al., 2012). In this chapter, we present
findings on language development in children who sustained a pre-or perinatal uni-
lateral stroke, and to complement these studies, we also look at the development of an-
other communicative system, affective expression, in these same children, as it reflects a
rather different developmental trajectory. These prospective studies of this rare group of
children (1/4,000 live births; Lynch & Nelson, 2001) provide a unique window into the
development of the neural substrates for language and affect. Specifically, they afford a
context to investigate the degree to which particular brain regions may be privileged for
specific behavioral functions, as well as how the developing brain adapts to organize al-
ternative pathways in the wake of an early insult.
The fields of linguistics and affective science both have long, but rather independent
histories (see van Berkum, Chapter 29 in this volume). However, in our view, the devel-
opment of affective expression provides an intriguing and informative complement to
language. As adult speakers, we consider language to be our primary communicative
system; however, each utterance is produced and interpreted in an emotional context.
We can color or modulate our spoken utterances lexically, as well as with prosody and fa-
cial expression. As such, for adults, the two expressive systems are smoothly integrated,
and their co-expression constitutes the expressed message. In children, however, affec-
tive communication significantly predates the onset of productive language. Neonates
Neuroplasticity and Perinatal Stroke 233
orient and attend to faces (or face-like stimuli; Grossman & Johnson, 2007; Johnson &
Morton, 1991); moreover, newborns can and do produce a broad range of emotional fa-
cial behaviors (Oster, 1978). First social smiles appear at about 4–6 weeks of age, when
children also recognize their familiars by facial features alone (de Shonen & Mathivet,
1989); babies are smiling frequently at their family members by three months and are
using a smile or a cry instrumentally to engage or summon others by 3–4 months of age.
By the infant’s first birthday, when first words typically emerge, the child is already a
competent affective communicator, employing facial expression, gestures, and affective
utterances not only when interpreting the communications of others, but also to com-
municate her own interests and desires. Interestingly, whereas the perisylvian regions
of the left hemisphere are most implicated in language processing, when we look to the
neural substrates of affective processing in adults, it is those with late-onset right hemi-
sphere lesions who show flattened affect both facially and vocally (e.g., Blonder, Bowers,
& Heilman, 1991; Blonder, Burns, Bowers, Moore, & Heilman, 1993; Blonder et al., 2005;
Borod, 2000; Borod, Bloom, Brickman, Nakhutina, & Curko, 2002; Pell, 2006). Given
the developmental precocity of emotional signaling, and the differential adult neural
profiles for emotion and language, investigating emotional expression in children with
PS provides a counterpoint to their language development and offers a more compre-
hensive understanding of neuroplasticity.
The chapter is organized in the following manner: we begin with a description of the
children with PS; we then turn to the adult model of brain-language and brain-emotion
relations to provide a backdrop for our developmental studies; the adult profiles can be
considered an end or steady state of development. We go on to document language de-
velopment in children with PS from first babbling in infancy to recounting personal
narratives in adolescence. We then look at the development of emotional expression in
these children to refine and contextualize our understanding of the extent and nature of
neuroplasticity. Finally, we consider how the developing neural substrates for language
and emotion in neuro-typical children play a role in the extent of plasticity observed fol-
lowing a stroke.
Children with Perinatal Stroke
Perinatal stroke is a rare occurrence; the incidence is about 1 in 4,000 live births
(Lynch, Hirtz, DeVeber, & Nelson, 2002; Lynch & Nelson, 2001). These children have
a documented unilateral lesion of acute onset, the result of a stroke (infarct or hemor-
rhage), which occurred sometime in the last trimester of gestation up until 28 days post-
natally with no history of more global damage (e.g., bacterial meningitis, encephalitis, or
severe closed head trauma). Their lesions have been documented by computed tomog-
raphy (CT) or magnetic resonance imaging (MRI), and tend to be in the left, rather than
the right, hemisphere. The children in our study group have normal or corrected audi-
tory and visual acuity, and they are of English-speaking background. Their very early
cerebrovascular events provide a unique opportunity to investigate neuroplasticity for

language and other cognitive systems. The children also offer an informative compar-
ison group to the well-studied population of adults with late-onset unilateral strokes, as
the differences in cognitive/behavioral profiles between these groups can be attributed
to the development and the plasticity of the developing brain.
The Neural Underpinnings

of Language: Adults
Adults with unilateral late-onset stroke have been studied for well over a century, with
the aim of identifying regions of the brain that are associated with specific functional
language processes (see, in this volume, Blumstein, Chapter 1, and Wilson, Chapter 2).
In 1861, Broca described a patient with damage to the left inferior frontal gyrus (Broca’s
area) who exhibited severely impaired expressive language, ostensibly without a com-
prehension deficit. Shortly thereafter, it was observed that patients with damage to the
posterior temporal lobe (Wernicke’s area) had significant deficits in comprehension
yet apparently fluent productive grammar (Goodglass, 1993). From these observations
came the Wernicke-Lichtheim model of language (Lichtheim, 1885; Wernicke, 1874), in
which it was proposed that auditory word comprehension occurs in the left posterior
temporal region, and motor word representations (for word production) are localized
in the left inferior frontal regions of the brain. Structural and functional neuroimaging
research has contributed to a greater understanding of the complexities underlying
these elegant models of language (see Price, 2010, for a review), yet the basic tenets of the
early models are broadly substantiated in current neuroimaging and patient research.
Specifically, language networks in the anterior left hemisphere (particularly inferior
frontal gyrus) tend to be preferentially involved in language production and semantic
processing; regions in the left hemisphere superior temporal lobe tend to be involved in
the comprehension of formal aspects of language, such as grammar and syntax.
While it was previously believed that the right hemisphere was uninvolved or only
marginally involved in processing language, it is now clear that the right hemisphere
plays an important (albeit qualitatively distinct) role (Joanette & Brownell, 1993;
Joanette, Goulet, & Hannequin, 1990; Jung-Beeman, 2005; Kaplan, Brownell, Jacobs, &
Gardner, 1990). It is vital for discourse processing, cohesion, pragmatic language (Borod
et al., 2000), and nonliteral language such as metaphors, idioms, and sarcasm (Bottini
et al., 1994; see Rapp, Chapter 28 in this volume), as well as affective communication
(e.g., Borod, Bloom, Brickman, Nakhutina, & Curko, 2002; see van Berkum, Chapter 29
in this volume). With this brief overview of brain-language relations in adults, we now
consider the adult profile for emotional expression.
The Neural Underpinnings

of Affect: Adults
In adults with late-onset right hemisphere lesions, identity and recognition of emo-
tional facial expression and affective prosody are negatively impacted. Both clinical
reports and experimental studies document affective deficits for both facial and vocal
channels in those with right hemisphere stroke as compared to those with left hemi-
sphere stroke and neurotypical controls (e.g., Blonder et al., 2005; Borod, 2000; Borod,
Koff, Lorch, & Nicholas, 1985; Ross, 1981). Haxby and colleagues (Haxby, Hoffman,
& Gobbini, 2000, 2002) have proposed a “core” face network for the recognition of
faces, as well as an “extended” network to extract meaningful information (e.g., emo-
tion) from that face. The core network includes the fusiform gyrus, occipital gyri, and
the superior temporal sulcus bilaterally; for emotion processing, the extended net-
work also includes limbic areas, including amygdala and insula (see also Fusar-Poli
et al., 2009).
Drawing from both behavioral and imaging studies from neurotypical adults and
stroke patients, Adolphs (2002) proposed a model for the perception of emotional fa-
cial expression that recruits the right visual cortices, especially occipital gyrus, right fu-
siform gyrus, and superior temporal gyrus, insula and amygdala, orbitofrontal cortex
(R>L), and right frontoparietal cortices. This model suggests an interactive network
in which visual cortices and fusiform gyrus respond to the face; bodily movement is
processed by the superior temporal sulcus (STS) (e.g., gaze); and both the amygdala and
orbital frontal cortex modulate the ongoing processing. As an extensive discussion of the
neural substrates of emotion is beyond the scope of this chapter, in addition to this brief
overview, we highlight two studies that are particularly pertinent to the discussion of
neural plasticity for language and emotion. The first is that of Adolphs, Damasio, Tranel,
Cooper, & Damasio (2000), who investigated two aspects of emotion processing: recog-
nition of facial emotion, and emotion labeling. Using 3D comparisons of the structural
MRIs of 108 adult late-onset stroke patients, they found that those with deficits in emo-
tion recognition were those with injury to right somatosensory cortices; and those with
difficulties in labeling emotions were those whose lesions included the left operculum.
They concluded that interpretation of emotional expression involves a simulation or
representation of that emotion, and that emotional labels draw on left hemisphere lan-
guage regions. An early set of case studies of adolescents with presumed right hemi-
sphere damage (Weintraub & Mesulam, 1983) reported poor social skills, problems with
eye contact, and flattened affect. Such findings suggest that neuroplasticity for affect
may well follow a different trajectory from that of language. With these adult/adolescent
profiles in mind, we now turn to the development of language and emotion expression
in children with PS.
Early Language Development

in Children with PS
One of the most startling early findings in the study of language development in chil-
dren with PS is that, unlike adult stroke patients, in whom language is impacted far
more when the stroke is in the left hemisphere, children with either right or left hemi-
sphere injury were delayed in the emergence of language milestones. This delay, regard-
less of which hemisphere was injured, was noted early in development, in a study of
infant babbling. For typically developing children, early vocalizations include cooing at
about 3 months of age, and then canonical babbling (CVCV sequences) appears at about
6–8 months of age. Marchman, Miller, and Bates (1991) began following five infants
with PS at 10 months of age, before they produced their first words. They report that
the infants with PS were delayed in babbling compared to controls, and there was var-
iability across both groups. However, in spite of the overall delay, the infants with PS
followed a similar developmental trajectory with respect to both place and manner of
articulation of consonants relative to the typically developing (TD) control group, and
for both groups, consonant production was associated with word production. In the PS
group, they found no differences between those with left and those with right hemi-
sphere injury.
After the onset of canonical babbling, word comprehension is the next developmental
step in language development. Early indications of word comprehension are evident
in TD children about 9–10 months of age when a child turns in response to her name,
or looks for the cat in response to “Where’s the kitty?” First productive words then ap-
pear, on average, around 12 months of age (although there is extensive variability), with
the productive lexicon increasing slowly. At about 16–18 months, a burst in vocabulary
occurs as the child infers that objects have labels. It is not until 20–24 months that the
first word combinations typically emerge.
Word comprehension is particularly challenging to measure in infants. Habituation/
looking paradigms, which can be employed, require dozens of trials and participants;
they are thus problematic for rare populations. Not surprisingly, parents offer a rich
source of information regarding their toddlers’ development. One successful and widely
used means to chronicle early comprehension, word production, and first sentences has
been the MacArthur Bates Communicative Development Inventory (MBCDI; Fenson,
et al., 1993), a parental report form. In 1997, Bates and colleagues (1997) reported data from
53 children with PS (10–44 months of age) using the MBCDI. From the preschoolers,
transcripts from free-play situations were also used to investigate lexical comprehension
and production, as well as early word combinations. They document delay in the acquisi-
tion of each of these milestones across the group of children with PS, regardless of lesion
side or site; that is, those with right or left hemisphere injury were delayed in the onset of
language, word comprehension, word production, and first word combinations compared
to their typically developing peers. In the context of this broad delay, they found some
site-specific deficits: the infants with right posterior damage showed increased delay in
comprehension, whereas those with left posterior injury were more delayed in word pro-
duction. Note that this profile is unlike what might be predicted from the adult model.
Results from the transcribed language samples in the toddler/preschool group (aged
20–44 months) also demonstrate an overall delay in the PS group compared to controls.
In addition, they found a persistent decrement in lexical production and first word
combinations as measured by MLU (mean length of utterance) in those with left poste-
rior lesions. Again, this profile is in contrast to that of adults, where one would expect
those with left frontal, not temporal, injury to have problems with word and sentence
production.
A case study of a child with a large left hemisphere lesion illustrates this develop-
mental picture. As can be seen in Figure 10.1 (A), her lesion includes frontal, temporo-
parietal lobes as well as subcortical structures. Note that in lexical development, her
vocabulary growth reflects a similar developmental slope to that of the controls, but
with a delay (Figure 10.1 B). Her use of word combinations is also significantly below
that of the TD norms (Figure 10.1 C).
(A)
(B) Vocabulary production (MCDI) (C) Mean Length of Utterance (IPSYN)

600 4.5
4
Number of Words
500
3.5
Morphemes
400
Produced
MLU in
3
300 2.5
200 2
100 1.5
0 1
16 21 23 30 36 41 48 24 36 48
Age in Months Age in Months
10th % 25th% 50th% Sarah Sarah TD
Figure 10.1. (A) Three views of a child’s brain with a left front-temporo-parietal lesion.
Source: With permission from Moses (1999).
(B) Trajectory of this child’s early lexical development as compared to a large normative sample.
(C) Syntactic growth of this same child as measured by mean length of utterance. Although
both lexical and syntactic development are delayed, the developmental slopes are quite similar,
suggesting a late, but comparable path of acquisition.
A complementary study by Rowe, Levine, Fisher, and Goldin-Meadow (2009)

also analyzed productive language from free-play sessions in a group of 27 PS and
53 TD children monthly from the age of 14 months to age 40 months. They report no
differences in language performance according to laterality of lesion; however, the
size and type of lesion (middle cerebral artery versus periventricular) contributed to
different profiles for both vocabulary and for syntactic development. In addition to
the children’s language development, the authors were interested in the role of lin-
guistic input on language acquisition. For both typically developing (TD) and PS
groups, lexical growth was positively correlated with the amount of linguistic input,
but for only the PS group was the use of syntax in the input associated with syn-
tactic development. Such results have important implications for clinicians: rather
than simplifying speech to make language more accessible, this study suggests that
clinicians should encourage parents of children with PS to converse frequently with
their child as a genuine conversational partner, and to use child-directed, yet syntac-
tically rich sentences.
Additional studies of children with PS have confirmed the delay in the onset and ac-
quisition of language milestones in the PS group compared to controls (Feldman 2005;
Thal et al., 2004); some have additionally noted a production delay in those with left
hemisphere injury (Chilosi et al., 2005; Vicari et al., 2000), while others (Rowe et al.,
2009; Thal et al., 2004) have reported an overall delay and a flatter developmental slope
in those with PS, without site-or side-specific profiles. Together, these results demon-
strate a delay in the onset of language as well as in the acquisition of early linguistic
milestones in the PS group, implying a slowed, but iterative and continued development.
Importantly, unlike adults who were already proficient speakers when their strokes
occurred, in the acquisition of language, the comparable delays in those with either left
or right hemisphere injury demonstrate that children are relying on both hemispheres to
acquire their native tongue.
Later Language in Children

with PS: Studies Focused on Specific
Linguistic Forms
As children reach school age and become more compliant, targeted experimental meas-
ures and standardized tests of language are often the preferred means to assess language
abilities. Standardized tests are designed to assess a child’s ability to respond to spe-
cific linguistic stimuli and/or to produce targeted structures. Similar to the studies of
younger children with PS, the studies using standardized tests have found no significant
differences in those with left or right hemisphere injury. However, they did find that
school-age children with PS as a group tend to perform behind their age-matched TD
peers (Ballantyne, Spilkin, Hessleink, & Trauner, 2008; MacWhinney, Feldman, Sacco,
& Valdés-Pérez, 2000). Using a comprehensive language assessment, Ballantyne and

colleagues (2007, 2008) tested children aged 5–16 years and found that while there were
no differences in test performance between the children with left and right hemisphere
injury, there was substantial variability in their scores, and the presence of seizures was
significantly correlated with negative language outcome. MacWhinney and colleagues
(2000) noted an exception to the overall poorer performance of the PS group: results
from vocabulary comprehension subtests were within the normal limits. Another
contrast to the low performance on standardized measures persistently reported by
Ballantyne and colleagues (2007, 2008) are results from a younger group of children with
PS (aged 5–6 years), which showed no difference in performance between the TD and
PS groups (Demir, Levine, & Goldin-Meadow, 2010). It may well be that differences are
emerging as the children acquire more complex language and are required to make in-
creasingly subtle and complex linguistic distinctions. Alternatively, the different results
may stem from the inclusion or exclusion of children with periventricular leukomalacia
(PVL) in the PS group, as they tend to perform better than those with middle cerebral
artery (MCA) strokes.
In addition to standardized tests, both online and offline experiments have shown
the PS groups to be delayed compared to their TD peers. In English, tag questions
(John likes chocolate, doesn’t he?) are complicated syntactic structures, requiring
pronoun substitution, production of the appropriate auxiliary verb, both local and
long-distance verb agreement, and reversing polarity to construct the appropriate
tag question. They represent a challenge for children and second-language learners
as well. Weckerly, Wulfeck, and Reilly (2004) report results from a group of children
with PS and their TD controls on their ability to supply the correct tag (e.g., doesn’t
he?) when given the stem (e.g., John likes chocolate . . .). They found no differences in
performance between those with left and those with right hemisphere lesions, and
they found that the PS group followed the same developmental sequence as the TD
group in mastering the various aspects of tag questions: agreement, pronoun, aux-
iliary, and lastly, polarity. Most interesting was looking at performance across age.
Whereas the PS group performed similarly to the TD controls at the younger and
middle ages (4–7 and 8–11 years), in the oldest age group (12–16 years), when the TD
group made substantial gains, the PS group was significantly behind their typically
developing peers. These findings suggest that plasticity for later language acquisition
might be limited.
Another avenue to investigate children’s linguistic knowledge is to ask them to
evaluate the grammaticality of a sentence. Wulfeck, Bates, Krupa-Kwiatkowski, and
Saltzman (2004) performed an online grammaticality judgment task with school-
age children with PS and their TD controls. For both groups of children, grammatical
sensitivity increased with age, with the TD group exhibiting the adult pattern by age
11–12. With respect to reaction time, the PS group was slower than controls, and gram-
matical sensitivity of the PS group (while below controls) was better than adults with
homologous late-onset lesions, again demonstrating the increased flexibility of the
developing brain.
Standardized tests and experimental measures focusing on individual sentences or

constructions evaluate a child’s ability to respond to certain questions about language
devoid of contextual and paralinguistic cues. However, they do not tell us how well
a child uses language in his or her everyday life. In fact, in typically developing chil-
dren aged 7–14 years, we found little correlation between a child’s performance on
standardized measures and his or her performance in natural language situations (Reilly
& Polse, 2016). For a broader perspective on language development, we next review
studies that focus on language production in more naturalistic discourse situations.
Later Language Development

in Children with PS: Discourse
By age 5, typically developing children are proficient language users and have access
to a repertoire of grammatical structures, although language development continues
well into adolescence (Berman & Verhoeven, 2002; Nippold, 1998; Reilly, Zamora, &
McGivern, 2005). Later language development involves not only an increasingly wide
vocabulary and access to more complex structures, but also a growing ability to flexibly
recruit and employ such structures for various discourse purposes. As they enter pri-
mary school, children can participate in a conversation, describe pictures and events,
and are beginning to tell coherent stories. To better understand later language devel-
opment in children with PS and its resilience as discourse requirements become more
challenging, in this section we review a series of discourse studies that reflect increasing
cognitive demands: dyadic conversation, telling a story from a picture book, and per-
sonal narrative.
A dyadic conversation represents the earliest discursive genre for children; in fact, dy-
adic interactions begin early in the first year, well before first words are produced, with
children showing conversation-like turn-taking in their early vocalizations, gestures, and
behaviors (e.g., playing peek-a-boo) (Bruner, 1975; Stern, 2009). Rather than reflecting a
broad structure, in a conversation, interlocutors respond to the preceding utterance or
turn, and the partner reciprocates. As such, conversations are locally organized by turns.
In 2001, Bates and colleagues (2001) used a semi-structured interview to compare the
conversational discourse of 38 children with PS (aged 5–8 years) and their TD controls
to 21 adults with homologous late-onset unilateral strokes and their neurotypical adult
peers. The adult stroke patients broadly followed the classical profile: those with left hem-
isphere injury who were severely aphasic produced fewer utterances, had lower syntactic
diversity, and made more morphological errors and omissions than their controls; those
with Wernicke’s aphasia were characterized by neologisms and lexical substitutions.
As expected, those with right hemisphere injury made few linguistic errors. However,
they produced twice as many propositions as controls, and their speech tended to be
disinhibited and lacking in content. The children with PS presented a distinctively
different profile. Unlike the adult stroke patients, the children with PS performed com-
parably to their TD peers on all linguistic measures. There were no differences be-
tween those with left or right hemisphere injury and, importantly, the language of the
PS group was similar to that of the TD group with respect to vocabulary, morphological
proficiency, and in recruiting complex syntax. Figure 10.2 compares (A) morphological
errors and (B) the frequency of recruiting complex sentences across the groups. In form,
these interviews represent a quasi-naturalistic context for the evaluation of spontaneous
conversational language; as such, they are the age-appropriate equivalent to the “free-
play” sessions used with younger children described in the research presented earlier.
Performance in these interviews is evidence that the earlier noted deficits in the PS group
have resolved and that by early school age, the children with PS appear to have “caught
up” to their peers in productive conversational language.
(A)
350
Z-scores erros/proposition
300
250
200
150
100
50
0
Child LHD
Child RHD
Adult LHD
Adult RHD
Broca
Wernicke
Anomic
Non-Aphasic
LHD
(B)
0.5
Mean complex structures
0.4
per utterance
0.3
0.2
0.1
0
Child Normal
Child LHD
Child RHD
Adult Normal
Adult LHD
Adult RHD
Broca
Wernicke
Anomic
Non-Aphasic LHD
Figure 10.2. Comparing the morphological error rate (A) and the use of complex syntax (B) in
children and adults with homologous strokes, it is evident that a perinatal stroke does not exact
the same devastating effect as a late onset stroke.
Source: Reprinted with permission from Bates et al. (2001).
To extend our understanding of discourse, we asked some of these same children

(and some older and younger) to tell a story from a picture book, Frog, Where Are You?
(Mayer, 1969; see Berman & Slobin, 1994, for a review of typical narrative development).
Unlike conversations, stories have a canonical structure: good stories have a setting, a
problem, an attempt at resolving this problem, and a conclusion or resolution. As such,
they represent a cognitively more challenging task than a conversation, as the child must
plan, as well as integrate the local events of the story with the overall theme. The story,
Frog, Where Are You? is a traditional “quest story”: a boy has captured a frog who then
escapes. The hunt for the missing frog includes encounters with other animals until the
frog (and its mate and babies) are discovered by the boy (and his dog). In narrative de-
velopment, typically developing children have some idea of “story” by their third year
(Appleby, 1978); by age 5, children tend to produce horizontally linked narratives using
“and” and “and then” to sequentially relate events. It is not until about 8 years of age that
children are able to integrate the horizontal (local) episodes with the overarching theme
(e.g., searching for the frog).
The Frog Story studies of children with PS (Reilly, Bates & Marchman, 1998; Reilly
et al., 2004) included 52 children with PS and age-and gender-matched controls from
3;11–12;3 years of age. The children’s stories were evaluated for both linguistic measures,
for example, morphosyntactic errors, use and diversity of complex syntax; as well as dis-
course features, for example, whether the search theme was articulated, and whether the
child included all the story components. Consistent with earlier studies, the first finding
was no difference between those with right and those with left hemisphere injury on any
linguistic or discourse measure after age 5.1 Looking at their performance as a group, the
stories from the children with PS were shorter than those of the TD group. The children
in the younger and middle PS groups (aged 3; 11–6; and 7–9) made more morphosyn-
tactic errors and used less complex syntax, as well as fewer syntactic types than their TD
peers, but both groups made fewer errors and produced more complex sentences as they
got older. To give a flavor of the children’s language, Box 10.1 presents an excerpt from a
Frog Story from Sarah (from Figure 10.1) at ages 4;11 and 8;2.
It is important to note that the morphosyntactic errors of the PS children are the same
types of errors seen in the speech of younger TD children, suggesting that children with
PS are following a similar path in acquiring English to their TD peers, but at a slower
pace. For these linguistic measures in the narratives, it was not until age 10 that the per-
formance of the PS group matched that of their TD peers. It appears that a story from the
picture book challenges the language capacities of the PS group (as measured by gram-
matical proficiency) more than those of the TD children. However, in this discourse
context, children with PS seem to have “caught up” to their TD peers by 10 years of age,
a few years later than was observed in a less challenging, dyadic conversational context.
1
(The four children under age 5 years whose lesion affected the left temporal lobe had made more
morphological errors than their age mates, but this difference was no longer present after age 5.)
Box 10.1. Excerpts from the “Frog Story” from a child with LPS.
Narrative excerpts from a child with Left Perinatal Stroke (LPS) telling, Frog, Where Are
You? (at ages 4;11 and 8;2).
4;11:
‘The boy's looking at fwog. Then the dog... began to bark. Then the fwog...he comed out.
Then the dog and the boy couldn't see the fwog! He woked up. He looked outside. Then the
dog fell down. Then the, then the boy got pushed. Um . . . he screamed.”
8;2:
“The boy and the dog are looking at the frog. The boy is sleeping on his bed and the frog got
out. And when the boy woke up, the frog was gone. He was peeking in his vest, and he was
calling the frog. And then the dog fell out, fell through the window. He was mad. He was
yelling really really really loud.”
With respect to the more global discourse measures of narrative, although they in-
clude more story components with age, the stories from both older and younger chil-
dren in the PS group included fewer episodes than those of the TD group, even though
they were all telling the same story with the picture book in front of them. Children in
both TD and PS groups were likely to remark that the frog had initially disappeared;
however, the children with PS were less likely to mention that the boy’s activities (e.g.,
looking in the tree) were motivated by his search for the frog; that is, the PS group was
less likely to present a goal or theme of the story to render the story cohesive. In a qual-
itative analysis of the texts from all the 7-to 8-year-olds (both TD and PS), the stories
were classified into three groups (High [best stories], Middle, and Low) based on such
elements as plot, elaboration, reference, and “storiness,” which was defined as “use of
linguistic connectors to integrate events.” The stories in the Low group were all from the
PS children (with one exception), whereas all the highest scoring stories (High group)
were from the control group, except for one. In sum, although stories from both groups
were more complete and complex in the older groups of children, the stories from chil-
dren with PS were less cohesive and more impoverished than those of controls overall.
In the dyadic conversation data presented above, the language of the PS group was
comparable to that of their TD peers at age 5; in the picture book narrative, it was not
until age 10 that their morphological proficiency and use of syntax was similar to the TD
group. The differences in their performance on the picture story narratives compared to
that of the interviews suggests that language is more fragile for the children with PS than
for the TD children, and that discourse context and cognitive complexity are influential
factors in the quality of their language performance.
It is clear from these studies that children with either right or left hemisphere lesions
are affected in language development and that language in the children with PS is
delayed. However, it is also evident that their language continues to develop and appears
to be following a similar behavioral path to that of typically developing children, but
on a somewhat different time schedule. Discourse context and cognitive demands also
appear to have a greater impact on language performance in children with PS than in

typically developing children. Together, these findings raise questions regarding the na-
ture of later language development in the PS group. Will they continue to make prog-
ress and finally “catch up” to their age mates in all linguistic contexts? To address these
questions, we now turn to an even more challenging discourse context—personal
narratives—as we follow language development into adolescence. In contrast to the pic-
ture book, where the story is provided, in the case of a personal narrative, it is the child’s
responsibility to recall, organize, and recount events from his or her life into a narra-
tive. Although a common occurrence in quotidian life, recounting a personal narrative
draws on a range of social, linguistic, memorial, and organizational skills.
Children and adolescents were asked to “[t]‌ell about a time when someone made you
sad or angry; what happened, how it began and how it ended” (Reilly, Wasserman, &
Appelbaum, 2013). Ninety-five children participated (20 LPS, 15 Right Perinatal Stroke
(RPS), and 60 TD). Children were divided into a younger (7–11 years) and an older
(12–16 years) group, and similar to the picture stories, the children’s transcribed texts
were assessed for both linguistic and narrative measures. Linguistic measures included
Productivity, Morphological Errors, and Complex Syntax; narrative measures included
Setting (Time, Situation, Characters, and Location), and story components (Setting,
Problem, Attempt at Resolution, Resolution). On the language measures, the PS group
as a whole performed more poorly than their age-matched TD peers, and those with
left hemisphere lesions fared the worst. The children with left hemisphere injury made
more morphological errors and used fewer complex sentences and sentence types
than the controls. Those with right hemisphere lesions did not differ significantly from
controls on these measures, but their average morphological error rate and frequency
of complex syntax fell between those with LPS and the TD group. With respect to the
narrative components, all the children produced stories with the requisite components.
However, those with LPS produced more impoverished settings than controls, whereas
those from the RPS group did not differ from the TD group. Including a Setting to intro-
duce one’s story is a developmentally later acquisition; socially, it orients the listener and
establishes the world in which the story will unfold. As such, for these older children
the quality of the setting has proven to be a sensitive measure of later language devel-
opment (Tolchinsky, Johansson, & Zamora, 2002). The children in this study ranged
from 7 to 16 years of age, and we found that children in the older TD group made sig-
nificantly fewer errors and produced richer story settings than the younger TD group.
Surprisingly, however, there were no comparably significant developmental differences
in the younger and older LPS and RPS children. One reason for this lack of apparent
change with age in the PS group may well be the elevated level of variability in their data.
To illustrate this variability, Figure 10.3 represents the morphological error data from
the older and younger TD, LPS, and RPS groups.
Variability characterizes the early stages of language acquisition in typical develop-
ment. For example, assessing morphological proficiency in spontaneous speech, chil-
dren aged 3 and 4 years show high variability: some make numerous errors (e.g., more
than one error per clause), whereas other preschool-age children are making few errors
Typically developing Left hemisphere Right hemisphere

MorphErr MorphErr MorphErr
0.9 0.9 0.9 0.9 0.9 0.9

0.8 0.8 0.8 0.8 0.8 0.8
0.7 0.7 0.7 0.7 0.7 0.7
0.6 0.6 0.6 0.6 0.6 0.6
0.5 0.5 0.5 0.5 0.5 0.5
0.4 0.4 0.4 0.4 0.4 0.4
0.3 0.3 0.3 0.3 0.3
0.3
0.2 0.2 0.2 0.2 0.2
0.2
0.1 0.1
0.1 0.1 0.1 0.1
0
0 0 0 0 0
–0.1
–0.1 –0.1 –0.1 –0.1 –0.1
Younger Older Younger Older Younger Older
Figure 10.3. Variability in morphosyntactic errors. The older groups in all three populations
make fewer errors than their younger counterparts, but the variability of the children with brain
injury, both those with LPS and RPS, far outstrips that of the TD group.
Source: Reprinted with permission from Reilly, Wasserman, & Appelbaum (2013).
(e.g., one error per every 10 clauses). However, by about age 5, typically developing
children make few errors, averaging one error every 10 clauses in spontaneous speech;
and by age 10, TD children are making about one error per 20 clauses in spontaneous
speech (Reilly et al., 2004). As such, the early pattern of variability becomes increas-
ingly homogenous as children master the morphology. Consequently, if an 8-year-old
were making significantly more errors than his or her peers, he or she would be a can-
didate for clinical evaluation. In contrast to school-age typically developing children,
the extensive variability in language performance in the PS group has been noted across
studies and research groups (e.g., Feldman, 1994; Marchman et al., 1991; Reilly et al.,
2012, Rowe et al., 2009; Thal et al., 2004), and makes it difficult to determine linguistic
norms for this group. What might underlie this broad variability? We address this ques-
tion later in the discussion.
Personal narratives from children with PS provide an additional perspective on later
aspects of language development: those with RPS performed somewhat lower, but did
not differ significantly from controls on either linguistic or discourse measures; how-
ever, those with LPS performed worse than controls in all areas. In this more challenging
discourse genre, it appears that the language of the PS group (and especially of those
with left hemisphere lesions) is more vulnerable than that of the TD group. Moreover,
the results bring into relief the increased fragility of productive language for those with
left hemisphere lesions, whose performance subtly mirrors adults with homologous
late-onset stroke.
To briefly summarize language acquisition in children with PS, language onset
is delayed, as is each of the early developmental milestones: babbling, word compre-
hension, word production, first word combinations, and the acquisition of mor-
phology. Within this broad landscape of delay, studies have found some site-specific
characteristics: early on (ages 10–17 months), those with right posterior injury showed

greater comprehension deficits than others in the PS group, and those with left hemi-
sphere injury reflected greater delays in word and early sentence production. However,
these side-specific profiles appear to resolve by about 5 years of age. Whereas the lan-
guage of the PS group was initially characterized by delay and development, by early
school age (5–8), the PS group performs comparably to their age-matched controls
with respect to lexical diversity, morphological proficiency, and in employing complex
syntax in conversation. At this age, TD children are considered to have “acquired” the
basic structures of their language (Slobin, 1985, 1992, 1997), and from the acquisition lit-
erature on TD children, we would predict relative stability across naturalistic contexts.
As we saw from the two narrative studies presented earlier, this is not the case for the
children with PS. In contrast to the biographical interview, in the Frog Story, it is not
until age 10–12 that the PS group performs comparably to controls on morphological
and syntactic measures. Such findings suggest that the added cognitive task of inferring
a story from pictures challenges their language abilities, and that their language is more
fragile than that of the TD group. In reviewing data from the more demanding personal
narrative, not only are the PS stories more impoverished and more errorful than those
of their TD peers, but it is the children and adolescents with left hemisphere injury who
fare the worst. These findings may constitute evidence of limits on neuroplasticity for
spontaneous language; moreover, they suggest that the left hemisphere may indeed be
privileged for language.
Profiles of Affective Expression

in Children with PS
As noted earlier, by their first birthday, when first words emerge, TD infants are al-
ready good affective communicators. They use facial expression, gesture, and affective
vocalizations to convey their desires and to interpret the behavior of others. Moreover,
they can initiate and participate in, first, dyadic and then triadic interactions using these
nonlinguistic tools. The early developmental appearance of this communicative system
affords another perspective on development and neuroplasticity.
In our first study of emotional expression in infants with PS (Reilly, Stiles, Larsen, &
Trauner, 1995), we asked mothers to play with their pre-linguistic infants (6–22 months
of age) as they did at home. Mother-infant dyads were video-and audio-taped with
multiple cameras to capture the faces, language/vocalizations, and behaviors of both
participants. The results were surprising. As can be seen from Figure 10.4, the typically
developing infants demonstrated a range of positive responses to their mothers’ bids
for attention: some infants were very smiley and others more serious, but overall they
smiled easily. Those infants with posterior left hemisphere lesions clustered tightly in the
mid-range of “smiliness” compared to the typical group, whereas those with posterior
1.0
0.8
Proportion of smiles
0.6 Normal Left
controls posterior
0.4
Right
0.2 posterior
0.0
Figure 10.4. Infant smiles in response to maternal bids for conversation. Whereas TD infants
show a range of “smiliness,” those with LPS cluster tightly in the mid-range of the TD group; in
contrast, those with RPS fall below those with TD and LPS in their frequency of smiles.
Source: Reprinted with permission from Reilly, Stiles, Larsen, & Trauner (1995).
right hemisphere lesions smiled significantly less frequently than either the TD or LPS
groups (see Figure 10.4).
Moreover, the RPS group also produced more negative vocalizations than either
the controls or those with LPS. In sum, whereas those infants with left hemisphere
lesions cluster with controls for both affective facial and vocal expression, the infants
with RPS smile significantly less frequently and produce more negative vocalizations
than their LPS and TD peers. A complementary parental report study on young chil-
dren with PS (Nass & Koch, 1987) that used a temperament questionnaire reported
similar results: those toddlers with RPS displayed more negative expressive behaviors
than those with LPS or the TD group, and this profile continued into preschool. Such
findings are consistent with affective expression in adults with late-onset right hemi-
sphere stroke. Blonder and colleagues have reported a series of studies depicting the
spontaneous use of facial expression and prosody in adults with stroke in conversation
(Blonder et al., 1993; Blonder et al., 2005). They found a decrement in affective expres-
sion overall, as well as increased negativity and a decreased expression of positive emo-
tion in adults with right hemisphere damage (RHD )compared to neurotypical adults
and those with left hemisphere damage (LHD). As such, unlike the emergence of lan-
guage where the developmental profile contrasts with that of adults, that is, children
with both right and left hemisphere lesions show considerable delay, the profile for af-
fective expression (both facial and vocal) in the PS group mirrors that of adults. Such
findings suggest that the neural underpinnings for affective expression are established
very early, sometime in the first year of life. The question that arises concerns plasticity
and development: what is the nature of the affective profile of the children with RPS as
they develop and as they acquire language?
To address this question, we returned to the biographical interview data of 20 children
with PS aged 5–6 years (10 RPS, 10 LPS, and 20 TD) (Lai & Reilly, 2015). This context
is most similar to free play with infants and the conversational context in the Blonder
studies, thus affording an appropriate comparison. The videotapes were analyzed for fa-
cial expression production and the transcribed texts for the content and affective valence
of the children’s utterances. Similar to their infant and adult counterparts, the children
with right perinatal stroke used less facial expression overall during the interview than
either those with left hemisphere lesions or controls; moreover, they also used less affec-
tive language than with those with LPS or the TD group. In assessing the valence of their
speech, whereas both the TD and LPS groups relate mostly positive experiences in the
interviews, those with RPS recount slightly more negative than positive experiences. It
must be noted here that the children with RPS can tell stories about emotional events
and did so reasonably successfully in their personal narratives discussed earlier. It is
rather that spontaneously they do not express emotion with the same frequency or the
same valence as their LPS or TD peers.
To summarize our findings on spontaneous production of emotional expression in
children with early stroke, we found that from early in infancy, those with LPS cluster
with their typically developing peers, while those with RPS express less positive affect
overall, as well as more negative affect than either those with LPS or their TD peers.
This profile mirrors that of adults with late-onset homologous lesions. As these chil-
dren develop and acquire language, we continue to see an analogous profile in which
those with right hemisphere lesions persistently express less emotion overall, with a ten-
dency toward negative affect. Interestingly, this profile extends to facial expression and
vocalizations, as well as to the content of spontaneous language. The most striking aspect
of the school-age profile for emotional expression in the PS group is the apparent lack
of development or change up through age six years. Similar to their infant patterns, the
school-age children continue to mirror the adult profile in their spontaneous use of af-
fective expression; that is, unlike the slower, but steady development of language in chil-
dren with PS, their affective profile remains relatively stable, with those children with
RPS demonstrating a persistent decrement in emotional expression, especially positive
expression. What might explain these contrastive developmental profiles? Considering
the development of the neural substrates of these two communicative systems (language
and affect) in typically developing children may provide clues to this question. The fol-
lowing sections are thus devoted to what we currently know about the development of
the neural bases of language and emotion in infants and children.
Neural Organization for Language

in Childhood
Neuroimaging evidence suggests that early in development, at 3 months of age (fol-

lowing approximately 4 months of language exposure in utero), infants show some
left-
lateralization for language in response to forward versus backward speech
(Dehaene-Lambertz, Dehaene, & Hertz-Pannier, 2002). By 12–18 months, they are
recruiting left frontotemporal networks in the left hemisphere (similar to adults) in

response to a single word matched with a picture (Travis et al., 2011). When considered
in light of these neuroimaging results, it is not surprising that children with left hem-
isphere perinatal stroke, who lack the neural tissue to support this same early laterali-
zation pattern, are delayed in early language milestones relative to typically developing
children and children with right hemisphere lesions. It is somewhat surprising, how-
ever, that children with right hemisphere injury also show delays in initial language
milestones such as babbling and producing their first words (e.g., Bates et al., 1997;
Marchman et al., 1991); that is, if word recognition in the first year and a half recruits
primarily left hemisphere anterior areas, why do we see a delay in this skill particularly
in children with right posterior injury? Pediatric neuroimaging research that takes into
account both language proficiency and development sheds some light on this apparent
enigma.
Using event-related potentials (ERPs; see Leckey & Federmeier, Chapter 3 in this
volume), Mills, Coffey-Corina, and Neville (1997) reported that in typically devel-
oping infants 13–20 months of age, known words elicited larger amplitude ERPs at
200–400 milliseconds following word onset relative to nonwords. In the younger half
of the sample (13–17 months), this amplitude difference was observed bilaterally over
frontal, temporal, parietal, and occipital electrodes. By 20 months, however, the differ-
ence occurred mostly over left hemisphere temporal and parietal sites. Importantly, the
older infants in the sample (20 months) who knew more than 150 words elicited more
focal ERPs over left hemisphere electrodes, whereas the 20-month-olds who under-
stood fewer than 50 words elicited more distributed ERPs over bilateral electrodes. This
research suggests that initially, word processing is more distributed and is processed
bilaterally; it is only with age and increasing vocabularies that word comprehension
becomes more focally organized in left hemisphere perisylvian regions. Together these
neuroimaging results provide a context for understanding early language development
in children with PS, and demonstrate that lateralization for language is a dynamic devel-
opmental process, one highly dependent on language acquisition itself. Such results are
consistent with the finding that children typically recruit both hemispheres in the initial
stages of word acquisition and thus, both children with left hemisphere lesions and chil-
dren with right hemisphere lesions show an initial language delay.
As children grow older, language quickly becomes more than the production and
comprehension of single words. Their vocabularies expand, and they begin to string
words together, first into simple and then into syntactically complex sentences. These
more complex linguistic processes require broader cognitive faculties that recruit bi-
lateral neural networks. There is now ample evidence that as children develop through
the elementary school years and into adolescence, these bilaterally distributed neural
networks for language become more localized to left hemisphere perisylvian regions
(e.g., Booth et al., 2003; Brown et al., 2005; Holland et al., 2001; Schlaggar, Brown,
Lugar, Visscher, & Miezin, 2002; Schmithorst, Holland, & Plante, 2007; Szaflarski,
Holland, Schmithorst, & Byars, 2006). Importantly, this shift is not dependent on age
alone, but also depends on the task (i.e., what aspect of language is probed), the child’s
proficiency with that linguistic element, and the area of the brain recruited for a partic-
ular linguistic task.
In a covert semantic fluency task with adults and children, Gaillard and colleagues
(2003) found left lateralization for a lexical task by age 7, with no differences between
child and adult lateralization activation. However, another functional magnetic reso-
nance imaging (fMRI; see Heim & Specht, Chapter 4 in this volume) investigation by
Holland and colleagues (2009) of children aged 5–18, which probed multiple aspects of
language processing (lexico-semantic, syntactic, prosodic), found that it was the later
acquired (more complex) aspects of language (i.e., syntactic tasks without semantic
content) that had more bilateral representation in the developing brain. Earlier acquired
(simpler) language functions, such as lexico-semantic tasks (measured by a word-
picture matching), on the other hand, showed stronger left hemisphere lateralization,
which was significant in both anterior and posterior areas. An additional fMRI inves-
tigation using spoken, rather than covert, responses in a verb-generation task provides
further evidence that the lateralization process is progressive; it is not only dependent on
task, age, and brain areas recruited, but also on the level of proficiency with the language
structure assessed (Brown et al., 2005). Brown and his colleagues also noted that during
this developmental process of lateralization for language, activation in areas associated
with earlier processing “lower-level” mechanisms (e.g., bilateral occipital and temporal
cortex) decreased in the older children, concomittant with increases in activation in left
frontal and parietal cortex (regions associated with “higher-level” top-down control).
It is clear that the language system, unlike other communicative systems such as af-
fect, has a remarkable capacity to exploit neural plasticity, and the preceding discussion
provides a possible explanation for the remarkable language development in children
with PS. However, it is not clear how the brain responds to an early insult; that is, what
neural tissue are these children recruiting for language use? Some studies suggest that
for those with left hemisphere injury, the remaining healthy neural tissue in the left
hemisphere takes on language functions (Liegeois et al., 2004; Saccumen et al., 2006).
Additional research indicates that homologous right hemisphere regions more often
support language processing (Holland et al., 2009; Jacola et al., 2006; Staudt et al., 2002;
Szaflarski et al., 2014; Tillema et al., 2008). Additional investigations have found evi-
dence for both: In one of the few imaging studies with children with PS, Raja Beharelle
et al. (2010) found that language performance was positively correlated with left later-
alization in the inferior frontal gyrus and bilateral activation in the superior temporal
regions. These apparently contradictory results between investigations may reflect the
variability in language function, as well as the variability in neural organization that
is characteristic of this population. An imaging study by Fair and colleagues (2009)
confirms that each child with a left perinatal stroke recruits a unique and individual
right hemisphere network in response to lexical stimuli. In sum, it is likely that the dis-
tributed, bilateral language network in typical childhood provides these children the
opportunity to exploit ipsilateral neural substrates surrounding the lesion, as well as
contralateral homologous tissue (which are typically involved in language processing
in childhood) to support language. This period of bilaterality for language may well
contribute to the high degree of neural plasticity and development observed in language
in this population, which contrasts with the development of communicative affect.
Neural development is a dynamic process, with each step laying the foundation for
the emergence of new neural structures and systems (Stiles, 2008; Stiles, Brown, Haist &
Jernigan, 2013). Thus, it is the iterative nature of development itself that dictates the func-
tional and structural organization of a child’s brain. For the children with PS, therefore, it
is not only the event of a cerebral injury, but the way in which this event impacts successive
brain development that determines the neuroanatomy for and proficiency of language. We
next consider the neural underpinnings and development of affect and emotion.
Neural Organization for Processing

Emotional Facial Expression in Infancy
and Childhood
In contrast to the burgeoning pediatric neuroimaging literature for language, the par-
allel literature on emotion is more limited, with the majority of investigations focusing
on face processing. As processing faces is imperative to understanding emotional facial
expression, this literature will be briefly discussed.
From behavioral studies (see Herba & Phillips, 2004, for a review of emotion devel-
opment and Pascalis et al., 2011, for a review of face processing) we know that processing
faces and emotional facial expression, as well as the neural underpinnings involved in
face and emotional face processing (Batty & Taylor, 2006; Swartz, Carrasco, Wiggins,
Thomason, & Monk, 2014; Gee et al., 2013), continue to develop well into adolescence
and young adulthood. However, it appears that very early in development, the in-
fant brain is already attuned to faces and to the information they provide (Grossman
& Johnson, 2007). Newborn infants attend to face-like patterns, and such behavior is
supported by a subcortical neural network (Johnson, 2005). Evidence for the prenatal
development of such subcortical connections, and their role in affective development,
stems from studies of nonhuman primates. After lesioning the amygdala in rhesus
monkeys during the neonatal period, as adults, these monkeys showed significantly
blunted affect, with decreased responses to both positive and negative stimuli (Bliss-
Moreau, Bauman, & Amaral, 2011).
With respect to face processing, ERP studies in human infants, aged 3–12 months,
show activation of left and right lateral occipital areas, the right fusiform face area (FFA),
and the right superior temporal sulcus (STS) in discriminating upright and inverted
faces (Johnson et al., 2005). In addition, in a rare imaging study of a small group of 2-
month-old infants, Tzourio-Mazoyer and colleagues (2002) found that in response to
faces (versus colored lights) a network comparable to the core adult system for face pro-
cessing was activated in the infants’ brain, including the right inferior temporal gyrus
(the infant homologue of the adult fusiform gyrus), bilateral occipital, and right inferior
parietal cortices. They also found activation of left superior temporal and inferior frontal
gyri, areas typically associated with language processing in adults. Together these studies
suggest that a face-sensitive neural network is functioning in the first months of life, and
although broader than that of adults, many of the specified areas are also implicated in
the adult face-processing network (Haxby, Hoffman, & Gobbini, 2000).
Studies investigating infants’ responses to emotional facial expressions show that by
7 months of age, infants respond differentially to fear and happy/neutral faces, (Nelson
& de Haan, 1996; Leppänen & Nelson, 2008). However, responses did not distinguish
between angry and fearful faces (Nelson & de Haan, 1996), suggesting that emergent
emotional categories are broad and reflect simple positive/negative parameters. Batty
and Taylor (2006) report a developmental ERP study on the neural bases of emotion
processing in children and adolescents (ages 4–15). Their study follows changes of P1
and N170, components associated with face and emotional processing in adults. In the
children, the P1 was sensitive to faces across childhood, and by age 4, the N170 was
also face sensitive. They found decreases with age in both the amplitude and latency
of the P1, suggesting increased automated visual processing for faces. The P1 latency
showed an effect for emotion even in the youngest children; in contrast, the N170 was
only sensitive to emotion type in the oldest group (14–15 years). These results indicate
that even though infants and young children use emotional facial expression fluently
and frequently, their neural bases reflect a protracted developmental course well into
adolescence.
Using fMRI to look at emotion responsivity, an early pediatric imaging study
investigated amygdala response to fearful faces in children (average age: 11 years)
and adults (Thomas et al., 2001). They found differential response patterns: Whereas
adults demonstrated increased left amygdala activation to fearful over neutral faces,
the children showed greater amygdala response to neutral than fearful faces. A com-
plementary study by Todd, Evans, Morris, Lewis, and Taylor (2010) with children 3;6–
8;6 showed that unlike adults, the children showed increased amygdala response to
happy over angry faces, but that amygdala activation to angry faces increased with age.
Recently, several pediatric neuroimaging studies of emotion processing have focused
on amygdala-prefrontal connectivity. The results from Gee et al., (2013) and Swartz
et al. (2014) suggest that with development, increased structural connectivity is related
to decreased amygdala activation in response to sad and happy faces, and that such
increased connectivity affects the development of emotion regulation. Together these
studies attest to the extended development of the neural bases for emotion and its reg-
ulation, indicating refinements in function and connectivity that continue to develop
beyond childhood.
Despite this protracted development, a nascent neural network for face and emotion
processing is present early in the first months of life, perhaps at birth, and this network
recruits both cortical and subcortical structures that are implicated in the mature adult
network. As such, a pre-or perinatal stroke involving the posterior regions of the right
hemisphere may well have long-term consequences for their development.
Conclusions
An accruing literature has noted that language function is especially resistant to neural
injury early in life, and has concluded that language enjoys a particularly high de-
gree of neural plasticity in childhood. The data on affect extend and refine our under-
standing of neuroplasticity for communicative systems more broadly. The remarkable
development of language in these children with either right or left hemisphere injury
as compared to the specific deficits in affective expression that are associated with le-
sion site bear witness to the range and limits of neuroplasticity of the developing brain.
Possible explanations for these gradients of plasticity may stem from the intersection of
phylogeny and ontogeny.
Relative to language, the affective system is evolutionarily old. Even in lower animals,
we see evidence of affect: an angry tortoise will bob his head repeatedly on sighting an-
other tortoise whom he perceives as a threat; prairie voles show affiliative behaviors to-
ward their lifelong mates. As such, evolutionarily “older” brain structures, such as the
visual cortices and limbic system, support affect, while language processing recruits rel-
atively newly evolved cortical structures, such as regions in the frontal and prefrontal
cortex. It is likely that these early affective brain structures are less flexible and more
constrained than later evolving cortical regions. As such they may well be less capable of
co-opting alternate neural tissue to adapt and reorganize following injury. In addition to
being a more evolutionarily primitive system relative to language, communicative affect
also emerges earlier in development than either expressive or receptive language.
With respect to ontogeny, it is not until later in the first year of life that infants show
some understanding of language, suggesting that the neural circuitry involved in lan-
guage processing for communicative purposes is just beginning to function. As we have
noted, the neural substrates recruited for language in childhood are broadly distributed
across both hemispheres in typical development. This distributed state provides a nat-
ural opportunity for children with PS to strengthen and refine supplementary pathways
in ipsilateral surrounding and contralateral homologous brain regions to support lan-
guage. By late adolescence, when typically developing language has become more focally
organized in the left hemisphere (Szflarski et al., 2006), children with PS have co-opted
these early additional neural pathways, and have established them for language. For
adults with late-occurring stroke, language has already lateralized to the left perisylvian
regions. As such, they have missed this natural developmental period of bilaterality for
language, and thus do not have the same opportunity to capitalize on naturally existing
alternate cortical circuitry.
In contrast to language, the onset of emotion and affective communication emerges
just after birth, with infants producing facial configurations respecting canonical emo-
tional expressions when they are only a few days old (Oster, 1978). Critical to emotion
and face processing are subcortical networks including the amygdala (Adolphs 2003;
Johnson, 2005). Johnson has proposed that the subcortical neural substrates involved
in early emotional processing may be established prenatally, and support for this hy-
pothesis comes from research from nonhuman primates. Thus, like the adults with
late-occurring stroke who have missed the opportunity to utilize naturally existing
distributed networks for language, the opportunity to develop and strengthen more
distributed neural substrates for face and emotion processing may be significantly
decreased because critical aspects of the neural tissues that support such processing are
already active at birth. This early neural specification for emotion processing may ex-
plain the similarity of the affective profiles between children with PS and adults with
late-occurring stroke.
The preceding discussion sheds some light on the gradients of neuroplasticity in
different communicative systems, and suggests that the interaction between ontogeny
and phylogeny plays a critical role. For language, it is clear that early-occurring per-
inatal strokes do not produce the same irreparable damage that we witness in adults
with homologous late-onset strokes. Furthermore, for the children, the side of the le-
sion has relatively little impact on their behavioral linguistic profile, whereas in adults,
the side and site of the lesion are good predictors of language function following
stroke. If we look uniquely at language and its acquisition, it appears that plasticity of
the young, developing brain reflects the possibility for extensive (re)organization fol-
lowing a neural insult. However, our findings on affect, a complementary communi-
cative system, indicate that children with early injury show similar, albeit more subtle,
effects from their early stroke to those of their adult counterparts. These distinctive
profiles and developmental trajectories for language and affect demonstrate that the
processes underlying neural organization (i.e., plasticity) do not operate indiscrimi-
nately; these processes are inherently tied both to functional neuroanatomy and de-
velopment itself.
Acknowledgments
This work was partially supported by NIH P50 NS 22343: Neurological Bases of Language,
Learning and Cognition. We would like to thank those working on the Project for Cognitive
and Neural Development, and we are especially grateful to the children and their families who
generously participated in these studies.
References
Adolphs, R. (2002). Neural systems for recognizing emotion. Current Opinion in Neurobiology,
12(2), 169–177.
Adolphs, R. (2003). Cognitive neuroscience: Cognitive neuroscience of human social
behaviour. Nature Reviews Neuroscience, 4(3), 165.
Adolphs, R., Damasio, H., Tranel, D., Cooper, G., & Damasio, A. R. (2000). A role for somat-
osensory cortices in the visual recognition of emotion as revealed by three-dimensional le-
sion mapping. The Journal of Neuroscience, 20(7), 2683–2690.
Appleby, A. (1978). The child’s concept of story. Chicago: University of Chicago Press.

Ballantyne, A. O., Spilkin, A. M., & Trauner, D. A. (2007). Language outcome after perinatal
stroke: Does side matter? Child Neuropsychology, 13, 494–509.
Ballantyne, A. O., Spilkin, A. M., Hesselink, J., & Trauner, D. A. (2008). Plasticity in the devel-
oping brain: Intellectual, language and academic functions in children with ischaemic peri-
natal stroke. Brain, 131(11), 2975–2985.
Basser, L. S. (1962). Hemiplegia of early onset and the faculty of speech with special reference to
the effects of hemispherectomy. Brain, 85, 427–460.
Bates, E., Reilly, J., Wulfeck, B., Dronkers, N., Opie, M., Fenson, . . . Herbst, K. (2001).
Differential effects of unilateral lesions on language production in children and adults. Brain
and Language, 79(2), 223–265.
Bates, E., & Roe, K. (2001). Language development in children with unilateral brain injury. In
C. A. Nelson & M. Luciana (Eds.), Handbook of developmental cognitive neuroscience (pp.
281–307). Cambridge, MA: MIT Press.
Bates, E., Thal, D., Trauner, D., Fenson, J., Aram, D., Eisele, J., & Nass, R. (1997). From first
words to grammar in children with focal brain injury. In D. Thal and J. Reilly (Eds.), Special
Issue on Origins of Communication Disorders, Developmental Neuropsychology, 13(3),
275–343.
Batty, M., & Taylor, M. J. (2006). The development of emotional face processing during child-
hood. Developmental Science, 9(2), 207–220.
Berman, R., & Slobin, D. (1994). Relating events in narrative: A cross-linguistic developmental
study. Hillsdale, NJ: Lawrence Erlbaum.
Berman, R., & Verhoeven, L. (2002). Crosslinguistic perspectives on developing text produc-
tion abilities in speech and writing. Written Language and Literacy, 5,  1–44.
Bliss-Moreau, E., Bauman, M. D., & Amaral, D. G. (2011). Neonatal amygdala lesions result in
globally blunted affect in adult rhesus macaques. Behavioral Neuroscience, 125(6), 848.
Blonder, L. X., Bowers, D., & Heilman, K. M. (1991). The role of the right hemisphere in emo-
tional communication. Brain, 114(3), 1115–1127.
Blonder, L. X., Burns, A. F., Bowers, D., Moore, R. W., & Heilman, K. M. (1993). Right-
hemisphere facial expressivity during natural conversation. Brain and Cognition,
21(1),  44–56.
Blonder, L. X., Heilman, K. M., Ketterson, T., Rosenbek, J., Raymer, A., Crosson, B., . . . Rothi, L.
G. (2005). Affective facial and lexical expression in aprosodic versus aphasic stroke patients.
Journal of the International Neuropsychological Society, 11(6), 677–685.
Booth, J. R., Burman, D. D., Meyer, J. R., Lei, Z., Choy, J., Gitelman, D. R., . . . Mesulam, M. M.
(2003). Modality-specific and -independent developmental differences in the neural sub-
strate for lexical processing. Journal of Neurolinguistics, 16(4–5), 383–405.
Borod, J. C. (Ed.). (2000). The neuropsychology of emotion. New York: Oxford
University Press.
Borod, J. C., Bloom, R. L., Brickman, A. M., Nakhutina, L., & Curko, E. A. (2002). Emotional
processing deficits in individuals with unilateral brain damage. Applied Neuropsychology,
9(1),  23–36.
Borod, J. C., Koff, E., Lorch, M. P., & Nicholas, M. (1985). Channels of emotional expression in
patients with unilateral brain damage. Archives of Neurology, 42(4), 345–348.
Borod, J. C., Rorie, K. D., Pick, L. H., Bloom, R. L., Andelman, F., Campbell, A. L., . . . Sliwinski,
M. (2000). Verbal pragmatics following unilateral stroke: Emotional content and valence.
Neuropsychology, 14(1), 112–124.
Bottini, G., Corcoran, R., Sterzi, R., Paulesu, E., Schenone, P., Scarpa, P., . . . Frith, C. D. (1994).
The role of the right hemisphere in the interpretation of figurative aspects of language: A
positron emission tomography activation study. Brain, 117, 1241–1253.
Broca, P. ([1861] 1960). Remarks on the seat of the faculty of articulate language, followed
by an observation of aphemia. In G. von Bonin (Ed.), Some papers on the cerebral cortex.
Oxford: Blackwell Scientific Publications.
Brown, T. T., Lugar, H. M., Coalson, R. S., Miezin, F. M., Petersen, S. E., & Schlaggar, B. L.
(2005). Developmental changes in human cerebral functional organization for word genera-
tion. Cerebral Cortex (New York, NY, 1991), 15(3), 275–290.
Bruner, J. S. (1975). The ontogenesis of speech acts. Journal of Child Language, 2(1),  1–19.
Catani, M., Jones, D. K., & ffytche, D. H. (2005). Perisylvian language networks of the human
brain. Annals of Neurology, 57(1),  8–16.
Chilosi, A. M., Pecini, C., Cipriani, P., Brovedani, P., Brizzolara, D., Ferretti, G., . . . Cioni, G.
(2005). Atypical language lateralization and early linguistic development in children with
focal brain lesions. Developmental Medicine and Child Neurology, 47, 725–730.
Dehaene-Lambertz, G., Dehaene, S., & Hertz-Pannier, L. (2002). Functional neuroimaging of
speech perception in infants. Science, 298, 2013–2015.
Demir, O. E., Levine, S. C., & Goldin-Meadow, S. (2010). Narrative skill in children with early uni-
lateral brain injury: A possible limit to functional plasticity. Developmental Science, 13, 636–647.
De Schonen, S., & Mathivet, E. (1989). First come first serve: A scenario about the development
of hemispheric specialization in face processing in infancy. European Bulletin of Cognitive
Psychology, 9,  3–44.
Fair, D., Choi, A., Dosenbach, Y., Coalson, R., Miezin, F., Petersen, S., Schlaggar, B. (2009).
The functional organization of trial-related activity in lexical processing after early left
hemispheric brain lesions: An event-related fMRI study. Brain and Language, 114, 135–146.
doi: 10.1371/journal.pcbi.1000381
Feldman, H. M. (1994). Language development after early brain injury: A replication study. In
H. Tager-Flusberg (Ed.), Constraints on language acquisition: Studies of atypical children (pp.
75–90). Hillsdale, NJ: Lawrence Erlbaum.
Feldman, H. M. (2005). Language learning with an injured brain. Language Learning and
Development, 1, 265–288.
Feldman, H. M., Holland, A. L., Kemp, S. S., & Janosky, J. E. (1992). Language development
after unilateral brain injury. Brain and Language, 42, 89–102.
Fenson, L., Dale, P. S., Reznick, J. S., Thal, D., Bates, E., Hartung, J. P., Pethick, S., & Reilly, J. S.
(1993). The MacArthur Communicative Development Inventories: User’s guide and technical
manual. San Diego: Singular Publishing Group.
Fusar-Poli, P., Placentino, A., Carletti, F., Landi, P., Allen, P., Surguladze, S., . . . Politi, P. (2009).
Functional atlas of emotional faces processing: A voxel-based meta-analysis of 105 func-
tional magnetic resonance imaging studies. Journal of Psychiatry & Neuroscience, 34(6), 418.
Gaillard, W. D., Sachs, B. C., Whitnah, J. R., Ahmad, Z., Balsamo, L. M., Petrella, J.
R., . . . Grandin, C. B. (2003). Developmental aspects of language processing: fMRI of verbal
fluency in children and adults. Human Brain Mapping, 18(3), 176–185.
Gee, D. G., Humphreys, K. L., Flannery, J., Goff, B., Telzer, E. H., Shapiro, M., . . . Tottenham, N.
(2013). A developmental shift from positive to negative connectivity in human amygdala–
prefrontal circuitry. The Journal of Neuroscience, 33(10), 4584–4593.
Goodglass, H. (1993). Understanding aphasia. San Diego, CA: Academic Press.
Grossmann, T., & Johnson, M. H. (2007). The development of the social brain in human in-
fancy. European Journal of Neuroscience, 25(4), 909–919.
Haxby, J. V., Hoffman, E. A., & Gobbini, M. I. (2000). The distributed human neural system for
face perception. Trends in Cognitive Sciences, 4(6), 223–233.
Haxby, J. V., Hoffman, E. A., & Gobbini, M. I. (2002). Human neural systems for face recogni-
tion and social communication. Biological Psychiatry, 51(1),  59–67.
Herba, C., & Phillips, M. (2004). Annotation: Development of facial expression recognition
from childhood to adolescence: Behavioural and neurological perspectives. Journal of Child
Psychology and Psychiatry, 45(7), 1185–1198.
Holland, S. K., Plante, E., Weber Byars, A., Strawsburg, R. H., Schmithorst, V. J., & Ball, W. S.,
Jr. (2001). Normal fMRI brain activation patterns in children performing a verb generation
task. NeuroImage, 14, 837–843.
Holland, S. K., Vannest, J., Mecoli, M., Jacola, L. M., Tillema, J., Karunanayaka, P. R., . . . Byars,
W. (2009). Functional MRI of language lateralization during development in children.
International Journal of Audiology, 46(9), 533–551.
Jacola, L. M., Schapiro, M. B., Schmithorst, V. J., Byars, A. W., Strawsburg, R. H., Szaflarski,
J. P., . . . Holland, S. K. (2006). Functional magnetic resonance imaging reveals atypical
language organization in children following perinatal left middle cerebral artery stroke.
Neuropediatrics, 37(1), 46.
James, W. ([1890] 1981). The principles of psychology, Vol. I. Cambridge, MA: Harvard
University Press.
Johnson, M. H. (2005). Subcortical face processing. Nature Reviews Neuroscience, 6(10),
766–774.
Johnson, M. H., Griffin, R., Csibra, G., Halit, H., Farroni, T., De Haan, M., . . . Richards, J.
(2005). The emergence of the social brain network: Evidence from typical and atypical de-
velopment. Development and Psychopathology, 17(03), 599–619.
Johnson, M. H., & Morton, J. (1991). Biology and cognitive development: The case of face recogni-
tion. Oxford: Blackwell.
Joanette, Y., & Brownell, H. H. (Eds.). (1993). Narrative discourse in neurologically impaired and
normal aging adults. San Diego, CA: Singular.
Joanette, Y., Goulet, P., Hannequin, D., & Boeglin, J. (1990). Right hemisphere and verbal com-
munication. New York: Springer-Verlag.
Jung-Beeman, M. (2005). Bilateral brain processes for comprehending natural language.
Kaplan, J. A., Brownell, H. H., Jacobs, J. R., & Gardner, H. (1990). The effects of right hem-
isphere damage on the pragmatic interpretation of conversational remarks. Brain and
Language, 38(2), 315–333.
Lai, P. T., & Reilly, J. S. (2015). Language and affective facial expression in children with peri-
natal stroke. Brain and Language, 147,  85–95.
Lenneberg, E. H. (1967). Biological foundations of language. New York: Wiley.
Leppänen, J. M., & Nelson, C. A. (2008). Tuning the developing brain to social signals of
emotions. Nature Reviews Neuroscience, 10(1),  37–47.
Lichtheim, L. (1885). On aphasia. Brain, 7(4), 433–484.
Liegeois, F., Connelly, A., Cross, J. H., Boyd, S. G., Gadian, D. G., Vargha‐Khadem, F., &
Baldeweg, T. (2004). Language reorganization in children with early‐onset lesions of the left
hemisphere: An fMRI study. Brain, 127(6), 1229–1236.
Lynch, J. K., & Nelson, K. B. (2001). Epidemiology of perinatal stroke. Current Opinion in
Pediatrics, 13(6), 499–505.
Lynch, J. K., Hirtz, D. G., DeVeber, G., & Nelson, K. B. (2002). Report of the National Institute
of Neurological Disorders and Stroke workshop on perinatal and childhood stroke.
Pediatrics, 109(1), 116–123.
Macwhinney, B., Feldman, H., Sacco, K., & Valdès-Pérez, R. (2000). Brain and Language, 71,
400–431.
Marchman, V. A., Miller, R., & Bates, E. A. (1991). Babble and first words in children with focal
brain injury. Applied Psycholinguistics, 12,  1–22.
Mayer, M. (1969). Frog, where are you? New York: Dial Press.
Mills, D. L., Coffey-Corina, S., & Neville, H. J. (1997). Language comprehension and cerebral
specialization from 13 to 20 months. Developmental Neuropsychology, 13, 397–446.
Nass, R., & Koch, D. (1987). Temperament differences in toddlers with early unilateral right-
and-left brain damage. Developmental Neuropsychology, 3(2),  93–99.
Nelson, C. A., & de Haan, M. (1996) Neural correlates of infants’ visual responsiveness to facial
expression of emotion. Developmental Psychobiology, 29, 577–595.
Nippold, M. (1998). Later language development: The school-age and adolescent years (2nd ed.).
Austin, TX: Pro-Ed.
Oster, H. (1978). Facial expression and affect development. In M. Lewis & L. Rosenblum (Eds.),
The development of affect. New York: Plenum.
Pascalis, O., de Martin de Viviés, X., Anzures, G., Quinn, P. C., Slater, A. M., Tanaka, J. W., &
Lee, K. (2011). Development of face processing. Wiley Interdisciplinary Reviews: Cognitive
Science, 2(6), 666–675. doi: 10.1002/wcs.146
Pascual-Leone, A., Amedi, A., Fregni, F., & Merabet, L. B. (2005). The plastic human brain
cortex. Annual Review of Neuroscience, 28, 377–401.
Pearn, J., & Gardner-Thorpe, C. (2002). Jules Cotard (1840–1889): His life and the unique syn-
drome which bears his name. Neurology, 58, 1400–1403.
Pell, M. D. (2006). Judging emotion and attitudes from prosody following brain damage.
Progress in Brain Research, 156, 303–317.
Price, C. J. (2010). The anatomy of language: A review of 100 fMRI studies published in 2009.
Annals of the New York Academy of Sciences, 1191(1),  62–88.
Raja Beharelle, A., Dick, A. S., Josse, G., Solodkin, A., Huttenlocher, P. R., Levine, S. C., &
Small, S. L. (2010). Left hemisphere regions are critical for language in the face of early left
focal brain injury. Brain, 133, 1707–1716.
Ramon y Cajal, S. (1904). Histology of the nervous system, Vol. 2. N. Swanson & L. Swanson
(Trans.). . New York: Oxford University Press.
Reilly, J. S., Bates, E. A., & Marchman, V. (1998) Narrative discourse in children with early focal
brain injury. Brain and Language, 61, 335–375.
Reilly, J. S., Levine, S. C., Nass, R., & Stiles, J. (2008). Brain plasticity: Evidence from chil-
dren with perinatal brain injury. In J. Reed & J. Warner-Rogers (Eds.), Child neuropsy-
chology: Concepts, theory, and practice (pp. 58–91). Oxford: Blackwell.
Reilly, J. S., Levine, S. C., Trauner, D. A., & Nass, R. (2012). Neural plasticity and cognitive devel-
opment: Insights from children with perinatal brain injury. New York: Oxford University Press.
Reilly, J., Losh, M., Bellugi, U., & Wulfeck, B. (2004). “Frog, where are you?” Narratives in chil-
dren with specific language impairment, early focal brain injury, and Williams Syndrome.
Brain and Language, 88, 229–247.
Reilly, J., & Polse, L. (2016). Perspectives on spoken and written language: Evidence from
English speaking children. In J. Perera, M. Aparici, E. Rosado, & N. Salas (Eds.), Written
and spoken language development across the lifespan (pp. 125– 140). Cham: Springer
International.
Reilly, J. S., Stiles, J., Larsen, J., & Trauner, D. (1995). Affective facial expression in infants with
focal brain damage. Neuropsychologia, 33(1),  83–99.
Reilly, J. S., Wasserman, S., & Appelbaum, M. (2013). Later language development in narratives
in children with perinatal stroke. Developmental Science, 16(1),  67–83.
Reilly, J., Zamora, A., & McGivern, R. F. (2005). Acquiring perspective in English: The develop-
ment of stance. Journal of Pragmatics, 37(2), 185–208.
Ross, E. D. (1981). The aprosodias: Functional- anatomic organization of the affective
components of language in the right hemisphere. Archives of Neurology, 38(9), 561–569.
Rowe, M. L., Levine, S., Fisher, J., & Goldin-Meadow, S. (2009). The joint effects of biology and
input on the language development of brain-injured children. Developmental Psychology, 45,
90–102.
Saccuman, M. C., Dick, F., Kwiatowski, M., Moses, P., Bates, E., Perani, D., . . . Wulfeck, B.
(2006). Language processing in children and adolescents with early unilateral focal brain le-
sion: A fMRI study. Paper presented at the Meeting of the Organization for Human Brain
Mapping, Florence, Italy.
Schlaggar, B. L., Brown, T. T., Lugar, H. M., Visscher, K. M., & Miezin, F. M., & Petersen, S. E.
(2002). Functional neuroanatomical differences between adults and school-age children in
the processing of single words. Science, 296(5572), 1476–1479.
Schmithorst, V. J., Holland, S. K., & Plante, E. (2007). Object identification and lexical/se-
mantic access in children: A functional magnetic resonance imaging study of word-picture
matching. Human Brain Mapping, 28(10), 1060–1074.
Slobin, D. I. (1997). The crosslinguistic study of language acquisition, Vol. 1: The data (1985); Vol.
2: Theoretical issues (1985); Vol. 3 (1992); Vol. 4 (1997); Vol. 5: Expanding the contexts (1997).
Hillsdale, NJ: Lawrence Erlbaum.
Staudt, M., Lidzba, K., Grodd, W., Wildgruber, D., Erb, M., & Krägeloh-Mann, I. (2002). Right-
hemispheric organization of language following early left-sided brain lesions: Functional
MRI topography. NeuroImage, 16, 954.
Stern, D. N. (2009). The first relationship: Infant and mother. Cambridge, MA: Harvard
University Press.
Stiles, J. (2008). The fundamentals of brain development: Integrating nature and nurture.
Cambridge, MA: Harvard University Press.
Stiles, J., Brown, T. T., Haist, F., & Jernigan, T. L. (2013). Brain and cognitive development. In D.
Kuhn & R. S. Siegler (Eds.), Handbook of child psychology (7th ed.). New York: John Wiley
& Sons.
Stiles, J., Reilly, J. S., Levine, S. C., Trauner, D. A., & Nass, R. (2012). Neural plasticity and cog-
nitive development: Insights from children with perinatal brain injury. New York: Oxford
University Press.
Swartz, J. R., Carrasco, M., Wiggins, J. L., Thomason, M. E., & Monk, C. S. (2014). Age-related
changes in the structure and function of prefrontal cortex–amygdala circuitry in children
and adolescents: A multi-modal imaging approach. NeuroImage, 86, 212–220.
Szaflarski, J. P., Holland, S. K., Schmithorst, V. J., & Byars, A. W. (2006). fMRI study of language
lateralization in children and adults. Human Brain Mapping, 27(3), 202–212.
Szaflarski, J. P., Allendorfer, J. B., Byars, A. W., Vannest, J., Dietz, A., Hernando, K. A., &
Holland, S. K. (2014). Age at stroke determines post- stroke language lateralization.
Restorative Neurology and Neuroscience, 32(6), 733–742.
Thal, D. J., Marchman, V. A., Stiles, J., Aram, D., Trauner, D., Nass, R., & Bates, E. (1991).
Early lexical development in children with focal brain injury. Brain and Language, 40,
491–527.
Thal, D. J., Reilly, J., Seibert, L., Jeffries, R., & Fenson, J. (2004). Language development in chil-
dren at risk for language impairment: Cross-population comparisons. Brain and Language,
88(2), 167–179.
Thomas, K. M., Drevets, W. C., Whalen, P. J., Eccard, C. H., Dahl, R. E., Ryan, N. D., & Casey,
B. J. (2001). Amygdala response to facial expressions in children and adults. Biological
Psychiatry, 49(4), 309–316.
Tillema, J. M., Byars, A. W., Jacola, L. M., Schapiro, M. B., Schmithorst, V. J., Szaflarski, J. P., &
Holland, S. K. (2008). Cortical reorganization of language functioning following perinatal
left MCA stroke. Brain and Language, 105, 99–111.
Todd, R. M., Evans, J. W., Morris, D., Lewis, M. D., & Taylor, M. J. (2010). The changing face of
emotion: Age-related patterns of amygdala activation to salient faces. Social Cognitive and
Affective Neuroscience, 6(1),  12–23.
Tolchinsky, L., Johansson, V., & Zamora, A. (2002). Text openings and closings in writing and
speech: Autonomy and differentiation. Written Language and Literacy, 5, 219–254.
Travis, K. E., Leonard, M. K., Brown, T. T., Hagler, D. J., Curran, M., Dale, A. M., . . . Halgren,
E. (2011). Spatiotemporal neural dynamics of word understanding in 12-to 18-month-old-
infants. Cerebral Cortex, 21(8), 1832–1839.
Tzourio-Mazoyer, N., De Schonen, S., Crivello, F., Reutter, B., Aujard, Y., & Mazoyer, B. (2002).
Neural correlates of woman face processing by 2-month-old infants. NeuroImage, 15(2),
454–461.
Vicari, S., Albertoni, A., Chilosi, A. M., Cipriani, P., Cioni, G., & Bates, E. (2000). Plasticity
and reorganization during language development in children with early brain injury. Cortex,
36,  31–46.
and sentence processing. NeuroImage, 30(4), 1414–1432.
Weckerly, J., Wulfeck, B., & Reilly, J. (2004). The development of morphosyntactic ability in
atypical populations: The acquisition of tag questions in children with early focal lesions and
children with specific-language impairment. Brain and Language, 88(2), 190–201.
Weintraub, S., & Mesulam, M. (1983). Developmental learning disabilities of the right hem-
isphere: Emotional, interpersonal, and cognitive components. Archives of Neurology,
40(8), 463.
Wernicke C. (1874). Der aphasische Symptomenkomplex. Breslau: Cohn & Weigert.
Wulfeck, B., Bates, E., Krupa-Kwiatkowski, M., & Saltzman, D. (2004). Grammaticality sensi-
tivity in children with early focal brain injury and children with specific language impair-
ment. Brain and Language, 88(2), 215–228.
Chapter 11
T he Neu roling u i st i c s
of Bilingua l i sm
Plasticity and Control
David W. Green and Judith F. Kroll
Introduction
Language use is a form of communicative action, and the languages we use comple-
ment and extend our ability to act jointly with others. To achieve our communicative
goals, we recruit diverse representations and skills. In a conversation, for example, we
recruit skills in reading mental states, drawing inferences, and mapping experiential
states into spoken words. For many of us, achieving our communicative goals involves
using more than one language over the course of the day. Our contention is that the na-
ture of such usage exerts profound effects on the brain.
Strikingly, bilingualism appears protective of cognitive decline in the elderly (Bak,
Nissan, Allerhand, & Deary, 2014; see also Kavé, Eyal, Shorek, & Cohen-Mansfield,
2008, and Perquin et al., 2013, for data on multilingual speakers) and delays the onset of
dementia relative to monolingual samples by an average of 4–5 years independent of so-
cioeconomic status, immigration status, or language type (Alladi et al., 2013; Bialystok,
Craik, & Freedman, 2007; Woumans et al., 2015; see Bak, 2016, for a discussion of pos-
sible counter-evidence and an appraisal of potential confounds). It also appears predic-
tive of cognitive function post-stroke (Alladi et al., 2016). Such neuroprotective effects
plausibly derive from the adaptive changes required to represent and use two languages.
Why should there be adaptive changes?
We have no theory yet of how the use of more than one language changes the brain
and on how such changes may affect even nonlinguistic performance. Instead, we start
from a single premise and explore the questions that arise from it and the neural data
that bear on them. Our premise is that the use of more than one language imposes
262 David W. Green and Judith F. Kroll
additional demand relative to the use of a single language. It is this increased demand,
and how it is met, that drives plastic change. Our premise gives rise to two immediate
questions: What kinds of demand are there? How might the brain respond to increased
demands? We consider these questions in the next two sections.
Two Types of Demand
We can think about the nature of the demand by distinguishing between the language
network that captures a person’s knowledge of each language (e.g., its syntactic patterns
and vocabulary) and how it is used (the network or circuits involved in language con-
trol). By “control processes” we refer broadly to processes that allow individuals to per-
form a range of language tasks (e.g., naming a picture rather than describing it), making
a request or issuing a command, and speaking one language rather than another.
Neuropsychological data not only support the face validity of the distinction be-
tween the language network and its control, but also indicate that control processes
are essential to normal language use. In bilingual speakers, stroke damage can im-
pair some aspects of language control while sufficiently sparing the language network
(Green, 1986). For example, Fabbro, Skrap, and Aglioti (2000) reported the case of an
Italian-Friulian speaker (S. J.) whose lesion included the left prefrontal cortex, part of
the anterior cingulate, and the left striatum. Clausal processing for speech compre-
hension and production was intact for both languages, but S. J. was unable to avoid
switching into Friulian (his L1) even when addressing an Italian speaker who spoke no
Friulian. Likewise, when required to speak Friulian only, S. J. would switch into Italian
(his L2).
Evidently, the use of more than one language must alter the language network
compared to that of a monolingual speaker of those languages, but it may also alter
the network involved in its control. We suppose that language control recruits neural
regions involved in the control of action in general (e.g., Green, 1986, 1998; see
Pliatsikas & Luk, 2016, for a short review of the overlap of brain regions involved).
The language network and the control network must also interact. Indeed, domain-
general control processes operate in Broca’s region classically held to be specialized
for language (Fedorenko, Duncan, & Kanwisher, 2012; Snyder et al., 2010). Such
interpenetration undermines the notion that any linguistically relevant region is
encapsulated in the sense of Fodor (Fedorenko & Thompson-Schill, 2014) and may
prove to be the norm in all regions comprising the language network. Indeed, as
will be seen, we identify common regions. Granted the distinction between the lan-
guage network and its control, we distinguish between the demand on the language
network and the demand on the control network. Interpreting adaptive changes
requires us to consider both types of demand. In order to do so, we need to consider
our second question.
The Neurolinguistics of Bilingualism 263
Neural Response to Increased Demand
What kinds of changes in the neural substrates might be expected in the light of
increased demand? Responding to increased demand may lead to an increase in net-
work efficiency or in the patterns of connectivity. Increased neural resources may be
allocated to perform the neurocomputations required. Indeed, radically distinct re-
gions might become recruited. However, it is surely more plausible that regions and
networks specialized for the processing of lexical, syntactic, and prosodic information
for one language are recruited to process information of the same type in a person’s
other language (see Paz-Alonso, Oliver, Quiñones, & Carreiras, Chapter 24 in this
volume). There should be neural convergence (e.g., Consonni et al., 2013; Green, 2003).
Adaptive change might then be detected in structural properties of the tissue (e.g., gray
matter density). Within these common regions, neural representations for each lan-
guage should be interleaved so that bilinguals can selectively use one language rather
than another (Green, 2008; Paradis, 2004). However, on this issue of neural conver-
gence, opinion diverges. Some researchers acknowledge convergence only for vocab-
ulary, but not for syntax. We present evidence in later sections that supports neural
convergence.
The Language-C ontrol Network
Language- control processes are critical to both monolingual (e.g., Snyder et al.,
2010) and bilingual speakers. Behavioral and event-related potential (ERP) studies (see
Leckey & Federmeier, Chapter 3 in this volume) suggest that in bilingual speakers the
two languages compete to control output (see Kroll, Dussias, Bice, & Perrotti, 2015, for
a review). Control processes are needed for a range of purposes (Green & Abutalebi,
2013), for example to implement the intention to speak in one language rather than an-
other, to monitor and regulate interference from the nontarget language (Abutalebi &
Green, 2007; Costa, Miozzo, & Caramazza, 1999), though not to the same extent when
the other language is a sign language (Emmorey, Giezen, & Gollan, 2016; see Corina
& Lawyer, Chapter 16 in this volume). Control processes are also needed to disengage
from using one language and to engage in using the other, or to code-switch between
different languages within a conversational turn. Research has identified a network of
cortical, subcortical, and cerebellar regions that orchestrate these processes and has
identified some of their functions, but has yet to reveal details of the temporal dynamics
that their orchestration requires. The primary regions involved are the dorsal anterior
cingulate cortex (dACC)/pre-supplementary motor (pre-SMA) area, the left prefrontal
cortex, the left caudate, and the inferior parietal lobules bilaterally (Abutalebi & Green,
2007), together with control input from the right prefrontal cortex, the thalamus and
the putamen of the basal ganglia, and the cerebellum (Abutalebi & Green, 2016; Green
& Abutalebi, 2013). We offer a selective review of the functions of each of these regions in
the following section.
The dACC/pre-SMA Complex

ACC (anterior cingulate cortex) activity is usually related to conflict and error
monitoring (Botvinick, Braver, Barch, Carter, & Cohen, 2001) and activates during
cross-linguistic conflict (Rodriguez-Fornells et al., 2005; van Heuven, Schriefers,
Dijkstra, & Hagoort, 2008) and language switching (e.g., Guo, Liu, Misra, & Kroll,
2011; Hosoda, Hanakawa, Nariai, Ohno, & Honda, 2012; Luk, Green, Abutalebi, &
Grady, 2012). Activation also increases when a relatively proficient L2 is re-engaged
after a period of L1 immersion (Tu et al., 2015). It increases too when naming pictures
in L2 after naming in L1 (e.g., Branzi, Della Rosa, Canini, Costa, & Abutalebi, 2015).
In both cases, there is an increased need to monitor potential responses before overt
speech. The increased demands in bilingual compared to monolingual speakers ap-
pear to tune its responsiveness to nonlinguistic conflict. Bilinguals, as compared to
monolinguals, displayed reduced interference on a nonverbal flanker task (Abutalebi
et al., 2012). Structurally, their performance correlated with increased gray matter
density in the dorsal ACC, and functionally, bilinguals displayed a more efficient use
of this structure.
The Left Inferior and Right Frontal Gyri

In contrast to dACC/pre-SMA, the left inferior frontal gyrus (IFG) is involved in re-
sponse selection rather than conflict monitoring. It is commonly activated in language
switching (e.g., Hernandez, Martinez, & Kohnert, 2000; Hosoda et al., 2012; Lehtonen
et al., 2005) and during word generation, picture naming, and verbal fluency (see
Abutalebi & Green, 2007), especially when producing words in the weaker language
(De Bleser et al., 2003; Kovelman, Baker, & Pettito, 2008; Marian, Spivey, & Hirsch,
2003; Parker-Jones et al., 2012; Perani et al., 2003). In contrast with the response se-
lection processes of the left IFG, the right prefrontal cortex appears to be associated
with domain-general inhibitory control (Aron, Robbins, & Poldrack, 2014). Indeed,
in the study by Branzi et al. (2015), switching back into L1 after naming in L2 induced
increased activation consistent with the need to release inhibition of the dominant lan-
guage. By contrast, they report significant deactivation for naming in L2 after naming
in L1 (see Videsott et al., 2010, for a comparable pattern of response). Conceivably,
increased additional demand on these regions for bilingual speakers may account for
the greater cortical thickness in bilateral inferior frontal regions for these speakers
(Klein, Mok, Chen, & Watkins, 2014).
The Inferior Parietal Lobules

The inferior parietal lobules comprise two distinct gyri: the supramarginal gyrus and the
angular gyrus. We know that a posterior region of the supramarginal gyrus is responsive
to vocabulary knowledge in monolingual and bilingual speakers (e.g., Lee et al., 2007;
Mechelli et al., 2004). This region appears ideally placed to represent vocabulary, as it is
connected to an anterior parietal region that processes phonology and the angular gyrus
that processes meaning. We consider the representation and processing of vocabulary
in more detail in a later section.
Inferior parietal lobules are also recruited in the context of attentional tasks, both
in orienting attention in a voluntary manner and in responding to stimulus changes
(Majerus et al., 2010; Shomstein, 2012). We should expect their involvement in language
switching, and indeed both clinical (e.g., Pötzl, 1925) and functional neuroimaging data
(e.g., Price, Green, & von Studnitz, 1999) confirm their involvement, but their coordina-
tion with other regions involved in language switching remains an open question.
Subcortical Structures: The Left Caudate, Putamen,

and Thalamus
Green and Abutalebi (2013) postulated that a circuit involved in the detection of sa-
lient cues (e.g., Aron, Behrens, Smith, Frank, & Poldrack, 2007) would be important
in ensuring sensitivity to the language context and in initiating a change in language.
Neural circuitry would support such a role. Regions in the right inferior frontal cortex
connect to the thalamus, which in turn connects two subcortical regions involved in lan-
guage control: the left head of caudate and the putamen (e.g., Smith, Surmeier, Redgrave,
& Kimura, 2011). The thalamus plays a role in shifting attention and action selection and
connects to anterior and posterior regions of the left IFG (Ford et al., 2013). We sup-
pose, then, that it is likely to play an important role in language production in bilingual
speakers by aiding the selection of intended lexical and semantic representations, espe-
cially in the weaker or less exposed language in highly proficient bilingual speakers (see
Consonni et al., 2013, for support).
Language switching and selection in production or in comprehension elicits ac-
tivity in the left caudate (e.g., Abutalebi et al., 2007; Crinion et al., 2006; Lehtonen
et al., 2005; Price et al., 1999) and lesions to the left caudate can induce a breakdown
in language control (e.g., Gil Robles et al., 2005; Green & Abutalebi, 2008, for review).
In contrast to dACC/pre-SMA activation, activation in the left caudate is contingent
on language proficiency, with increased activation associated with a switch to the
least proficient language (Abutalebi et al., 2013a). Such an outcome is consistent with
a more general role of the left caudate in the control of action, where it is most ac-
tive in overcoming habitual action plans (Ali, Green, Kherif, Devlin, & Price, 2010;
Shadmehr & Holcomb, 1999).
Left putamen activity may reflect control of articulatory processes. In monolingual

speakers (see Oberhuber et al., 2013), the left putamen is involved in overt speech pro-
duction, with the anterior region involved in initiating novel articulatory sequences (as
in the production of pseudo-words) and a posterior region associated with known or
memory-guided sequences (producing words). The putamen is therefore likely to be
sensitive to the articulatory demands of a second language. Indeed, Burgaleta, Sanjuán,
Ventura-Campos, Sebastián-Gallés, and Ávila (2016) report volumetric increases in
this structure, along with the thalamus, in simultaneous bilinguals compared to mon-
olingual speakers (see also Abutalebi et al., 2013b). Its activation is also likely to be
modulated by the demands for language control. In simultaneous interpreting, for in-
stance, speakers must avoid articulating words they hear while producing words in the
target language. In this circumstance, putamen activation varies with the duration of the
overlap between listening and speaking (Hervais-Adelman, Moser-Mercer, Michel, &
Golestani, 2015).
The Cerebellum
We supposed earlier that language control recruits structures involved in the control of
action in general. The cerebellum is a further critical structure. It is linked to all the key
regions of the language-control network (Green & Abutalebi, 2013), including the right
inferior frontal cortex, which, as we noted, is connected via the thalamus to caudate and
putamen regions of the basal ganglia. Research has established its contribution to a wide
range of language and cognitive functions in addition to motor processes (see Tyson,
Lantrip, & Roth, 2014, for a review). For example, the right cerebellum forms a circuit
with the left inferior frontal cortex (Krienen & Buckner, 2009). Clinical data indicate
that such a circuit supports morphosyntactic processing: lack of right cerebellar activa-
tion impaired speech production in bilingual speakers (Marien, Engelborghs, Fabbro, &
De Deyn, 2001; Silveri, Leggio, & Molinari, 1994).
The Language Network
What are the neural substrates of the language network? On a classical view, the lan-
guage network comprises a set of regions in left frontal and temporal cortices to-
gether with the inferior parietal cortex. Multiple long-range white matter fiber bundles
(pathways) interconnect these regions. Together, they are held to support language
production and comprehension (Vigneau et al., 2006). Most interestingly, distinct
dorsal and ventral pathways seem to support the syntactic, semantic, and sensori-
motor processes required in language use (Friederici, 2015), though precisely how these
pathways work together to integrate sound, syntax, and meaning in speech production
is underdetermined. We can, though, make use of the functional dissociations implied.
They allow us to determine whether the neural signatures for processing vocabulary or
processing syntax in a second language are the same or different from those identified in
the first language; accordingly, we describe them briefly.
Broadly speaking, regions in the dorsal pathways mediate sensorimotor and syn-
tactic processes, whereas those in ventral pathways mediate semantic processing. The
dorsal pathway from the temporal cortex via the inferior parietal area to the premotor
cortex (the superior longitudinal fasciculus) supports sensory to motor processing, as in
speech repetition (Hickok & Poeppel, 2007; Saur et al., 2008). A second dorsal pathway
(the arcuate fasciculus; Wernicke, 1874), which directly connects the pars opercularis
(BA 44 of Broca’s area) and Wernicke’s area, appears necessary to the processing of syn-
tactically complex sentences (Wilson et al., 2011). The pars opercularis may support
such processing because it is involved in phonemic discrimination and phonological re-
hearsal and so may act to retain the wording of the sentence. Phonological rehearsal may
be critical, too, in the learning of new vocabulary. Undoubtedly, other dorsal pathways
are likely to support speech production. For example, the frontal aslant tract connecting
the pars opercularis to the superior frontal regions (Catani et al., 2012) may support
speech planning and initiation (Dick, Bernal, & Tremblay, 2014).
Of the designated ventral pathways, one supports semantic processing through its
linking of BA 45, the pars triangularis in Broca’s area and BA 47, to occipito-temporo-
parietal regions, via the extreme capsule fiber system (the longitudinal inferior-fronto-
occipital fasciculus). A second pathway (the uncinate fasciculus) connects the anterior
temporal cortex (a region viewed as a conceptual hub) and the orbitofrontal cortex. It
serves to integrate the meaning of lexical concepts (e.g., cherry-cake vs. bread-cake;
Feng, Chen, Zhu, & Wang, 2016) and so may complement the dorsal pathway for syntax
by processing local phrase structure (e.g., Friederici, 2015; Griffiths, Marslen-Wilson,
Stamatakis, & Tyler, 2013).
This view of the language network is noticeably cortico-centric, but subcortical and
cerebellar regions are also pertinent to language comprehension and production (see
Price, 2012, for an overview of the regions involved). Recent work provides good evi-
dence of connections between regions in Broca’s area and basal ganglia structures (Ford
et al., 2013; Kotz, Anwander, Axer, & Knösche, 2013), also via the insula (Oh, Duerden,
& Wang, 2014), and between the left inferior frontal cortex and the right cerebellum
(Krienen & Buckner, 2009; see also Tyson et al., 2014). Basal ganglia structures are likely
to be integral to sentence formation.
On one proposal, consistent with the multiple reciprocal connections between the
frontal cortex and the basal ganglia, control signals serve to update, maintain, or output
the constructed sentence plan (Kriete, Noelle, Cohen, & O’Reilly, 2013). For bilingual
speakers, language-control signals will be needed to select a relevant structure in the in-
tended language and to select among competing items that may be from both languages
(see also Stocco, Yamasaki, Natalenko, & Prat, 2014, for a related view).
Distinct activation profiles have been identified during sentence production for
two basal ganglia structures: the putamen and the caudate. The putamen contributes
to articulation (together with cortical areas such as the pars opercularis) during both
sentence repetition and sentence generation, whereas the caudate supports the process
of selecting sentence structure and form during sentence generation (Argyropoulos,
Tremblay, & Small, 2013). We know of no comparable published study with bilingual
speakers that allocates a functional role to these two structures during sentence gener-
ation. However, interesting data from a simultaneous interpretation task support the
role of the caudate in language selection and indicate the sensitivity of the putamen
to increased articulatory demand (Hervais-Adelman et al., 2015). In other research,
increased articulatory demand during sentence generation in a less proficient L2 also
increases functional connectivity of the left putamen with the left pars opercularis
(Dodel et al., 2005).
Interestingly, both the pars opercularis, implicated in phonology and syntax, and
the pars triangularis, implicated in semantic processing, are connected to the anterior
putamen. Ford et al. (2013) proposed that this arrangement ensures that the intended
semantic response is articulated with appropriate phonemes. Overall, the fronto-
subcortical circuits (along with thalamic input) may serve to sharpen and fine-tune
output during lexical selection, with the caudate actively involved in selecting the rele-
vant linguistic structure. These regions and circuits should be sensitive to the demands
of bilingual speech.
Cerebellar regions also play an important role in both overt speech (e.g., verbal flu-
ency) and inner speech (Tyson et al., 2014). In the case of verbal fluency, gray matter
density in bilateral inferior cerebellar regions correlated with the number of words
produced during letter and semantic fluency tasks for both L1 and L2 (Grogan et al.,
2009). In line with the role of the cerebellum in sentence processing, gray matter cer-
ebellar volume, though not specifically right cerebellar volume, correlated with effi-
cient processing of L2 morphosyntax in immersed, proficient L2 speakers (Pliatsikas,
Johnstone, & Marinis, 2014).
Functional imaging studies using sentence production and comprehension tasks
have yet to elucidate the set of cerebellar contributions to language processing, but
one general function may be predictive (Ito, 2006). Repetitive transcranial magnetic
stimulation applied to the right cerebellum delays eye movements to a target object
predicted by sentence content (Lesage et al., 2012). Cerebellar regions may also aid
in maintaining an ongoing representation of a sentence during comprehension or in
resolving conflicting inputs. In line with the latter possibility, gray matter density
in a region of the right cerebellum predicts resistance to speech interference as bi-
lingual speakers comprehend an utterance in their second language while listening
to speech in their first language (Filippi et al., 2011). A re-analysis of data in Crinion
et al. (2006) with this area as a region of interest confirmed that it was most active
when resisting interference from L1. Such an outcome supports the notion that func-
tional demand may underlie structural response. We also note that the same region
responds to language conflict in monolingual English speakers (e.g., when silently
reading words with irregular spellings; Osipowicz et al., 2011), indicating that it is not
specialized for language control in bilingual speakers. It provides a further example

of neural convergence.
Adaptive Changes in the Language

and Control Networks
Basic differences between languages provide a potential trigger for adaptive change.
A study by Krizman, Marian, Shook, Skoe, and Kraus (2012) provides an excellent il-
lustration. In a multi-speaker context, bilinguals must be able to select the relevant
target language and adjust their own speech. Fundamental pitch (F0) provides a cue
for identifying the language of use and is encoded in the auditory brainstem. Given its
functional relevance, Krizman et al. (2012) reasoned that bilinguals might attend to this
cue and amplify its neural representation. In line with expectation, relative to mon-
olingual English speakers, bilingual speakers showed enhanced auditory brainstem
responses to changes in fundamental pitch. Most interestingly, given our interest in the
language network and in its control, they also showed enhanced performance on a test
of sustained attention, with performance correlating strongly with brainstem response
in the multi-talker context. A language demand, then, can induce a strong coupling be-
tween an attentional process and the physiological encoding of a relevant auditory cue.
Bilinguals must be able to not only discriminate the relevant auditory cues, but also
process and represent the words of the languages in order to derive the meaning of an
utterance and its significance. Auditory cortex is clearly implicated, and so it is fair to
ask if there is evidence that the learning and lifelong use of two languages affects it in any
way. Given that musical training can lead to an increase in its volume (e.g., Schneider
et al., 2002), does the learning and use of more than one language yield an auditory ex-
pertise effect? A study examining the volume of Heschl’s gyrus (HG) showed greater
volume in young Catalan-Spanish students compared to their age-matched monolin-
gual Spanish controls consistent with the notion that bilingual experience (at least when
both languages are learned simultaneously at an early age) does indeed cause changes in
this region of the auditory cortex (Ressel et al., 2012).
Other research suggests that the specific auditory properties of a language can exert
an effect. For example, Mandarin uses tone to mark lexical differences, and so regions
associated with the processing of pitch might be expected to differ between Chinese
and non-Chinese speakers, regardless of ethnicity differences. In one voxel-based mor-
phometry study, native Chinese speakers and Europeans who had learned Chinese as a
second language showed increased gray and white matter in the right temporal pole and
left insula compared to monolingual English speakers or bilingual speakers who spoke
no Chinese (Crinion et al., 2009).
The Representation and Processing

of Vocabulary
Knowledge of words, their senses, referents, sounds, and written forms is distributed
over many regions of the brain. We consider the implications for bilingual lexical pro-
cessing, for acquiring vocabulary knowledge, and for speaking words in two languages.
Lexical Access
Initial evidence for the idea that there are interleaved representations within common
areas comes from intra-operative cortical stimulation studies (see Duffau, Chapter 8
in this volume) that identify sites where speech arrest arises for one language but not
another (Lucas, McKhann, & Ojemann, 2004; Ojemann & Whitaker, 1978; Roux &
Trémoulet, 2002). Such results do not imply localization of an object name, but rather
interruption of a naming circuit. Data better suited to address this question come from
studies that examine patterns of neural response in the functional magnetic resonance
imaging (fMRI) response to translation-equivalent words. In processing such words,
we should expect to find two kinds of data: language-distinct patterns within common
regions that reflect their distinct acoustic and phonological properties of the transla-
tion equivalents, and language-independent patterns that reflect the fact that such
equivalents access modality-independent knowledge of the referents of those words.
Using multi-voxel pattern analyses, Correia et al. (2014) reported that discriminating
neural response to individual spoken animal nouns (horse/duck) within each language
(English/Dutch) involved multiple temporal, parietal, and frontal cortical regions.
Some regions (such as the left anterior temporal lobe) revealed an invariant response
pattern to the translation equivalents (paard/eend), indicative of access to common se-
mantic/conceptual knowledge. The anterior temporal lobe also shows adaptation effects
in cross-language semantic priming in reading single words from different categories
(Crinion et al., 2006). The processing of verbs loads on a different set of regions (e.g.,
Tyler, Randall, & Stamatakis, 2008), but neuroimaging data confirm that the set of re-
gions involved is identical in the two languages of proficient adult bilingual speakers
(Consonni et al., 2013). Our expectation is that a multi-voxel pattern analysis would re-
veal a comparable interleaved pattern in the processing of verbs.
Further evidence for the tight neural coupling of translation equivalents comes from
studies using ERPs to monitor the first moments when bilinguals process words in one
language alone. Thierry and Wu (2007) asked Chinese-English bilinguals immersed in
an English-dominant context to perform a semantic relatedness judgment on a pair of
English words. Unbeknownst to these bilinguals, the two English words had translations
in Chinese that sometimes shared the same characters. They found that bilinguals, but
not monolingual speakers of English, showed a modulated N400 when there was a
conflict between the semantic match of the English words and the form mismatch of
the Chinese characters. Because no actual Chinese was present in the experiment, the
result suggests that the Chinese translations of the English words were activated im-
plicitly in the first few hundreds of milliseconds of processing the English as the L2. The
result is particularly striking because the two languages do not share the same written
script. Similar behavioral results have been reported for deaf readers in the United States
who appear to activate translations in American Sign Language when they are reading
English words (Morford, Wilkinson, Villock, Piñar, & Kroll, 2011). Three features of
these results are critical to our discussion. One is that the co-activation of the bilingual’s
two languages does not depend on the similarity of word form. Form relations may
modulate responses, but the very existence of the two languages is what appears to be
important. A second is that co-activation across the two languages happens quickly
and without conscious intention. A third is that these interactions are not a reflection of
early stages of learning, but characterize the performance of highly proficient bilingual
speakers and readers.
Acquiring Vocabulary Knowledge

In order to access modality-independent knowledge through language, speakers need
vocabulary. Current evidence favors the notion that temporal and parietal regions me-
diate key components of vocabulary knowledge. In monolingual speakers, inferior
temporal regions are actively engaged in semantic fluency tasks, that is, in the paced re-
trieval of words from a specified semantic category (e.g., Mummery, Patterson, Hodges,
& Wise, 1996). Bilingual speakers engage the same region, with gray matter density pre-
dictive of the relative performance in a semantic fluency compared to a phonemic flu-
ency task in both L1 and L2 (Grogan, Green, Ali, Crinion, & Price, 2009).
Vocabulary knowledge as such is linked in monolingual adolescents to gray matter
density in posterior regions of the supramarginal gyrus (pSMG) (Lee et al., 2007),
whereas in adult monolinguals, vocabulary knowledge correlates with gray matter
density in two left temporal regions associated with sentence processing (Richardson,
Thomas, Filippi, Harth, & Price, 2009). In bilingual speakers, vocabulary knowledge
appears to load on the pSMG. This region is not typically activated in phonological tasks
but, as indicated earlier, it connects to an anterior region that does activate and to the an-
gular gyrus activated in semantic tasks. It thus may serve as a region that integrates two
sources of information fundamental to vocabulary: how a word sounds and its meaning.
Indeed, Mechelli et al. (2004) reported higher gray matter density in pSMG in Italian-
English bilinguals as compared to English monolinguals. Furthermore, gray matter
density covaried positively with L2 proficiency and negatively with age of learning the
L2 (see also Grogan et al., 2012).
Given the results of Lee et al. (2007), the correlation with proficiency is most plausibly
attributed to an increase in L2 vocabulary knowledge (see Grogan et al., 2012, for an
association with the number of languages spoken). Further corroboration comes from
a longitudinal study in children by Della Rosa et al. (2013). A regional increase in gray
matter density correlated with increased language proficiency over a one-year period.
Abutalebi, Canini, Della Rosa, Green, and Weekes (2015a) confirmed the persistence of
this correlational effect in older bilinguals. The persistence of the association over time
may suggest a reduced reliance in bilingual speakers on sentence contexts to expand vo-
cabulary, or a continued reliance on processes that served to expand vocabulary in the
first place. Conceivably, there are individual differences within speakers such that those
better able to induce meaning from context rely on more temporal regions, whereas
those better at phonological processing rely on more parietal regions.
Individual differences in the structure of the regions associated with subvocal re-
hearsal (see Golestani & Pallier, 2007; Golestani, Molko, Dehaene, LeBihan, & Pallier,
2007) or in their functional connectivity (see Veroude, Norris, Shumskaya, Gullberg, &
Indefrey, 2010; also Xiang et al., 2012) do appear to affect the ease with which individuals
can learn novel words. In a short-term training study using lexical tone in a word-
picture task, Yang, Gates, Molenaar, and Li (2015; see, earlier, Wong, Chandrasekaran,
Garibaldi, & Wong, 2011) examined the causal structure of the network mediating suc-
cessful word learning. In line with the importance of preexisting individual differences,
successful learners displayed a better integrated fronto-temporo-parietal network (in-
cluding SMG) before learning as well as after learning (see Li, Legault, & Litcofsky, 2014,
for a more detailed review of such training studies). Individual differences in resting-
state functional connectivity between frontal and temporal regions, and between
frontal regions and dorsal ACC, also predict the efficiency of lexical retrieval following
an intensive French immersion course (Chai et al., 2016). Such data suggest that neural
signatures can be used to predict the ease with which adults can learn a new language.
Subcortical structures are also likely to play an important role in word learning and
help explain individual differences in word learning. In the case of reading, for ex-
ample, differential activation of the caudate-fusiform circuit is predictive of changes
in reading skill (see Tan et al., 2011). Reward circuits may also be critical. Subcortical
areas mediating reward, such as the ventral striatum, are activated by a range of re-
warding stimuli covering food, money, and artistic or intellectual pleasures. It is plau-
sible that language learning (see Syal & Finlay, 2011) recruits the same areas. A number
of interrelated predictions arising from this proposal were tested in a study examining
the neural substrates for learning the meanings of novel words (Ripollés et al., 2014). The
study showed that the ventral striatal region involved in correctly learning the meaning
of new words was active in a task involving monetary gains and so established a tight
coupling between correctly learning the meaning of a new word and the modulation of a
phylogenetically older system involved in reinforcement learning.
More specifically, the study established that activation in the ventral striatum
increased for new words that were learned, and that such learning also increased acti-
vation in cortical regions of the language network (left IFG, left middle temporal lobe,
left inferior parietal gyrus; the left hippocampus was also active). If reward signals drive
regions in the language network, then there should be increased connectivity between
ventral striatum and pertinent regions in the language and control network. In line with
expectation, functional connectivity analysis confirmed that learning increased the

coupling of left ventral striatal activation with activation in regions in the left IFG (pars
opercularis [BA 44], pars triangularis [BA 45], and BA 47), the supplementary motor
area, and the left caudate. A further analysis nicely showed that the percentage of suc-
cess in learning new words was related to the integrity of the white matter pathways
reaching the ventral striatum, as well as pathways in the language network (inferior
fronto-occipital fasciculus associated with semantic processing and the left uncinate
fasciculus, linking the anterior temporal pole and the orbitofrontal cortex). Overall,
these functional and anatomical data cohere to support the role of subcortical reward-
related areas, together with regions and pathways in the language network, in predicting
the success with which individuals learn the meanings of new words.
Spoken Lexical Production

Adaptive changes to the language network can also be seen in the performance of highly
proficient bilinguals. Studies that characterize skilled bilingualism are important not
only because they enable us to identify the scope of language plasticity, but also because
they demonstrate that skilled bilinguals are not monolingual native speakers in either
of their two languages, a point made many years ago by Grosjean (1989). A theme in
recent studies of bilingual lexical production is that the native language is continually
regulated, presumably to engage those control mechanisms that eventually allow for
fluent performance in the weaker language and to coordinate the use of the two lan-
guages in different contexts (e.g., Green & Abutalebi, 2013). What has become evident
is that this coordination is not a simple effect, but depends on the level of resource
demands associated with particular task goals.
A second language increases the articulatory repertoire involved in producing words.
Given monolingual data on the involvement of the anterior putamen in producing less
familiar forms (nonwords; Oberhuber et al., 2013; see earlier discussion in the chapter),
we should expect the functional activation of this region during naming in L2 to reduce
with proficiency, and this seems to be the case (Abutalebi et al., 2013b). Other frontal
regions are known to be sensitive to lexical demands in monolingual speakers. The left
IFG is implicated in the paced retrieval of words (e.g., Thompson-Schill, D’Esposito, &
Dan, 1999). Bilingual speakers, at least in their L2, need to resolve lexical competition
with L1, and the left pars opercularis in the left IFG shows such an effect (Parker-Jones
et al., 2012). If structural change follows functional demand, then we may predict that
the speed and accuracy with which bilingual speakers recognize and produce words in
their L2 (lexical efficiency) should correlate positively with gray matter density in this
region, and it does (Grogan et al., 2012; see also Stein et al., 2012). The precise nature
of the lexical demand detected here is not resolved, but it is noteworthy that struc-
tural differences in this region (along with others in auditory and parietal regions)
are predictive of individual differences in the ease of perceiving and producing for-
eign speech sounds (Golestani & Pallier, 2007; Golestani, Molko, Dehaene, LeBihan,
& Pallier, 2007). Such data are consistent with the role of the left pars opercularis in
phonemic processing and subvocal rehearsal in working memory (e.g., Gough, Nobre,
& Devlin, 2005).
A comparison of phonemic and semantic fluency in bilingual speakers reveals fur-
ther regional correlations with gray matter density. Gray matter density in the head of
caudate bilaterally predicted relative performance for the phonemic over the semantic
fluency task, particularly in L2 (Grogan et al., 2009) This outcome is consistent with the
idea that the caudate helps to control interference from prepotent responses (e.g., L1) or
to assemble phonemes into words as part of a procedural system (Ullman, 2001)—a task
that would be more effortful for the L2. Gray matter density in bilateral pre-SMA also
predicted relative phonemic over semantic fluency performance. This region is associ-
ated with the planning and preparation of movement (e.g., Petrides, Alivisatos, Meyer,
& Evans, 1993) but surprisingly, there was no differential effect of language in this re-
gion, where one might have expected the correlation to reflect the increased monitoring
demands for L2.
Recent studies also have examined the carryover effects of prior use of one lan-
guage on lexical production in the other (see Kroll & Gollan, 2014, and Kroll, Gullifer,
McClain, Rossi, & Martín, 2015, for recent reviews). In brief, the evidence suggests that
when bilinguals are required to speak their native or dominant L1 following speaking
the weaker or less dominant L2, there is inhibition of the L1. The need to select one
language alone requires the engagement of resources to reduce the activation of the
more available alternative. The consequence of speaking the L2 for the L1 can be seen
in ERP data (e.g., Branzi, Martin, Abutalebi, & Costa, 2014; Misra, Guo, Bobb, & Kroll,
2012), in behavior (Van Assche, Duyck, & Gollan, 2013), and in fMRI data (e.g., Branzi
et al., 2015; Guo et al., 2011). The modulation of the L1 in the face of demands to speak
the L2 suggests that bilinguals acquire skill in language regulation. What is notable
in the studies mentioned is that the requirement to regulate the more dominant lan-
guage is not simply a momentary event that is observed when bilinguals switch from
one language to the other in forced trial-to-trial switching paradigms (e.g., Meuter &
Allport, 1999). Rather, it occurs even when bilinguals have an opportunity to speak in
the L1 for an extended period of time, suggesting that there are sustained inhibitory
control mechanisms that may have the consequence of suppressing the non-target
language globally. A focus in recent studies is to identify the mechanisms that may
contribute to inhibitory control in different circumstances, with some hypothesized
to operate more locally, for specific lexical alternatives and perhaps with short-term
consequences, and others that may operate more globally, with respect to both scope
and time course.
It is of interest to note that a series of studies has asked about the consequences for
cognitive control when these selection demands are absent in production. For hearing
individuals who use a signed language, there is the possibility of co-gesturing while
speaking. Therefore, a choice along the same output channel is not required. Both
behavioral (Emmorey, Luk, Pyers, & Bialystok, 2008) and structural imaging data
(Olulade et al., 2015) on bimodal bilinguals support the claim that the ease of selecting
the language to be produced is one source of the observed consequences of bilingualism.

In the absence of a high selection demand, neither behavioral advantages nor struc-
tural changes in the brain were observed for bimodal bilinguals relative to their uni-
modal counterparts. The precise contexts of bimodal language use may be important;
Zou, Ding, Abutalebi, Shu, and Peng (2012) report increased gray matter volume of head
of caudate, relative to monolingual speakers, for late learners of Chinese sign language
who were teachers of it. In this sample, conceivably because of the teaching context, se-
lection demand may be high. Indeed, activation of the left head of caudate was associ-
ated with language switching consistent with the role of the left caudate in the selection
of competing action plans.
Representation and Processing

of Syntax in the Bilingual Brain
Where scientific opinion, at least until recently, has been most strongly divided is
whether or not the learning of a second language necessarily leads to a difference in
the representation of syntax. Post-stroke, languages tend to recover in line with their
relative premorbid proficiency (e.g., Paradis, 2004), but some neuropsychological case
reports do associate distinct lesion sites with the selective recovery of L1 or L2 (e.g.,
García-Caballero et al., 2007; see also Aladdin, Snyder, & Ahmed, 2008). However, such
selective recovery may also be the result of damage to circuits involved in language se-
lection, and so we do not consider such evidence decisive.
Why might L2 syntax be represented in a different fashion, at least at low levels
of L2 proficiency (Paradis, 2004, 2008; Ullman, 2001)? The argument here is that
syntax is acquired implicitly for L1 and is represented in subcortical regions of the
basal ganglia that contribute to procedural or skill-based memory. By contrast, L2
grammatical rules are learned explicitly and are represented declaratively in cor-
tical regions. Stroke may then yield different effects in L2 because subcortical
damage, for instance, will leave the declarative representations of L2 to drive pro-
cessing, whereas these will not be available to drive processing in L1. Much larger
samples of patients with subcortical lesions are needed to test this idea, but current
lesion-deficit data favor common representations for L1 and L2. In an analysis of the
lesion-deficit associations involving a wide battery of language skills, including those
requiring syntactic processing, Hope et al. (2015) found that monolingual and bilin-
gual patients were sensitive to damage in the same sets of regions, though bilingual
patients showed greater sensitivity to damage, perhaps reflecting their lower pre-
morbid proficiency. fMRI data also indicate the involvement of cortical, subcortical,
and cerebellar regions in common with those activated in monolingual speakers,
when bilingual speakers read sentences aloud either in their native or in their non-
native language (Berken et al., 2015).
If neural convergence holds, ERP, magnetoencephalography (MEG), and functional

imaging data should provide evidence that L2 learners engage in native-like syntactic
processing at the very earliest stages of L2 learning.
Native-L ike Processing of L2 Syntax at

the Earliest Stages of L2 Learning
Longitudinal studies of beginning learners of a language provide critical evidence for

the way in which an L2 grammar is learned. Neurocomputationally, convergence with
native-speaker profiles of processing L2 syntax should operate at relatively early stages
of L2 learning. That assumption is contrary to the traditional claim that the syntax is
inaccessible to late L2 learners, leading them to adopt alternative means to achieve
comprehension (e.g., Clahsen & Felser, 2006), by using semantic or pragmatic infor-
mation to circumvent the syntax itself. In fact, the neurocognitive evidence provides
strong support for the proposal that most features of the syntax and morphosyntax
are available to late L2 learners, although some may require greater adjustment than
others as learning proceeds (see Roncaglia-Denissen & Kotz, 2016, for an overview).
The approach that has been most revealing in identifying the nature of processing the
syntax in the L2 has been the use of ERPs. ERPs provide a temporally sensitive window
into the first moments of sentence processing as they occur online. If bilinguals who
have acquired the L2 past early childhood are required to adopt different strategies
to process the grammar than those used by native speakers, then the ERP record
would be expected to reveal those differences. As we will see, recent ERP data pro-
vide evidence for much greater adult plasticity with respect to syntax than previously
assumed by the traditional view (see Duñabeitia, Dimitropoulou, Dowens, Molinaro,
& Martin, 2016, and Van Hell & Tokowicz, 2010, for reviews on ERPs and language
processing).
Two early studies using ERPs to investigate the role of age of acquisition (AoA) for
sensitivity to L2 syntax initially appeared to support the traditional claims about critical
periods in language learning (Hahne & Friederici, 2001; Weber-Fox & Neville, 1996).
These studies adopted the logic of the classic behavioral study reported by Johnson and
Newport (1989) in which late learners were more likely than native speakers to make
errors in judging the grammaticality of sentences. The initial ERP data seemed to bolster
this claim by demonstrating that in the presence of syntactic violations, late L2 learners
did not appear to produce the signature ERP effects that have been observed for na-
tive speakers, an LAN effect (left anterior negativity) and a P600. Since these reports,
there has been a stream of criticism centering around methodological issues and, crit-
ically, the observation that AoA is confounded with L2 proficiency (e.g., Steinhauer,
2014). Although all would agree that there are consequences of age of exposure that may
be reflected in brain structure and function (e.g., Klein, Mok, Chen, & Watkins, 2014),
those consequences may not present hard constraints in acquiring sensitivity to the L2
grammar.
Subsequent studies have taken two different approaches to examine features of the
syntax. One is to ask whether highly proficient but late L2 learners process the syntax
and morphosyntax like native speakers. The other is to examine neural response to
the learning of artificial grammar or to use longitudinal methods to track the develop-
mental trajectory of adult L2 learners during the earliest stages of learning. fMRI data
of neural response to the processing of syntactic structures from an artificial grammar
indicate commonality with the neural substrate mediating syntactic processing in a nat-
ural language. For example, the arcuate fasciculus, with its direct connection between
BA 44 and the temporal cortex, underlay the processing of novel syntactic structures in
a comprehension task (Friederici, Bahlman, Heim, Schubotz, & Anwander, 2006) iden-
tical to the pathway required in the processing of complex structures in a natural lan-
guage (Wilson et al., 2011). Learning novel syntactic structures also elicited an identical
profile in MEG responses to syntactic structures in English (Ding, Mellon, Zhang, Tian,
& Poeppel, 2016).
More germane to the bilingual case, Morgan-Short, Steinhauer, Sanz, and Ullman
(2012) used an artificial language under conditions of implicit or explicit training to
ask whether learners can acquire native-like responses to syntactic violations. They
compared the two types of training for learners who had become proficient in the arti-
ficial language or not. The two types of training were taken to be a model of immersion
(implicit) versus classroom (explicit) learning conditions. Although the two learning
groups at the two levels of proficiency did not differ in behavioral measures, the ERP
record revealed clear consequences of both factors. Individuals who had learned im-
plicitly and had achieved a relatively high level of proficiency in the new language
produced an ERP pattern that has been associated with native-speaker responses to syn-
tactic violations; there was an anterior negativity, followed by a P600 and a late anterior
negativity. Individuals who had become relatively proficient following explicit training
did not reveal the late anterior negativity, suggesting that implicit learning may enable
native-like performance, even following relatively brief exposure to the new language.
In a follow-up study, Morgan-Short, Finger, Grey, and Ullman (2012) compared rela-
tively high-proficiency learners in this paradigm after they had a period of 3–6 months
with no exposure to the language. Contrary to the expectation that attrition would be
observed, upon retesting they found that both implicit and explicit learners produced
a pattern of ERPs that was more native-like than the initial test, although the implicit
learners maintained a native-like profile relative to the explicit learners. Beyond the
nuances of learning method, the significance of these findings is that they demonstrate,
first, that it is possible for adult learners to acquire sensitivity to the syntax following rel-
atively brief exposure, and second, that ERPs may reveal learning prior to behavior (see
Tokowicz & MacWhinney, 2005, for a similar claim).
An ideal approach to investigate the trajectory of new learning is to track learners
longitudinally as they become more proficient in the L2. The pragmatic constraints re-
quired by longitudinal designs make these studies difficult to carry out, but there are
a number that have used ERPs to assess the presence and changes in L2 learning over
time. McLaughlin, Osterhout, and Kim (2004) first examined word learning in a group
of college students learning French as a foreign language. Quite remarkably, they
demonstrated that after only 14 hours of classroom exposure, the ERPs to discrimi-
nate words and nonwords began to reveal recognition of the new French vocabulary, as
indexed by an N400. In contrast, behavioral measures of recognition remained at chance
at this early stage of learning, suggesting that the brain is beginning to register new
learning before it is manifest in behavior. Using the same approach, with just a month
of instruction, Osterhout et al. (2008) showed that at least some learners were able to
discriminate between sentences that were well formed and those that contained a syn-
tactic violation. However, unlike native speakers, the learners initially produced an N400
to those violations rather than a P600. After four months of L2 instruction, the N400
was replaced by a P600, suggesting a developmental trajectory with changes occurring
very early in learning, although only for some learners and for some structures (and see
McLaughlin et al., 2010). Related studies have begun to examine individual differences
in sensitivity to syntactic violations and in the trajectory of new language learning (e.g.,
Tanner, McLaughlin, Herschensohn, & Osterhout, 2013). Although there is a great deal
that is yet unknown about the factors that contribute to the developmental course of
new L2 learning or the individual differences that modulate the trajectory of learning,
the general implications are clear. Obstacles to complete acquisition of the L2 syntax re-
flect aspects of the learner and the learning context, rather than a set of hard constraints.
Critically, the neurolinguistic evidence reveals a system that is more open to new learning
and to even subtle aspects of the syntax than previously understood and also is more
plastic during even the very first moments of exposure to the second language.
Native-L ike Processing of L2 Syntax

for Highly Proficient Late Bilinguals
The approach taken by Morgan-Short and colleagues, and others, in using an arti-
ficial language learning paradigm can be criticized on the grounds that artificial lan-
guage learning necessarily involves a smaller number of tokens and structures that
may overestimate the ability of adult learners. Thus, an important question is whether
similar patterns can be seen in studies with actual languages with relatively proficient
L2 speakers who acquired the L2 as adults. There are many studies that have taken this
approach, and although the evidence is complex, with varied findings across different
types of structures and speakers, there is general support for the notion that late L2
learners who are able to become highly skilled reveal similar patterns of sentence pro-
cessing relative to native speakers of the language (e.g., Bowden, Steinhauer, Sanz, &
Ullman, 2013, and see Steinhauer, 2014, for a review of the recent studies; and earlier sec-
tion of this chapter for fMRI data).
A number of factors may determine the ability of late bilinguals to achieve native-
like status, including structural differences across the two languages and individual
differences among L2 learners. The exercise becomes more complex when one considers
that even for the most proficient bilinguals, the ease of activating particular syntactic
structures may be influenced by cross-language processing. While bilinguals may acti-
vate the syntax of both languages in parallel (Sanoudaki & Thierry, 2015), there also may
be constraints imposed when structures within each language are incongruent. For ex-
ample, Hartsuiker, Pickering, and Veltkamp (2004) used a syntactic priming paradigm
behaviorally to demonstrate that it was possible to observe priming across languages
when structures are shared. Other studies have shown that when the languages diverge,
for example, in their reliance on word order, then priming is typically not found (e.g.,
Bernolet, Hartsuiker, & Pickering, 2007; Loebell & Bock, 2004). The ability to acquire
sensitivity to the L2 syntax does not mean that all structures are necessarily processed in
parallel (but see Vaughan-Evans, Kuipers, Thierry, & Jones, 2014, for some ERP evidence
to the contrary). Likewise, not all learners may reveal the same trajectory of learning or
adopt the same strategies to process all structures. A number of recent ERP studies have
shown striking individual differences in patterns of ERP sentence processing for both
monolinguals and bilinguals (e.g., Pakulak & Neville, 2010; Tanner, Inoue, & Osterhout,
2014), suggesting that caution needs to be exercised when interpreting differences
across groups.
In a recent review of the research using ERPs to assess L2 sensitivity to syntax,
Caffarra, Molinaro, Davidson, and Carreiras (2015) analyzed the aggregated data from a
large number of studies to ask which factors determine the presence or absence of sen-
sitivity to syntactic violations. Their analysis suggested that both L2 proficiency and lan-
guage immersion experience have consequences for late acquirers, but that each of these
variables affected a different aspect of L2 processing, with early effects for immersion
and later control or monitoring effects associated with proficiency. An interesting result
in the Caffarra et al. paper was that there was no evidence within the limits of the anal-
ysis performed to suggest that the similarity between the L1 and L2 was critical.
Another logic that has been adopted to investigate these issues is to select features
of the syntax or morphosyntax that are subtle and typically difficult to acquire. A long
line of studies has investigated the acquisition of grammatical gender from this perspec-
tive to ask whether the acquisition of a gender-marked language is more difficult when
the learner’s native language also marks gender but differently than the L2, or does not
mark gender at all, as is the case for English learners. Recent ERP studies have produced
results that are somewhat mixed but generally suggest that late bilinguals are indeed
able to acquire sensitivity to gender (e.g., Foucart & Frenck-Mestre, 2012; Meulman,
Stowe, Sprenger, Bresser, & Schmid, 2014) and that knowledge and use of gender in one
language may influence the other language even when it does not mark gender (e.g.,
Ganushchak, Verdonschot, & Schiller (2011). Rossi, Kroll, and Dussias (2014) took the
approach of focusing on a subtle feature of Spanish to examine the ability of late English-
Spanish bilinguals to process clitic pronouns. Clitics do not exist in English and are also
marked for gender and number in Spanish. Rossi et al. found that relatively proficient
late English-Spanish bilinguals were sensitive to violations of number marked on the

clitic, resulting in a P600 in the ERP record, but were not sensitive to violations of
gender. When proficiency was taken into consideration, the most proficient bilinguals
among these late learners were indeed sensitive to violations of gender as well as number,
suggesting that it is possible to acquire native-like ability to process structures that are
unique to the L2.
The literature on the processing of syntax in two languages is only beginning to ex-
amine the neural basis of those aspects of language use that may be uniquely bilingual,
such as intra-sentence code-switching. Some bilinguals code-switch with others sim-
ilarly bilingual in the middle of a sentence, and the linguistic data demonstrate that
these switches are far from arbitrary in that they follow constraints that appear to be
imposed by grammar and by the relationship between production and comprehension
(e.g., Guzzardo Tamargo, Valdés Kroff, & Dussias, 2016). Understanding the neural con-
trol of code-switching will be an important direction for future research on the syntax
(see Green & Wei, 2016, for a discussion of code-switching and language control).
Other studies show that the L1 grammar comes to be influenced by the L2, again re-
vealing a level of plasticity that might not have traditionally been expected (e.g., Dussias
& Sagarra, 2007). The neural basis of these bilingual phenomena are not yet well under-
stood but will be important in shaping the next stages of research. They are compatible
with imaging data that suggest that the same neural tissue supports both languages. The
effects of the L2 on the L1 are of additional interest in light of discussions of acquiring
native-speaker facility in an L2. If proficient bilinguals change the way they process the
native language as a consequence of the active use of a second language, then the native-
language model may be that of a bilingual rather than monolingual speaker.
Neural Plasticity and Neural Reserve
In the introduction to this chapter, we referenced studies on cognitive decline and re-
ported diagnosis of Alzheimer’s disease that point to the potential neuroprotective
role of lifelong bilingualism. Neuroprotective effects plausibly derive from the adap-
tive changes required to represent and use two languages. To the extent that everyday
language use involves distinguishable demands on the control and language networks,
then the cumulative effects will be realized in structural changes in the regions most
affected and in the networks that interconnect them. In addition to the changes in
specific regions noted in earlier sections, there are reports of widespread neural dif-
ference in the brains of young adults who are bilingual rather than monolingual. For
instance, Burgaleta et al. (2016) reported bilateral gray matter volumetric differences
in inferior frontal, inferior temporal, parietal, and cerebellar regions, in addition to the
volume differences in the putamen already discussed. By contrast, they noted increased
volumes for monolinguals in the left middle and superior temporal gyrus, arguably con-
sistent with the adult monolingual vocabulary data of Richardson et al. (2009). At the
network level, García-Pentón, Perez, Iturria-Medina, Gillon-Dowens, and Carreiras

(2014) identified two networks with increased connectivity in bilinguals compared to
monolinguals: one comprising left hemisphere language-related and language-control
regions and the other arguably more linked to semantic processing but including a
frontal control region (see also Berken, Chai, Chen, Gracco, & Klein, 2016). Such data
argue for profound neural differences arising from the experience of learning and using
languages. However, determining potential neuroprotective effects of bilingualism in
the elderly requires structural studies of older bilingual and monolingual individuals.
Such studies exist but are currently few in number (for reviews, see Gold, 2015; Perani &
Abutalebi, 2015). We consider two such studies in the following.
In pioneering research, Luk, Bialystok, Craik, and Grady (2011) compared white
matter integrity in a sample of lifelong bilingual adults (with the L2 learned before the
age of 11) and monolingual adults matched for age (mean age = 70 years) and perfor-
mance on neuropsychological tests. Diffusion tensor imaging (DTI) (see Catani &
Forkel, Chapter 9 in this volume) revealed greater white matter integrity in the corpus
callosum and a number of major long-range tracts of the language network (the bilateral
superior longitudinal fasciculi, right inferior fronto-occipital fasciculus, and uncinate
fasciculus) (see also data from bilingual children, Mohades et al., 2012). Enhanced white
matter integrity may be the outcome of the repetitive demands on the mechanisms
of language control. Indeed, resting-state data in the same study showed significantly
stronger distributed functional connectivity in bilinguals. Whereas monolinguals
showed greater connectivity within frontal regions, bilinguals showed enhanced con-
nectivity between regions associated with language control (left inferior frontal and left
caudate) and posterior regions of tracts in the language network (see also Grady, Luk,
Craik, & Bialystok, 2015). The results of Luk et al. (2011) usefully corroborate a struc-
ture (white matter) and functional correspondence (the resting-state data). They are
tantalizing clinically because progressive disruption of white matter is typical of the
aging brain (Gunning-Dixon, Brickman, Cheng & Alexopoulos, 2009; Pfefferbaum,
Adalsteinson, & Sullivan, 2005) and disruption of prefrontal networks predicts im-
paired cognitive control (Mayda, Westphal, Carter, & DeCarli, 2011). Maintained in-
tegrity of white-matter tracts, as Luk et al. suggested, may compensate for gray matter
atrophy. Gold, Johnson, and Powell (2013) found a different pattern in a sample of 60-
year-olds and suggested that compensation in fact arose from increased cognitive con-
trol skills associated with bilingualism. Work showing increased gray matter in the ACC
for older bilinguals compared to aged-matched controls may offer support for this posi-
tion (Abutalebi et al., 2015b), but clearly there is a need for further work that includes a
range of cognitive control and linguistic tasks.
Our second study investigated gray matter density differences between elderly bilin-
gual and monolingual speakers. Bilinguals, compared to aged-matched monolinguals
(with equivalent scores on education, socioeconomic status, and cognitive perfor-
mance) displayed increased gray matter volume bilaterally in the temporal poles and the
orbitofrontal cortex (Abutalebi et al., 2014). Interestingly, L2 proficiency also correlated
with increased gray matter volume in the left anterior temporal pole, indicating that
increased control demands may continue to shape the structure of the regions they
target. Such an outcome is provocative because the temporal poles, along with the or-
bitofrontal cortex, are among the earliest cortical regions to suffer age-related brain at-
rophy (Kalpouzos et al., 2009).
Only further research can corroborate the indicative white matter and gray matter
differences in the brains of elderly bilingual and monolinguals speakers that we have
described. Where groups are matched on relevant explanatory variables (socioec-
onomic status, education, exercise), bilingual experience is plausibly causative (see
García-Pentón, García, Costello, Duñabeita, & Carreiras, 2016, and commentaries
for critical discussion). However, in line with the adaptive control hypothesis (Green
& Abutalebi, 2013), the precise nature of the bilingual experience, accumulated and
maintained over time, should be decisive in terms of the impact of bilingualism on the
language-control network and its interaction with the language network. If so, it follows,
as we briefly consider in the next section, that exploring such individual differences may
prove critical in refining our understanding of the potential protective effects of bilin-
gualism. Whether bilingual experience is directly, or indirectly, related to the integrity
of the language-control network, and its interaction with the language network, requires
identification and use of other markers of aging. It is conceivable, for example, that the
mental challenge of language use in bilinguals is sufficient to maintain the integrity of a
system in the brainstem: the locus coeruleus-norepinephrine system that helps prevent
age-related neuronal damage (Mather & Harley, 2016). Bilingual experience would then
be an indirect neuroprotective factor.
Conclusions and Future Prospects
We have endeavored to illustrate how the brain adapts to the experience of learning and
using a further language. Fundamental to that experience is the way a speaker engages
with the language practices of the communities in which they participate. From a theo-
retical point of view, the recurrent patterns of conversational exchange impose different
demands on the processes of language control and induce different habits of language
control (Green, 2011; Green & Abutalebi, 2013). For example, in some communities,
speakers switch between their two languages to address different speakers. Despite po-
tential activation, words and structures from the current, nontarget language must be
blocked from the speech plan. In such circumstances, selection of one language involves
non-selection of the other: language control is competitive. Other communities foster
dense code-switching between languages and thus free the speaker to access resources
from either language in order to achieve the communicative goal. Language control is
cooperative: there is no target language, and output reflects momentary fitness in the
speech plan (Green & Wei, 2014). These different contexts (the behavioral ecologies of
language use) and the range of contexts that speakers inhabit mean that we need to de-
scribe in detail the nature of the contexts that shape adaptive response.
Current research indicates an unanticipated neuroplasticity of the bilingual brain.

However, malleability is not unconstrained: there can be traces of early language expe-
rience. Intriguing data on international adoptees with early exposure to Chinese tone
but with no later exposure showed that their neural response in a lexical tone discrim-
ination task matched that of Chinese-French bilinguals exposed to Chinese from birth
but differed from that of French speakers (Pierce, Klein, Chen, Delcenserie, & Genesee,
2014). Of equal importance is that such early experience influences the processing of
the adopted language. In a French phonological working-memory task, international
adoptees who were no longer exposed to Mandarin and who spoke only French differed
from monolingual French speakers in the neural regions they recruited. Together with
bilingual Mandarin-French speakers, they recruited regions typically associated with
the control of language in order to perform the task to the same level (Pierce, Chen,
Delcenserie, Genesee, & Klein, 2015). Early language experience may then shape the re-
cruitment of regions involved in language control when processing the current native
language. Such data point to the methodological need to use tasks that challenge the
neural representation of language and its control in order to determine the nature of in-
dividual variety. Understanding such variety is essential to the construction of a theory
of adaptive change in the bilingual brain. How exactly do control regions work together
to perform a given task? This requires researchers to test causal models (i.e., to examine
effective connectivity). Such models can also allow researchers to explore how engage-
ment of control regions changes with language proficiency or as function of acquisition
history.
Acknowledgments
The writing of this chapter was supported in part by NSF Grants BCS-1535124, OISE-0968369,
and OISE-1545900 and NIH Grant HD082796 to J. F. Kroll.
References
Abutalebi, J., Brambati, S. M., Annoni, J. M., Moro, A., Cappa, S. F., & Perani, D. (2007).
The neural cost of the auditory perception of language switches: An event-related func-
tional magnetic resonance imaging study in bilinguals. Journal of Neuroscience, 27,
13762–13769.
Abutalebi, J., Canini, M., Della Rosa, P. A., Sheung, L. P., Green, D. W., & Weekes, B. S. (2014).
Bilingualism protects anterior temporal lobe integrity in aging. Neurobiology of Aging, 35,
2126–2133.
Abutalebi, J., Canini, M., Della Rosa, P. A., Green, D. W., & Weekes, B. S. (2015a). The
neuroprotective effects of bilingualism upon the inferior parietal lobule: A structural neuro-
imaging study in aging Chinese bilinguals. Journal of Neurolinguistics, 33, 3–13.
Abutalebi, J., Della Rosa, P. A., Ding, G., Weekes, B., Costa, A., & Green, D. W. (2013a).
Language proficiency modulates the engagement of cognitive control areas in multilinguals.
Cortex, 49, 905–911.
Abutalebi, J., Della Rosa, P. A., Castro Gonzaga, A. K., Keim, R., Costa, A., & Perani, D. (2013b).
The role of the left putamen in multilingual language production. Brain and Language, 125,
307–315.
Abutalebi, J., Della Rosa, P. A., Green, D. W., Hernandez, M., Scifo, P., Keim, R., . . . Costa, A.
(2012). Bilingualism tunes the anterior cingulate cortex for conflict monitoring. Cerebral
Cortex, 22, 2076–2086.
Abutalebi, J., & Green, D. W. (2007). Bilingual language production: The neurocognition of
language representation and control. Journal of Neurolinguistics, 20, 242–275.
Abutalebi, J., & Green, D. W. (2016). Neuroimaging of language control in bilinguals: Neural
adaptation and reserve. Bilingualism: Language and Cognition, 19, 689–698.
Abutalebi, J., Guidi, L., Borsa, V., Canini, M., Della Rosa, P. A., Parris, B. A., & Weekes, B. A.
(2015b). Bilingualism provides a neural reserve for aging populations. Neuropsychologia, 69,
201–210.
Aladdin, Y., Snyder, T. J., & Ahmed, S. N. (2008). Pearls & Oy-sters: Selective postictal aphasia
cerebral language organization in bilingual patients. Neurology, 71, e14–e17.
Ali, N., Green, D. W., Kherif, F., Devlin, J. T., & Price, C. J. (2010). The role of the left head of
caudate in suppressing irrelevant words. Journal of Cognitive Neuroscience, 22, 2369–2386.
Alladi, S., Bak, T. H., Duggirala, V., Surampudi, B., Shailaja, M., Shukla, A. K., . . . Kaul, S. (2013).
Bilingualism delays age at onset of dementia, independent of education and immigration
status. Neurology, 81, 1938–1944.
Alladi, S., Bak, T. H., Mekala, S., Rajan, A., Chaudhuri, J. R., Mioshi, E., . . . Kaul, S. (2016).
Impact of bilingualism on cognitive outcome after stroke. Stroke, 47, 258–261.
Argyropoulos, G. P., Pascale Tremblay, P., & Small, S. L. (2013). The neostriatum and response
selection in overt sentence production: An fMRI study. NeuroImage, 82,  53–60.
Aron, A. R., Behrens, T. E., Smith, S., Frank, M. J., & Poldrack, R. A. (2007). Triangulating
a cognitive control network using diffusion-weighted magnetic resonance imaging (MRI)
and functional MRI. Journal of Neuroscience, 27, 3743–3752.
Aron, A. R., Robbins, T. W., & Poldrack, R. A. (2014). Inhibition and the right inferior frontal
cortex: One decade on. Trends in Cognitive Sciences, 18, 177–185.
Bak, T. H. (2016). The impact of bilingualism on cognitive aging and dementia: Finding a
path through a forest of confounding variables. Linguistic Approaches to Bilingualism, 6,
205–226.
Bak, T. H., Nissan, J. J., Allerhand, M. M., & Deary, I. J. (2014) Does bilingualism influence cog-
nitive aging? Annals of Neurology, 75, 959–963.
Berken, J. A., Chai, X., Chen, J.-K., Gracco, V. L., & Klein, D. (2016). Effects of early and
late bilingualism on resting-state functional connectivity. Journal of Neuroscience, 36,
1165–1172.
Berken, J. A., Gracco, V. L., Chen, J.-K., Watkins, K. E., Baum, S., Callahan, M., & Klein, D.
(2015) Neural activation in speech production and reading aloud in native and non-native
languages. NeuroImage, 112, 208–217.
Bernolet, S., Hartsuiker, R. J., & Pickering, M. J. (2007). Shared syntactic representations
in bilinguals: Evidence for the role of word-order repetition. Journal of Experimental
Psychology: Language, Memory & Cognition, 33, 931–949.
Bialystok, E., Craik, F. I., & Freedman, M. (2007). Bilingualism as a protection against the onset
of symptoms of dementia. Neuropsychologia, 45, 459–464.
Botvinick, M. M., Braver, T. S., Barch, D. M., Carter, C. S., & Cohen, J. D. (2001). Conflict
monitoring and cognitive control. Psychological Review, 108, 624–652.
Bowden, H. W., Steinhauer, K., Sanz, C., & Ullman, M. T. (2013). Native-like brain processing
of syntax can be attained by university foreign language learners. Neuropsychologia, 51,
2492–2511.
Branzi, F. M., Della Rosa, P. A., Canini, M., Costa, A., & Abutalebi, J. (2015). Language control
in bilinguals: Monitoring and response selection. Cerebral Cortex, 26, 2367–2380.
Branzi, F. M., Martin, C. D., Abutalebi, J., & Costa, A. (2014). The after-effects of bilingual lan-
guage production. Neuropsychologia, 52, 102–116.
Burgaleta, M., Sanjuán, A., Ventura-Campos, N., Sebastián-Gallés, N., & Ávila, C. (2016).
Bilingualism at the core of the brain: Structural differences between bilinguals and
monolinguals revealed by subcortical shape analysis. NeuroImage, 125, 437–445.
Caffara, S., Monlinaro, N., Davidson, D., & Carreiras, M. (2015). Second language syntactic
processing revealed through event-related potentials: An empirical review. Neuroscience &
Biobehavioral Reviews, 51,  31–47.
Catani, M., Dell’Acqua, F., Vergani, F., Malik, F., Hodge, H., Roy, P., . . . Thiebaut de Schotten, M.
(2012). Short frontal lobe connections of the human brain. Cortex, 48, 273–291.
Chai, X. J., Berken, J. A., Barbeau, E. B., Soles, J., Callahan, M., Chen, J.-K., & Klein, D. (2016).
Intrinsic functional connectivity in the adult brain and success in second-language learning.
Journal of Neuroscience, 36, 755–761.
Clahsen, H., & Felser, C. (2006). Grammatical processing in language learners. Applied
Psycholinguistics, 27,  3–42.
Consonni, M., Cafiero, R., Tettamanti, M. D., Iadanza, A., Fabbro, F., & Perani, D. (2013).
Neural convergence for language comprehension and grammatical class production in
highly proficient bilinguals is independent of age of acquisition. Cortex, 49, 1252–1258.
Correia, J., Formisano, E., Valente, G., Hausfeld, L., Bernadette, J., & Bonte, M. (2014).
Brain-based translation: fMRI decoding of spoken words in bilinguals reveals language-
independent semantic representations in anterior temporal lobe. Journal of Neuroscience,
34, 332–338.
Costa, A., Miozzo, M., & Caramazza, A. (1999). Lexical selection in bilinguals: Do words in the
bilingual’s two lexicons compete for selection? Journal of Memory and Language, 41, 365–397.
Crinion, J. T., Green, D. W., Chung, R., Ali, N., Grogan, A., Price, G. R., . . . Price C. J. (2009).
Neuroanatomical markers of speaking Chinese. Human Brain Mapping, 30, 4108–4115.
Crinion, J., Turner, R., Grogan, A., Hanakawa, T., Noppeney, U., Devlin, J. T., . . . Price, C. J.
(2006). Language control in the bilingual brain. Science, 312, 1537–1540.
De Bleser, R., Dupont, P., Postler, J., Bormans, G., Speelman, D., Mortelmans, L., & Debrock, M.
(2003). The organisation of the bilingual lexicon: A PET study. Journal of Neurolinguistics,
16, 439–456.
Della Rosa, P. A., Videsott, G., Borsa, V. M., Canini, M., Weekes, B. S., Franceschini, R.,
& Abutalebi, J. (2013). A neural interactive location for multilingual talent. Cortex, 49,
605–608.
Dick, A. S., Bernal, B., & Tremblay, P. (2014). The language connectome: New pathways, new
concepts. The Neuroscientist, 20, 453–467.
Ding, N., Mellon, L., Zhang, H., Tian, X., & Poeppel, D. (2016). Cortical tracking of hierarchical
linguistic structures in connected speech. Nature Neuroscience, 19, 158–164. doi: 10.1038/
nn.4186
Dodel, S., Golestani, N., Pallier, C., Elkouby, V., Le Bihan, D., & Poline, J. (2005). Condition-
dependent functional connectivity: Syntax networks in bilingualism. Philosophical
Transactions of the Royal Society B, 360, 921–935.
Duñabeitia, J. A., Dimitropoulou, M., Dowens, M. G., Molinaro, N., & Martin, C. (2016).
The electrophysiology of the bilingual brain. In R. R. Heredia, J. Altaribba, & A. B.
Cieślicka (Eds.), Methods in bilingual reading comprehension research (pp. 265– 312).
New York: Springer.
Dussias, P. E., & Sagarra, N. (2007). The effect of exposure on syntactic parsing in Spanish–
English bilinguals. Bilingualism: Language and Cognition, 10, 101–116.
Emmorey, K., Giezen, M. R., & Gollan, T. H. (2016). Psycholinguistic, cognitive, and neural
implications of bimodal bilingualism. Bilingualism: Language and Cognition, 19, 223–242.
Emmorey, K., Luk, G., Pyers, J. E., & Bialystok, E. (2008). The source of enhanced cognitive
control in bilinguals evidence from bimodal bilinguals. Psychological Science, 19, 1201–1206.
Fabbro, F., Skrap, M., & Aglioti, S. (2000). Pathological switching between languages following
frontal lesions in a bilingual patient. Journal of Neurology, Neurosurgery and Psychiatry, 68,
650–652.
Fedorenko, E., Duncan, J., & Kanwisher, N. (2012). Language-selective and domain general re-
gions lie side by side within Broca’s area. Current Biology, 22, 2059–2062.
Feng, G., Chen, Q., Zhu, Z., & Wang, S. (2016). Separate brain circuits support integra-
tive and semantic priming in the human language system. Cerebral Cortex, 26, 3169–3182.
doi: 10.1093/cercor/bhv148
Filippi, R., Richardson, F. M., Dick, F., Leech, R., Green, D. W., Thomas, M. S. C., & Price, C. J.
(2011). The right posterior paravermis and the control of language interference. Journal of
Ford, A., Triplett, W., Sudhyadhom, A., Gullett, J., McGregor, K., Fitzgerald, D. B., . . . Crosson,
B. (2013). Broca’s area and its striatal and thalamic connections: A diffusion- MRI
tractography study. Frontiers in Neuroanatomy, 7, 8:1–12.
Foucart, A., & Frenck-Mestre, C. (2012). Can late L2 learners acquire new grammatical
features? Evidence from ERPs and eye-tracking. Journal of Memory and Language, 66,
226–248.
Friederici, A. (2015). White-matter pathways for speech and language processing. In G. G.
Celesia & G. Hickok (Eds.), Handbook of clinical neurology, Vol. 129 (3rd series): The human
auditory system (pp. 177–186). Philadelphia: Elsevier.
Friederici, A. D., Bahlmann, J., Heim, S., Schubotz, R. I., & Anwander, A. (2006). The brain
differentiates human and non-human grammars: Functional localization and structural
connectivity. Proceedings of the National Academy of Sciences USA, 103, 2458–2463.
Ganushchak, L. Y., Verdonschot, R. G., & Schiller, N. O. (2011). When leaf becomes
neuter: Event-related potential evidence for grammatical gender transfer in bilingualism.
García- Caballero, A., García- Lado, I., González- Hermida, J., Area, R., Recimil, M. J.,
Rabadán, O. J., . . . Jorge, F. J. (2007). Paradoxical recovery in a bilingual patient with aphasia
after right capsuloputaminal infarction. Journal of Neurology, Neurosurgery and Psychiatry,
78,  89–91.
García-Pentón, L., García, Y., Costello, B., Duñabeita, J. A., & Carreiras, M. (2016). The neuro-
anatomy of bilingualism: How to turn a hazy view into the full picture. Language, Cognition,
and Neuroscience, 31, 303–327.
García-Pentón, L., Perez, F. A., Iturria-Medina, Y., Gillon-Dowens, M., & Carreiras, M. (2014).
Anatomical connectivity changes in the bilingual brain. NeuroImage, 84, 495–504.
Neurology, Neurosurgery & Psychiatry, 76, 940–946.
Gold, B. T. (2015). Lifelong bilingualism and neural reserve against Alzheimer’s disease: A re-
view of findings and potential mechanisms. Behavioural Brain Research, 28,  9–15.
Gold, B. T., Johnson, N. F., & Powell, D. K. (2013). Lifelong bilingualism contributes to cog-
nitive reserve against white matter integrity declines in aging. Neuropsychologia, 51,
2841–2846.
Golestani, N., Molko, N., Dehaene, S., LeBihan, D., & Pallier, C. (2007). Brain structure predicts
the learning of foreign speech sounds. Cerebral Cortex, 17, 575–582.
Golestani, N., & Pallier, C. (2007). Anatomical correlates of foreign speech sound production.
Gough, P. M., Nobre, A. C., & Devlin, J. T. (2005). Dissociating linguistic processes inthe left
inferior frontal cortex with transcranial magnetic stimulation. Journal of Neuroscience, 25,
8010–8016.
Grady, C. L., Luk, G., Craik, F. I. M., & Bialystok, E. (2015). Brain network activity in monolin-
gual and bilingual older adults. Neuropsychologia, 66, 170–181.
Green, D. W. (1986). Control, activation, and resource: A framework and a model for the con-
trol of speech in bilinguals. Brain and Language, 27, 210–223.
Green, D. W. (1998). Mental control of the bilingual lexico-semantic system. Bilingualism:
Language and Cognition, 1,  67–81.
Green, D. W. (2003). Neural basis of lexicon and grammar in L2 acquisition. In R. van Hout, A.
Hulke, O. Kuiken, & R. J. Towell (Eds.), The lexicon syntax interface in second language acqui-
sition (pp. 197–218). Amsterdam: John Benjamins.
Green, D. W. (2008). Bilingual aphasia: Adapted language networks and their control. Annual
Review of Applied Linguistics, 28,  25–48.
Green, D. W. (2011). Language control in different contexts: The behavioral ecology of bilingual
speakers. Frontiers in Psychology, 2: 103.
Green, D. W., & Abutalebi, J. (2008). Understanding the link between bilingual aphasia and
language control. Journal of Neurolinguistics, 21, 558–576.
Green, D. W., & Abutalebi, J. (2013). Language control in bilinguals: The adaptive control hypo-
thesis. Journal of Cognitive Psychology, 25, 515–530.
Green, D. W., & Wei, L. (2014). A control process model of code-switching. Language,
Cognition & Neuroscience, 29, 499–511.
Green, D. W., & Wei, L. (2016) Code-switching and language control. Bilingualism: Language
and Cognition, 19, 883–884.
Griffiths, J. D., Marslen-Wilson, W.cD., Stamatakis, E.cA., & Tyler, L.cK. (2013). Functional or-
ganization of the neural language system: Dorsal and ventral pathways are critical for syntax.
Grogan, A., Green, D. W., Ali, N., Crinion, J., & Price, C. J. (2009). Structural correlates of
semantic and phonemic fluency ability in first and second languages. Cerebral Cortex, 19,
2690–2698.
Grogan, A., Parker-Jones, O., Ali, N., Crinion, J., Orabona, S., Mechias, M. L., . . . Price, C. J.
(2012). Structural correlates for lexical efficiency and number of languages in non-native
speakers of English. Neuropsychologia, 50, 1347–1352.
Grosjean, F. (1989). Neurolinguists, beware! The bilingual is not two monolinguals in one
person. Brain and Language, 36,  3–15.
Gunning-Dixon, F. M., Brickman, A. M., Cheng, J. C., & Alexopoulos, G. S. (2009). Aging of ce-
rebral white matter: A review of MRI findings. International Journal of Geriatric Psychiatry,
24, 109–117.
Guo, T., Liu, H., Misra, M., & Kroll, J. F. (2011). Local and global inhibition in bilingual word
production: fMRI evidence from Chinese–English bilinguals. NeuroImage, 56, 2300–2309.
Guzzardo Tamargo, R. E., Kroff, J. R. V., & Dussias, P. E. (2016). Examining the relationship
between comprehension and production processes in code-switched language. Journal of
Memory and Language, 89, 138–161.
Hahne, A., & Friederici, A. D. (2001). Processing a second language: Late learners comprehen-
sion mechanisms as revealed by event-related brain potentials. Bilingualism: Language and
Cognition, 4, 123–141.
Hartsuiker, R. J. R., Pickering, M. M. J., & Veltkamp, E. (2004). Is syntax separate or shared
between languages? Cross- linguistic syntactic priming in Spanish- English bilinguals.
Psychological Science, 15, 409–414.
Hernandez, A. E., Martinez, A., & Kohnert, K. (2000). In search of the language switch: An
fMRI study of picture naming in Spanish–English bilinguals. Brain and Language, 73,
421–431.
Hervais-Adelman, A., Moser-Mercer, B., Michel, C. M., & Golestani, N. (2015). fMRI of simul-
taneous interpretation reveals the neural basis of extreme language control. Cerebral Cortex,
25, 4727–4739. doi:10.1093/cercor/bhu158
Hickok, G., & Poeppel, D. (2007). The cortical organization of speech perception. Nature
Review Neuroscience, 8, 393–402.
Hope, T. M., Parker-Jones, O., Grogan, A., Crinion, J., Rae, J., Ruffle, L., Leff, A. P., Seghier, M.
L., Price, C. J., & Green, D. W. (2015) Comparing language outcomes in monolingual and bi-
lingual stroke patients. Brain, 138, 1070–1083.
Hosoda, C., Hanakawa, T., Nariai, T., Ohno, K., & Honda, M. (2012). Neural mechanisms of
language switch. Journal of Neurolinguistics, 25,  44–61.
Ito, M. (2008). Opinion: Control of mental activities by internal models in the cerebellum.
Nature Review Neuroscience, 9, 304–313.
Johnson, J. S., & Newport, E. L. (1989). Critical period effects in second language learning: The
influence of maturational state on the acquisition of English as a second language. Cognitive
Kalpouzos, G., Chetelat, G., Baron, J. C., Landeau, B., Mevel, K, Godeau, C., . . . Desgranges, B.
(2009). Voxel-based mapping of brain gray matter volume and glucose metabolism profiles
in normal aging. Neurobiology of Aging, 30, 112–124.
Kavé, G., Eyal, N., Shorek, A., & Cohen-Mansfield, J. (2008). Multilingualism and cognitive
state in the oldest old. Psychology and Aging, 23,  70–78.
Klein, D., Mok, K., Chen, J-K., Watkins, K. E. (2014). Age of language learning shapes brain
structure: A cortical thickness study of bilingual and monolingual individuals. Brain &
Language, 131,  20–24.
Kotz, S. A., Anwander, A., Axer, H., & Knösche, T. R. (2013). Beyond cytoarchitectonics: The
internal and external connectivity structure of the caudate nucleus. PLOS One, 8, e701741.
Kovelman, I., Baker, S. A., & Petitto, L.-A. (2008). Bilingual and monolingual brains compared: A
functional magnetic resonance imaging investigation of syntactic processing and a possible
“neural signature” of bilingualism. Journal of Cognitive Neuroscience, 20, 153–169.
Krienen, F. M., & Buckner, R. L. (2009). Segregated fronto-cerebellar circuits revealed by in-
trinsic functional connectivity. Cerebral Cortex, 19, 2485–2497.
Kriete, T., Noelle, D. C., Cohen, J. D., & O’Reilly, R. C. (2013). Indirection and symbol-like pro-
cessing in the prefrontal cortex and basal ganglia. Proceedings of the National Academy of
Sciences U.S.A, 110: 16390–16395.
Krizman, J., Marian, V., Shook, A., Skoe, E., & Kraus, N. (2012). Subcortical encoding of sound
is enhanced in bilinguals and relates to executive function advantages. Proceedings of the
Kroll, J. F., Dussias, P. E., Bice, K., & Perrotti, L. (2015). Bilingualism, mind, and brain. In M.
Liberman & B. H. Partee (Eds.), Annual Review of Linguistics, 1, 377–394.
Kroll, J. F., & Gollan, T. H. (2014). Speech planning in two languages: What bilinguals tell us
about language production. In V. Ferreira, M. Goldrick, & M. Miozzo (Eds.), The Oxford
handbook of language production (pp. 165–181). Oxford: Oxford University Press.
Kroll, J. F., Gullifer, J. W., McClain, R., Rossi, E., & Martín, M. C. (2015). Selection and control
in bilingual comprehension and production. In J. Schwieter (Ed.), Cambridge handbook of
bilingual processing (pp. 485–507). New York: Cambridge University Press.
Lee, H., Devlin, J. T., Shakeshaft, C., Stewart, L. H., Brennan, A., Glensman, J., . . . Price, C.
J. (2007). Anatomical traces of vocabulary acquisition in the adolescent brain. Journal of
Lehtonen, M., Laine, M., Niemi, J., Thomson T., Vorobyev, V. A., & Hughdal, K. (2005).
Brain correlates of sentence translation in Finnish-Norwegian bilinguals. NeuroReport, 16,
607–610.
Lesage, E., Morgan, B. E., Olson, A. C., Meyer, A. S., & Miall, R. C. (2012). Cerebellar rTMS
disrupts predictive language processing. Current Biology, 22, 794–795.
Li, P., Legault, J., & Litcofsky, K. A. (2014). Neuroplasticity as a function of second language
learning: Anatomical changes in the human brain. Cortex, 58, 301–324.
Loebell, H., & Bock, K. (2003). Structural priming across languages. Linguistics, 41, 791–824.
Lucas, T. H., McKhann, G. M., & Ojemann, G. A. (2004). Functional separation of languages in
the bilingual brain: A comparison of electrical stimulation language mapping in 25 bilingual
patients and 117 monolingual control patients. Journal of Neurosurgery, 101, 449–457.
Luk, G., Bialystok, E., Craik, F. I., & Grady, C. L. (2011). Lifelong bilingualism maintains white
matter integrity in older adults. Journal of Neuroscience, 31, 16808–16813.
Luk, G., Green, D. W., Abutalebi, J., & Grady, C. (2012). Cognitive control for language
switching in bilinguals: A quantitative meta-analysis on functional neuroimaging studies.
Language and Cognitive Processes, 27, 1479–1488.
Majerus, S., D’Argembeau, A., Martinez Perez, T., Belayachi, S., Van der Linden, M., & Collette,
F. (2010). The commonality of neural networks for verbal and visual short-term memory.
Journal of Cognitive Neuroscience, 22, 2570–2593.
Marian, V., Spivey, M., & Hirsch, J. (2003). Shared and separate systems in bilingual language
processing: Converging evidence from eyetracking and brain imaging. Brain and Language,
86,  70–82.
Marien, P., Engelborghs, S., Fabbro, F., & De Deyn, P. P. (2001). The lateralized linguistic cere-
bellum: A review and a new hypothesis. Brain and Language, 79, 580–600.
Mather, M., & Harley, C. W. (2016). The locus coeruleus: Essential for maintaining cognitive
function and the aging brain. Trends in Cognitive Sciences, 20, 214–226.
Mayda, A. B. V., Westphal, A., Carter, C. S., & DeCarli, C. (2011). Later life cognitive control
deficits are accentuated by white matter disease burden. Brain, 134, 1673–1683.
McLaughlin, J., Osterhout, L., & Kim, A. (2004). Neural correlates of second-language word
learning: Minimal instruction produces rapid change. Nature Neuroscience, 7, 703–704.
McLaughlin, J., Tanner, D., Pitkänen, I., Frenck-Mestre, C., Inoue, K., Valentine, G., &
Osterhout, L. (2010). Brain potentials reveal discrete stages of L2 grammatical learning.
Language Learning, 60, 123–150.
Mechelli, A., Crinion, J. T., Noppeney, U., O’Doherty, J., Ashburner, J., Frackowiack, R. S.,
& Price, C. J. (2004). Neurolinguistics: Structural plasticity in the bilingual brain. Nature,
431, 757.
Meulman, N., Stowe. L. A., Sprenger, S. A., Bresser, M., & Schmid, M. S. (2014) An ERP study
on L2 syntax processing: When do learners fail? Frontiers in Psychology, 5, 1072.
Meuter, R. F. I., & Allport, A. (1999). Bilingual language switching in naming: Asymmetrical
costs of language selection. Journal of Memory and Language, 40,  25–40.
Misra, M., Guo, T., Bobb, S. C., & Kroll, J. F. (2012). When bilinguals choose a single word
to speak: Electrophysiological evidence for inhibition of the native language. Journal of
Mohades, S. G., Struys, E., Van Schuerbeek, P., Mondt, K., Van Den Craen, P., & Luypaert, R.
(2012). DTI reveals structural differences in white matter tracts between bilingual and mon-
olingual children. Brain Research, 1435,  72–80.
Morford, J. P., Wilkinson, E., Villwock, A., Piñar, P., & Kroll, J. F. (2011). When deaf signers read
English: Do written words activate their sign translations? Cognition, 118, 286–292.
Morgan-Short, K., Finger, I., Grey, S., & Ullman, M. T. (2012). Second language processing shows
increased native-like neural responses after months of no exposure. PLoS One, 7: e32974.
Morga-Short, K., Steinhauer, K., Sanz, C., & Ullman, M. T. (2012). Explicit and implicit second
language training differentially affects the achievement of native-like brain activation
patterns. Journal of Cognitive Neuroscience, 24, 933–947.
Mummery, C. J., Patterson, K., Hodges, J. R., & Wise, R. J. (1996). Generating “tiger” as an an-
imal name or a word beginning with T: Differences in brain activation. Proceedings of the
Royal Society of London B: Biological Sciences, 263, 989–995.
Oberhuber, M., ParkerJones, O., Hope, T. M. H., Prejawa, S., Seghier, M. L., Green, D. W., &
Price, C. J. (2013). Functionally distinct contributions of the anterior and posterior putamen
during sublexical and lexical reading. Frontiers in Human Neuroscience, 7: 787.
Oh, A., Duerden, E. G., & Pang, E. W. (2014). The role of the insula in speech and language pro-
cessing. Brain & Language, 135, 96–103.
Ojemann, G. A., & Whitaker, H. A. (1978.) The bilingual brain. Archives of Neurology, 35,
409–412.
Olulade, O. A., Jamal, N. I., Koo, D. S., Perfetti, C. A., LaSasso, C., & Eden, G. F. (2015).
Neuroanatomical evidence in support of the bilingual advantage theory. Cerebral Cortex, 26,
3196–3204.
Osipowicz, K. Z., Rickards, T., Shah, A., Sharan, A., Sperling, M. Kahn, W., & Tracy, J. (2011).
A test of the role of the medial temporal lobe in single-word decoding. NeuroImage, 54,
1455–1464.
Osterhout, L., Poliakov, A., Inoue, K., McLaughlin, J., Valentine, G., Pitkanen,
I., . . . Hirschensohn, J. (2008). Second-language learning and changes in the brain. Journal
of Neurolinguistics, 21, 509–521
Pakulak, E., & Neville, H. J. (2010). Proficiency differences in syntactic processing of monolin-
gual native speakers indexed by event-related potentials. Journal of Cognitive Neuroscience,
22, 2728–2744.
Paradis, M. (2004). A neurolinguistic theory of bilingualism. Amsterdam: John Benjamins.
Paradis, M. (2008). Declarative and procedural determinants of second languages. Studies in

Bilingualism 40. Amsterdam: John Benjamins.
Parker-Jones, O., Green, D. W., Grogan, A., Pliatsikas, C., Filippopolitis, K., Ali, N., . . . Price,
C. J. (2012). Where, when and why brain activation differs for bilinguals and monolinguals
during picture naming and reading aloud. Cerebral Cortex, 22, 892–902.
Perani, D., & Abutalebi, J. (2015). Bilingualism, dementia, cognitive and neural reserve. Current
Opinion of Neurology, 28, 618–625.
Perani, D., Abutalebi, J., Paulesu, E., Brambati, S., Scifo, P., Cappa, S. F., & Fazio, F. (2003). The
role of age of acquisition and language usage in early, high-proficient bilinguals: An fMRI
study during verbal fluency. Human Brain Mapping, 19, 170–182.
Perquin, M., Vaillant, M., Schuller, A.-M., Pastore, J., Dartigues, J.-F., Lair, M.-L., & Diederich,
N. (2013). Lifelong exposure to multilingualism: New evidence to support cognitive reserve
hypothesis. PLoS One, 8, 4. doi: 10.1371/journal.pone.0062030
Petrides, M., Alivisatos, B., Meyer, E., & Evans, A. C. (1993). Functional activation of the human
frontal cortex during the performance of verbal working memory tasks. Proceedings of the
National Academy of Sciences USA, 90, 878–882.
Pfefferbaum, A., Adalsteinsson, E., & Sullivan, E. V. (2005). Frontal circuitry degradation marks
healthy adult aging: Evidence from diffusion tensor imaging. NeuroImage, 26, 891–899.
Pierce, L., Klein, D., Chen, J.-K., Delcenserie, A., & Genesee, F. (2014). Mapping the uncon-
scious maintenance of a lost first language. Proceedings of the National Academy of Sciences
USA, 111, 7314–7319.
Pierce, L. J., Chen, J.-K., Delcenserie, A., Genesee, F., & Klein, D. (2015). Past experience shapes
ongoing neural patterns for language. Nature Communications, 6, 10073.
Pliatsikas, C., Johnstone, T., & Marinis, T. (2014). Grey matter volume in the cerebellum is re-
lated to the processing of grammatical rules in a second language: A structural voxel-based
morphometry study. Cerebellum, 13,  55–63.
Pliatsikas, C., & Luk, G. (2016). Executive control in bilinguals: A concise review of fMRI
studies. Bilingualism: Language and Cognition, 19, 699–705.
Pötzl, O. (1925).Über die parietal bedingte Aphasie und ihren Einfluss auf das Sprechen mehrer
Sprachen. Zeitschrift für die gesamte Neurologie und Psychiatrie, 96, 100–1124.
Price, C. J. (2012). A review and synthesis of the first 20 years of PET and fMRI studies of heard
speech, spoken language and reading. NeuroImage, 62, 816–847. doi: 10.1038/ncomms10073
Price, C. J., Green, D. W., & von Studnitz, R. A. (1999). Functional imaging study of translation
and language switching. Brain, 122, 2221–2236.
Ressel, V., Pallier, C., Ventura-Campos, N., Díaz, B., Roessler, A., Ávila, C., & Sebastián-Gallés,
N. (2012). An effect of bilingualism on the auditory cortex. Journal of Neuroscience, 32,
16597–16601.
Richardson, F., Thomas, M. S. C., Filippi, R., Harth, H., & Price, C. J. (2009). Contrasting effects
of vocabulary knowledge on temporal and parietal brain structure across the life span.
Ripollés, P., Marco-Pallarés, J., Hielscher, U., Mestres-Missé, A., Tempelmann, C., Heinze,
H.-J., . . . Noesselt, T. (2014). The role of reward in word learning and its implications for
language acquisition. Current Biology, 24, 2606–2611.
Rodriguez-Fornells, A., Van der Lugt, A., Rotte, M., Britti, B., Heinze, H. J., & Muente, T. F.
(2005). Second language interferes with word production in fluent bilinguals: Brain poten-
tial and functional imaging evidence. Journal of Cognitive Neuroscience, 17, 422–433.
Roncaglia-Denissen, M. P., & Kotz, S. A. (2016). What does neuroimaging tell us about the
morhosyntactic processes in the brain of second language learners? Bilingualism: Language
and Cognition, 19, 665–673.
Rossi, E., Kroll, J. F., & Dussias, P. E. (2014). Clitic pronouns reveal the time course of pro-
cessing gender and number in a second language. Neuropsychologia, 62,  11–25.
Roux, F.-E., & Trémoulet, M. (2002). Organization of language areas in bilingual patients: A
cortical stimulation study. Journal of Neurosurgery, 97, 857–864.
Sanoudaki, E., & Thierry, G. (2015). Language non-selective syntactic activation in early
bilinguals: The role of verbal fluency. International Journal of Bilingual Education and
Bilingualism, 18, 548–560.
Saur, D., Kreher, B. W., Schnell, S., Kümmerer, P., Kellmeyer, P., Vry, M-S., . . . Weiller, C. (2008).
Ventral and dorsal pathways for language. Proceedings of the National Academy of Sciences
USA, 105, 18035–18040.
Schneider, P., Scherg, M., Dosch, H. G., Specht, H. J., Gutschalk, A., & Rupp, A. (2002).
Morphology of Heschl’s gyrus reflects enhanced activation in the auditory cortex of
musicians. Nature Neuroscience, 5, 688–694.
Shadmehr, R., & Holcomb, H. H. (1999). Inhibitory control of competing motor memories.
Experimental Brain Research, 126, 235–251.
Shomstein, S. (2012). Cognitive functions of the posterior parietal cortex: Top-down and
bottom-up attentional control. Frontiers in Integrative Neuroscience, 6, 38. doi: 10.3389/
fnint.2012.00038.
Silveri, M. C., Leggio, M. G., & Molinari, M. (1994). The cerebellum contributes to linguistic
production: A case of agrammatism of speech following right hemicerebellar lesion.
Neurology, 44, 2047–2050.
Smith, Y., Surmeier, D. J., Redgrave, P., & Kimura, M. (2011). Thalamic contributions to
basal gangliarelated behavioral switching and reinforcement. Journal of Neuroscience, 31,
16102–16106.
Snyder, H. R., Hutchison, N., Nyhus, E., Curran, T., Banich, M. T., O’Reilly, R. C., & Munakata,
Y. (2010). Neural inhibition enables selection during language processing. Proceedings of the
Stein, M., Federspiel, A., Koenig, T., Wirth, M., Strik, W., & Wiest, R. (2012). Structural plas-
ticity in the language system related to increased second language proficiency. Cortex, 48,
458–465.
Steinhauer, K. (2014). Event-related potentials (ERPs) in second language research: A brief
introduction to the technique, a selected review, and an invitation to reconsider critical
periods in L2. Applied Linguistics, 35, 393–417.
Stocco, A., Yamasaki, B., Natalenko, R., & Prat, C. S. (2014). Bilingual brain training: A neurobi-
ological framework of how bilingual experience improves executive function. International
Journal of Bilingualism, 18,  67–92.
Syal, S., & Finlay, B. L. (2011). Thinking outside the cortex: Social motivation in the evolution
and development of language. Developmental Science, 14, 417–430.
Tan, L. H., Chen, L., Yip, V., Chan, A. H. D., Yang, J., Gao, J.-H., & Siok, W. T. (2011).
Activity levels in the left hemisphere caudate-fusiform circuit predict how well a second
language will be learned. Proceedings of the National Academy of Sciences USA, 108,
62540–62544.
Tanner, D., Inoue, K., & Osterhout, L. (2014). Brain-based individual differences in online L2
grammatical comprehension. Bilingualism: Language and Cognition, 17, 277–293.
Tanner, D., McLaughlin, J., Herschensohn, J., & Osterhout, L. (2013). Individual differences
reveal stages of L2 grammatical acquisition: ERP evidence. Bilingualism: Language and
Cognition, 16, 367–382.
Thierry, G., & Wu, Y. J. (2007). Brain potentials reveal unconscious translation during foreign-
language comprehension. Proceedings of the National Academy of Sciences, 104, 12530–12535.
Thompson-Schill, S. L., D’Esposito, M., & Dan, I. P. (1999). Effects of repetition and competi-
tion on activity in left prefrontal cortex during word generation. Neuron, 23, 513–522.
Tokowicz, N., & MacWhinney, B. (2005). Implicit and explicit measures of sensitivity to
violations in second language grammar: An event-related potential investigation. Studies in
Second Language Acquisition, 27, 173–204.
Tu, L., Wang, J., Abutalebi, J., Jiang, B., Pan, X., Li, M., . . . Huang, R. (2015). Language exposure
induced neuroplasticity in the bilingual brain: A follow-up fMRI study. Cortex, 64,  8–19.
Tyler, L. K., Randall, B., & Stamatakis, E. A. (2008). Cortical differentiation for nouns and verbs
depends on grammatical markers. Journal of Cognitive Neuroscience, 20, 1381–1389.
Tyson, B., Lantrip, A. C., & Roth, R. M. (2014). Cerebellar contributions to implicit learning
and executive function. Cognitive Sciences, 9, 179–217.
Ullman, M. (2001). The neural basis of lexicon and grammar in first and second language: The
declarative/procedural model. Bilingualism: Language and Cognition, 4, 105–122.
Van Assche, E., Duyck, W., & Gollan, T. (2013). Whole-language and item-specific control in
bilingual language production. Journal of Experimental Psychology: Memory, Language and
Cognition, 39, 1781–1792.
Van Hell, J. G., & Tokowicz, N. (2010). Event-related brain potentials and second language
learning: Syntactic processing in late L2 learners at different L2 proficiency levels. Second
Language Research, 26,  43–74.
Van Heuven, W. J. B., Schriefers, H., Dijkstra, T., & Hagoort, P. (2008). Language conflict in the
bilingual brain. Cerebral Cortex, 18, 2706–2716.
Vaughan-Evans, A., Kuipers, J. R., Thierry, G., & Jones, M. W. (2014). Anomalous transfer of
syntax between languages. Journal of Neuroscience, 34, 8333–8335.
Veroude, K., Norris, D. G., Shumskaya, E., Gullberg, M., & Indefrey, P. (2010). Functional
connectivity between brain regions involved in learning words in a new language. Brain &
Language, 113,  21–27.
Videsott, G., Herrnberger, B., Hoenig, K., Schilly, E., Grothe, J., Wiater, W., . . . Kiefer, M. (2010).
Speaking in multiple languages: Neural correlates of language proficiency in multilingual
word production. Brain & Language, 113, 103–112.
Vigneau, M., Beaucousin, V., Herve, P. Y., Duffau, H., Crivello, F., Houdé, O., & Tzourio-
and sentence processing. NeuroImage, 30, 1414–1432.
Weber-Fox, C., & Neville, H. J. (1996). Maturational constraints on functional specialization
for language processing: ERP and behavioral evidence in bilingual speakers. Journal of
Wernicke, C. (1874). Der aphasische Symptomencomplex. Berlin: Springer-Verlag.
Wilson, S. M., Galantucci, S., Tartaglia, M. C. R. K., Patterson, D. K., Henry, M. L., Ogar, J.
M., . . . Gorno-Tempini, M. L. (2011). Syntactic processing depends on dorsal language
tracts. Neuron, 72, 397–403.
Wong, F. C. K., Chandrasekaran, B., Garibaldi, K., & Wong, P. C. M. (2011). White matter ani-
sotropy in the ventral language pathway predicts sound-to-word learning success. Journal of
Woumans, E., Santens, P., Sieben, P., Versijpt, J., Stevens, M., & Duyck, W. (2015). Bilingualism
delays clinical manifestation of Alzheimer’s disease. Bilingualism: Language and Cognition,
18, 568–574.
Xiang, H., Dediu, D., Roberts, L., van Oort, E., Norris, D. G., & Hagoort, P. (2012). The struc-
tural connectivity underpinning language aptitude, working memory, and IQ in the
perisylvian language network. Language Learning, 62(Suppl 2), 110–130.
Yang, J., Gates, K. M., Molenaar, P., & Li, P. (2015). Neural changes underlying successful second
language word learning: An fMRI study. Journal of Neurolinguistics, 33,  29–49.
Zou, L., Ding, G., Abutalebi, J., Shu, H., & Peng, D. (2012). Structural plasticity of the left cau-
date in bimodal bilinguals. Cortex, 48, 1197–1206.
Chapter 12
L anguage and   Ag i ng
Jonathan E. Peelle
Introduction
As we age, sensory and cognitive changes occur that create challenges for language pro-
cessing. At the same time, we gain increased experience and expertise with language and
may adopt more efficient processing strategies. These complementary facets of normal
aging suggest that age-related changes in language processing will be multifactorial.
Understanding language processing in older adulthood is important for both prac-
tical and theoretical reasons. Practically, a mechanistic understanding of how older
brains process language may help an increasingly aging population communicate. From
a theoretical standpoint, predictions made regarding how sensory and cognitive factors
jointly contribute to language processing should be borne out in older adults, who typ-
ically show more variability than their younger counterparts in both of these domains.
That is, normal aging is just one example of neuroanatomical and behavioral variability
that can be useful to test theoretically important issues in neurolinguistics. For example,
to what degree is language processing modular (Fodor, 1983)? If older adults show dif-
ferent patterns of neural processing during language processing from those observed in
young adults, what implications might this have for our understanding of cognitive or-
ganization? Healthy aging is therefore a useful model for studying behavioral success in
the presence of changing biology, and the lessons learned have implications for a variety
of demographic and clinical populations.
In this chapter, I present an overview of some of the main themes associated with
cognitive aging, followed by specific examples of how older adults process language,
focusing on neural mechanisms wherever possible (for comprehensive reviews of be-
havioral findings, see Kemper, 1992; Wingfield & Stine-Morrow, 2000). Although I will
focus on spoken language, many of the general principles also hold true for written and
signed language.
296 Jonathan E. Peelle
Cognitive Aging
Language processing in older adults can only be understood within a broader picture
of age-related cognitive change and mechanisms related to neural adjustment and
compensation. I thus begin with a brief overview of the key points in the cognitive
aging literature, many of which return more specifically in the context of language
processing.
Age-Related Changes in Brain Structure

and Cognitive Function
As we age, our brains get smaller and lighter: gyri widen and ventricles enlarge to a
degree that is apparent to the naked eye on structural brain scans. It is not surprising
then, to find significant reductions in gray matter volume and cortical thickness in older
adulthood (Good et al., 2001; Raz et al., 1997; Raz, Gunning-Dixon, Head, Dupuis, &
Acker, 1998; Raz et al., 2005; Salat et al., 2004). Historically, age-related changes in gray
matter have been discussed most often in the context of frontal and prefrontal cortex
(Raz et al., 1997), although there are age-related changes in nearly every region of the
brain (Fjell et al., 2009; Peelle, Cusack, & Henson, 2012). Changes in the composition
of white matter are also common (Salat et al., 2005) and are typically measured using
fractional anisotropy or other diffusion coefficients from diffusion-weighted imaging
(see Catani & Forkel, Chapter 9 in this volume). White matter changes are frequently
interpreted as relating to demyelination and thus less efficient communication between
brain regions, although caution should be used when interpreting diffusion-weighted
measures (Jones, Knösche, & Turner, 2013).
It is important to keep in mind methodological issues, such as changes in image con-
trast mechanisms associated with age (Salat et al., 2009) or the interpretation of re-
gional change relative to global effects (Peelle et al., 2012), which may affect these results.
Nevertheless, there is clear evidence that our brains change as we age, even in the ab-
sence of neurodegenerative disease.
In addition to widespread neuroanatomical change, healthy aging is also associ-
ated with altered cognitive processing across a wide number of domains, including
working memory (Mitchell, Johnson, Raye, Mather, & D’Esposito, 2000), attention
and inhibitory control (Hasher, Stoltzfus, Zacks, & Rypma, 1991), general processing
speed constructs (Salthouse, 1996), and many others. Verbal knowledge is typically
well preserved (and frequently increases with age, as evidenced by increasing vocabu-
lary sizes; Verhaeghen, 2003), but nearly every other domain studied by cognitive aging
researchers reveals significant age-related changes in accuracy, reaction time, or pro-
cessing strategy (Hedden & Gabrieli, 2004).
Language and Aging 297
Adaptive Processing and Compensation

in Older Adulthood
Given the concomitant neuroanatomical and cognitive changes that occur in normal
aging, it would be surprising if older adults achieved a behavioral outcome with iden-
tical neural processes to those of young adults. As a first step, it can also be useful to
examine differences in neural processing at the group level. Are there any consistent
patterns that distinguish older adults’ successful processing?
Since the early days of functional brain imaging, a frequent observation is that older
adults tend to show less activity than young adults in the regions associated with that
task in young adults, but increased activity in other regions (Grady, 2000). For ex-
ample, Cabeza and colleagues (1997) performed a positron emission tomography (PET)
study of memory for pairs of words, and noted that older adults appeared to show
reduced activity during encoding compared to the young adults (and increased bilat-
eral activity during retrieval, discussed in more detail later in this chapter). Reports of
underactivation—that is, older adults showing reduced activity compared to young
adults in what are assumed to be “core” processing areas for a task—have led to the
suggestion that neural processing in older adults is less specialized than that of young
adults (Park et al., 2004). The concurrent increases in activity viewed outside core re-
gions are often viewed as compensatory, reflecting an older brain making up for lost
processing efficiency by recruiting additional resources. Interpreting the older adults’
“overactivation” as a beneficial adaptation has intuitive appeal, and is also supported
by studies in which older adults’ extra activity is correlated with behavioral accuracy.
(However, such a correlation is not always found, meaning that compensation-related
explanations for age-related differences in brain activity need to be viewed with caution.)
As hinted at earlier, a number of early studies examining age-related differences in
neural processing used tasks in which young adults showed left-lateralized activation
(that is, hemispheric asymmetry). When older adults performed the same task suc-
cessfully, they frequently showed activity in the contralateral homologous region, and
thus a lesser degree of hemispheric asymmetry. In Cabeza et al. (1997), for example, the
authors found that during retrieval young adults showed right-lateralized activity in
frontal cortex, whereas older adults showed bilateral activity in the same task. Although
at the time it was uncommon to statistically assess the degree of lateralization in neuro-
imaging studies, qualitative results strongly suggested age differences in the degree of
lateralized activity, at least for some tasks. Findings such as these led to the Hemispheric
Asymmetry Reduction in OLDer adults (HAROLD) model (Cabeza, 2002), which
proposed that reduction in hemispheric asymmetry is a key feature of neural processing
in older adulthood, and is thus a guiding principle often used to consider age differences
in activity. Additional frameworks proposing a key role for neural compensation in
aging include the Compensation-Related Utilization of Neural Circuits Hypothesis
(CRUNCH; Reuter-Lorenz & Lustig, 2005) and the Scaffolding Theory of Aging and
Cognition (STAC; Reuter-Lorenz & Park, 2014). CRUNCH and STAC also focus on the
importance of compensatory activity, but are more agnostic about the specific type of
activity to be expected (that is, there is less emphasis on reductions in lateralization).
What these models have in common is that all propose that increased activity is needed
for older adults to maintain the same performance as young adults.
An important question concerns the nature of the compensatory “resources” older
adults up-regulate. What are these additional cognitive processes? One way to answer
this question is to classify them as domain-preferential or domain-general. At a broad
level, HAROLD-type activation may be considered as tending more toward the domain-
preferential end of the spectrum, under the assumption that contralateral homologous
regions frequently perform related functions. In contrast, another type of compensa-
tion would reflect up-regulation of domain-general systems such as executive atten-
tion networks (Duncan, 2010; Power & Petersen, 2013). These networks are considered
domain-general because they can be dynamically engaged in the context of current
task demands and goals, flexibly coding task-relevant information (Stokes et al., 2013;
Woolgar, Hampshire, Thompson, & Duncan, 2011). Examples of both are seen during
language processing.
Keeping in mind the brain’s ability to flexibly compensate for task-related challenge,
successful aging can be seen as relying on the relationship between supply and demand
of processing resources (Figure 12.1). Processing efficiency reflects the degree to which
any individual has a neurocognitive capacity relevant for a given task; demand is deter-
mined by the current task requirements, which for language tasks might consist of the
joint challenges of perceptual, sensory, and meta-linguistic (i.e., thinking about the use
of language) processing. If an individual’s neural resources are sufficient to meet the de-
mand, performance will be high. If not, performance will begin to deteriorate. Poor per-
formance thus results from a mismatch between cognitive supply and demand. (In some
sense, this can be thought of equally as reflecting too great of a behavioral challenge or
Number of neurons Word frequency

Dendritic branching Task Lexical competition
Myelin integrity Processing requirements Semantic ambiguity
Working memory efficiency Syntactic complexity
Attention Task demands
... ...
Neural
Behavior
engagement
Figure 12.1. Schematic framework within which to consider the neural networks supporting
language processing. The neural systems engaged and degree of behavioral success reflect a com-
plex balance between the specific task requirements (including the level of linguistic processing
required) and the level of cognitive resources listeners have available (which is tied to underlying
neurophysiology).
too few neural resources, although the degree to which this distinction matters is an
open question.)
Finally, as with structural magnetic resonance imaging (MRI) data, it is important
to consider methodological concerns that might lead to age-related changes in func-
tional measures that are unrelated to cognitive processing. These include differences
in the shape or timing of the hemodynamic response (D’Esposito, Zarahn, Aguirre, &
Rypma, 1999), neurovascular coupling (Tsvetanov et al., 2015), and movement within
the scanner that may affect estimates of activity or network connectivity (Power, Barnes,
Snyder, Schlaggar, & Petersen, 2012). Increased variability in the timing of neural
responses can also significantly impact group-averaged event-related potentials (ERPs).
Having thus briefly set the stage with a general framework for successful performance
in normal aging, I now turn to reviewing comprehension and production of language in
older adulthood as specific applications of these general principles.
Language Comprehension
in Normal Aging
Most neuroantomically constrained models of spoken language comprehension have

been formulated based on known anatomical connectivity, behavioral data from stroke
patients, and functional brain imaging in healthy (typically young) adults. There is
good general agreement on the importance of bilateral temporal cortex for auditory
and single-word processing, extending into left lateral temporal cortex and inferior
frontal gyrus (IFG) in the context of sentence-level material (Hickok & Poeppel, 2007;
see, in this volume, Poeppel, Cogan, Davidesco, & Flinker, Chapter 26, and Bornkessel-
Schlesewsky & Schlesewsky, Chapter 27; Peelle, Johnsrude, & Davis, 2010; Rauschecker
& Scott, 2009). However, much less is known regarding how these networks change in
the context of healthy aging. In the following subsections, I discuss patterns of neural
processing for language in older adults relative to those seen in young adults, some
illustrations of which are shown in Figure 12.2.
Word Perception
Contemporary theories of spoken word perception frequently operate within a lex-
ical competition framework in which listeners must correctly identify a target word by
inhibiting (or not selecting) similar-sounding items in the lexicon (Gagnepain, Henson, &
Davis, 2012; Luce & Pisoni, 1998; Marslen-Wilson & Tyler, 1980; Norris & McQueen, 2008).
Behaviorally, older adults have more difficulty recognizing spoken words, particularly in
noise (Dubno, Dirks, & Morgan, 1984; Pichora-Fuller, Schneider, & Daneman, 1995), and
studies show that older adults’ spoken-word processing is affected by both lexical and
(a) Words in noise

Errors Reduced SNR
(b) Anomolous sentences
Young adults
Older adults
(c) Syntactic complexity
All listeners
Older > Young
Age differences
Young > Older
Figure 12.2. (A) Increased activity in the cingulo-opercular network during spoken word
processing in response to both errors and poorer signal-to-noise ratio (SNR).
Source: Vaden, et al., Journal of Neuroscience (2015). Permission granted.
(B) Activity for spoken sentences with anomalous prose in young and older adults during a target
word-monitoring task (Tyler, et al., 2010). Older adults show increased activity in right frontal
cortex to support sentence comprehension.
(C) Regions showing differential activation for spoken sentences with subject-relative versus
subject-relative embedded clauses (Peelle, Troiani, et al., 2010). Older adults show less activity in
left inferior frontal gyrus, but greater activity in regions of prefrontal cortex outside the common
syntax network.
Source: Used with permission of Oxford University Press; License number 3756501173119.
cognitive factors (Humes, Kidd, & Lentz, 2013; Lash, Rogers, Zoller, & Wingfield, 2013;
Sommers & Danielson, 1999). What are the neural processes that are engaged?
Shafto and colleagues (2012) presented young and older adults with spoken words
and nonwords that varied in both cohort competition (Marslen-Wilson, 1987) and
imageability, along with a complex acoustic baseline. Young adults show increased
activity in IFG as a function of increased lexical competition and selection demands
(Zhuang, Tyler, Randall, Stamatakis, & Marslen-Wilson, 2014). In contrast, older
adults appear to be less sensitive to competition and selection manipulations, although
potentially more sensitive to imageability effects (Shafto et al., 2012). Research with
visual word identification also suggests some age differences in processing that may dis-
sociate lexical and sublexical components (Whiting et al., 2003). These studies suggest
that even at the single-word level, well-known lexical properties may be processed dif-
ferently in older adults compared to young adults.
Functional MRI (fMRI) studies of spoken word perception in older adults also im-
plicate executive attention systems in speech perception when noise is present. Eckert
and colleagues (2008) presented listeners with single spoken words that were low-pass
filtered (to mimic the effects of high-frequency hearing loss); intelligibility varied as a
function of the cutoff frequency. Overall intelligibility across age groups was equated
using broadband noise. On each trial, listeners repeated the presented word back if pos-
sible, providing a trial-by-trial measure of recognition accuracy. For correct trials, older
adults showed different patterns of activation compared to young adults, with increased
activity in middle frontal gyrus, anterior cingulate, and visual cortex. These results sug-
gest that in challenging listening situations, older adults rely to a greater degree on ex-
ecutive attention mechanisms (middle frontal gyrus and anterior cingulate) than do
young adults, even when they are successful.
It is difficult to talk about the neural systems contributing to speech perception in
adverse listening conditions without discussing the cingulo-opercular network. The
cingulo-opercular network is part of the multiple-demand network, comprising the
dorsal anterior cingulate and bilateral frontal operculum (or anterior insula, depending
on the characterization) (Dosenbach, Fair, Cohen, Schlaggar, & Petersen, 2008;
Duncan, 2010; Power & Petersen, 2013). Across a variety of tasks, the cingulo-opercular
network shows elevated responses for sustained attention and error trials (Neta et al.,
2015). In spoken-word processing, elevated cingulo-opercular activity is seen in speech
comprehension tasks when the acoustic signal is degraded enough to result in reduced
intelligibility (and thus behavioral errors), including in older adults (Eckert et al., 2009).
Further evidence for the functional role of the cingulo-opercular network in language
processing comes from studies examining whether activity following one trial can pre-
dict accuracy on the next. Vaden and colleagues (2015) examined this issue using ge-
neral linear mixed-model analysis to predict the accuracy of participants’ single-word
recognition in multi-talker babble. The authors found, first, that activity in older adults’
cingulo-opercular network predicted their accuracy on a subsequent trial, and, second,
that the level of cingulo-opercular modulation was lower for older adults than for young
adults (Vaden et al., 2013). These findings implicate cognitive control mechanisms
supported by the cingulo-opercular network in single-word perception.
Of course, regions beyond the cingulo-opercular network have also been implicated
in older adults’ word perception. Kuchinsky and colleagues (2012) also presented single
words that had been bandpass filtered to different levels in order to manipulate intelli-
gibility. As expected, word perception was associated with bilateral temporal cortex ac-
tivity. Both increasing age and decreasing word intelligibility were associated with more
activation in occipital cortex. Interestingly, occipital cortex was functionally connected
to left temporal cortex (including middle temporal cortex, superior temporal gyrus, and
Heschl’s gyrus), suggesting that it may have a relationship to task-relevant processing.

These results suggest that cross-modal attention to speech cues may be impacted by age,
as well as by the quality of the auditory stimuli being presented.
In summary, single-word processing in older adults appears to be supported broadly
by similar systems to those used by young adults—namely, bilateral temporal cortex.
However, differences have been reported with respect to psycholinguistic and atten-
tional factors. Published studies to date suggest that older adults use a more extensive
set of brain regions when processing single words, and suggest a complex interaction
between brain structure, brain function, and behavior (Bilodeau-Mercure, Lortie, Sato,
Guitton, & Tremblay, 2015). A further point is that the acoustic clarity of items (in-
cluding age-related changes in hearing sensitivity) appears to have a significant impact;
I return to the important issue of perceptual challenge in more detail in the following.
Sentence Comprehension
In addition to the phonological, lexical, and semantic computations involved in single-
word perception, sentences require additional processes related to correctly parsing the
syntax of an utterance, and frequently contain words (such as pronouns or ambiguous
words) that require context to interpret (Rodd, Johnsrude, & Davis, 2012). These added
complexities mean that spoken sentences generally rely on a broader network of brain
regions than that required for single words, including more extensive regions of bilateral
temporal cortex and left IFG (Peelle, 2012).
A productive line of research on semantic processing has used electroencephalography
(EEG) (see Leckey & Federmeier, Chapter 3 in this volume) to investigate time-locked
activity to words occurring with different levels of preceding predictability. These studies
therefore assess the ability of listeners to make use of semantic context during online sen-
tence comprehension. Federmeier and Kutas (2005) presented young and older adults
with written sentences in which the final word was highly predictable (e.g., “No one at the
reunion recognized Dan because he had grown a beard”) or not very predictable (e.g., “At
the children’s park next to the beach she saw a man with a beard”). Young adults showed a
pronounced difference in time-locked response between strongly and weakly constrained
sentences, an expected N400 response (Kutas & Hillyard, 1984). In contrast, older adults
as a group showed less difference between the strongly and weakly constrained sentence
types, suggesting that older adults may be less able to use contextual constraints during
online sentence processing (Wlotko, Federmeier, & Kutas, 2012). Later studies suggest that
individual differences exist, with some older adults showing patterns that are more sim-
ilar to those seen in young adults (Federmeier, Kutas, & Schul, 2010). (These online EEG
measures can be contrasted with behavioral measures that frequently show older adults
showing greater reliance on context compared to young adults [Dubno, Ahlstrom, &
Horwitz, 2000; Pichora-Fuller, et al., 1995; Wingfield, Aberdeen, & Stine, 1991].)
Behaviorally, there is considerable evidence that older adults have difficulty pro-
cessing syntactically complex sentences, including reduced use of grammatical
complexity in production (Kemper, Greiner, Marquis, Prenovost, & Mitzner, 2001;

Kemper, Marquis, & Thompson, 2001), and differentially longer response times on com-
prehension tasks (Kemtes & Kemper, 1997; Waters & Caplan, 2001; Wingfield, Peelle, &
Grossman, 2003). Functional neuroimaging studies of age-related changes in sentence
processing have reported at least two patterns of results.
In the first fMRI study, young and older adults were presented with short spoken
sentences that contained either a subject-relative or object-relative center-embedded
clause, making a keypress response to indicate the gender of the character performing
the action (Peelle, Troiani, Wingfield, & Grossman, 2010). For example, when hearing
the sentence “Brothers that sisters assist are happy,” participants would press a button
to indicate that a female is performing the action (in this case, assisting). Older adults’
accuracy was generally high on this task, consistent with a high level of successful com-
prehension. Both young and older listeners showed increased activity for the more com-
plex object-relative sentences compared to subject-relative sentences. However, older
adults showed less activity than the young adults in left ventral IFG, and increased ac-
tivity in regions of dorsolateral prefrontal cortex and premotor cortex. Thus, to main-
tain high levels of sentence comprehension accuracy, older adults showed additional
frontal recruitment.
Tyler and colleagues (2010) took a slightly different approach, having young and older
participants perform a word-monitoring task in sentences with normal prose, anoma-
lous prose, and random word lists. The anomalous prose sentences were grammatically
correct but did not convey a coherent meaning (e.g., “Stephen didn’t catch himself very
much. Her tooth was driven because he had a weak nail and she couldn’t heat anyone
properly.”). The authors focused on regions showing increased activation for the anoma-
lous prose condition compared to an acoustic baseline. Overall, as in Peelle et al. (2010),
young and older adults relied on a largely similar network of regions (bilateral supe-
rior temporal gyrus and left IFG), although there was some indication that older adults
showed increased activity in the right IFG. Most interesting, however, was a second
analysis in which they looked at whether individual differences in gray matter in left IFG
were related to the activation they saw in right IFG. Indeed, older listeners with reduced
gray matter in left IFG and left middle temporal gyrus showed increased activity in right
hemisphere homologues of these regions, producing a more bilateral pattern of activa-
tion in the older adults relative to young adults. Importantly, these findings suggest that
individual differences in cortical integrity may help explain patterns of compensatory
activity.
A critical distinction to consider is whether processing is interpretive (occurring
when the meaning of the sentence is being calculated) or post-interpretive (occurring
after the meaning has been determined). A classic example of post-interpretive pro-
cessing would be making a meta-linguistic decision about a sentence, such as whether
it is grammatically appropriate. These two stages may also be thought of as “online” (in-
terpretive) versus “offline” (post-interpretive) processing. The online–offline distinc-
tion may be particularly important in the context of experimental paradigms requiring
a response (such as a button press, or recall) from participants, as this adds task-related
cognitive demands that are likely separate from those required for comprehension
(Davis, Zhuang, Wright, & Tyler, 2014). It has been argued that older adults show similar
online processing to that of young adults, but differences in offline processing (Caplan &
Waters, 1999; Waters & Caplan, 2001). Alternatively, others have suggested that domain-
general cognitive systems are required for interpreting challenging sentences, and not
simply for meta-linguistic decisions (Wingfield & Grossman, 2006). Although it seems
likely that both levels of processing are affected during aging, the issue remains a topic
of debate.
In summary, age differences in neural activity during sentence comprehension are
routinely observed, particularly when semantic or syntactic challenges are explicitly
introduced. However, disagreement persists regarding the degree to which these reflect
online versus offline processing demands, and whether they reflect domain-preferential
or domain-general cognitive processes.
Perceptual Challenge and Effortful Listening

Normal aging is associated with changes at nearly every level of the auditory system
(Peelle & Wingfield, 2016). Loss of hearing sensitivity is common in normal aging,
particularly in higher frequencies (Morrell, Gordon-Salant, Pearson, Brant, & Fozard,
1996). Other age-related changes in hearing include broadening frequency filters and
reduced accuracy of temporal processing (Humes, Kewley-Port, Fogerty, & Kinney,
2010; Schneider, 1997). Thus, understanding the processes supporting speech compre-
hension in older adults requires considering the effects of reduced auditory sensitivity.
Subjectively, hearing impairment is frequently associated with “effortful” listening
(Pichora-Fuller et al., 2016). There is growing evidence that listening effort reflects, in
part, additional cognitive processing that is required to make sense of degraded speech
(Rönnberg et al., 2013; Wingfield, Tun, & McCoy, 2005). The role of acoustic challenge
on word perception is seen in cases where adding external noise decreases intelligibility
in proportion to how many phonological neighbors a word has—that is, the more lex-
ical competitors, the more difficult accurate perception becomes (Luce & Pisoni, 1998),
particularly for older adults (Sommers, 1996). Hearing loss similarly affects perception,
with hearing-impaired listeners requiring more acoustic information before identifying
a word (Lash et al., 2013).
Some of the most compelling behavioral evidence for the cognitive consequences
of acoustic challenge comes from episodic memory paradigms in which listeners are
asked to recall lists of unrelated words or digits, some of which are presented in noise.
In these situations, memory is worse not only for the item presented in noise, but also
for previous items that were not degraded (Piquado, Cousins, Wingfield, & Miller,
2010; Rabbitt, 1968). The disruption of memory encoding is consistent with increased
working-memory resources being needed to comprehend noisy speech (Cousins, Dar,
Wingfield, & Miller, 2014; Miller & Wingfield, 2010). Memory effects for spoken lan-
guage are also seen in participants with age-related hearing loss (McCoy et al., 2005;
Rabbitt, 1991), although depending on the specific task, individual differences in

hearing ability may not correlate with memory measures (Ward, Rogers, Van Engen, &
Peelle, 2016).
Neuroimaging evidence for effects of perceptual challenge in older adults has been
discussed earlier for words that have been acoustically degraded through background
noise or low-pass filtering (Eckert et al., 2009; Eckert et al., 2008; Vaden et al., 2015).
Additional support is emerging in the context of spoken sentences. Erb and Obleser
(2013) presented spoken sentences to young and older adults; the sentences were normal
or acoustically degraded using noise vocoding with four channels. Noise vocoding
reduces the spectral detail in the speech, but leaves the temporal amplitude envelope
preserved (Shannon, Zeng, Kamath, Wygonski, & Ekelid, 1995); speech vocoded with
four channels is typically somewhat intelligible. The authors found that older adults
showed increased activation in dorsal anterior cingulate cortex in response to degraded
speech compared to the younger adults. Furthermore, individual differences in activity
in this region were positively correlated with word-report scores, suggesting a func-
tional role for this increased activity.
In summary, there is ample behavioral evidence that perceptual challenges caused by
acoustic degradation or distortion can alter the neural processes required for spoken
language comprehension. Although particularly relevant in the context of age-related
hearing loss, these same principles also likely apply in the case of background noise
(Scott & McGettigan, 2013), competing talkers, or non-native accents (Van Engen
& Peelle, 2014). Thus, again, issues raised in the context of normal aging have broad
implications for models of speech perception.
Language Production in Normal Aging
Syllable Production
Older adults experience more word-finding failures and more filled pauses (for example,
saying “um”) during speech production, along with overall slower rates of production
(Duchin & Mysak, 1987). These production challenges might be caused by difficulty
with lexical retrieval, difficulty accessing the correct speech sounds (phonology) asso-
ciated with a lexical item, or difficulty forming the appropriate motor plan. The fact that
older adults take longer to make sequential speech movements and make more articu-
latory errors (Bilodeau-Mercure et al., 2015) is consistent with at least some age effect on
motor planning or production.
Sörös and colleagues (2011) had young and older adults produce syllabic sounds (the
vowel /a/and the multisyllabic token /pa-ta-ka) in an fMRI study. They found signif-
icantly lower activity in older adults in bilateral temporal cortex (near primary and
secondary auditory cortex), but significantly more activity in older adults in bilateral
middle temporal gyrus and large portions of bilateral frontal cortex. Tremblay and
Deschamps (2016) investigated the relationship of brain structure for young and older
adults producing sequences of syllables that were simple (e.g., /pa-pa-pa-pa-pa-pa/)
or complex (e.g., /pa-ta-ka-pa-ta-ka/). Behaviorally, older adults took longer, and were
less accurate, in their production compared to young adults. Individual differences in
speech production behavior correlated with regions of temporal and frontal cortex
that also showed age-related decreases in cortical thickness in the same participants.
Together these findings suggest a link between age-related neuroanatomical change and
concomitant alterations in speech production.
Word Retrieval and Picture Naming

There is ample evidence that age-related differences in speech production go beyond
motor planning. Speech production is frequently assessed in the context of object
naming, which beyond visual object recognition requires retrieving the name of an ob-
ject (lexical information) and its corresponding phonological form prior to producing
it (Indefrey & Levelt, 2004). During picture-naming tasks, older adults often have more
difficulty producing correct names for pictures than young adults, particularly for low-
frequency items (Nicholas, Obler, Albert, & Goodglass, 1985; Rogalski, Peelle, & Reilly,
2011). Weirenga and colleagues (2008) used fMRI to examine neural activity during pic-
ture naming in young and older adults. Although there were no age differences in accu-
racy, older adults appeared to show increased activity in several regions of both temporal
and frontal cortex, including right IFG and bilateral insula. Individual differences in ac-
tivity in several of these regions showed positive correlations with accuracy.
Perhaps the most compelling evidence for age-related difficulties in word retrieval
is found in the tip-of-the-tongue (TOT) experience, in which a speaker is certain they
know a word but is unable to produce it (for example, in response to the question, “what
is the name of the strait between Alaska and Siberia?”). Older adults experience TOT
states significantly more often than young adults, particularly for proper nouns (Burke,
MacKay, Worthley, & Wade, 1991). Phonological (homophone) priming improves older
adults’ retrieval of proper names (Burke, Locantore, Austin, & Chase, 2004), suggesting
that difficulty retrieving phonological information may underlie these TOT states; that
is, semantic and/or lexical information has been accessed (resulting in a strong feeling
of knowing), but phonological retrieval has not yet occurred (preventing the word from
being produced).
In an effort to identify the neural systems involved in TOT states, Shafto and
colleagues (2007) examined gray matter density and TOT occurrence in older adults,
and found that increased TOTs were associated with reduced gray matter in the left in-
sula and anterior cingulate cortex. These findings were anatomically dissociable from
regions of gray matter that correlated with scores on an executive processing task
(Ravens progressive matrices), suggesting that age-related changes in TOTs are not
simply reflecting an overall cognitive decline. Converging evidence from fMRI on a
similar experiment also showed increased left anterior insula activity for TOT states
(Shafto, Stamatakis, Tam, & Tyler, 2010). (It is interesting to consider these structural
findings in the anterior insula in the context of the error-related activity often reported
in the cingulo-opercular network—it may be that TOTs are related to general states of
attention and error-monitoring seen in the cases of perceptual errors.)
Aging is thus associated with differences in picture naming and word production.
Although some of these differences may be due to changes in articulatory fluency or
control, there is also evidence suggesting age-related changes to lexical or phonological
retrieval stages of processing.
Sentence and Discourse Production

In comparison to single-word production and naming, there has been little work on the
cognitive neuroscience of sentence or discourse production in healthy aging. However,
it is worth briefly noting that a number of striking behavioral age differences have been
observed. A particularly interesting way of assessing sentence production is to examine
diaries or other writing samples produced over time. In these cases, the complexity
of written sentences has been found to decrease over time (Kemper, 1987; Kemper,
Greiner, et al., 2001). This includes decreases in syntactic complexity and idea density.
Provocatively, longitudinal studies have found that the linguistic complexity of writing
samples in early life has been found to predict development of Alzheimer’s disease in
older age (Snowdon et al., 1996). Measures of speech production in the context of sto-
rytelling can also help distinguish between patients with a variety of neurodegenerative
diseases (Ash et al., 2006; Ash et al., 2014). Thus, connected speech offers a number of
additional insights into language production that may not be available with word or pic-
ture naming.
Aging and the Effects

of Multilingualism on Language
Processing and Cognitive Function
Recent years have seen increasing interest in the effects of bilingualism (or more gen-
erally, multilingualism) on language and cognitive processing (see, in this volume,
Green & Kroll, Chapter 11, and Paz-Alonso, Oliver, Quiñones, & Carreiras, Chapter 24).
Multilingualism presents a number of complex cognitive challenges. For example,
storing multiple lexicons may provide extra linguistic support, but also requires
choosing between a greater number of words referring to an intended concept. Code
switching—that is, changing from one language to another in production (or com-
prehension)—also requires cognitive control and flexibility (Gollan & Ferreira, 2009;
Olson, 2017). Because of these added demands, multilingualism is frequently associated
with inhibition and task-switching (Abutalebi & Green, 2008; De Baene, Duyck, Brass,
& Carreiras, 2015; Green & Abutalebi, 2013). Multilingualism is also associated with a
number of neural changes, including functional processing changes in the caudate
(Crinion et al., 2006), and structural changes in left frontal and parietal cortices (Grogan
et al., 2012; Mechelli et al., 2004; Stein et al., 2012). Although some potential converging
links may be observed between specific brain regions and cognitive processes implicated
in multilingual processing, the correspondence is not always straightforward (Higby,
Kim, & Obler, 2013).
Given age-related changes in cognitive function, it interesting to consider how multi-
lingualism fits into the aging process. In particular, an area of burgeoning interest is the
degree to which bilingualism may lead to improved cognitive processing, especially in
older age (Bak, Nissan, Allerhand, & Deary, 2014; Bialystok, Craik, & Luk, 2012; Kavé,
Eyal, Shorek, & Cohen-Mansfield, 2008). Early behavioral work suggested that bilin-
gualism was associated with altered executive processing (Bialystok, Craik, Klein, &
Viswanathan, 2004; Prior & MacWhinney, 2009), a finding that also has found some
support in brain imaging studies (Gold, Johnson, & Powell, 2013; Gold, Kim, Johnson,
Kryscio, & Smith, 2013; Schweizer, Ware, Fischer, Craik, & Bialystok, 2012). One com-
pelling hypothesis is that years of multilingual processing result in cognitive changes,
building cognitive reserve that can have a protective effect against age-related cognitive
decline (Craik, Bialystok, & Freedman, 2010). That being said, disagreement exists as
to the extent to which bilingualism affects cognitive processing in older adults (Antón,
García, Carreiras, & Duñabeitia, 2016), and further data are likely needed to fully resolve
the issue.
Beyond Individual Brain

Regions: Network Balance
during Language Processing
Owing in part to the prominent role that language processing has played in the history
of localizing brain function, it can be tempting to think about age-related changes in
neural activity in terms of discrete regions being activated differently in older adults
compared to young adults. Although there is some truth to be found from this per-
spective, a more accurate framework is likely one in which individual differences in
sensory and cognitive ability lead to altered dynamics of interacting brain networks
(Shafto & Tyler, 2014). From this network-centered perspective, levels of activity in any
given network arise naturally out of the balance between processing efficiency and task
requirements illustrated in Figure 12.1. Individual differences in neurobiology, and con-
comitant sensory and cognitive ability, thus assume the primary explanatory position,
rather than age per se. Operationally, a network-based perspective might be supported
by analyses incorporating structural brain measures, functional brain measures, and be-
havior to investigate systematic variability across participants (Meunier, Stamatakis, &
Tyler, 2014). Recent studies that have taken network-based approaches to aging suggest
age-related changes in network connectivity (Tsvetanov et al., 2016) that underlie older
adults success with at least some language tasks (Campbell et al., 2016).
Conclusions
Adult aging is an ideal framework within which to study how individual differences in
cognitive and perceptual factors influence language processing. Evidence from func-
tional brain imaging suggests that age-related changes in the neural underpinnings of
language processing are present at nearly every level of linguistic processing—syllables,
words, sentences, and discourse—in both production and comprehension. Thus, al-
though language processing is generally well preserved in older adulthood, this be-
havioral success is accomplished using a different balance of cognitive and neural
processing in older adults. Emerging work specifically looking at individual differences
in older adults’ brain activity is an exciting new direction. It may well be that improved
measures of neural integrity, sensory acuity, and cognitive ability ultimately prove more
useful in predicting language processing than chronological age.
Acknowledgments
This work was supported by grants R01DC14281 and R01AG038490 from the US National
Institutes of Health.
References
Abutalebi, J., & Green, D. W. (2008). Control mechanisms in bilingual language produc-
tion: Neural evidence from language switching studies. Language and Cognitive Processes,
23, 557–582.
Antón, E., García, Y. F., Carreiras, M., & Duñabeitia, J. A. (2016). Does bilingualism shape in-
hibitory control in the elderly? Journal of Memory and Language, 90, 147–160. doi: 10.1016/
j.jml.2016.04.007
Ash, S., Menaged, A., Olm, C., McMillan, C. T., Boller, A., Irwin, D. J., . . . Grossman, M. (2014).
Narrative discourse deficits in amyotrophic lateral sclerosis. Neurology, 83, 520–528.
Ash, S., Moore, P., Antani, S., McCawley, G., Work, M., & Grossman, M. (2006). Trying to
tell a tale: Discourse impairments in progressive aphasia and frontotemporal dementia.
Neurology, 66, 1405–1413.
Bak, T. H., Nissan, J. J., Allerhand, M. M., & Deary, I. J. (2014). Does bilingualism influence cog-
nitive aging? Annals of Neurology, 75, 959–963. doi: 10.1002/ana.24158
Bialystok, E., Craik, F. I. M., Klein, R., & Viswanathan, M. (2004). Bilingualism, aging, and
cognitive control: Evidence from the Simon Task. Psychology and Aging, 19, 290–303.
doi: 10.1037/0882-7974.19.2.290
Bialystok, E., Craik, F. I. M., & Luk, G. (2012). Bilingualism: Consequences for mind and brain.
Trends in Cognitive Sciences, 16, 240–250. doi: 10.1016/j.tics.2012.03.001
Bilodeau-Mercure, M., Kirouac, V., Langlois, N., Ouellet, C., Gasse, I., & Tremblay, P. (2015).
Movement sequencing in normal aging: Speech, oro-facial, and finger movements. Age,
37, 78.
Bilodeau-Mercure, M., Lortie, C. L., Sato, M., Guitton, M., & Tremblay, P. (2015). The neu-
robiology of speech perception decline in aging. Brain Structure and Function, 220,
979–997.
Burke, D. M., Locantore, J., Austin, A., & Chase, B. (2004). Cherry pit primes Brad
Pitt: Homophone priming effects on young and older adults’ production of proper names.
Psychological Science, 15, 164–170.
Burke, D. M., MacKay, D. G., Worthley, J. S., & Wade, E. (1991). On the tip of the tongue: What
causes word finding failures in young and older adults? Journal of Memory and Language, 30,
542–579.
Cabeza, R. (2002). Hemispheric asymmetry reduction in older adults: The HAROLD model.
Psychology and Aging, 17(1), 85–100.
Cabeza, R., Grady, C. L., Nyberg, L., McIntosh, A. R., Tulving, E., Kapur, S., . . . Craik, F. I. M.
(1997). Age-related differences in neural activity during memory encoding and retrieval: A
positron emission tomography study. Journal of Neuroscience, 17, 391–400.
Campbell, K. L., Samu, D., Davis, S. W., Geerligs, L., Mustafa, A., Tyler, L. K., & Cam-
CAN. (2016). Robust resilience of the frontotemporal syntax system to aging. Journal of
Caplan, D., & Waters, G. S. (1999). Verbal working memory and sentence comprehension.
Behavioral and Brain Sciences, 22(1), 77–126.
Cousins, K. A. Q., Dar, H., Wingfield, A., & Miller, P. (2014). Acoustic masking disrupts time-
dependent mechanisms of memory encoding in word-list recall. Memory and Cognition, 42,
622–638.
Craik, F. I. M., Bialystok, E., & Freedman, M. (2010). Delaying the onset of Alzheimer di-
sease: Bilingualism as a form of cognitive reserve. Neurology, 75, 1726–1729. doi: 10.1212/
WNL.0b013e3181fc2a1c
Crinion, J. T., Turner, R., Grogan, A., Hanakawa, T., Uppeney, U., Devlin, J. T., . . . Price, C.
J. (2006). Language control in the bilingual brain. Science, 312, 1537–1540. doi: 10.1126/
science.1127761
D’Esposito, M., Zarahn, E., Aguirre, G. K., & Rypma, B. (1999). The effect of normal aging on
the coupling of neural activity to the bold hemodynamic response. NeuroImage, 10(1),  6–14.
Davis, S. W., Zhuang, J., Wright, P., & Tyler, L. K. (2014). Age-related sensitivity to task-related
modulation of language-processing networks. Neuropsychologia, 63, 107–115.
De Baene, W., Duyck, W., Brass, M., & Carreiras, M. (2015). Brain circuit for cognitive control
is shared by task and language switching. Journal of Cognitive Neuroscience, 27, 1752–1765.
doi: 10.1162/jocn_a_00817
Dosenbach, N. U. F., Fair, D. A., Cohen, A. L., Schlaggar, B. L., & Petersen, S. E. (2008). A dual-
networks architecture of top-down control. Trends in Cognitive Sciences, 12, 99–105.
Dubno, J. R., Ahlstrom, J. B., & Horwitz, A. R. (2000). Use of context by young and aged adults
with normal hearing. Journal of the Acoustical Society of America, 107, 538–546.
Dubno, J. R., Dirks, D. D., & Morgan, D. E. (1984). Effects of age and mild hearing loss on
speech recognition in noise. Journal of the Acoustical Society of America, 76(1),  87–96.
Duchin, S. W., & Mysak, E. D. (1987). Disfluency and rate characteristics of young adult,
middle-aged, and older males. Journal of Communication Disorders, 20, 245–257.
Duncan, J. (2010). The multiple-demand (MD) system of the primate brain: Mental programs
for intelligent behaviour. Trends in Cognitive Sciences, 14, 172–179.
Eckert, M. A., Menon, V., Walczak, A., Ahlstrom, J., Denslow, S., Horwitz, A., & Dubno, J. R.
(2009). At the heart of the ventral attention system: The right anterior insula. Human Brain
Mapping, 30, 2530–2541.
Eckert, M. A., Walczak, A., Ahlstrom, J., Denslow, S., Horwitz, A., & Dubno, J. R. (2008).
Age-related effects on word recognition: Reliance on cognitive control systems with
structural declines in speech-responsive cortex. Journal of the Association for Research in
Otolaryngology, 9, 252–259.
Erb, J., & Obleser, J. (2013). Upregulation of cognitive control networks in older adults’ speech
comprehension. Frontiers in Systems Neuroscience, 7, 116.
Federmeier, K. D., & Kutas, M. (2005). Aging in context: Age-related changes in context use
during language comprehension. Psychophysiology, 42, 133–141.
Federmeier, K. D., Kutas, M., & Schul, R. (2010). Age-related and individual differences in the
use of prediction during language comprehension. Brain and Language, 115, 149–161.
Fjell, A. M., Walhovd, K. B., Fennema-Notestine, C., McEvoy, L. K., Hagler, D. J., Holland,
D., . . . Dale, A. M. (2009). One-year brain atrophy evident in healthy aging. Journal of
Fodor, J. A. (1983). The modularity of mind. Cambridge, MA: MIT Press.
Gagnepain, P., Henson, R. N., & Davis, M. H. (2012). Temporal predictive codes for spoken
words in auditory cortex. Current Biology, 22, 615–621.
Gold, B. T., Johnson, N. F., & Powell, D. K. (2013). Lifelong bilingualism contributes to cog-
nitive reserve against white matter integrity declines in aging. Neuropsychologia, 51, 2841–
2846. doi: 10.1016/j.neuropsychologia.2013.09.037
Gold, B. T., Kim, C., Johnson, N. F., Kryscio, R. J., & Smith, C. D. (2013). Lifelong bilingualism
maintains neural efficiency for cognitive control in aging. Journal of Neuroscience, 33, 387–
396. doi: 10.1523/JNEUROSCI.3837-12.2013
Gollan, T. H., & Ferreira, V. S. (2009). Should I stay or should I switch? A cost-benefit anal-
ysis of voluntary language switching in young and aging bilinguals. Journal of Experimental
Psychology: Learning, Memory, and Cognition, 35, 640–665. doi: 10.1037/a0014981
Good, C. D., Johnsrude, I. S., Ashburner, J., Henson, R. N. A., Friston, K. J., & Frackowiak, R.
S. J. (2001). A voxel-based morphometric study of ageing in 465 normal adult human brains.
NeuroImage, 14,  21–36.
Grady, C. L. (2000). Functional brain imaging and age-related changes in cognition. Biological
Psychology, 54, 259–281.
Green, D. W., & Abutalebi, J. (2013). Language control in bilinguals: The adaptive control hy-
pothesis. Journal of Cognitive Psychology, 25(5), 515–530. doi: 10.1080/20445911.2013.796377
Grogan, A., Parker Jones, O., Ali, N., Crinion, J., Orabona, S., Mechias, M. L., . . . Price,
C. J. (2012). Structural correlates for lexical efficiency and number of languages
in non- native speakers of English. Neuropsychologia, 50, 1347– 1352. doi: 10.1016/
j.neuropsychologia.2012.02.019
Hasher, L., Stoltzfus, E. R., Zacks, R. T., & Rypma, B. (1991). Age and inhibition. Journal of
Experimental Psychology: Learning, Memory, and Cognition, 17(1), 163–169.
Hedden, T., & Gabrieli, J. D. E. (2004). Insights into the ageing mind: A view from cognitive
neuroscience. Nature Reviews Neuroscience, 5,  87–96.
Higby, E., Kim, J., & Obler, L. K. (2013). Multilingualism and the brain. Annual Review of
Applied Linguistics, 33, 68–101. doi: 10.1017/S0267190513000081
Humes, L. E., Kewley-Port, D., Fogerty, D., & Kinney, D. (2010). Measures of hearing threshold
and temporal processing across the adult lifespan. Hearing Research, 264,  30–40.
Humes, L. E., Kidd, G. R., & Lentz, J. J. (2013). Auditory and cognitive factors underlying indi-
vidual differences in aided speech-understanding among older adults. Frontiers in Systems
components. Cognition, 92, 101–144. doi: 10.1016/j.cognition.2002.06.001
Jones, D. K., Knösche, T. R., & Turner, R. (2013). White matter integrity, fiber count, and other
fallacies: The do’s and don’ts of diffusion MRI. NeuroImage, 73, 239–254.
Kavé, G., Eyal, N., Shorek, A., & Cohen-Mansfield, J. (2008). Multilingualism and cognitive
state in the oldest old. Psychology and Aging, 23, 70–78. doi: 10.1037/0882-7974.23.1.70
Kemper, S. (1987). Life span changes in syntactic complexity. Journal of Gerontology, 42,
323–328.
Kemper, S. (1992). Language and aging. In F. I. M. Craik & T. A. Salthouse (Eds.), The handbook
of aging and cognition (pp. 213–270). Hillsdale, NJ: Lawrence Erlbaum Associates.
Kemper, S., Greiner, L. H., Marquis, J. G., Prenovost, K., & Mitzner, T. L. (2001). Language de-
cline across the life span: Findings from the Nun Study. Psychology and Aging, 16(2), 227–239.
Kemper, S., Marquis, J., & Thompson, M. (2001). Longitudinal change in language produc-
tion: Effects of aging and dementia on grammatical complexity and propositional content.
Psychology and Aging, 16(4), 600–614.
Kemtes, K. A., & Kemper, S. (1997). Younger and older adults’ on-line processing of syntacti-
cally ambiguous sentences. Psychology and Aging, 12(2), 362–371.
Kuchinsky, S. E., Vaden, K. I., Jr., Keren, N. I., Harris, K. C., Ahlstrom, J. B., Dubno, J. R., &
Eckert, M. A. (2012). Word intelligibility and age predict visual cortex activity during word
listening. Cerebral Cortex, 22, 1360–1371.
Kutas, M., & Hillyard, S. A. (1984). Brain potentials during reading reflect word expectancy
and semantic association. Nature, 307, 161–163.
Lash, A., Rogers, C. S., Zoller, A., & Wingfield, A. (2013). Expectation and entropy in spoken
word recognition: Effects of age and hearing acuity. Experimental Aging Research, 39,
235–253.
Luce, P. A., & Pisoni, D. B. (1998). Recognizing spoken words: The neighborhood activation
model. Ear and Hearing, 19,  1–36.
Marslen-Wilson, W. D. (1987). Functional parallelism in spoken word recognition. Cognition,
25, 71–102.
Marslen-Wilson, W. D., & Tyler, L. K. (1980). The temporal structure of spoken language pro-
cessing. Cognition, 8,  1–7 1.
McCoy, S. L., Tun, P. A., Cox, L. C., Colangelo, M., Stewart, R., & Wingfield, A. (2005).
Hearing loss and perceptual effort: Downstream effects on older adults’ memory for speech.
Quarterly Journal of Experimental Psychology, 58(1),  22–33.
Mechelli, A., Crinion, J. T., Noppeney, U., O’Doherty, J., Ashburner, J., Frackowiak, R. S., &
Price, C. J. (2004). Structural plasticity in the bilingual brain. Nature, 431, 757.
Meunier, D., Stamatakis, E. A., & Tyler, L. K. (2014). Age-related functional reorganization,
structural changes, and preserved cognition. Neurobiology of Aging, 35,  42–54.
Miller, P., & Wingfield, A. (2010). Distinct effects of perceptual quality on auditory word recog-
nition, memory formation and recall in a neural model of sequential memory. Frontiers in
Systems Neuroscience, 4, 4. doi: 10.3389/fnsys.2010.00014
Mitchell, K. J., Johnson, M. K., Raye, C. L., Mather, M., & D’Esposito, M. (2000). Aging and re-
flective processes of working memory: Binding and test load deficits. Psychology and Aging, 15,
527–541.
Morrell, C. H., Gordon-Salant, S., Pearson, J. D., Brant, L. J., & Fozard, J. L. (1996). Age-and
gender-specific reference ranges for hearing level and longitudinal changes in hearing level.
Journal of the Acoustical Society of America, 100(4, Pt 1), 1949–1967.
Neta, M., Miezin, F. M., Nelson, S. M., Dubis, J. W., Dosenbach, N. U. F., Schlaggar, B. L., &
Petersen, S. E. (2015). Spatial and temporal characteristics of error-related activity in the
human brain. Journal of Neuroscience, 35, 253–266.
Nicholas, M., Obler, L., Albert, M., & Goodglass, H. (1985). Lexical retrieval in healthy aging.
Cortex, 21, 595–606.
Norris, D., & McQueen, J. M. (2008). Shortlist B: A Bayesian model of continuous speech rec-
ognition. Psychological Review, 115, 357–395.
Olson, D. J. (2017). Bilingual language switching costs in auditory comprehension. Language,
Cognition and Neuroscience, 32, 494–513. doi: 10.1080/23273798.2016.1250927
Park, D. C., Polk, T. A., Park, R., Minear, M., Savage, A., & Smith, M. R. (2004). Aging reduces
neural specialization in ventral visual cortex. Proceedings of the National Academy of
Sciences, 101(35), 13091–13095.
Peelle, J. E. (2012). The hemispheric lateralization of speech processing depends on what
“speech” is: A hierarchical perspective. Frontiers in Human Neuroscience, 6, 309. doi: 10.3389/
fnhum.2012.00309
Peelle, J. E., Cusack, R., & Henson, R. N. A. (2012). Adjusting for global effects in voxel-based
morphometry: Gray matter decline in normal aging. NeuroImage, 60, 1503–1516.
Peelle, J. E., Johnsrude, I. S., & Davis, M. H. (2010). Hierarchical processing for speech in
human auditory cortex and beyond. Frontiers in Human Neuroscience, 4, 51. doi: 10.3389/
fnhum.2010.00051
Peelle, J. E., Troiani, V., Wingfield, A., & Grossman, M. (2010). Neural processing during older
adults’ comprehension of spoken sentences: Age differences in resource allocation and con-
nectivity. Cerebral Cortex, 20, 773–782.
Peelle, J. E., & Wingfield, A. (2016). The neural consequences of age-related hearing loss. Trends
in Neurosciences, 39, 486–497. doi: 10.1016/j.tins.2016.05.001
Pichora-Fuller, M. K., Kramer, S. E., Eckert, M. A., Edwards, B., Hornsby, B. W. Y., Humes, L.
E., . . . Wingfield, A. (2016). Hearing impairment and cognitive energy: The framework for
understanding effortful listening (FUEL). Ear and Hearing, 37, 5S–27S.
Pichora-Fuller, M. K., Schneider, B. A., & Daneman, M. (1995). How young and old adults listen
to and remember speech in noise. Journal of the Acoustical Society of America, 97(1), 593–608.
Piquado, T., Cousins, K. A. Q., Wingfield, A., & Miller, P. (2010). Effects of degraded sensory
input on memory for speech: Behavioral data and a test of biologically constrained compu-
tational models. Brain Research Bulletin, 1365,  48–65.
Power, J. D., Barnes, K. A., Snyder, A. Z., Schlaggar, B. L., & Petersen, S. E. (2012). Spurious but
systematic correlations in functional connectivity MRI networks arise from subject motion.
NeuroImage, 59, 2142–2154.
Power, J. D., & Petersen, S. E. (2013). Control-related systems in the human brain. Current
Opinion in Neurobiology, 23, 223–228.
Prior, A., & MacWhinney, B. (2009). A bilingual advantage in task switching. Bilingualism:
Language and Cognition, 13, 253–262. doi: 10.1017/S1366728909990526
Rabbitt, P. M. A. (1968). Channel capacity, intelligibility and immediate memory. Quarterly
Journal of Experimental Psychology, 20, 241–248.
Rabbitt, P. M. A. (1991). Mild hearing loss can cause apparent memory failures which increase
with age and reduce with IQ. Acta Otolaryngolica, 476, 167–176.
Rauschecker, J. P., & Scott, S. K. (2009). Maps and streams in the auditory cortex: Nonhuman
primates illuminate human speech processing. Nature Neuroscience, 12, 718–724.
Raz, N., Gunning, F. M., Head, D., Dupuis, J. H., McQuain, J., Briggs, S. D., . . . Acker, J. D.
(1997). Selective aging of the human cerebral cortex observed in vivo: Differential vulnera-
bility of the prefrontal gray matter. Cerebral Cortex, 7, 268–282.
Raz, N., Gunning-Dixon, F. M., Head, D., Dupuis, J. H., & Acker, J. D. (1998). Neuroanatomical
correlates of cognitive aging: Evidence from structural magnetic resonance imaging.
Raz, N., Lindenberger, U., Rodrigue, K. M., Kennedy, K. M., Head, D., Williamson, A.,
. . . Acker, J. D. (2005). Regional brain changes in aging healthy adults: General trends, indi-
vidual differences and modifiers. Cerebral Cortex, 15, 1676–1689.
Reuter-Lorenz, P. A., & Lustig, C. (2005). Brain aging: Reorganizing discoveries about the
aging mind. Current Opinion in Neurobiology, 15, 245–251.
Reuter-Lorenz, P. A., & Park, D. C. (2014). How does it STAC up? Revisiting the scaffolding
theory of aging and cognition. Neuropsychology Review, 24, 355–370.
Rodd, J. M., Johnsrude, I. S., & Davis, M. H. (2012). Dissociating frontotemporal contributions
to semantic ambiguity resolution in spoken sentences. Cerebral Cortex, 22, 1761–1773.
Rogalski, Y., Peelle, J. E., & Reilly, J. (2011). Effects of perceptual and contextual enrichment on
visual confrontation naming in aging. Journal of Speech, Language, and Hearing Research, 54,
1349–1360.
Rönnberg, J., Lunner, T., Zekveld, A., Sörqvist, P., Danielsson, H., Lyxell, B., . . . Rudner, M.
(2013). The ease of language understanding (ELU) model: Theoretical, empirical, and clin-
ical advances. Frontiers in Systems Neuroscience, 7, 31. doi: 10.3389/fnsys.2013.00031
Salat, D. H., Buckner, R. L., Snyder, A. Z., Greve, D. N., Desikan, R. S. R., Busa, E., . . . Fischl, B.
(2004). Thinning of the cerebral cortex in aging. Cerebral Cortex, 14, 721–730.
Salat, D. H., Lee, S. Y., van der Kouwe, A. J., Greve, D. N., Fischl, B., & Rosas, H. D. (2009). Age-
associated alterations in cortical gray and white matter signal intensity and gray to white
matter contrast. NeuroImage, 48,  21–28.
Salat, D. H., Tuch, D. S., Greve, D. N., van der Kouwe, A. J. W., Hevelone, N. D., Zaleta, A.
K., . . . Dale, A. M. (2005). Age-related alterations in white matter microstructure measured
by diffusion tensor imaging. Neurobiology of Aging, 26, 1215–1227.
Salthouse, T. A. (1996). The processing-speed theory of adult age differences in cognition.
Psychological Review, 103(3), 403–428.
Schneider, B. A. (1997). Psychoacoustics and aging: Implications for everyday listening. Journal
of Speech-Language Pathology and Audiology, 21(2), 111–124.
Schweizer, T. A., Ware, J., Fischer, C. E., Craik, F. I. M., & Bialystok, E. (2012). Bilingualism
as a contributor to cognitive reserve: Evidence from brain atrophy in Alzheimer’s disease.
Cortex, 48, 991–996. doi: 10.1016/j.cortex.2011.04.009
Scott, S. K., & McGettigan, C. (2013). The neural processing of masked speech. Hearing
Research, 303,  58–66.
Shafto, M. A., Burke, D. M., Stamatakis, E. A., Tam, P. P., & Tyler, L. K. (2007). On the tip-of-
the-tongue: Neural correlates of increased word-finding failures in normal aging. Journal of
Shafto, M. A., Randall, B., Stamatakis, E. A., Wright, P., & Tyler, L. K. (2012). Age-related neural
reorganization during spoken word recognition: The interaction of form and meaning.
Shafto, M. A., Stamatakis, E. A., Tam, P. P., & Tyler, L. K. (2010). Word retrieval failures in old
age: The relationship between structure and function. Journal of Cognitive Neuroscience, 22,
1530–1540.
Shafto, M. A., & Tyler, L. K. (2014). Language in the aging brain: The network dynamics of de-
cline and preservation. Science, 346, 583–587.
Shannon, R. V., Zeng, F.-G., Kamath, V., Wygonski, J., & Ekelid, M. (1995). Speech recognition
with primarily temporal cues. Science, 270, 303–304.
Snowdon, D. A., Kemper, S. J., Mortimer, J. A., Greiner, L. H., Wekstein, D. R., & Markesbery, W.
R. (1996). Linguistic ability in early life and cognitive function and Alzheimer’s disease in late
life: Findings from the Nun Study. Journal of the American Medical Association, 275, 528–532.
Sommers, M. S. (1996). The structural organization of the mental lexicon and its contribution
to age-related declines in spoken-word recognition. Psychology and Aging, 11(2), 333–341.
Sommers, M. S., & Danielson, S. M. (1999). Inhibitory processes and spoken word recogni-
tion in young and older adults: The interaction of lexical competition and semantic context.
Psychology and Aging, 14, 458–472.
Sörös, P., Bose, A., Sokoloff, L. G., Graham, S. J., & Stuss, D. T. (2011). Age-related changes in the
functional neuroanatomy of overt speech production. Neurobiology of Aging, 32, 1505–1513.
Stein, M., Federspiel, A., Koenig, T., Wirth, M., Strik, W., Wiest, R., . . . Dierks, T. (2012).
Structural plasticity in the language system related to increased second language profi-
ciency. Cortex, 48, 458–465. doi: 10.1016/j.cortex.2010.10.007
Stokes, M. G., Kusunoki, M., Sigala, N., Nili, H., Gaffan, D., & Duncan, J. (2013). Dynamic
coding for cognitive control in prefrontal cortex. Neuron, 78, 364–375.
Tremblay, P., & Deschamps, I. (2016). Structural brain aging and speech production: A surface-
based brain morphometry study. Brain Structure and Function, 221, 3275–3299.
Tsvetanov, K. A., Henson, R. N. A., Tyler, L. K., Davis, S. W., Shafto, M. A., Taylor, J. R., . . . Rowe,
J. B. (2015). The effect of ageing on fMRI: Correction for the confounding effects of vascular re-
activity evaluated by joint fMRI and MEG in 335 adults. Human Brain Mapping, 36, 2248–2269.
Tsvetanov, K. A., Henson, R. N. A., Tyler, L. K., Razi, A., Geerligs, L., Ham, T. E., . . . Cam-CAN.
(2016). Extrinsic and intrinsic brain network connectivity maintains cognition across the
lifespan despite accelerated decay of regional brain activation. Journal of Neuroscience, 36,
3115–3126. doi: 10.1523/JNEUROSCI.2733-15.2016
Tyler, L. K., Shafto, M. A., Randall, B., Wright, P., Marslen-Wilson, W. D., & Stamatakis, E. A.
(2010). Preserving syntactic processing across the adult life span: The modulation of the
frontotemporal language system in the context of age-related atrophy. Cerebral Cortex, 20,
352–364.
Vaden, K. I., Jr., Kuchinsky, S. E., Ahlstrom, J. B., Dubno, J. R., & Eckert, M. A. (2015). Cortical
activity predicts which older adults recognize speech in noise and when. Journal of
Vaden, K. I., Jr., Kuchinsky, S. E., Cute, S. L., Ahlstrom, J. B., Dubno, J. R., & Eckert, M. A. (2013).
The cingulo-opercular network provides word-recognition benefit. Journal of Neuroscience,
33, 18979–18986.
Van Engen, K. J., & Peelle, J. E. (2014). Listening effort and accented speech. Frontiers in Human
Neuroscience, 8, 577. doi: 10.3389/fnhum.2014.00577
Verhaeghen, P. (2003). Aging and vocabulary score: A meta-analysis. Psychology and Aging,
18(2), 332–339.
Ward, C. M., Rogers, C. S., Van Engen, K. J., & Peelle, J. E. (2016). Effects of age, acoustic
challenge, and verbal working memory on recall of narrative speech. Experimental Aging
Research, 42, 126–144.
Waters, G. S., & Caplan, D. (2001). Age, working memory, and on-line syntactic processing in
sentence comprehension. Psychology and Aging, 16(1), 128–144.
Whiting, W. L., Madden, D. J., Langley, L. K., Denny, L. L., Turkington, T. G., Provenzale, J.
M., . . . Coleman, R. E. (2003). Lexical and sublexical components of age-related changes
in neural activation during visual word identification. Journal of Cognitive Neuroscience, 15,
475–487.
Wierenga, C. E., Benjamin, M., Gopinath, K., Perlstein, W. M., Leonard, C. M., Gonzalez Rothi,
L. J., . . . Crosson, B. (2008). Age-related changes in word retrieval: Role of bilateral frontal
and subcortical networks. Neurobiology of Aging, 29, 436–451.
Wingfield, A., Aberdeen, J. S., & Stine, E. A. (1991). Word onset gating and linguistic context
in spoken word recognition by young and elderly adults. Journals of Gerontology, 46(3),
127–129.
Wingfield, A., & Grossman, M. (2006). Language and the aging brain: Patterns of neural com-
pensation revealed by functional brain imaging. Journal of Neurophysiology, 96, 2830–2839.
Wingfield, A., Peelle, J. E., & Grossman, M. (2003). Speech rate and syntactic complexity
as multiplicative factors in speech comprehension by young and older adults. Aging,
Neuropsychology, and Cognition, 10(4), 310–322.
Wingfield, A., & Stine-Morrow, E. A. L. (2000). Language and speech. In F. I. M. Craik & T.
A. Salthouse (Eds.), The handbook of aging and cognition (2nd ed) (pp. 359–416). Mahwah,
NJ: Lawrence Erlbaum Associates.
Wingfield, A., Tun, P. A., & McCoy, S. L. (2005). Hearing loss in older adulthood: What it is and
how it interacts with cognitive performance. Current Directions in Psychological Science, 14,
144–148.
Wlotko, E. W., Federmeier, K. D., & Kutas, M. (2012). To predict or not to predict: Age-related
differences in the use of sentential context. Psychology and Aging, 27, 975–988.
Woolgar, A., Hampshire, A., Thompson, R., & Duncan, J. (2011). Adaptive coding of
task-relevant information in human frontoparietal cortex. Journal of Neuroscience, 31,
14592–14599.
Zhuang, J., Tyler, L. K., Randall, B., Stamatakis, E. A., & Marslen-Wilson, W. D. (2014).
Optimially efficient neural systems for processing spoken language. Cerebral Cortex, 24,
908–918.
Chapter 13
L anguage Pl ast i c i t y
in Epile psy
Jeffrey R. Cole and Marla J. Hamberger
Introduction
Epilepsy provides a rich context in which to study brain organization of language, and
the plasticity of language organization in response to electrophysiological and struc-
tural disturbances. Clinically, issues regarding the localization of language areas often
become central when pharmacologically refractory patients are considered for resective
surgical treatment, as surgery can be an option for patients whose seizures arise from
a focal brain region. Surgical resection for seizure control involves removal of the ep-
ileptogenic cortex that generates seizure activity, and thus, one of the main challenges
of resective surgery is to remove a sufficient extent of epileptogenic brain tissue, yet
without significant compromise to function. Depending on the location of the epilep-
togenic cortex and the proposed surgery, language is frequently at risk, and therefore
various pre-surgical procedures are performed to assess and reduce this risk of postop-
erative language decline, mainly by identifying the location of essential language areas.
It is these procedures that provide a unique and valuable opportunity to study language
plasticity in epilepsy.
In the context of epilepsy surgery and risk to language function, there are two types
of brain plasticity mechanisms to consider. The type most often associated with epi-
lepsy is the reorganization of language that is attributed to the ongoing functional dis-
ruption or slowly progressive structural disturbances from chronic epileptic activity
(Janszky, Mertens, Janszky, Ebner, & Woermann, 2006). This abnormal electrophysio-
logical activity could shift language from the left to the right hemisphere or might re-
route language pathways during development from traditional to nontraditional sites
intra-hemispherically within the dominant left hemisphere (Duchowny et al., 1996;
Liegeois et al., 2004). As the incidence of atypical language organization is significantly
higher among people with epilepsy relative to the healthy population, the surgical
318 Jeffrey R. Cole and Marla J. Hamberger
team cannot simply rely on typical language landmarks (Berl et al., 2014; Helmstaedter,
Kurthen, Linke, & Elger, 1997; Janszky, Jokeit, et al., 2003; Risse, Gates, & Fangman,
1997). Rather, disruptive techniques that capitalize on the “lesion method,” such as
Wada testing and electrocortical stimulation mapping, are frequently employed to de-
termine hemispheric laterality and intra-hemispheric localization essential language
areas. Activation techniques such as functional magnetic resonance imaging (fMRI)
are increasingly utilized as well, although disruptive methods remain the gold standard
(Hamberger, 2007).
In contrast to the slowly progressive changes related to chronic epileptiform activity,
advances in noninvasive neuroimaging have brought to light the more acute changes in
language organization that result from surgical resection. These changes may or may
not coincide with recovery of function, and determining which factors are more or
less likely to be associated with reorganization and recovery is currently fertile ground
for research. Although acute brain injury is, unfortunately, not uncommon, such phe-
nomena are typically unexpected, and premorbid conditions can only be inferred.
Conversely, epilepsy surgery is planned in advance, and therefore offers the opportunity
for thorough examination prior to resection, enabling within-subject, pre-and postop-
erative comparisons.
Clinically, both of these mechanisms play a role in surgical decision-making. The lo-
cation of essential language areas must be identified in the individual patient, with the
goal of sparing these regions from resection. However, in cases in which language areas
might be disrupted, it would be helpful to predict which patients or which brain regions
are more likely to undergo successful reorganization and recovery.
In the following sections, we present findings that have emerged from studies of both
chronic epilepsy patients who may or may not have undergone progressive language re-
organization during development, and patients assessed pre-and postoperatively who
may have undergone acute language reorganization due to surgical intervention. These
data are based primarily on findings from clinical tools, which include Wada testing,
cortical stimulation, and neuroimaging, as well as language assessment. Drawing from
both of these domains and these various methods, we consider the influence of demo-
graphic, clinical, and epilepsy-related factors that appear to influence progressive and
acute brain plasticity of language.
Effects of Chronic Epilepsy
In considering language plasticity in epilepsy, it is important to note that epilepsy is a

heterogeneous disorder, with individual variability in multiple factors that could po-
tentially affect the cerebral organization of language. These include laterality and intra-
hemispheric location of seizure onset and spread, underlying neuropathology, age of
seizure onset, duration of epilepsy, handedness, seizure frequency, frequency and lo-
cation of abnormal electroencephalogram (EEG) activity between seizures, and type,
Language Plasticity in Epilepsy 319
amount, and duration of pharmacological treatment. Most of the work on language or-
ganization in epilepsy has involved temporal lobe epilepsy (TLE) patients, mainly be-
cause these patients represent the largest and most homogenous subgroup of patients
who undergo surgical treatment. Nevertheless, these patients vary with respect to many
of the factors listed previously, as well as other factors that can influence language orga-
nization, including medial temporal versus lateral neocortical onset and presence/ab-
sence of hippocampal sclerosis (HS).
In many patients with focal epilepsy, structural abnormalities such as neoplasms of
vascular malformations can be identified, and the ability to detect some of the more
subtle morphological irregularities (e.g., sclerosis, dysplasia) has increasingly improved
with advances in neuroimaging. On the other hand, some patients have no structural le-
sion identified, yet the brain tissue is functionally abnormal due to epileptiform activity
(i.e., seizures and abnormal interictal electrophysiological discharges). Thus, chronic
epilepsy can give rise to language reorganization in response to structural aberration,
electrophysiological aberration, or the combination of these. Our knowledge regarding
language plasticity in epilepsy comes primarily from Wada testing, which has brought
to light variations in the hemispheric distribution of language, and electro-cortical map-
ping, which has revealed intra-hemispheric variations in the topography of brain areas
that mediate language.
Alterations in Language Laterality:

Evidence from Wada Testing
Wada testing is an invasive procedure that enables assessment of cognitive functioning

during temporary anesthesia of one cerebral hemisphere, induced by injection of a
short-acting anesthetic agent into the internal carotid artery (Wada & Rasmussen,
1960). The procedure was originally developed to determine hemispheric language
dominance in epilepsy surgery candidates, and was subsequently modified to include
assessment of hemispheric memory capacity as well (Milner, Branch, & Rasmussen,
1962). Given the territory perfused by the internal carotid artery, the procedure enables
assessment of language ability in each cerebral hemisphere, separately, serving as
a crude reversible model for the effects of surgical resection. Although the particular
format of Wada testing varies among surgery centers, the hemispheric anesthetic effect
is typically confirmed via scalp EEG (ipsilateral, unilateral slowing) and contralateral
hemiplegia/hemiparesis. Language testing typically includes automatic speech (e.g.,
counting), object naming, repetition, execution of verbal commands, and reading.
Unilateral language dominance is inferred when all tasks are performed accurately with
anesthesia in one hemisphere, and inability to perform tasks with anesthesia of the con-
tralateral cerebral hemisphere. Bilateral language representation is typically inferred
when a combination of accurate and inaccurate performances occur following both
left-and right-hemisphere anesthesia. Clinically, some centers determine hemispheric

language representation based on qualitative observations, such as occurrence of speech
arrest or presence of paraphasic errors, whereas others utilize more empirically based
scoring methods with calculated laterality indices (Hamberger & Walczak, 1996; Loring,
Meader, Lee, & King, 1992).
Prior to the advent of Wada testing, it was generally assumed that handedness could
be used as an indicator of hemispheric language dominance. Wada testing with epilepsy
patients revealed discrepancies in the relation between these two variables (Gloning,
Gloning, Harb, & Quantember, 1969; Rasmussen & Milner, 1977; Rausch & Walsh, 1984).
Wada testing in mixed neurological samples has shown right-hemisphere language
dominance in approximately 4%–37% of right-handers and 25%–52% of left-handed
or ambidextrous individuals (Branch, Milner, & Rasmussen, 1964; Helmstaedter et al.,
1997; Loring et al., 1990; Mateer & Dodrill, 1983; Powell, Polkey, & Canavan, 1987;
Rausch & Walsh, 1984; Rey, Dellatolas, Bancaud, & Talairach, 1988; Risse et al., 1997;
Strauss & Wada, 1983; Woods, Dodrill, & Ojemann, 1988; Zatorre, 1989), whereas right-
hemisphere language dominance in healthy individuals is estimated at approximately
4% based on Doppler sonography (Knecht et al., 2000).
Similar to that shown in lesion studies of patients with early left-hemisphere injury
and increased likelihood of right-hemisphere language, left-hemisphere seizure onset
has been shown to be the most significant predictor of atypical (i.e., right or bilateral)
language representation (Branch et al., 1964; Loring et al., 1992). Rausch and Walsh
(1984) found that 15% of their sample of right-handed patients with left TLE were right-
hemisphere language dominant, and several investigators have reported a significant re-
lation between of age of “injury” (e.g., left-cerebral injury, left-sided seizure onset) and
atypical language lateralization (Rasmussen & Milner, 1977; Rausch, Boone, & Ary, 1991;
Rey et al., 1988; Satz, Strauss, Wada, & Orsini, 1988; Strauss & Wada, 1983), with earlier
age of injury, typically before age 5, more likely resulting in right or mixed language
dominance. A retrospective study of 445 epilepsy patients who had undergone bilat-
eral Wada testing found that 46% of left-handers with early left-hemisphere lesions were
right-hemisphere language dominant, whereas 37% of left-handers with late neocor-
tical left-hemisphere lesions were more likely to have bilateral language representation
(Möddel, Lneweaver, Schuele, Reinholz, & Loddenkemper, 2009). These results suggest
an influence of age of onset/injury, in that right dominance might reflect early insult, be-
fore left-hemisphere dominance is established, whereas bilateral language might reflect
compromise to the left language system at a time when left-hemisphere dominance has
already started to take hold.
These studies, however, fail to tease apart the effects of space-occupying lesions from
more epilepsy-specific features. With this in mind, Janszky et al. (2003) used Wada
testing to determine language dominance in a relatively homogenous subgroup of 184
TLE patients, all of whom had medial TLE (i.e., hippocampal seizure onset), and uni-
lateral hippocampal sclerosis (HS), yet no other lesions, thereby eliminating the influ-
ence of type of pathology, location of pathology, and age of precipitating injury. HS is
understood to occur during infancy or early childhood (Engel, Williamson, & Wieser,
1997), and is not adjacent to classic language areas. These investigators found that 24%
of left medial temporal lobe epilepsy (MTLE) patients with HS had atypical hemi-
spheric language. Moreover, atypical language representation in these patients was as-
sociated with a significantly higher frequency of interictal discharges and with sensory
auras, which reflects seizure propagation to lateral temporal structures. These findings
suggest that in addition to structural factors, functional factors such as abnormal EEG
activity (i.e., interictal EEG discharges and seizure spread) can alter cerebral language
organization.
Although Wada testing is a powerful disruptive technique, enabling assessment of
language in a functionally isolated hemisphere, the procedure is limited in its rendering
of a fairly diffuse unilateral lesion and its inability to provide information regarding
intra-hemispheric organization of language. Electrical stimulation mapping, discussed
in the following section, enables detailed intra-hemispheric testing of relatively small
cortical areas, and thus could be considered a refined application of the lesion method.
Intra-hemispheric Variations:
Evidence from Electrical
Stimulation Mapping
Electrical stimulation mapping (ESM) is an invasive procedure in which electrical stim-

ulation is applied briefly (~2–10 sec) to the cortical surface, producing a reversible func-
tional lesion in the discrete region below the stimulating electrodes (Hamberger, 2011).
The procedure is used to identify essential sensory, motor, or language areas when there is
concern that the proposed brain surgery will involve removal or disruption of functional
brain tissue. Sites identified as positive are typically spared from resection, with the goal
of preserving function postoperatively. Although primarily used for clinical purposes,
the procedure provides a unique opportunity to study the intra-hemispheric represen-
tation of language (Ojemann, 1983b; Penfield & Roberts, 1959). Nevertheless, in consid-
ering results from ESM studies, as with any technique, it is important to understand the
nature of data that can be obtained, and the potential limitations of the procedure.
As noted earlier, ESM for the identification of language cortex utilizes the lesion
method, and therefore relies on “negative” responses. That is, unlike ESM for the iden-
tification of sensory or motor cortex, which is based on stimulation-evoked “posi-
tive” responses such as subjective sensation (e.g., tingling) or observable movement
(e.g., muscle twitch), stimulation of language cortex does not elicit language behavior.
Instead, the patient must be engaged in a language task, and stimulation of language
cortex will disrupt task performance. Theoretically, an ESM trial simulates the func-
tional consequences of damage to the cortical site(s) being stimulated.
Several investigations have demonstrated topographical specificity of language
functions using ESM; specifically, electro-cortical stimulation of a particular brain area
can disrupt one language function (e.g., naming), yet a different function (e.g., reading)
will remain intact during stimulation. Consequently, unless the particular function
supported by the cortical area is tested during stimulation, the site could erroneously
be classified as negative for language. Although this might suggest that a wide breadth
of language functions should be tested at each site under consideration, practical
considerations, such as time limitations, especially during intra-operative mapping, and
patient discomfort (i.e., headache, fatigue) frequently restrict the number of tasks typ-
ically employed. Language tasks utilized among epilepsy surgery centers vary widely,
although visual object naming is the most commonly used language task for ESM
(approximately 60%) (Hamberger, Williams, & Schevon, 2014). Other language tasks
utilized include auditory description naming, reading, and modified forms of the Token
Test (for comprehension) (Boatman, Lesser, & Gordon, 1995; Hamberger, Goodman,
Perrine, & Tamny, 2001; Luders et al., 1986; Malow et al., 1996; Ojemann, 1983a, 1990;
Schwartz, Devinsky, Doyle, & Perrine, 1999).
Because it is invasive, ESM studies are limited to the areas considered relevant to the
clinical situation, so that only the areas considered likely to be involved in seizure onset
and spread will receive electrode coverage. As such, no normative data are available for
ESM, and it remains unknown where ESM-based language areas would be found in
healthy individuals. Although such information would be useful heuristically, it is not
critical to the procedure’s clinical utility, as stimulation-induced language errors are typ-
ically sufficient to infer an important functional role of the region stimulated. Further,
it is generally assumed that if ESM could be performed in healthy individuals, positive
language areas would mimic the areas associated with aphasia in stroke patients (i.e., a
classic anterior/Broca’s and posterior/Wernicke’s pattern). There is some evidence that
late-onset (i.e., >10 years of age) epilepsy patients might serve as a reasonable model of
normative ESM patterns, as by age 10, “normal” language organization would likely al-
ready be established (Kadis et al., 2011). However, it could also be argued that abnormal
EEG discharges or other neurological abnormalities present early in life, yet preceding
clinical seizures could have interfered with normal language representation (Devinsky
et al., 2000).
Finally, it is important to consider that, whereas language is complex, likely involving
collaboration of multiple brain areas, ESM enables investigation of only small cor-
tical areas, for only a short period of time. Similarly, although it might be tempting to
attribute the function disrupted directly to the brain tissue stimulated, it can only be
stated that inactivation of the stimulated area disrupted the particular function; how-
ever, this area may be part of a larger, integrated network. In this way, functional im-
aging (discussed later) may offer complementary data to ESM.
Certainly, the advantages of ESM are its high level of precision, and as a disruptive
technique, the ability to identify brain areas that are necessary for language, not merely
areas that might participate in ancillary fashion in language functioning. Nevertheless,
these advantages should be considered together with limitations when interpreting
ESM language results. With these cautions in mind, ESM-based findings related to lan-
guage plasticity in epilepsy are presented in the following sections.
Influence of Patient-Related Characteristics

ESM studies in epilepsy patients have revealed two main alterations from the traditional
left hemisphere, anterior (Broca’s) and posterior (Wernicke’s) language areas that would
be expected in the normal population: (1) positive sites including, yet extending be-
yond, traditional language areas; and (2) positive sites adjacent or repositioned within
the left hemisphere. Due to the invasive nature of ESM, the procedure is essentially
always performed only unilaterally; we are aware of no published studies of bilateral
ESM within the same patient. Although ESM results have been reported for the right
hemisphere in patients whose pre-surgical evaluation revealed right-hemisphere in-
volvement in language (Duchowny et al., 1996; Jabbour, Hempel, Gates, Zhang, & Risse,
2005), ESM cannot contribute significantly to the study of full versus partial transfer of
positive areas to the right hemisphere. This area of study is better served via bilateral
Wada testing and noninvasive neuroimaging.
The first pattern, characterized by an overall greater number and more diffuse spatial
representation of positive language sites throughout the left hemisphere, tends to occur
in “disadvantaged” groups. For example, Devinsky et al. (Devinsky, Perrine, Llinas,
Luciano, & Dogali, 1993) and Hamberger et al. (Hamberger, McClelland, Williams,
Goodman, & McKhann, 2007) reported relatively widespread distribution of naming
sites in patients with earlier age of seizure onset (Figure 13.1). This pattern was also
more likely to be found in patients with lower IQ scores, whereas patients with higher
IQ scores tended to have naming sites limited to classic language areas (Devinsky et al.,
2000; Ojemann & Whitaker, 1978). In a sample of left (dominant) epilepsy patients
ranging in age from 19 to 64 years, Hamberger and colleagues (Hamberger, Williams,
McKhann II, & Schevon, 2012) found that older patients had more diffuse spatial repre-
sentation of visual naming sites, and a greater number of auditory description naming
(a) (b)
Figure 13.1. Topographic distribution of auditory (solid circles) and visual (open circles)
naming sites in nonlesional epilepsy patients (A) and patients with space occupying temporal
lobe lesions (B).
Source: Hamberger et al., Epilepsia (2007).
sites relative to younger patients. This might reflect a type of compensatory reorgan-
ization in which a greater amount of brain tissue is needed to support naming func-
tion. Interestingly, despite age differences in naming sites, there were no age-related
differences in naming performance.
Consistent with these findings, Devinsky et al. (2000) reported atypical
left-hemisphere language organization, characterized by naming and reading sites
in anterior or inferior temporal cortex, in TLE patients with lower education levels,
and poorer verbal memory and fluency scores. This contrasted with the more typical
left-hemisphere organization limited to the posterior superior temporal gyrus (STG)
(classic Wernicke’s area) observed in TLE patients with higher education and stronger
verbal performances (Devinsky et al., 2000; Hamberger, 2007).
Greater diffusivity of language cortex and even “atypical” language patterns associ-
ated with clinical features differ from the type of reorganization that appears to be more
directly related to focal cerebral insult. In epilepsy, these generally include electro-
physiological disruption and developmental abnormalities such as dysplasia, as well as
more typical, space-occupying lesions. For example, Haglund et al. (Haglund, Berger,
Shamseldin, Lettich, & Ojemann, 1994) found that epilepsy patients with left temporal
lobe gliomas had proportionally fewer STG naming sites compared to similar patients
without lesions, suggesting that the lesions either displaced or destroyed language sites
that would ordinarily reside there.
In an ESM study comparing TLE patients with HS and without structural pa-
thology, Hamberger, Seidel, and colleagues (2007) found proportionally fewer
naming sites in anterior temporal cortex in HS patients relative to non-HS patients
(consistent with their lower risk of naming decline following anterior temporal re-
section for seizure control) and that overall, auditory description naming sites in
the HS patients were distributed more posteriorly relative to that in TLE patients
without structural pathology (Hamberger, Seidel, et al., 2007). These results were
interpreted to reflect intra-hemispheric reorganization of language in response to the
likely, early development of HS. The posterior displacement of auditory-description
naming sites was considered a possible result of the anterior temporal propagation of
seizures and epileptiform discharges in TLE (Emerson, Turner, Pedley, Walczak, &
Forgione, 1995).
As noted earlier, results from several Wada studies suggest that early onset
left-hemisphere epilepsy often induces language transfer to the right hemisphere.
Nevertheless, using ESM in both left-and right- hemisphere epilepsy patients,
Duchowny and colleagues found language sites remaining within the left hemisphere,
often adjacent to, and sometimes overlapping with, developmental lesions and epilep-
togenic regions, even in patients with epilepsy onset prior to age 5 years (Duchowny
et al., 1996). Right-hemisphere language was rare, found only in patients with very
large, early lesions acquired before age 5 that had clearly destroyed language-relevant
cortical areas.
Right-Hemisphere  ESM
As ESM is generally performed only in the hemisphere considered dominant for lan-
guage, ESM data are inherently biased, and little is known regarding ESM language
testing in the non-dominant hemisphere. In a unique study by Wyllie and colleagues
(Wyllie et al., 1990), 15/15 patients who were left-hemisphere language dominant on
Wada testing showed no evidence of right-hemisphere language using ESM. However, 2/
7 patients who were shown to be right-hemisphere language dominant on Wada testing
had language sites identified in the left hemisphere with ESM. This suggests the pos-
sibility of incomplete transfer of language to the right hemisphere, and highlights that
the left-hemisphere language contribution was not detected with Wada testing (Wyllie
et al., 1990). Using ESM in patients determined to have bilateral language representa-
tion, Jabbour and colleagues found frontal and/or temporal language areas analogous
to the classic essential language areas of the dominant left hemisphere in 4/6 patients
identified with left-sided ESM (Jabbour et al., 2005).
As noted, one shortcoming of ESM is that the procedure enables investigation of only
small cortical areas on a given stimulation trial, using relatively simplified tasks, due to
the time constraints associated with administration of an electrical current to the brain.
Human language is complex, involving integration and coordination of multiple brain
regions, including sulcal and subcortical brain areas that are not accessible to ESM. As
described in the following, neuroimaging techniques are not constrained in these ways,
thereby offering other, or possibly complementary information to ESM findings.
Inter- and Intra-hemispheric

Reorganization: Evidence from fMRI
In recent years, “activation techniques” have been used increasingly to measure changes
in brain function during performance of language-related tasks, allowing for in vivo
or real-time detection of regions that are presumed to be involved in the task at hand.
One promising method is fMRI, which extends beyond the traditional structural tech-
nology of MRI by measuring hemodynamic changes (e.g., increased blood flow to
local vasculature) that are assumed to accompany neural activity in that specific area
(for more details regarding technical aspects of fMRI, see Heim & Specht, Chapter 4
in this volume; Belliveau et al., 1990; Ogawa et al., 1993; Tank, Ogawa, & Ugurbil, 1992;
Turner, Le Bihan, Moonen, Despres, & Frank, 1991). As fMRI is noninvasive, it can be
used repeatedly with minimal time restrictions, and with both patients and healthy con-
trol subjects due to minimal, if any, health risks. Also, and perhaps more important,
it provides detailed information about changes in activation both within and across
cerebral hemispheres, thereby providing useful information regarding language lateral-

ization and localization at the same time. This is particularly relevant, as language repre-
sentation is not a purely unilateral phenomenon (see Van der Haegen & Cai, Chapter 34
in this volume; Springer et al. 1999).
Supporting the use of fMRI as a valid alternative for language lateralization, a
number of studies have demonstrated high concordance rates with the Wada procedure
(Benke et al., 2006; Binder et al., 1996; Rutten, van Rijen, van Veelen, & Ramsey, 1999;
Woermann et al., 2003), although it also has been shown that the level of congruence
varies significantly depending on the specific methods and protocols utilized (Balsamo
& Gaillard, 2002; Benson et al., 1999; Lehericy et al., 2000). The use of fMRI for language
localization has also been bolstered by several studies showing good concordance rates
with ESM (Carpentier et al., 2001; FitzGerald et al., 1997; Pouratian, Bookheimer, Rex,
Martin, & Toga, 2002; Ruge et al., 1999), although another study rightly pointed out that
caution must be exercised when using the information for surgical decisions because
the techniques were not perfectly correlated (Roux et al., 2003).
A number of fMRI studies have added critical knowledge to our understanding of
language-related substrates in epilepsy. It is well known that patients with epilepsy have
a higher incidence of atypical language representation than the general population
(Rasmussen & Milner, 1977; Springer et al., 1999). Not surprisingly, in a retrospective
analysis comparing a mixed sample of pediatric epilepsy patients with healthy controls,
Yuan and colleagues (2006) found a higher rate of atypical language lateralization in the
patient group. In addition, based on prior evidence that increased specialization toward
the left hemisphere is a function of normal development (Holland et al., 2001; Schapiro
et al., 2004; Szaflarski, Holland, Schmithorst, & Byars, 2006), they showed a near signif-
icant association between age and language lateralization in the control group (implying
a trend that may have been obscured by small sample size), yet no significant association
in the patient group, suggesting that epilepsy can either disrupt the normal develop-
mental convergence toward left-hemisphere language areas and/or lead to the activa-
tion or establishment of compensatory areas in the right hemisphere.
The propensity for inter-hemispheric reorganization has been demonstrated in
a number of fMRI studies. For example, similar to other investigators who have
demonstrated right- hemisphere language activation in left- sided seizure patients
(Berl et al., 2005; Brazdil, Zakopcan, Kuba, Fanfrdlova, & Rektor, 2003; Janszky, Jokeit,
et al., 2003; Rosenberger et al., 2009), Adcock and colleagues (Adcock, Wise, Oxbury,
Oxbury, & Matthews, 2003) used an auditory-based word decision to explore activation
asymmetries in Broca’s and Wernicke’s areas in patients with left-hemisphere seizure
foci compared to normal control subjects. As expected, the patients were more likely
to have right-sided activation overall. Furthermore, when the investigators specifically
looked at patients with right-hemisphere activation, the identified sites were right-sided
homologues of Broca’s and broadly defined Wernicke’s areas. However, patients who
remained left-dominant showed only slight and variable differences from the control
group within the left mid-temporal gyrus, indicating only minimal intra-hemispheric
reorganization among these patients.
In another compelling study, Janszky and colleagues (2006) compared patients with
left and right MTLE to investigate whether frequent left-sided interictal abnormalities
(i.e., spikes or sharp waves) would induce a shift in language functioning to the right
hemisphere. Using a covert word-generation task and calculated activation asym-
metry indices, they indeed found a higher incidence of atypical language representa-
tion among the left MTLE group compared to the right MTLE group. Most interesting,
within the left MTLE group only, a higher frequency of interictal abnormalities was as-
sociated with greater left-to-right shift of language functioning. Importantly, this shift
was not influenced by other variables, including gender, age, age of epilepsy onset, sei-
zure frequency, IQ, or verbal fluency, leading the investigators to conclude that chronic
and frequent interictal abnormalities might represent a primary influence in inter-
hemispheric language reorganization independently of other factors.
It is important to note that fMRI research does not always suggest a complete shift of
language from the left to the right hemisphere among patients with left-sided seizures.
For instance, Brazdil et al. (2005) studied right-handed patients with unilateral left TLE
(including suspected medial onset and documented hippocampal sclerosis) versus a
healthy control sample. Using a silent word-generation task, they showed significantly
greater bi-hemispheric representation in the patient group. Furthermore, while acti-
vation patterns among controls generally corresponded with known language circuits,
the patients showed less consistent and more widespread changes in both hemispheres,
including a lack of activation in traditional Broca’s area, with greater activation in the
left medial frontal gyrus extending into the right anterior cingulate gyrus, as well as the
right inferior frontal gyrus. Other clear differences from controls were also noted in the
anterior cingulate, basal ganglia, and cerebellum, but these did not reach the level of sta-
tistical significance. The authors concluded that language does not simply shift from one
hemisphere to the other; rather, there is a very complex and highly individualized pat-
tern of reorganization involving both inter-and intra-hemispheric changes in neuronal
networks.
In related research by Mbwana and colleagues (2009), control participants again
demonstrated primary activation in known language areas. However, language ac-
tivation patterns were much less consistent among a heterogeneous group of patients
with left-hemisphere seizure foci, resulting in the identification of several subgroups,
as follows: group 1a (predominant left-sided activation with a significant cluster in the
left posterior superior temporal sulcus); group 1b (predominant left-sided activation
with no significant difference from the control group); group 2a (predominant right-
sided activation with significant clusters in the right inferior middle superior frontal
gyri, middle temporal gyrus, right cingulate, and left cerebellum); and group 2b (pre-
dominant right-sided activation in the right inferior middle superior frontal gyri, right
angular gyrus, and left cerebellum). The results were interpreted as showing tendencies
for both intra-hemispheric reorganization (i.e., compensation by ipsilateral adjacent
regions as seen in group 1a) and inter-hemispheric reorganization (i.e., recruitment of
contralateral homologous regions as seen in groups 2a and 2b) among individuals with
left-hemisphere seizure disorders. With the exceptions of handedness and MRI detected
pathology, other factors such as gender, age, age of onset, indications of early insult, du-
ration of seizures, and location of seizure focus were not significantly different between
left-and right-lateralized patients, highlighting the importance of investigating lan-
guage patterns on an individual basis.
With respect to symptomatic epilepsy (i.e., seizures associated with known struc-
tural lesions), the location of the lesion(s) can have significant bearing on language
organization, although results are not always predictable. To illustrate, Liegeois and
colleagues (2004) used fMRI to study children and adolescents with lesions that were
either adjacent to or within anterior language cortex (i.e., Broca’s area) or lesions that
were remote from traditional language sites (i.e., hippocampus, parahippocampus,
temporal pole), compared to healthy controls (see Figure 13.2). Not surprisingly, the
patients had a higher rate of atypical language lateralization (50% left-, 10% bilateral-,
40% right-dominant). However, inconsistent with previous reports in the literature
(Devinsky et al., 1993; Isaacs, Christie, Vargha-Khadem, & Mishkin, 1996; Lazar et al.,
2000; Rasmussen & Milner, 1977), lesions near traditional language areas did not result
in higher inter-hemispheric reorganization, as 80% of patients with lesions near Broca’s
area remained left-hemisphere dominant, whereas 80% of patients with lesions away
from traditional language areas had bilateral or right-sided dominance. Importantly,
other clinical factors such as handedness, EEG abnormalities in the left frontal lobe or
Right
1.0
.8
.6
.4
Lateralization Indec (LI)
.2
–.0
–.2
–.4
–.6
–.8
Left
–1.0
Controls Patients
Lesion location
Near Broca’s area Remote from language regions
Figure 13.2. fMRI lateralization index (LI) in control participants and patients. Bars indicate
group means.
Source: Liegeois et al., Brain (2004).
right hemisphere, early onset of chronic epilepsy, and age at first seizure did not appear
to have any bearing on language lateralization.
In similar fMRI research, Weber and colleagues (2006) demonstrated that patients
with left hippocampal sclerosis were more likely to have atypical language lateraliza-
tion than patients with left frontal or left temporal (lateral) lesions, and that the latter
patient groups displayed similar left-sided activation patterns compared to control
participants. The results suggested that the left hippocampus may be important in deter-
mining language dominance, although it is structurally remote from so-called language
eloquent areas. One possible explanation is that damage prior to or during language ac-
quisition can induce a shift toward the right hippocampus for the development of lex-
ical, semantic, and learning of grammatical rules, with subsequent involvement of the
right neocortex through reciprocal connections with the medial temporal region.
The studies reviewed here are intended as a brief yet representative sampling of the
growing body of literature on fMRI and language organization in epilepsy. Despite
the clear advantages of using fMRI for this purpose, one of the major shortcomings
of this technique is that specific functions might be difficult to isolate (e.g., language
tasks might also activate areas associated with auditory processing and sustained atten-
tion), and careful research methods are therefore needed to rule out extraneous factors
when interpreting the results. It is also assumed that any underlying pathology has
not disrupted hemodynamic activity in the regions of interest or expected activation.
Finally, as with any other mode of investigation, differences in methodology between
studies are bound to yield different results. Nevertheless, it seems fairly clear that lan-
guage organization and reorganization in epilepsy is a very complex and heterogeneous
process, and that further exploration is warranted to better our understanding and im-
prove the clinical utility of this technique.
Epilepsy Surgery and Language

Reorganization
As noted earlier, chronic and medically refractory seizure disorders often lead to elec-
tive surgery, which, unlike unexpected neurological conditions (e.g., traumatic brain in-
jury, stroke), allows for careful pre-and postoperative evaluation of language functions.
When considering language reorganization, this is of vital importance because potential
changes can be measured objectively in a controlled fashion, rather than simply inferred
retrospectively. An ideal situation for demonstrating true language plasticity would in-
volve persons with established language networks/abilities prior to neurological illness
and surgery and verifiable changes in those networks/abilities afterward. A number of
case examples and patient series are presented in the following, which provide com-
pelling evidence of both functional and structural alterations, with implications for the
malleability of language circuits in the brain.
In keeping with the fMRI research reviewed in the preceding section, Hertz-Pannier
and colleagues presented a case report of a boy who developed intractable epilepsy re-
lated to Rasmussen’s syndrome at age 5 years and 6 months and who later underwent
a left hemispherotomy (e.g., complete disconnection but not removal of the left hem-
isphere) at age 9 years, after previously normal and complete language acquisition
(Hertz-Pannier et al., 2002). This provided a unique opportunity to compare fMRI
studies from both before and after his surgery. The preoperative fMRI (obtained at age
6 years and 10 months) reportedly showed left lateralization during a covert semantic
word-generation task. By contrast, the postoperative fMRI (obtained at age 10 years and
6 months) reportedly showed right-sided activation during word-generation, sentence-
generation, and story-listening tasks. Most important, it was reported that the latter
activation was “seen mainly in regions that could not be detected preoperatively, but
mirrored those previously found in the left hemisphere (inferior frontal, temporal, and
parietal cortex), suggesting reorganization in a pre-existing bilateral network” (Hertz-
Pannier et al., 2002, p. 361). It was concluded that this de novo activation provided lon-
gitudinal evidence that the boy’s previously nondominant right hemisphere was capable
of sustaining late plasticity changes for language, although it is important to note that
his receptive language functions recovered more quickly than his expressive language
functions, which were still quite limited compared to healthy peers at the last time he
was tested (e.g., verbal fluency = –2 SD, naming = –3 SD).
A number of other reports have also challenged the traditional belief that the crit-
ical period for language acquisition ends around 6 years of age (see Marcotte & Morere,
1990; Woods & Carey, 1979; Woods & Teuber, 1978). In one case, Telfeian et al. (Telfeian,
Berqvist, Danielak, Simon, & Duhaime, 2002) described a female patient with seizures
secondary to Rasmussen’s syndrome beginning at age 11 who underwent a domi-
nant hemispherectomy at age 16. Despite progressive language deterioration before
the surgery and severe global aphasia immediately after surgery, she subsequently
demonstrated a remarkable recovery of language functions over time. Specifically,
performance on the Western Aphasia Battery 1 and 2 years after surgery showed
improvements of 600% in spontaneous speech, 9% in comprehension, 45% in repeti-
tion, 27% in naming, and 89% in the aphasia quotient. Not citing neural plasticity per
se, the authors speculated that this recovery was due primarily to the interruption of a
disease process, allowing the unmasking of right-hemisphere functions that had been
suppressed by diffuse seizure activity. However, they particularly emphasized the point
that language recovery is very much possible, even in adolescence after a late-onset di-
sease process.
In another case report, Voets and colleagues performed fMRI studies on a patient with
Rasmussen’s encephalitis diagnosed at age 6 both before and after surgical removal of
the left hemisphere at age 14 (Voets et al., 2006). Using phonemic and semantic fluency
tasks, the preoperative scan revealed an activation pattern involving primarily inferior
frontal and superior temporal regions bilaterally, whereas postoperative activation was
limited almost exclusively to the right inferior frontal gyrus (IFG). Importantly, mean
word-generation scores were similar (average of 4 words in 30 seconds), indicating no

real change in function as a result of the surgery. In addition, a more specific region of in-
terest analysis showed that the peak preoperative activation in the right IFG was seen in
the pars triangularis for letter fluency and just superior to the pars orbitalis for category
fluency, but the peak activation patterns both shifted more medially and posteriorly in
the frontal operculum/anterior insula postoperatively. The findings were interpreted as
longitudinal evidence of relative changes in localization of activation in the right IFG
after left hemispherectomy.
In yet another case of late-onset language development, though involving a different
underlying pathology, Vargha-Khadem and colleagues described a boy with Sturge-
Weber syndrome and related mutism, who suddenly began to develop speech at age
9 years and 5 months, 10 months after left hemidecortication and withdrawal of anticon-
vulsant medications (Vargha-Khadem et al., 1997). He subsequently displayed a signifi-
cant improvement in verbal communication, with increasing mean length of utterance
from 0 to 11.6 words over the next 13 months, eventually reaching the normal range for
his age on tests of English morphology and phonological imitation of multisyllabic
words. Based on this remarkable case, the authors concluded that the complex neural
systems underpinning speech can remain viable even if unused for the first 9+ years of
life, and that clearly articulated and well-structured language functions can then be de-
veloped entirely within an isolated right hemisphere.
Similar descriptions of language reorganization have been reported in adolescents
with Rasmussen’s encephalitis using intracarotid amobarbital testing (IAT)
(Loddenkemper, Wyllie, Lardizabal, Stanford, & Bingaman, 2003) and others with in-
tractable epilepsy and surgical resection using magnetoencephalography (MEG)
(Papanicolaou et al., 2001). Further, there is growing evidence that patients can de-
velop essentially normal language functioning after left hemispherectomy (Devlin et al.,
2003; Vining et al., 1997). The possibility of postoperative language development or re-
covery is critical with respect to the catastrophic epilepsies that require radical surgical
procedures such as hemispherectomy. Hopefully, further research will shed light on the
circumstances/conditions that promote the likelihood of this phenomenon.
Evidence of Structural Change

in Language Circuits
after Epilepsy Surgery
Moving beyond reported changes in language activation measured by fMRI BOLD

signal, a newer line of inquiry has also shown important structural changes measured
by diffusion tensor imaging (DTI) (see Catani & Forkel, Chapter 9 in this volume) that
occur after neurological surgery involving language network areas. In one such study,
Schoene-Bake and colleagues used postoperative DTI to demonstrate that epilepsy

patients with left or right hippocampal sclerosis exhibit widespread degradation of frac-
tional anisotropy (FA; a measure of fiber density, axonal diameter, and myelination in the
white matter) in main fiber tracts that are not limited to the temporal lobe—including
uncinate fasciculus, fronto-occipital fasciculus, superior longitudinal fasciculus, corpus
callosum, and corticospinal tract in the surgical hemisphere—and that these changes
seem to be more extensive in patients who undergo left-sided surgery (Schoene-Bake
et al., 2009). Although pre-surgical data were not available for direct comparison, it was
inferred from comparisons with other published data that both epilepsy and surgery
can lead to neuronal loss, as fiber tracts undergo a process of Wallerian degeneration
when they are disconnected from afferent and/or efferent structures.
Building on this idea, Yogarajah and colleagues carried out a longitudinal study of left
and right TLE patients also using tractography (Yogarajah et al., 2010). Pre-and post-
surgical comparison revealed widespread and significant (mean 7%) reductions in frac-
tional anisotropy in white matter networks connected to the area of resection in both
groups. At the same time, they also observed a widespread (mean 8%) increase in frac-
tional anisotropy after left anterior temporal lobe resection in the ipsilateral external
capsule and posterior limb of the internal capsule, and corona radiata, believed to rep-
resent a ventro-medial language network. Moreover, these morphometric findings were
correlated with changes in verbal fluency after resective surgery, in that larger increases
in parallel diffusivity (a measure of diffusion along the fiber axis) were associated with
smaller declines in language proficiency. The authors concluded that this ventro-medial
network represented structural reorganization in response to the anterior temporal lobe
resection, which tends to cause greater damage in the more susceptible dorsolateral lan-
guage pathway.
Concluding Remarks
In summary, research in epilepsy has undoubtedly made significant contributions

to the understanding of language organization and plasticity. In this chapter, we have
reviewed relevant findings from disruption (e.g., Wada, ESM) and activation methods
(e.g., fMRI), as well as more recent advances in structural imaging (e.g., DTI), all of
which have enhanced our knowledge about this incredibly complex phenomenon. One
clear message is that, despite an ever-improving theoretical understanding regarding
both functional and structural anomalies that can influence brain-language relations at
varying points in development, recent findings have challenged previously held beliefs
arguing against neural plasticity with increasing age, and demonstrated that the clin-
ical presentation of language is quite heterogeneous among patients with epilepsy,
thereby underscoring the practical importance of using available techniques on a case-
by-case basis to optimize patient outcomes. The study of language in epilepsy is very
much a burgeoning field of inquiry, and more refined research paradigms, together with
ongoing improvements in technology, promise continued advancement of our know-

ledge regarding the neuroscience of language organization and reorganization.
References
Adcock, J. E., Wise, R. G., Oxbury, J. M., Oxbury, S. M., & Matthews, P. M. (2003). Quantitative
fMRI assessment of the differences in lateralization of language-related brain activation in
patients with temporal lobe epilepsy. NeuroImage, 18(2), 423–438.
Balsamo, L. M., & Gaillard, W. D. (2002). The utility of functional magnetic resonance imaging
in epilepsy and language. Current Neurology and Neuroscience Reports, 2, 141–149.
Belliveau, J. W., Rosen, B. R., Kantor, H. L., Rzedzian, R. R., Kennedy, D. N., McKinstry, R.
C., . . . Brady, T. J. (1990). Functional cerebral imaging by susceptibility-contrast NMR.
Magnetic Resonance in Medicine, 14(3), 538–546.
Benke, T., Koylu, B., Visani, P., Karner, E., Brenneis, C., Bartha, L., . . . Willmes, K. (2006).
Language lateralization in temporal lobe epilepsy: A comparison between fMRI and the
Wada Test. Epilepsia, 47(8), 1308–1319.
Benson, R. R., FitzGerald, D. B., LeSueur, L. L., Kennedy, D. N., Kwong, K. K., Buchbinder, B.
R., . . . Rosen, B. R. (1999). Language dominance determined by whole brain functional MRI
in patients with brain lesions. Neurology, 52(4), 798–809.
Berl, M. M., Balsamo, L. M., Xu, B., Moore, E. N., Weinstein, S. L., Conry, J. A., . . . Gaillard, W.
D. (2005). Seizure focus affects regional language networks assessed by fMRI. Neurology,
65(10), 1604–1611.
Berl, M. M., Zimmaro, L. A., Khan, O. I., Dustin, I., Ritzl, E., Duke, E. S., . . . Gaillard, W. D.
(2014). Characterization of atypical language activation patterns in focal epilepsy. Annals of
Neurology, 75(1), 33–42. doi: http://dx.doi.org/10.1002/ana.24015
Binder, J. R., Swanson, S. J., Hammeke, T. A., Morris, G. L., Mueller, W. M., Fischer,
M., . . . Haughton, V. M. (1996). Determination of language dominance using functional
MRI: A comparison with the Wada test. Neurology, 46, 978–984.
Boatman, D., Lesser, R. P., & Gordon, B. (1995). Auditory speech processing in the left temporal
lobe: An electrical interference study. Brain and Language, 51, 269–290.
Branch, C., Milner, B., & Rasmussen, T. (1964). Intracarotid sodium amytal for the lateraliza-
tion of cerebral speech dominance. Journal of Neurosurgery, 21, 399–405
Brazdil, M., Chlebus, P., Mikl, M., Pazourkova, M., Krupa, P., & Rektor, I. (2005).
Reorganization of language-related neuronal networks in patients with left temporal lobe
epilepsy—an fMRI study. European Journal of Neurology,12, 268–275.
Brazdil, M., Zakopcan, J., Kuba, R., Fanfrdlova, Z., & Rektor, I. (2003). Atypical hemispheric
language dominance in left temporal lobe epilepsy as a result of the reorganization of lan-
guage functions. Epilepsy & Behavior, 4(4), 414–419.
Carpentier, A., Pugh, K. R., Westerveld, M., Studholme, C., Skrinjar, O., Thompson,
J.L., . . . Constable, R. T. (2001). Functional MRI of language processing: Dependence on
input modality and temporal lobe epilepsy. Epilepsia, 42(10), 1241–1254.
Devinsky, O., Perrine, K., Hirsch, J., McMullen, W., Pacia, S., & Doyle, W. (2000). Relation of
cortical language distribution and cognitive function in surgical epilepsy patients. Epilepsia,
41(4), 400–404.
Devinsky, O., Perrine, K., Llinas, R., Luciano, D. J., & Dogali, M. (1993). Anterior temporal lan-
guage areas in patients with early onset of TLE. Annals of Neurology, 34, 727–732.
Devlin, A. M., Cross, J. H., Harkness, W., Chong, W. K., Harding, B., Vargha-Khadem, F., &
Neville, B. G. (2003). Clinical outcomes of hemispherectomy for epilepsy in childhood and
adolescence. Brain, 126, 556–566.
Duchowny, M., Jayakar, P., Harvey, A. S., Resnick, T., Alvarez, L., Dean, P., & Levin, B. (1996).
Language cortex representation: Effects of developmental versus acquired pathology. Annals
of Neurology, 40,  31–38.
Emerson, R. G., Turner, C. A., Pedley, T. A., Walczak, T. S., & Forgione, M. (1995). Propagation
patterns of temporal spikes. Electroencephalography and Clinical Neurophysiology, 94,
338–348.
Engel, J., Jr., Williamson, P. D., & Wieser, H. G. (1997). Mesial temporal lobe epilepsy. In
J. Engel, Jr., & T. A. Pedley (Eds.), Epilepsy: A comprehensive textbook (pp. 2417–2426).
New York: Lippincott-Raven.
FitzGerald, D. B., Cosgrove, G. R., Ronner, S., Jiang, H., Buchbinder, B. R., Belliveau, J.
W., . . . Benson, R. R. (1997). Location of language in the cortex: A comparison between func-
tional MR imaging and electrocortical stimulation. American Journal of Neuroradiology,
18(8), 1529–1539.
Gloning, I., Gloning, K., Harb, G., & Quantember, R. (1969). Comparison of verbal behavior in
right-handed and nonright-handed patients with anatomically verified lesions in one hemi-
sphere. Cortex, 5,  43–52.
Haglund, M., Berger, M., Shamseldin, M., Lettich, E, & Ojemann, G.A. (1994). Cortical locali-
zation of temporal lobe language sites in patients with gliomas. Neurosurgery, 34(4), 567–576.
Hamberger, M. J. (2007). Cortical language mapping in epilepsy: A critical review.
Neuropsychology Review, 4, 477–489.
Hamberger, M. J. (2011). Cortical mapping. In J. S. Kreutzer, B. Caplan, & J. DeLuca (Eds.),
Encyclopedia of clinical neuropsychology (pp. 719–721). New York: Springer Science.
Hamberger, M. J., Goodman, R. R., Perrine, K., & Tamny, T. (2001). Anatomical dissociation of
auditory and visual naming in the lateral temporal cortex. Neurology, 56,  56–61.
Hamberger, M. J., McClelland, S., Williams, A. C., Goodman, R. R., & McKhann, G. M.
(2007). Distribution of auditory and visual naming sites in nonlesional temporal lobe ep-
ilepsy patients and patients with space-occupying temporal lobe lesions. Epilepsia, 48(3),
531–538.
Hamberger, M. J., Seidel, W. T., Goodman, R. R., Williams, A. C., Perrine, K., Devinsky, O., &
McKhann, G. M., 2nd. (2007). Evidence for cortical reorganization of language in patients
with hippocampal sclerosis. Brain, 130, 2942–2950.
Hamberger, M. J., & Walczak, T. S. (1996). The Wada test: A critical review. In T. A. Pedley
& B. S. Meldrum (Eds.), Recent advances in epilepsy (Vol. 6, pp. 57–78). Dallas, TX: W.
B. Saunders.
Hamberger, M. J., Williams, A. C., McKhann, G.M., 2nd, & Schevon, C. (2012). Increasing age
and stimulation identified naming sites. Paper presented at the American Epilepsy Society
Annual Meeting, San Diego.
Hamberger, M. J., Williams, A. C., & Schevon, C. A. (2014). Extraoperative neurostimulation
mapping: Results from an international survey of epilepsy surgery programs. Epilepsia,
55(6), 933–939. doi: http://dx.doi.org/10.1111/epi.12644
Helmstaedter, C., Kurthen, M., Linke, D. B., & Elger, C. E. (1997). Patterns of language domi-
nance in focal left and right hemisphere epilepsies: Relation to MRI findings, EEG, sex, and
age at onset of epilepsy. Brain & Cognition, 33(2), 135–150. doi: https://dx.doi.org/10.1006/
brcg.1997.0888
Hertz-Pannier, L., Chiron, C., Jambaque, I., Renaux-Kieffer, V., Van de Moortele, P. F.,
Delalande, O., . . . Le Bihan, D. (2002). Late plasticity for language in a child’s non-dominant
hemisphere: A pre-and post-surgery fMRI study. Brain, 125(Pt 2), 361–372.
Holland, S. K., Plante, E., Weber Byars, A., Strawsburg, R. H., Schmithorst, V. J., & Ball, W. S.,
Jr. (2001). Normal fMRI brain activation patterns in children performing a verb generation
task. NeuroImage, 14(4), 837–843.
Isaacs, E., Christie, D., Vargha-Khadem, F., & Mishkin, M. (1996). Effects of hemispheric
side of injury, age at injury, and presence of seizure disorder on functional ear and hand
asymmetries in hemiplegic children. Neuropsychologia, 34(2), 127–137.
Jabbour, R. A., Hempel, A., Gates, J. R., Zhang, W., & Risse, G. L. (2005). Right hemisphere lan-
guage mapping in patients with bilateral language. Epilepsy & Behavior, 6(4), 587–592.
Janszky, J., Jokeit, H., Heinemann, D., Schultz, R., Woermann, F. G., & Ebner, A. (2003).
Epileptic activity influences the speech organization in medial temporal lobe epilepsy Brain,
126(9), 2043–2051.
Janszky, J., Mertens, M., Janszky, I., Ebner, A., & Woermann, F.G. (2006). Left-sided interictal
epileptic activity induces shift of language lateralization in temporal lobe epilepsy: An fMRI
study. Epilepsia, 47(5), 921–927.
Kadis, D. S., Pang, E. W., Mills, T., Taylor, M. J., McAndrews, M. P., & Smith, M. L. (2011).
Characterizing the normal developmental trajectory of expressive language lateralization
using magnetoencephalography. Journal of the International Neuropsychological Society,
17(5), 896–904. doi: http://dx.doi.org/10.1017/S1355617711000932
Knecht, S., Drager, B., Deppe, M., Bobe, L., Lohmann, H., Floel, A., . . . Henningsen, H. (2000).
Handedness and hemispheric language dominance in healthy humans. Brain, 123, 2512–2518.
Lazar, R. M., Marshall, R. S., Pile-Spellman, J., Duong, H. C., Mohr, J. P., Young, W.
L., . . . DeLaPaz, R. L. (2000). Interhemispheric transfer of language in patients with left
frontal cerebral arteriovenous malformation. Neuropsychologia, 38(10), 1325–1332.
Lehericy, S., Cohen, L., Bazin, B., Samson, S., Giacomini, E., Rougetet, R., . . . Baulac, M. (2000).
Functional MR evaluation of temporal and frontal language dominance compared with the
Wada test. Neurology, 54(8), 1625–1633.
Liegeois, F., Connelly, A., Cross, J. H., Boyd, S. F., Gadian, D. G., Vergha-Khadem, F., &
Baldeweg, T. (2004). Language reorganization in children with early-onset lseions of the left
hemisphere: An fMRI study. Brain, 127, 1229–1236.
Loddenkemper, T., Wyllie, E., Lardizabal, D., Stanford, L. D., & Bingaman, W. (2003). Late lan-
guage transfer in patients with Rasmussen encephalitis. Epilepsia, 44(6), 870–871.
Loring, D. W., Meader, K. J., Lee, G. P., & King, D. W. (1992). Amobarbital effects and lateralized
brain function. New York: Springer-Verlag.
Loring, D. W., Meador, K. J., Lee, G. P., Murro, A. M., Smith, J. R., Flanigin, H. F., . . . King, D.
W. (1990). Cerebral language lateralization: Evidence from intracarotid amobarbital testing.
Luders, H., Lesser, R., Hahn, J., Dinner, D. S., Morris, H. H., Resor, S. R., & Harrison, M.
(1986). Basal temporal language area demonstrated by electrical stimulation. Neurology, 36,
505–510.
Malow, B. A., Blaxton, T. A., Susumu, S., Bookheimer, S., Kufta, C. V., Figlozzi, C. M., &
Theodore, W. H. (1996). Cortical stimulation elicits regional distinctions in auditory and
visual naming. Epilepsia, 37(3), 245–252.
Marcotte, A. C., & Morere, D. A. (1990). Speech lateralization in deaf populations: Evidence for
a developmental critical period. Brain & Language, 39(1), 134–152.
Mateer, C. A., & Dodrill, C. B. (1983). Neuropsychological and linguistic correlates of atypical
language lateralization: Evidence from sodium amytal studies. Human Neurobiology, 2(3),
135–142.
Mbwana, J., Berl, M. M., Ritzl, E. K., Rosenberger, L., Mayo, J., Weinstein, S., . . . Gaillard, W. D.
(2009). Limitations to plasticity of language network reorganization in localization related
epilepsy. Brain, 132(Pt 2), 347–356. doi: http://dx.doi.org/10.1093/brain/awn329
Milner, B., Branch, C., & Rasmussen, T. (1962). Study of short term memory after intracarotid
injection of sodium amobarbital. Transactions of the American Neurological Association, 91,
306–308.
Möddel, G., Lneweaver, T., Schuele, S. U., Reinholz, J., & Loddenkemper, T. (2009). Atypical
language lateralization in epilepsy patients. Epilepsia, 50(6), 1505–1016.
Ogawa, S., Menon, R. S., Tank, D. W., Kim, S. G., Merkle, H., Ellermann, J. M., & Ugurbil,
K. (1993). Functional brain mapping by blood oxygenation level-dependent contrast mag-
netic resonance imaging: A comparison of signal characteristics with a biophysical model.
Biophysical Journal, 64(3), 803–812.
Ojemann, G. A. (1983a). Brain organization for language from the perspective of electrical
stimulation mapping. Behavioral Brain Research, 6, 189–230.
Ojemann, G. A. (1983b). Electrical stimulation and the neurobiology of language. The
Behavioral and Brain Sciences, 2, 221–230.
Ojemann, G. A. (1990). Organization of language cortex derived from investigations during
neurosurgery. Seminars in Neuroscience, 2, 297–305.
Ojemann, G. A., & Whitaker, H. (1978). Language localization and variability. Brain and
Language, 6, 239–260.
Papanicolaou, A. C., Simos, P. G., Breier, J. I., Wheless, J. W., Mancias, P., Baumgartner, J.
E., . . . Butler, I. I. (2001). Brain plasticity for sensory and linguistic functions: A functional
imaging study using magnetoencephalography with children and young adults. Journal of
Child Neurology, 16(4), 241–252.
Penfield, W., & Roberts, L. (1959). Evidence from cortical mapping. In W. Penfield & L.
Roberts (Eds.), Speech and brain mechanisms (pp. 119–137). Princeton, NJ: Princeton
University Press.
Pouratian, N., Bookheimer, S. Y., Rex, D. E., Martin, N. A., & Toga, A. W. (2002). Utility of
preoperative functional magnetic resonance imaging for identifying language cortices in
patients with vascular malformations. Journal of Neurosurgery, 97(1),  21–32.
Powell, G. E., Polkey, C. E., & Canavan, A. G. (1987). Lateralisation of memory functions in
epileptic patients by use of the sodium amytal (Wada) technique. Journal of Neurology,
Neurosurgery and Psychiatry, 50(6), 665–672.
Rasmussen, T., & Milner, B. (1977). The role of early left-brain injury in determining lateraliza-
tion of cerebral speech functions. New York Academy of Science, 299, 355–379.
Rausch, R., Boone, K., & Ary, C. M. (1991). Right-hemisphere language dominance in tem-
poral lobe epilepsy: Clinical and neuropsychological correlates. Journal of Clinical and
Rausch, R., & Walsh, G. O. (1984). Right-hemisphere language dominance in right-handed ep-
ileptic patients. Archives of Neurology, 41(10), 1077–1080.
Rey, M., Dellatolas, G. J., Bancaud, J., & Talairach, J. (1988). Hemispheric lateralization of motor
and speech functions after early brain lesion: Study of 73 epileptic patients with intracarotid
amytal test. Neuropsychologia, 26, 167–172.
Risse, G. L., Gates, J. R., & Fangman, M. C. (1997). A reconsideration of bilateral language
representation based on the intracarotid amobarbital procedure. Brain & Cognition, 33(1),
118–132.
Rosenberger, L. R., Zeck, J., Berl, M. M., Moore, E. N., Ritzl, E. K., Shamim, S., . . . Gaillard, W. D.
(2009). Interhemispheric and intrahemispheric language reorganization in complex partial
epilepsy. Neurology, 72(21), 1830–1836. doi: http://dx.doi.org/10.1212/WNL.0b013e3181a7114b
Roux, F. E., Boulanouar, K., Lotterie, J. A., Mejdoubi, M., LeSage, J. P., & Berry, I. (2003).
Language functional magnetic resonance imaging in preoperative assessment of language
areas: Correlation with direct cortical stimulation. Neurosurgery, 52(6), 1335–1345; discus-
sion 1345–1337.
Ruge, M. I., Victor, J., Hosain, S., Correa, D. D., Relkin, N. R., Tabar, V., . . . Hirsch, J. (1999).
Concordance between functional magnetic resonance imaging and intraoperative language
mapping. Stereotactic & Functional Neurosurgery, 72(2–4), 95–102.
Rutten, G. J., van Rijen, P. C., van Veelen, C. W., & Ramsey, N. F. (1999). Language area locali-
zation with three-dimensional functional magnetic resonance imaging matches intrasulcal
electrostimulation in Broca’s area. Annals of Neurology, 46(3), 405–408.
Satz, P., Strauss, E., Wada, J., & Orsini, D. L. (1988). Some correlates of intra-and
interhemispheric speech organization after left focal brain injury. Neuropsychologia, 26(2),
345–350.
Schapiro, M. B., Schmithorst, V. J., Wilke, M., Byars, A. W., Strawsburg, R. H., & Holland,
S. K. (2004). BOLD fMRI signal increases with age in selected brain regions in children.
Neuroreport, 15(17), 2575–2578.
Schoene-Bake, J. C., Faber, J., Trautner, P., Kaaden, S., Tittgemeyer, M., Elger, C. E., & Weber, B.
(2009). Widespread affections of large fiber tracts in postoperative temporal lobe epilepsy.
NeuroImage, 46(3), 569–576. doi: http://dx.doi.org/10.1016/j.neuroimage.2009.03.013
Schwartz, T. H., Devinsky, O., Doyle, W., & Perrine, K. (1999). Function-specific high-
probability “nodes” identified in posterior language cortex. Epilepsia, 40(5), 575–583.
Springer, J. A., Binder, J. R., Hammeke, T. A., Swanson, S. J., Frost, J. A., Bellgowan, P.
S., . . . Mueller, W. M. (1999). Language dominance in neurologically normal and epilepsy
subjects: A functional MRI study. Brain, 122(Pt 11), 2033–2046.
Strauss, E., & Wada, J. . (1983). Lateral preferences and cerebral speech dominance. Cortex, 19,
165–177.
Szaflarski, J. P., Holland, S. K., Schmithorst, V. J., & Byars, A. W. (2006). fMRI study of language
lateralization in children and adults. Human Brain Mapping, 27(3), 202–212.
Tank, D. W., Ogawa, S., & Ugurbil, K. (1992). Mapping the brain with MRI. Brain Imaging,
2(10), 525–528.
Telfeian, A. E., Berqvist, C., Danielak, C., Simon, S. L., & Duhaime, A. C. (2002). Recovery
of language after left hemispherectomy in a sixteen-year-old girl with late-onset seizures.
Pediatric Neurosurgery, 37(1),  19–21.
Turner, R., Le Bihan, D., Moonen, C. T., Despres, D., & Frank, J. (1991). Echo-planar time course
MRI of cat brain oxygenation changes. Magnetic Resonance in Medicine, 22(1), 159–166.
Vargha-Khadem, F., Carr, L. J., Isaacs, E., Brett, E., Adams, C., & Mishkin, M. (1997). Onset of
speech after left hemispherectomy in a nine-year-old boy. Brain, 120(Pt 1), 159–182.
Voets, N. L., Adcock, J. E., Flitney, D. E., Behrens, T. E., Hart, Y., Stacey, R., . . . Matthews, P.
M. (2006). Distinct right frontal lobe activation in language processing following left hemi-
sphere injury. Brain, 129(Pt 3), 754–766.
Wada, J., & Rasmussen, T. (1960). Intracarotid injection of sodium amytal for the lateral-
ization of cerebral speech dominance: Experimental and clinical observations. Journal of
Weber, B., Wellmer, J., Reuber, M., Mormann, F., Weis, S., Urbach, H., . . . Fernandez, G. (2006).
Left hippocampal pathology is associated with atypical language lateralization in patients
with focal epilepsy. Brain, 29(2), 346–351.
Woermann, F. G., Jokeit, H., Luerding, R., Freitag, H., Schulz, R., Guertler, S., . . . Ebner, A.
(2003). Language lateralization by Wada test and fMRI in 100 patients with epilepsy.
Neurology, 61(5), 699–701.
Woods, B. T., & Carey, S. (1979). Language deficits after apparent clinical recovery from child-
hood aphasia. Annals of Neurology, 6(5), 405–409.
Woods, B. T., & Teuber, H. L. (1978). Changing patterns of childhood aphasia. Annals of
Neurology, 3(3), 273–280.
Woods, R. P., Dodrill, C. B., & Ojemann, G. A. (1988). Brain injury, handedness, and speech lat-
eralization in a series of amobarbital studies. Annals of Neurology, 23(5), 510–518.
Wyllie, E., Luders, H., Murphy, D., Morris, H., 3rd., Dinner, D. S., Lesser, R., . . . Kanner, A.
(1990). Intracarotid amobarbital (Wada) test for language dominance: Correlation with
results of cortical stimulation. Epilepsia, 31(2), 156–161.
Yogarajah, M., Focke, N. K., Bonelli, S. B., Thompson, P., Vollmar, C., McEvoy, A.
W., . . . Duncan, J. S. (2010). The structural plasticity of white matter networks following an-
terior temporal lobe resection. Brain, 133(Pt 8), 2348–2364. doi: http://dx.doi.org/10.1093/
brain/awq175
Yuan, W., Szaflarski, J. P., Schmithorst, V. J., Schapiro, M., Byars, A. W., Strawsburg, R. H., &
Holland, S. K. (2006). fMRI shows atypical language lateralization in pediatric epilepsy
patients. Epilepsia, 47(3), 593–600.
Zatorre, R. J. (1989). Perceptual asymmetry on the dichotic fused words test and cerebral
speech lateralization determined by the carotid sodium amytal test. Neuropsychologia,
27(10), 1207–1219.
Chapter 14
L anguage Dev e l opme nt

in Deaf Ch i l dre n
Sign Language and Cochlear Implants
Aaron J. Newman
Introduction
Deafness and other forms of hearing loss represent the fifth leading cause of disability
in the world; the 2013 Global Burdens of Disease study (Vos et al., 2015) estimated the
total prevalence of hearing loss worldwide at over 1.2 billion people, with over 400 mil-
lion people—over 5% of the world’s population—having moderate or greater loss (de-
fined as a reduction in thresholds of 35 dB or greater, the level at which hearing loss is
considered “disabling”). In the United States, it is estimated that 2–3 out of every 1,000
children are born with detectable hearing loss in one or both ears, and hearing loss is
one of the three most common congenital disorders (Vohr, 2003). The prevalence of
hearing loss is higher in other parts of the world, with highest incidence in South Asia,
Asia Pacific, and Sub-Saharan Africa (Vos et al., 2015).
The consequences of hearing loss can be quite severe. Because language is used both
in education and for self-regulation (“self-talk”), children with hearing loss are more
likely to have delays in language development, reading, and social development, as well
as behavioral problems—especially if they do not have the opportunity to learn and use
sign language, or to have their hearing restored. The consequences of this can be far-
reaching and lifelong: on average, deaf people have lower levels of educational attain-
ment, lower incomes, less likelihood of holding a managerial or professional occupation
(even when adjusted for education level), and report lower levels of job satisfaction,
along with higher incidences of work stress and depression (Rydberg, Gellerstedt, &
Danermark, 2009; Shein, 2003). People who become deaf as adults suffer language com-
prehension difficulties, difficulties in normal activities of daily living, participation re-
striction, social isolation, and high rates of depression.
340 Aaron J. Newman
Clinically, “deafness” can be defined as moderate (40–60 dB loss), severe (60–80 dB),
and profound (>80 db) hearing loss. In practice, these different levels are distinguished
clinically because they affect activities of daily living to different degrees, and have dif-
ferent standards of treatment. People with moderate or severe impairment are generally
still able to detect environmental noises, and are at least aware of speech. However, even
with moderate loss, speech perception can be challenging, and worse in environments
with background noise (such as outside, in a room with multiple people talking, or with
background music, radio, TV, etc.). With severe impairment, speech perception may
be difficult or impossible, even under ideal listening conditions. It is important to rec-
ognize as well that hearing loss is often asymmetric (worse in one ear); generally the
rating of severity of loss is based on the better ear, but one can expect worse functional
hearing in someone with profound loss in one ear and moderate in the other, compared
to someone with moderate loss in both ears.
Numerous options are available for persons with hearing loss, including learning
and using sign language, hearing aids, and cochlear implants (CIs). Hearing aids
are devices that sit on or around the ear and amplify incoming sounds. In contrast,
CIs involve a device that is implanted in the base of the skull, with a wire that is
threaded into the cochlea (see Figure 14.1). A microphone attaches to the outside of
the head via a magnet, and transmits sound across the skull to the CI, which then
stimulates the cochlea via electrodes on the inserted wire. CIs take advantage of the
place-coding organization of the cochlea, whereby different auditory frequencies are
received through electrical stimulation at different points along the cochlea. These
rehabilitative options are not mutually exclusive; for example, many deaf people have
hearing aids or CIs, and CI users may have a CI on one side and a hearing aid on the
other. Profoundly deaf children are routinely referred for cochlear implantation, and
in most cases this is effective in bringing hearing into the normal range. However,
even with hearing restored to audiologically “normal” levels, many children with CIs
show functional deficits (e.g., speech perception in noisy environments; reading)
that persist into adulthood. As well, these treatment options have been the subject of
significant debate in cultural, educational, and health-care spheres. Although these
debates encompass arguments on both empirical and cultural grounds, a solid un-
derstanding of how deafness affects brain organization and language abilities is cen-
tral to better understanding how society can best address the needs of people with
severe hearing loss.
This chapter is focused on language development in deaf children, with a particular
emphasis on children who use CIs and the factors that influence language outcomes.
After briefly reviewing sign language (see also Corina & Lawyer, Chapter 16 in this
volume), we first consider the effects of auditory deprivation on neurodevelopment,
followed by a description of language outcomes in deaf children. This will be followed by
a review of the neuroimaging literature on neuroplastic reorganization in deafness and
after hearing restoration. We then discuss the major predictors of language outcomes
in children with CIs. The chapter concludes with a summary and recommendations for
practice and future research.
Language Development in Deaf Children 341
Figure 14.1. A cochlear implant. The external microphone unit is shown in the top left, and
includes a piece that fits over the external part of the ear, as well as a magnetic transmitter that
connects to the implanted unit. The implant itself is shown above and behind the ear, partially
occluded by the external magnet. Running from the implant is a wire that is threaded into the
cochlea (shell-shaped structure in the center-right of the figure) and turns sound input into elec-
trical stimulation to the cochlea. Signals are then conducted via the auditory nerve (right side of
figure) to the brain.
Source: Image courtesy of MED-EL.
Sign Language and Deaf Culture
Throughout the world, natural sign languages—which have all the linguistic complexity
and characteristics of spoken languages—have evolved naturally among communities
of deaf people. Deaf children exposed to sign language from birth develop normal lan-
guage abilities, and indeed children who learn sign language from birth later show far
greater mastery of spoken language (oral and written) than deaf children not exposed
to sign or exposed only later in life (Mayberry, Lock, & Kazmi, 2002). However, only
an estimated 4%–5% of congenitally deaf children are born to deaf parents, and so
342 Aaron J. Newman
unfortunately the majority of deaf children do not have a fluent signing parent from
whom to learn a sign language. Although parents can of course begin to learn sign lan-
guage and use it with their children, this is a significant undertaking for any parent of a
newborn—and in general cannot be expected to provide the child with optimal, fluent
early language input.
One very important thing to recognize is that the term “sign language” actually
encompasses a variety of manual communication systems, which are very different in
their origins, linguistic status, and the situations and cultures in which they are used. In
the preceding paragraph, the term “natural sign languages” was used, very intention-
ally, to describe visual-manual languages that have evolved independently of each other
in Deaf communities around the world. Contrary to popular misunderstanding, there
is no universally intelligible sign language—just like spoken languages, independent
communities of people have developed their own sign languages that differ from each
other in their phonological structure, grammar, and form- to-meaning mappings
(i.e., vocabulary). Because sign languages have evolved within communities of deaf
users, they are not visual-manual versions of the spoken languages used in the same
geographic locations. For example, American Sign Language (ASL), which is used by
the Deaf community throughout the United States and Canada, has many syntactic
differences from spoken English. Attempting to literally translate an English sentence
into ASL will produce a sentence that may well violate ASL’s syntactic rules; many words
and morphemes that are required in English are not required in ASL (and may actu-
ally not have an ASL equivalent), or are required in different sentence contexts. Further
underscoring the distinction between signed languages and the spoken languages used
in the surrounding hearing communities is the fact that although English is the primary
spoken language in the United States, Australia, Ireland, and the United Kingdom, each
of these countries has its own, linguistically distinct sign language. For example, British
Sign Language (BSL) is mutually unintelligible with ASL; ASL is derived from langue des
signes française (LSF; French Sign Language), not BSL. In spite of their having evolved
largely independently of one another, all natural sign languages appear to follow some
universal principles. Some of these are universal principles of human language that
cross spoken and signed forms (such as having a limited phonological inventory and
phonotactic rules; having hierarchical and recursive syntactic structure), while others
are unique to sign language but common across sign languages (for example, the pho-
netic inventories of sign languages are defined based on the parameters of handshape,
orientation, location, and movement; syntactic morphemes are produced simultane-
ously with their root, rather than sequentially).
In contrast to natural sign languages, a variety of auxiliary sign languages, or manually
coded languages (MCL), have been developed by hearing educators in efforts to help pro-
vide deaf people with access to spoken languages. Generally speaking, these are ways of
representing a spoken language manually—so the vocabulary, word order, and syntactic
morphemes of a spoken language are given manual equivalents. MCLs are often used in
deaf education settings (and parents of deaf children are encouraged to use them) spe-
cifically because they offer access to the spoken language of the milieu. The assumption
underlying their use is that by having visual-manual access to the spoken language, the
child will have an awareness and understanding of that language and its associated phono-
logical and morpho-syntactic systems, which will in turn facilitate tasks such as learning
to read and, if a CI is provided, learning to hear and understand the spoken language.
Language is a defining aspect of many cultures, and this is certainly true of sign lan-
guages: there is a widely recognized distinction between deafness (profound hearing
loss) and Deaf culture (spelled with a capital “D”). Deaf culture revolves around a shared
(signed) language, but encompasses more than just language (including shared beliefs,
values, traditions, and arts) and includes both deaf and hearing people who sign. Within
Deaf culture, hearing loss is not considered a medical condition, and Deaf people—
more than most other groups of people who have a condition defined by the medical
community—tend to resent views of deafness that center around concepts of deficits,
impairment, or disability. Rather, hearing loss is viewed as a central aspect of cultural
identity (although many hearing signers are part of Deaf culture as well). The notion that
deafness needs to be treated, or “fixed,” is viewed as a threat to an individual’s identity
and to Deaf culture more generally. Cochlear implants were thus not initially welcomed
by most of the Deaf community, but rather were viewed as a significant threat, and even
an attempted act of “cultural genocide” by mainstream culture.
Sign Language Development

An immense amount of linguistic development occurs in the first year of life, going from
tuning children’s brains to the phonemic inventory of their native language, through
becoming able to segment the incoming, continuous speech stream into words, to
comprehending and understanding first words. Deaf infants exposed to fluent signing
from birth show similar developmental milestones, at similar ages, to their normally
hearing peers (Petitto et al., 2001). For example, they begin canonical babbling (using
native language phonemes) at the same age (7 months) as hearing babies (Petitto &
Marentette, 1991), and show increasing sensitivity to, and mastery of, ASL phonological
structure over the course of development (Marentette & Mayberry, 2000). Deaf babies
generally produce their first signs some months earlier than hearing children, which has
been attributed to more rapid development of motor control of the hands than the vocal
articulators. As adults, extensive neuropsychological and neuroimaging data demon-
strate that in the deaf, native sign languages activate the same classical, left-lateralized
brain regions as spoken languages (see Corina & Lawyer, Chapter 16 in this volume).
Together with the developmental data, these findings emphasize the fact that signed
and spoken languages are linguistically and neurologically equivalent, and that native
signers are at no linguistic disadvantage in their native language compared to native
learners of a spoken language.
Some of the strongest evidence in favor of sensitive periods in L1 development comes
from studies of deaf people who learned a natural sign language at different ages (such
cases are more prevalent among deaf people due to great variability in how deaf children
344 Aaron J. Newman
were historically educated). On numerous tests, including grammatical judgment, com-

plex grammatical morphology, and sentence recall, there was a clear gradient whereby
native ASL learners outperformed later learners, and those who learned as young chil-
dren outperformed those who only began L1 acquisition after age 12 (Mayberry, 1993;
Newport, 1990). Notably, in all of these studies the participants were tested as adults and
so all had many years of ASL signing experience, confirming that even delays of a few
years in L1 exposure cannot be overcome by years of subsequent practice. Mayberry and
colleagues have further demonstrated that these linguistic deficits in late L1 acquisition
are mirrored by changes in brain organization for language. Examining adults who first
learned ASL as their L1 between the ages of 0–14, they found linear declines in activa-
tion of left hemisphere frontal and temporal language areas with increasing age of ac-
quisition (AoA), along with increased activation in occipital regions, which may have
indicated greater effort required for processing the low-level (visual) features of signs at
the expense of fluent linguistic processing (Mayberry, Chen, Witcher, & Klein, 2011a).
Using another imaging technique, magnetoencephalography (MEG) (see Salmelin,
Kujala and Liljeström, Chapter 6 in this volume), Ferjan Ramirez and colleagues (2013)
similarly found unusual patterns of brain activity, involving occipital and parietal re-
gions, in deaf signers who learned ASL after age 14.
This research confirms that delays in L1 exposure, as experienced by deaf people who
do not receive signed language input from birth, lead to lifelong delays in L1 perfor-
mance. Further evidence also indicates that such delays in signed L1 exposure have
permanent impacts on spoken L2 learning. Mayberry and colleagues (2002) compared
deaf native ASL signers with deaf people who first learned ASL between ages 9–15,
and a group of hearing people who learned English as a second language at a range of
ages matched to the ASL late learners (thus all groups had learned English late as their
L2). On a test of English grammatical judgment, native ASL users performed compa-
rably to hearing English L2 learners, and both groups performed significantly better
than deaf late learners of ASL. In other words, poorer L2 performance was attributable
to the lack of L1 learning, but—critically—not whether the L1 was spoken or signed,
nor whether people were deaf or hearing. Mayberry and Lock (2003) extended these
findings, showing that on both English grammatical judgment and sentence compre-
hension, native ASL signers and hearing English L2 learners performed comparably to
each other, and not significantly worse than native English speakers. More recent work
has extended these findings to deaf CI users. Hassanzadeh (2012) compared two groups
of CI users, matched in age, duration of deafness, and age of implantation. The groups
differed only in that one had deaf parents, and the other had hearing parents. Children
of deaf parents outperformed those of hearing parents on spoken language compre-
hension and production at every time point tested, from 6 months to 10 years post-
implantation. (Although the language use of the parents in this study was not explicitly
reported, the author indicated that sign language was used, and speculated that more
generally, deaf parents adopted communication strategies from birth that were heavily
visual and accommodated children’s lack of hearing.) A similar finding was reported
by Sarant and colleagues (2001), who found that having a family history of hearing
loss predicted significantly better preschool language skills in young CI users. Taking
a slightly different tack, Davidson and colleagues (2014) compared spoken language
outcomes between deaf CI users and hearing children, all of whom had ASL as their L1
by virtue of being born to deaf, signing families. The CI users’ English perception and
production skills on standardized tests were within (and in some cases even exceeded)
age-appropriate norms and were indistinguishable from those of the hearing children.
This was particularly notable because the deaf children had received their CIs after the
age of 1 year (between 16–35 months of age; they had 1–4 years of experience with the CI)
and so, on the basis of the literature reviewed earlier, would be expected to perform on
average below hearing peers on these standardized tests.
Similar findings have been obtained in a number of studies investigating deaf people’s
reading abilities. Learning to read is obviously challenging for deaf people, since it normally
involves mapping novel visual shapes (letters) to sounds (phonemes), which someone who
is pre-lingually deaf has no experience with. On average, (non CI-using) deaf high school
graduates read at a grade 3–4 level (Marschark et al., 2009). However, this statistic masks the
fact that many deaf people do become skilled readers, with many succeeding in higher edu-
cation and earning professional and doctoral degrees. Recent research has focused on what
cognitive skills deaf people utilize in becoming successful readers (Marschark et al., 2009).
A notable finding from these studies is that although phonological abilities (e.g., phonolog-
ical awareness, phonological coding ability) are among the strongest predictors of reading
ability in hearing people, they account for a much smaller proportion of variance in deaf
readers (Holmer, Heimann, & Rudner, 2016; Mayberry, Del Giudice, & Lieberman, 2011b).
This indicates that while lack of access to spoken language phonology may create a challenge
for learning to read, it is not insurmountable, and skilled deaf readers develop alternative
skills for fluent reading, perhaps focused more on visual processing and efficient, whole-
word recognition. Most notably for the current discussion, several studies have found that
signing abilities predict better reading abilities (T. E. Allen, 2015; Chamberlain & Mayberry,
2008; Freel et al., 2011); notably, Allen found this relationship to hold in signing families
of both deaf and hearing children, but not in families of deaf children who only learned
sign language later in childhood, outside of their family setting—again emphasizing the
important role of learning a native language from birth. However, as Godin-Meadow and
Mayberry (2001) note, sign language fluency does not guarantee good reading ability—
reading ability is a specific skill that needs to be taught. An intriguing recent suggestion
is the “functional equivalence” hypothesis (McQuarrie & Parrila, 2014): that exposure to a
natural language with a phonetic structure—regardless of whether that language is spoken
or signed—during sensitive periods for language development in infancy is essential for de-
veloping normal reading abilities.
Language Development with a Cochlear Implant

As noted in the introduction to this chapter, CIs are considered standard of care for
children with severe hearing loss. Overall, the evidence overwhelmingly supports
346 Aaron J. Newman
the efficacy of CIs to restore hearing. At the same time, many CI users perform below
their normally hearing peers on standardized outcome measures, especially those re-
lated to language and scholastic performance—effects that can persist into high school
and are thus not ameliorated simply by years of hearing experience. A significant factor
contributing to heterogeneity of CI outcomes—but also providing useful information—
is the fact that children may receive their implants at a wide variety of ages. There are
several reasons for this. First, in the early days of cochlear implantation, surgeons (and
practice guidelines) were more conservative and tended to wait until children were 3 or
more years old prior to CI surgery. However, more recent evidence has suggested that
infants not only tolerate CI surgery well, but that their outcomes tend to be better with
earlier implantation. Even with this evidence, many parents and clinicians may choose
to wait longer prior to the surgery; as well, some children are not born deaf but become
so sometime during childhood. In other cases, limitations in health-care resources or
insurance may preclude children from receiving CIs as early as might be desired. These
variables result in “natural experiments” that have allowed researchers to study a range
of variables that affect CI outcomes.
Most of the evidence concerning age of implantation comes from relatively young
children, likely because of the increasing recognition that earlier implantation results
in better outcomes, and because longitudinal research is more difficult, costly, and
prone to dropouts. On both comprehension and production measures, several studies
have shown better performance by children implanted before 2 years of age compared
to those implanted between ages 3–5 (Dettman et al., 2016; Tobey et al., 2013); infants
who received CIs within the first two years of life show increased phonetic complexity
in their utterances, which correlates with language skills at 4 years of age (Walker &
Bass-Ringdahl, 2008). Other studies have looked at even younger ages of implanta-
tion, and found significantly better outcomes in children implanted before 12 months
of age than from 13–24 months (Colletti, Mandalà, Zoccante, Shannon, & Colletti, 2011;
Cuda, Murri, Guerzoni, Fabrizi, & Mariani, 2014; Dettman et al., 2016; Dettman, Pinder,
Briggs, & Dowell, 2007; Leigh, Dettman, Dowell, & Briggs, 2013).
It is important to recognize, however, that the fact that CIs are “effective” does not
necessarily mean that CI users’ speech production and perception are indistinguishable
from their normally hearing peers: mean levels of performance decrease with age of im-
plantation, and variability increases. For example, children implanted before 2 years of
age showed an average of 12 months’ delay in receptive language abilities at 2–4 years,
relative to hearing children (Ceh, Bervinchak, & Francis, 2013). In another study of a
group of children who received CIs prior to age 5 and were tested at ages 5–13 years,
only approximately 50% were in the normative range for their age on assessments of
spoken language (Boons et al., 2013). In a large national study of a cohort of children
who were among the first in the United States to receive multichannel CIs, only about
50% of children were within one standard deviation of norms obtained from hearing
children on tasks of receptive and expressive language, although this increased to 70%–
80% in the normal range by high school (Geers, Strube, Tobey, Pisoni, & Moog, 2011).
However, there was still considerable variability in individual outcomes, and the authors
of this study noted significant gaps between verbal and nonverbal IQ scores in CI users,
suggesting that the CI users had not reached levels of spoken language competence that
they might have been able to without a hearing impairment. These studies further noted
that children who evidenced greater language difficulties in early grades were also those
with worse performance in high school. This suggests that although the proportion of
children falling within the normal range increased with age, time alone is not a panacea;
identifying and addressing the needs of CI users who are struggling early on might help
improve long-term outcomes. Further, CI users’ performance varied widely across dif-
ferent outcome measures; generally they performed worse on more challenging tests
(e.g., connected speech compared to isolated words), and better on tests where strategies
or executive skills (such as use of context) could compensate for hearing difficulties
(Geers, Pisoni, & Brenner, 2013; Geers & Sedey, 2011).
While CIs are able to open up the world of sound to children at any age (and even
adults), the data on age of implantation are consistent with the animal literature
pointing to sensitive periods in auditory development (reviewed in the next section),
as well as the many linguistic milestones that normally hearing infants achieve in the
first year of life (see, e.g., Kuhl, 2004, for a review). The precise timing of such sensitive
periods, and the extent to which language outcomes are affected by auditory versus lin-
guistic sensitive periods, are not clear because there are very few cases in which chil-
dren develop with normal hearing but no linguistic input—and those cases that do exist,
such as neglected children and those in some orphanages, are confounded by much
broader deficits in the children’s environments, such as impoverished social and emo-
tional interactions. Furthermore, although the outcomes of children implanted prior to
1 year of age are significantly better than those implanted later, not all of those children
necessarily achieve the same language outcomes as normally hearing peers. It will be
important in the future to study outcomes as a function of age of implantation among
children implanted prior to 1 year—given the rapid rate of development in this period, it
may well be that implantation at 6, 8, or 12 months has as-yet undocumented differences
in outcomes. On the other hand, the phenomenon of suspended auditory development
in animal models (see next section) suggests that a “younger is always better” approach
may not necessarily hold true across the first 12 months of life. Rather, there may be a
lower limit on how young a child needs to be to receive maximum benefit from a CI.
It is also important to note that—while neurobiological sensitive periods doubtless
play a major role in determining CI outcomes—when children receive a CI, they are
not provided with acoustic information that is as rich as normal hearing. Thus it is not
simply a case of auditory deprivation followed by normal hearing. CIs have between
8–32 channels (frequency bands) that they encode. In other words, whereas the in-
tact cochlea encodes frequency along an effectively continuous range, a CI stimulates
only 8–32 locations along the cochlea, although the auditory system adapts over time
in such a way that experienced CI users are able to resolve a much finer-grained range
of frequencies than the few that are directly encoded by the CI electrode. Nevertheless,
children receiving CIs face (at least) two challenges: the effects of auditory depriva-
tion on neurodevelopment, and learning to extract and make sense of the information
348 Aaron J. Newman
necessary to understand speech from a degraded stimulus. In the following sections

we will first examine the neurodevelopmental consequences of auditory deprivation,
followed by consideration of the factors—other than age at implantation—that seem to
mediate language outcomes in deaf children.
Neurodevelopmental Consequences
of Auditory Deprivation
In order to better understand how deafness and CIs affect language processing, it is
helpful to understand the effects of acoustic deprivation on brain organization and
structure. This has been extensively studied in an animal model of congenital deaf-
ness: blue-eyed white cats (Ponton & Eggermont, 2002). A large proportion of these cats
are born with a mutation that causes a lack of hair cells (the acoustic receptor cells in
the cochlea), which is a common cause of deafness in humans as well. Studies using
these cats have allowed for a better understanding of how cells in the auditory system
develop in the absence of acoustic input, and also what impact cochlear implantation
at different ages has on the development and organization of the auditory system. These
studies (reviewed in Kral & Sharma, 2012) have shown that in the course of normal audi-
tory development, cells become tuned to particular acoustic features (such as frequency,
or sensitivity to inter-aural timing differences) through experience. However, even in
the absence of auditory input, a very rudimentary organization around these features
still exists. This suggests that coarse genetic coding guides the organization of the au-
ditory system, but that experience is critical to refine the general “outline” provided by
genetics, and to properly tune cells. In addition, with experience, hearing animals (and
people) develop auditory object representations, learning to associate particular sounds
with particular objects in the world (including environmental noises, such as food
being poured into a bowl; voices associated with individuals; and—at least in humans—
speech). Thus cells’ tuning during normal auditory cortex development is driven by a
combination of both bottom-up (sensory) and top-down (cognitive) influences.
Since the top-down influences rely on representations of complex combinations of
acoustic features, the bottom-up development necessarily starts first. This process has
been shown to be subject to sensitive periods, such that normal development can occur
only with acoustic exposure early in life. In cats, the sensitive period appears to be in the
first 4 months of life: cats receiving CIs prior to this age show normal, or nearly normal,
patterns of electrophysiological responses to sound after CI experience (measured by
electrodes placed directly in A1—primary auditory cortex), whereas cats implanted
later show much weaker and less organized responses—even after the same total dura-
tion of acoustic stimulation (Kral & Sharma, 2012). These changes in sensitivity seem to
be caused by a combination of factors, including changes in neurotransmitter receptor
density, the duration of postsynaptic potentials, dendritic branching, synaptogenesis,
overall cortical inhibition, and structural changes in auditory cortex. In cases of audi-
tory deprivation, the development of these processes is delayed for the first two months
of life, and then proceeds in the absence of stimulation, albeit with much-degraded or-
ganization and sensitivity. Thus, to a certain extent, a lack of auditory stimulation seems
to extend the temporal window of the sensitive period—creating a wider window for
restoring hearing with minimal consequences—but this delay only lasts so long before
self-organization begins to occur, even without input. A further consequence of this de-
velopment that occurs in the absence of hearing is that it seems to reduce neuroplastic
sensitivity to external input (Kral & Sharma, 2012).
In humans, neuroimaging offers a powerful way to study the effects of deafness and
cochlear implantation on language processing, and potentially provide insights into
what underlies suboptimal outcomes. Several imaging modalities have been used
to study deaf people and CI users, including positron emission tomography (PET),
structural magnetic resonance imaging (MRI), functional MRI (fMRI) (see Heim &
Specht, Chapter 4 in this volume), functional near infrared spectroscopy (fNIRS) (see
Minagawa & Cristia, Chapter 7 in this volume) and electroencephalography (EEG),
including event-related potentials (ERP; EEG time-locked to stimulus events of in-
terest) (see Leckey & Federmeier, Chapter 3 in this volume). One significant limita-
tion imposed by cochlear implants is the fact that they are implanted electromagnetic
devices containing metal. Since MRI uses strong magnetic fields and radio frequency
waves, it is not possible to perform research MRI scans on people with CIs.
The developmental time course in humans is (not surprisingly) more protracted
than in cats. Auditory brainstem potentials take approximately 2 years to reach adult-
like shape and timing, and cortically generated auditory evoked potentials (AEPs;
electrical recordings from the scalp) take longer, ranging from 2 years (for the P2 com-
ponent) into adolescence (for the N1; Eggermont & Ponton, 2003). The development
of these potentials reflects underlying changes in the maturation of auditory cortex,
most notably myelination of axons; postmortem histological studies have shown that
in most cortical layers this myelination begins between 4.5–12 months and matures be-
tween 3–5 years, though the auditory system is not fully mature until around 12 years.
Studies of AEPs in CI users show results that parallel this developmental time course.
A large study of people who received CIs at different ages examined the P1 compo-
nent, which reflects the earliest processing of sound in the auditory cortex (Sharma,
Dorman, & Kral, 2005; Sharma, Dorman, & Spahr, 2002). This study demonstrated that
children who received CIs before 3.5 years of age ultimately showed P1 AEP latencies
within the normal range, whereas those implanted after age 7 never showed normal P1
latencies, even after many years of use. Those implanted between 3.5–7 years showed
more variable, intermediate outcomes, leading Sharma and colleagues to conclude that
implantation prior to 3.5 years was optimal from a developmental neurophysioloical
point of view, with a window of decreasing sensitivity up to 7 years. Eggermont and
Ponton (2003) note that for early-implanted children, the delays in P1 latency quite
closely track the duration of deafness; in other words, P1 latencies appear normal when
adjusted for the duration that the child has received auditory stimulation. They further
350 Aaron J. Newman
note that the P1 likely reflects projections from the thalamus to cortical layers III and
IV, which are relatively slow to mature.
Similar results have been obtained for the N1 AEP, which occurs after the P1 and
also reflects early stages of cortical auditory processing. The N1 normally begins to be
detectable in the AEP at around 7 years of age, associated with maturation of cortical
layer II in A1 and thalamo-cortical projections (Eggermont & Ponton, 2003). The de-
velopment of this component continues up to approximately age 9–12 years, which
coincides cortically with the development of A1 layers II and III, and behaviorally with
improvements in speech in noise perception (Eggermont & Ponton, 2003). Sharma
and colleagues (2015) examined N1 development in 80 CI users aged 2–16 years, as
well as normally hearing children of similar age. The N1 began to be detectable in the
AEP waveforms in both normally hearing and early-implanted (<3.5 years) CI users
by 6–9 years, and was identifiable in all early implantees by 9–12 years. In contrast,
among children implanted after the age of 7, the N1 was not detectable in any children
under age 12, and only a small percentage of the older children; as with the P1, children
implanted between 3.5–7 years showed intermediate responses, with less robust N1s
than younger-implanted children. Ponton and Eggermont (2002) also examined N1 la-
tency, in a smaller group of CI users, and found that N1 showed permanent increases in
latency compared to normally hearing children. In a large study of 79 CI children with
a wide range of hearing experience, Jiwani and colleagues (2013) noted that although
differences in AEPs between normally hearing and CI children decreased over time,
they were still present even with 10 or more years of hearing experience. Ponton and
Eggermont suggested that immature myelination of layer I, arising from lack of early
auditory stimulation, might result in poorly synchronized firing patterns that create a
smeared and imprecise signal, affecting the N1 as well as auditory performance in noisy
situations.
Structurally, congenitally deaf adults show no changes in gray matter volume in the su-
perior temporal gyrus (STG) generally, nor in A1/Heschl’s gyrus specifically (Emmorey,
Allen, Bruss, Schenker, & Damasio, 2003; Leporé et al., 2010; Penhune, Cismaru,
Dorsaint-Pierre, Petitto, & Zatorre, 2003; Shibata, 2007). However, reductions in white
matter volume have been reported (Emmorey et al., 2003; Olulade, Koo, LaSasso, &
Eden, 2014). A study of deaf infants similarly showed decreased white matter in Heschl’s
gyrus, as well as increased gray matter volume (Smith et al., 2011). These findings are
consistent with Ponton and Eggermont’s (2002) suggestion of immature myelination.
Outside of the auditory cortex, several other structural differences have been reported,
including increased white matter volumes in the frontal lobes, corpus callosum, and
insula (J. S. Allen, Emmorey, Bruss, & Damasio, 2008; Leporé et al., 2010); also, gray
matter volume is increased in the left insula (J. S. Allen et al., 2008) and left motor cortex
(Penhune et al., 2003). Conflicting results have been obtained for the primary visual
cortex, where one study reported increases in gray matter (J. S. Allen, Emmorey, Bruss,
& Damasio, 2013), while another reported decreases (Olulade et al., 2014). While these
structural changes have not been correlated with specific functional abilities, nor with
CI outcomes, it can be speculated that white matter reductions in auditory cortex reflect
a lack of activity-dependent strengthening of these pathways, which could in turn nega-

tively affect CI outcomes.
While most functional neuroimaging studies have focused on task-related activity in
CI users (e.g., during speech processing), one PET study examined baseline (resting, not
task-related) glucose metabolism measured prior to implantation in a group of children
aged 1–11, and related this to their speech processing outcomes 3 years later (H.-J. Lee
et al., 2007). Auditory speech comprehension was positively correlated with metabolic
activity in the left dorsolateral prefrontal cortex (associated with motor/premotor tasks
and verbal working memory) and angular gyrus (associated with phonological pro-
cessing). Conversely, speech comprehension was negatively correlated with metabolism
in other areas, notably the right STG including Heschl’s gyrus. The authors interpreted
these findings as indicating that higher spontaneous neural activity in auditory/peri-
auditory regions may reflect neuroplastic reorganization, whereby these regions assume
other functions, and thus could not later assume a role in auditory processing—a topic
discussed further in the next section.
A number of other studies have examined brain activation after cochlear implanta-
tion, using functional PET imaging to measure blood flow reflecting task-related ac-
tivity. Studies have consistently reported auditory cortex activation by auditory stimuli
(speech and non-speech) in deaf adults after cochlear implantation, and many have
shown activation of classical left hemisphere language areas as well, including Broca’s
and Wernicke’s areas (Giraud et al., 2000; Herzog et al., 1991; Ito, Iwasaki, Sakakibara,
& Yonekura, 1993; Limb, Molloy, Jiradejvong, & Braun, 2010; Naito et al., 1995; Okazawa
et al., 1996; Truy et al., 1995). A few studies have related brain activation to CI outcomes,
and generally show higher levels of activation within auditory and language regions
in people with better speech comprehension (Fujiki et al., 1999; K. M. J. Green, Julyan,
Hastings, & Ramsden, 2005; Mortensen, Mirz, & Gjedde, 2006). While these findings—
showing correlations between CI outcomes and brain activity markers—make intuitive
sense, several caveats are important to note. First, all of these studies were based on ex-
tremely heterogeneous groups of CI users—varying in their ages of hearing loss (in-
cluding a mix of pre-and post-lingually deafened people), duration of deafness, and
duration of CI use. This reflects the difficulty of performing this type of research: adult
CI recipients are inherently a highly heterogeneous group, and there are relatively few
CI users in any given location. Of these, only a small subset is likely to be interested in
undergoing in a PET scan (which involves radioactive tracers) and also to meet the in-
clusion criteria for the study. However, this poses significant challenges for interpreting
the imaging data, especially the comparisons based on speech comprehension. In all of
these studies, good and poor comprehenders differed in important variables, such as
age of onset of hearing loss, duration of loss, and duration of CI use—all of which likely
contributed to an individual’s performance level. Moreover, most of these studies had
relatively few participants overall, and fewer when these were divided into subgroups
based on performance. Thus, what appears as no activation in a lower-performing
group may be weaker activation combined with insufficient statistical power to detect
that activation. Overall, the small sample sizes in the neuroimaging studies of CI users
352 Aaron J. Newman
published to date is of real concern, especially given the growing recognition that many
neuroimaging studies are underpowered (Button et al., 2013). As such we are posed
with a “chicken and egg” question: Do CI users who achieve better speech perception
outcomes do so because they are better able to engage particular brain areas, or do they
show greater brain activation as a consequence of their better performance?
Neuroplastic Reorganization
in Deafness
The fate of auditory cortices in the absence of acoustic stimulation is mixed. On the
one hand—consistent with the folk adage that when one sense is lost, the others are
enhanced—some parts of the auditory system show evidence of takeover by other sen-
sory systems. Evidence for this was first provided in humans, with congenitally deaf
adults outperforming normally hearing adults on a task involving motion detection in
the attended, peripheral visual field (Neville & Lawson, 1987). Notably, in this study deaf
adults also showed both an enhanced visual N1 ERP component to attended periph-
eral visual stimuli (but not centrally presented stimuli), and an altered scalp topography
of the N1, whereby its peak was shifted anteriorly from occipital to temporal regions.
Although the location of an electrical potential on the scalp cannot accurately inform us
as to the location of its generators in the brain, Neville and Lawson speculated that this
reflected takeover of auditory cortex for visual processing.
Subsequent work has replicated and extended Neville and Lawson’s (1987) original
findings, and has revealed that although deaf people’s visual abilities are not enhanced
across the board, they do show consistent improvements on some specific tasks. Studies
have failed to find any consistent differences between deaf and hearing individuals in
most visual functions, including sustained attention, visual search, orienting or alerting
aspects of attention, or low-level functions such as contrast sensitivity, motion-detection
thresholds, or brightness detection (Bavelier et al., 2001; Bavelier, Dye, & Hauser, 2006;
Bosworth & Dobkins, 2002a; Horn, Davis, Pisoni, & Miyamoto, 2005). Rather, congen-
itally deaf people—both adults and children—show a very specific profile of greater
sensitivity in tasks requiring selective attention to the visual periphery (Bosworth &
Dobkins, 2002a; Bottari, Nava, Ley, & Pavani, 2010; Dye, Baril, & Bavelier, 2007; Dye,
Hauser, & Bavelier, 2009; Neville & Lawson, 1987), and to visual motion (Bosworth
& Dobkins, 1999, 2002b; Mohammed et al., 2005; Neville & Lawson, 1987; Stevens &
Neville, 2006). Congenitally deaf adults and children also evidence more interference
when distracting stimuli are presented in the visual periphery, but less interference from
stimuli presented centrally, than normally hearing individuals (Dye et al., 2007; Dye
et al., 2009). Sign language experience itself leads to a right visual field superiority for
motion detection, in contrast to the left field advantage typical of hearing non-signers
(Bosworth & Dobkins, 1999, 2002b; Hauthal, Sandmann, Debener, & Thorne, 2013).
Similar selective enhancements have been shown in cats, along with associated reor-
ganization of specific auditory regions. In an elegant study by Lomber and colleagues
(Lomber, Meredith, & Kral, 2010), small cooling coils were implanted in cats’ brains
to allow transient deactivation of specific cortical regions. When the posterior au-
ditory field was transiently deactivated, congenitally deaf cats’ enhanced peripheral
visual localization abilities were reduced to levels consistent with normally hearing
cats, indicating a direct causal link between deaf cats’ enhanced performance and this
brain area. Interestingly, in normally hearing cats this area subserves the localization of
sounds, so it seems that the takeover of auditory processing regions by vision was spe-
cific to regions already involved in spatial localization. As well, deaf cats’ enhanced sen-
sitivity to visual motion was reduced to the levels of normally hearing cats by transient
deactivation of a different part of auditory cortex, the dorsal zone. Although the func-
tion of this region in normally hearing cats is unknown, later work showed that in deaf
cats, this area receives some direct axonal projections from visual and somatosensory
cortices (Barone, Lacassagne, & Kral, 2013). At the same time, this work demonstrated
no effects on visual processing when either A1 or several other auditory cortical regions
were deactivated. This indicates that in the absence of auditory input, cortical auditory
regions are not universally taken over for other purposes; rather, much of this cortex
retains a rough approximation of its normal organization. In terms of the implications
of this for CI outcomes, the animal evidence clearly suggests the existence of sensitive
periods in development such that earlier implantation is likely to ultimately result in a
more typical pattern of organization. Nonetheless, in the absence of early intervention,
much of the auditory system appears to remain unoccupied, rather than being taken
over for other sensory or cognitive functions.
The behavioral work in cats and humans, along with the neurophysiological work in
cats, is also supported by some neuroimaging evidence, although the picture is less clear
than in the animal model. These data are almost entirely from congenitally deaf adult
sign language users who did not have CIs. In one set of studies, deaf signers (but not
hearing people) showed right STG activation (including Heschl’s gyrus) when viewing
moving dots in the peripheral visual field (Fine, Finney, Boynton, & Dobkins, 2005;
Finney, Fine, & Dobkins, 2001). However, this finding was not replicated in other studies
using very similar stimuli (Bavelier et al., 2000; Bavelier et al., 2001)—which found in-
stead changes in lateralization and functional connectivity within the same set of re-
gions used for visual processing in hearing people. More recently, Cardin and colleagues
(2013, 2016) showed primary auditory cortex activation by sign language both in deaf
signers and—most notably—in deaf non-signers, but not in hearing non-signers.
Another study showed that cortical thickness in right planum temporale correlated with
visual motion-detection thresholds in congenitally deaf signers (Shiell, Champoux, &
Zatorre, 2016). Further, in a recent study of deaf, non-signing adults with a range of ages
of hearing loss, Muise-Hennessey and colleagues (2015, 2016), found that both earlier
onset and longer duration of deafness were associated with stronger left STG (though
not A1) activity when viewing communicative gestures; this activation was not pre-
sent in normally hearing people. Collectively, these data suggest that deafness increases
354 Aaron J. Newman
the sensitivity of auditory cortex to moving visual stimuli, regardless of sign language
experience.
Some research has specifically investigated relationships between CI use and visual
processing or visual interference. Doucet and colleagues (2006) reported that while
proficient CI users showed enhanced visual N1 and P2 ERP components in response to
moving visual stimuli, poor CI users showed smaller visual ERP components that, crit-
ically, were shifted anteriorly on the scalp compared to the ERPs of normally hearing or
proficient CI users—similar to Neville and Lawson’s (1987) early findings. Other studies
have investigated more directly how visual information affects speech processing in CI
users. Champoux and colleagues (2009) investigated speech processing in the pres-
ence of different types of visual distractors. Proficient CI users performed comparably
to normally hearing adults; however, CI users with poorer auditory-only speech com-
prehension showed greater interference during speech processing in the presence of
moving visual stimuli (but not color-change distractors). Other studies investigated
how the auditory perception of phonemes is influenced by whether the listener sees the
same or a different phoneme being mouthed (Desai, Stickney, & Zeng, 2008; Rouger,
Fraysse, Deguine, & Barone, 2008; Tremblay, Champoux, Lepore, & Théoret, 2010).
CI users showed greater influence of visual information on their speech perception
overall, and those with poorer speech perception showed relatively greater reliance
on visual information (Tremblay et al., 2010). However, reliance on visual informa-
tion during normal speech processing does not appear to impede auditory processing.
A longitudinal study found that CI users who showed significant gains in auditory-only
processing after receiving a CI maintained high levels of speechreading (sometimes re-
ferred to as lipreading) ability; the authors suggested that this was due to the fact that
CI users have greater difficulty perceiving auditory speech in noisy environments, and
so continue to rely on visual cues to aid speech perception, even when the CI is effective
(Rouger et al., 2007).
Given the evidence suggesting cross-modal reorganization in deaf people, an impor-
tant question is whether enhanced sensitivity of auditory cortex to visual information
interferes with hearing after cochlear implantation. One possibility is that if areas nor-
mally specialized for auditory processing do take on other functions, then restoration of
hearing through a CI might be less effective if these areas are forced to reorganize them-
selves, or are even unable to assume auditory processing functions once years of activity-
dependent neural connections have been established. On the other hand, Campbell
and MacSweeney (2014) have noted that even normally hearing people show activation
within auditory regions of the STG for some types of stimuli, such as speechreading
(Calvert & Campbell, 2003) and sign language (in hearing native signers; Neville et al.,
1998). Further, Rouger and colleagues’ (2007) data suggest beneficial effects of at least
one specific enhanced visual ability (speechreading) on speech perception, even after a
CI. Thus if there is visual “takeover” in deafness of brain areas typically used for auditory
processing, it appears to be more nuanced than an area being simply unimodally “au-
ditory” in normally hearing people, and “visual” in deaf people. Although studies have
shown visual interference and altered ERP topography in CI users with poor speech
comprehension, the directionality of these effects is not clear. One interpretation is that
visual takeover of auditory cortex mediates these effects, leading to poorer CI perfor-
mance. However, there are many factors that contribute to CI outcomes, as discussed
previously and in the following two sections. Therefore it is possible that visual activa-
tion of auditory cortex does not interfere with speech processing, but rather represents
a compensatory mechanism whereby people who are unable to obtain optimal audi-
tory information continue to rely to a greater extent on visual information. As a result,
they show overall greater sensitivity to such information—which can manifest in some
circumstances as visual interference. However, outside of experimental lab settings,
visual information is usually beneficial to speech comprehension rather than detri-
mental, and so evidence of such interference should not necessarily be interpreted as a
negative phenomenon.
Factors Influencing CI Outcomes

in Children
As we have seen, age at implantation is a very strong predictor of CI outcomes, with

implantation within the first year of life yielding the best outcomes in congenitally deaf
children. Related to this, in children not born deaf, earlier onset and longer duration of
deafness both predict worse outcomes (Geers, 2003; Sarant et al., 2001). However, this
does not explain all the variance in CI outcomes; a number of other factors have been
identified consistently across multiple studies that influence CI outcomes.
The home and family setting in which a child is raised has very strong influences on
the linguistic and educational achievements of all children (Hart & Risley, 1995; 2003;
Hoff-Ginsberg, 1998), and children with CIs are no different. Socioeconomic status
(SES) is a significant predictor of language outcomes in children with CIs (Convertino,
Marschark, Sapere, Sarchet, & Zupan, 2009; Geers, 2003; Hodges, Ash, Balkany,
Schloffman, & Butts, 1999; Niparko, Tobey, Thal, & Eisenberg, 2010), as is maternal ed-
ucation status (which is closely related to SES; Sarant, Harris, Bennet, & Bant, 2014)—
higher SES and maternal education levels predict better outcomes. Parental behavior
also impacts outcomes in several ways. Mothers who speak in longer, more complex
sentences to their children (mean length of utterance—MLU) have children with better
language outcomes (DesJardin & Eisenberg, 2007), whereas the children of mothers
who use more directives and prohibitions have poorer language outcomes (Fagan,
Bergeson, & Morris, 2014). Maternal sensitivity, the level of parental involvement, and
cognitive stimulation provided to children also influence outcomes (Moeller, 2000;
Quittner et al., 2013; Sarant, Holt, Dowell, Rickards, & Blamey, 2008).
Parental sensitivity can be a double-edged sword, however: data suggest that many
mothers adapt their speech to their expectations of deaf children’s limitations in un-
derstanding speech—producing simpler utterances on the expectation that this will be
356 Aaron J. Newman
easier for the children to understand—which reduces the complexity of language input
and may adversely affect language development (Fagan et al., 2014). It seems likely that
the effects of SES and maternal education on outcomes are at least in part mediated by
these parenting style variables, since better-educated mothers tend to provide their chil-
dren with richer input (Hart & Risley, 1995, 2003). However, studies have also shown
that positive interaction behaviors can be taught, and indeed intervention programs
that teach parents positive interaction styles with their deaf children significantly im-
prove the children’s CI outcomes (Moog & Geers, 2010). Related to this, having a deaf
family member improves CI users’ outcomes, likely due to a combination of increased
sensitivity to the specific needs of deaf children, and more sign language knowledge and
use (Hassanzadeh, 2012; Sarant et al., 2014). Beyond SES and parenting effects, children
born earlier than their siblings (birth order) and/or having fewer siblings show better
outcomes (Geers, Brenner, & Davidson, 2003; Sarant et al., 2014), and girls often do
better than boys (Geers, 2003; Sarant et al., 2014).
Cognitive factors also significantly influence CI outcomes. Among these, the ones
that have been most consistently identified as strong predictors are nonverbal IQ
(Geers, 2003; Geers et al., 2003; Pyman, Blamey, Lacy, Clark, & Dowell, 2000; Sarant
et al., 2014; Sarant et al., 2008) and aspects of working memory—including working
memory capacity, and the speed and efficiency of rehearsal and retrieval (Geers & Sedey,
2011). Motor skills are also closely related to CI outcomes: children with a history of fine
motor problems have poorer outcomes (Pyman et al., 2000; Sarant et al., 2014), as do
those with slower trajectories of motor development (Horn, Fagan, Dillon, Pisoni, &
Miyamoto, 2007; Horn, Pisoni, Sanders, & Miyamoto, 2005).
Genetics may also influence outcomes. This has been studied primarily in the con-
text of mutations to the connexin-26 (GJB2) gene—a leading cause of congenital deaf-
ness in many countries (Denoyelle et al., 1997; G. E. Green et al., 2002). The data are
equivocal, however, with some studies reporting better outcomes for children with
GJB2 mutations than with other causes of deafness (C.-M. Wu et al., 2015), but others
reporting no differences (C.-C. Wu, Lee, Chen, & Hsu, 2008). Another mutation, to
the SLC26A4 gene, has been associated with more positive outcomes (C.-M. Wu et al.,
2015); this study further found that GJB2 and SLC26A4 mutations positively affected
outcomes only in children implanted prior to age 3, but not in children implanted later.
Finally, whether cochlear implantation is unilateral or bilateral may influence
outcomes. Bilateral implants were initially proposed because with normal hearing,
sounds are localized using timing and phase difference cues from the two ears. Not
only does localizing the origins of sounds help in orienting to the environment (which
can help both identify where a speaker is, and where a danger such as oncoming traffic
may be coming from), but it may help speech perception in noisy environments as well,
by helping to focus auditory attention. Indeed, children with bilateral implants outper-
form those with unilateral implants—and also do better with both implants activated
than only one—both on sound localization (Beijen, Snik, & Mylanus, 2007; Galvin,
Mok, & Dowell, 2007; Litovsky, Johnstone, Godar, & Agrawal, 2006; Lovett, Kitterick,
Hewitt, & Summerfield, 2010) and speech in noise perception (Beijen et al., 2007;
Lovett et al., 2010; Peters, Litovsky, Parkinson, & Lake, 2007; Wolfe, Baker, Caraway,
& Kasulis, 2007; Zeitler, Kessler, & Terushkin, 2008). These results also translate into
better use of auditory and vocal skills in social interactions (Tait et al., 2010), and better
performance on standardized language tests (Sarant et al., 2001).
Although virtually all studies comparing bilateral with unilateral implantation have
reported bilateral advantages, there are some caveats to this. First, not all measures used
show bilateral advantages, including subjective assessments of speech and quality of
hearing (Beijen et al., 2007) and preschool language scales (Sarant et al., 2001). Second,
the timing of the implants plays an important role in outcomes. Bilateral implantation is
rarely simultaneous; it typically occurs sequentially, with the second implant being re-
ceived months or even years after the first. The data indicate that the closer together the
two CIs can be implanted, the better the outcomes are (Sharma et al., 2005). With longer
separations between the first and second implants, children have more difficulty both
adjusting to the second implant, and in integrating auditory input from the two sources.
Related to this, speech in noise perception and localization are often significantly better
when the noise occurs on the side of the first implant, and more generally hearing
assessments show better performance for the first-implanted ear, especially with longer
durations between the first and second implants (Galvin et al., 2007; Peters et al., 2007).
Finally, in considering the influence of so many variables on CI outcomes, it is impor-
tant to consider that no research in this area has used randomized, double-blind studies
that are the gold standard in clinical research (Vlastarakos et al., 2010). Rather, studies of
CIs and the effects of deafness more generally reflect the results of “natural experiments”
that occur as a result of the choices made by individual parents based on the options
made available to them in a particular location at a particular point in time. For in-
stance, a child born deaf in the past year in a major urban center of a developed country
is very likely to be identified as deaf within days—if not hours—of being born and may
well have an active CI before his or her first birthday. In contrast, this author has met
people from low socioeconomic status (SES), rural communities who were identified as
deaf only upon entering school and whose parents were informed by their family phy-
sician that there were no treatment options—and who thus only obtained a CI in adult-
hood. While this range of experiences creates a natural degree of heterogeneity that can
be analyzed in an attempt to determine the influence of certain variables, the end result
is that the variables are confounded and often difficult to disentangle.
Conclusions
The first and clearest take-home message from this chapter is that cochlear implanta-
tion is highly effective in restoring hearing, and earlier implantation (certainly before
3.5 years, and probably optimally before 1 year) leads to better outcomes than later im-
plantation. Nevertheless, CIs are not magical spells that fix all the effects of hearing
loss: children with CIs need specific supports in their language development, including
358 Aaron J. Newman
parenting styles and in many cases additional supports in school, especially in early
grades.
Even when CIs are readily available (which is not the case in many places for reasons
such as cost and access to services), there will inevitably be a waiting period prior to the
CI surgery, and an additional period is required post-surgery before the CI is activated
and the child begins receiving auditory input—followed by additional time adapting to
the CI before speech is perceived at all clearly. Although hearing aids may be provided
during the interval before CI activation, for children whose hearing loss is so profound
as to warrant a CI, these will provide little support for language development. The pe-
riod between diagnosis and a level of restored hearing capable of supporting speech
comprehension is thus likely to be months or even years. It is clear that parents should
not simply wait for a CI before their children begin to receive language input—the first
year of life is filled with critical/sensitive periods for language development that rely on
language input. Sign language is the only natural human language that deaf children are
able to perceive, and the evidence suggests that this should be provided, as much as pos-
sible and as early as possible—even if it is expected that children will rely primarily or
exclusively on spoken language after their implant.
There is room in the middle ground for hearing parents to provide the optimal lin-
guistic environment for their infant prior to cochlear implantation. Although parents
who are new to signing cannot provide fluent, native-like input, there is strong evidence
that young children pick up on linguistic structure and regularities even in imperfect,
inconsistent input from nonfluent parents, and end up producing more linguistically
regular output (Brentari, Coppola, Mazzoni, & Goldin-Meadow, 2011)—a testament to
the readiness of infant brains to learn from language input. Parents do not need to be-
come fluent signers in order to provide their deaf children with constructive language
input—indeed, in recent years many parents of children with normal hearing have
taken it upon themselves to learn “baby sign” (ASL vocabulary) to facilitate communi-
cation and bonding with their infants. This practice is supported by empirical evidence
pointing to positive language development and stronger parent-child interactions,
suggesting that there is benefit—and critically, no cost—to exposing children to sign
language (Goodwyn, Acredolo, & Brown, 2000; Kirk, Howlett, Pine, & Fletcher, 2012;
Mueller, Sepulveda, & Rodriguez, 2013). Put another way, there is no evidence that
depriving children of natural language input leads to better language outcomes than
providing them with imperfect input.
Providing deaf infants with sign language input prior to their receiving a CI—and
even continuing afterward in some settings—should simply be viewed as a form
of bilingualism. A vast proportion of children in the world are raised in multilin-
gual households, without negative consequences to their development (and possibly
even benefits; see, in this volume, Green & Kroll, Chapter 11, and Paz-Alonso, Oliver,
Quiñones, & Carreiras, Chapter 24). For parents who worry that their child’s deafness
is a disability and that bilingualism will be an additional burden, we can point to the
data showing that children exposed to sign language from birth show normal language
development, regardless of whether they are deaf and learn only sign language as an L1
(Mayberry et al., 2002), or whether they are hearing, native sign-speech bilinguals (K.
Davidson et al., 2014). In this context, it is also worth noting that even children with
severe intellectual disabilities such as Down’s syndrome and autism spectrum disorder
do not show any costs to being raised bilingually (e.g., English-French) as opposed to
monolingually (Bird et al., 2005; Ohashi et al., 2012). Beyond the irreplaceable benefits
of natural language input during the critical first year of life, providing children with
sign language prepares them for participation in Deaf culture. Although many deaf
children may be raised in hearing families, educated in mainstream classrooms, and
rarely or never encounter other deaf children, they may ultimately find important sup-
port and identity within Deaf culture—even if they achieve excellent hearing outcomes
with a CI.
References
Allen, J. S., Emmorey, K., Bruss, J., & Damasio, H. (2008). Morphology of the insula in relation
to hearing status and sign language experience. The Journal of Neuroscience, 28(46), 11900–
11905. http://doi.org/10.1523/JNEUROSCI.3141-08.2008
Allen, J. S., Emmorey, K., Bruss, J., & Damasio, H. (2013). Neuroanatomical differences in
visual, motor, and language cortices between congenitally deaf signers, hearing signers,
and hearing non- signers. Frontiers in Neuroanatomy, 7, 1–10. http://doi.org/10.3389/
fnana.2013.00026
Allen, T. E. (2015). ASL skills, fingerspelling ability, home communication context and early al-
phabetic knowledge of preschool-aged deaf children. Sign Language Studies, 15(3), 233–265.
http://doi.org/10.1353/sls.2015.0006
Barone, P., Lacassagne, L., & Kral, A. (2013). Reorganization of the connectivity of cortical
field DZ in congenitally deaf cats. PloS One, 8(4), e60093–21. http://doi.org/10.1371/journal.
pone.0060093
Bavelier, D., Brozinsky, C., Tomann, A., Mitchell, T., Neville, H., & Liu, G. (2001). Impact of
early deafness and early exposure to sign language on the cerebral organization for motion
processing. The Journal of Neuroscience, 21(22), 8931–8942.
Bavelier, D., Dye, M. W. G., & Hauser, P. C. (2006). Do deaf individuals see better? Trends in
Cognitive Sciences, 10(11), 512–518. http://doi.org/10.1016/j.tics.2006.09.006
Bavelier, D., Tomann, A., Hutton, C., Mitchell, T., Corina, D., Liu, G., & Neville, H. (2000).
Visual attention to the periphery is enhanced in congenitally deaf individuals. The Journal of
Neuroscience, 20(17), RC93. http://www.jneurosci.org/content/20/17/RC93
Beijen, J. W., Snik, A., & Mylanus, E. (2007). Sound localization ability of young children with
bilateral cochlear implants. Otology and Neurotology, 28(4), 479–485. http://doi.org/10.1097/
mao.0b013e3180430179
Bird, E. K.-R., Cleave, P., Trudeau, N., Thordardottir, E., Sutton, A., & Thorpe, A. (2005). The
language abilities of bilingual children with Down syndrome. American Journal of Speech-
Language Pathology, 14(3), 187–199. http://doi.org/10.1044/1058-0360(2005/019)
Boons, T., De Raeve, L., Langereis, M., Peeraer, L., Wouters, J., & van Wieringen, A. (2013).
Expressive vocabulary, morphology, syntax and narrative skills in profoundly deaf children
after early cochlear implantation. Research in Developmental Disabilities, 34(6), 2008–2022.
http://doi.org/10.1016/j.ridd.2013.03.003
360 Aaron J. Newman
Bosworth, R. G., & Dobkins, K. R. (1999). Left-hemisphere dominance for motion processing
in deaf signers. Psychological Science, 10(3), 256–262. http://doi.org/10.1111/1467-9280.00146
Bosworth, R. G., & Dobkins, K. R. (2002a). The effects of spatial attention on motion pro-
cessing in deaf signers, hearing signers, and hearing nonsigners. Brain and Cognition, 49(1),
152–169. http://doi.org/10.1006/brcg.2001.1497
Bosworth, R. G., & Dobkins, K. R. (2002b). Visual field asymmetries for motion processing
in deaf and hearing signers. Brain and Cognition, 49(1), 170–181. http://doi.org/10.1006/
brcg.2001.1498
Bottari, D., Nava, E., Ley, P., & Pavani, F. (2010). Enhanced reactivity to visual stimuli in deaf
individuals. Restorative Neurology and Neuroscience, 28(2), 167–179. http://doi.org/10.3233/
RNN-2010-0502
Brentari, D., Coppola, M., Mazzoni, L., & Goldin-Meadow, S. (2011). When does a system be-
come phonological? Handshape production in gesturers, signers, and homesigners. Natural
Language & Linguistic Theory, 30(1), 1–31. http://doi.org/10.1007/s11049-011-9145-1
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò,
M. R. (2013). Power failure: Why small sample size undermines the reliability of neurosci-
ence. Nature Reviews Neuroscience, 14(5), 365–376. http://doi.org/10.1038/nrn3475
Calvert, G. A., & Campbell, R. (2003). Reading speech from still and moving faces: The neural
substrates of visible speech. Journal of Cognitive Neuroscience, 15(1), 57–70. http://doi.org/
10.1162/089892903321107828
Campbell, R., & MacSweeney, M. (2014). Cochlear implantation (CI) for prelingual deaf-
ness: The relevance of studies of brain organization and the role of first language acquisition
in considering outcome success. Frontiers in Human Neuroscience, 8, 1–11. http://doi.org/
10.3389/fnhum.2014.00834
Cardin, V., Orfanidou, E., Rönnberg, J., Capek, C. M., Rudner, M., & Woll, B. (2013).
Dissociating cognitive and sensory neural plasticity in human superior temporal cortex.
Nature Communications, 4, 1473–1475. http://doi.org/10.1038/ncomms2463
Cardin, V., Smittenaar, R. C., Orfanidou, E., Rönnberg, J., Capek, C. M., Rudner, M., & Woll,
B. (2016). Differential activity in Heschl’s gyrus between deaf and hearing individuals is due
to auditory deprivation rather than language modality. NeuroImage, 124, 96–106. http://doi.
org/10.1016/j.neuroimage.2015.08.073
Ceh, K. M., Bervinchak, D. M., & Francis, H. W. (2013). Early literacy gains in children with
cochlear implants. Otology and Neurotology, 34, 416–421.
Chamberlain, C., & Mayberry, R. I. (2008). American Sign Language syntactic and narrative
comprehension in skilled and less skilled readers: Bilingual and bimodal evidence for the
linguistic basis of reading. Applied Psycholinguistics, 29(03), 367–388. http://doi.org/10.1017/
S014271640808017X
Champoux, F., Lepore, F., Gagné, J.-P., & Théoret, H. (2009). Visual stimuli can impair auditory
processing in cochlear implant users. Neuropsychologia, 47(1), 17–22. http://doi.org/10.1016/
Colletti, L., Mandalà, M., Zoccante, L., Shannon, R. V., & Colletti, V. (2011). Infants versus
older children fitted with cochlear implants: Performance over 10 years. International
Journal of Pediatric Otorhinolaryngology, 75(4), 504–509. http://doi.org/10.1016/
j.ijporl.2011.01.005
Convertino, C. M., Marschark, M., Sapere, P., Sarchet, T., & Zupan, M. (2009). Predicting ac-
ademic success among deaf college students. Journal of Deaf Studies and Deaf Education,
14(3), enp005–343. http://doi.org/10.1093/deafed/enp005
Cuda, D., Murri, A., Guerzoni, L., Fabrizi, E., & Mariani, V. (2014). Pre-school children
have better spoken language when early implanted. International Journal of Pediatric
Otorhinolaryngology, 78(8), 1327–1331. http://doi.org/10.1016/j.ijporl.2014.05.021
Davidson, K., Lillo-Martin, D., & Pichler, D. C. (2014). Spoken English language development
among native signing children with cochlear implants. Journal of Deaf Studies and Deaf
Education, 19(2), 238–250. http://doi.org/10.1093/deafed/ent045
Denoyelle, F., Weil, D., Maw, M. A., Wilcox, S. A., Lench, N. J., Allen-Powell, D. R., et al. (1997).
Prelingual deafness: High prevalence of a 30delG mutation in the connexin 26 gene. Human
Molecular Genetics, 6(12), 2173–2177.
Desai, S., Stickney, G., & Zeng, F.-G. (2008). Auditory-visual speech perception in normal-
hearing and cochlear-implant listeners. The Journal of the Acoustical Society of America,
123(1), 428–440. http://doi.org/10.1121/1.2816573
DesJardin, J. L., & Eisenberg, L. S. (2007). Maternal contributions: Supporting language devel-
opment in young children with cochlear implants. Ear and Hearing, 28(4), 456–469. http://
doi.org/10.1097/aud.0b013e31806dc1ab
Dettman, S. J., Dowell, R. C., Choo, D., Arnott, W., Abrahams, Y., Davis, A., et al. (2016).
Long-term communication outcomes for children receiving cochlear implants younger
than 12 months. Otology and Neurotology, 37(2), e82–e95. http://doi.org/10.1097/
MAO.0000000000000915
Dettman, S. J., Pinder, D., Briggs, R., & Dowell, R. C. (2007). Communication development in
children who receive the cochlear implant younger than 12 months: Risks versus benefits.
Ear and Hearing, 28(Supplement), 11S–18S. http://doi.org/10.1097/aud.0b013e31803153f8
Doucet, M. E., Bergeron, F., Lassonde, M., Ferron, P., & Lepore, F. (2006). Cross-modal reor-
ganization and speech perception in cochlear implant users. Brain, 129, 3376–3383. http://
doi.org/10.1093/brain/awl264
Dye, M. W. G., Baril, D. E., & Bavelier, D. (2007). Which aspects of visual attention are changed
by deafness? The case of the Attentional Network Test. Neuropsychologia, 45(8), 1801–1811.
http://doi.org/10.1016/j.neuropsychologia.2006.12.019
Dye, M. W. G., Hauser, P. C., & Bavelier, D. (2009). Is visual selective attention in deaf
individuals enhanced or deficient? The case of the useful field of view. PloS One, 4(5), e5640.
http://doi.org/10.1371/journal.pone.0005640
Eggermont, J. J., & Ponton, C. W. (2003). Auditory-evoked potential studies of cortical matu-
ration in normal hearing and implanted children: Correlations with changes in structure
and speech Perception. Acta Oto-Laryngologica, 123(2), 249–252. http://doi.org/10.1080/
0036554021000028098
Emmorey, K., Allen, J. S., Bruss, J., Schenker, N., & Damasio, H. (2003). A morphometric
analysis of auditory brain regions in congenitally deaf adults. Proceedings of the National
Academy of Sciences USA, 100(17), 10049–10054. http://doi.org/10.1073/pnas.1730169100
Fagan, M. K., Bergeson, T. R., & Morris, K. J. (2014). Synchrony, complexity and directiveness
in mothers’ interactions with infants pre-and post-cochlear implantation. Infant Behavior
and Development, 37(3), 249–257. http://doi.org/10.1016/j.infbeh.2014.04.001
Ferjan Ramirez, N., Leonard, M. K., Torres, C., Hatrak, M., Halgren, E., & Mayberry, R. I.
(2013). Neural language processing in adolescent first-language learners. Cerebral Cortex, 1,
2772–2783. http://doi.org/10.1093/cercor/bht137
Fine, I., Finney, E. M., Boynton, G. M., & Dobkins, K. R. (2005). Comparing the effects of
auditory deprivation and sign language within the auditory and visual cortex. Journal of
Cognitive Neuroscience, 17(10), 1621–1637. http://doi.org/10.1162/089892905774597173
362 Aaron J. Newman
Finney, E. M., Fine, I., & Dobkins, K. R. (2001). Visual stimuli activate auditory cortex in the
deaf. Nature Neuroscience, 4(12), 1171–1173. http://doi.org/10.1038/nn763
Freel, B. L., Clark, M. D., Anderson, M. L., Gilbert, G. L., Musyoka, M. M., & Hauser, P. C. (2011).
Deaf individuals’ bilingual abilities: American Sign Language proficiency, reading skills, and
family characteristics. Psychology, 2(1), 18–23. http://doi.org/10.4236/psych.2011.21003
Fujiki, N., Naito, Y., Hirano, S., Kojima, H., Shiomi, Y., Nishizawa, S., et al. (1999). Correlation
between rCBF and speech perception in cochlear implant users. Auris Nasus Larynx, 26(3),
229–236.
Galvin, K. L., Mok, M., & Dowell, R. C. (2007). Perceptual benefit and functional outcomes
for children using sequential bilateral cochlear implants. Ear and Hearing, 28(4), 470–482.
http://doi.org/10.1097/aud.0b013e31806dc194
Geers, A. E. (2003). Predictors of reading skill development in children with early coch-
lear implantation. Ear and Hearing, 24(1 Suppl), 59S–68S. http://doi.org/10.1097/
01.AUD.0000051690.43989.5D
Geers, A., Brenner, C., & Davidson, L. (2003). Factors associated with development of speech
perception skills in children implanted by age five. Ear and Hearing, 24(1 Suppl), 24S–
35S. http://doi.org/10.1097/01.AUD.0000051687.99218.0F
Geers, A. E., Pisoni, D. B., & Brenner, C. (2013). Complex working memory span in cochlear
implanted and normal hearing teenagers. Otology and Neurotology, 34, 396–401.
Geers, A. E., & Sedey, A. L. (2011). Language and verbal reasoning skills in adolescents with 10
or more years of cochlear implant experience. Ear and Hearing, 32, 39S–48S. http://doi.org/
10.1097/AUD.0b013e3181fa41dc
Geers, A. E., Strube, M. J., Tobey, E. A., Pisoni, D. B., & Moog, J. S. (2011). Epilogue: Factors
contributing to long-term outcomes of cochlear implantation in early childhood. Ear and
Hearing, 32, 84S–92S. http://doi.org/10.1097/AUD.0b013e3181ffd5b5
Giraud, A. L., Truy, E., Frackowiak, R. S., Gregoire, M. C., Pujol, J. F., & Collet, L. (2000).
Differential recruitment of the speech processing system in healthy subjects and
rehabilitated cochlear implant patients. Brain, 123(Pt 7), 1391–1402.
Goldin-Meadow, S., & Mayberry, R. I. (2001). How do profoundly deaf children learn to read?
Learning Disabilities Research & Practice, 16(4), 222–229. http://doi.org/10.1111/0938-8982.00022
Goodwyn, S. W., Acredolo, L. P., & Brown, C. A. (2000). Impact of symbolic gesturing on early
language development. Journal of Nonverbal Behavior, 24(2), 81–103. http://doi.org/10.1023/
A:1006653828895
Green, G. E., Scott, D. A., McDonald, J. M., Teagle, H. F. B., Tomblin, B. J., Spencer, L. J., et al.
(2002). Performance of cochlear implant recipients with GJB2-related deafness. American
Journal of Medical Genetics, 109(3), 167–170. http://doi.org/10.1002/ajmg.10330
Green, K. M. J., Julyan, P. J., Hastings, D. L., & Ramsden, R. T. (2005). Auditory cortical ac-
tivation and speech perception in cochlear implant users: Effects of implant experience
and duration of deafness. Hearing Research, 205(1–2), 184–192. http://doi.org/10.1016/
j.heares.2005.03.016
Hart, B., & Risley, T. R. (1995). Meaningful differences in the everyday experience of young
American children. Baltimore, MD: P. H. Brookes.
Hart, B., & Risley, T. R. (2003). The early catastrophe: The 30 million word gap by age 3.
American Educator, 27(10),  4–9.
Hassanzadeh, S. (2012). Outcomes of cochlear implantation in deaf children of deaf
parents: Comparative study. The Journal of Laryngology & Otology, 126(10), 989–994. http://
doi.org/10.1017/S0022215112001909
Hauthal, N., Sandmann, P., Debener, S., & Thorne, J. (2013). Visual movement perception in
deaf and hearing individuals. Advances in Cognitive Psychology, 9(2), 53–61. http://doi.org/
10.5709/acp-0131-z
Herzog, H., Lamprecht, A., Kühn, A., Roden, W., Vosteen, K.-H., & Feinendegen, L. E. (1991).
Cortical activation in profoundly deaf patients during cochlear implant stimulation
demonstrated by H215O PET. Journal of Computer Assisted Tomography, 15(3), 369.
Hodges, A. V., Ash, M. D., Balkany, T. J., Schloffman, J. J., & Butts, S. L. (1999). Speech percep-
tion results in children with cochlear implants: Contributing factors. Otolaryngology—Head
and Neck Surgery, 121(1), 31–34. http://doi.org/10.1016/S0194-5998(99)70119-1
Hoff-Ginsberg, E. (1998). The relation of birth order and socioeconomic status to children’s
language experience and language development. Applied Psycholinguistics, 19(04), 603–629.
http://doi.org/10.1017/S0142716400010389
Holmer, E., Heimann, M., & Rudner, M. (2016). Evidence of an association between sign
language phonological awareness and word reading in deaf and hard-of-hearing chil-
dren. Research in Developmental Disabilities, 48, 145–159. http://doi.org/10.1016/
j.ridd.2015.10.008
Horn, D. L., Davis, R. A. O., Pisoni, D. B., & Miyamoto, R. T. (2005). Development of visual
attention skills in prelingually deaf children who use cochlear implants. Ear and Hearing,
26(4), 389–408.
Horn, D. L., Fagan, M. K., Dillon, C. M., Pisoni, D. B., & Miyamoto, R. T. (2007). Visual-motor
integration skills of prelingually deaf children: Implications for pediatric cochlear implan-
tation. The Laryngoscope, 117(11), 2017–2025. http://doi.org/10.1097/MLG.0b013e3181271401
Horn, D. L., Pisoni, D. B., Sanders, M., & Miyamoto, R. T. (2005). Behavioral assessment of pre-
lingually deaf children before cochlear implantation. The Laryngoscope, 115(9), 1603–1611.
http://doi.org/10.1097/01.mlg.0000171018.97692.c0
Ito, J., Iwasaki, Y., Sakakibara, J., & Yonekura, Y. (1993). Positron emission tomography of au-
ditory sensation in deaf patients and patients with cochlear implants. The Annals of Otology,
Rhinology, and Laryngology, 102(10), 797–801. http://doi.org/10.1177/000348949310201011
Jiwani, S., Papsin, B. C., & Gordon, K. A. (2013). Central auditory development after long-term
cochlear implant use. Clinical Neurophysiology, 124(9), 1868–1880. http://doi.org/10.1016/
j.clinph.2013.03.023
Kirk, E., Howlett, N., Pine, K. J., & Fletcher, B. C. (2012). To sign or not to sign? The impact of
encouraging infants to gesture on infant language and maternal mind-mindedness. Child
Development, 84(2), 574–590. http://doi.org/10.1111/j.1467-8624.2012.01874.x
Kral, A., & Sharma, A. (2012). Developmental neuroplasticity after cochlear implantation.
Trends in Neurosciences, 35(2), 111–122. http://doi.org/10.1016/j.tins.2011.09.004
Kuhl, P. K. (2004). Early language acquisition: Cracking the speech code. Nature Reviews
Neuroscience, 5(11), 831–843. http://doi.org/10.1038/nrn1533
Lee, H.-J., Giraud, A.-L., Kang, E., Oh, S. H., Kang, H., Kim, C. S., & Lee, D. S. (2007). Cortical
activity at rest predicts cochlear implantation outcome. Cerebral Cortex, 17(4), 909–917.
http://doi.org/10.1093/cercor/bhl001
Leigh, J., Dettman, S., Dowell, R., & Briggs, R. (2013). Communication development in chil-
dren who receive a cochlear implant by 12 months of age. Otology & Neurotology, 34(3), 443–
450. http://doi.org/10.1097/mao.0b013e3182814d2c
Leporé, N., Vachon, P., Lepore, F., Chou, Y.-Y., Voss, P., Brun, C. C., et al. (2010). 3D mapping of
brain differences in native signing congenitally and prelingually deaf subjects. Human Brain
Mapping, 31(7), 970–978. http://doi.org/10.1002/hbm.20910
364 Aaron J. Newman
Limb, C. J., Molloy, A. T., Jiradejvong, P., & Braun, A. R. (2010). Auditory cortical activity
during cochlear implant-mediated perception of spoken language, melody, and rhythm.
Journal of the Association for Research in Otolaryngology, 11(1), 133–143. http://doi.org/
10.1007/s10162-009-0184-9
Litovsky, R. Y., Johnstone, P. M., Godar, S., & Agrawal, S. (2006). Bilateral cochlear implants
in children: Localization acuity measured with minimum audible angle. Ear and Hearing,
27(1), 43–59. http://doi.org/10.1097/01.aud.0000194515.28023.4b
Lomber, S. G., Meredith, M. A., & Kral, A. (2010). Cross-modal plasticity in specific auditory
cortices underlies visual compensations in the deaf. Nature Neuroscience, 13(11), 1421–1427.
http://doi.org/10.1038/nn.2653
Lovett, R. E. S., Kitterick, P. T., Hewitt, C. E., & Summerfield, A. Q. (2010). Bilateral or unilat-
eral cochlear implantation for deaf children: An observational study. Archives of Disease in
Childhood, 95(2), 107–112. http://doi.org/10.1136/adc.2009.160325
Marentette, P. F., & Mayberry, R. I. (2000). Principles for an emerging phonological system: A
case study of early ASL acquisition. In C. Chamberlain, J. P. Morford, & R. I. Mayberry
(Eds.), Language acquisition by eye (pp. 71–90). Mahwah, NJ: Psychology Press.
Marschark, M., Sapere, P., Convertino, C. M., Mayer, C., Wauters, L., & Sarchet, T. (2009). Are
deaf students’ reading challenges really about reading? American Annals of the Deaf, 154(4),
357–370. http://doi.org/10.1353/aad.0.0111
Mayberry, R. I. (1993). First-language acquisition after childhood differs from second-language
acquisition: The case of American Sign Language. Journal of Speech, Language, and Hearing
Research, 36(6), 1258–1270. http://doi.org/10.1044/jshr.3606.1258
Mayberry, R. I., & Lock, E. (2003). Age constraints on first versus second language acquisi-
tion: Evidence for linguistic plasticity and epigenesis. Brain and Language, 87(3), 369–384.
http://doi.org/10.1016/S0093-934X(03)00137-8
Mayberry, R. I., Chen, J.-K., Witcher, P., & Klein, D. (2011a). Age of acquisition effects on the
functional organization of language in the adult brain. Brain and Language, 119(1), 16–29.
http://doi.org/10.1016/j.bandl.2011.05.007
Mayberry, R. I., Del Giudice, A. A., & Lieberman, A. M. (2011b). Reading achievement in rela-
tion to phonological coding and awareness in deaf readers: A meta-analysis. Journal of Deaf
Studies and Deaf Education, 16(2), 164–188. http://doi.org/10.1093/deafed/enq049
Mayberry, R. I., Lock, E., & Kazmi, H. (2002). Development: Linguistic ability and early lan-
guage exposure. Nature, 417(6884), 38–38. http://doi.org/10.1038/417038a
McQuarrie, L., & Parrila, R. (2014). Literacy and linguistic development in bilingual deaf chil-
dren: Implications of the “and” for phonological processing. American Annals of the Deaf,
159(4), 372–384.
Moeller, M. P. (2000). Early intervention and language development in children who are deaf
and hard of hearing. Pediatrics, 106(3), e43–e43. http://doi.org/10.1542/peds.106.3.e43
Mohammed, T., Campbell, R., MacSweeney, M., Milne, E., Hansen, P., & Coleman, M. (2005).
Speechreading skill and visual movement sensitivity are related in deaf speechreaders.
Perception, 34(2), 205–216. http://doi.org/10.1068/p5211
Moog, J. S., & Geers, A. E. (2010). Early educational placement and later language outcomes for
children with cochlear implants. Otology and Neurotology, 31(8), 1315–1319. http://doi.org/
10.1097/mao.0b013e3181eb3226
Mortensen, M. V., Mirz, F., & Gjedde, A. (2006). Restored speech comprehension linked to
activity in left inferior prefrontal and right temporal cortices in postlingual deafness.
NeuroImage, 31(2), 842–852. http://doi.org/10.1016/j.neuroimage.2005.12.020
Mueller, V., Sepulveda, A., & Rodriguez, S. (2013). The effects of baby sign training on child
development. Early Child Development and Care, 184(8), 1178–1191. http://doi.org/10.1080/
03004430.2013.854780
Muise-Hennessey, A., Tremblay, A., White, N. C., McWhinney, S. R., Zaini, W. H., Maessen,
H., et al. (2015). Age of onset and duration of deafness influence neuroplastic reorganization
of biological motion processing in deaf non-signers. Poster presented at the annual meeting
of the Organization for Human Brain Mapping, Honolulu, HI.
Muise-Hennessey, A., Tremblay, A., White, N. C., McWhinney, S. R., Zaini, W. H., Maessen,
H., et al. (2016). Age of onset and duration of deafness drive brain organization for biological
motion perception in non-signers. DalSpace. Halifax, NS, Canada. http://dalspace.library.
dal.ca/handle/10222/72221
Naito, Y., Okazawa, H., Honjo, I., Hirano, S., Takahashi, H., Shiomi, Y., et al. (1995). Cortical
activation with sound stimulation in cochlear implant users demonstrated by positron
emission tomography. Cognitive Brain Research, 2(3), 207–214. http://doi.org/10.1016/
0926-6410(95)90009-8.
Neville, H. J., Bavelier, D., Corina, D., Rauschecker, J. P., Karni, A., Lalwani, A., et al. (1998).
Cerebral organization for language in deaf and hearing subjects: Biological constraints and
effects of experience. Proceedings of the National Academy of Sciences USA, 95(3), 922–929.
http://doi.org/10.1073/pnas.95.3.922
Neville, H. J., & Lawson, D. (1987). Attention to central and peripheral visual space in a move-
ment detection task. III. Separate effects of auditory deprivation and acquisition of a visual
language. Brain Research, 405(2), 284–294.
Newport, E. L. (1990). Maturational constraints on language learning. Cognitive Science, 14(1),
11–28. http://doi.org/10.1207/s15516709cog1401_2
Niparko, J. K., Tobey, E. A., Thal, D. J., & Eisenberg, L. S. (2010). Spoken language development
in children following cochlear implantation. Journal of the American Medical Association,
303(15), 1498. http://doi.org/10.1001/jama.2010.451
Ohashi, J. K., Mirenda, P., Marinova-Todd, S., Hambly, C., Fombonne, E., Szatmari, P., et al.
(2012). Comparing early language development in monolingual-and bilingual-exposed
young children with autism spectrum disorders. Research in Autism Spectrum Disorders,
6(2), 890–897. http://doi.org/10.1016/j.rasd.2011.12.002
Okazawa, H., Naito, Y., Yonckura, Y., Sadato, N., Hirano, S., Nishizawa, S., et al. (1996).
Cochlear implant efficiency in pre-and postlingually deaf subjects. Brain, 119(4), 1297–1306.
http://doi.org/10.1093/brain/119.4.1297
Olulade, O. A., Koo, D. S., LaSasso, C. J., & Eden, G. F. (2014). Neuroanatomical profiles of
deafness in the context of native language experience. The Journal of Neuroscience, 34(16),
5613–5620. http://doi.org/10.1523/JNEUROSCI.3700-13.2014
Penhune, V. B., Cismaru, R., Dorsaint-Pierre, R., Petitto, L.-A., & Zatorre, R. J. (2003). The
morphometry of auditory cortex in the congenitally deaf measured using MRI. NeuroImage,
20(2), 1215–1225. http://doi.org/10.1016/S1053-8119(03)00373-2
Peters, B. R., Litovsky, R., Parkinson, A., & Lake, J. (2007). importance of age and
postimplantation experience on speech perception measures in children with sequential
bilateral cochlear implants. Otology and Neurotology, 28(5), 649–657, S1–S2. http://doi.org/
10.1097/01.mao.0000281807.89938.60
Petitto, L.-A., & Marentette, P. F. (1991). Babbling in the manual mode: Evidence for
the ontogeny of language. Science, 251(5000), 1493–1496. http://doi.org/10.1126/
science.2006424
366 Aaron J. Newman
Petitto, L.-A., Katerelos, M., Levt, B. G., Gauna, K., Tétreault, K., & Ferraro, V. (2001). Bilingual
signed and spoken language acquisition from birth: Implications for the mechanisms un-
derlying early bilingual language acquisition. Journal of Child Language, 28(2), 453–496.
http://doi.org/10.1017/S0305000901004718
Ponton, C. W., & Eggermont, J. (2002). Of kittens and kids: Altered cortical maturation fol-
lowing profound deafness and cochlear implant use. Audiology and Neuro-Otology, 6,
363–380.
Pyman, B., Blamey, P., Lacy, P., Clark, G., & Dowell, R. (2000). The development of speech
perception in children using cochlear implants: Effects of etiologic factors and delayed
milestones. The American Journal of Otology, 21(1), 57–61. http://doi.org/10.1590/
S0325-00752011000100012
Quittner, A. L., Cruz, I., Barker, D. H., Tobey, E., Eisenberg, L. S., & Niparko, J. K. (2013). Effects
of maternal sensitivity and cognitive and linguistic stimulation on cochlear implant users’
language development over four years. The Journal of Pediatrics, 162(2), 343–351. http://doi.
org/10.1016/j.jpeds.2012.08.003
Rouger, J., Fraysse, B., Deguine, O., & Barone, P. (2008). McGurk effects in cochlear-
implanted deaf subjects. Brain Research, 1188, 87–99. http://doi.org/10.1016/
j.brainres.2007.10.049
Rouger, J., Lagleyre, S., Fraysse, B., Deneve, S., Deguine, O., & Barone, P. (2007). Evidence
that cochlear-implanted deaf patients are better multisensory integrators. Proceedings
of the National Academy of Sciences USA, 104(17), 7295–7300. http://doi.org/10.1073/
pnas.0609419104
Rydberg, E., Gellerstedt, L. C., & Danermark, B. (2009). Toward an equal level of educational
attainment between deaf and hearing people in Sweden? Journal of Deaf Studies and Deaf
Education, 14(3), 312–323. http://doi.org/10.1093/deafed/enp001
Sarant, J. Z., Blamey, P. J., Dowell, R. C., Clark, G. M., & Gibson, W. P. R. (2001). Variation
in speech perception scores among children with cochlear implants. Otolaryngology,
22(1),  18–28.
Sarant, J. Z., Holt, C. M., Dowell, R. C., Rickards, F. W., & Blamey, P. J. (2008). Spoken language
development in oral preschool children with permanent childhood deafness. Journal of Deaf
Studies and Deaf Education, 14(2), enn034–217. http://doi.org/10.1093/deafed/enn034
Sarant, J., Harris, D., Bennet, L., & Bant, S. (2014). Bilateral versus unilateral cochlear implants
in children: A study of spoken language outcomes. Ear and Hearing, 35(4), 396–409.
Sharma, A., Campbell, J., & Cardon, G. (2015). Developmental and cross-modal plasticity in
deafness: Evidence from the P1 and N1 event related potentials in cochlear implanted chil-
dren. International Journal of Psychophysiology, 95(2), 135–144. http://doi.org/10.1016/
j.ijpsycho.2014.04.007
Sharma, A., Dorman, M. F., & Kral, A. (2005). The influence of a sensitive period on central
auditory development in children with unilateral and bilateral cochlear implants. Hearing
Research, 203(1–2), 134–143. http://doi.org/10.1016/j.heares.2004.12.010
Sharma, A., Dorman, M. F., & Spahr, A. J. (2002). A sensitive period for the develop-
ment of the central auditory system in children with cochlear implants: Implications
for age of implantation. Ear and Hearing, 23(6), 532–539. http://doi.org/10.1097/
01.AUD.0000042223.62381.01
Shein, J. (2003). At home among strangers: Exploring the Deaf community in the United States.
Washington, DC: Gallaudet University Press.
Shibata, D. K. (2007). Differences in brain structure in deaf persons on MR imaging studied

with voxel-based morphometry. American Journal of Neuroradiology, 28(2), 243–249.
Shiell, M. M., Champoux, F., & Zatorre, R. J. (2016). The right hemisphere planum temporale
supports enhanced visual motion detection ability in deaf people: Evidence from cortical
thickness. Neural Plasticity, 2016(6), 1–9. http://doi.org/10.1155/2016/7217630
Smith, K. M., Mecoli, M. D., Altaye, M., Komlos, M., Maitra, R., Eaton, K. P., et al. (2011).
Morphometric differences in the Heschl’s Gyrus of hearing impaired and normal hearing
infants. Cerebral Cortex, 21(5), 991–998. http://doi.org/10.1093/cercor/bhq164
Stevens, C., & Neville, H. (2006). Neuroplasticity as a double-edged sword: Deaf enhancements
and dyslexic deficits in motion processing. Journal of Cognitive Neuroscience, 18(5), 701–7 14.
Tait, M., Nikolopoulos, T. P., De Raeve, L., Johnson, S., Datta, G., Karltorp, E., et al. (2010).
Bilateral versus unilateral cochlear implantation in young children. International Journal of
Pediatric Otorhinolaryngology, 74(2), 206–211. http://doi.org/10.1016/j.ijporl.2009.11.015
Tobey, E. A., Thal, D., Niparko, J. K., Eisenberg, L. S., Quittner, A. L., & Wang, N.-Y. (2013).
Influence of implantation age on school-age language performance in pediatric cochlear
implant users. International Journal of Audiology, 52(4), 219–229. http://doi.org/10.3109/
14992027.2012.759666
Tremblay, C., Champoux, F., Lepore, F., & Théoret, H. (2010). Audiovisual fusion and cochlear
implant proficiency. Restorative Neurology and Neuroscience, 28(2), 283–291. http://doi.org/
10.3233/RNN-2010-0498
Truy, E., Deiber, M. P., Cinotti, L., Mauguiere, F., Froment, J. C., & Morgon, A. (1995). Auditory
cortex activity changes in long-term sensorineural deprivation during crude cochlear
electrical stimulation: Evaluation by positron emission tomography. Hearing Research,
86(1–2),  34–42.
Vlastarakos, P. V., Proikas, K., Papacharalampous, G., Exadaktylou, I., Mochloulis, G.,
& Nikolopoulos, T. P. (2010). Cochlear implantation under the first year of age: The
outcomes: A critical systematic review and meta-analysis. International Journal of Pediatric
Otorhinolaryngology, 74(2), 119–126. http://doi.org/10.1016/j.ijporl.2009.10.004
Vohr, B. (2003). Overview: Infants and children with hearing loss. Part I. Mental Retardation
and Developmental Disabilities Research Reviews, 9(2), 62–64. http://doi.org/10.1002/
mrdd.10070
Vos, T., Barber, R. M., Bell, B., Bertozzi-Villa, A., Biryukov, S., Bolliger, I., et al. (2015). Global,
regional, and national incidence, prevalence, and years lived with disability for 301 acute
and chronic diseases and injuries in 188 countries, 1990–2013: A systematic analysis for
the Global Burden of Disease Study 2013. The Lancet, 386(9995), 743–800. http://doi.org/
10.1016/S0140-6736(15)60692-4
Walker, E. A., & Bass-Ringdahl, S. (2008). Babbling complexity and its relationship to speech
and language outcomes in children with cochlear implants. Otology and Neurotology, 29(2),
225–229. http://doi.org/10.1097/mao.0b013e31815f6673
Wolfe, J., Baker, S., Caraway, T., & Kasulis, H. (2007). 1 year postactivation results for sequen-
tially implanted bilateral cochlear implant users. Otology and Neurotology, 28(5), 589–596.
http://doi.org/10.1097/mao.0b013e318067bd24
Wu, C.-C., Lee, Y.-C., Chen, P.-J., & Hsu, C.-J. (2008). Predominance of genetic diagnosis and
imaging results as predictors in determining the speech perception performance outcome
after cochlear implantation in children. Archives of Pediatrics & Adolescent Medicine, 162(3),
269–276. http://doi.org/10.1001/archpediatrics.2007.59
368 Aaron J. Newman
Wu, C.-M., Ko, H.-C., Tsou, Y.-T., Lin, Y.-H., Lin, J.-L., Chen, C.-K., et al. (2015). Long-term
cochlear implant outcomes in children with GJB2 and SLC26A4 mutations. PloS One, 10(9),
e0138575–13. http://doi.org/10.1371/journal.pone.0138575
Zeitler, D. M., Kessler, M. A., & Terushkin, V. (2008). Speech perception benefits of sequential
bilateral cochlear implantation in children and adults: A retrospective analysis. Otology and
Neurotology, 29(3), 314–325. http://doi.org/10.1097/mao.0b013e3181662cb5^#
Pa rt   I I I
A RT IC U L AT ION A N D
P RODU C T ION
Chapter 15
Neu romotor
Organiz ation of Spe e c h
Produ ct i on
Pascale Tremblay, Isabelle Deschamps,

and Anthony Steven Dick
Introduction
The act of speaking is a complex and integrative behavior, which associates high-level
cognitive and linguistic processes with fine motor-control mechanisms. Speaking
begins with an intention to communicate, continues to the translation of the message
into lexical units (words), which in turn need to be broken down into sequences of tem-
porally ordered syllables and adjusted to the context in terms of rhythm, loudness, and
prosody. The final stage requires the coordination of multiple sensorimotor systems, in-
cluding the respiratory system, which provides the airflow necessary to set the vocal
folds into vibrations, the intrinsic and extrinsic laryngeal muscles that convert this air-
flow into phonation, and, finally, the supralaryngeal muscles that modulate the config-
uration of the vocal tract to convert the laryngeal output into vowels and consonants
(articulation). Speaking also requires the interaction of sensorimotor mechanisms with
linguistic processes including grammatical, syntactic, and semantic processes, as well
as cognitive/executive processes such as verbal memory and audiovisual attention. In
spite of this remarkable complexity, adult speakers are able to produce as many as six
to nine syllables per second (Kent, 2000). Perhaps given this extraordinary speed and
complexity, the manner in which this process is accomplished at the neural level has not
been fully elucidated. Historical models of spoken language production have focused
on a single region for speech production (“Broca’s area”) and a single region for com-
prehension (“Wernicke’s area”), and a single white matter pathway connecting these re-
gions (the arcuate fasciculus). The complexity of the sensorimotor processes supporting
372 Pascale Tremblay, Isabelle Deschamps, and Anthony Steven Dick
articulation has thus been minimized, and is often represented as a “final-output

stage” black box within serial and compartmentalized models of language (for a dis-
cussion of this topic, see Tremblay & Dick, 2016). More recently, advances in brain im-
aging and brain stimulation techniques have contributed tremendously to uncovering
the sophisticated neural system that underlies the production of speech. The seminal
meta-analysis conducted by Indefrey and Levelt (2004) was among the first to high-
light the complexity of the cortical network underlying word production. More recent
analyses of functional connectivity support the notion of a highly distributed network
(Simonyan, Ackermann, Chang, & Greenlee, 2016; Simonyan & Fuertinger, 2015). Here
we propose, in Figure 15.1, an overview of the cortico-subcortical network that underlies
speech production from phonological planning to articulation. In this chapter, we re-
view and discuss the most up-to-date knowledge on speech production, from phono-
logical planning to motor execution, with an emphasis on functional neuroanatomy
and anatomical connectivity.
Phonological Planning
for Speech Production
Phonological planning refers to the specification of the abstract phonological

representations that are used during speech production. It is preceded by conceptual,
lexical, semantic, and syntactic processes, and followed by motor planning, thereby
representing an interface between the cognitive and the motor dimensions of speech
production. The mechanisms underlying phonological planning have been studied in-
tensively (Cholin, Levelt, & Schiller, 2006; Costa & Caramazza, 2002; Damian & Dumay,
2007; Dell & Reich, 1981; Jescheniak, Schriefers, Garrett, & Friederici, 2002; Meyer, 1990,
1991; Roelofs, 1999; Schiller, Bles, & Jansma, 2003; Schnur, Costa, & Caramazza, 2006;
Shattuck-Hufnagel & Klatt, 1979; Treiman, 1985; Wheeldon & Lahiri, 1997). According
to several prominent models, the phoneme and the syllable are the basic phonological
units necessary to produce fluent speech (Bohland, Bullock, & Guenther, 2010; Dell,
1986; Indefrey & Levelt, 2004; Levelt, 1992, 1999; Levelt & Wheeldon, 1994; MacNeilage,
1998; Roelofs, 1997; Tourville & Guenther, 2010). During phonological planning, pho-
nological encoding processes are called upon to build a word’s phonemic and syllabic
codes from abstract linguistic representations containing conceptual, semantic, and
syntactic information.
Most models of speech production assume that phonological planning involves the
serial retrieval of phonemic-and syllabic-level information for items that are to be
produced (e.g., Bohland et al., 2010; Hickok, 2014; Indefrey & Levelt, 2004; Van der
Merwe, 2009). Indefrey and Levelt (2004) propose that the phonological word forms
are built online through a segment-by-segment internal syllabification process, a pro-
cess associated with the left inferior frontal gyrus (IFG). According to the GODIVA
Cerebral cortex
STGp pre-
IFG SMA MFG
PMv SMA CMA

Somato-
sensory
areas Insula
M1v
Basal ganglia
Globus Striatum
pallidus ext
Thalamus
Subthalamic
nuclei Cerebellum
Globus
pallidus int/
substantia nigra
Midbrain
Sensory Mesencephalic
receptors in CN V trigeminal
the face, tongue, nucleus
larynx, pharynx
and abdomen
(respiration)
CN V Pontine Brainstem
cranial
Reticular formation
nerve
Red nucleus
CN V, VII nuclei
Muscle
contractions Medullar
CN IX, X, XI, XII cranial
(for respiration,
phonation and nerve
CN IX, X
articulation) nuclei
Spinal
CN V, VII, IX, X trigeminal
nucleus
Upper cervical
CN IX, X, XI, XII spinal cord
Figure 15.1. Schematic representation of the neurocognitive organization of speech produc-

tion. Blue lines denote connections predominantly conveying motor information. Orange lines
denote connections predominantly conveying sensory information. Purple denotes connections
predominantly conveying cognitive/ linguistic (higher-
order) information. For simplicity,
the thalamus, cerebellum, and striatum were not subdivided into their subcomponents, and
connections between specific cortical areas and these structures are also not included. Lines with
arrows indicate the direction of the information flow. Lines without arrows indicate bidirectional
connections.
Abbreviations: Globus pallidus ext: external globus pallidus; Globus pallidus int: internal globus pallidus;
STGp: posterior superior temporal gyrus; PMv: ventral premotor cortex; CN: cranial nerve; IFG: inferior frontal gyrus;
SMA: supplementary motor area; CMA: cingulate motor area; MFG: middle frontal gyrus.
model (gradient order DIVA) (Bohland et al., 2010)—an extension of the well-known
DIVA ((Directions Into Velocities of Articulators)) model—phonemic codes and syl-
labic position specification are computed within the left inferior frontal sulcus, whereas
syllabic frame specifications (i.e., abstract structure above the phonemic level) are associ-
ated with the pre-supplementary motor area (Bohland et al., 2010; Guenther, 1995, 2006;
Guenther, Ghosh, & Tourville, 2006b; Tourville & Guenther, 2011). The notion that pho-
nological planning is accomplished in a two-step fashion (operating serially or in parallel,
depending on the speech production model) is supported by a few functional neuroim-
aging studies that have focused either on specific phonological representations or on the
mechanisms involved in building a word’s phonological form. For instance, in a recent
functional magnetic resonance imaging (fMRI) experiment, Markiewicz and Bohland
(2016) investigated, using multi-voxel pattern analysis (MVPA), whether the response
patterns (i.e., hemodynamic response) of specific cortical areas predicted the linguistic
class (i.e., phonemes and syllables) of the stimuli during the production of Consonant-
Vowel-Consonant (CVC) syllables. The authors identified numerous regions spanning
across both the left and right hemispheres in which predictive phoneme and\or syllable
information was found. Of particular interest is the finding that the left inferior frontal
sulcus was among the regions whose activation pattern was predictive of phoneme level
information, consistent with GODIVA (Bohland et al., 2010). Syllable-level predictive
information was found in numerous regions, including the left ventral premotor cortex
(PMv) and motor cortex, as well as the left posterior superior temporal gyrus (STGp),
and the right posterior superior temporal sulcus. Similarly, Peeva and colleagues (2010),
using an fMRI repetition-suppression paradigm (Grill-Spector, Henson, & Martin, 2006;
Grill-Spector & Malach, 2001), also identified the left PMv as a region that responds to
syllabic complexity during speech-production, suggesting a role for this region in the
encoding of entire syllables, perhaps through speech sound maps that represent syllabic
motor programs, as proposed in the DIVA model (Guenther, Ghosh, & Tourville, 2006a).
The authors also identified regions that specifically responded to phonemic complexity
(the left supplementary motor area, the left globus pallidus, the left posterior superior
temporal gyrus, and the left superior posterior lateral cerebellum).
Other functional neuroimaging studies have manipulated psycholinguistic variables
during speech production (i.e., picture naming, reading, and spoken word/pseudo-
word repetition) to target phonological encoding mechanisms, including phonolog-
ical neighborhood density, and syllable and word frequency (Acheson, Hamidi, Binder,
& Postle, 2011; de Zubicaray, McMahon, Eastburn, & Wilson 2002; Indefrey & Levelt,
2000; Papoutsi et al., 2009; Peramunage, Blumstein, Myers, Goldrick, & Baese-Ber, 2011;
Vihla, Laine, Matti, & Salmelin, 2006). These studies have found that one region—the
posterior superior temporal cortex (including the posterior segment of both the supe-
rior and middle temporal gyri)—is sensitive to these manipulations, suggesting a role
for this region in phonological planning (i.e., mechanisms involved in encoding a word’s
phonological form).
In sum, neuroimaging experiments suggest that both the syllable and the phoneme are
used during phonological planning for speech production. While many issues remain
Neuromotor Organization of Speech Production 375
regarding the networks involved with the implementation of syllabic-and phonemic-

level processes during phonological planning in speech production, empirical evidence
suggests that regions distributed across the frontal and temporal lobes are involved in
phonological planning. Recent promising developments of neurocomputational models
of speech production (e.g., Bohland et al., 2010; Hickok, 2012) will allow researchers
to test specific predictions regarding the implementation of phonological planning
mechanisms.
Motor Planning/P rogramming

Speech motor planning builds upon phonological planning and possibly proceeds si-
multaneously. There is general agreement that the output of phonological encoding
is a phonological word, in which metrical, syllabic, and segmental properties are
specified. However, models of speech production differ in terms of the output of pho-
nological planning, with some positing that the phonological form is pre-syllabified,
and others postulating that syllabification occurs within the sensorimotor system. In
either case, motor preparation for speech involves several steps, including response
selection, motor timing, and sequencing processes. This speech motor preparation
process, often globally referred to as “supra-motor” or “motor cognition” (Freund,
Jeannerod, Hallett, & Leiguarda, 2005), may not be speech specific and may have
evolved from more general motor functions. Thus, the system “[ . . . ] gets what service
it can out of organs and functions, nervous and muscular, that have come into being
and are maintained for very different ends than its own” (Sapir, 1921, p. 8). Several
steps in this motor-planning process are specified in specific neural systems, which
are reviewed in the following paragraphs. There is a growing consensus in the field of
speech motor control that speech motor programs for well-practiced sounds are stored
within the left PMv, as proposed in the DIVA model (Guenther, 2006). Consistent
with this idea, it was recently shown, using fMRI, that the process of learning new
speech sequences is associated with a decline in activation in the left PMv, suggesting
that individual motor programs become merged into one, thus requiring the activa-
tion of only one motor program (Segawa, Tourville, Beal, & Guenther, 2015). Similarly,
a recent electrocorticographic study conducted in awake patients showed that tissue
adjacent to the PMv in the left IFG is active early on during auditory repetition, pos-
sibly reflecting the activation of speech motor plans (Flinker et al., 2015). It was found
that activity was stronger for pseudo-words, which are not associated with motor
programs, in contrast to real words. The contribution of the PMv and IFG to speech
production has been notoriously difficult to untangle. In the Flinker study, activity
in the precentral sulcus containing the PMv was not reported, leaving the question
unanswered.
Motor Response Selection

The neural mechanism by which a phonological form is transformed into stored motor
routines is often referred to as response selection, or sometimes, action selection. Though
it is still not included in most neurobiological models of spoken-language production,
motor response selection is a well-documented mechanism in the non-speech motor-
control literature. Several neuroimaging studies have examined the process of selecting
non-speech motor responses, including finger and hand movements. These studies
have shown activation in the pre-supplementary motor area (pre-SMA) (Brodmann’s
medial area 6) that varied as a function of response selection modality (voluntary se-
lection vs. forced selection) (Deiber, Ibanez, Sadato, & Hallett, 1996; Lau, Rogers, &
Passingham, 2006; Weeks, Honda, Catalan, & Hallett, 2001). The pre-SMA is a region
located in the medial aspect of the superior frontal gyrus, immediately anterior to the
SMA and corresponding to the anterior part of Brodmann’s medial area 6. The pre-
SMA is densely connected to the prefrontal cortex, particularly the dorsolateral pre-
frontal cortex in the middle frontal gyrus (MFG) (Lu, Preston, & Strick, 1994; Luppino,
Matelli, Camarda, & Rizzolatti, 1993), and to several non-primary motor areas, such
as the SMA-proper and the PMv (Luppino & Rizzolatti, 2000), for controlling motor
output. Recent data suggest that the pre-SMA is also connected to the posterior part
of the inferior frontal gyrus through the frontal aslant tract (FAT) (Catani et al., 2012a;
Ford, McGregor, Case, Crosson, White, 2010, & Dick et al., 2018). Thus, the pre-SMA
has a connectivity pattern that is ideal for linking higher-level cognitive functions (in-
cluding linguistic processes) and motor processes, which is critical to implementing
response selection. In agreement with the literature on motor cognition, a number
of fMRI studies have shown that manipulating response selection during word pro-
duction modulates the pre-SMA, the neighboring cingulate motor area (CMA), and
the PMv (Alario, Chainay, Lehericy, & Cohen, 2006; Crosson et al., 2001; Tremblay
& Gracco, 2006; Tremblay & Small, 2011). This was also found during the selection of
non-speech orofacial movements (Braun, Guillemin, Hosey, & Varga, 2001; Tremblay
& Gracco, 2010). There is also evidence from neurostimulation studies that noninvasive
transcranial magnetic stimulation (TMS; see Schuhmann, Chapter 5 in this volume)
to the pre-SMA impairs the voluntary selection of words and non-speech orofacial
movements (Tremblay & Gracco, 2009). Taken together, these results suggest that the
pre-SMA plays a central role in selecting motor responses for speech production and,
more generally, for all voluntary actions.
In addition to a key role for the pre-SMA in response selection, recent fMRI evi-
dence suggests a role for the striatum, and more specifically the caudate nucleus, in re-
sponse selection during speech production (Argyropoulos, Tremblay, & Small, 2013).
Consistent with this observation, it has been suggested that the caudate is involved
in speech planning, and more specifically sequencing speech motor responses, re-
ceiving input from both the inferior frontal sulcus and the pre-SMA (Bohland et al.,
2010). The caudate also has been found to be activated in memory-driven manual
sequencing tasks (Menon, Anagnoson, Glover, & Pfefferbaum, 2000), and during
sequence learning (Bischoff-Grethe, Goedert, Willingham, & Grafton, 2004), which

suggests the region has a role in higher- order cognitive/ executive aspects of move-
ment preparation role in higher-order cognitive/executive aspects of movement
preparation. The caudate has a connectivity pattern consistent with a role in motor
planning rather than execution, as it is connected to prefrontal structures and not to
the primary motor area (Alexander, DeLong, & Strick, 1986; Hoover & Strick, 1999;
Postuma & Dagher, 2006).
Speech Motor Sequencing

One of the most distinctive features of spoken-language production is its serial
ordering, that is, the organization of speech movements into precise, smooth, and co-
ordinated temporal sequences of movements of the lips, tongue, and jaw (Lashley, 1951).
Indeed, normal communicative speech consists of a continuous, connected stream of
sounds. Motor sequencing is the planning of the specific sequences of movements re-
quired to produce smooth and co-articulated temporal sequences of movements of
the lips, tongue, and jaw. Sequencing mechanisms are closely related to motor-timing
mechanisms (see earlier discussion), which involve initiating movement sequences
and timing each movement within a sequence. The production of streams of speech
sounds thus requires the close coordination of sequencing and timing mechanisms.
Neuroimaging and neuromodulation evidence shows that, for manual movements,
motor sequencing is implemented in a complex and distributed neural network
involving cortical premotor areas (SMA-proper, PMv), the cerebellum, and the striatum
(caudate and putamen) (Bengtsson, Ehrsson, Forssberg, & Ullen, 2005; Gerloff, Corwell,
Chen, Hallett, & Cohen, 1997; Macar et al., 2002); for example, repetitive TMS of the
SMA-proper results in sequential timing disruptions in a complex finger-movement
task (Gerloff et al., 1997). SMA-proper activation also occurs in tasks requiring the pro-
cessing of temporal patterns (Macar et al., 2002). Consistent with the idea of a role for
the striatum and other basal ganglia structures in motor sequencing, it has been shown
that patients with basal ganglia disorders, such as Parkinson’s disease (PD), a neuro-
degenerative disorder characterized by progressive loss of dopaminergic neurons in
the substantia nigra, have difficulty with switching rapidly from one movement to an-
other, reflecting a deficit in the initiation and termination of sequential movements
(e.g., Agostino, Berardelli, Formica, Accornero, & Manfredi, 1992; Benecke, Rothwell,
Dick, Day, & Marsden, 1987). It is also suspected that FOXP2, the transcription factor
associated with impairments in articulation (Fisher & Scharff, 2009; Vargha-Khadem,
Gadian, Copp, & Mishkin, 2005), affects these cortico-basal ganglia circuits (Reimers-
Kipping, Hevers, Pääbo, & Enard, 2011).
For speech sequencing more specifically, a broad network appears to be involved
in the process of producing sequential speech (Bohland & Guenther, 2006; Shuster
& Lemieux, 2005). For example, using fMRI, Bohland and Guenther (2006) showed
that several cortical regions, including the PMv, the inferior parietal lobule (IPL), the
inferior frontal sulcus, the SMA, and the pre-SMA, are sensitive to serial complexity in
speech-sequence production. Peeva et al. (2010) further demonstrated a contribution of
the right superior lateral cerebellum to syllable sequencing.
Other studies have shown that the anterior insula, which is functionally connected to the
striatum (Postuma & Dagher, 2006), is also involved in motor timing/sequencing during
finger-tapping tasks (Bengtsson, Ehrsson, Forssberg, & Ullen, 2004; Lewis & Miall, 2002).
Consistent with these findings, Dronkers and colleagues (Dronkers, 1996; Ogar et al.,
2006) have shown, using voxel-based morphometry, that lesions to the anterior insula are
related to speech apraxia, a disorder of speech planning that affects speech sequencing.
Though it is unlikely that the anterior insula is involved in either speech-specific or
motor-specific processes, it is a region that is active during demanding speech tasks (e.g.,
Ackermann & Riecker, 2004; Bilodeau-Mercure, Lortie, Sato, Guitton, & Tremblay, 2015;
Bohland & Guenther, 2006; Peeva et al., 2010). It has been suggested that the anterior in-
sula contributes to the attention-orientation system (Corbetta, Patel, & Shulman, 2008),
as well as to general executive processes involved during goal-oriented tasks (Nelson et al.,
2010). It is therefore possible that the insula provides the attentional control necessary to
produce precisely ordered and timed sequences, which is cognitively demanding.
In sum, while there remain several questions regarding the implementation of selec-
tion and sequencing mechanisms for speech, the available empirical evidence, though
limited, suggests that sequencing for speech production relies largely upon common
action-control mechanisms involving the pre-SMA, SMA, PMv, insula, cerebellum, and
striatum.
Speech Motor Timing
Motor timing refers to the ability to initiate movement sequences and to time each
sub-movement within a sequence. It occurs at the interface of sequencing mechanisms
and motor execution. For speech, initiation corresponds to the moment at which air
is expelled from the lungs; it is followed by phonation, articulation, and resonance
(opening or closing of the nasal cavity by the soft palate to produce nasal and oral sounds,
respectively). Several regions appear to participate in motor timing, including the
SMA-proper, the pre-SMA, the basal ganglia, and the prefrontal cortex. Unfortunately,
though, there is little neuroimaging data focusing specifically on the neural basis of
speech motor timing. An important distinction for both the neural control systems and
for assessing speech motor disorders is the manner in which movements are initiated,
whether externally by sensory events, or by internal events. Motor responses initiated
by external stimuli are associated with activity in the SMA-proper (Lee, Chang, &
Roh, 1999; Thickbroom et al., 2000; Wiese et al., 2004), as well as in the left dorsal PM
(PMd) (Krams, Rushworth, Deiber, Frackowiak, & Passingham, 1998; Lepage et al.,
1999; Weeks et al., 2001). In humans, inhibition of the left PMd results in a response
delay in an externally triggered choice reaction-time task, which suggests a role in the
initiation of movements (Schluter, Rushworth, Passingham, & Mills, 1998). The direct
comparison of self-initiated and externally triggered finger movements reveals acti-

vation in the pre-SMA (Deiber, Ibanez, Sadato, & Hallett, 1996; Jenkins, Jahanshahi,
Jueptner, Passingham, & Brooks, 2000; Tsujimoto, Ogawa, Tsukada, Kakiuchi, & Sasaki,
1998), which suggests that the pre-SMA may be generating internal triggers to produce
actions. Using both fMRI and electroencephalography (EEG), it has been shown that
the pre-SMA/SMA-proper is involved in preparing sequences of finger movements
with high demands on ordering (sequencing), while the dorsolateral prefrontal cortex
is involved in preparing sequences with high demands on temporal precision and
timing, suggesting distinct roles for these regions in movement preparation (Bortoletto
& Cunnington, 2010). Comparing self-initiated sequences to sequences in which the
internal order of the sub-movements was varied also points to a role for the dorsolat-
eral prefrontal cortex in initiation and the SMA for sub-movement timing (Bortoletto,
Cook, & Cunnington, 2011). These findings are supported by TMS evidence suggesting
a role for the right dorsolateral prefrontal cortex in temporal preparation for finger
movements (Vallesi, Shallice, & Walsh, 2007). The right dorsolateral prefrontal cortex
in the middle frontal gyrus is also thought to be part of the speech preparation network
(Brendel et al., 2010; Garnier et al., 2013; Riecker et al., 2005). Consistent with evidence
from finger sequence paradigms, fMRI research on speech motor preparation, though
limited, suggests that several regions are involved, including the pre-SMA and SMA-
proper, the bilateral putamen, the bilateral thalamus, the bilateral anterior insula, and
the right dorsolateral prefrontal cortex (Brendel et al., 2010). In a recent study by Long
and colleagues (Long et al., 2016), a promising cortical cooling approach was used to
study speech production in awake participants awaiting brain surgery. The results show
that cooling of the left IFG leads to slower speech, while cooling of the primary motor
cortex (M1v) leads to articulation errors. Based on these results, the authors suggested a
role for the left IFG in the control of speech timing. A role for the left IFG in the planning
of speech more generally was also suggested by recent electrocorticographic recordings,
which demonstrated early activity in this region during an auditory word-repetition
task, which was followed by activity in M1v (Flinker et al., 2015).
There is also converging clinical and imaging data for a role for the basal ganglia in
motor timing, particularly movement initiation. In patients with Parkinson’s disease,
there is a decline in the ability to initiate movements at will. However, externally trig-
gered actions are preserved (Cunnington, Iansek, & Bradshaw, 1999; Freeman, Cody,
& Schady, 1993; Praamstra, Stegeman, Cools, Meyer, & Horstink, 1998), which suggests
a role in movement initiation for the basal ganglia, directly or through its connections
with prefrontal and premotor areas. This notion is supported by fMRI studies comparing
self-initiated to externally paced finger-movement sequences, which show activity in the
left anterior putamen for self-initiated, but not externally paced, movements (Boecker,
Jankowski, Ditter, & Scheef, 2008). For speech, Brendel et al. (2010) have shown activa-
tion of the bilateral anterior putamen during speech preparation. There is also evidence
that basal ganglia dysfunction (including Parkinson and Huntington diseases) is asso-
ciated with abnormal speech movement timing, including impaired movement dura-
tion (Ludlow, Connor, & Bassich, 1987), rate (Skodda, 2011; Skodda & Schlegel, 2008;
Volkmann, Hefter, Lange, & Freund, 1992), and pace stability (Schmitz-Hubsch, Eckert,
Schlegel, Klockgether, & Skodda, 2012; Skodda, 2011; Skodda, Flasskamp, & Schlegel,
2010, 2011; Skodda & Schlegel, 2008). The available literature therefore suggests a role
for medial premotor areas (both pre-SMA and SMA-proper), prefrontal cortex, and the
basal ganglia in the timing of speech actions.
Speech Motor Programming

Once selected, ordered, and precisely timed, speech movements need to be fine-tuned,
which includes adjustments of velocity, muscle tone, movement range, and direction, all
accomplished within specific phonetic, environmental, emotional, and social contexts.
The sensory systems provide information about the initial conditions such that the
motor commands for a desired outcome can be successfully achieved given the state of
the vocal tract. In each language, a closed set of syllables and phonemes is available. These
syllables and phonemes are stored as a set of sensorimotor routines with invariant goals
and include information about the proprioceptive, tactile, and auditory feedback asso-
ciated with each movement. Nevertheless, adjustments to these routines are constantly
needed because phonemes, syllables, and words are never produced in an identical
manner. Indeed, one important notion for speech production is that of motor equiva-
lency, which is the capacity to achieve a movement goal in various ways, from different
starting points and under different conditions. A related notion is that of co-articulation,
which is the temporal-spatial overlap of movements associated with the production of
more than one sound occurring at a single point in time (Liberman, Harris, Hoffman, &
Griffith, 1957; MacNeilage & DeClerk, 1969). In every language, phonemes in the speech
stream have a strong effect on the phonemes that are produced close to them (both be-
fore or after) leading to anticipatory and carryover co-articulation effects. This suggests
either that motor commands for adjacent vowels and consonants are processed simul-
taneously, or that plans for moving the articulators (tongue, lips, soft palate, mandible)
from one position to the other are established in advance. Because of the speed at which
speech is produced, and the difficulty in dissociating motor programming from motor
planning steps with current neuroimaging and neuromodulation techniques, little is
known about the neural basis of co-articulation processes. According to the model de-
veloped by Van Der Merwe (2009), brain regions involved in speech motor program-
ming would include the cerebellum, SMA-proper, M1, and the basal ganglia (especially
the putamen), but empirical evidence still awaits, and research is ongoing to provide
additional information about the neural underpinning of these important processes.
Speech Movement Execution
Once motor planning and programming is terminated and a trigger for initiation has
been generated, the execution of speech movements begins. Understanding the neural
basis of speech movement production—including both phonation and articulation—

requires an understanding of the central and peripheral nervous system because both
are essential to produce speech. Indeed, in addition to the complex organization in
the central nervous system (CNS) that we have touched upon in the previous sections,
the production of speech sounds requires the orchestration of ~100 striated and vis-
ceral muscles located in the abdomen, neck, larynx, pharynx, and oral cavity, which are
innervated through six pairs of cranial nerves.
Motor commands for speech are sent to the ventral part of M1v, sometimes referred
to as the final common pathway, which contains (most of) the neurons controlling the
vocal apparatus (Penfield & Boldrey, 1937). From M1v, the motor commands are sent
toward the relevant muscles through the pyramidal system. The pyramidal system is
one of the most important efferent pathways for the control of voluntary movements,
including speech. It includes two major divisions: the corticospinal and corticobulbar
tracts. For speech, the primary division is the corticobulbar one, but both divisions play
a part. While the corticospinal tract innervates motor nuclei located in the spinal cord,
the corticobulbar tract innervates motor nuclei located in the brainstem. Motor nuclei
are collections of neuron bodies in the brainstem and spinal cord that are associated
with one or more cranial or spinal nerve nerves, respectively. The pyramidal system
serves to connect neurons in the cortex, called the upper motor neurons (UMNs), to
neurons located in the brainstem (forming the corticobulbar tract) and spinal cord
(forming the corticospinal tract), which are called lower motor neurons (LMNs). That
is, UMNs in the cortex do not directly connect to the muscles. It is the LMNs that inner-
vate the muscle fibers located in the face, neck, and abdomen.
M1 is the cortical area that contains the largest number of pyramidal neurons
(Kuypers, 1973; Murray & Coulter, 1981; Ralston & Ralston, 1985), in particular, the giant
Betz cells located in cortical layer V. However, it is not the only cortical area that projects
to LMNs. Indeed, anatomical studies have shown that other cortical areas contain a high
density of neurons directly projecting to LMNs (for a review of the connectivity of non-
primary motor areas, see Picard & Strick, 1996, 2001). These are the same regions that
are directly connected to M1 through long and short association fibers: the SMA (Dum
& Strick, 1991, 1996; Muakkassa & Strick, 1979), the CMA located just beneath the SMA
on the dorsal and ventral banks of the cingulate sulcus (Dum & Strick, 1991; Muakkassa
& Strick, 1979), the PMd and the PMv (Barbas & Pandya, 1987; He, Dum, & Strick, 1993).
For example, electrical stimulation of the SMA (Fried et al., 1991; Penfield & Welch, 1951;
Talairach & Bancaud, 1966) and CMA (von Cramon & Jurgens, 1983) induces vocali-
zation and speech arrest in humans, suggesting a role in the control of phonation and
articulation for these regions. This means that premotor areas have some level of di-
rect control over the generation and control of movement, independently of M1v (Dum
& Strick, 1991). Axons originating from M1v, SMA, CMA, and PMv are arranged in a
fan-shaped collection of axons that is directed downward toward the brainstem. At
the level of the thalamus and basal ganglia, the axons are compacted into a dense band
of axons, which is known as the internal capsule and within which axons originating
from different locations are segregated (Beevor & Horsley, 1890; Dejerine, 1901). Most
corticobulbar axons are located in the genu of the internal capsule, while corticospinal
axons are located in the posterior limb of the internal capsule. Because all the projections
to and from the cortex go through the internal capsule, even small lesions can produce
widespread motor deficits, including speech deficits, particularly if the lesion occurs in
the genu of the internal capsule. Most pyramidal axons cross from one hemisphere to
the other before entering the spinal cord at the level of the medulla oblongata (i.e., the
pyramidal decussation). Corticobulbar axons also cross at the level of the brainstem,
though there is substantial bilateral innervation of the cranial nerve (CN) motor nuclei,
making the speech system fairly resistant to unilateral UMN damages. The exceptions
include contralateral innervation of ventral cell groups of the motor nucleus of the facial
nerve (CN VII), and the hypoglossal nucleus (CN XII).
The production of speech and voice depends upon the integrity of both the
corticospinal tract, which innervates the muscles of respiration located in the ab-
domen, neck, and shoulder, and the corticobulbar tract, for the innervation of laryn-
geal and supra-laryngeal muscles (for reviews, see Jurgens, 2002, 2009). Six pairs of
cranial nerves are involved: (1) the trigeminal nerve (CN V), which controls the mus-
cles of mastication (jaw) (important for articulation) and carries general sensory in-
formation from the pharynx and anterior two-third of the tongue; (2) the facial nerve
(CN VII), which controls muscles of facial expression (e.g., those controlling the lips)
and eye movements; (3) the glossopharyngeal nerve (CN IX), which controls the
stylopharyngeus muscle (which elevates the pharynx during speech and swallowing)
and carries general sensory information from the face and posterior one-third of the
tongue; (4) the vagus nerve (CN X), which controls muscles of the larynx and pharynx
and carries general sensory information from the larynx; (5) the accessory nerve (CN
XI), which controls the sternomastoid (head rotation and chin elevation) and trapezius
muscle; and, finally, (6) the hypoglossal nerve (CN XII), which controls intrinsic and ex-
trinsic tongue muscles (except the palatoglossus).
Another peculiarity of the speech system is that some of its components are under
dual control (voluntary and autonomous). The autonomous pathway involves the ante-
rior cingulate cortex (ACC), the periaqueductal gray matter (PAG), and the reticular for-
mation (Jurgens, 2002, 2009; von Cramon & Jurgens, 1983). This pathway is important
for the innate control of nonverbal and emotional vocalizations (e.g., crying, laughing,
and moaning) (Scheiner, Hammerschmidt, Jurgens, & Zwirner, 2004). Vocalizations
can also occur without a cortical intervention through the reticular formation’s connec-
tion to brainstem motoneurons. Interestingly, unlike humans, other mammals lack a
direct control of M1 on laryngeal motoneurons. In mammals, M1 connects to the laryn-
geal motoneurons indirectly through the reticular formation (Jurgens, 2009; Simonyan
& Jurgens, 2003). In addition to these two pathways, it is worth mentioning that there
are other connections, often called extrapyramidal or indirect, between the cortex,
the reticular formation, and laryngeal motoneurons in the brainstem (see Figure 15.1
for an overview of these connections). These extrapyramidal pathways link the cortex,
including M1v, premotor, and sensory areas, with brainstem nuclei, in particular the
reticular formation (through the corticoreticular tract), which in turn modulate la-
ryngeal motoneurons in the brainstem. Though the roles of these pathways have not
been completely elucidated, they would be involved in controlling posture and muscle
tone, thus providing a framework upon which voluntary speech movements can take
place. They may also play a role in coordinating reflect activities within different cranial
nerves.
Sensorimotor Integration
Sensory and motor systems for speech are constantly interacting, and sensorimotor inte-
gration is key to successful and efficient speech production. There is strong empirical ev-
idence that the unanticipated alteration of auditory and somatosensory feedback during
speech production can lead to compensatory movement adjustments, which show the im-
portance of sensory feedback for speech motor control. Indeed, compensatory movements
have been shown following unanticipated alteration of auditory feedback for pitch (Burnett,
Freedland, Larson, & Hain, 1998; Jones & Munhall, 2000), and vowel formants (Purcell &
Munhall, 2006a, 2006b), as well as for unexpected perturbation of lip or jaw movements
(Abbs, Gracco, & Cole, 1984; Ito, Kimura, & Gomi, 2005; Tremblay et al., 2003).
Though sensory feedback is important for speech motor control, if all movement
adjustments depended solely on afferent sensory (external) feedback, speech motor con-
trol would be inefficient because of the delay involved in receiving and processing this in-
formation as it takes about 100 milliseconds for the auditory signal to reach the auditory
association areas. The speech production system therefore also relies upon predictive
feedforward processes for movement control. When a speech act is ready to be produced,
a copy of the prepared motor commands, the efference copy, or corollary discharge
(Sperry, 1950; Von Holst & Mittelstaedt, 1973), is sent to an internal model of the vocal
tract to predict movement sensory consequences, while in parallel a copy is sent through
the descending pathway to the muscles involved. The efference copy signal inhibits acti-
vation of the STGp, which results in a phenomenon referred to as speech-induced suppres-
sion in which the auditory cortex responds less to speaking than to hearing a recording
of a similar utterance (Christoffels, van de Ven, Waldorp, Formisano, & Schiller, 2011;
Eliades & Wang, 2003; Houde, Nagarajan, Sekihara, & Merzenich, 2002; Meekings et al.,
2016). The internal model is useful for online movement control because the sensory
consequences of a movement command can be evaluated and corrected even before ex-
ternal feedback has reached the central nervous system. When a mismatch is detected
between expected and planned sensory consequences, an error signal is generated.
fMRI studies of altered auditory feedback have shown an increased activation in the su-
perior temporal cortex in the presence of altered feedback compared to unaltered feed-
back (Christoffels et al., 2011; Meekings et al., 2016; Tourville, Reilly, & Guenther, 2008),
suggesting that the error signal is generated in the superior temporal cortex. Similar
mechanisms would exist for the somatosensory modality as well. According to Houde
and Nagarajan (2011), the signal to correct the motor commands is sent from the tem-
poral cortex to M1v through PMv, which is connected to both somatosensory areas,
including area SII (Matelli, Camarda., Glickstein, & Rizzolatti, 1986) and the STGp
(Chavis & Pandya, 1976; Schmahmann et al., 2007).
Most models of speech production acknowledge a role for both external and in-
ternal feedback in speech motor control. External feedback is critical to learn the re-
lationship between motor commands and their sensory consequences (the internal
model), to continuously update the internal model and to detect and correct for
sudden perturbations (Hickok, 2014), while internal feedback is critical for fast online
adjustments.
White Matter Cortical and

Subcortical Tracks and Articulation
Given the complexity of the speech production system, vast networks of fiber pathways
are required to support speech processes, which can be together considered as part
of three speech streams: (1) a dorsal speech stream also involved in speech perception
and sensorimotor integration (Hickok & Poeppel, 2007; see Hickok, Chapter 20 in this
volume); (2) a ventral stream also involved in semantic processing (Hickok & Poeppel,
2007); and (3) a motor-speech stream involved in the implementation of phonation,
articulation, and resonance (Dick, Bernal, & Tremblay, 2014). These fiber pathways
(see Figure 15.2 for an overview) include (1) long-association cortical fiber pathways,
comprising the superior longitudinal/arcuate fasciculus (SLF/AF), the frontal aslant
tract (FAT), and cortico-striatal fibers (including the external capsule, and fronto-
striatal tract or subcallosal fasciculus of Muratoff [i.e., the Muratoff bundle]); (2) the
descending motor tracts, especially the corticobulbar pathway; and (3) the cortico-
ponto-cerebellar system.
Long Association Cortical Pathways

Superior Longitudinal/Arcuate Fasciculus
The SLF/AF is the most prominent fiber pathway associated with speech and language.
Historically, the SLA/AF was proposed to connect the posterior IFG with the inferior
parietal lobule (including the supramarginal gyrus) and posterior superior middle tem-
poral gyrus/sulcus (MTG/MTS) (von Monakow, 1897; Wernicke, 1874). However, the
anatomical connectivity of this pathway has been re-evaluated in recent years, and it
is now thought that the SLF/AF can be differentiated into several subcomponents.
For example, research in macaque suggests that the SLF/AF can be differentiated into
four subcomponents (SLF I, II, III, and AF) (Schmahmann & Pandya, 2006; Yeterian,
To Pre-SMA
Frontal aslant tract

Superior longitudinal and arcuate fasciculus
Inferior longitudinal fasciculus
Extreme capsule fiber system
Inferior fronto-occipital fasciculus
Middle longitudinal fasciculus
Uncinate fasciculus
Figure 15.2. Long association fiber pathways of the perisylvian cortex. The superior longi-
tudinal and arcuate fasciculus are the major fibers of the dorsal stream, while the inferior lon-
gitudinal fasciculus, extreme capsule fiber system, inferior fronto-occipital fasciculus, middle
longitudinal fasciculus, and uncinate fasciculus comprise the ventral stream. The frontal aslant
tract connects the pre-SMA with the pars opercularis of the IFG.
Pandya, Tomaiuolo, & Petrides, 2011). Within this parcellation, the SLF III and AF
components are the components most likely involved in articulation. In the macaque,
the SLF III is a parieto-frontal pathway connecting the anterior inferior parietal lobule
with the PMv and posterior IFG. In contrast, the AF component is a temporo-premotor
pathway thought to connect the STGp with more dorsal premotor and lateral prefrontal
cortex (areas 9/46d, 8Ad, and 6d), though not to the IFG (Schmahmann & Pandya,
2006), as previously believed.
In contrast to the macaque data, however, other models based on diffusion-weighted
imaging of the human brain do support direct temporo-inferior frontal connectivity in
humans (Bernal & Altman, 2010; Brauer, Anwander, Perani, & Friederici, 2013; Brown
et al., 2013; Catani et al., 2005; Glasser & Rilling, 2008; Makris et al., 2005; Patterson,
Van Petten, Beeson, Rapcsak, & Plante, 2014; Perani et al., 2011; Thiebaut de Schotten,
Dell’Acqua, Valabregue, & Catani, 2012). Given the inability of nonhuman primates to
speak naturally, it is possible that the connectivity of this pathway may differ between
species. In Catani’s influential model, the “long (direct) segment” of the SLF connects
the posterior superior temporal, middle, and inferior temporal gyrus with the IFG
(pars triangularis and pars opercularis) and PMv (see Catani & Forkel, Chapter 9 in this
volume). This is closer to the classical definition of the SLF/AF. Two “indirect” segments
are also identified in this model: anterior and posterior. The anterior part is a fronto-
inferior parietal-posterior temporal segment, and can be considered to constitute the
SLF III. A posterior part constitutes a posterior temporal-inferior parietal segment
(Catani et al., 2005; Martino et al., 2013).
Because of the connectivity between posterior IFG and supramarginal gyrus,
the anterior component may be an important component of the articulatory loop.
Electrostimulation of the white matter underneath the IFG and IPL/pSTG results in
speech arrest/articulation disturbance (Duffau et al., 2002; Duffau, Gatignol, Denvil,

Lopes, & Capelle, 2003; Maldonado et al., 2011), and damage to the pathway is associated
with transient dysarthria (Duffau et al., 2003; see, in this volume, Duffau, Chapter 8, and
Ziegler, Schölderle, Aichert, & Staiger, Chapter 18). A number of studies have suggested
that the white matter of the long segment and the anterior segment is related to speech
articulation/speech fluency. For example, measures of fluency (Marchina et al., 2011;
Tanabe et al., 1987) and verbal repetition (Breier, Hasan, Zhang, Men, & Papanicolaou,
2008; Fridriksson et al., 2010) in people with left hemisphere stroke are associated with
white matter damage to these pathways.
The involvement of the ventral stream in speech production is less well established,
but if it is involved in speech production it is likely supporting semantic processing.
According to a recent computational model (called Lichtheim 2; Ueno, Saito, Rogers, &
Lambon Ralph, 2011), this is the presumed pathway, via the left uncinate fasciculus (UF)
and left extreme capsule fiber system (EmC-fs), by which meaning is mapped to artic-
ulation. Damage to this pathway is mainly associated with deficits in naming famous
faces (Papagno et al., 2011), which fits with the model’s predictions. However, this pro-
posal is disputed by another model, the Weaver++/ARC model (Roelofs, 2014). These
authors suggest that the dorsal stream involvement is the primary pathway in speech
production. Roelofs points to data showing that production impairments are more as-
sociated with damage to the left AF than to damage to the ventral pathways (namely,
UF; Marchina et al., 2011; Wang, Marchina, Norton, Wan, & Schlaug, 2013). Another
pathway that may play a role in semantic memory and semantic processing during
speech production is the inferior fronto-occipital fasciculus (IFOF; de Zubicaray, Rose,
& McMahon, 2011; Duffau et al., 2005; Mandonnet, Nouet, Gatignol, Capelle, & Duffau,
2007). The IFOF originates in the inferior and medial occipital lobe (and possibly the
medial parietal lobe), sends projections to the ventral temporal lobe, travels through the
temporal stem dorsal to the UF, and projects to the IFG, the medial and orbital frontal
cortex, and the frontal pole (Catani et al., 2003). Connectivity of the temporal and
frontal lobes, and its importance for speech, remains an area of intense research.
The Frontal Aslant Tract

Although it does not appear in historical atlases of white matter, the frontal aslant tract
(FAT) has been recently identified using diffusion weighted imaging in humans (Broce,
Bernal, Altman, Tremblay, & Dick, 2015; Catani et al., 2012b; Catani et al., 2013; Ford,
McGregor, Case, Crosson, & White, 2010; Henry, Berman, Nagarajan, Mukherjee, &
Berger, 2004; Kinoshita et al., 2014; Klein et al., 2007; Lawes et al., 2008; Oishi et al.,
2008; Vergani et al., 2014). As shown in Figure 15.3, the FAT runs obliquely (aslant) from
lateral to medial cortex, and is proposed to connect the posterior IFG, pars opercularis,
to the pre-SMA and (to a lesser degree) the SMA-proper. This connectivity profile is con-
sistent with a number of emerging studies of the functional association between the FAT
and speech. For example, Catani and colleagues (2013) reported that reduced integrity
of the FAT is associated with decreased verbal fluency in people with primary progres-
sive aphasia (PPA). In a case study using both electrostimulation and diffusion tensor
Superior frontal gyrus Frontal aslant tract
Fronto-striatal tract
Inferior frontal gyrus,
pars opercularis
Dorsal striatum
(caudate and putamen)
Cingulate gyrus
Figure 15.3. Connectivity of the frontal aslant and fronto-striatal tracts. The frontal aslant tract
connects pre-supplementary motor area (pre-SMA) with the IFG, pars opercularis. The dorsal
striatum (caudate and putamen) has dense connectivity with the SMA and CMA, in addition
to other frontal (dorsolateral and orbitofrontal) projections not shown. Cortico-striatal connec-
tivity is also supported by fibers of the external capsule (not shown).
imaging (DTI), Vassal and colleagues (Vassal, Boutet, Lemaire, & Nuti, 2014) showed
that stimulation of the left FAT induced speech arrest, with normalization of speech
occurring when stimulation stopped. Kinoshita and colleagues (2014) also used both
electrostimulation and DTI in 19 participants, and showed that electrostimulation of the
left, but not the right, FAT induced speech arrest. Finally, emerging evidence suggests
that this tract is associated with stuttering (Kemerdere et al., 2016; Kronfeld-Duenias,
Amir, Ezrati-Vinacour, Civier, & Ben-Shachar, 2016; see Ziegler, Schölderle, Aichert, &
Staiger, Chapter 18 in this volume). The evidence thus suggests that the FAT is a tract
that can be reliably identified, and that it has a functional relevance to the production of
speech. For a full review of the FAT, refer to Dick et al (2018)
Fronto-Striatal Pathways: External Capsule and Subcallosal

Fasciculus of Muratoff (Muratoff Bundle, or Fronto-Striatal Tract)
The medial frontal cortex makes connections with the dorsal striatum, and planning
and execution of fluent speech relies on the integrity of these cortico-striatal pathways
(Civier, Bullock, Max, & Guenther, 2013). There are at least two fiber bundles important
for these cortico-striatal connections. Primary and non-primary motor areas connect
to the posterior third of the striatum (particularly the posterior putamen), and travel
mainly in the external capsule (Schmahmann & Pandya, 2006). After targeting the pu-
tamen and caudate nuclei, connections are established with the external and internal
pallidum, subthalamic nucleus, and ventrolateral thalamic nuclei. The ventrolateral
thalamic nuclei project back to primary and nonprimary cortical motor areas, forming
the motor component (“motor loop”) of the cortico-striatal circuit. The pre-SMA, in
contrast, along with prefrontal cortex, connects to more anterior parts of the striatum,
also via the external capsule. From there, these regions target the internal pallidum, and
the ventral anterior thalamic nuclei, which in turn projects back to these same cortical
areas, forming a cortico-striatal loop. In addition to the external capsule, the fronto-
striatal tract (FST; Kinoshita et al., 2014; Lehéricy et al., 2004a; Lehéricy et al., 2004b),
essentially the frontal component of the subcallosal fasciculus of Muratoff (Muratoff
bundle), has received more recent interest. It is more easily detected in the macaque,
and can be differentiated reliably from the fronto-occipital-fasciculus (Schmahmann &
Pandya, 2006). Yet, Kinoshita and colleagues (2014) reported successful tractography
in patients undergoing awake surgery for tumor resection, and also showed that stimu-
lation of the pathway (namely caudate-pre-SMA connections) evokes motor initiation
impairment and speech disturbances. Thus, fibers traveling in this pathway may be im-
portant for speech.
Cortico-Cerebello-Cortico  Loops
The majority of the afferent connections to the cerebellum are conveyed through the
middle and inferior cerebellar peduncles, while the majority of the efferent fibers travel
via the superior cerebellar peduncle. Major inputs from the cortex travel via various pon-
tine nuclei, where almost all the cortico-ponto-cerebellar fibers cross the midline in the
basal pons to terminate in the contralateral half of the cerebellar cortex. The cerebellum
itself projects to primary and nonprimary motor areas and to the prefrontal cortex (in-
cluding the pre-SMA) through projections from the cerebellar dentate nucleus via the
ventrolateral nucleus of the thalamus, which projects to the contralateral primary and
nonprimary motor areas and prefrontal cortex (Salmi et al., 2010). These connections
form a cortico-cerbello-cortical loop involved in the control of actions in general, in-
cluding speech. Empirical studies indicate that cerebellar lesions impact speech and lan-
guage function (Fiez, Petersen, Cheney, & Raichle, 1992 Vias & Dick (2017)). Fluency
deficits in both semantic (category) and letter fluency tasks are observed in adult
patients with focal and degenerative cerebellar pathologies (Akshoomoff, Courchesne,
Press, & Iragui, 1992; Appollonio, Grafman, Schwartz, Massaquoi, & Hallett, 1993;
Leggio, Silveri, Petrosini, & Molinari, 2000; Neau, Arroyo-Anllo, Bonnaud, Ingrand, &
Gil, 2000; Richter et al., 2007; Schweizer et al., 2010).
Summary
Many fiber pathways support the production of speech, integrating information from
large segments of the cortex, basal ganglia, thalamus, and cerebellum. The importance
of the basal ganglia and cerebellum to speech production has been known for some time
(Robin & Schienberg, 1990; see, in this volume, Ziegler, Schölderle, Aichert, & Staiger,
Chapter 18, and Copland & Angwin, Chapter 33), but the study of the particular fiber
pathways supporting the communication among these regions, using newly established
methodologies such as diffusion-weighted imaging or connectivity modeling, is a more
recent phenomenon (Manes et al., 2014). What this brief review has hopefully accom-
plished is to reinforce that these fiber pathways should be understood as part of a dis-
tributed network supporting speech production, and attempts should be made to study
them as such.
Conclusions
In this chapter, we have reviewed the neural systems that controls speech produc-
tion at all levels of the nervous system, from a complex web of cortical regions such
as M1v, PMv, SMA, pre-SMA, CMA, and the insula, interconnected through both
short-and long-association pathways, all the way down to the cranial nerve nuclei
in the brainstem through the corticobulbar tract. We have seen that loops of internal
control involving the basal ganglia, thalamus, and cerebellum are also involved in
speech movement preparation, including sequencing and motor timing. Though sev-
eral aspects of speech motor-control processes remain to be clarified, such as motor
programming and co-articulation, it is clear that the advent of multimodal imaging
and brain stimulation techniques have allowed us to make major strides toward un-
derstanding the neural basis of human communication, one of the most distinctive
features of our species.
References
Abbs, J. H., Gracco, V. L., & Cole, K. J. (1984). Control of multimovement coordina-
tion: Censorimotor mechanisms in speech motor programming. Journal of Motor Behavior,
16(2), 195–231.
Acheson, D. J., Hamidi, M., Binder, J. R., & Postle, B. R. (2011). A common neural substrate for
language production and verbal working memory. Journal of Cognitive Neurosciences, 23(6),
1358–1367. doi: 10.1162/jocn.2010.21519
Ackermann, H., & Riecker, A. (2004). The contribution of the insula to motor aspects of speech
production: A review and a hypothesis. Brain and Language, 89(2), 320–328. doi: 10.1016/
S0093-934X(03)00347-X
Agostino, R., Berardelli, A., Formica, A., Accornero, N., & Manfredi, M. (1992). Sequential arm
movements in patients with Parkinson’s disease, Huntington’s disease and dystonia. Brain,
115(Pt 5), 1481–1495.
Akshoomoff, N. A., Courchesne, E., Press, G. A., & Iragui, V. (1992). Contribution of the cere-
bellum to neuropsychological functioning: Evidence from a case of cerebellar degenerative
disorder. Neuropsychologia, 30(4), 315–328.
Alario, F. X., Chainay, H., Lehericy, S., & Cohen, L. (2006). The role of the supplementary
motor area (SMA) in word production. Brain Research, 1076(1), 129–143.
Alexander, G. E., DeLong, M. R., & Strick, P. L. (1986). Parallel organization of functionally
segregated circuits linking basal ganglia and cortex. Annual Review of Neuroscience, 9,
357–381.
Appollonio, I. M., Grafman, J., Schwartz, V., Massaquoi, S., & Hallett, M. (1993). Memory in
patients with cerebellar degeneration. Neurology, 43(8), 1536–1536.
Argyropoulos, G. P., Tremblay, P., & Small, S. L. (2013). The neostriatum and response selec-
tion in overt sentence production: An fMRI study. NeuroImage, 82, 53–60. doi: 10.1016/
j.neuroimage.2013.05.064
Barbas, H., & Pandya, D.N. (1987). Architecture and frontal cortical connections of the premotor
cortex (area 6) in the rhesus monkey. Journal of Comparative Neurology, 256(2), 211–228.
Beevor, C. E., & Horsley, V. (1890). An experimental investigation into the arrangement of the
excitable fibres of the internal capsule of the bonnet monkey (Macacus sinicus). Philosophical
Transactions of the Royal Society of London B Biological Sciences, 181,  49–88.
Benecke, R., Rothwell, J. C., Dick, J. P., Day, B. L., & Marsden, C. D. (1987). Disturbance of se-
quential movements in patients with Parkinson’s disease. Brain, 110(Pt 2), 361–379.
Bengtsson, S. L., Ehrsson, H. H., Forssberg, H., & Ullen, F. (2004). Dissociating brain regions
controlling the temporal and ordinal structure of learned movement sequences. European
Journal of Neuroscience, 19(9), 2591–2602. doi: 10.1111/j.0953-816X.2004.03269.x
Bengtsson, S. L., Ehrsson, H. H., Forssberg, H., & Ullen, F. (2005). Effector-independent vol-
untary timing: Behavioural and neuroimaging evidence. European Journal of Neuroscience,
22(12), 3255–3265. doi: 10.1111/j.1460-9568.2005.04517.x
Bernal, B., & Altman, N. (2010). The connectivity of the superior longitudinal fasciculus: A
tractography DTI study. Magnetic Resonance Imaging, 28(2), 217–225.
Bilodeau-Mercure, M., Lortie, C. L., Sato, M., Guitton, M. J., & Tremblay, P. (2015). The neuro-
biology of speech perception decline in aging. Brain Structure & Function, 220(2), 979–997.
doi: 10.1007/s00429-013-0695-3
Bischoff-Grethe, A., Goedert, K. M., Willingham, D. T., & Grafton, S. T. (2004). Neural
substrates of response- based sequence learning using fMRI. Journal of Cognitive
Neurosciences, 16(1), 127–138. doi: 10.1162/089892904322755610
Boecker, H., Jankowski, J., Ditter, P., & Scheef, L. (2008). A role of the basal ganglia and mid-
brain nuclei for initiation of motor sequences. NeuroImage, 39(3), 1356–1369. doi: 10.1016/
Bohland, J. W., Bullock, D., & Guenther, F. H. (2010). Neural representations and mechanisms
for the performance of simple speech sequences. Journal of Cognitive Neurosciences, 22(7),
1504–1529. doi: 10.1162/jocn.2009.21306
Bohland, J. W., & Guenther, F. H. (2006). An fMRI investigation of syllable sequence produc-
tion. NeuroImage, 32(2), 821–841. doi: 10.1016/j.neuroimage.2006.04.173
Bortoletto, M., Cook, A., & Cunnington, R. (2011). Motor timing and the preparation for se-
quential actions. Brain and Cognition, 75(2), 196–204. doi: 10.1016/j.bandc.2010.11.016
Bortoletto, M., & Cunnington, R. (2010). Motor timing and motor sequencing contribute
differently to the preparation for voluntary movement. NeuroImage, 49(4), 3338–3348.
doi: 10.1016/j.neuroimage.2009.11.048
Brauer, J., Anwander, A., Perani, D., & Friederici, A. D. (2013). Dorsal and ventral pathways in
language development. Brain and Language, 127(2), 289–295. doi: 10.1016/j.bandl.2013.03.001
Braun, A. R., Guillemin, A., Hosey, L., & Varga, M. (2001). The neural organization of dis-
course: An H2 15O-PET study of narrative production in English and American sign lan-
guage. Brain, 124(10), 2028–2044.
Breier, J. I., Hasan, K. M., Zhang, W., Men, D., & Papanicolaou, A. C. (2008). Language dys-
function after stroke and damage to white matter tracts evaluated using diffusion tensor im-
aging. American Journal of Neuroradiology, 29(3), 483–487. doi: 10.3174/ajnr.A0846
Brendel, B., Hertrich, I., Erb, M., Lindner, A., Riecker, A., Grodd, W., & Ackermann,
H. (2010). The contribution of mesiofrontal cortex to the preparation and execu-
tion of repetitive syllable productions: An fMRI study. NeuroImage, 50(3), 1219–1230.
doi: S1053-8119(10)00059-5  [pii]
Broce, I., Bernal, B., Altman, N., Tremblay, P., & Dick, A. S. (2015). Fiber tracking of the frontal
aslant tract and subcomponents of the arcuate fasciculus in 5-8-year-olds: Relation to speech
and language function. Brain and Language, 149, 66–76. doi: 10.1016/j.bandl.2015.06.006
Brown, E. C, Jeong, J.-W., Muzik, O., Rothermel, R., Matsuzaki, N., Juhász, C., . . . Asano, E. (2013).
Evaluating the arcuate fasciculus with combined diffusion-weighted MRI tractography and
electrocorticography. Human Brain Mapping. 35(5), 2333–2347. doi: 10.1002/hbm.22331
Burnett, T. A., Freedland, M. B., Larson, C. R., & Hain, T. C. (1998). Voice F0 responses to
manipulations in pitch feedback. Journal of the Acoustical Society of America, 103(6),
3153–3161.
Catani, M., Dell’acqua, F., Vergani, F., Malik, F., Hodge, H., Roy, P., . . . Thiebaut de Schotten,
M. (2012a). Short frontal lobe connections of the human brain. Cortex, 48(2), 273–291.
doi: 10.1016/j.cortex.2011.12.001
Catani, M., DellAcqua, F., Vergani, F., Malik, F., Hodge, H., Roy, P., . . . Thiebaut de Schotten, M.
(2012b). Short frontal lobe connections of the human brain. Cortex, 48(2), 273–291.
Catani, M., Jones, D. K., Donato, R., & Ffytche, D. H. (2003). Occipito-temporal connections in
the human brain. Brain, 126(Pt 9), 2093–2107. doi: 10.1093/brain/awg203
Catani, M., Jones, D. K., & Ffytche, D H. (2005). Perisylvian language networks of the human
brain. Annals of Neurology, 57,  8–16.
Catani, M., Mesulam, M. M., Jakobsen, E., Malik, F., Martersteck, A., Wieneke, C., . . . Rogalski,
E. (2013). A novel frontal pathway underlies verbal fluency in primary progressive aphasia.
Brain, 136(Pt 8), 2619–2628. doi: 10.1093/brain/awt163
Chavis, D. A., & Pandya, D. N. (1976). Further observations on corticofrontal connections in
the rhesus monkey. Brain Research, 117(3), 369–386. doi: 0006-8993(76)90089-5  [pii]
Cholin, J., Levelt, W. J., & Schiller, N. O. (2006). Effects of syllable frequency in speech produc-
tion. Cognition, 99(2), 205–235.
Christoffels, I. K., van de Ven, V., Waldorp, L. J., Formisano, E., & Schiller, N. O. (2011). The sen-
sory consequences of speaking: Parametric neural cancellation during speech in auditory
cortex. PLoS One, 6(5), e18307. doi: 10.1371/journal.pone.0018307
Civier, O., Bullock, D., Max, L., & Guenther, F. H. (2013). Computational modeling of stuttering
caused by impairments in a basal ganglia thalamo-cortical circuit involved in syllable selec-
tion and initiation. Brain and Language, 126(3), 263–278.
Corbetta, M., Patel, G., & Shulman, G. L. (2008). The reorienting system of the human
brain: From environment to theory of mind. Neuron, 58(3), 306– 324. doi: 10.1016/
j.neuron.2008.04.017
Costa, A., & Caramazza, A. (2002). The production of noun phrases in English and
Spanish: Implications for the scope of phonological encoding in speech production. Journal
of Memory and Language, 46(1), 178–198.
Crosson, B., Sadek, J. R., Maron, L., Gökçay, D., Mohr, C., Auerbach, E. J., . . . Briggs, R. W.
(2001). Relative shift in activity from medial to lateral frontal cortex during internally versus
externally guided word generation. Journal of Cognitive Neuroscience, 13(2), 272–283.
Cunnington, R., Iansek, R., & Bradshaw, J. L. (1999). Movement- related potentials in
Parkinson’s disease: External cues and attentional strategies. Movement Disorders,
14(1),  63–68.
Damian, M. F., & Dumay, N. (2007). Time pressure and phonological advance planning in
spoken production. Journal of Memory and Language, 57(2), 195–209.
de Zubicaray, G. I., McMahon, K. L., Eastburn, M. M., & Wilson, S. J. (2002). Orthographic/
phonological facilitation of naming responses in the picture-word task: An event-related
fMRI study using overt vocal responding. NeuroImage, 16(4), 1084–1093.
de Zubicaray, G. I., Rose, S. E., & McMahon, K. L. (2011). The structure and connectivity of se-
mantic memory in the healthy older adult brain. NeuroImage, 54(2), 1488–1494. doi: 10.1016/
Deiber, M. P., Ibanez, V., Sadato, N., & Hallett, M. (1996). Cerebral structures participating
in motor preparation in humans: A positron emission tomography study. Journal of
Dejerine, J. J. (1901). Anatomie des centres nerveux (Vol. 2). Paris: Rueff et Cie.
Dell, G. S. (1986). A spreading- activation theory of retrieval in sentence production.
Dell, G. S., & Reich, P. A. (1981). Stages in sentence production: An analysis of speech error
data. Journal of Verbal Learning and Verbal Behavior, 20(6), 611–629.
Dick, A. S., Garic, D., Graxiano, P., & Tremblay, P. (2018). The frontal aslant tract (FAT) and its
role in speech, language and executive function. Cortex; a journal devoted to the study of the
nervous system and behavior, 111, 148–163. doi:https://doi.org/10.1016/j.cortex.2018.10.015
Dick, A, Bernal, B., & Tremblay, P. (2014). The language connectome: New pathways, new
concepts. The Neuroscientist, 20, 453–467.
Dronkers, N. F. (1996). A new brain region for coordinating speech articulation. Nature,
384(6605), 159–161. doi: 10.1038/384159a0
Duffau, H., Capelle, L., Sichez, N., Denvil, D., Lopes, M., Sichez, J.-.P., . . . Fohanno, D. (2002).
Intraoperative mapping of the subcortical language pathways using direct stimulations: An
anatomo-functional study. Brain, 125, 199–214.
Duffau, H., Gatignol, P., Denvil, D., Lopes, M., & Capelle, L. (2003). The articulatory loop: Study
of the subcortical connectivity by electrostimulation. Neuroreport, 14, 2005–2008.
Duffau, H., Gatignol, P., Mandonnet, E., Peruzzi, P., Tzourio-Mazoyer, N., & Capelle, L. (2005).
New insights into the anatomo-functional connectivity of the semantic system: A study
using cortico-subcortical electrostimulations. Brain, 128, 797–810.
Dum, R. P., & Strick, P. L. (1991). The origin of corticospinal projections from the premotor
areas in the frontal lobe. Journal of Neuroscience, 11(3), 667–689.
Dum, R. P., & Strick, P.L. (1996). Spinal cord terminations of the medial wall motor areas in
macaque monkeys. Journal of Neuroscience, 16(20), 6513–6525.
Eliades, S. J., & Wang, X. (2003). Sensory-motor interaction in the primate auditory cortex
during self-initiated vocalizations. Journal of Neurophysiology, 89(4), 2194–2207. doi: 10.1152/
jn.00627.2002
Fiez, J. A, Petersen, S. E., Cheney, M. K., & Raichle, M. E. (1992). Impaired non-motor learning
and error detection associated with cerebellar damage: A single case study. Brain, 115(1),
155–178.
Fisher, S. E., & Scharff, C. (2009). FOXP2 as a molecular window into speech and language.
Trends in Genetics, 25(4), 166–177. doi: 10.1016/j.tig.2009.03.002
Flinker, A., Korzeniewska, A., Shestyuk, A. Y., Franaszczuk, P. J., Dronkers, N. F., Knight, R.
T., & Crone, N. E. (2015). Redefining the role of Broca’s area in speech. Proceedings of the
National Academy of Sciences USA, 112(9), 2871–2875. doi: 10.1073/pnas.1414491112
Ford, A., McGregor, K. M., Case, K., Crosson, B., & White, K. D. (2010). Structural connectivity
of Broca’s area and medial frontal cortex. NeuroImage, 52(4), 1230–1237.
Freeman, J. S., Cody, F. W., & Schady, W. (1993). The influence of external timing cues upon the
rhythm of voluntary movements in Parkinson’s disease. Journal of Neurology, Neurosurgery
and Psychiatry, 56(10), 1078–1084.
Freund, H. J., Jeannerod, M., Hallett, M., & Leiguarda, R. (2005). Higher-order motor
disorders: From neuroanatomy and neurobiology to clinical neurology. Oxford: Oxford
University Press.
Fridriksson, J., Kjartansson, O., Morgan, P. S., Hjaltason, H., Magnusdottir, S., Bonilha, L.,
& Rorden, C. (2010). Impaired speech repetition and left parietal lobe damage. Journal of
Neuroscience, 30(33), 11057–11061. doi: 10.1523/JNEUROSCI.1120-10.2010
Fried, I., Katz, A., McCarthy, G., Sass, K. J., Williamson, P., Spencer, S. S., & Spencer, D. D.
(1991). Functional organization of human supplementary motor cortex studied by electrical
stimulation. Journal of Neuroscience, 11(11), 3656–3666.
Garnier, M., Lamalle, L., & Sato, M. (2013). Neural correlates of phonetic convergence and
speech imitation. Frontiers in Psychology, 4, 600. doi: 10.3389/fpsyg.2013.00600
Gerloff, C., Corwell, B., Chen, R., Hallett, M., & Cohen, L. G. (1997). Stimulation over the
human supplementary motor area interferes with the organization of future elements in
complex motor sequences. Brain, 120(Pt 9), 1587–1602.
Glasser, M. F., & Rilling, J. K. (2008). DTI tractography of the human brain’s language pathways.
Grill-Spector, K., Henson, R., & Martin, A. (2006). Repetition and the brain: Neural models
of stimulus- specific effects. Trends in Cognitive Sciences, 10(1), 14– 23. doi: 10.1016/
j.tics.2005.11.006
Grill-Spector, K., & Malach, R. (2001). fMR-adaptation: A tool for studying the functional
properties of human cortical neurons. Acta Psychologica (Amsterdam), 107(1–3), 293–321.
Guenther, F. H. (1995). Speech sound acquisition, coarticulation, and rate effects in a neural
network model of speech production. Psychological Review, 102(3), 594.
Guenther, F. H. (2006). Cortical interactions underlying the production of speech sounds.
Journal of Communication Disorders, 39(5), 350–365. doi: 10.1016/j.jcomdis.2006.06.013
Guenther, F. H., Ghosh, S. S., & Tourville, J. A. (2006a). Neural modeling and imaging of the
cortical interactions underlying syllable production. Brain and Language, 96(3), 280–301.
Guenther, F. H., Ghosh, S. S., & Tourville, J. A. (2006b). Neural modeling and imaging of the
cortical interactions underlying syllable production. Brain and Language, 96(3), 280–301.
He, S. Q., Dum, R. P., & Strick, P. L. (1993). Topographic organization of corticospinal
projections from the frontal lobe: Motor areas on the lateral surface of the hemisphere.
Henry, R. G., Berman, J. I., Nagarajan, S. S., Mukherjee, P., & Berger, M. S. (2004). Subcortical
pathways serving cortical language sites: Initial experience with diffusion tensor imaging
fiber tracking combined with intraoperative language mapping. NeuroImage, 21, 616–622.
Hickok, G. (2012). Computational neuroanatomy of speech production. Nature Reviews
Neuroscience, 13(2), 135–145. doi: 10.1038/nrn3158
Hickok, G. (2014). The architecture of speech production and the role of the phoneme
in speech processing. Language and Cognitive Processes, 29(1), 2– 20. doi: 10.1080/
01690965.2013.834370
Reviews Neuroscience, 8(5), 393–402.
Hoover, J. E., & Strick, P. L. (1999). The organization of cerebellar and basal ganglia outputs to
primary motor cortex as revealed by retrograde transneuronal transport of herpes simplex
virus type 1. Journal of Neuroscience, 19(4), 1446–1463.
Houde, J. F., & Nagarajan, S. S. (2011). Speech production as state feedback control. Frontiers in
Human Neuroscience, 5, 82. doi: 10.3389/fnhum.2011.00082
Houde, J. F., Nagarajan, S. S., Sekihara, K., & Merzenich, M. M. (2002). Modulation of the au-
ditory cortex during speech: An MEG study. Journal of Cognitive Neuroscience, 14(8), 1125–
1138. doi: 10.1162/089892902760807140
Indefrey, P., & Levelt, W. J. (2000). The neural correlates of language production. In M. S.
Gazzaniga (Ed.), The new cognitive neurosciences (pp. 845–865). Cambridge: MIT Press.
components. Cognition, 92(1–2), 101–144.
Ito, T., Kimura, T., & Gomi, H. (2005). The motor cortex is involved in reflexive compensatory
adjustment of speech articulation. Neuroreport, 16(16), 1791–1794.
Jenkins, I. H., Jahanshahi, M., Jueptner, M., Passingham, R. E., & Brooks, D. J. (2000). Self-
initiated versus externally triggered movements. II. The effect of movement predictability
on regional cerebral blood flow. Brain, 123, 1216–1228.
Jescheniak, J. D., Schriefers, H., Garrett, M. F., & Friederici, A. D. (2002). Exploring the activa-
tion of semantic and phonological codes during speech planning with event-related brain
potentials. Journal of Cognitive Neuroscience, 14(6), 951–964.
Jones, J. A., & Munhall, K. G. (2000). Perceptual calibration of F0 production: Evidence from
feedback perturbation. Journal of the Acoustical Society of America, 108(3 Pt 1), 1246–1251.
Jurgens, U. (2002). Neural pathways underlying vocal control. Neuroscience and Biobehavioral
Reviews, 26(2), 235–258.
Jurgens, U. (2009). The neural control of vocalization in mammals: A review. Journal of Voice,
23(1), 1–10. doi: 10.1016/j.jvoice.2007.07.005
Kemerdere, R., de Champfleur, N. M., Deverdun, J., Cochereau, J., Moritz- Gasser, S.,
Herbet, G., & Duffau, H. (2016). Role of the left frontal aslant tract in stuttering: A brain
stimulation and tractographic study. Journal of Neurology, 263(1), 157–167. doi: 10.1007/
s00415-015-7949-3
Kent, R. D. (2000). Research on speech motor control and its disorders: A review
and prospective. Journal of Communication Disorders, 33(5), 391– 427; quiz 428.
doi: S0021-9924(00)00023-X  [pii]
Kinoshita, M., de Champfleur, N. M., Deverdun, J., Moritz-Gasser, S., Herbet, G., & Duffau,
H. (2014). Role of fronto-striatal tract and frontal aslant tract in movement and speech: An
axonal mapping study. Brain Structure & Function, 220(6), 3399– 3412, doi: 10.1007/
s00429-014-0863-0
Klein, J. C., Behrens, T. E. J., Robson, M. D., Mackay, C. E., Higham, D. J., & Johansen-Berg, H.
(2007). Connectivity-based parcellation of human cortex using diffusion MRI: Establishing
reproducibility, validity and observer independence in BA 44/45 and SMA/pre-SMA.
NeuroImage, 34(1), 204–211. doi: 10.1016/j.neuroimage.2006.08.022
Krams, M., Rushworth, M. F., Deiber, M. P., Frackowiak, R. S., & Passingham, R. E. (1998).
The preparation, execution and suppression of copied movements in the human brain.
Experimental Brain Research, 1203, 386–398.
Kronfeld-Duenias, V., Amir, O., Ezrati-Vinacour, R., Civier, O., & Ben-Shachar, M. (2016). The
frontal aslant tract underlies speech fluency in persistent developmental stuttering. Brain
Structure & Function, 221(1), 365–381. doi: 10.1007/s00429-014-0912-8
Kuypers, H. G. J. M. (1973). The anatmomical organization of the descending pathways
and their contribution to motor control especially in primates. In J. Desmedt (Ed.), New
developments in EMG and clinical neurophysiology (pp. 38–68). Basel, Switzerland: Karger.
Lashley, K. S. (1951). The problem of serial order in behavior. In L. A. Jeffress (Ed.), Cerebral
mechanisms in behavior (pp. 112–131). New York: John Wiley & Sons.
Lau, H., Rogers, R. D., & Passingham, R. E. (2006). Dissociating response selection and con-
flict in the medial frontal surface. NeuroImage, 29(2), 446–451.
Lawes, I. N. C., Barrick, T. R., Murugam, V., Spierings, N., Evans, D. R., Song, M., & Clark, C. A.
(2008). Atlas-based segmentation of white matter tracts of the human brain using diffusion
tensor tractography and comparison with classical dissection. NeuroImage, 39(1), 62–79.
doi: 10.1016/j.neuroimage.2007.06.041
Lee, K. M., Chang, K. H., & Roh, J. K. (1999). Subregions within the supplementary motor area
activated at different stages of movement preparation and execution. NeuroImage, 9, 117–123.
Leggio, M. G., Silveri, M. C., Petrosini, L., & Molinari, M. (2000). Phonological grouping is
specifically affected in cerebellar patients: A verbal fluency study. Journal of Neurology,
Neurosurgery and Psychiatry, 69(1), 102–106.
Lehéricy, S., Ducros, M., Krainik, A., Francois, C., Van de Moortele, P.-F., Ugurbil, K., & Kim,
D.-S. (2004a). 3-D diffusion tensor axonal tracking shows distinct SMA and pre-SMA
projections to the human striatum. Cerebral Cortex, 14(12), 1302–1309. doi: 10.1093/cercor/
bhh091
Lehéricy, S., Ducros, M., Van de Moortele, P.-F., Francois, C., Thivard, L., Poupon, C., . . . Kim,
D.-S. (2004b). Diffusion tensor fiber tracking shows distinct corticostriatal circuits in
humans. Annals of Neurology, 55(4), 522–529. doi: 10.1002/ana.20030
Lepage, M., Beaudoin, G., Boulet, C., O’Brien, I., Marcantoni, W., Bourgouin, P., & Richer,
F. (1999). Frontal cortex and the programming of repetitive tapping movements in
man: Lesion effects and functional neuroimaging. Brain Research: Cognitive Brain Research,
8(1),  17–25.
Levelt, W. J. (1992). Accessing words in speech production: Stages, processes and
representations. Cognition, 42(1–3),  1–22.
Levelt, W. J. (1999). Models of word production. Trends in Cognitive Sciences, 3(6), 223–232.
Levelt, W. J., & Wheeldon, L. (1994). Do speakers have access to a mental syllabary? Cognition,
50(1), 239–269.
Lewis, P. A., & Miall, R. C. (2002). Brain activity during non-automatic motor production of
discrete multi-second intervals. Neuroreport, 13(14), 1731–1735.
Liberman, A. M., Harris, K. S., Hoffman, H. S., & Griffith, B. C. (1957). The discrimination of
speech sounds within and across phoneme boundaries. Journal of Experimental Psychology,
54(5), 358–368.
Long, M. A., Katlowitz, K. A., Svirsky, M. A., Clary, R. C., Byun, T. M., Majaj, N., . . . Greenlee, J.
D. (2016). Functional segregation of cortical regions underlying speech timing and articula-
tion. Neuron, 89(6), 1187–1193. doi: 10.1016/j.neuron.2016.01.032
Lu, M. T., Preston, J. B., & Strick, P. L. (1994). Interconnections between the prefrontal cortex
and the premotor areas in the frontal lobe. Journal of Comparative Neurology, 341(3),
375–392.
Ludlow, C. L., Connor, N. P., & Bassich, C. J. (1987). Speech timing in Parkinson’s and
Huntington’s disease. Brain and Language, 32(2), 195–214.
Luppino, G., Matelli, M., Camarda, R., & Rizzolatti, G. . (1993). Corticocortical connections
of area F3 (SMA-proper) and area F6 (pre-SMA) in the macaque monkey. Journal of
Comparative Neurology, 338(1), 114–140.
Luppino, G., & Rizzolatti, G. (2000). The organization of the frontal motor cortex. News and
Views, 15, 219–224.
Macar, F., Lejeune, H., Bonnet, M., Ferrara, A., Pouthas, V., Vidal, F., & Maquet, P. (2002).
Activation of the supplementary motor area and of attentional networks during temporal
processing. Experimental Brain Research, 142(4), 475–485. doi: 10.1007/s00221-001-0953-0
MacNeilage, P. F. (1998). The frame/content theory of evolution of speech production.
Behavioral and Brain Sciences, 21(4), 499–511; discussion 511–446.
MacNeilage, P. F., & DeClerk, J. L. (1969). On the motor control of coarticulation in CVC
monosyllables. Journal of the Acoustic Society of America, 45(5), 1217–1233.
Makris, N., Kennedy, D., McInerney, S., Sorensen, A. G., Wang, R., Caviness, V., & Pandya,
D. N. (2005). Segmentation of subcomponents within the superior longitudinal fascicle in
humans: A quantiative, in vivo, DT-MRI study. Cerebral Cortex, 15, 854–869.
Maldonado, I. L., Moritz-Gasser, S., de Champfleur, N. M., Bertram, L., Moulinié, G., & Duffau,
H. (2011). Surgery for gliomas involving the left inferior parietal lobule: New insights into
the functional anatomy provided by stimulation mapping in awake patients. Journal of
Neurosurgery, 115(4), 770–779. doi: 10.3171/2011.5.JNS112
Mandonnet, E., Nouet, A., Gatignol, P., Capelle, L., & Duffau, H. (2007). Does the left inferior
longitudinal fasciculus play a role in language? A brain simulation study. Brain, 130, 623–629.
Manes, J. L., Parkinson, A. L., Larson, C. R., Greenlee, J. D., Eickhoff, S. B., Corcos, D. M.,
& Robin, D. A. (2014). Connectivity of the subthalamic nucleus and globus pallidus pars
interna to regions within the speech network: A meta-analytic connectivity study. Human
Brain Mapping, 35(7), 3499–3516.
Marchina, S., Zhu, L. L., Norton, A., Zipse, L., Wan, C.Y., & Schlaug, G. (2011). Impairment of
speech production predicted by lesion load of the left arcuate fasciculus. Stroke, 42(8), 2251–
2256. doi: 10.1161/STROKEAHA.110.606103
Markiewicz, C. J., & Bohland, J. W. (2016). Mapping the cortical representation of speech sounds
in a syllable repetition task. NeuroImage, 141, 174–190. doi: 10.1016/j.neuroimage.2016.07.023
Martino, J., De Witt Hamer, P. C., Berger, M. S., Lawton, M. T., Arnold, C. M., de Lucas, E.
M., & Duffau, H. (2013). Analysis of the subcomponents and cortical terminations of the
perisylvian superior longitudinal fasciculus: A fiber dissection and DTI tractography study.
Brain Structure & Function, 218(1), 105–121. doi: 10.1007/s00429-012-0386-5
Matelli, M., R., Camarda., Glickstein, M., & Rizzolatti, G. (1986). Afferent and efferent
projections of the inferior area 6 in the macaque monkey. Journal of Comparative Neurology,
251(3), 281–298.
Meekings, S., Evans, S., Lavan, N., Boebinger, D., Krieger-Redwood, K., Cooke, M., & Scott,
S. K. (2016). Distinct neural systems recruited when speech production is modulated by
different masking sounds. Journal of the Acoustic Society of America, 140(1), 8. doi: 10.1121/
1.4948587
Menon, V., Anagnoson, R. T., Glover, G. H., & Pfefferbaum, A. (2000). Basal ganglia involve-
ment in memory-guided movement sequencing. Neuroreport, 11(16), 3641–3645.
Meyer, A. S. (1990). The time course of phonological encoding in language production: The
encoding of successive syllables of a word. Journal of Memory and Language, 29(5), 524–545.
doi: http://dx.doi.org/10.1016/0749-596X(90)90050-A
Meyer, A. S. (1991). The time course of phonological encoding in language produc-
tion: Phonological encoding inside a syllable. Journal of Memory and Language, 30(1),  69–89.
Muakkassa, K. F., & Strick, P.L. (1979). Frontal lobe inputs to primate motor cortex: Evidence
for four somatotopically organized “premotor” areas. Brain Research, 177(1), 176–182.
Murray, E. A., & Coulter, J. D. (1981). Organization of corticospinal neurons in the monkey.
Journal of Comparative Neurology, 195(2), 339–365. doi: 10.1002/cne.901950212
Neau, J. P., Arroyo-Anllo, E., Bonnaud, V., Ingrand, P., & Gil, R. (2000). Neuropsychological
disturbances in cerebellar infarcts. Acta Neurologica Scandinavica, 102(6), 363–370.
Nelson, S. M., Dosenbach, N. U., Cohen, A. L., Wheeler, M. E., Schlaggar, B. L., & Petersen, S. E.
(2010). Role of the anterior insula in task-level control and focal attention. Brain Structure &
Function, 214(5-6), 669–680. doi: 10.1007/s00429-010-0260-2
Ogar, J., Willock, S., Baldo, J., Wilkins, D., Ludy, C., & Dronkers, N. (2006). Clinical and an-
atomical correlates of apraxia of speech. Brain and Language, 97(3), 343–350. doi: 10.1016/
j.bandl.2006.01.008
Oishi, K., Zilles, K., Amunts, K., Faria, A., Jiang, H., Li, X., . . . Mori, S. (2008). Human brain
white matter atlas: Identification and assignment of common anatomical structures in su-
perficial white matter. NeuroImage, 43(3), 447–457. doi: 10.1016/j.neuroimage.2008.07.009
Papagno, C., Miracapillo, C., Casarotti, A., Romero Lauro, L. J., Castellano, A., Falini,
A., . . . Bello, L. (2011). What is the role of the uncinate fasciculus? Surgical removal and
proper name retrieval. Brain, 134(Pt 2), 405–414. doi: 10.1093/brain/awq283
Papoutsi, M., de Zwart, J. A., Jansma, J. M., Pickering, M. J., Bednar, J. A., & Horwitz, B. (2009).
From phonemes to articulatory codes: An fMRI study of the role of Broca’s area in speech
production. Cerebral Cortex, 19(9), 2156–2165.
Patterson, D. K., Van Petten, C., Beeson, P. M., Rapcsak, S. Z., & Plante, E. (2014).
Bidirectional iterative parcellation of diffusion weighted imaging data: Separating cortical
regions connected by the arcuate fasciculus and extreme capsule. NeuroImage, 102(Pt 2),
704–7 16.
Peeva, M. G., Guenther, F. H., Tourville, J. A., Nieto-Castanon, A., Anton, J. L., Nazarian, B.,
& Alario, F. X. (2010). Distinct representations of phonemes, syllables, and supra-syllabic
sequences in the speech production network. NeuroImage, 50(2), 626–638.
Penfield, W., & Boldrey, E. (1937). Somatic motor and sensory representation in the cerebral
cortex of man as studied by electrical stimulation. Brain, 60, 389–443.
Penfield, W., & Welch, K. (1951). The supplementary motor area of the cerebral cortex: A clin-
ical and experimental study. AMA Archives Neurology & Psychiatry, 66(3), 289–317.
Peramunage, D., Blumstein, S. E., Myers, E. B., Goldrick, M., & Baese-Berk, M. (2011).
Cognitive Neuroscience, 23(3), 593–603. doi: 10.1162/jocn.2010.21489
Perani, D., Saccuman, M. C., Scifo, P., Anwander, A., Spada, D., . . . Friederici, A. D. (2011).
Neural language networks at birth. Proceedings of the National Academy of Sciences USA,
108(38), 16056–16061. doi: 10.1073/pnas.1102991108
Picard, N., & Strick, P. L. (1996). Motor areas of the medial wall: A review of their location and
functional activation. Cerebral Cortex, 6, 342–353.
Picard, N., & Strick, P. L. (2001). Imaging the premotor areas. Current Opinion in Neurobiology,
11(6), 663–672.
Postuma, R. B., & Dagher, A. (2006). Basal ganglia functional connectivity based on a meta-
analysis of 126 positron emission tomography and functional magnetic resonance imaging
publications. Cerebral Cortex, 16(10), 1508–1521. doi: 10.1093/cercor/bhj088
Praamstra, P., Stegeman, D. F., Cools, A. R., Meyer, A. S., & Horstink, M. W. (1998). Evidence
for lateral premotor and parietal overactivity in Parkinson’s disease during sequential and
bimanual movements. A PET study. Brain, 121, 769–772.
Purcell, D. W., & Munhall, K. G. (2006a). Adaptive control of vowel formant fre-
quency: Evidence from real-time formant manipulation. Journal of the Acoustic Society of
America, 120(2), 966–977.
Purcell, D. W., & Munhall, K. G. (2006b). Compensation following real-time manipulation

of formants in isolated vowels. Journal of the Acoustic Society of America, 119(4), 2288–2297.
Ralston, D. D., & Ralston, H. J., 3rd. (1985). The terminations of corticospinal tract axons in the ma-
caque monkey. Journal of Comparative Neurology, 242(3), 325–337. doi: 10.1002/cne.902420303
Reimers-Kipping, S., Hevers, W., Pääbo, S., & Enard, W. (2011). Humanized Foxp2 spe-
cifically affects corticobasal ganglia circuits. Neuroscience, 175, 75– 84. doi: 10.1016/
j.neuroscience.2010.11.042
Richter, S., Gerwig, M., Aslan, B., Wilhelm, H., Schoch, B., Dimitrova, A., . . . Timmann, D.
(2007). Cognitive functions in patients with MR-defined chronic focal cerebellar lesions.
Journal of Neurology, 254(9), 1193–1203. doi: 10.1007/s00415-006-0500-9
Riecker, A., Mathiak, K., Wildgruber, D., Erb, M., Hertrich, I., Grodd, W., & Ackermann,
H. (2005). fMRI reveals two distinct cerebral networks subserving speech motor control.
Neurology, 64, 700–706.
Robin, D. A., & Schienberg, S. (1990). Subcortical lesions and aphasia. Journal of Speech and
Hearing Disorders, 55(1), 90–100.
Roelofs, A. (1997). The WEAVER model of word-form encoding in speech production.
Cognition, 64(3), 249–284.
Roelofs, A. (1999). Phonological segments and features as planning units in speech production.
Language and Cognitive Processes, 14(2), 173–200.
Roelofs, A. (2014). A dorsal- pathway account of aphasic language production: The
WEAVER++/ARC model. Cortex, 59,  33–48.
Salmi, J., Pallesen, K. J., Neuvonen, T., Brattico, E., Korvenoja, A., Salonen, O., & Carlson, S.
(2010). Cognitive and motor loops of the human cerebro-cerebellar system. Journal of
Cognitive Neuroscience, 22(11), 2663–2676. doi: 10.1162/jocn.2009.21382
Sapir, E. (1921). Language: An introduction to the study of speech. New York: Harcourt, Brace.
Scheiner, E., Hammerschmidt, K., Jurgens, U., & Zwirner, P. (2004). The influence of
hearing impairment on preverbal emotional vocalizations of infants. Folia Phoniatrica et
Logopaedica, 56(1), 27–40. doi: 10.1159/000075326
Schiller, N. O., Bles, M., & Jansma, B. M. (2003). Tracking the time course of phonological
encoding in speech production: An event-related brain potential study. Cognitive Brain
Research, 17(3), 819–831.
Schluter, N. D., Rushworth, M. F., Passingham, R. E., & Mills, K. R. (1998). Temporary in-
terference in human lateral premotor cortex suggests dominance for the selection of
movements: A study using transcranial magnetic stimulation. Brain, 121(5), 785–799.
Schmahmann, J. D., & Pandya, D. N. (2006). Fiber pathways of the brain. Oxford: Oxford
University Press.
Schmahmann, J. D., Pandya, D. N., Wang, R., Dai, G., D’Arceuil, H. E., de Crespigny, A. J., &
Wedeen, V. J. (2007). Association fibre pathways of the brain: Parallel observations from diffusion
spectrum imaging and autoradiography. Brain, 130(Pt 3), 630–653. doi: 10.1093/brain/awl359
Schmitz-Hubsch, T., Eckert, O., Schlegel, U., Klockgether, T., & Skodda, S. (2012). Instability of
syllable repetition in patients with spinocerebellar ataxia and Parkinson’s disease. Movement
Disorders, 27(2), 316–319. doi: 10.1002/mds.24030
Schnur, T. T., Costa, A., & Caramazza, A. (2006). Planning at the phonological level during
sentence production. Journal of Psycholinguistic Research, 35(2), 189–213.
Schweizer, T. A., Alexander, M. P., Susan Gillingham, B. A., Cusimano, M., & Stuss, D. T. (2010).
Lateralized cerebellar contributions to word generation: A phonemic and semantic fluency
study. Behavioural Neurology, 23(1-2), 31–37. doi: 10.3233/BEN-2010-0269
Segawa, J. A., Tourville, J. A., Beal, D. S., & Guenther, F. H. (2015). The neural correlates of
speech motor sequence learning. Journal of Cognitive Neuroscience, 27(4), 819–831.
doi: 10.1162/jocn_a_00737
Shattuck-Hufnagel, S., & Klatt, D. H. (1979). The limited use of distinctive features and mark-
edness in speech production: Evidence from speech error data. Journal of Verbal Learning
and Verbal Behavior, 18(1),  41–55.
Shuster, L. I., & Lemieux, S. K. (2005). An fMRI investigation of covertly and overtly
produced mono-and multisyllabic words. Brain and Language, 93(1), 20–31. doi: 10.1016/
j.bandl.2004.07.007
Simonyan, K., Ackermann, H., Chang, E. F., & Greenlee, J. D. (2016). New developments in
understanding the complexity of human speech production. Journal of Neuroscience, 36(45),
11440–11448. doi: 10.1523/JNEUROSCI.2424-16.2016
Simonyan, K., & Fuertinger, S. (2015). Speech networks at rest and in action: Interactions be-
tween functional brain networks controlling speech production. Journal of Neurophysiology,
113(7), 2967–2978. doi: 10.1152/jn.00964.2014
Simonyan, K., & Jurgens, U. (2003). Efferent subcortical projections of the laryngeal
motorcortex in the rhesus monkey. Brain Research, 974(1–2),  43–59.
Skodda, S. (2011). Aspects of speech rate and regularity in Parkinson’s disease. Journal of the
Neurological Sciences, 310(1–2), 231–236. doi: 10.1016/j.jns.2011.07.020
Skodda, S., Flasskamp, A., & Schlegel, U. (2010). Instability of syllable repetition as a model for
impaired motor processing: Is Parkinson’s disease a “rhythm disorder”? Journal of Neural
Transmission, 117(5), 605–612. doi: 10.1007/s00702-010-0390-y
Skodda, S., Flasskamp, A., & Schlegel, U. (2011). Instability of syllable repetition in Parkinson’s
disease: Influence of levodopa and deep brain stimulation. Movement Disorders, 26(4), 728–
730. doi: 10.1002/mds.23439
Skodda, S., & Schlegel, U. (2008). Speech rate and rhythm in Parkinson’s disease. Movement
Disorders, 23(7), 985–992. doi: 10.1002/mds.21996
Sperry, R. W. (1950). Neural basis of the spontaneous optokinetic response produced by visual
inversion. Journal of Comparative Physiology and Psychology, 43, 482–489.
Talairach, J., & Bancaud, J. (1966). The supplementary motor area in man. International Journal
of Neurology, 5330–5347.
Tanabe, H., Sawada, T., Inoue, N., Ogawa, M., Kuriyama, Y., & Shiraishi, J. (1987). Conduction
aphasia and arcuate fasciculus. Acta Neurologica Scandinavica, 76(6), 422–427.
Thickbroom, G. W., Byrnes, M. L., Sacco, P., Ghosh, S., Morris, I. T., & Mastaglia, F. L. (2000).
The role of the supplementary motor area in externally timed movement: The influence of
predictability of movement timing. Brain Research, 874(2), 233–241.
human comparative anatomy of the frontal lobe association tracts. Cortex, 48(1), 82–96.
doi: 10.1016/j.cortex.2011.10.001
Tourville, J. A., & Guenther, F. H. (2010). The DIVA model: A neural theory of speech acqui-
sition and production. Language and Cognitive Processes, 26(7), 952–981. doi: 10.1080/
01690960903498424
Tourville, J. A., & Guenther, F. H. (2011). The DIVA model: A neural theory of speech acqui-
sition and production. Language and Cognitive Processes, 26(7), 952–981. doi: 10.1080/
01690960903498424
Tourville, J. A., Reilly, K. J., & Guenther, F. H. (2008). Neural mechanisms underlying auditory
feedback control of speech. NeuroImage, 39(3), 1429–1443. doi: S1053-8119(07)00883-X  [pii]
Treiman, R. (1985). Onsets and rimes as units of spoken syllables: Evidence from children.
Journal of Experimental Child Psychology, 39(1), 161–181. doi: http://dx.doi.org/10.1016/
0022-0965(85)90034-7
Tremblay, P., & Dick, A. S. (2016). Broca and Wernicke are dead, or moving past the
classic model of language neurobiology. Brain and Language, 162, 60–7 1. doi: 10.1016/
j.bandl.2016.08.004
Tremblay, P., & Gracco, V. L. (2009). Contribution of the pre-SMA to the production of words
and non-speech oral motor gestures, as revealed by repetitive transcranial magnetic stimu-
lation (rTMS). Brain Research, 1268, 112–124. doi: 10.1016/j.brainres.2009.02.076
Tremblay, P., & Gracco, V. L. (2010). On the selection of words and oral motor
responses: Evidence of a response-independent fronto-parietal network. Cortex, 46(1), 15–
28. doi: 10.1016/j.cortex.2009.03.003
Tremblay, P., & Gracco, V.L. (2006). Contribution of the frontal lobe to externally and inter-
nally specified verbal responses: fMRI evidence. NeuroImage, 33(3), 947–957.
Tremblay, P., & Small, S. L. (2011). Motor response selection in overt sentence production: A
functional MRI study. Frontiers in Psychology, 2, 253. doi: 10.3389/fpsyg.2011.00253
Tremblay, S., Shiller, D. M., & Ostry, D. J. (2003). Somatosensory basis of speech production.
Nature, 423(6942), 866–869. doi: 10.1038/nature01710
Tsujimoto, T., Ogawa, M., Tsukada, H., Kakiuchi, T., & Sasaki, K. (1998). Activation of the ven-
tral and mesial frontal cortex of the monkey by self-initiated movement tasks as revealed by
positron emission tomography. Neuroscience Letters, 258(2), 117–120.
Ueno, T., Saito, S., Rogers, T. T., & Lambon Ralph, M. A. (2011). Lichtheim 2: Synthesizing
aphasia and the neural basis of language in a neurocomputational model of the dual dorsal-
ventral language pathways. Neuron, 72(2), 385–396.
Vallesi, A., Shallice, T., & Walsh, V. (2007). Role of the prefrontal cortex in the foreperiod ef-
fect: TMS evidence for dual mechanisms in temporal preparation. Cerebral Cortex, 17(2),
466–474. doi: 10.1093/cercor/bhj163
Van der Merwe, A. (2009). A theoretical framework for the characterization of pathological
speech sensorimotor control. In M. R. McNeil (Ed.), Clinical management of sensorimotor
speech disrders (pp. 3–18). New York: Thieme.
Vargha-Khadem, F., Gadian, D. G., Copp, A., & Mishkin, M. (2005). FOXP2 and the neuro-
anatomy of speech and language. Nature Reviews Neuroscience, 6(2), 131–138. doi: 10.1038/
nrn1605
Vassal, F., Boutet, C., Lemaire, J.-J., & Nuti, C. (2014). New insights into the functional signifi-
cance of the frontal aslant tract: An anatomo-functional study using intraoperative electrical
stimulations combined with diffusion tensor imaging-based fiber tracking. British Journal of
Neurosurgery, 28(5), 685–687. doi: 10.3109/02688697.2014.889810
Vergani, F., Lacerda, L., Martino, J., Attems, J., Morris, C., Mitchell, P., . . . Dell’acqua, F. (2014).
White matter connections of the supplementary motor area in humans. Journal of Neurology,
Neurosurgery and Psychiatry, 85(12), 1377–1385, doi: 10.1136/jnnp-2013-307492
Vias, C., & Dick, A. S. (2017). Cerebellar Contributions to Language in Typical and Atypical
Development: A Review. Dev Neuropsychol, 42(6), 404–421. doi:10.1080/87565641.2017.133
4783
Vihla, M., Laine, M., & Salmelin, R. (2006). Cortical dynamics of visual/semantic vs. phono-
logical analysis in picture confrontation. NeuroImage, 33(2), 732–738. doi: http://dx.doi.org/
10.1016/j.neuroimage.2006.06.040
Volkmann, J., Hefter, H., Lange, H. W., & Freund, H. J. (1992). Impairment of temporal or-
ganization of speech in basal ganglia diseases. Brain and Language, 43(3), 386–399.
doi: 0093-934X(92)90108-Q  [pii]
von Cramon, D., & Jurgens, U. (1983). The anterior cingulate cortex and the phonatory control
in monkey and man. Neuroscience and Biobehavioral Reviews, 7(3), 423–425.
Von Holst, E., & Mittelstaedt, H. . (1973). The reafference principle. In R. Martin (Ed.), The be-
havioral physiology of animals and man: The collected papers of Erich von Holst (Vol. 1, pp.
139–173). Coral Gables, FL: University of Miami Press.
von Monakow, C. (1897). Gehirnpathologie. I. Allgemeine einleitung. II. Localisation. III.
Gehirnblutungen. IV. Verstopfung der Hirnarterien. (Vol. 9[1]). Wien: Alfred Hölder.
Wang, J., Marchina, S., Norton, A. C., Wan, C. Y., & Schlaug, G. (2013). Predicting speech flu-
ency and naming abilities in aphasic patients. Frontiers in Human Neuroscience, 7, 831.
Weeks, R. A., Honda, M., Catalan, M. J., & Hallett, M. . (2001). Comparison of auditory, somat-
osensory, and visually instructed and internally generated finger movements: A PET study.
NeuroImage, 14(1), 219–230.
Wernicke, C. (1874). Der aphasiche symptomenkomplex: Eine psychologische studie auf
anatomischer basis. Breslau: Cohn & Weigert.
Wheeldon, L., & Lahiri, A. (1997). Prosodic units in speech production. Journal of Memory and
Language, 37(3), 356–381.
Wiese, H., Stude, P., Nebel, K., de Greiff, A., Forsting, M., Diener, H. C., & Keidel, M. (2004).
Movement preparation in self-initiated versus externally triggered movements: An event-
related fMRI-study. Neuroscience Letters, 371(2–3), 220–225.
Yeterian, E. H., Pandya, D. N., Tomaiuolo, F., & Petrides, M. (2011). The cortical connectivity of
the prefrontal cortex in the monkey brain. Cortex. doi: 10.1016/j.cortex.2011.03.004
Chapter 16
The Neural Org a ni z at i on

of Signed L a ng uag e
Aphasia and Neuroscience Evidence
David P. Corina and Laurel A. Lawyer
Introduction
Signed languages used in Deaf communities are naturally occurring human languages
that exhibit the full range of linguistic complexity found in spoken languages (see
Newman, Chapter 14 in this volume). Just as there are a multitude of spoken language
communities around the world (e.g., speakers of Farsi/Persian, Thai, German, and
English), there are also many distinct sign language communities (e.g., signers of Taiwan
Ziran Shouyu [TZS], Deutsche Gebärdensprache [DGS], Lengua de Sigños Española
[LSE], and British Sign Language [BSL], to name but a few). Although the histories and
geographies of signed languages are less well documented, it is known that signed lan-
guages arise spontaneously, over several generations, from isolated communities that
have a preponderance of deaf individuals. Such situations are not as rare as one might
think, as genetic influences on the transmission of deafness are well attested (Groce,
1985; Sandler, Meir, Padden, & Arnoff, 2005).
Despite the widespread use of signed languages in Deaf communities, most neurobi-
ological models of language processing have been predicated on research using spoken
languages only. Neurolinguistic studies using signed languages provide a unique oppor-
tunity to distinguish neurobiological substrates that universally support language pro-
cessing from those that are dependent on aural-oral processes. Studies of sign language
processing in the adult brain highlight a largely modality-independent language system,
with some important distinctions. These differences can largely be traced to perceptual
processes related to the modality of the language itself (visuospatial as opposed to au-
ditory). The following review will address the ways in which sign-language research
broadly supports current views of language processing in a modality-independent
the Neural Organization of Signed Language 403
manner, before turning to several areas in which sign-language processing differs from
spoken-language processing. Finally, we conclude with two areas in which sign lan-
guage has bearing on current issues in neurolinguistics: the profile of neurodegenerative
diseases and their effect on signing, and the relationship between sign perception and
human action understanding.
Language Processing Is Primarily

Modality-Independent
Evidence from Sign Language Aphasia

Deaf signers, like hearing speakers, exhibit distinct language disturbances when cor-
tical regions are damaged. Neurolinguistic studies of lifelong signers whose primary
and preferred mode is a signed language have documented patterns of sign-language
disturbances following acute brain damage and have helped to identify brain re-
gions that are critical for sign-language processing. These studies indicate remarkable
parallels between left hemisphere regions that support spoken-language processing and
those that support signed-language processing. Sign-language disturbances vary sys-
tematically depending on the locus of the lesion (Hickok, Love-Geffen, & Klima, 2002;
Marshall, Atkinson, Smulovitch, Thacker, & Woll, 2004; Poziner, Bellugi, & Klima,
1987; for a review see Corina, 1998a, 1998b), supporting long-standing generalizations
regarding differential contributions of left hemisphere anterior-frontal and posterior-
temporal regions in language processing. In the following, we review the details of sev-
eral case studies that provide support for this analysis.
Broca-Like Signing
In spoken language, language production impairments are typically associated with
lesions that involve the lower posterior portion of the left frontal lobe (e.g., Broca’s
area, Brodmann area 44/45) and often extend into the periventricular white matter (cf.
Goodglass, 1993; Mohr et al., 1978; see Wilson, Chapter 2 in this volume). Two well-
documented cases of sign language aphasia provide additional support for the critical
role that left frontal regions play in language production.
Patient G. D., reported in Poizner et al. (1987), an active mother married to a deaf
signing man with three deaf signing children, had a large lesion to left frontal areas that
encompassed traditional Broca’s territory (including BA 44/45), resulting in a profound
expressive aphasia. The patient presented with halting dysfluent sign articulation and
an agrammatic language profile. In particular, American Sign Language (ASL) sentence
structure was greatly simplified (i.e., telegraphic), and signs did not include the required
movement modulations that signal morpho-syntactic inflection. In contrast to her ex-
pressive deficits, G. D.’s comprehension was well within normal limits.
404 David P. Corina and Laurel A. Lawyer
Patient R. S., reported in Hickok, Kritchevsky, Bellugi, and Klima (1996), is a native
signer, who learned ASL in infancy from her deaf signing parents. She had a left infe-
rior frontal lesion that was more restricted to classically defined Broca’s territory. This
lesion involved inferior motor cortex, most of the pars opercularis, and anterior, supe-
rior insula. Subcortically, this lesion also undercut most of the pars triangularis and the
lower motor cortex. During the acute phase of this stroke, the patient showed a marked
expressive aphasia with nonfluent, effortful production, which also effected sign rep-
etition. Sign language comprehension, however, was intact. Over time, the expressive
aphasia largely resolved, though word-finding difficulties remained.
R. S. exhibited an unusual paraphasia restricted to two-handed signs. In signs that
require two hands to move independently, R. S. would move one of her hands incor-
rectly. In other cases, R. S. would fail to preserve the required spatial relationships
between the two hands. Moreover, during one-handed signing, R. S. mirrored the
movements and handshapes of the dominant hand on the nondominant hand, but
somewhat reduced in degree of movement. This mirroring was not seen during
nonlinguistic movements and appears different from pathological mirror movements
seen in some cases of hemiparesis. This case illustrates that the modality of the lin-
guistic system can place unique demands on the neural mediation and implemen-
tation of language. These errors may be taken as evidence of modality-dependent
linguistic impairment.
Wernicke-Like Signing
Spoken language comprehension deficits are well attested after left hemisphere temporal
lobe damage (cf. Damasio, 1992; Dronkers & Baldo, 2009; Naeser, Helm-Estabrooks,
Haas, Auerbach, & Srinivasan, 1987). Wernicke’s aphasia, which is expressed as im-
paired language comprehension accompanied with fluent, but often paraphasic (se-
mantic and phonemic) output, is often associated with damage to the posterior regions
of the left superior temporal gyrus. More recent work has suggested the contribution of
the posterior middle temporal gyrus in cases of chronic Wernicke’s aphasia (Dronkers,
Redfren, & Knight, 2000; Dronkers, Redfern, & Ludy, 1995; see Wilson, Chapter 2 in
this volume). In two group-level analyses, one on American Sign Language and one
on British Sign Language (BSL; a completely distinct language from ASL), damage
to the left hemisphere was observed to factor in sign-comprehension deficits as well
(Atkinson, Marshall, Woll, & Thacker, 2005; Hickok et al., 2002).
Case studies of signers with damage in parietal areas evidence fluent sign aphasia with
associated comprehension deficits, as is also seen in spoken language users. One such ex-
ample is patient L. K., described in Chiarello, Knight, and Mandell (1982) and in Poizner
et al. (1987), who had a left anterior parietal lesion in the region of the supramarginal
and angular gyri, but which spared both inferior frontal areas (BA 44/45) and the poste-
rior superior temporal plane (BA 41, 42, and 22). The patient presented with fluid, facile
signing, but with phonemic paraphasias and severe anomia. L. K. showed difficulty in
grasping test instructions and was unable to carry out 2-or 3-part commands. The pa-
tient experienced profound and lasting sign comprehension impairment.
Patient P. D., reported in Poizner et al. (1987), also had a left subcortical lesion with
an anterior focus deep to Broca’s area, with posterior extension into the white matter
underlying the supramarginal and angular gryi. His signing, while fluent, exhibited a
number of semantic paraphasias, such as signing “bed” for “chair,” “daughter” for “son,”
and “quit” for “depart.” In addition, he produced grammatically inappropriate signs
(paragrammatism), and his expression of sentence-level grammatical roles was im-
paired. Notably, P. D. showed a lack of consistency in spatial agreement required of ASL
syntax (Bellugi, Poizner, & Klima, 1989). That is, he would establish nominal referents
in space, but he was inconsistent in maintaining co-reference to these previously estab-
lished locations.
As with the case studies presented in the preceding discussion, the group analysis by
Hickok et al. (2002) highlights the role of the left posterior temporal lobe in sign com-
prehension deficits. There is, however, some controversy over the degree of anatom-
ical overlap between signers with comprehension deficits and users of spoken language
who present with Wernicke’s aphasia, as others have remarked on the potentially unique
role of the left inferior parietal region in sign processing (Corina, 1998a, 1998b; Corina,
Lawyer, Hauser, & Hirshorn, 2013; Chiarello et al., 1982; Leischner, 1943; Poizner et al.,
1987). For instance, patient W. L. (Corina et al., 1992) exhibited fluent aphasia with severe
comprehension deficits, in addition to producing numerous phonemic errors, nearly all
of which were errors involving substitutions of handshape. For example, he produced the
sign “sister” with an F handshape rather than the required L handshape. The lesion in this
case did not include classic Wernicke’s area, but rather undercut both frontal and infe-
rior parietal areas, including the supramarginal gyrus. Based on cases such as this, some
have suggested that sign-language comprehension may be more dependent than speech
on left hemisphere inferior parietal areas, traditionally associated with somatosensory
and visual motor integration, while spoken-language comprehension might weigh more
heavily on posterior temporal lobe association regions whose input includes networks
intimately involved with auditory speech processing (Corina, Lawyer, & Cates, 2012).
Evidence from Cortical Stimulation Mapping

Data from rare clinical cases of cortical stimulation mapping (CSM; see Duffau,
Chapter 8 in this volume) performed in deaf and signing neurosurgical patients provide
further clues to neural regions that contribute differentially to the patterns of deficits
outlined in the preceding. Corina et al. (1999) report the effects of left hemisphere
cortical stimulation on sign-language production in a deaf individual, patient S. T.,
undergoing an awake CSM procedure. Stimulation to two anatomical sites, one in the
frontal opercular region of Broca’s area (BA 44) and one in the parietal operculum, both
led to consistent naming disruptions. However, the errors were qualitatively different.
Stimulation to the frontal opercular region resulted in errors involving the motor ex-
ecution of signs. These errors were characterized by a lax articulation of the intended
sign, with nonspecific movements (repeated tapping or rubbing) and a reduction in
handshape configurations to a lax closed fist handshape. These types of errors were
observed during trials of both sign and non-sign repetition. These results suggest that
the posterior portion of Broca’s area is involved in the motoric execution of complex
articulatory forms, especially those underlying the phonetic level of language structure
(see Tremblay, Deschamps, & Dick, Chapter 15 in this volume).
The sign errors observed with stimulation to the supramarginal gyrus (SMG) (BA40)
resulted in both formational and semantic errors. Formational errors were characterized
by repeated attempts to articulate the intended targets, and successive approximations of
the target sign were common. Notably, we do not find the lax and reduced articulations
characteristic of the signs seen with stimulation to Broca’s area. Semantic errors were
also observed with stimulation to the SMG. Interestingly, many of these errors involve
semantic substitutions that were formationally similar to the intended targets. As these
errors did not affect non-signs, this suggests that the SMG plays a critical role in the se-
lection of the individual semantic and phonological (i.e., handshape, movement, and, to
a lesser extent, location) components of sign forms.
Evidence from Neuroimaging Studies

Early functional imaging studies often emphasized the overlap of activation between
regions subserving sign processing and those regions traditionally thought to me-
diate speech processing (cf. Bavelier et al., 1998; McGuire et al., 1997; Neville et al., 1998;
Nishimura et al., 1999; Petitto et al., 2000; Soderfeldt et al., 1997). For example, Pettito
et al. (2000) and McGuire et al. (1997) reported significant activation in left inferior
frontal regions, specifically Broca’s area (BA 44 and 45), and anterior insula during overt
and covert tasks of sign generation. Subsequent work using more refined cytoarchitec-
tonic references have indicated that BA 45 is activated during both sign and speech, and
can be differentiated from complex oral/laryngeal and limb movements, which result
in more ventral activation of BA 44. This interpretation accords well with the cortical
stimulation finding reported in Corina et al. (1999). Overall this research speaks to the
modality independence of the pars triangularis (BA 45) (Horowitz et al., 2003), a region
that has been traditionally considered a “speech-motor” region.
Neuroimaging studies of sign-language production reveal further commonalities in
the neural systems underlying core properties of language function in sign and speech.
In studies of ASL verb generation, in which patients are required to generate a verb in
response to a given noun, José-Robertson et al. (2004) reported left-lateralized activa-
tion within perisylvian frontal and subcortical regions commonly observed in spoken-
language generation tasks. In an extension of this work, Corina, San José-Robertson,
Guillemin, High, and Braun (2003) reported that the left-lateralized pattern was not sig-
nificantly different when the production of a repeat-generate task was conducted with a
signer’s right/dominant or left/nondominant hand. This finding is also consistent with
the observation that aphasic deficits in right-handed signers are often realized on the
ipsilesional limb (i.e., the left hand in cases of left hemisphere damage), indicating that
the observed errors are not an artifact of motoric weakness, but rather a central disorder
of linguistic control mediated by the left hemisphere.
In a large-scale analysis of deaf signers and hearing non-signers engaged in object-
naming tasks, Emmorey, Mehta, and Grabowski (2007) reported areas of overlap in
neural activation for sign production in deaf participants and word production in hearing
participants. This analysis identified regions supporting modality-independent lexical
access. Common regions included the left medial temporal cortex and the left inferior
frontal gyrus. Emmorey et al. (2007) suggest that left temporal activation reflects concep-
tually driven lexical access (Indefrey & Levelt, 2004; see de Zubicaray & Piai, Chapter 19
in this volume). In this view, for both speakers and signers, activation within the left in-
ferior temporal gyrus may reflect pre-lexical conceptual processing of the pictures to be
named, while activation within the more middle temporal regions may reflect lemma
selection prior to phonological code retrieval. These results argue for a modality-
independent fronto-temporal network that subserves both sign and word production.
Subcortical Impairments
While previous aphasia and neuroimaging studies have done an admirable job
in describing the commonalities in cortical regions between users of signed and
spoken languages, more recent studies have begun to expand our knowledge of the
contributions of subcortical structures on signing. For instance, Corina et al. (2014)
present contrasting profiles of two patients with damage to left hemisphere subcor-
tical regions. Patient O. S. had damage to the left putamen/basal ganglia and anterior
striatal-subcortical white matter regions. O.S. exhibited a Wernicke-like aphasia, with
notably hypokinetic signing. In this case, articulatory precision was compromised,
evincing reduced movement, and a lack of differentiation between lexical and transi-
tional movements in signing. Patient D.T., on the other hand, had a lesion restricted
to the left putamen, and showed no evidence of aphasia. However, her signing was hy-
perkinetic, showing an exuberance of movement, often overshooting and repeating
movements, and with inability to dampen contacting movements. She also showed diffi-
culty in right-and left-hand coordination, and had a somewhat flaccid wrist tone. These
two case studies provide some of the first evidence for dysarthria in a sign language, and
provide evidence for the role of the basal ganglia in the motor execution of signing.
Sign-L anguage Processing Engages

Specialized Cortical Regions
As outlined earlier, studies using a variety of methodologies (aphasia, neuroimaging,

cortical stimulation mapping) provide converging evidence for the uniformity of neural
systems underlying spoken and signed languages. However, numerous studies have also
revealed instances where sign-language processing invokes neural resources that differ
from those observed for spoken languages. These instances illuminate the intimate
connections between language modality and brain structures that support language
processing. In the following, we introduce several research areas that highlight the roles
of linguistic modality, sensory experience, and language development, and which un-
derlie the cortical organization of sign language processing.
The Right Hemisphere

The once popular premise that the left hemisphere plays an exclusive role in the pro-
cessing of spoken language has given way to a more nuanced assessment of the cerebral
distribution of language functions (McGettigan & Scott, 2012). While researchers have
recognized that the right hemisphere may contribute to extra-linguistic functions such
as discourse and pragmatic processes (see Rapp, Chapter 28 in this volume), we also
observe new evidence that both hemispheres participate in core aspects of speech per-
ception (see, for example, Poeppel, 2003; Scott, Blank, Rosen, & Wise, 2000; Shalom &
Poeppel, 2007; Zatorre & Belin, 2001; see also Poeppel, Cogan, Davidesco, & Flinker,
Chapter 26 in this volume). Within the realm of sign-language processing there have
been many indications that the right hemisphere may also play a heightened role in the
mediation of the visuospatial properties of signed languages.
Facial Processing
Imaging studies of sign-language processing have often noted significant right hemi-
sphere activation. For example, activations in right hemisphere superior temporal,
inferior frontal, and posterior parietal regions have been reported for sentence-level
processing tasks (e.g., MacSweeney et al., 2006; MacSweeney et al., 2002a, 2002b;
Neville et al., 1998; Newman, Bavelier, Corina, Jezzard, & Neville, 2002). The question
of whether these patterns of activation are unique to sign has been the topic of debate
(Corina, Neville, & Bavelier, 1998; Hickok, Bellugi, & Klima, 1998). Mounting evidence
suggests that some aspects of this right hemisphere activation may be attributable to the
processing of facial information, which factors significantly in the sign signal. Studies
of auditory and audiovisual speech have observed right hemisphere activations that ap-
pear similar to those reported in signing (Capek et al., 2004; Davis & Johnsrude, 2003;
Schlosser, Aoyagi, Fulbright, Gore, & McCarthy, 1998).
Sentence Comprehension
Researchers have speculated that right hemisphere activation (and damage, in
cases of aphasia) is associated with the linguistic use of space in sign language.
Signed languages routinely capitalize on the postural properties of the body, the
manual articulators, and the spatial affordances of the visual system to convey com-
plex meanings, including grammatical roles (such as subject/object), prepositional
meaning, locative relations, and speaker viewpoint in ways that may not have direct
parallels in spoken languages.
One area in which spatial processing interacts with the grammar of signed languages
comes in the form of so-called classifier constructions. Signed languages are unique
in their ability to spatially represent prepositional relationships between objects, such
as on, above, under, and so on. These concepts are often conveyed via the depiction of
the physical relation itself, rather than encoded by a discrete lexical item. Many signed
languages express spatial relationships and events in this manner and have discrete
inventories of highly productive grammatical forms, often referred to as “classifiers”
or “classifier predicates,” which participate in a wide range of depictive constructions
(Zwitserlood, 2012). The linguistic status of these classifier forms and their conventions
of use have important implications for our understanding of the neurolinguistics of
signed languages.
For instance, in a positron emission tomography (PET) study by Emmorey et al.
(2002), deaf subjects were asked to examine line drawings of two spatially arrayed
objects, and to produce either a classifier description or a description of the spa-
tial relationship using ASL lexical prepositions. This study found evidence for right
hemisphere SMG activation for both prepositional forms and classifiers compared
to object naming; however, the direct comparison between classifier constructions
and lexical prepositions revealed only left hemisphere inferior parietal lobule acti-
vation. This counterintuitive finding suggests that right hemisphere activation must
be related to some common process, perhaps the spatial analysis of the stimulus
to be described, rather than a special spatial-linguistic property of ASL classifiers
per se.
Data from aphasic signers support these conclusions. Atkinson and colleagues
conducted a group study of signers of BSL with damage to the left or right hemi-
sphere (Atkinson et al., 2005). They devised comprehension tests that examined a
wide range of sentence types, including tests of classifier placement, orientation,
and rotation. Their findings indicated that signers with right hemisphere damage
differed from age-matched controls only on tests of locative relationships expressed
via classifier constructions, and on classifier placement, orientation, and rota-
tion (see also Hickok, Pickell, Klima, & Bellugi, 2009). As signers with left hem-
isphere damage exhibited deficits on all comprehension tests, including classifier
constructions, this suggests that the comprehension of classifier constructions
requires left hemisphere resources as well as intact right hemisphere visuospatial
processing mechanisms. Atkinson et al. (2005, p. 247) state, “some people with RH
damage can no longer map non-arbitrary sign locations on to real-world spatial
positions.” That is, in the case of right hemisphere damage, the signers’ difficulties
stem from a more general extra-linguistic visuospatial deficit, rather than linguistic
malfunction.
Discourse
Spatial relationships also figure into larger, grammatically determined discourse
elements in sign language. An analysis of language use in two right hemisphere lesioned
patients, J. H. and D. N. (Emmorey, Corina, & Bellugi, 1995), reveals the subtle way in
which spatial processing interacts with discourse content and cohesion.
Patient D. N. had a right hemisphere lesion that was predominantly medial, and in-
volved the upper part of the occipital lobe and superior parietal lobule. In everyday
signing and in picture-description tasks, D. N. showed a pattern of language disrup-
tion that affected discourse cohesion via impaired spatial indexing abilities. Spatial
indexing is a commonly used device for pronominal reference in signed languages.
Signers will point to the spatial location of a previously signed nominal, effectively re-
referencing this previously introduced concept. Although D. N. showed unimpaired
spatial indexing abilities within sentences, her use of spatial indexing across sentences
was inconsistent; that is, she did not consistently use the same indexic points to refer to
individuals throughout a discourse.1 In order to salvage intelligibility, D. N. used a com-
pensatory strategy in which she restated the noun phrase in each sentence, resulting in
an overly repetitive discourse style.
The case study of J. H., a deaf signer with a large right hemisphere lesion involving
frontal, temporal and parietal cortex, illustrates yet another pattern in the complex re-
lationship between visuospatial processing sign-language abilities. J. H. had severely
compromised visuospatial abilities, but nevertheless exhibited remarkably little im-
pairment in sign-language comprehension (Corina, Kritchevsky, & Bellugi, 1996).
J. H. produced occasional nonsequiturs and showed abnormal attention to detail—
behaviors that are typically found in the discourse of hearing patients with right hem-
isphere lesions. J. H. also exhibited persistent left-neglect, such that if two identical
objects were presented for identification, one in the left visual field and the other in the
right visual field, J. H. would show consistent extinction for the physical object in his left
visual field (see Corina et al., 1996, for a discussion). Despite this, J. H. showed little to
no evidence that this visuospatial neglect affected his perception of ASL. Formal testing
showed that even in cases of face-to-face signing, where the perception of a two-handed
sign such as “deer” (the right and left hands, finger spread, are placed on each side of the
forehead like antlers) could easily be perceived as the well-formed one-handed sign “fa-
ther” (only the dominant hand, fingers spread, is placed in the forehead), J. H. faithfully
reports “deer.” It is speculated that J. H. was able to use tacit representational knowledge
of ASL formational constraints on two-handed signs to process the otherwise neglected
information. Similar interactions between abstract lexical representations and atten-
tional neglect have been reported in some neglect-dyslexics (Behrmann, Moscovitch,
Black, & Mozer, 1990).
1
Contrast this impairment with previously discussed patient P. D., who had left hemisphere damage
and showed a lack of spatial agreement within single sentences.
Multilingualism in
the Deaf Community
Bilingualism
A prominent question in the field of neurolinguistics is whether the neural represen-
tation of language changes as a function of the number of languages one knows (see
Paradis, 2004, for a review; see, in this volume, Green & Kroll, Chapter 11, and Paz-
Alonso, Oliver, Quiñones & Carreiras, Chapter 24). This question takes on additional
complexities when considering the expression of multilingualism in the Deaf commu-
nity, particularly as bilingualism is both the norm and quite variable across individuals.
Studies of bilingualism in Deaf communities afford unique opportunities to explore the
role of language modality in the expression of neural systems and to evaluate issues re-
lated to cognitive control across languages and modalities, as well as issues related to
delayed language exposure.
Deaf individuals who have been exposed to and acquired a sign language from birth
by virtue of having deaf signing parents are referred to as native signers, for whom a
sign language is considered their native language (L1). As these signers typically receive
significant educational training based on English, they are also in large part bilingual.
However, the language that constitutes the second language (L2) varies, from spoken
or written English to a second signed language (e.g., DGS). There is also considerable
variability in the linguistic competences in L2 regardless of mode of L2 (spoken, cued,
written, etc.), as is observed with spoken-language bilinguals.
To date, the majority of studies investigating the neural representation of signed lan-
guages have been conducted on deaf native signers. Comparisons between different
populations of sign language users are of great interest, however, as they serve to elu-
cidate the role of early language experience and deafness on cortical organization. One
primary group of comparison is hearing native signers (i.e., hearing children who have
deaf signing parents), who often acquire a sign language as their L1 in conjunction with
spoken English. These individuals have been termed “bimodal bilinguals,” illustrating
the fact that the two languages known are expressed in different modalities.
Neuroimaging studies of hearing native signers indicate considerable overlap in cor-
tical regions that support sign-language and spoken-language comprehension. This ob-
servation is consistent with studies of unimodal bilinguals that indicate shared neural
systems mediating L1 and L2. However, some differences in the engagement of left and
right hemisphere parietal regions have been observed. For instance, MacSweeney et al.
(2002b) tested deaf and hearing native signers in a paradigm that utilized BSL sentence
contexts that either made use of spatial-topographic signing space or did not require
spatial mapping. While native deaf signers showed activation in the right angular gyrus
(BA 39) for spatial-topographic sentences, parietal activation in the hearing native
signers was modulated by accuracy on the task.
Neville et al. (1998) compared native deaf signers with native hearing signers pro-
cessing English and ASL sentences. In this case, Neville et al. (1998) report differences
in lateralization across the two groups, with hearing native signers more strongly left-
lateralized than the deaf signers. Hearing native signers also showed greater activation
in posterior parietal regions, for example intraparietal sulcus (IPS) and anterior occip-
ital sulcus, that may be indicative of greater attentional demands and visual-feature pro-
cessing during the ASL task. This increased processing for the hearing native signers is
consistent with the claim that this group may hold English as their dominant language
even if exposed to ASL from birth. Both groups also showed considerable precentral
and central sulcus activation in the ASL condition, which is not typically observed in
studies of spoken language monolinguals. This may be an indication of greater acti-
vation of manual, facial, and body-related pre-motor and motor regions during sign
comprehension.
Another group comparison is between native deaf signers and deaf sign language
users who were exposed to their L1 at a relatively late age, ranging from early child-
hood to adulthood. These late learners may also have secondary L1 based on ex-
plicit and extensive oral English training, though outcomes are variable, and some
individuals may stop using spoken English as they grow into young adulthood. There
are well-documented consequences of delayed language exposure (Mayberry, 2010;
Mayberry & Eichen, 1991). A study by Newman et al. (2002) also finds late exposure to
sign language to contribute significantly to cortical organization even within hearing
participants. In this study, deaf and hearing native signers, hearing late learners of ASL,
and hearing non-signers viewed sign-language sentences and sign gibberish. Deaf and
hearing native signers showed significant activation in right hemisphere posterior-
parietal and posterior-temporal regions. These activation patterns were not seen in
non-signers, nor were they observed in hearing late learners of sign language. A group
analysis of hearing participants confirmed that only hearing native users of ASL, but
not late learners who had learned ASL after puberty, recruited the right angular gyrus
during this task. Newman and colleagues suggested that the activation of this neural
region may be a signature of sign language being acquired during the critical period for
language.
Finally, data from spoken-language bilinguals strongly argue for distinguishing
neural regions serving aspects of language control from those involved in language
switching (see Green & Kroll, Chapter 11 in this volume). Some psycholinguistic studies
have argued that there remains less potential for modality-based inference in sign-
speech bilinguals owing to the difference in input and output modalities (for example,
lack of shared phonological representations). Indeed, Emmorey et al. (2008) report
that bimodal bilinguals fail to show enhancements in executive control that are claimed
to arise from implicit suppression and selection of languages in unimodal bilinguals.
Whether there is less need for subcortical, frontal parietal, and basal ganglia involve-
ment in adjudicating the demand of two languages when they are expressed in decid-
edly different modalities is an active area of research (see Emmorey, Giezen, & Gollan,
2015, for a review of the psycholinguistic literature).
Recent fMRI studies have reported resting-state and task-dependent functional con-
nectivity analyses of the dorsal anterior cingulate (dACC) and the left caudate nucleus
(CN) in a small group of late Mandarin-CSL bimodal bilinguals (L1 spoken Mandarin,
L2 Chinese Sign Language) (Li et al., 2015; Li et al., 2016). The modality of naming
(speech or sign) modulated the functional connectivity of the dACC either to a pre-
defined speech-based region in the left middle superior temporal gyrus (STG) or to a
predefined sign language region in the left superior pre/postcentral gyrus (PCG). The
increase in functional connectivity may reflect the control process of monitoring and
promoting the activation level of the target language. Functional connectivity between
the left CN and left PCG (the region for signed language) was similar when producing
either CSL or Mandarin within a single-language context. The left CN was more
connected with the left STG (the region for spoken language) when producing CSL than
when producing Mandarin. This asymmetry may reflect an inherent dependency of L2
(CSL) on L1 (Mandarin) in this population and the left CN’s greater role in controlling
the less proficient language (Tan et al., 2011; Zou et al., 2012). These initial studies suggest
that despite differences in modality, cognitive control mechanisms are engaged during
single-language and dual-language tasks in ways that appear similar to language control
in spoken-language bilinguals.
Reading
Studies investigating the processing of written language in deaf populations are of in-
terest for two reasons. First, as noted earlier, the L2 of many signers, both native and
late learners, is a written form of a language such as English. This is itself a unique situ-
ation in terms of bilingual language processing and linguistic representation. Second,
many deaf individuals do not reach a high level of reading competency, despite years
of instruction and educational interventions. Studying orthographic processing in deaf
individuals can illuminate the relationship between cognitive organization and reading
proficiency.
Data from deaf bilinguals have contrasted patterns of activation for sign language
and written language. Neville and colleagues (1998) examined responses in deaf and
hearing native signers, and a control group of hearing monolinguals, while participants
read English sentences. The hearing native signers’ activation during English sentence
reading was a subset of activation seen in the monolingual hearing control group and
included prominent frontal lobe activation, including the dorsolateral prefrontal cortex
(DLPC) and Broca’s area, posterior STS, and posterior parietal regions including angular
gyrus. This activity was strongly left-lateralized and showed that regions active during
English reading were also active while observing ASL signing. In contrast, deaf native
signers’ responses to written English were quite variable and statistically much weaker.
While left frontal regions, including the DLPC, Broca’s area, and precentral sulcus, were
activated during reading, superior temporal sulcus activation was limited to middle
and posterior regions, with the greatest activity seen in the right hemisphere, a pattern
that contrasts with both monolingual controls and hearing native signers. Activation in
posterior parietal regions, including IPS, anterior occipital sulcus, and angular gyrus,
were largely bilateral and more strongly right-lateralized in the deaf participants; these
patterns were not observed in either of the hearing groups.
Subsequent work has indicated that patterns of activation for L2 reading in ASL
signing deaf participants may vary as a function of proficiency. Corina et al. (2013)
examined neural activation during implicit word reading. Proficient deaf readers ac-
tivated left inferior frontal gyrus and left middle and superior temporal gyrus in a
pattern that is consistent with regions reported in hearing readers. In contrast, the less-
proficient readers exhibited a pattern of responses characterized by inferior and middle
frontal lobe activation that was stronger on the right hemisphere. This bears some sim-
ilarity to areas reported in studies of logographic reading, raising the possibility that
these individuals are using a qualitatively different mode of orthographic processing
than is traditionally observed in hearing individuals reading sound-based scripts. The
variability may also point to differences in age of acquisition/mastery, and differential
reliance upon declarative memory systems.
Cross-modal Plasticity
Studies have routinely reported activation of the temporal lobes in response to
signing. Petitto et al. (2000), and Nishimura et al. (1999) reported significant acti-
vation in the left superior temporal cortex, a region often associated with auditory
processing, in response to the perception of single signs. MacSweeney et al. (2002b)
also observed the activation of auditory association areas in deaf and hearing native
signers of BSL. Their findings revealed relatively greater activation for deaf native
signers than hearing native signers in the left temporal auditory association cortex
during the perception of signed sentences (see also Kassubek, Hickok, Erhard, &
2004). These findings are taken as evidence of cross-modal plasticity whereby audi-
tory cortex and auditory association cortex may be modified in the absence of audi-
tory input to become specialized for visual language input. While this left hemisphere
auditory activation is presumed to reflect linguistic processing, the functional signif-
icance of these activations is not well understood. Cardin et al. (2013) has presented
data that suggest that right superior temporal lobe activation reflects cross-modal
plasticity related to auditory deprivation.
A small number of morphometric studies that examine the presence of anatomical
differences in the brains of deaf and hearing participants have provided further evi-
dence for cross-modal plasticity. Two studies have reported reduced white matter in the
left posterior superior temporal gyrus in deaf subjects, but no difference in gray matter
volume in temporal auditory and speech areas (Emmorey, Allen, Bruss, Schenker, &
Damasio, 2003; Shibata, 2007). It is speculated that the reduced white matter volume
may indicate a hypoplasia in the development of specific tracts related to speech. The
finding that auditory cortices show no differences in gray matter volume has been taken
as evidence for preserved functionality of these regions, perhaps in the form of cross-
modal plasticity.
Sign Language and

Neurodegenerative Diseases
There is growing interest in the behavioral sequela of sign-language processing in the

face of neurodegenerative diseases. These studies hold potential for understanding the
contributions of specific neurotransmitter systems and synaptic molecular mechanism
in the expression of sign language behaviors.
Parkinson’s Disease
Parkinson’s disease is a degenerative disorder of the brain that affects the extrapyramidal
motor system and is mainly characterized by progressive tremor, bradykinesia (slow-
ness of movement), and rigidity. Degeneration of the substantia nigra with subsequent
decrease of striatial dopamine is responsible for the motor and cognitive disturbances
seen in Parkinson’s patients (Rodriguez-Oroz et al., 2009). The motor deficits observed
in these subjects are not specific to language, but are evidenced across the general do-
main of motor behaviors (see Copland & Angwin, Chapter 33 in this volume).
Studies of deaf signers with Parkinson’s disease highlight the importance of the ex-
trapyramidal motor system and basal ganglia in the mediation of sign language. The
sign-language deficits observed in Parkinsonian signing are an expression of the
generalized motor impairments common in Parkinson’s disease. Poizner and colleagues
have presented several case studies of Parkinsonian signers (Brentari, Poizner, & Kegl,
1995). Signers with Parkinson’s disease have been described as signing in a monotonous
fashion, with a severely restricted range of temporal rates and tension in signing, two
features that normally serve to signal emphasis. Accompanying restrictions in limb
movements are deficits in the motility of facial musculature, which further reduce ex-
pressivity in signing.
In Parkinsonian signing, the scope of sign movement and the precision of articula-
tion are compromised. In Parkinsonian signers, physical contacts between the major
articulators and body locations may be entirely lost. The lack of contact is consistent
with a description of overall smoothing of articulation in signers with Parkinson’s di-
sease; maintenance of physical contacts with body locations probably places signifi-
cant demands on the updating and programming of movement plans used in signing.
There is often a systematic reduction of sign articulation as well, which results in signs
being articulated at the torso (as opposed to distinct, lexically specified locations).
For example, Brentari and Poizner (1994) report their subject R. H. as signing “boy,”
normally articulated on the forehead, at the torso level. In fact, most of R. H.’s signs
were articulated near the waist during free conversation. The handshapes of signers
with Parkinson’s disease are often quite lax, and do not maintain the crisp closure
or opening of finger postures seen in normal signing. Importantly, the handshapes
observed (and indeed all the articulatory distinctions) are approximations of the
intended form, and not incorrect substitutions (Brentari & Poizner, 1994). These
findings are consistent with a general disruption of motor control that leads to pho-
netic disruptions in signing. This can be contrasted to the phonemic disruption ap-
parent in aphasic signing.
Alzheimer’s Disease
Recent studies of patients with suspected Alzheimer’s disease reveal the relationship
between memory deficits and sign language. A case study reported by Falchook and
colleagues (2013) illustrates a patient with deficits in episodic and working memory that
resulted in a tendency to produce short sentences, and deficits in sign comprehension,
syntax, and pronouns, as well as sign fluency, sign repetition, and naming problems
(anomia). A PET scan revealed hypometabolism bilaterally in the parietal and anterior
temporal lobes (greater in the left hemisphere than right), consistent with a diagnosis of
Alzheimer’s dementia. Synaptic loss in the cortex is the major correlate of the patterns of
cognitive decline in Alzheimer’s disease.
While there was no evidence of semantic or formational paraphasias in this patient’s
signing, two specific deficits were particularly striking: impaired fingerspelling2 com-
prehension and recall, and impaired sentence repetition. In deaf signers, fMRI studies
of fingerspelling comprehension show activation of the visual word form area, which
is more left-lateralized than signs, paralleling the difference between reading and lis-
tening for spoken language (Emmorey, McCullough, & Weisberg, 2015). The deficits
in sentence repetition were, on the other hand, attributed to impairments in working
memory for verbal and visual spatial abilities. This is consistent with neuroimaging
studies of working memory processes in users of sign language, which also have re-
ported modality-specific activations in response to the perception and maintenance
of sign-based information. These activations typically include bilateral temporal-
occipital activations and activations in the posterior parietal cortex (Buchsbaum
et al., 2005; Pa, Wilson, Pickell, Bellugi, & Hickok, 2008; Rönnberg, Rudner, &
Ingvar, 2004).
2 Fingerspellings used in signed languages are often considered ancillary systems that are used to
borrow orthographic representations into a manual form. Fingerspelling in ASL requires complex
sequences of handshapes that represent letters of the English alphabet.
Sign Language and Human

Action Understanding
There is growing interest in the relationship between sign-language processing and

human gesture understanding, fueled by mirror neuron-based accounts of language
evolution and language processing (Arbib, 2005, 2008). Rizzolatti and Arbib (1998)
propose that a human analog of the bilateral fronto-parietal macaque mirror neuron
system that subserves action execution and action perception supports a variety of
complex socio-cognitive phenomena, most notably human action understanding and
human language. Signed languages provide an important opportunity to examine such
hypotheses, as sign production requires the orchestration of complex manual gestural
forms and its comprehension depends on the direct perception of these same gestures.
Some support for the involvement of a unified action perception/action execution
system for sign language comes from consideration of the functional role of the pari-
etal lobes in sign language behaviors. Mirror neuron system accounts of language pro-
cessing hold that one and the same region should participate both in the production and
comprehension of language forms. Corina and Knapp (2006) reviewed neuroimaging
data from studies of sign-language production and sign-language comprehension and
examined regions that showed significant overlap. One region that showed compelling
overlap was in the left SMG of the inferior parietal lobe. Lesion studies have shown ev-
idence for sign-comprehension deficits with damage to left inferior parietal regions,
and both lesion and stimulation mapping studies have shown impairment in sign pro-
duction implicating the SMG. This is a region that is often observed in studies of sign-
language working memory (Buchsbaum et al., 2005; Pa et al., 2008; Rönnberg et al.,
2004). In hearing individuals, this region is considered a polymodal association region,
with involvement in tasks that include phonological short-term memory (Jacquemot &
Scott, 2006), judgments of hand gestures (Hermsdorfer et al., 2001), motor attention to
limb movements (Rushworth, Krams, & Passingham, 2001), and phonological agraphia
(Alexander, Friedman, Loverso, & Fischer, 1992). Given this multiplicity of functions
of the SMG, it is perhaps not surprising that aspects of sign comprehension and sign
production are subserved by this region. Whether individual neurons within this re-
gion exhibit mirror properties remains an important question (see Chong, Cunnington,
Williams, Kanwisher, & Mattingley, 2008, for example).
It must be noted, however, that data from lesion and neuroimaging studies also place
some limits on the feasibility of the mirror neuron account of sign language. For ex-
ample, researchers have suggested that Broca’s region is a likely homologue of ma-
caque F5, a frontal ventral region where neurons with mirroring properties have
been observed (Gallese, Fadiga, Fogassi, & Rizzolatti, 1996). Yet, as discussed, studies
of aphasic patients show that damage to left frontal ventral and opercular regions do
not result in comprehension deficits in signers, as would be predicted from a mirror
neuron account of human language. Recent group-level analyses of signing aphasics
reinforce this point (Rogalsky et al., 2013). In addition, case studies of deaf signers have
shown compelling dissociations between nonlinguistic pantomime and sign-language
abilities. Patient W. L., for example, produced and comprehended pantomime normally,
but demonstrated marked deficits in sign-language production and comprehension
(Corina, et al., 1992). His gestures were clearly intended to convey symbolic information
that he ordinarily would have imparted with sign language. Marshall et al. (2004) report
on the dissociation of sign and gesture in Charles, an aphasic user of BSL. His ability to
use nonlinguistic gesture was intact, while sign language expression showed significant
impairment.
These accounts are difficult to reconcile within a human mirror neuron system view
of action execution and perception that fails to distinguish between different classes
of human actions. Data from neuroimaging studies that have directly compared the
perception of different classes of human actions and signs in deaf signers also show
little evidence of overlap in the neural system mediating the perception of transi-
tive and intransitive human actions, pantomimes, and linguistic signs (Corina et al.,
2007; Emmorey, Xu, Gannon, Goldin-Meadow, & Braun, 2010). In these studies, sign-
language forms activate the familiar perisylvian language areas, while nonlinguistic
human actions and pantomimes show prominent activation in the inferior occipital-
temporal regions in deaf signers. One explanation for these differences is that deaf
signers may show a greater reliance on top-down processing in the recognition of signs,
leading to more automatic and efficient early visual processing of highly familiar lin-
guistic features. In contrast, nonlinguistic gesture detection may be driven by bottom-
up processing, in which preliminary visual analysis is crucial to interpretation of
these forms.
Conclusions
Patterns of deficits seen in deaf signers following left and right hemisphere lesions
show great commonalities to the deficits seen in users of spoken languages. Functional
imaging studies of sign-language comprehension and production in deaf and hearing
signers provide further evidence for the uniformity in the neural systems underlying
spoken and signed languages. Taken together, these studies provide evidence for a
core neurobiological system that underlies human language, regardless of the mo-
dality of expression (Fedorenko, Behr, & Kanwisher, 2011). At the same time, studies
of sign language have also revealed instances in which the properties of sign languages
invoke neural- processing resources that differ from those observed for spoken lan-
guages. These instances illuminate the intimate connections between language mo-
dality and brain structures for language. Structural imaging studies of deaf signers
have begun to observe and differentiate the roles of sensory and language experience
in sculpting cortical connections, while neurolinguistics studies of signed languages
have been used to both motivate and constrain models of language evolution and
language biology. Studies of signed languages and the experiences of deaf individuals
provide an important means to advance our understanding of the neurobiology of
human language.
Acknowledgments
This work was support by grants NIH NIDCD R01 DC011538 and RO1 DC014767.
References
Alexander, M. P., Friedman, R. B., Loverso, F., & Fischer, R. S. (1992). Lesion localization of pho-
nological agraphia. Brain and Language, 43(1), 83–95. doi: 10.1016/0093-934X(92)90022-7
Arbib, M. A. (2005). From monkey-like action recognition to human language: An evolu-
tionary framework for neurolinguistics. Behavioral and Brain Sciences, 28(02), 105–124.
doi: 10.1016/j.jphysparis.2008.03.001
Arbib, M. A. (2008). From grasp to language: Embodied concepts and the challenge of abstrac-
tion. Journal of Physiology-Paris, 102(1),  4–20.
Atkinson, J., Marshall, J., Woll, B., & Thacker, A. (2005). Testing comprehension abilities
in users of British Sign Language following CVA. Brain and Language, 94(2), 233–248.
doi: 10.1016/j.bandl.2004.12.008
Bavelier, D., Corina, D., Jezzard, P., Clark, V., Karni, A., Lalwani, A., Rauschecker, J. P., Braun,
A., Turner, R., & Neville, H. J. (1998). Hemispheric specialization for English and ASL: Left
invariance‐right variability. Neuroreport, 9(7), 1537–1542.
Behrmann, M., Moscovitch, M., Black, S. E., & Mozer, M. (1990). Perceptual and conceptual
mechanisms in neglect dyslexia. Brain, 113(4), 1163–1183. doi: 10.1093/brain/113.4.1163
Bellugi, U., Poizner, H., & Klima, E. S. (1989). Language, modality and the brain. Trends in
Neurosciences, 12(10), 380–388. doi: 10.1016/0166-2236(89)90076-3
Brentari, D., & Poizner, H. (1994). A phonological analysis of a deaf Parkinsonian signer.
Language and Cognitive Processes, 9(1), 69–99. doi: 10.1080/01690969408402110
Brentari, D., Poizner, H., & Kegl, J. (1995). Aphasic and Parkinsonian signing: Differences in
phonological disruption. Brain and Language, 48(1), 69–105. doi: 10.1006/brln.1995.1003
Buchsbaum, B., Pickell, B., Love, T., Hatrak, M., Bellugi, U., & Hickok, G. (2005). Neural
substrates for verbal working memory in deaf signers: fMRI study and lesion case report.
Brain and Language, 95(2), 265–272. doi: 10.1016/j.bandl.2005.01.009
Capek, C. M., Bavelier, D., Corina, D., Newman, A. J., Jezzard, P., & Neville, H. J. (2004). The
cortical organization of audio-visual sentence comprehension: An fMRI study at 4 Tesla.
Cognitive Brain Research, 20(2), 111–119. doi: 10.1016/j.cogbrainres.2003.10.014
Cardin, V., Orfanidou, E., Rönnberg, J., Capek, C. M., Rudner, M., & Woll, B. (2013).
Dissociating cognitive and sensory neural plasticity in human superior temporal cortex.
Nature Communications, 4, 1473. doi: 10.1038/ncomms2463
Chiarello, C., Knight, R., & Mandel, M. (1982). Aphasia in a prelingually deaf woman. Brain,
105(1), 29–51. doi: 10.1093/brain/105.1.29
Chong, T. T. J., Cunnington, R., Williams, M. A., Kanwisher, N., & Mattingley, J. B. (2008).
fMRI adaptation reveals mirror neurons in human inferior parietal cortex. Current Biology,
18(20), 1576–1580. doi: 10.1016/j.cub.2008.08.068
Corina, D., Chiu, Y. S., Knapp, H., Greenwald, R., San Jose-Robertson, L., & Braun, A. (2007).
Neural correlates of human action observation in hearing and deaf subjects. Brain Research,
1152, 111–129. doi: 10.1016/j.brainres.2007.03.054
Corina, D. P. (1998a). Aphasia in users of signed language. In P. Coppens, Y. Lebrun, & A. Basso
(Eds.), Aphasia in atypical populations (pp. 261–309). Mahwah, NJ: Lawrence Erlbaum
Associates.
Corina, D. P. (1998b). The processing of sign language: Evidence from aphasia. In H.
Whitaker & B. Stemmer (Eds.), Handbook of neurology (pp. 313–329). San Diego,
CA: Academic Press.
Corina, D. P., & Knapp, H. (2006). Sign language processing and the mirror neuron system.
Cortex, 42(4), 529–539. doi: 10.1016/S0010-9452(08)70393-9
Corina, D. P., Kritchevsky, M., & Bellugi, U. (1996). Visual language processing and unilateral
neglect: Evidence from American Sign Language. Cognitive Neuropsychology, 13(3), 321–356.
doi: 10.1080/026432996381935
Corina, D. P., Lawyer, L. A., & Cates, D. (2012). Cross-linguistic differences in the neural
representation of human language: Evidence from users of signed languages. Frontiers in
Psychology, 3, 587. doi: 10.3389/fpsyg.2012.00587
Corina, D. P., Lawyer, L. A., Hauser, P., & Hirshorn, E. (2013). Lexical processing in deaf
readers: An fMRI investigation of reading proficiency. PLoS One, 8(1), e54696. doi: 10.1371/
journal.pone.0054696
Corina, D. P., McBurney, S. L., Dodrill, C., Hinshaw, K., Brinkley, J., & Ojemann, G. (1999).
Functional roles of Broca’s area and SMG: Evidence from cortical stimulation mapping in a
deaf signer. NeuroImage, 10(5), 570–581. doi: 10.1006/nimg.1999.0499
Corina, D. P., Neville, H. J., & Bavelier, D. (1998). Response from Corina, Neville and Bavelier.
Trends in Cognitive Sciences, 2(12), 468–470. doi: 10.1016/S1364-6613(98)01251-0
Corina, D. P., Petersen, S. Farnady, C., Rogalsky, C., Bellugi, U., & Hickok, G. (2014). Effects of
subcortical lesions on the production of American Sign Language. Annual meeting of the
Cognitive Neuroscience Society, Boston, MA.
Corina, D. P., Poizner, H., Bellugi, U., Feinberg, T., Dowd, D., & O’Grady- Batch, L.
(1992). Dissociation between linguistic and nonlinguistic gestural systems: A case for
compositionality. Brain and Language, 43(3), 414–447. doi: 10.1016/0093-934X(92)90110-Z
Corina, D. P., San Jose-Robertson, L., Guillemin, A., High, J., & Braun, A. R. (2003). Language
lateralization in a bimanual language. Journal of Cognitive Neuroscience, 15(5), 718–730.
doi: 10.1162/jocn.2003.15.5.718
Damasio, A. R. (1992). Aphasia. The New England Journal of Medicine, 326(8), 351–539.
Davis, M. H., & Johnsrude, I. S. (2003). Hierarchical processing in spoken language compre-
hension. The Journal of Neuroscience, 23(8), 3423–3431.
Dronkers, N., & Baldo, J. (2009). Language: Aphasia. In L. R. Squire (Ed.), Encyclopedia of neu-
roscience (Vol. 5, pp. 343–348). Oxford: Academic Press.
Dronkers, N. F., Redfern, B. B., & Ludy, C. A. (1995). Lesion localization in chronic Wernicke’s
aphasia. Brain and Language, 51(1),  62–65.
Dronkers, N. F., Redfern, B. B., & Knight, R.T. (2000). The neural architecture of language
disorders. In M. S. Gazzaniga (Ed.), The new cognitive neurosciences, (pp. 949–958).
Cambridge, MA: MIT Press.
Emmorey, K., Allen, J. S., Bruss, J., Schenker, N., & Damasio, H. (2003). A morphometric
analysis of auditory brain regions in congenitally deaf adults. Proceedings of the National
Academy of Sciences, 100(17), 10049–10054. doi: 10.1073/pnas.1730169100
Emmorey, K., Corina, D. P., & Bellugi, U. (1995). Differential processing of topographic and
referential functions of space. In K. Emmorey and J. S. Reilly (Eds.), Language, gesture, and
space (pp. 43–62). Hillsdale, NJ: Lawrence Erlbaum Associates.
Emmorey, K., Damasio, H., McCullough, S., Grabowski, T., Ponto, L. L., Hichwa, R. D., &
Bellugi, U. (2002). Neural systems underlying spatial language in American Sign Language.
NeuroImage, 17(2), 812–824. doi: 10.1006/nimg.2002.1187
Emmorey, K., Giezen, M. R., & Gollan, T. H. (2015). Psycholinguistic, cognitive, and neural
implications of bimodal bilingualism. Bilingualism: Language and Cognition, 19(2), 223–242.
http://doi.org/10.1017/S1366728915000085
Emmorey, K., Luk, G., Pyers, J. E., & Bialystok, E. (2008). The source of enhanced cognitive
control in bilinguals: Evidence from bimodal bilinguals. Psychological Science, 19(12), 1201–
1206. doi: 10.1111/j.1467-9280.2008.02224.x
Emmorey, K., McCullough, S., & Weisberg, J. (2015). Neural correlates of fingerspelling,
text, and sign processing in deaf American Sign Language–English bilinguals. Language,
Cognition and Neuroscience, 30(6), 749–767. doi: 10.1080/23273798.2015.1014924\
Emmorey, K., Mehta, S., & Grabowski, T. J. (2007). The neural correlates of sign versus word
production. NeuroImage, 36(1), 202–208. doi: 10.1016/j.neuroimage.2007.02.040
Emmorey, K., Xu, J., Gannon, P., Goldin-Meadow, S., & Braun, A. (2010). CNS activation and
regional connectivity during pantomime observation: No engagement of the mirror neuron
system for deaf signers. NeuroImage, 49(1), 994–1005. doi: 10.1016/j.neuroimage.2009.08.001
Falchook, A. D., Mayberry, R. I., Poizner, H., Burtis, D. B., Doty, L., & Heilman, K. M. (2013).
Sign language aphasia from a neurodegenerative disease. Neurocase, 19(5), 434–444.
doi: 10.1080/13554794.2012.690427
Fedorenko, E., Behr, M. K., & Kanwisher, N. (2011). Functional specificity for high-level lin-
guistic processing in the human brain. Proceedings of the National Academy of Sciences,
108(39), 16428–16433. doi: 10.1073/pnas.1112937108
Gallese, V., Fadiga, L., Fogassi, L., & Rizzolatti, G. (1996). Action recognition in the premotor
cortex. Brain, 119(2), 593–609. doi:10.1093/brain/119.2.593
Groce, N. E. (1985). Everyone here spoke sign language: Hereditary deafness on Martha’s
Vineyard. Cambridge, MA: Harvard University Press.
Hermsdörfer, J., Goldenberg, G., Wachsmuth, C., Conrad, B., Ceballos-Baumann, A. O.,
Bartenstein, P., Schwaiger, M., & Boecker, H. (2001). Cortical correlates of gesture pro-
cessing: Clues to the cerebral mechanisms underlying apraxia during the imitation of mean-
ingless gestures. NeuroImage, 14(1), 149–161. doi: 10.1006/nimg.2001.0796
Hickok, G., Bellugi, U., & Klima, E. S. (1998). What’s right about the neural organization of sign
language? A perspective on recent neuroimaging results. Trends in Cognitive Sciences, 2(12),
465–468. doi: 10.1016/S1364-6613(98)01263-7
Hickok, G., Kritchevsky, M., Bellugi, U., & Klima, E. S. (1996). The role of the left frontal oper-
culum in sign language aphasia. Neurocase, 2(5), 373–380. doi: 10.1080/13554799608402412
Hickok, G., Love-Geffen, T., & Klima, E. S. (2002). Role of the left hemisphere in sign language
comprehension. Brain and Language, 82(2), 167–178. doi: 10.1016/S0093-934X(02)00013-5
Hickok, G., Pickell, H., Klima, E., & Bellugi, U. (2009). Neural dissociation in the produc-
tion of lexical versus classifier signs in ASL: Distinct patterns of hemispheric asymmetry.
Neuropsychologia, 47(2), 382–387. doi: 10.1016/j.neuropsychologia.2008.09.009
Horwitz, B., Amunts, K., Bhattacharyya, R., Patkin, D., Jeffries, K., Zilles, K., & Braun, A. R.
(2003). Activation of Broca’s area during the production of spoken and signed language: A
combined cytoarchitectonic mapping and PET analysis. Neuropsychologia, 41(14), 1868–

1876. doi: 10.1016/S0028-3932(03)00125-8
components. Cognition, 92(1), 101–144. doi: 10.1016/j.cognition.2002.06.001
Jacquemot, C., & Scott, S. K. (2006). What is the relationship between phonological short-term
memory and speech processing? Trends in Cognitive Sciences, 10(11), 480–486. doi: 10.1016/
j.tics.2006.09.002
José‐Robertson, S., Corina, D. P., Ackerman, D., Guillemin, A., & Braun, A. R. (2004). Neural
systems for sign language production: Mechanisms supporting lexical selection, phono-
logical encoding, and articulation. Human Brain Mapping, 23(3), 156–167. doi: 10.1002/
hbm.20054
Kassubek, J., Hickok, G., & Erhard, P. (2004). Involvement of classical anterior and posterior
language areas in sign language production, as investigated by 4 T functional magnetic reso-
nance imaging. Neuroscience Letters, 364(3), 168–172. doi: 10.1016/j.neulet.2004.04.088
Leischner, A. (1943). Die “Apasie” der Taubstummen. Archiv für Psychiatrie und
Nervenkrankheiten, 115, 469–548.
Li, L., Abutalebi, J., Zou, L., Yan, X., Liu, L., Feng, X., et al. (2015). Bilingualism
alters brain functional connectivity between “control” regions and “language” re-
gions: Evidence from bimodal bilinguals. Neuropsychologia, 71, 236–247. doi: 10.1016/
Li, L., Emmorey, K., Feng, X., Lu, C., & Ding, G. (2016). Functional connectivity reveals which
language the “control regions” control during bilingual production. Frontiers in Human
Neuroscience, 10, 616.
MacSweeney, M., Campbell, R., Woll, B., Brammer, M. J., Giampietro, V., David, A. S., Calvert,
G. A., & McGuire, P. K. (2006). Lexical and sentential processing in British Sign Language.
Human Brain Mapping, 27(1), 63–76. doi: 10.1002/hbm.20167
MacSweeney, M., Woll, B., Campbell, R., Calvert, G. A., McGuire, P. K., David, A. S.,
Simmons, A., & Brammer, M. J. (2002a). Neural correlates of British sign language com-
prehension: Spatial processing demands of topographic language. Journal of Cognitive
MacSweeney, M., Woll, B., Campbell, R., McGuire, P. K., David, A. S., Williams, S., Suckling, J.,
Calvert, G. A., & Brammer, M. J. (2002b). Neural systems underlying British Sign Language
and audio‐visual English processing in native users. Brain, 125(7), 1583–1593. doi: 10.1093/
brain/awf153
Marshall, J., Atkinson, J., Smulovitch, E., Thacker, A., & Woll, B. (2004). Aphasia in a user of
British Sign Language: Dissociation between sign and gesture. Cognitive Neuropsychology,
21(5), 537–554. doi: 10.1080/02643290342000249
Mayberry, R. I. (2010). Early language acquisition and adult language ability: What sign lan-
guage reveals about the critical period for language. In M. Marschark, & P. E. Spencer
(Eds.), The Oxford handbook of deaf studies, language, and education (Vol. 2, pp. 281–291).
Mayberry, R. I., & Eichen, E. B. (1991). The long-lasting advantage of learning sign language in
childhood: Another look at the critical period for language acquisition. Journal of Memory
and Language, 30(4), 486–512. doi: 10.1016/0749-596X(91)90018-F
McGettigan, C., & Scott, S. K. (2012). Cortical asymmetries in speech perception: What’s
wrong, what’s right and what’s left? Trends in Cognitive Sciences, 16(5), 269–276. doi: 10.1016/
j.tics.2012.04.006
McGuire, P. K., Robertson, D., Thacker, A., David, A. S., Kitson, N., Frackowiak, R. S., & Frith,
C. D. (1997). Neural correlates of thinking in sign language. Neuroreport, 8(3), 695–698.
(1978). Broca aphasia: Pathologic and clinical. Neurology, 28(4), 311–311. doi: 10.1212/
WNL.28.4.311
Naeser, M. A., Helm-Estabrooks, N., Haas, G., Auerbach, S., & Srinivasan, M. (1987).
Relationship between lesion extent in “Wernicke’s area” on computed tomographic scan and
predicting recovery of comprehension in Wernicke’s aphasia. Archives of Neurology, 44(1),
73. doi: 10.1001/archneur.1987.00520130057018
Neville, H. J., Bavelier, D., Corina, D., Rauschecker, J. P., Karni, A., Lalwani, A., Braun, A.,
Clark, V., Jezzard, P. & Turner, R. (1998). Cerebral organization for language in deaf and
hearing subjects: Biological constraints and effects of experience. Proceedings of the National
Academy of Sciences, 95(3), 922–929. doi:10.1073/pnas.95.3.922
Newman, A. J., Bavelier, D., Corina, D., Jezzard, P., & Neville, H. J. (2002). A critical period for
right hemisphere recruitment in American Sign Language processing. Nature Neuroscience,
5(1), 76–80. doi: 10.1038/nn775
Nishimura, H., Hashikawa, K., Doi, K., Iwaki, T., Watanabe, Y., Kusuoka, H., Nishimura, T.,
& Kubo, T. (1999). Sign language “heard” in the auditory cortex. Nature, 397(6715), 116–116.
Pa, J., Wilson, S. M., Pickell, H., Bellugi, U., & Hickok, G. (2008). Neural organization of
linguistic short-term memory is sensory modality–dependent: Evidence from signed
and spoken language. Journal of Cognitive Neuroscience, 20(12), 2198–2210. doi: 10.1162/
jocn.2008.20154
Paradis, M. (2004). A neurolinguistic theory of bilingualism (Vol. 18). Amsterdam: John
Benjamins.
Petitto, L. A., Zatorre, R. J., Gauna, K., Nikelski, E. J., Dostie, D., & Evans, A. C. (2000). Speech-
like cerebral activity in profoundly deaf people processing signed languages: Implications
for the neural basis of human language. Proceedings of the National Academy of Sciences,
97(25), 13961–13966. doi: 10.1073/pnas.97.25.13961
Poeppel, D. (2003). The analysis of speech in different temporal integration windows: Cerebral
lateralization as “asymmetric sampling in time.” Speech Communication, 41(1), 245–255.
doi: 10.1016/S0167-6393(02)00107-3
Poizner, H., Bellugi, U. & Klima, E. S. (1987). What the hands reveal about the brain. Cambridge,
MA: MIT Press.
Rizzolatti, G., & Arbib, M. A. (1998). Language within our grasp. Trends in Neurosciences, 21(5),
188–194. doi: 10.1016/S0166-2236(98)01260-0
Rodriguez-Oroz, M. C., Jahanshahi, M., Krack, P., Litvan, I., Macias, R., Bezard, E., &
Obeso, J. A. (2009). Initial clinical manifestations of Parkinson’s disease: Features and
pathophysiological mechanisms. The Lancet Neurology, 8(12), 1128– 1139. doi: 10.1016/
S1474-4422(09)70293-5
Rogalsky, C., Raphel, K., Tomkovicz, V., O’Grady, L., Damasio, H., Bellugi, U., & Hickok,
G. (2013). Neural basis of action understanding: Evidence from sign language aphasia.
Aphasiology, 27(9), 1147–1158. doi: 10.1080/02687038.2013.812779
Rönnberg, J., Rudner, M., & Ingvar, M. (2004). Neural correlates of working memory for sign
language. Cognitive Brain Research, 20(2), 165–182. doi: 10.1016/j.cogbrainres.2004.03.002
Rushworth, M. F., Krams, M., & Passingham, R. E. (2001). The attentional role of the left pa-
rietal cortex: The distinct lateralization and localization of motor attention in the human
brain. Journal of Cognitive Neuroscience, 13(5), 698–7 10. doi: 10.1162/089892901750363244
Sandler, W., Meir, I., Padden, C., & Aronoff, M. (2005). The emergence of grammar: Systematic
structure in a new language. Proceedings of the National Academy of Sciences USA, 102(7),
2661–2665. doi: 10.1073/pnas.0405448102
Schlosser, M. J., Aoyagi, N., Fulbright, R. K., Gore, J. C., & McCarthy, G. (1998). Functional
MRI studies of auditory comprehension. Human Brain Mapping, 6(1), 1–13. doi: 10.1002/
(SICI)1097-0193(1998)6:1<1::AID-HBM1>3.0.CO;2-7
Scott, S. K., Blank, C. C., Rosen, S., & Wise, R. J. (2000). Identification of a pathway for in-
telligible speech in the left temporal lobe. Brain, 123(12), 2400–2406. doi: 10.1093/brain/
123.12.2400 2400-2406
Shalom, D. B., & Poeppel, D. (2007). Functional anatomic models of language: Assembling the
pieces. The Neuroscientist, 14, 119–127. doi: 10.1177/1073858407305726
Shibata, D. K. (2007). Differences in brain structure in deaf persons on MR imaging studied
with voxel-based morphometry. American Journal of Neuroradiology, 28(2), 243–249.
Söderfeldt, B., Ingvar, M., Rönnberg, J., Eriksson, L., Serrander, M., & Stone-Elander, S.
(1997). Signed and spoken language perception studied by positron emission tomography.
Neurology, 49(1), 82–87. doi:10.1212/WNL.49.1.82
Tan, L. H., Chen L., Yip V., Chan A. H., Yang J., Gao J. H., et al. (2011). Activity levels in the
left hemisphere caudate-fusiform circuit predict how well a second language will be
learned. Proceedings of the National Academy of Sciences USA, 108, 2540–2544. doi: 10.1073/
pnas.0909623108
Cerebral Cortex, 11(10), 946–953. doi: 10.1093/cercor/11.10.946
Zou, L., Ding, G., Abutalebi, J., Shu, H., & Peng, D. (2012). Structural plasticity of the left cau-
date in bimodal bilinguals. Cortex, 48, 1197–1206. doi: 10.1016/j.cortex.2011.05.022
Zwitserlood, I. (2012). Classifiers. In R. Pfau, M. Steinbach, & B. Woll (Eds.), Sign language: An
international handbook (pp. 158–186). Berlin: Mouton de Gruyter.
Chapter 17
Understanding H ow W e
Produ ce Writ t e n  Word s
Lessons from the Brain
Brenda Rapp and Jeremy Purcell
The vast majority of our current understanding of how we produce written language has
come from the behavioral study of neurologically intact individuals and individuals with
neural impairments: psycholinguistic and cognitive neuropsychological investigations,
respectively. More recently, however, neural evidence from both neuroimaging and
lesion-based investigations has provided a source of valuable information not only re-
garding the neural bases of the cognitive processes involved in written language pro-
duction, but also regarding the cognitive processes and representations themselves. This
chapter will focus on the latter and will summarize and evaluate the contributions to
our understanding of written word production that come from the study of the brain,
highlighting the specific issues to which neural data have made particularly significant
contributions.
The Cognitive Processes and

Representations of Written Word
Production: An Overview
A basic functional architecture of word spelling is depicted in Figure 17.1. The produc-
tion of written words occurs in a variety of situations, based on different types of inputs
to the writing system. For example, we write the words we hear when we take notes in
a lecture, or write a phone message, or we write to express internally generated word
meanings when composing letters, text messages, essays, and so on. The operations
426 Brenda Rapp and Jeremy Purcell
Spoken word: /breIn/

Visual image:
Phonological Long-Term Memory (LTM): Store of

the phonological forms of familiar words
Lexical Semantics: Representations

of word meanings
Phonology-Orthography Conversion (PGC): Mapping of Orthographic Long-Term Memory (LTM): Store of

sounds to letters the spellings of familiar words
Orthographic Working Memory (WM): Limited capacity system for maintaining letter identity
and order during selection and serial production by peripheral processes.
Allographic/Letter-Shape Selection:
Representation of case-specific letter forms
Letter-name selection Graphic-Motor Planning for Typing: Graphic-Motor Planning for handwriting:
for oral spelling Finger stroke sequences Sequence of wrist and finger movements
/bi/ /ar/ /∂/ /aI/ /εn/
Figure 17.1. The cognitive operations used in written word production. Note that the non-
semantic lexical route is indicated with dashed lines; components that are specifically involved in
spelling are bolded.
involved in processing the different input forms and formats and contacting meaning
representations are not specific to writing; rather, they are engaged whenever heard
words or objects are processed or message representations are internally generated.
Only once the system for word meaning representation (lexical semantics) is en-
gaged can we retrieve stored knowledge of word spellings from orthographic long-term
memory (O-LTM), also referred to as the orthographic lexicon (Buchwald & Rapp,
2009). In addition to the retrieval of spellings from O-LTM, spellings may also be
generated through the application of knowledge regarding the relationships between
sounds and letters via a process that is often referred to as phonology-orthography
conversion (POC). This process generates plausible spellings for known words
(“sauce” → SOSS) or for unfamiliar names or other novel words (“flove” → FLOAVE).
Orthographic representations, whether retrieved from O-LTM or assembled through
POC, are assumed to consist of, at a minimum, abstract letter identities and their order
(D1+O2+G3) that are processed by an orthographic working memory (O-WM) system
Understanding How We Produce Written Words 427
(often referred to as the graphemic buffer). Like other working memory systems (e.g.,
phonological working memory), O-WM is a complex system with multiple functions,
including those required to keep orthographic information active and differentiated
(Costa, Fischer-Baum, Capasso, Miceli, & Rapp, 2011) as well as processes needed
for the serial selection of letters in their correct order for production by subsequent
processes. O-LTM, O-WM, and POC are all involved in producing spellings, regard-
less of production format or motor effector: oral spelling (saying the names of letters),
typing, producing letter shapes with the right or left hand, with a foot in the sand, a can
of spray paint, and so on. For written spelling, letter shapes are selected via processes
sometimes referred to as allographic conversion (specifying case and font) and then
graphic motor plans are assembled and executed to produce specific letter forms
(e.g., dOg). For oral spelling, letter names are selected from phonological long-term
memory (i.e., the phonological lexicon) and the necessary phonological, phonetic, and
articulatory processes are deployed to say the letter names (e.g., /di//oʊ/  /ʤi/).
Evidence for these component processes and for the connectivity structure among
them (Figure 17.1) has come primarily from cognitive neuropsychological investigations
of individuals with acquired dysgraphia (for recent reviews, see Miceli & Costa, 2014;
Rapp & Fischer-Baum, 2015), although psycholinguistic studies with neurotypical
individuals have increasingly constituted an important source of evidence (Bonin,
Peereman, & Fayol 2001; Breining, Nozari, & Rapp, 2016; Damian & Bowers, 2003).
Despite considerable consensus regarding the architecture depicted in Figure 17.1, nu-
merous questions remain regarding even fundamental aspects. This chapter reviews
some of these fundamental issues, highlighting the contribution of neural evidence to
ongoing debates.
The Relationship Between

Orthographic Long-Term
and Working Memory
In the broader context of the literature on working memory, one persistent question has
concerned the relationship between working memory (WM) and long-term memory
(LTM) and specifically the extent to which these are distinct, independent systems (see
Buchsbaum, Chapter 32 in this volume; see Miyake & Shah, 1999, for a review). There
are theories that assume structural distinctions between the two, and others that assume
that WM refers to the portion of LTM that is activated during task performance. For ex-
ample, Cowan (1999, p. 62) proposed that “WM information comes from hierarchically
arranged faculties comprising: (a) LTM; (b) the subset of LTM that is currently activated;
and (c) the subset of activated memory that is the focus of attention and awareness”
(see also Ruchkin, Grafman, Cameron, & Berndt, 2003). The “embedded processes”
account stems from numerous findings indicating that different aspects of LTM know-
ledge and skills have effects on WM tasks and capacity. These findings have, in fact, led
most researchers to acknowledge that the two systems cannot be entirely independent.
On the other hand, various empirical findings, including neuropsychological ones, have
supported independent systems accounts. This position is well-represented by Baddeley
and Logie’s (1999, p. 31) statement that “although some have viewed working memory
as an activated portion of LTM, such a view is probably an unhelpful oversimplifica-
tion.” It would seem that a plausible hybrid account is that the two are structurally inde-
pendent systems that, nonetheless, interact during processing. For a hybrid account, key
questions concern the extent, nature, and conditions of the interaction.
While work on LTM and WM distinctions has most commonly occurred in the con-
text of tasks that involve the presentation of sensory information (e.g., lists of visually or
auditorily presented letters, words, shapes, spatial locations) for short-term retention
and processing, there is also work that specifically involves the retrieval of information
from LTM for processing by WM (Roth & Courtney, 2007). In fact, as described earlier,
this is precisely the context in which the LTM and WM distinction applies in written
word production. As reviewed previously, theories of written word production posit a
distinction between an orthographic long-term memory store of known word spellings
(O-LTM) and a limited-capacity orthographic working memory system (O-WM) that
maintains active information about letter identities and their order and allows for the se-
rial selection of the letter identities in their proper order for production by downstream
processes. This distinction was motivated by the computational demands of word
spelling (Caramazza, Miceli, Villa, & Romani, 1987) and supported by cognitive neu-
ropsychological findings. In terms of computational demands, Caramazza et al. (1987)
argued that O-WM (graphemic buffering) is required to interface between O-LTM
processes that retrieve whole-word spelling representations (consisting of multiple let-
ters) and subsequent processes involved in producing the individual letters sequentially.
Supporting this distinction, cognitive neuropsychological findings have revealed
two highly distinct patterns of acquired dysgraphia (for a review, see Buchwald & Rapp,
2009). The pattern attributed to O-LTM impairment is characterized by higher error
rates for low-compared to high-frequency words. This is attributed to differences in
the robustness of LTM representations due to frequency of use, with the effect that
more robust representations are less susceptible to disruption under conditions of brain
damage. Furthermore, in these deficits accuracy is not affected by the length of words,
with errors occurring equally often for longer words and shorter words. In these cases,
errors are typically (although not necessarily) “word-level” errors and may include
other words, semantic errors, or phonologically plausible spellings. In contrast to this
pattern, the characteristics of O-WM impairments are thought to be the consequence
of disruption and further limitations on the capacity-limited O-WM system. In fact,
a deficit in O-WM is characterized by a largely complementary pattern in which there
are no (or only mild) effects of word frequency, and there are effects of word length such
that there is a greater probability of error on any given letter, depending on the total
number of letters in a word. Errors in cases of O-WM deficits are typically “letter-level”
errors consisting of letter deletions, substitutions, transpositions, or additions. The pat-

tern observed for O-WM deficits corresponds to what is generally expected in situations
where WM processes are damaged or taxed.
While the reported patterns of impairment appear quite distinct, Sage and Ellis
(2004) questioned the true distinctiveness of the patterns and, therefore, the inde-
pendence of the O-LTM and O-WM (see also Glasspool, Shallice, & Cipolotti, 2006;
Ward & Romani, 1998). Furthermore, a computational model of spelling—Competitive
Queueing (Houghton, Glasspool, & Shallice, 1994)—does not clearly differentiate these
components. Buchwald and Rapp (2009) responded to these concerns by laying out
a number of arguments in favor of the robustness of the differences between the two
patterns while, at the same time, allowing for some interaction between the O-LTM
and O-WM components. Recently, Rapp, Purcell, Hillis, Capasso, and Miceli (2016) re-
ported neural findings highly relevant to understanding the nature and extent of the dis-
tinction between O-LTM and O-WM. This study was based on the analysis of lesion loci
of 33 individuals with acquired dysgraphia: 17 with O-LTM impairment, 10 with O-WM
impairment, and 6 with both (O-Mixed). Individuals were identified for the study based
only on their behavioral profiles, with no consideration of lesion loci. A voxel-based
lesion mapping analysis was then carried out in order to determine if there were voxels
specifically associated with O-LTM or O-WM impairments. The results of the anal-
ysis are depicted in Figure 17.2, revealing (1) a distinct neurotopographic distribution
of lesions producing O-LTM and O-WM deficits; (2) that lesions selectively associated
O-Long Term Memory > O-Working Memory

Z-Value: +2.4 +2.1 +1.7 –1.7 –2.1 –3.1 –4.1
Left
Figure 17.2. Voxel-based lesion mapping comparison of the lesions of individuals with ortho-
graphic long-term memory (O-LTM) and working memory (O-WM) deficits, Depicted are the
results of testing (at each voxel) for differences in presence/absence of lesion for individuals with
deficits affecting O-LTM or O-WM. Clusters of significant difference are rendered on a left hem-
isphere standard brain template. All clusters are FDR (false discovery rate) corrected for multiple
comparisons at a p <0.05. The color scale reflects the range z-values for the significant clusters.
Positive z-values (orange/red) indicate the O-LTM deficit clusters; negative z-values (blue/
purple/pink) indicate O-WM deficit clusters.
Source: Rapp et al. (2016).
with O-WM impairments occurred in the left superior parietal lobe (centered on the
intra-parietal sulcus, –39, –50, 36); and (3) a fractionation of O-LTM into a left ventral
temporal and a posterior inferior frontal gyrus (IFG) site, with lesion densities centered
on –49, –39, –19 and –52, –19, 18, respectively. These findings advance our understanding
of the spelling process in three ways.
First, the highly distinct neurotopographic distribution of the lesions provides strong
support for the notion of structurally distinct O-LTM and O-WM processes. In this
regard, it is also worth noting that none of the lesions in the seven O-LTM cases with
ventral temporal damage extended outside the temporal lobe. Likewise, the lesions for
the 10 O-WM cases did not extend into ventral temporal cortex. In other words, these
subsets of O-LTM and O-WM cases had non-overlapping lesions. While these findings
do not, of course, reveal the nature and extent of the processing interaction between
the components, they do reveal that distinct and quite distant brain regions subserve
these computationally “close” processes. The lesion locations identified in Rapp et al.
(2016) are consistent with previous lesion studies linking O-LTM deficits with lesions
to the left ventral temporal region (Rapcsak & Beeson, 2004) as well as the left posterior
IFG (Hillis et al., 2002). Importantly, the locations identified with O-LTM and O-WM
lesions are also highly convergent with previous fMRI findings with neurotypical adults.
In an fMRI study during which participants carried out written spelling while being
scanned, Rapp and Dufor (2011) identified specific regions that were associated with
sensitivity to word length but not frequency (O-WM), as well as others associated with
the complementary pattern of sensitivity to frequency but not length (O-LTM). The
key activation sites identified by Rapp and Dufor (2011; see also Rapp & Lipka, 2011) in-
cluded frontal and ventral temporal sites associated with O-LTM, as well as a parietal
site associated with O-WM. In fact, the areas identified by Rapp and Dufor (2011) were
in close proximity to the centers of high-density lesion overlap depicted in Figure 17.2.
Second, the parietal area associated with O-WM lesions has often been associated
with spatial and/or visual WM/attentional processes (Jonides et al., 1998; Wager &
Smith, 2003; Todd & Marois, 2004), although it is worth noting that these processes
have been more consistently associated with the homologous area in the right hemi-
sphere. Also associated with this parietal region are mechanisms supporting atten-
tion switching in visual (Serences & Yantis, 2006) and auditory (Shomstein & Yantis,
2006) displays, as well as more abstractly in the context of moving attention from one
task to another (Shomstein & Yantis, 2004). In fact, the superior parietal locus of O-
WM deficits raises questions regarding the domain specificity versus generality of the
working memory processes subserved by this region. One possibility is that there are
domain-specific WM processes that occupy adjacent parietal subregions with ortho-
graphic WM corresponding to a distinctly orthographic WM process; another is that
this region supports domain-general WM processes. Favoring the orthography-specific
hypothesis is the additional finding by Rapp et al. (2016) that the individuals with O-
WM deficits performed within the normal range on visuospatial working memory, with
spatial spans on the Corsi blocks task (Kessels, Van Zandvoort, Postma, Kappelle, & De
Haan, 2000) averaging 4.75 (range = 4–5). This is a strong indication that visuospatial
WM was not generally disrupted by the (left hemisphere) lesions in these cases. Also
favoring the possibility of orthography-specific WM processes in the left superior pa-
rietal lobule are functional neuroimaging findings of letter position WM processing
for visually presented letters in this area (Carreiras, Armstrong, Perea, & Frost, 2014;
Cohen, Dehaene, Vinckier, Jobert, & Montavont, 2008).
A third key finding form Rapp et al. (2016) was the fractionation of O-LTM into
frontal and ventral temporal sites. This finding was somewhat unexpected given that the
participants presented with the same overall behavioral profile of O-LTM deficits. One
possibility is that the IFG and ventral temporal areas together instantiate a single, dis-
tributed O-LTM function. Another possibility is that the theory depicted in Figure 17.1 is
underspecified with regard to its characterization of O-LTM processes and that, in fact,
there are two distinct functions that both contribute to O-LTM, each supported by these
different brain areas. Along these lines, Rapp et al. (2016) speculated that the frontal site
may be involved in cognitive control for lexical orthographic selection, while the ven-
tral temporal region may support the representation and storage of orthographic word
forms. Consistent with this role for the ventral temporal site is the fact that this area
(mid-fusiform gyrus, specifically) has been associated with orthographic lexical repre-
sentation in reading (Cohen et al., 2002; Glezer, Jiang, & Riesenhuber, 2009), a topic we
discuss in a later section.
In sum, the neural data—lesion and neuroimaging results—reveal substantially in-
dependent O-WM and LTM mechanisms. However, the findings do not provide a spe-
cific characterization of the nature of the interaction between the systems and, in this
regard, key questions remain. A fundamental question is the following: Does O-WM
constitute a short-term storage structure/work-space with “copies” of orthographic
representations, or does it, instead, consist of specialized attentional processes that
provide “top-down” amplification of orthographic information in O-LTM (Rapcsak &
Beeson, 2015) and the serially directed attention required for letter selection for produc-
tion? One can expect that, in the future, neural data may contribute to furthering our
understanding of issues such as these.
The Relationship between Lexical

and Sublexical Processes
The distinction between lexical and sublexical processes has been vociferously debated
with regard to reading, although there has been little discussion of the parallel issues in
the context of spelling. Nonetheless, the same issues do also apply to spelling, and so
we provide a brief review of the debate in reading as a backdrop for the discussion of
spelling.
Coltheart and colleagues (1993) proposed multiple routes to reading, including both
semantically and non-semantically mediated lexical routes for reading words, and a
non-lexical route for pseudoword reading (Figure 17.1 depicts an “analogous” architec-
ture for spelling). Seidenberg and McClelland (1989) proposed an alternative theoret-
ical position, arguing that the distinction between lexical and sublexical processing is
unnecessary and that, instead, all oral reading can take place within a single network
that maps orthography to phonology. In effect, this proposal reduces the two non-
semantic routes proposed by Coltheart and colleagues to a single, non-semantic route.
Importantly, in their view, this network does not include explicit lexical orthographic
representations. Instead, knowledge of the relationships between letters and sounds
at all levels of granularity—from letters to words—is encoded in the connections be-
tween letter and sound representations. In sum, the debate concerns if, in addition to
a semantically mediated lexical route for word reading, there is a single non-semantic,
non-lexical process that maps letters to sounds for all written strings, including both
exception words and pseudowords, or if, alternatively, distinct lexical and non-lexical
routes are responsible for non-semantic reading of exception words and pseudowords,
respectively (see Rapp, Folk, & Tainturier, 2001, for a review).
Key evidence in support of the lexical/sublexical distinction has been the behavioral
double dissociation consisting of individuals with intact pseudoword reading in the face
of poor reading of exception words, and vice versa (Coltheart, Curtis, Atkins, & Haller,
1993). However, both patterns of dissociation have been questioned by opponents of this
distinction. On the one hand, Plaut, McClelland, Seidenberg, and Patterson (1996) have
argued that the pattern of intact pseudoword reading in the context of severely impaired
exception-word reading is extremely rare and dependent on premorbid individual
differences across individuals. Given the rather vague nature of the relevant individual
differences, this argument has been difficult to evaluate. A different type of argument has
been made to challenge the complementary pattern of impaired pseudoword reading in
the context of intact word reading. Specifically, the argument has been that these cases
do not reflect damage to reading processes at all but, instead, arise from disruption to
phonological processes (Harm & Seidenberg, 1999; Patterson & Lambon-Ralph, 1999).
Proponents of this claim have pointed to the strong association in the literature be-
tween phonological deficits and pseudoword reading difficulties. On this basis, these
researchers have questioned the existence of reading-specific sublexical processes. In
response, proponents of sub-lexical reading processes have pointed to the several well-
documented reports of individuals with significant pseudoword reading difficulties plus
good reading in the face of intact phonological processes (for a review, see Caccappolo-
van Vliet, Miozzo, & Stern, 2004). These cases constitute a clear and strong challenge for
the view that pseudoword reading deficits are reducible to phonological impairments
and, in so doing, provide support for the view of independent lexical and sublexical
reading processes.
As indicated earlier, the same issues and questions apply to spelling: Is the spelling
of words and pseudowords handled by the same or different processes? Are there
orthography-specific processes used for pseudoword spelling? Given the goals of this
chapter, we will specifically focus on the neural evidence—neuroimaging and lesion
analyses—that are relevant to these questions.
The appeal of a single-process proposal for spelling (analogous to Seidenberg and

McClelland’s [1989] proposal for reading) lies in the fact that our knowledge of the rela-
tionship between sounds and letters is learned largely from our experience with spelling
words. Given that we must account for our word-spelling abilities, placing know-
ledge of word and pseudoword spelling within the same network eliminates the need
to posit a mechanism by which a separate process or network is learned for sublexical
spelling. Despite this positive aspect of the proposal, an important challenge has been
to account for the many reports of behavioral dissociations between word and nonword
spelling performance (e.g., for a review, see Tainturier & Rapp, 2001). The behavioral
dissociations are matched by neural ones as well, which we will focus on here.
As indicated in the previous section, the neural substrates associated with O-LTM
lesions correspond to the left inferior ventral region and areas of the posterior left frontal
cortex (Rapcsak & Beeson, 2004; Rapp et al., 2016). With regard to pseudoword spelling
deficits, Henry, Beeson, Stark, and Rapcsak (2007) reported that a group of individuals
with perisylvian lesions exhibited greater difficulty in spelling pseudowords than words.
For this group, the area of greatest lesion overlap was within the IFG/frontal operculum,
precentral gyrus and insula, and, to a lesser degree, the superior temporal gyrus (STG)
and the supramarginal gyrus. Other reports of selective pseudoword spelling deficits
have also implicated posterior perisylvian regions, including the left supramarginal
gyrus (Roeltgen, Sevush, & Heilman, 1983; also Alexander, Fischer, & Friedman, 1992;
Bub & Kertesz, 1982; Philipose et al., 2007) and the angular gyrus (Hillis et al., 2002;
Sheldon, Malcolm, & Barton, 2008). Generally, consistent with these findings, Rapp
et al. (2016) found that the individuals with the ventral temporal lesions and O-LTM
deficits all had good pseudoword spelling (85%–98% accuracy). This indicates that key
pseudoword spelling processes are unlikely to be situated within the ventral-temporal
lesion region and points, instead, to some role for frontal and/or perisylvian areas in
pseudoword spelling. Thus, in combination, the lesion findings point to clear differences
in the neuroanatomical regions supporting word and pseudoword spelling.
With regard to neuroimaging data, as indicated earlier, the findings with regard to
O-LTM are convergent with the lesion-based findings, providing evidence for the role of
ventral temporal cortex and posterior IFG. Rapp and Lipka (2011) found these regions
to be sensitive to word frequency during spelling, and Rapp and Dufor (2011) confirmed
those results while also providing evidence that these areas are not sensitive to word
length in spelling. However, with regard to POC processes, there has been scant func-
tional neuroimaging research. Ludersdorfer, Kronbichler, and Wimmer (2015) reported
on the most detailed examination of word and pseudoword spelling to date. As depicted
in Figure 17.3, within the left IFG and ventral temporal cortex, Ludersdorfer and
colleagues found a neurotopographic differentiation between word and pseudoword
spelling such that word spelling produced greater activity than pseudoword spelling.
The word spelling recruitment of ventral temporal cortex was associated with O-
LTM processing, and the left IFG recruitment was attributed to resolving competition
among multiple orthographic representations that become active when spelling a word.
Importantly, the only area in which pseudoword spelling resulted in more activity than
Word Spelling> Pseudoword Spelling
Left
Positive Negative
Figure 17.3. Areas selectively associated with word and pseudoword spelling. fMRI spelling
experiment comparing neural response for word to pseudoword spelling. The red clusters in
the left IFG and ventral occipitotemporal cortex (vOTC) depict areas with greater activity for
word spelling than for pseudoword spelling. The blue cluster in the left STG depicts the area with
greater activity for pseudoword than word spelling. Activation is projected onto a standard tem-
plate brain in Montreal Neurological Institute (MNI) space.
Source: Adapted with permission from Figure 3 in Ludersdorfer et al. (2015).
word spelling was a region of posterior STG. This latter finding suggests that the STG re-
gion may be especially important for pseudoword but not necessarily for word spelling.
Overall, the functional neuroimaging and lesion mapping results point to a distinc-
tion between the roles of ventral temporal cortex for lexical processing and posterior
perisylvian cortex for pseudoword processing. However, this does not identify which
of the processes involved in pseudoword spelling are specifically supported by this re-
gion. At a minimum, these include the phonological processing of the input (e.g., seg-
mentation), phonological working memory maintenance of the phonological input, or
the phoneme-orthography correspondences themselves. With respect to POC, is this
knowledge represented in a single region, or would it be encoded in the connection
pattern between regions that otherwise represent phonemes and letters? In this con-
text, the comparable recruitment of posterior ventral temporal cortex by both word
and pseudoword spelling reported by Ludersdorfer et al. (2015) raises the possibility
that single letter and multi-letter groups that are often assumed to be represented in this
region are shared by both word and pseudoword spelling processes. However, on that
basis, one would predict pseudoword spelling deficits with lesions in this region, but
that has not been reported.
An intriguing aspect of the neuroimaging findings is the interpretation of the superior
temporal/perisylvian findings provided by both Ludersdorfer et al. (2015) and Rapcsak
et al. (2009). While both groups of researchers found an association of pseudoword
spelling with either neural activity or lesions in this region, they both interpreted their
findings as indicating that this region is specifically responsible for the phonological
processing aspects of pseudoword spelling. For example, Rapcsak et al. (2009) re-
ported that the degree of phonological impairment in their participants was predictive
of spelling (and reading) accuracy and specifically concluded that pseudoword spelling
difficulties (at least in cases of perisylvian lesions) are the result of a central phono-
logical deficit, rather than damage to spelling knowledge per se. Along similar lines,
Ludersdorfer et al. (2015) attributed the greater recruitment of STG by pseudoword
compared to word spelling to the greater phonological demands of pseudoword
spelling. Interestingly, these arguments parallel those we referred to in the context of
reading (Patterson & Lambon-Ralph, 1999), according to which pseudoword deficits
are reduced to phonological ones.
Although this is an area in which strong conclusions are premature, there are two
things worth noting. First, while the interpretation offered by Ludersdorfer et al. (2015)
and Rapcsak et al. (2009) is plausible, there is nothing in either the neuroimaging or le-
sion results that requires the interpretation that only phonological processing, rather
than POC spelling processes, is associated with the STG/perisylvian sites. Second,
these interpretations fail to provide an account of the neural (and cognitive) bases of
pseudoword spelling. We certainly must have knowledge of the relationships between
sounds and letters, which cannot be reduced to phonological processing since phono-
logical processes alone cannot generate spelling responses. However, the “reductionist”
phonological account of pseudoword spelling (or reading) and phonological dysgraphia
leaves this fundamental question unanswered.
In sum, the differences in the neurotopography of brain regions active for word and
pseudoword spelling support a distinction between lexical and non-lexical spelling
processes that is independently motivated by the behavioral data from neuropsycholog-
ical cases. Despite this, we still lack a detailed understanding of the distinctions between
these processes and, in particular, we have little data regarding the neural instantiation
of pseudoword spelling processes.
The Relationships between Reading

and Writing
A key question in written language research is whether or not reading and spelling
share representations and processes. These questions have been investigated in behav-
ioral studies with neurologically intact adults and children, and with individuals with
acquired deficits. The basic approach in much of this research has involved examining
whether specific aspects of performance are highly similar or different in reading and
spelling. Most of the research to date has focused on the possibility of shared O-LTM
representations, with relatively little attention directed at the possibility of shared O-
WM processes (but see Tainturier & Rapp, 2003) or at the relationship between letter
recognition and production (but see Longcamp, Anton, Roth, & Velay, 2003; Rapp
& Caramazza, 1997). Consequently, here we will primarily focus on the question of
whether or not spelling and reading share O-LTM, and then touch briefly on this issue
as it concerns O-WM.
In behavioral studies with neurologically intact participants, Holmes and Carruthers
(1998) and also Burt and Tate (2002) have shown that individuals are slower and/
or less accurate in reading tasks (such as lexical decision or visual spelling accuracy
judgments) for the specific words that they cannot spell correctly, providing support
for the notion that O-LTM systems are shared for reading and spelling. Consistent
with this, Monsell (1987) found significant repetition priming from the task of spelling
(without visual feedback) to a subsequent reading task. While the behavioral evidence
from neurotypical individuals would seem to generally favor shared O-LTM, arguments
can be raised against this conclusion. For example, with regard to the more fine-grained
patterns of behavioral associations/dissociations, the fact that the findings are largely
correlational allows for the possibility that some additional factor is the source of the re-
lationship across the modalities or, alternatively, that episodic memory traces from one
task influence performance on the other.
In terms of acquired deficits, the straightforward prediction is that associations of
reading and spelling deficits affecting lexical orthographic processing favor the view of
shared O-LTM component(s), while dissociations of O-LTM deficits across reading and
spelling would present a challenge to that view. While there have been a number of reports
describing associations and dissociations between reading and spelling, for our purposes,
the most relevant are those studies that determined whether or not O-LTM was specifi-
cally affected. As discussed earlier, Rapp et al. (2016) reported a number of cases with O-
LTM deficits in spelling with lesions in ventral temporal fusiform cortex or posterior IFG.
Interestingly, while many of those cases also had reading deficits, many did not; however,
the reading deficits were not specifically evaluated in terms of whether or not they specifi-
cally affected O-LTM. However, Rapcsak and Beeson (2004) reported on eight individuals
who suffered damage to left hemisphere Brodmann areas 37 and 20 and exhibited lexical
impairments in both spelling and reading (though more pronounced in spelling) (see
Philipose et al., 2007, for similar evidence from acute stroke). One of the challenges in
interpreting the findings of associations and dissociations in acquired deficits is that asso-
ciated acquired deficits could either signal a shared process or, alternatively, be explained
as resulting from coincidental damage to independent O-LTM components for reading
and spelling that are instantiated in adjacent neural substrates. Similarly, dissociations
could indicate distinct O-LTM systems for word reading and spelling or result from lesions
affecting modality-specific access to a single, shared O-LTM system (Allport & Funnell,
1981). In fact, Purcell, Shea, and Rapp (2014) used what they referred to as “cognitive disso-
ciation lesion mapping” to examine the lesion distributions of three individuals: two with
lexical deficits in both reading and spelling, and one with a lexical deficit in spelling but
not reading. They suggested that there are separate substrates that support the interface
between O-LTM and semantics for reading and spelling, respectively.
While one might have thought that it would be a relatively straightforward matter
to determine if spelling and reading share processing mechanisms and representations,
the number of alternative interpretations for existing deficit/lesion findings leaves open
the possibility for functional neuroimaging data to make a significant contribution to
the debate. There have been a large number of studies evaluating the neural substrates of
reading, and a much smaller number that have considered spelling substrates (for meta-
analyses of functional neuroimaging studies of reading, see Jobard, Crivello, & Tzourio-
Mazoyer, 2003; Martin, Schurz, Kronbichler, & Richlan, 2015; and Turkeltaub, Eden,
Jones, & Zeffiro, 2002; for spelling meta-analyses, see Planton, Jucla, Roux, & Démonet,
2013; Purcell, Napoliello, & Eden, 2011). However, there have been only a handful of
studies that have examined both reading and spelling in the same individuals, some-
thing that is critical for addressing this issue via functional neuroimaging (Purcell et al.,
2011; Purcell, Jiang, & Eden 2017; Rapp & Dufor, 2011; Rapp & Lipka, 2011) and these
studies reported co-activation for reading and spelling in the left mid-fusiform gyrus
and in left IFG/junction. Furthermore, both Rapp and Lipka (2011) and Rapp and Dufor
(2011) found sensitivity to lexical frequency in these two regions. This latter finding
strengthens the argument that the shared substrates between reading and spelling were
specifically involved in lexical (orthographic) processing.
However, all of these studies except Purcell et al. (2017) relied on identifying
overlapping areas of activation for reading and spelling. This leaves open the possi-
bility that different O-LTM processes and/or representations for reading and spelling
could be supported by different subpopulations of neurons within these overlapping re-
gions. The neural adaptation approach taken by Purcell et al. (2017) is particularly well-
suited for evaluating this possibility. Neural adaptation allows one to examine whether
different cognitive functions (or representational types) share neural substrates. It is
based on the finding that when a neural population is repeatedly engaged, its response
diminishes. To determine whether the same neural population processes different
tasks or stimulus types, one can compare the response of the area to the repetition of
the same tasks/stimuli versus its response to the consecutive processing of the rele-
vant different tasks/stimuli. If the same neural substrates support the different tasks/
stimuli, then similar decreases in neural responses are expected in both situations. The
finding of neural adaptation across different tasks/stimuli supports the conclusion that
the brain area supports a cognitive function or representational type that is shared by
the tasks/stimuli. Although fMRI-adaptation is a commonly used method to index
the selectivity of a brain region to a task/stimulus representation (Barron, Garvert, &
Behrens, 2016), these experiments require careful design, as the adaptation effect can
be influenced by neural adaptation due to representational selectivity as well as by at-
tentional expectation (e.g., heightened attention when a stimulus/task is different as
compared to same) (Larsson & Smith, 2012; Summerfield, Trittschuh, Monti, Mesulam,
& Egner, 2008). Therefore, fMRI-adaptation designs should compare conditions for
which task/stimulus attentional expectations are equated. Purcell et al. (2017) used an
fMRI-adaptation method of this sort to test the hypothesis that reading and spelling
share orthographic representations. On different trials, participants were asked to read,
spell, and repeat words. There were two critical conditions defined by the pairings of
consecutive trial types: (1) participants spelled and then read the same or different words
(i.e., spell-READ); (2) participants verbally repeated and then read the same or different
words (i.e., repeat-READ). If, within a neural region, the same functions/representations
support both reading and spelling, then we expect a benefit for consecutive processing
of the same word (compared to different words) for the spell-READ condition. On the
other hand, if the neural region does not involve functions/representations shared by
both reading and spelling, then no more neural adaptation would be expected from
this region for same compared to different words across the two tasks. The repeat-Read
condition serves to establish whether any observed neural adaptation is specific to or-
thographic processing or is due, instead, to shared non-orthographic processes (e.g.,
phonologic or semantic). Figure 17.4 A depicts the lateral ventral mid-fusiform region,
referred to as the VWFA (visual word form area; Cohen et al., 2002), that was function-
ally defined in each participant in the study; these individually defined regions served as
the analysis region. Figures 17.4 B and C show the neural responses associated with each
of the experimental conditions. These data clearly indicate a neural adaptation response
when the same (vs. different) word is spelled and then read (spell-READ) but, critically,
not when it is repeated and then read (repeat-READ); these conditions are equated on
attentional expectation (i.e., the relative expectation of performing a same or different
word READ task is equated across both the critical spell-READ and the repeat-READ
condition pairs). These findings provide strong evidence that spelling and reading share
orthographic-specific representations/processes within this brain region. The results are
stronger than the previously described overlap results, as they constitute a stronger test
that the same neural populations are involved in both reading and spelling.
Finally, we briefly consider the relationship between spelling and reading with re-
gard to O-WM. In terms of the role of O-WM in spelling and reading, the few behav-
ioral studies of acquired impairments that have examined the issue concluded that
there is a shared O-WM (Caramazza, Capasso, & Miceli, 1996; Hillis & Caramazza,
1995; Tainturier & Rapp, 2003). In these papers, it was argued that in the case of spelling,
words and pseudowords place roughly comparable demands on the O-WM system.
In contrast, in reading, words place minimal demands on the system due to the fact
that they are largely read in parallel, while pseudowords are considerably more taxing
of O-WM (see Ans, Carbonnel, & Valdois, 1998 for similar arguments with regard to
reading). Although the neural data are scant, they provide some contributions to this
question. As reported earlier and depicted in Figure 17.2, Rapp et al. (2016) found that
O-WM deficits in spelling can arise from lesions to the left superior parietal lobule and
that this region has been associated with sensitivity to word length in neuroimaging
studies of spelling (Rapp & Dufor, 2011). Interestingly, this general neural area has also
been associated with reading under attentionally demanding conditions (Carreiras,
Quiñones, Hernández-Cabrera, & Duñabeitia, 2015; Cohen, Dehaene, Vinckier, Jobert,
& Montavont, 2008), leading to the suggestion that the O-WM area may provide “top-
down,” serially directed attention for letter selection in O-LTM regions during reading.
While in reading this may be required only under demanding viewing conditions or
specific tasks, this type of O-WM memory involvement may be routinely required for
spelling.
(a)
Left
VWFA MNI Coordinates
–41 –55 –16
(b) across Task condition BOLD plots
spell-READ repeat-READ
0.5
0.4
% Signal Change
0.3
0.2
0.1
0.0
–0.1
1 5 9 13 1 5 9 13
Seconds
Different Same
(c) Peak across Task Condition Mean Peak

BOLD Response
*
0.5 ***
0.4
% Signal Change
0.3
0.2
0.1
–0.1 spell-READ repeat-READ
Different Same
Figure 17.4. Neural adaptation in the left visual word form area (VWFA) region of interest,
demonstrating orthographic representations/processes shared across spelling and reading. (A) The
mean location of the participant specific VWFA regions projected onto a transparent left hemi-
sphere standard brain. Red dot refers to the mean location across subjects (–41, –55, –16); the blue
dots refer to the individual subject peak locations. (B) Blood oxygenation level dependent (BOLD)
responses for the (spell-READ) and the repeat-READ consecutive task pairs of either different
words (solid lines) or the same words (dotted lines). Error bars are standard error. (C) Average peak
BOLD response (4–8 sec post-stimulus) differences for each condition. Positive values refer to an
adaptation effect (i.e., different > same). Error bars are standard error. These results indicate a sig-
nificant effect for the spell-READ conditions, but critically not for the repeat-READ condition,
indicating shared orthographic representations across spelling and reading within the left VWFA.
P-values: *** p <0.0001; * p <0.01.
Source: Adapted with permission from Figure 4 in Purcell, Jiang, & Eden (2017).
In summary, the current neural evidence supports the hypothesis of shared lexical
and working-memory processes in reading and spelling. However, much work remains
to be done to determine the specific nature of the shared functions and representations;
the possibility of dissociations in the access to and from these shared structures has been
scarcely addressed and may require different methodological approaches.
Abstract Letter Representations

in Writing
This section concerns the role of abstract letter identities (ALIs) in spelling. It is com-
monly assumed that the letter representations in O-LTM and O-WM are abstract or
amodal representations that do not correspond to letter names or shapes. In the context
of the architecture depicted in Figure 17.1, the ALIs retrieved from O-LTM or computed
from POC are held in O-WM while serially converted to specific letters shapes or
names, depending on whether the spelling knowledge is to be expressed in written or
oral spelling. An alternative view would be that spelling involves a “re-instatement”
of previous motor or perceptual experiences during writing, and therefore that letter
representations used for spelling are necessarily motor (or visual) representations. Such
a view would be consistent with the general embodied cognition framework (Barsalou,
2008; Pulvermüller, 2005).
The primary argument for ALIs in spelling is based on a combination of findings
from cases of acquired dysgraphia. First, individuals with deficits affecting the central
processes of spelling (O-LTM, POC, or O-WM) typically exhibit highly similar patterns
of accuracy and distribution of error types across output modalities—written and oral
spelling or typing (Caramazza & Miceli, 1990; see Tainturier & Rapp, 2001, for a review).
This indicates that a common representational type forms the basis of written and oral
spelling. However, it does not necessarily imply ALIs, as the common format could
be motor plans, visual shapes, or letter names. However, if spellings were represented
in a specific modality, then we would expect that disruption to representations in the
critical modality would create a deficit in other modalities. However, this has not been
the case. There have been reports of individuals with impaired oral spelling and intact
written spelling (e.g., Bub & Kertesz, 1982; Kinsbourne & Warrington, 1965) where the
oral spelling impairment did not involve apraxia or peripheral impairments. In a com-
plementary manner, there have been a number of cases of individuals with intact oral
spelling but impaired written spelling with no generalized motor deficits (among them,
Rothi & Heilman, 1981; Zangwill, 1954). This double dissociation of written and oral
spelling deficits argues against the position that either letter names or letter shapes cor-
respond to the format of letter representation at the level of the central components of
spelling and therefore is consistent with the ALI proposal. More direct evidence in favor
of ALIs comes from Rapp and Caramazza (1997) who examined the letter substitution
errors of individuals with deficits arising from O-WM deficits to determine if targets
and error letters were similar either visually or motorically. They found that there was
no more visual or motor similarity between target letters and errors than would be ex-
pected by chance, precisely what would be predicted under the view that orthographic
knowledge is represented in the abstract, amodal format of ALIs, and contrary to the
assumptions of an embodied account.
Neural data have also provided evidence relevant to this debate. Dufor and Rapp
(2013) used the fMRI adaptation paradigm described above to examine the various
representations of letters in writing, including ALIs. The logic was as follows: As depicted in
Figure 17.5A, on each trial, participants were prompted to write three repetitions of a partic-
ular letter. This was expected to produce increasingly smaller BOLD responses (adaptation/
habituation effects) in brain areas involved in letter production (see Figure 17.5B). On the
fourth trial, either the same letter (no-change trial) or a different letter (change trial) was
produced. Change trials could involve a change in case only (sssS), a change in case and
shape (qqqQ), a change in identity and shape (tttp), or a change in identity, case, and shape
(tttP). Comparisons of the magnitude of the BOLD response across the different types of
change trials should reflect the extent of neural habituation or release from inhibition that
has occurred, revealing the type(s) of representations processed in a given brain region.
Areas that encode letter shape should respond to changes in shape, and areas that encode
letter case should respond to changes in case regardless of changes in shape. Moreover, spe-
cifically as concerns ALIs, neural areas representing ALIs should show a release from adap-
tation when letter identity changes (tttP or tttp), but critically not when the letter identity
remains the same, despite changes in case and size (e.g., vvvV) or case and shape (qqqQ).
Dufor and Rapp (2013) found that within letter-sensitive cortex, there were multiple
regions that responded to changes in letter shape, and some that responded to both shape
and identity, but none that responded only to changes in case. In addition, and criti-
cally relevant to the ALI hypothesis, there was one region—the left fusiform gyrus (cen-
tered on –41, –60, –13)—that showed the pattern expected of a region encoding ALIs: it
responded to changes in letter identity more so than to changes in size, case, or shape only
(e.g., tttP/tttp trials compared to vvvV/qqqQ trials); see Figure 17.5 C.1 The implications
of these findings are twofold. First, they provide positive evidence of ALIs in letter
writing that demonstrates convergence with the cognitive neuropsychological evidence
described earlier. Second, the specific neural location of the results is also informative.
The neural responses consistent with ALIs were identified within the left fusiform region
that—as we have seen throughout this chapter—has been independently associated with
lexical orthographic representations in spelling (and reading) in neuroimaging with neu-
rologically intact individuals and in lesion-deficit correlation studies. This association
specifically strengthens the argument that the O-LTM orthographic representations that
form the basis for spelling consist of abstract, case-independent letter identities.
1 These ALI selective effects are unlikely to have been driven by task/stimulus expectation given
that, although there were more change than no-change trials, the different types of change trials were
equi-probable.
(a) fMRI Adaptation Writing Task: Single trial types with same or different (b) Predicted Writing Task BOLD Response
fourth repetition stimulus Same Trial (inhibition)
2
1.5
1000 ms 1
0.5
3000 ms 1000 ms 1000 ms 1000 ms
1500 ms 500 ms 500 ms 500 ms 500 ms 0
+ t P + + + + OR
–0.5
Neural response (% signal change)

Time
Different Trial (release from inhibition)

2
Time
1.5
1
(c) fMRI Adaptation Results
0.5
–0.5
Neural response (% signal change)
Time
Fusiform Gyrus (–41, –60, –13)
Right
Figure 17.5. Dufor and Rapp (2013) fMRI study of letter writing that provided evidence of abstract letter identities in spelling. (A) The sequence of events
in a trial of the task used to measure neural adaptation: after associating two letters with two colors, each trial consisted of three repetitions of the same letter,
followed by either a fourth repetition (same trials) or a different letter (change trials). Brain areas representing ALIs should show a release from adaptation when
letter identity changes, but not when it does not, despite changes in size, case, or shape. (B) Predicted blood oxygenation level dependent (BOLD), depending on
whether the trial’s fourth stimulus was the same or different. What constitutes a change will be determined by the types of representations processed in a given
brain area. (C) Within letter-sensitive cortical areas, only the left fusiform gyrus cluster (–41, –60, –13) exhibited greater release from inhibition for trials with
identity changes (e.g., tttP or tttp) than for trials when the identity stayed the same but the case-size or case-shape changed (e.g., vvvV or qqqQ).
Source: Dufor & Rapp (2013).
Looking Forward
In this chapter, we have examined four fundamental issues of cognitive representa-

tion and processing in spelling: the distinction between orthographic long-term and
working-memory, the distinction between lexical and sublexical spelling processes,
the relationship between reading and spelling, and the role of abstract letter representa
tions in spelling. For all of these issues, we have discussed neural findings that have
contributed significantly to our understanding of these issues. In some cases, the neural
data provided convergence with behavioral findings; in others, they provided unique
contributions to the arguments. These detailed examples of the contributions of neural
data to addressing cognitive questions is relevant to an ongoing debate between those
who have argued that this is either not possible in principle (van Orden & Paap, 1997) or
has not been achieved in practice (Coltheart, 2006) and those who have argued that
neural data can be relevant to furthering our understanding of cognitive questions
(Henson, 2006; see also de Zubicaray & Piai, Chapter 19 in this volume).
We have indicated many issues that remain unresolved, and many more important
issues in written language production were not discussed. Few of these have received
significant research attention, either behavioral or neural. Among others, these in-
clude similarities and differences in written production across different scripts and
orthographic codes; spelling processes beyond the single word; and, importantly, the
development and learning of spelling and writing (both normal development and de-
velopmental dysgraphia). In addition, there is a large variety of additional sources
of neural data to be considered in addressing these issues. These include different
forms of neural measurements and analysis with neurologicaly healthy individuals
(e.g., ERP, MEG, resting-state fMRI, among others), as well as different etiologies of
neural impairment. For example, in addition to investigations of deficits following
stroke, trauma, or surgery, there are an increasing number of studies that involve the
detailed study of the spelling deficits of individuals with progressive neurological di-
sease (e.g., Graham, Patterson, & Hodges, 1997; Haslam, Kay, Tree, & Baronm, 2009;
O’Dowd & Zubicaray, 2003; Primativo, Yong, Shakespeare, & Crutch, 2017). Clearly,
therefore, there is much to be done and many contributions to come from neurolin-
guistics research on written language production. The approaches and techniques
described here can be used to address some of these issues as well, but neuroimaging
methods and analysis techniques are evolving rapidly, and will surely contribute to
our ability to use these approaches to address increasingly detailed and sophisticated
cognitive questions.
Acknowledgments
We gratefully acknowledge the support for B. R. and J. P. from the multi-site NIH grant
DC012283, directed at examining the neurobiology of language recovery in aphasia.
References
Alexander, M. P., Fischer R. S., & Friedman, R. B. (1992). Lesion localization in apractic
agraphia. Archives of Neurology, 49(3), 246–251.
Allport, D. A., & Funnell, E. (1981). Components of the mental lexicon. Philosophical
Transactions of the Royal Society of London Series B, Biological Sciences, 295, 397–410
Ans, B., Carbonnel, S., & Valdois, S. (1998). A connectionist multiple-trace memory model for
polysyllabic word reading. Psychological Review, 105(4), 678–723.
Baddeley, A. D., & Logie, R. H. (1999). Working memory: The multiple-component model. In
A. Miyake & P. Shah (Eds.), Models of working memory: Mechanisms of active maintenance
and executive control (pp. 28–61). Cambridge: Cambridge University Press.
Barron, H. C., Garvert, M. M., & Behrens, T. E. J. (2016). Repetition suppression: A means to
index neural representations using BOLD? Philosophical Transactions of the Royal Society of
London. Series B, Biological Sciences, 371(1705). doi: 10.1098/rstb.2015.0355
Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59, 617–645.
Bonin, P., Peereman, R., & Fayol, M. (2001). Do phonological codes constrain the selection
of orthographic codes in written picture naming? Journal of Memory and Language, 45(4),
688–720.
Breining, B., Nozari, N., & Rapp, B. (2016). Does segmental overlap help or hurt? Evidence
from blocked cyclic naming in spoken and written production. Psychonomic Bulletin &
Review, 23(2), 500–506.
Bub, D., & Kertesz, A. (1982). Evidence for lexicographic processing in a patient with preserved
written over oral single word naming. Brain, 105(Pt 4), 697–7 17.
Buchwald, A., & Rapp, B. (2009). Distinctions between orthographic long-term memory and
working memory. Cognitive Neuropsychology, 26(8), 724–751.
Burt, J. S., & Tate, H. (2002). Does a reading lexicon provide orthographic representations for
spelling? Journal of Memory and Language, 46(3), 518–543.
Caccappolo-van Vliet, E., Miozzo, M., & Stern, Y. (2004). Phonological dyslexia without pho-
nological impairment? Cognitive Neuropsychology, 21(8), 820–839.
Caramazza, A., Capasso, R., & Miceli, G. (1996). The role of the graphemic buffer in reading.
Cognitive Neuropsychology, 13(5), 673–698. https://doi.org/10.1080/026432996381881
Caramazza, A., & Miceli, G. (1990). The structure of graphemic representations. Cognition,
37(3), 243–297.
Caramazza, A., Miceli, G., Villa, G., & Romani, C. (1987). The role of the graphemic buffer in
spelling: Evidence from a case of acquired dysgraphia. Cognition, 26(1),  59–85.
Carreiras, M., Armstrong, B. C., Perea, M., & Frost, R. (2014). The what, when, where, and how
of visual word recognition. Trends in Cognitive Sciences, 18(2),  90–98.
Carreiras, M., Quiñones, I., Hernández- Cabrera, J. A., & Duñabeitia, J. A. (2015).
Orthographic coding: Brain activation for letters, symbols, and digits. Cerebral Cortex,
25(12), 4748–4760.
Cohen, L., Dehaene, S., Vinckier, F., Jobert, A., & Montavont, A. (2008). Reading normal and
degraded words: Contribution of the dorsal and ventral visual Pathways. NeuroImage, 40 (1),
353–366.
Cohen, L., Lehericy, S., Chochon, F., Lemer, C., Rivaud, S., & Dehaene, S. (2002). Language-
specific tuning of visual cortex? Functional properties of the visual word form area. Brain,
125(Pt 5), 1054–1069.
Coltheart, M. (2006). Dual route and connectionist models of reading: An overview. London
Review of Education, 4(1),  5–17.
Coltheart, M., Curtis, B., Atkins, P., & Haller, M. (1993). Models of reading aloud: Dual-route
and parallel-distributed-processing approaches. Psychological Review, 100(4), 589.
Costa, V., Fischer-Baum, S., Capasso, R., Miceli, G., & Rapp, B. (2011). Temporal stability and
representational distinctiveness: Key functions of orthographic working memory. Cognitive
Cowan, N. (1999). An embedded-processes model of working memory. In A. Miyake & P. Shah
(Eds.), Models of working memory: Mechanisms of active maintenance and executive control
(pp. 62–101). Cambridge: Cambridge University Press.
Damian, M. F., & Bowers, J. S. (2003). Locus of semantic interference in picture-word interfer-
ence tasks. Psychonomic Bulletin & Review, 10(1), 111–117.
Dufor, O., & Rapp, B. (2013). Letter representations in writing: An fMRI adaptation approach.
Glasspool, D. W., Shallice, T., & Cipolotti, L. (2006). Towards a unified process model for gra-
phemic buffer disorder and deep dysgraphia. Cognitive Neuropsychology, 23(3), 479–512.
Glezer, L. S., Jiang, X., & Riesenhuber, M. (2009). Evidence for highly selective neuronal tuning
to whole words in the visual word form area. Neuron, 62(2), 199–204.
Graham, N. L., Patterson, K., & Hodges, J. R. (1997). Progressive dysgraphia: Co-occurrence of
central and peripheral impairments. Cognitive Neuropsychology, 14, 975–1005.
Harm, M. W., & Seidenberg, M. S. (1999). Phonology, reading acquisition, and dys-
lexia: Insights from connectionist models. Psychological Review, 106(3), 491–528.
Haslam, C., Kay, J., Tree, J., & Baron, R. (2009). Dysgraphia in dementia: A systematic investi-
gation of graphemic buffer features in a case series. Neurocase, 15(4), 338–351.
Henry, M. L., Beeson, P. M., Stark, A. J., & Rapcsak, S. Z. (2007). The role of left perisylvian cor-
tical regions in spelling. Brain and Language, 100(1),  44–52.
Henson, R. (2006). What has (neuro)psychology told us about the mind (so far)? A reply to
coltheart. Cortex, 42(3), 387–392. https://doi.org/10.1016/S0010-9452(08)70365-4
Hillis, A. E., & Caramazza, A. (1995). Representation of grammatical categories of words in the
brain. Journal of Cognitive Neuroscience, 7(3), 396–407.
Hillis, A. E., Wityk, R. J., Barker, P. B., Beauchamp, N. J., Gailloud, P., Murphy, K., Cooper, O.,
& Metter, E. J. (2002). Subcortical aphasia and neglect in acute stroke: The role of cortical
hypoperfusion. Brain, 125(Pt 5), 1094–1104.
Holmes, V. M., & Carruthers, J. (1998). The relation between reading and spelling in skilled
adult readers. Journal of Memory and Language, 39(2), 264–289.
Houghton, G., Glasspool, D. W., & Shallice, T. (1994). Spelling and serial recall: Insights
from a competitive queuing model. In G. D. A. Borwn & N. C. Ellis (Eds.), Handbook of
spelling: Theory, process and intervention (pp. 365–404). Chichester, UK: John Wiley.
Jobard, G., Crivello, F., & Tzourio-Mazoyer, N. (2003). Evaluation of the dual route theory of
reading: A metanalysis of 35 neuroimaging studies. NeuroImage, 20(2), 693–7 12.
Jonides, J., Schumacher, E. H., Smith, E. E., Koeppe, R. A., Awh, E., Reuter-Lorenz, P. A.,
Marshuetz, C., & Willis, C. R. (1998). The role of parietal cortex in verbal working memory.
The Journal of Neuroscience, 18(13), 5026–5034.
Kessels, R. P. C., Van Zandvoort, M. J. E., Postma, A., Kappelle, L. J., & De Haan, E. H. F. (2000).
The corsi block-tapping task: Standardization and normative data. Applied Neuropsychology,
7(4), 252–258.
Kinsbourne, M., & Warrington, E. K. (1965). A case showing selectively impaired oral spelling.
Journal of Neurology, Neurosurgery, and Psychiatry, 28(6), 563–566.
Larsson, J., & Smith, A. T. (2012). fMRI repetition suppression: Neuronal adaptation or stim-
ulus expectation? Cerebral Cortex, 22(3), 567–576.
Longcamp, M., Anton, J. L., Roth, M., & Velay, J. L. (2003). Visual presentation of single letters
activates a premotor area involved in writing. NeuroImage, 19(4), 1492–1500.
Ludersdorfer, P., Kronbichler, M., & Wimmer, H. (2015). Accessing orthographic
representations from speech: The role of left ventral occipitotemporal cortex in spelling.
Human Brain Mapping, 36(4), 1393–1406.
Martin, A., Schurz, M., Kronbichler, M., & Richlan, F. (2015). Reading in the brain of children
and adults: A meta‐analysis of 40 functional magnetic resonance imaging studies. Human
Brain Mapping, 36(5), 1963–1981.
Miceli, G., & Costa, V. (2014). The role of lexical and sublexical orthography in writing:
Autonomy, interactions and neurofunctional correlates. In M. A. Goldrick, V. Ferreira, & M.
Miozzo (Eds.), The Oxford handbook of language production (pp. 358–378), Oxford: Oxford
University Press.
Miyake, A., & Shah, P. (1999). Models of working memory: Mechanisms of active maintenance
and executive control, Cambridge: Cambridge University Press.
Monsell, S. (1987). On the relation between lexical input and output pathways for speech.
Cambridge, MA: Academic Press.
O’Dowd, B. S., & de Zubicaray, G. I. (2003) Progressive dysgraphia in a case of posterior cor-
tical atrophy. Neurocase, 9(3), 251–260.
Patterson, K., & Lambon Ralph, M. A. (1999). Selective disorders of reading? Current Opinion
in Neurobiology, 9(2), 235–239.
Philipose, L. E., Gottesman, R. F., Newhart, M., Kleinman, J. T., Herskovits, E. H., Pawlak,
M. A., Marsh, E. B., Davis, C., Heidler‐Gary, J., & Hillis, A. E. (2007). Neural regions es-
sential for reading and spelling of words and pseudowords. Annals of Neurology, 62(5),
481–492.
Planton, S., Jucla, M., Roux, F.-E., & Démonet, J.-F. (2013). The “handwriting brain”: A meta-
analysis of neuroimaging studies of motor versus orthographic processes. Cortex, 49(10),
2772–2787. https://doi.org/10.1016/j.cortex.2013.05.011
Plaut, D. C., McClelland, J. L., Seidenberg, M. S., & Patterson, K. (1996. Understanding
Psychological Review, 103(1), 56.
Pulvermüller, F. (2005). Brain mechanisms linking language and action. Nature Reviews
Primativo, S., Yong, K. X. X., Shakespeare, T. J., & Crutch, S. J. (2017). The oral spelling profile of
posterior cortical atrophy and the nature of the graphemic representation. Neuropsychologia,
94,  61–74.
Purcell, J. J., Napoliello, E. M., & Eden, G. F. (2011). A combined fMRI study of typed spelling
and reading. NeuroImage, 55(2), 750–762.
Purcell, J. J., Shea, J., & Rapp, B. (2014). Beyond the visual word form area: The
orthography–semantics interface in spelling and reading. Cognitive Neuropsychology,
31(5–6), 482–5 10.
Purcell, J. J., Turkeltaub, P. E., Eden, G. F., & Rapp, B. (2011). Examining the central and periph-
eral processes of written word production through meta-analysis. Frontiers in Psychology,
2(239), 10.3389.
Purcell, J. J., Jiang, X., & Eden, G. F. (2017). Shared orthographic neuronal representations for
spelling and reading. NeuroImage, 147, 554–567.
Rapcsak, S. Z., & Beeson, P. M. (2004). The role of left posterior inferior temporal cortex in
spelling. Neurology, 62(12), 2221–2229.
Rapcsak, S. Z., & Beeson, P. M. (2015). Neuroanatomical correlates of spelling and writing.
In A. E. Hillis (Ed.), Handbook on adult language disorders, 2nd Edition (pp. 87–116).
New York: Psychology Press.
Rapcsak, S. Z., Beeson, P. M., Henry, M. L., Leyden, A., Kim, E., Rising, K., Andersen, S., &
Cho, H. (2009). Phonological dyslexia and dysgraphia: Cognitive mechanisms and neural
substrates. Cortex, 45(5), 575–591.
Rapp, B., & Caramazza, A. (1997). From graphemes to abstract letter shapes: Levels of repre-
sentation in written spelling. Journal of Experimental Psychology: Human Perception and
Performance, 23(4), 1130–1152.
Rapp, B., & Dufor, O. (2011). The neurotopography of written word production: An fMRI in-
vestigation of the distribution of sensitivity to length and frequency. Journal of Cognitive
Neuroscience, 23(12), 4067–4081.
Rapp, B., & Fischer-Baum, S. (2015). Uncovering the cognitive architecture of spelling.
In A. E. Hillis (Ed.), The handbook of adult language disorders (2nd ed., pp. 59–86).
Rapp, B., Folk, J. R., & Tainturier, M. J. (2001). Word reading. In B. Rapp (Ed.), The handbook
of cognitive neuropsychology: What deficits reveal about the human mind (pp. 233–262).
Rapp, B., & Lipka, K. (2011). The literate brain: The relationship between spelling and reading.
Journal of Cognitive Neuroscience, 23(5), 1180–1197.
Rapp, B., Purcell, J. J., Hillis, A. E., Capasso, R., & Miceli, G. (2016). Neural bases of
orthographic long-term memory and working memory in dysgraphia. Brain, 139(Pt 2),
588–604.
Roeltgen, D. P., Sevush, S., & Heilman, K. M. (1983). Phonological agraphia: Writing by the
lexical-semantic route. Neurology, 33(6), 755–765.
Roth, J. K., & Courtney, S. M. (2007). Neural system for updating object working memory
from different sources: Sensory stimuli or long-term memory. NeuroImage, 38(3), 617–630.
Rothi, L. J., & Heilman, K. M. (1981). Alexia and agraphia with spared spelling and letter recog-
nition abilities. Brain and Language, 12(1),  1–13.
Ruchkin, D. S., Grafman, J., Cameron, K., & Berndt, R. S. (2003). Working memory reten-
tion systems: A state of activated long-term memory. Behavioral and Brain Sciences, 26(06),
709–728.
Sage, K., & Ellis, A. W. (2004). Lexical influences in graphemic buffer disorder. Cognitive
Neuropsychology, 21(2–4), 381–400.
Seidenberg, M. S., & McClelland, J. L. (1989). A distributed, developmental model of word rec-
ognition and naming. Psychological Review, 96(4), 523.
Serences, J. T., & Yantis, S. (2006). Selective visual attention and perceptual coherence. Trends
in Cognitive Sciences, 10(1),  38–45.
Sheldon, C. A., Malcolm, G. L., & Barton, J. J. (2008). Alexia with and without agraphia: An
assessment of two classical syndromes. The Canadian Journal of Neurological Sciences: Le
Journal Canadien Des Sciences Neurologiques, 35(5), 616–624.
Shomstein, S., & Yantis, S. (2004). Control of attention shifts between vision and audition in
human cortex. The Journal of Neuroscience, 24(47), 10702–10706.
Shomstein, S., & Yantis, S. (2006). Parietal cortex mediates voluntary control of spatial and
nonspatial auditory attention. Journal of Neuroscience, 26(2), 435–439. https://doi.org/
10.1523/JNEUROSCI.4408-05.2006
Summerfield, C., Trittschuh, E. H., Monti, J. M., Mesulam, M.-M., & Egner, T. (2008). Neural
repetition suppression reflects fulfilled perceptual expectations. Nature Neuroscience, 11,
1004–1006.
Tainturier, M. J., & Rapp, B. (2001). The spelling process. In B. Rapp (Ed.), The handbook of
cognitive neuropsychology: What deficits reveal about the human mind (pp. 263–289).
Tainturier, M. J., & Rapp, B. (2003). Is a single graphemic buffer used in reading and spelling?
Aphasiology, 17(6–7), 537–562.
Todd, J. J., & R. Marois. (2004). Capacity limit of visual short-term memory in human poste-
rior parietal cortex. Nature, 428(6984), 751–754.
Turkeltaub, P. E., Eden, G. F., Jones, K. M., & Zeffiro, T. A. (2002). Meta-analysis of the func-
tional neuroanatomy of single-word reading: Method and validation. NeuroImage, 16(3),
765–780.
Van Orden, G. C., & Paap, K. R. (1997). Functional neuroimages fail to discover pieces of mind
in the parts of the brain. Philosophy of Science, 64, S85–S94.
Wager, T. D., & Smith, E. E. (2003). Neuroimaging studies of working memory: A meta-
analysis. Cognitive, Affective & Behavioral Neuroscience, 3(4), 255–274.
Ward, J., & Romani, C. (1998). Serial position effects and lexical activation in spelling: Evidence
from a single case study. Neurocase, 4(3), 189–206.
Zangwill, O. L. (1954). Agraphia due to a left parietal glioma in a left-handed man. Brain, 77(4),
510–520.
Chapter 18
Motor Speech Di s orde rs
Wolfram Ziegler, Theresa Schölderle,

Ingrid Aichert, and Anja Staiger
Introduction
Like any other motor system of the human body, the motor system involved in speaking
can be impaired by lesions to the brain. As a consequence, the movements of the respi-
ratory, laryngeal, and supra-laryngeal motor organs that contribute to the generation of
speech are disordered, resulting in unnaturally sounding and/or unintelligible speech.
Depending on which parts of the brain’s speech motor network are affected, a diversity
of syndromes may emerge, which are subsumed under the clinical terms dysarthria and
apraxia of speech (e.g., Duffy, 2013). Patients with speech motor disorders represent the
largest group among those suffering from neurogenic communication disorders, with an
estimated prevalence of more than 4 per 1,000 in Germany. Dysarthria and apraxia of
speech constitute a severe health problem because the afflicted persons may suffer from
serious restrictions in their social and occupational participation (Schölderle, Staiger,
Schumacher, & Ziegler, in press).
In the organization of this chapter, we use the functional neuroanatomy of motor
speech as a framework to describe how speech motor control can be compromised
by brain lesions, with a particular focus on some of the most controversial issues.
In our interpretation of the available clinical data we will emphasize, in a final
section, how deeply the motor system of speaking is entrenched in language and
communication.
450 Wolfram Ziegler, Theresa Schölderle, Ingrid Aichert, Anja Staiger
The Neural Basis of Speech

Motor Disorders
Gross Architecture
Figure 18.1 gives a simplified overview of the gross architecture of the speech motor
system (for a more detailed description, see Tremblay, Deschamps, & Dick, Chapter 15 in
this volume). A core component is the primary motor cortex, located on the precentral
gyri of both hemispheres, with its descending pathway to the cranial (and also sev-
eral spinal) motor nuclei innervating the respiratory, laryngeal, and vocal tract mus-
cles. Along this pathway, the motor commands originating in the motor cortical areas
representing the speech muscles are conveyed to the speech organs (discussed later
in this chapter). This descending pathway is supported by two loop systems through
which the motor cortical information is continuously updated: the cerebellar loop and
the basal ganglia (“striatal”) loop (both discussed later in further detail). Together, these
components—the corticobulbar system with the striatal and cerebellar loops—may be
referred to as the speech motor execution system.
The motor execution network receives motor planning information from anterior
peri-and/or subsylvian cortical regions of the left hemisphere, in Figure 18.1 referred
to as left posterior inferior frontal gyrus (pIFG; see later discussion). This cortical area
is connected with higher auditory processing areas in the superior temporal gyrus
(STG) and with areas in the inferior-parietal and the parieto-temporal region of the left
hemisphere—a network known as the auditory dorsal pathway. This pathway has a cru-
cial role in higher order sensorimotor integration processes for speech, especially also as
Emotional/motivational input Motor planning

Left
Mesiofrontal cortex Left pIFG
IPG/STG
Sensorimotor cortex
Motor execution
BG CBL
PAG
Cranial motor nuclei
Speech muscles
Figure 18.1. Simplified outline of the neural architecture of speech motor control.

BG: Basal ganglia; CBL: cerebellum; IPG: inferior parietal gyrus; PAG: Peri-aqueductal grey; pIFG:
posterior inferior frontal gyrus; STG: superior temporal gyrus. Rectangles with rounded corners represent
feedforward-feedback loop systems.
Motor Speech Disorders 451
a relay system interfacing auditory with motor information (see, in this volume, Hickok,
Chapter 20, and Poeppel, Cogan, Davidesco, & Flinker, Chapter 26).
Finally, the speech motor network also receives input from the anterior cingu-
late cortex (ACC) located at the medial walls of the frontal lobes, which is supposed
to channel emotional and motivational input into the human motor control system for
speaking.
This architecture will in the following serve as a guideline to describe the sequelae
of brain lesions to speech motor control. For reasons of conciseness, the disturbances
resulting from dysfunctions of the mesio-frontal (“limbic”) motor network, including
the supplementary motor area (SMA), will not be described in a separate section (for
details on this issue, readers are referred to Ackermann, Hage, and Ziegler, 2014).
Moreover, we will concentrate on the central nervous system and spare the dysarthric
impairments resulting from lesions to the peripheral, lower motor neuron level of the
speech motor system. Furthermore, the ascending sensory pathways projecting from
the solitary tract and spinal trigeminal nuclei via thalamic nuclei to somatosensory
areas on the postcentral gyrus will not be considered (cf. Jürgens, 2002). For a more
comprehensive presentation of clinical issues, see Duffy (2013).
The Upper Motor Neuron System and Paretic Dysarthria

Functional Topography
The speech muscles receive their neural commands via the motor nuclei in the brain
stem and in parts of the spinal cord. These commands have their origin in the ventral
pre-central cortex (Brodmann area 4) of the left and right hemispheres, from where they
travel along the cortico-bulbar tracts to the trigeminal motor (Vmo), the facial (VII), the
ambigual (X), and the hypoglossal nuclei (XII) of the brain stem, as well as to motor nu-
clei in cervical and thoracic segments of the spinal cord. The corticobulbar and cortico-
spinal motor tracts are referred to as the upper motor neuron (UMN) system. From
the motor nuclei, the nerve fibers innervating the individual speech muscles (the lower
motor neuron, LMN) originate. The LMN system will not be considered further in this
chapter.
Apart from the facial nucleus innervating the muscles of the lower face, the brain
stem motor nuclei of either side receive fibers from both hemispheres, but with a pre-
dominant input from contralateral motor cortex. In humans, all brain stem motor nu-
clei, including the nucleus ambiguus, receive direct (i.e., monosynaptic) input from
primary motor cortex. This distinguishes the human speech motor system from the
(nonhuman) primate vocalization system, which lacks dense monosynaptic motor cor-
tical projections to the ambigual nuclei innervating the laryngeal musculature. As a hy-
pothesis, the monosynaptic differentiation of the human corticobulbar tracts during
phylogenesis is associated with the refinement of the behavioral repertoire of the human
larynx involved in speech and articulation (for references, see Ackermann et al., 2014).
The cranial motor nuclei also receive parallel cortical input via indirect fibers synapsing
at various sites in the striatum and the brain stem. These connections are considered
to mediate primarily the regulation of reflexes, posture, and tone, rather than the fine-
tuning of voluntary movements (e.g., Strominger, Demarest, & Laemle, 2012).
As in the limb muscles, the UMN system for speech has a somatotopic organization,
with selective representation sites of the lips, tongue, larynx, and respiratory organs in
the sensori-motor cortices (Brown et al., 2009). However, on a finer-grained level, the
motor cortical fields of the speech muscles are arranged in a more intricate manner,
with distributed and overlapping representational maps of individual muscles, and
representations of complex, macro-scale muscle synergies. Multi-electrode recordings
of speech-related motor-cortical electrical activity in individuals who underwent sur-
gical treatment of epileptic seizures revealed complex temporal activation patterns for
syllables, reflecting a high specificity of the organization of motor representations in
primary motor cortex for the phonetic patterning of speech, presumably as a result of
speech motor learning during language acquisition (Bouchard, Mesgarani, Johnson, &
Chang, 2013). This corresponds with findings relating to skilled motor actions of the
hands, according to which the learning of a motor skill shapes the representational maps
of the involved muscles in primary motor cortex through mechanisms of learning-
based neural plasticity (e.g., Matsuzaka, Picard, & Strick, 2007). We will come back to
this issue later in the chapter.
Clinical Aspects
Lesions to the motor cortical representation areas of the speech organs and/or the de-
scending fibers along their way to the brain stem lead to paresis (muscle weakness) and
a syndrome termed paretic dysarthria. All levels of the speech motor apparatus can
be afflicted to various degrees: weakness of the expiratory muscles may lead to short
breathing cycles, weakness of the laryngeal muscles to impaired voice quality and mo-
notonous intonation, and paresis of the vocal tract muscles to slowed and imprecise ar-
ticulation (Duffy, 2013). Due to the bilateral organization of the corticobulbar pathway,
one-sided lesions usually cause only mild dysarthria that may disappear within several
days or weeks (Urban et al., 2006).
Though the term paretic dysarthria suggests muscle weakness as the core mech-
anism underlying this impairment, existing clinical data are not conclusive regarding
the complex pathophysiological conditions resulting from lesions to the UMN system
supplying the speech muscles (upper motor neuron syndrome, UMNS). In the limb
muscles, paresis in UMNS is mostly accompanied by “positive” signs in addition to
weakness (i.e., muscular overactivity and spastic hypertonus; Sheean, 2002). Since
hypertonus and muscular overactivity can also be visible in the speech musculature
(e.g., hyperadduction of the vocal folds during laryngoscopy) and inferred from deviant
speech parameters (e.g., strained-strangled voice), the term spastic dysarthria has be-
come established. However, the patho-mechanisms underlying these signs are largely
unknown—not least because the spinal reflex mechanisms underlying spasticity in the
extremities cannot be transferred straightforwardly to the muscles implied in speaking.
Moreover, the methods to assess spasticity in the arms and legs are not applicable to the
speech motor apparatus. It is hypothesized that spastic hypertonus in the speech mus-
cles results from lesions of the indirect fibers traveling from motor cortex to the brain
stem nuclei. Since the signs of spastic hyperactivity are usually absent in patients suf-
fering from lesions to the lower motor neurons, the speech impairment resulting from
LMN lesions is referred to as flaccid dysarthria (Duffy, 2013).
Apart from weakness, another negative sign of the UMNS is loss of dexterity, for ex-
ample in hand and finger movements (Sheean, 2002). As is known from hand motor
control, an important neural basis of functional dexterity must be sought in (1) the task-
specific organization of representational maps in primary motor cortex, and (2) the
presence of direct, monosynaptic cortico-motoneuronal connections (Xu, Haith, &
Krakauer, 2015). Since both these principles are met in the UMN system for speech, as
was mentioned earlier, lesions to this network will not only reduce muscle strength, but
also interfere with the fine tuning of multi-articulator movements and lead to reduced
dexterity of articulation. This deficit is probably often masked by the more deleterious
effects of weakness and muscular overactivity in patients with severe spastic-paretic
dysarthria. Yet, in the usually mild speech impairment of unilateral UMNS, “clumsy”
articulation may emerge as a predominant symptom (Duffy, 2013).
Hemispheric Lateralization
There is a controversy about whether the corticobulbar system, with its bilateral organi-
zation, may exhibit a relative left lateralization for speech. In the motor cortex, hemody-
namic activation lateralized to the left hemisphere has been demonstrated by functional
neuroimaging during speech, but less so during non- speech tongue movements
(Wildgruber, Ackermann, Klose, Kardatzki, & Grodd, 1996). Moreover, there is clinical
evidence that left lateralization may also involve the descending corticobulbar fiber
tracts. Urban et al. (2006) demonstrated that lesions within the face area of the primary
motor cortex and along the course of the corticobulbar tract caused more severe dysar-
thria in patients with left as compared to right hemisphere infarctions. They concluded
that there is at least a slight left dominance for articulation at the motor cortical level and
the level of the descending white matter pathways. In a recent study based on magnetic
resonance imaging (MRI) tractography of the corticobulbar tract in healthy adults,
Liégeois, Butler, Morgan, Clayden, and Clark (2015) failed to find any evidence for struc-
tural or functional left asymmetry in their participants. However, this investigation was
not based on brain activations for speech, but rather on a rapid syllable repetition para-
digm that may involve speech-specific brain activity to a much lesser extent.
Another unresolved problem arising at this point is whether areas in the left primary
motor cortex not only are specialized for more than simply the fine-tuning of articu-
latory agility (i.e., dexterity), but are also involved in higher-order speech planning
mechanisms. In several case reports (e.g., Schiff, Alexander, Naeser, & Galaburda, 1983;
Tanji et al., 2001), patients with lesions in the left ventral precentral region demonstrated
a complex motor speech impairment that differed from dysarthria in two respects. First,
the symptoms were not mild and transient, as one would expect in dysarthria resulting
from circumscribed unilateral lesions, but were clinically remarkable and persisted
over longer periods. Second, the described symptom patterns were more complex
than one would expect from simply a weakness or clumsiness of the articulators—they
rather resembled the pattern of apraxia of speech described in patients with more ante-
rior left frontal lesions (see later discussion). This suggests that the prerolandic motor
area representing the articulatory muscles has a more elaborate organization in the left
than in the right hemisphere, and that left primary speech motor cortex (or a part of it)
subserves similar motor planning functions as the areas immediately anterior to it, that
is, ventro-lateral premotor cortex (BA 6), and adjacent posterior inferior frontal cortex
(Broca’s area, pars opercularis).
Emotional-Voluntary Dissociation
One of the most striking clinical observations related to paretic dysarthria resulting
from lesions above the level of the cranial motor nuclei is that even severe bilateral
damage to the corticobulbar system (e.g., in the so-called Foix-Chavany-Marie syn-
drome or in pseudobulbar palsy) may completely spare the motor functions of the
speech musculature during emotional-expressive movements (i.e., in laughing or
crying), while it destroys all volitional functions of the same muscles, including speech
(e.g., Nowak, Griebl, Dabitz, & Ochs, 2010). A similar observation of an “automatic-
voluntary” (better: “emotional-voluntary”) dissociation may occur in individuals with
unilateral UMN lesions. In these patients, voluntary lip functions (e.g., in cranial nerve
examinations or in speech) are impaired by one-sided weakness of the lower facial mus-
cles, although their spontaneous smiles are often perfectly symmetric (volitional facial
paresis; e.g., Hopf, Müller-Forell, & Hopf, 1992). Together, these clinical dissociations
are considered to evidence the existence of two independent pathways to the brain stem
motor nuclei: a voluntary pathway originating in primary motor cortex, and a “limbic”
pathway originating in the anterior cingulate cortex of the medial wall of the frontal lobe
and mediating intrinsic or emotional-expressive movements of the speech muscles (see
Figure 18.1). This latter system will be revisited later in this chapter (for a more compre-
hensive discussion, see Ackermann et al., 2014).
The Cerebellar Loop and Ataxic Dysarthria

The cerebellum, situated in the posterior cranial fossa and connected with the brain stem
via three pairs of cerebellar peduncles, is part of several loop systems interconnecting
it with diverse cerebral cortical regions (Middleton & Strick, 2000). The most exhaus-
tively studied cerebellar circuit is the motor loop, which interlinks, in a somatotopically
organized manner, motor areas of the cerebral cortex with cerebellar motor regions in
mostly the anterior, but also the posterior lobe of the contralateral cerebellar cortex. The
motor loop comprises (1) a feedforward limb involving projections from primary motor
cortex, ventral premotor cortex, and the SMA, traveling through pontine nuclei to the
contralateral cerebellar hemispheres, and (2) a feedback limb via the deep cerebellar nu-
clei to the motor nuclei of the contralateral thalamus and further on to the motor areas
of the cerebral cortex. Moreover, the cerebellum receives somatosensory information
from oral and facial muscles via the trigeminal nuclei. By this afferent pathway, sensory
information is channeled into cerebellar cortex, thus providing the cerebellar motor
loop with information about the sensory state of the speech apparatus (for reviews of the
pertinent literature, see Manto et al., 2012).
Through its topographic organization, the cerebellar motor loop is generally
considered to implement an inverse dynamics model (“forward model”), which
computes the motor commands needed to achieve a desired movement goal (Wolpert,
Miall, & Kawato, 1998). As for speech motor control, more specifically, the cerebellum is
known to be implicated in the evaluation of auditory and somatosensory error signals
required in adaptation and compensation mechanisms (Golfinopoulos et al., 2011), as
well as in the learning of novel articulations (Segawa, Tourville, Beal, & Guenther, 2014).
These functions are essential for the acquisition, maintenance, and constant updating
of a forward model of motor speech (e.g., Hickok, 2012; see Hickok, Chapter 20 in this
volume). However, there is still no detailed account of the sensorimotor computations
that the cerebellum performs during unperturbed speaking (see Tourville, Peeva, &
Guenther, 2014, for a recent review).
Clinical Aspects
As originally described by Holmes (1917), lesions to the cerebellum may cause a move-
ment disorder termed motor ataxia, that is, movement incoordination resulting in,
among other things, overshooting or undershooting movements (dysmetria) and a
decomposition of complex synergistic movements, as well as dysdiadochokinesia (i.e.,
slowing and irregular timing of repetitive movements). Ataxia is generally considered
to reflect difficulties regulating the force, range, direction, velocity, and timing of muscle
contractions, due to a dysfunction of the sensorimotor interactions mediating the feed-
forward and feedback control mechanisms involved in movement control.
Affliction of speech movements, resulting in ataxic dysarthria, may occur after lesions
to predominantly the paravermal parts of the anterior cerebellum (Schoch, Dimitrova,
Gizewski, & Timmann, 2006), for example, after infarctions of the superior cerebellar
artery or in the course of degenerative disorders of the cerebellum (Brendel et al., 2013;
Brendel et al., 2015). Regarding the lateralization of cerebellar ataxic dysarthria, a similar
(though mirroring) situation subsists, as in the descending corticobulbar tract: lesions
of both cerebellar hemispheres can cause dysarthria, but there seems to be a statistical
preponderance of right cerebellar hemispheric involvement (Urban et al., 2003).
Ataxic dysarthria is characterized by respiratory, phonatory, and articula-
tory symptoms that appear much less predictable than those resulting from pa-
resis: dyscoordination of respiratory muscles may lead to a “paradoxical” breathing
pattern with unforeseen involuntary inspirations, voice quality and vocal pitch may un-
dergo sudden changes, there may be voice tremor, and the precision of vowel and conso-
nant articulations may vary between over-and undershooting realizations. Speaking is
usually slow. Patients often compensate for dysmetria or cerebellar tremor by increased
muscular tension (e.g., in the larynx) (for a comprehensive description of ataxic dysar-
thria, see Duffy, 2013).
Is There Cerebellar Speech Impairment beyond Dysarthria?

In the tradition of Holmes (1917), the consequences of cerebellar pathology have long been
understood as exclusively motor or perceptuo-motor, which is compatible with the no-
tion of ataxic dysarthria as a sensorimotor speech disorder. During the last few decades,
however, the cerebellum has been discovered as an organ that is also implicated in cogni-
tion and emotion (Koziol et al., 2014), as well as in linguistic functions like syntactic pro-
cessing, reading and writing, or verbal working memory (Mariën et al., 2014). This raises
the question of whether cerebellar lesions may interfere not only with the motor execution
aspects of speaking, but also with higher-order processes of speech sound production,
such as phonetic or phonological planning, as discussed later in this chapter. As regards
phonological planning impairment (i.e., phonemic paraphasia or phonemic jargon), there
is not the slightest evidence from published case reports that problems of this kind may
occur after cerebellar lesions (see Ziegler, 2015). However, a small number of studies have
described symptoms resembling phonetic planning impairment (apraxia of speech) in
patients with right cerebellar hypoperfusion (Mariën, Pickut, Engelborghs, Martin, & De
Deyn, 2001; Mariën et al., 2006). As a primordial condition, however, these patients had
focal lesions in the left cerebral hemisphere, which caused hypoperfusion in the contra-
lateral cerebellum only secondarily (crossed cerebellar diaschisis). Hence, the left cerebral
lesions found in these individuals may explain their speech impairments without any ad-
ditional assumptions regarding a potential cerebellar contribution. As a conclusion, the
implication of the cerebellum in sound production impairment seems indeed to be con-
fined to the motor execution aspects of speech production (ataxic dysarthria) and does
not encroach upon “higher” phonetic or phonological processes. This result constrains the
linguistic cerebellum hypothesis and substantiates assumptions about the hierarchical orga-
nization of speech motor control in the brain (see Ziegler, 2015).
The Striatal Motor Loop and the Hypokinetic

and Dyskinetic Dysarthrias
As outlined in Figure 18.1, the descending speech motor pathway is also supported by
a second, striatal (or basal ganglia) loop system. The basal ganglia are an assemblage of
subcortical nuclei, consisting of the neostriatum (caudate nucleus and putamen), the
globus pallidus, sub-thalamic nucleus, substantia nigra, and the nucleus accumbens.
The basal ganglia are part of several parallel recurrent cortico-subcortical circuits
subserving motor, cognitive, and motivational functions. They receive input from dif-
ferent neocortical areas and project back, via thalamic nuclei, to their respective cortical
origins (Middleton & Strick, 2000). The basal ganglia loop implicated in sensorimotor
control involves projections from multiple motor cortical areas—primary motor cortex,
lateral premotor cortex, and the SMA—to the dorsal putamen, and back from the in-
ternal segment of the globus pallidus and the substantia nigra, via thalamic motor
nuclei, to their motor cortical origins. At least two separate motor pathways can be dis-
tinguished within the internal connectivity of the basal ganglia: a direct, excitatory, and
an indirect, inhibitory loop (Graybiel, 2000).
One other cortico-striato-thalamo-cortical circuit is important here, the so-called
limbic pathway, connecting the basal ganglia with mesiofrontal cortical areas, especially
anterior cingulate cortex and the SMA. Though the limbic and motor striatal loops are
parallel circuits, they are also interconnected within the basal ganglia, thereby allowing
motivational and affective input to influence the voluntary motor system (Ackermann
et al., 2014; Turner & Desmurget, 2010).
According to widely accepted theories, the motor functions of the basal ganglia in-
clude primarily the scaling of movement speed and magnitude (“control of move-
ment gain”; Turner & Desmurget, 2010), and the procedural learning of motor skills,
especially motor sequence learning (Doyon & Benali, 2005). Both these functions are
considered to be modulated, via the limbic striatal loop, by motivation-and emotion-
related input (Turner & Desmurget, 2010).
Clinical Aspects
The two most representative clinical models of basal ganglia dysfunction are Parkinson’s
disease (PD) and Huntington’s disease (HD) (see Copland & Angwin, Chapter 33
in this volume). In a simplified view, underactivity of the excitatory direct basal gan-
glia pathway leads to hypokinesia and bradykinesia in Parkinson’s disease, resulting
in poverty and slowing of movement, whereas underactivity of the inhibitory, indirect
loop causes excess motor activity in Huntington’s disease (hyperkinesia). These two
pathomechanisms are also considered to be effective in the dysarthrias accompanying
these disorders: Parkinson’s dysarthria is mainly characterized by hypophonia (i.e., a
soft, breathy, monotonous voice caused by hypokinesia of the respiratory-laryngeal
system) and by imprecise or unintelligible articulation as a result of hypokinetic articu-
lator movements. Huntington’s dysarthria, on the contrary, presents with uncontrolled
respiratory, laryngeal, and articulatory activity, resulting in highly dysfluent speech with
sudden inspirations, irregular alterations of loudness and pitch, unwanted articulatory
noises, and irregular over-or undershooting of articulations (for a more comprehensive
description of the two syndromes, see Duffy, 2013).
Other speech syndromes have also been associated with basal ganglia pathology, for
example, the dystonias (e.g., spasmodic dysphonia; Simonyan, Berman, Herscovitch, &
Hallett, 2013) or vocal tics (McCairn et al., 2016).
Movement Gain Hypothesis of Parkinson’s Dysarthria

In light of the gain control hypothesis of basal ganglia functions mentioned earlier, the
paucity of respiratory, laryngeal, and vocal tract movements in Parkinson’s dysarthria
would be explainable as a problem of scaling the speed and size of these movements
according to intrinsic motivation or external task requirements, that is, as a deficit in
applying the appropriate “movement vigor” (Turner & Desmurget, 2010). Recall that
the basal ganglia motor loop is strongly modulated by limbic cortical input via the ven-
tral striatum, hence hypokinetic dysarthria in PD may accrue from a weakening of the
link between motivation and movement gain (Ackermann et al., 2014). This model can
explain several characteristics of Parkinson’s dysarthria, for instance its strong depen
dence on situational parameters and its malleability through explicit instructions (Ho,
Iansek, & Bradshaw, 1999), through motivational reinforcement (Ramig et al., 2001),
or through strong external cues such as regularly structured rhythmical input (Späth
et al., 2016).
Apart from Parkinsonism, the movement gain hypothesis of basal ganglia dysfunc-
tion and the interactions between the motor and the limbic striatal loops may also yield
an explanation of inappropriate vocal behaviors and vocal tics, as they may occur, for
instance, in Tourette syndrome. McCairn et al. (2016) ascribed these symptoms to a dys-
function of the parallel motor and limbic circuits of the basal ganglia, leading to a re-
lease of overshooting motor activity. This is consistent with observations of abnormal
activation of paralimbic areas (e.g., anterior cingulate cortex) immediately preceding
the onset of vocal tics (Bohlhalter et al., 2006). Hence, these data corroborate the view
of the basal ganglia as an interface mediating human speech motor functions with the
limbic vocalization system, as proposed by Ackermann et al. (2014).
Basal Ganglia Dysfunction and the Learning and Retention

of Speech Motor Skills
According to available evidence, the basal ganglia are implicated in the procedural
learning of motor skills, especially in motor sequence learning, but not in the long-term
retention and recall of the acquired skills (Doyon & Benali, 2005; Turner & Desmurget,
2010). Considering speech as a sequential motor activity that is acquired throughout the
first decade of life, the cortico-striatal-thalamic motor loop must play a major role in
speech motor acquisition during childhood. Indirect evidence for this role was recently
obtained in a morphometric study of bilinguals who acquired their second language
either from birth or later during infancy (>5 years). The data pointed at significant gray
matter changes in the left putamen as a function of the time of inception of learning
and of articulatory proficiency in adulthood (Berken, Gracco, Chen, & Klein, 2015). This
model is compatible with clinical findings that early basal ganglia pathology interferes
massively with speech motor acquisition. As a matter of fact, early striatal lesions or ge-
netic conditions interfering with the development of striatal nuclei have a devastating
impact on speech acquisition in infants, as is known from case studies of patients with
developmental apraxia of speech subsequent to FOXP2 mutations (Vargha-Khadem,
Gadian, Copp, & Mishkin, 2005). This syndrome does not occur after basal ganglia
lesions affecting the adult brain, suggesting that the speech motor functions acquired
during childhood are not permanently stored in the deep nuclei (see the following sec-
tion). Hence, in adult speech, the striatal motor circuit is not involved in the planning
or programming of articulations. As mentioned earlier, the basal ganglia circuit of the

adult brain is rather implicated in the allocation of the amount of “movement vigor”
corresponding with the speaker’s emotional and motivational state and her or his per-
ception of the environment. As elucidated in Ackermann et al. (2014) and in Ziegler &
Ackermann (2017), the contribution of the basal ganglia obviously undergoes substan-
tial changes during the maturation of speech motor skills.
Left Posterior Inferior Frontal Cortex

and Apraxia of Speech
According to a long tradition, the posterior part of the inferior frontal gyrus in the left
hemisphere is viewed as a site where higher-order speech motor information is stored.
However, there is a lasting controversy about exactly which regions of left frontal cortex
are implied: Broca’s area (pars opercularis), caudally neighboring premotor cortex, pri-
mary (rolandic) motor cortex, or anterior insular cortex (Graff-Radford et al., 2014;
Richardson, Fillmore, Rorden, Lapointe, & Fridriksson, 2012). A contribution of one or
more of these regions to motor speech has primarily been inferred from lesion studies
of patients with articulation disorders, but there is an increasing number of functional
imaging studies pointing at the left anterior perisylvian cortex as a core region of speech
motor planning (e.g., Bohland, Bullock, & Guenther, 2010; Riecker, Brendel, Ziegler,
Erb, & Ackermann, 2008). Although it is usually the entire speech motor network in
the brain that is activated in these studies, the posterior inferior frontal region of the left
hemisphere is deemed to dominate the motor output system via efferent connections
with, inter alia, the neighboring ventral premotor region, the cerebellum, and the stri-
atum (Eickhoff, Heim, Zilles, & Amunts, 2009).
The motor information traded in left inferior frontal cortex and/or ventral premotor
cortex is often termed speech motor plans or programs, or phonetic plans. In the light
of what was said earlier about striatal motor learning capacities, one may hypothesize
that the speech-specific motor information acquired during childhood through striatal
functions is ultimately retained in these cortical areas (Ackermann et al., 2014; Ziegler
& Ackermann, 2017), either in the form of holistic motor plans for syllables or other
speech units, or as acquired implicit knowledge about how articulatory gestures are
typically combined in a speaker’s native language (Ziegler, Aichert, & Staiger, 2012).
Of note, the anterior speech motor region in the left brain is at the front end of a re-
current network connecting, via subcortical fiber tracts, the auditory association
areas in superior temporal cortex with the frontal speech motor areas, with a relay
area at the parieto-temporal junction. This connectivity has been termed the auditory
dorsal pathway (e.g., Hickok & Poeppel, 2004; Rauschecker, 2011; see Poeppel, Cogan,
Davidesco, & Flinker, Chapter 26 in this volume). Without going into the details of this
rapidly expanding theory, a subsegment of the dorsal pathway, connecting the superior
temporal gyrus to the premotor cortex via the arcuate fascicle and the superior longitu-
dinal fascicle, is supposed to map auditory onto motor representations and thereby con-
tribute to phonological processing in the production and perception of speech (Amunts
& Zilles, 2012; Murakami, Kell, Restle, Ugawa, & Ziemann, 2015). On this account, the au-
ditory dorsal pathway can be considered as the anatomical substrate of an internal model
subserving motor speech through forward and inverse mappings of sensory and motor
representations (e.g., Hickok, 2014; see Hickok, Chapter 20 in this volume). Within this
structure, the left parieto-temporal zone is viewed as a region where sensorimotor speech
information can be buffered through internal rehearsal (Buchsbaum & D’Esposito, 2008).
Clinical Aspects
Patients suffering from lesions to the left anterior peri-and sub-sylvian regions
mentioned in the preceding may suffer from a speech motor impairment termed
apraxia of speech (AOS). This syndrome is predominantly characterized by articulation
problems, with no or only minor phonatory and respiratory involvement. The patients
produce mispronounced (“phonetically distorted”) consonants and vowels, but also
phoneme errors sounding well articulated (e.g., shrosh for frosch, German “frog”). Their
speech is effortful and dysfluent due to frequent occurrences of articulatory groping,
false starts, and re-starts. The occurrence of speech errors and dysfluencies is influenced
by a number of factors, such as syllable complexity, syllabic length, syllable frequency, or
metrical structure (Aichert, Späth, & Ziegler, 2016; Staiger & Ziegler, 2008) (for detailed
descriptions of the clinical pattern of AOS, see Ziegler, 2008).
AOS is distinguished from the dysarthrias on several grounds. One specific property
of AOS is that it is a motor speech impairment that is strongly lateralized to the dom-
inant hemisphere. As mentioned earlier, this is not the case in the dysarthrias, which
may result from lesions to either hemisphere and are mostly mild and transient after
one-sided lesions. Another point is that the speech pattern of apraxic speakers cannot
be explained by any of the pathomechanisms underlying known neurologic movement
disorders (i.e., paresis, ataxia, hypokinesia, tremor, or the like). A sign that is particu-
larly incompatible with these “elementary” motor dysfunctions is the high variability
of apraxic errors and the occurrence of error-free productions, even in patients who are
moderately impaired (Staiger, Finger-Berg, Aichert, & Ziegler, 2012).
The term apraxia dates back originally to Liepmann, who termed the speech im-
pairment of a patient suffering from limb apraxia as an “apraxia of the language mus-
cles” (Liepmann, 1900, p. 129) because the patient’s speaking behavior and error pattern
reminded him of limb apraxia. Since Broca’s first description of a case of “speechless-
ness” (aphémie), the presumed nature of the motor impairment underlying AOS has
been characterized variably as a “loss of movement memories,” a “motor programming
disorder,” “phonetic disintegration,” a dissolution of “coordinative structures,” or a de-
struction of “phonetic/motor plans”—depending on the actual psycholinguistic and
motor theories of each time (Ziegler et al., 2012). Yet, there has always been a broad con-
sent about the superordinate, higher-order character of the speech impairment resulting
from corruption of left inferior-frontal motor mechanisms.
Apraxia of Speech versus Phonological Impairment

From a clinical viewpoint, there is often a problem of discriminating AOS from a speech
sound disorder termed phonological impairment, which occurs in patients with various
types of aphasia (Butterworth, 1992; Schwartz, 2014). The main symptom of this impair-
ment is phonemic paraphasia, that is, the occurrence of phoneme substitutions, deletions,
additions, or metatheses (e.g., film → flim). Unlike in AOS, where such errors can also be
observed, the phonemic paraphasias of aphasic patients are typically well articulated, and
the afflicted patients usually speak more fluently, with no articulatory groping, and with
less effort. Problems of differentiating between the two syndromes arise from the fact that
their symptoms overlap; that is, AOS patients, like patients with phonological impair-
ment, produce errors that are perceived as well articulated, and patients with phonolog-
ical impairment may at times be considerably dysfluent, similar to AOS patients, due to
their “conduite d’approche” attempts and self-corrections. Moreover, the occurrence of
errors is modulated by similar factors in both syndromes (i.e., word length, syllable com-
plexity, and syllable frequency) (Galluzzi, Bureca, Guariglia, & Romani, 2015).
The two types of sound production impairment have in common that they result from
lesions along the left auditory dorsal pathway: While AOS is caused by lesions to the
frontal target areas and the anterior segment of the arcuate and superior longitudinal
fascicles (Fridriksson, Guo, Fillmore, Holland, & Rorden, 2013), phonological impair-
ment is associated with damage to the more posterior parts (i.e., the posterior superior
temporal and inferior parietal lobes, the parieto-temporal region, or their connecting
fiber bundles) (Schwartz, Faseyitan, Kim, & Coslett, 2012). In the cognitive neurolin-
guistic tradition, phonological impairment is subdivided into a lexical and a post-lexical
subtype and is ascribed to a processing level where phonological form information is
retrieved, traditionally with the assumption that the underlying representations and
mechanisms are abstract and symbolic (Goldrick & Rapp, 2007; see Rapp & Purcell,
Chapter 17 in this volume). Considering the functional topography of the auditory
dorsal pathway, however, one would speculate that the mechanisms implied in pho-
nological impairment deal with auditory-motor integration processes rather than
with amodal symbols—a view supporting substance-based rather than abstractionist
approaches to phonology (Buchsbaum et al., 2011; Hickok, 2012). From this perspective,
the syndromes of the left auditory dorsal stream can be considered as a clinical model of
the phonetics-phonology interface (for references and a discussion, see Ziegler, 2016).
Speaking—A Task-Specific
Motor Action?
A controversial theoretical issue is about where the motor speech disorders find their
place between movement and language. This question hooks up with discussions re-
garding the specific tuning of the vocal tract and respiratory-laryngeal motor systems
for the production of spoken language. Broadly speaking, two opposing views are
discussed:
1. Speaking is grounded in a universal, domain-general motor proficiency of the

movement organs implied in speaking. It engages the same neural mechanisms
as any other “non-speech” or “speech-like” vocal tract motor task, such as isolated
tongue protrusion, visuomotor tracking by the jaw or tongue, sustained vowel pro-
duction, maximally rapid syllable repetitions, and so forth. All these tasks involve
individual combinations of universal motor skills based, for instance, on timing, co-
ordination, force generation, and so forth. Damage to the neural network subserving
these properties will therefore compromise all motor activities of the vocal organs.
As a corollary, clinical assessment and treatment of motor speech disorders need not
necessarily be based on speaking, but can principally also employ non-speech tasks
(Ballard, Granier, & Robin, 2000; Ballard, Robin, & Folkins, 2003).
2. Motor speech relies on a highly specialized neural organization of the movements
implied in speaking. The domain-specific organization of the motor act of speaking
results from extensive motor exercise during the long period of speech acquisition,
which—like all other types of motor learning—has an impact on the structural and
functional organization of the underlying neural network. Motor skill learning is
known to go hand in hand with organization processes at all levels of the brain’s
motor network: cortical and subcortical, gray and white matter, structural, topo-
graphic, as well as synaptic (Adkins, Boychuk, Remple, & Kleim, 2006; Doyon
& Benali, 2005), suggesting that after more than a decade of infant speech motor
learning (Kent & Vorperian, 1995), the neural network engaged in speech motor
control must be highly specialized. Another unique aspect of speaking is that its
motor organization is inseparably entrenched with auditory perception, both
in acquisition and in the mature system (Hickok, 2014), which is not the case for
chewing, isolated lip pursing, visuomotor tracking, and so forth. As a consequence,
brain lesions may impair speech and leave other motor functions largely intact, or
vice versa. Therefore, clinical assessment and treatment approaches based on non-
speech oral motor tasks are inappropriate (Ziegler & Ackermann, 2013).
In the following sections, we will revisit the pathomechanisms of motor speech disorders
from the perspective of whether and how they interfere with speech-specific aspects
of oral motor control and will provide explanations for the clinical findings of disso-
ciated speech and non-speech oral motor impairments (Staiger, Schölderle, Brendel &
Ziegler, 2017).
Upper Motor Neuron Dysfunction

Primary motor cortex is among the most extensively examined sites of practice-
related neural plasticity (e.g., Doyon & Benali, 2005; Monfils, Plautz, & Kleim, 2005).
According to Adkins et al. (2006), plasticity mechanisms also include the descending
fiber tracts projecting onto the motor nuclei. Of particular concern with regard to
speaking are investigations of plasticity mechanisms associated with motor learning
initiated early in childhood and extending over long periods, as in some professional
musicians and athletes. These studies have consistently revealed substantial practice-
related modifications of cortical motor maps (Gaser & Schlaug, 2003) and descending
fiber tracts (Imfeld, Oechslin, Meyer, Loenneker, & Jancke, 2009). Though the neural
plasticity mechanisms associated particularly with speech motor learning during in-
fancy are not accessible to direct investigations, electrocortigraphic (ECoG) recordings
of neuronal activity during speaking reported by Bouchard et al. (2013) revealed
syllable-specific patterns that can only be explained as a result of learning-induced cor-
tical plasticity. On the basis of this evidence, one would conjecture that especially the
fine motor skill aspects of speaking, relying on the integrity of such motor maps and of
the monosynaptic corticobulbar pathway, are particularly vulnerable to speech-specific
motor impairment (see earlier discussion in this chapter). Symptoms related especially
to these properties may therefore be inaccessible to non-speech assessment and treat-
ment approaches. On the contrary, lesions to the indirect, polysynaptic corticobulbar
fiber tracts, with their concomitant alterations of muscle tone and posture in spastic-
paretic dysarthria, may lead to less specific symptoms because this phylogenetically
older system is probably less malleable through motor plasticity mechanisms. Since
brain lesions most often involve both the direct and the indirect corticobulbar pathways,
speech–non-speech dissociations are often masked.
Cerebellar Dysfunction
Like the primary motor cortex, the cerebellum is also subject to practice-induced plas-
ticity (Doyon, Penhune, & Ungerleider, 2003); hence its structure and function must
also be considered to be shaped in specific ways during speech motor acquisition. The
contribution of the cerebellum to motor learning is known to be particularly distinct
during the fast, early learning phase. During the further course of learning and conso
lidation, its major activation shifts from the cerebellar cortex to the deep cerebellar nu-
clei and further on to other motor centres of the brain (Doyon et al., 2003). This entails
that cerebellar dysfunction will have a differential impact on novel motor tasks, such
as the tracking of a moving cursor through tongue or jaw movements, as opposed to
overlearned motor skills, such as speaking.
Regarding the ataxic pathomechanism resulting from cerebellar lesions, some move-
ment types are, in addition to being unlearned, also known to be particularly vulner-
able to motor dysfunctions. Among these are motor tasks requiring rapid sequential
inversions of agonist and antagonist movements (diadochokinesis), maintenance of
a fixed posture over longer periods, or minute online adaptations of motor output to
a slowly moving target (visuomotor tracking). These tasks call for highly adaptive sen-
sorimotor mechanisms to maintain a static target position, track a moving target, or
optimize strategies for fast repetition, and are therefore particularly prone to cerebellar
pathology (e.g., Miall & Reckess, 2002).
Tasks of these types also play a prominent role in experimental and clinical
assessments of neurological speech disorders: rapid repetitions of sequential or alter-
nating syllables (e.g., “puh-puh-puh,” “puh-tuh-kuh”) constitute a vocal tract variant of
diadochokinesis; sustaining a vowel or fricative for several seconds represents phona-
tory or articulatory variants of motor persistence tasks; and tracking a moving target
on a computer screen by appropriate adaptations of jaw or tongue forces is a variant of
visuomotor tracking. Like in the limbs, these tasks are particularly sensitive to cerebellar
involvement: for instance, patients with cerebellar pathology were found to be dis-
proportionately slow in oral diadochokinesis relative to speaking (Brendel et al., 2015;
Ziegler & Wessel, 1996). Furthermore, cerebellar patients are often highly conspicuous
for irregular pitch and loudness or voice tremor in sustained vowel production, although
these symptoms are much less remarkable or even absent during speech, where main-
tenance of stable postures is not a critical requirement (Ackermann & Ziegler, 1994).
Finally, patients with cerebellar ataxia were found to demonstrate severe impairment of
visuomotor airflow tracking that was, however, uncorrelated with their dysarthric im-
pairment (Deger, Ziegler, & Wessel, 1999). Obviously, clinical examinations based on
such tasks are not particularly predictive of the severity of cerebellar dysarthria, first be-
cause they are unlearned, and second because the motor organization of speaking does
not rely on the specific requirements of these oral/vocal motor tasks.
Striatal Dysfunction
As mentioned earlier, there is a shift in striatal motor functions from the acquisition
to the mature stage of speech motor control, with a major role in the learning of the
sequential and coordinative aspects of speaking during infancy, and predominantly a
gain-control function in adulthood (Ziegler & Ackermann, 2017). This entails—like in
the cerebellar system—a fundamental difference of striatal motor functions between
tasks with a high proficiency level, such as adult speech, and others that are new and
subject to immediate motor learning mechanisms, such as maximum performance or
visuomotor tracking tasks. There is even evidence that motor tasks relying on habituated
or automated skills, like speaking, and others relying on attentive, goal-directed control,
like visuomotor tracking, are differentially organized within the basal ganglia and may
therefore dissociate in striatal dysfunction (Redgrave et al., 2010). In an experimental
investigation of Parkinson’s patients, Connor and Abbs (1991), for instance, found
marked differences in jaw movement performance for patients with Parkinson’s disease
between a visually guided tracking task, a rapid syllable repetition task, and speaking.
The smallest deviations from normal, in terms of movement cycle durations, occurred
in the speech task, and the largest deviations in the tracking task, which is consistent
with differences in novelty/unfamiliarity between the three experimental settings and
demonstrates that striatal dysfunction has a differential effect on learned versus novel
oral motor tasks.
However, there may also be a more universal aspect of striatal dysfunction in vocal/
oral motor control and speech. As regards the gain control hypothesis of striatal motor
regulation and the influence of the limbic striatal loop, a plausible assumption is that
impairment of this mechanism may indeed have a more universal effect across dif-
ferent motor activities. More specifically, such a dysfunction probably involves non-
verbal emotional facial and vocal expression in the first place (e.g., Parkinsonian
“hypomimia”), and rubs off on speech or other voluntary oral movements only in the
second place. Likewise, motor disinhibition phenomena resulting from excess limbic
discharge across the ventral striatum (vocal and facial tics; cf. McCairn et al., 2016), or
from overactivity of the direct striatal pathway, may also exert unspecific influences on
speech and other vocal and oral motor activities, through a generalized excitation of
motor output channels.
Apraxia of Speech
Assuming, as suggested earlier, that the left ventral premotor and anterior opercular
region is a site where learned motor routines for speaking are stored, one would ex-
pect that lesions to this area result in speech impairment that is deeply moulded by
the linguistic (i.e., phonological) patterning of words and phrases. This is indeed the
case: Numerous factors are known to influence the occurrence of speech errors in
apraxia of speech that are related to phonological and statistical aspects of speech, for
instance syllable frequency and syllable structure (Aichert & Ziegler, 2004; Staiger &
Ziegler, 2008), or metrical rhythm (Aichert et al., 2016). Studies investigating the
relearning of syllables and words in apraxic speakers have revealed that learning and
transfer in these patients is strongly constrained by phonological structure (e.g., syllable
constituency) (Schoor, Aichert, & Ziegler, 2012), which is compatible with findings that
speech motor learning in healthy populations is extremely local and sensitive to phono-
logical context (Tremblay, Houle, & Ostry, 2008). We have recently proposed a model of
speech motor planning based on data from apraxic speakers that explains the phonetic
planning demands for spoken words by a hierarchical architecture, expanding from ar-
ticulatory gestures over syllable constituents to metrical structures (Ziegler & Aichert,
2015). Such a design cannot plausibly be based on domain-general motor mechanisms,
but must necessarily result from an implicit, statistical learning of the motor routines
associated with the phonological forms that are pervasive in a speaker’s language. The
view of apraxia of speech as an impairment of the acquired motor capacities involved in
speaking is compatible with clinical findings pointing at double dissociations between
apraxia of speech, on the one hand, and apraxia of non-speech movements (“oral-facial
apraxia”), on the other (cf. Bonilha, Moser, Rorden, Baylis, & Fridriksson, 2006; for
references see Ziegler, 2003).
Conclusions
Motor speech disorders result from dysfunctions of the neural network engaged in the con-
trol of speech movements, that is, a descending corticobulbar pathway supplemented by a
cerebellar and a striatal motor loop and dominated by a left-lateralized inferior-frontal and
ventral premotor planning center connected with auditory speech areas in the left temporal
lobe. The voluntary motor system engaged in speaking is modulated by motivational and
affective input from mesiofrontal cortex and limbic brain regions. The symptoms observed
in the different motor speech disorders reflect the motor control mechanisms of the partic-
ular brain sites involved in different neurologic conditions, but they also express compensa-
tory mechanisms evolving in reaction to the respective impairments.
Speech impairment caused by neurologic dysfunction should be understood as a
condition affecting motor functions that are specifically dedicated to verbal communi-
cation. This view is substantiated predominantly by evidence from studies of learning-
related neural plasticity. Yet, there may also be domain-general pathomechanisms
affecting speech motor control in dysarthria, such as generalized alterations of tone and
posture in spasticity or dysregulation of limbic activation in basal ganglia disorders.
Many of these points have not yet been substantiated by empirical evidence and still wait
for clinical approval.
References
Ackermann, H., Hage, S. R., & Ziegler, W. (2014). Brain mechanisms of acoustic communi-
cation in humans and nonhuman primates: An evolutionary perspective. Behavioral and
Brain Sciences, 37, 529–604.
Ackermann, H., & Ziegler, W. (1994). Acoustic analysis of vocal instability in cerebellar dys-
function. Annals of Otology, Rhinology, and Laryngology, 103, 98–104.
Adkins, D. L., Boychuk, J., Remple, M. S., & Kleim, J. A. (2006). Motor training induces
experience-specific patterns of platicity across motor cortex and spinal cord. Journal of
Applied Physiology, 101, 1776–1782.
Aichert, I., Späth, M., & Ziegler, W. (2016). The role of metrical information in apraxia of
speech: Perceptual and acoustic analyses of word stress. Neuropsychologia, 82, 171–178.
Aichert, I., & Ziegler, W. (2004). Syllable frequency and syllable structure in apraxia of speech.
Amunts, K., & Zilles, K. (2012). Architecture and organizational principles of Broca’s region.
Ballard, K. J., Granier, J. P., & Robin, D. A. (2000). Understanding the nature of apraxia of
speech: Theory, analysis, and treatment. Aphasiology, 14(10), 969–995.
Ballard, K. J., Robin, D. A., & Folkins, J. W. (2003). An integrative model of speech motor con-
trol: A response to Ziegler. Aphasiology, 17(1),  37–48.
Berken, J. A., Gracco, V. L., Chen, J.-K., & Klein, D. (2015). The timing of language learning
shapes brain structure associated with articulation. Brain Structure and Function, 221(7),
3591–3600.
Bohland, J. W., Bullock, D., & Guenther, F. H. (2010). Neural representations and mechanisms
for the performance of simple speech sequences. Journal of Cognitive Neuroscience, 22(7),
1504–1529.
Bohlhalter, S., Goldfine, A., Matteson, S., Garraux, G., Hanakawa, T., Kansaku, K., . . . Hallett,
M. (2006). Neural correlates of tic generation in Tourette syndrome: An event-related func-
tional MRI study. Brain, 129(8), 2029–2037.
Bonilha, L., Moser, D., Rorden, C., Baylis, G. C., & Fridriksson, J. (2006). Speech apraxia
without oral apraxia: Can normal brain function explain the physiopathology? NeuroReport,
17(10), 1027–1031.
Bouchard, K. E., Mesgarani, N., Johnson, K., & Chang, E. F. (2013). Functional organization of
human sensorimotor cortex for speech articulation. Nature, 495(7441), 327–332.
Brendel, B., Ackermann, H., Berg, D., Lindig, T., Schölderle, T., Schöls, L., . . . Ziegler, W. (2013).
Friedreich ataxia: Dysarthria profile and clinical data. Cerebellum, 12, 475–484.
Brendel, B., Synofzik, M., Ackermann, H., Lindig, T., Schölderle, T., Schöls, L., & Ziegler, W.
(2015). Comparing speech characteristics in spinocerebellar ataxias type 3 and type 6 with
Friedreich ataxia. Journal of Neurology, 262(1),  21–26.
Brown, S., Laird, A. R., Pfordresher, P. Q., Thelen, S. M., Turkeltaub, P., & Liotti, M. (2009). The
somatotopy of speech: Phonation and articulation in the human motor cortex. Brain and
Cognition, 70(1),  31–41.
Buchsbaum, B. R., Baldo, J., Okada, K., Berman, K. F., Dronkers, N., D’Esposito, M., &
Hickok, G. (2011). Conduction aphasia, sensory-motor integration, and phonological
short-term memory: An aggregate analysis of lesion and fMRI data. Brain and Language,
119(3), 119–128.
Buchsbaum, B. R., & D’Esposito, M. (2008). The search for the phonological store: From loop
to convolution. Journal of Cognitive Neuroscience, 20(5), 762–778.
Butterworth, B. (1992). Disorders of phonological encoding. Cognition, 42, 261–286.
Connor, N. P., & Abbs, J. H. (1991). Task-dependent variations in Parkinsonian motor
impairments. Brain, 114, 321–332.
Deger, K., Ziegler, W., & Wessel, K. (1999). Airflow tracking in patients with ataxic disorders.
Clinical Linguistics and Phonetics, 13, 433–447.
Doyon, J., & Benali, H. (2005). Reorganization and plasticity in the adult brain during learning
of motor skills. Current Opinion in Neurobiology, 15(2), 161–167.
Doyon, J., Penhune, V., & Ungerleider, L. G. (2003). Distinct contribution of the cortico-striatal
and cortico-cerebellar systems to motor skill learning. Neuropsychologia, 41(3), 252–262.
Duffy, J. R. (2013). Motor speech disorders: Substrates, differential diagnosis, and management
(3rd ed.). St. Louis, MO: Elsevier Mosby.
Eickhoff, S. B., Heim, S., Zilles, K., & Amunts, K. (2009). A systems perspective on the effective
connectivity of overt speech production. Philosophical Transactions of the Royal Society A,
367, 2399–2421.
Fridriksson, J., Guo, D., Fillmore, P., Holland, A., & Rorden, C. (2013). Damage to the ante-
rior arcuate fasciculus predicts non-fluent speech production in aphasia. Brain, 136(11),
3451–3460.
Galluzzi, C., Bureca, I., Guariglia, C., & Romani, C. (2015). Phonological simplifications,
apraxia of speech and the interaction between phonological and phonetic processing.
Neuropsychologia, 71,  64–83.
Gaser, C., & Schlaug, G. (2003). Brain structures differ between musicians and non-musicians.
Goldrick, M., & Rapp, B. (2007). Lexical and post-lexical phonological representations in
spoken production. Cognition, 102, 219–260.
Golfinopoulos, E., Tourville, J. A., Bohland, J. W., Ghosh, S. S., Nieto-Castanon, A., & Guenther,
F. H. (2011). fMRI investigation of unexpected somatosensory feedback perturbation during
speech. NeuroImage, 55(3), 1324–1338.
Graff-Radford, J., Jones, D. T., Strand, E. A., Rabinstein, A. A., Duffy, J. R., & Josephs, K. A.
(2014). The neuroanatomy of pure apraxia of speech in stroke. Brain and Language,
129,  43–46.
Graybiel, A. M. (2000). The basal ganglia. Current Biology, 10(14), R509–R511.
Hickok, G. (2014). Towards an integrated psycholinguistic, neurolinguistic, sensorimotor
framework for speech production. Language, Cognition and Neuroscience, 29(1),  52–59.
aspects of the functional anatomy of language. Cognition, 92(1–2),  67–99.
Ho, A. K., Iansek, R., & Bradshaw, J. L. (1999). Regulation of parkinsonian speech volume: The ef-
fect of interlocutor distance. Journal of Neurology, Neurosurgery, and Psychiatry, 67, 199–202.
Holmes, G. (1917). The symptoms of acute cerebellar injuries due to gunshot injuries. Brain, 40,
461–535.
Hopf, H. C., Müller-Forell, W., & Hopf, N. J. (1992). Localization of emotional and volitional
facial paresis. Neurology, 42, 1918–1923.
Imfeld, A., Oechslin, M. S., Meyer, M., Loenneker, T., & Jancke, L. (2009). White matter plas-
ticity in the corticospinal tract of musicians: A diffusion tensor imaging study. NeuroImage,
46(3), 600–607.
Jürgens, U. (2002). Neural pathways underlying vocal control. Neuroscience and Biobehavioral
Reviews, 26(2), 235–258.
Kent, R. D. & Vorperian, H. (1995). Development of the craniofacial-oral-laryngeal anatomy: A
review. Journal of Medical Speech-Language Pathology, 3, 145–190.
Koziol, L. F., Budding, D., Andreasen, N., Arrigo, S., Bulgheroni, S., Imamizu, H., . . . Parker,
K. (2014). Consensus paper: The cerebellum’s role in movement and cognition. Cerebellum,
13(1), 151–177.
Liégeois, F. J., Butler, J., Morgan, A. T., Clayden, J. D., & Clark, C. A. (2015). Anatomy and lat-
eralization of the human corticobulbar tracts: An fMRI-guided tractography study. Brain
Liepmann, H. (1900). Das Krankheitsbild der Apraxie (“motorischen Asymbolie”) auf Grund
eines Falles von einseitiger Apraxie (II). Monatsschrift für Psychiatrie und Neurologie,
VIII(1), 102–132.
Manto, M., Bower, J. M., Conforto, A. B., Delgado-Garcìa, J. M., Farias da Guarda, S. N.,
Gerwig, M., . . . Timmann, D. (2012). Consensus paper: Roles of the cerebellum in motor
control: The diversity of ideas on cerebellar involvement in movement. Cerebellum, 11,
457–487.
Mariën, P., Ackermann, H., Adamaszek, M., Barwood, C. H. S., Beaton, A., Desmond,
J., . . . Küper, M. (2014). Consensus Paper: Language and the cerebellum: An ongoing
enigma. Cerebellum, 13(3), 386–410.
Mariën, P., Pickut, B. A., Engelborghs, S., Martin, J.-J., & De Deyn, P. P. (2001). Phonological
agraphia following a focal anterior insulo-opercular infarction. Neuropsychologia, 39(8),
845–855.
Mariën, P., Verhoeven, J., Engelborghs, S., Rooker, S., Pickut, B. A., & De Deyn, P. P. (2006). A
role for the cerebellum in motor speech planning: Evidence from foreign accent syndrome.
Clinical Neurology and Neurosurgery, 108(5), 518–522.
Matsuzaka, Y., Picard, N., & Strick, P. L. (2007). Skill representation in the primary motor
cortex after long-term practice. Journal of Neurophysiology, 97(2), 1819–1832.
McCairn, K. W., Nagai, Y., Hori, Y., Ninomiya, T., Kikuchi, E., Lee, J.-Y., . . . Takada, M. (2016).
A primary role for nucleus accumbens and related limbic network in vocal tics. Neuron,
89(2), 300–307.
Miall, R. C., & Reckess, G. Z. (2002). The cerebellum and the timing of coordinated eye and
hand tracking. Brain and Cognition, 48, 212–226.
Middleton, F. A., & Strick, P. L. (2000). Basal ganglia and cerebellar loops: Motor and cognitive
circuits. Brain Research Reviews, 31(2–3), 236–250.
Monfils, M. H., Plautz, E. J., & Kleim, J. A. (2005). In search of the motor engram: Motor map
plasticity as a mechanism for encoding motor experience. Neuroscientist, 11(5), 471–483.
Murakami, T., Kell, C. A., Restle, J., Ugawa, Y., & Ziemann, U. (2015). Left dorsal speech stream
components and their contribution to phonological processing. Journal of Neuroscience,
35(4), 1411–1422.
Nowak, D. A., Griebl, G., Dabitz, R., & Ochs, G. (2010). Bilateral anterior opercular (Foix–
Chavany–Marie) syndrome. Journal of Clinical Neuroscience, 17(11), 1441–1442.
Ramig, L. O., Sapir, S., Countryman, S., Pawlas, A. A., O’Brien, C., Hoehn, M., & Thompson, L.
L. (2001). Intensive voice treatment (LSVT®) for patients with Parkinson’s disease: A 2 year
follow up. Journal of Neurology, Neurosurgery & Psychiatry, 71(4), 493–498.
Rauschecker, J. P. (2011). An expanded role for the dorsal auditory pathway in sensorimotor
control and integration. Hearing Research, 271(1),  16–25.
Redgrave, P., Rodriguez, M., Smith, Y., Rodriguez-Oroz, M. C., Lehericy, S., Bergman,
H., . . . Obeso, J. A. (2010). Goal- directed and habitual control in the basal gan-
glia: Implications for Parkinson’s disease. Nature Reviews Neuroscience, 11(11), 760–772.
Richardson, J. D., Fillmore, P., Rorden, C., Lapointe, L. L., & Fridriksson, J. (2012). Re-
establishing Broca’s initial findings. Brain and Language, 123(2), 125–130.
Riecker, A., Brendel, B., Ziegler, W., Erb, M., & Ackermann, H. (2008). The influence of syllable
onset complexity and syllable frequency on speech motor control. Brain and Language, 107,
102–113.
Schiff, H. B., Alexander, M. P., Naeser, M. A., & Galaburda, A. M. (1983). Aphemia. Clinical-
anatomic correlations. Archives of Neurology, 40, 720–727.
Schoch, B., Dimitrova, A., Gizewski, E. R., & Timmann, D. (2006). Functional localization
in the human cerebellum based on voxelwise statistical analysis: A study of 90 patients.
NeuroImage, 30(1),  36–51.
Schölderle, T., Staiger, A., Schumacher, B., & Ziegler, W. (in press). The impact of dysar-
thria on laypersons’ attitudes towards adults with cerebral palsy. Folia Phoniatrica et
Logopaedica.
Schoor, A., Aichert, I., & Ziegler, W. (2012). A motor learning perspective on phonetic syllable
kinships: How training effects transfer from learned to new syllables in severe apraxia of
speech. Aphasiology, 26(7), 880–894.
Schwartz, M. F. (2014). Theoretical analysis of word production deficits in adult aphasia.
Philosophical Transactions of the Royal Society B: Biological Sciences, 369(1634), 20120390.
Schwartz, M. F., Faseyitan, O., Kim, J., & Coslett, H. B. (2012). The dorsal stream contribution
to phonological retrieval in object naming. Brain, 135(12), 3799–3814.
Segawa, J. A., Tourville, J. A., Beal, D. S., & Guenther, F. H. (2014). The neural correlates of
speech motor sequence learning. Journal of Cognitive Neurosciences, 27, 819–831
Sheean, G. (2002). The pathophysiology of spasticity. European Journal of Neurology, 9(Suppl
1),  3–9.
Simonyan, K., Berman, B. D., Herscovitch, P., & Hallett, M. (2013). Abnormal striatal dopamin-
ergic neurotransmission during rest and task production in spasmodic dysphonia. Journal
of Neuroscience, 33(37), 14705–14714.
Späth, M., Aichert, I., Ceballos-Baumann, A., Wagner-Sonntag, E., Miller, N., & Ziegler, W.
(2016). Entraining with another person’s speech rhythm: Evidence from healthy speakers
and individuals with Parkinson’s disease. Clinical Linguistics & Phonetics, 30(1),  68–85.
Staiger, A., Finger-Berg, W., Aichert, I., & Ziegler, W. (2012). Error variability in apraxia of
speech: A matter of controversy. Journal of Speech, Language, and Hearing Research, 55(5),
S1544–S1561.
Staiger, A., Schölderle, T., Brendel, B., & Ziegler, W. (2017). Dissociating oral motor
capabilities: Evidence from patients with movement disorders. Neuropsychologia, 95,  40–53
Staiger, A., & Ziegler, W. (2008). Syllable frequency and syllable structure in the spontaneous
speech production of patients with apraxia of speech. Aphasiology, 22, 1201–1215.
Strominger, N. L., Demarest, R. J., & Laemle, L. B. (2012). Noback’s human nervous
system: Structure and function. Totowa, NJ: Humana Press.
Tanji, K., Suzuki, K., Yamadori, A., Tabuchi, M., Endo, K., Fujii, T., & Itoyama, Y. (2001). Pure
anarthria with predominantly sequencing errors in phoneme articulation: A case report.
Cortex, 37(5), 671–678.
Tourville, J. A., Peeva, M. G., & Guenther, F. H. (2014). Perception-production interactions and
their neural bases. In M. Goldrick, V. Ferreira, & M. Miozzo (Eds.), The Oxford handbook of
language production (pp. 460–481). Oxford: Oxford University Press.
Tremblay, S., Houle, G., & Ostry, D. J. (2008). Specificity of speech motor learning. Journal of
Neuroscience, 28(10), 2426–2434.
Turner, R. S., & Desmurget, M. (2010). Basal ganglia contributions to motor control: A vig-
orous tutor. Current Opinion in Neurobiology, 20(6), 704–7 16.
Urban, P. P., Marx, J., Hunsche, S., Gawehn, J., Vucurevic, G., Wicht, S., . . . Hopf, H. C. (2003).
Cerebellar speech representation: Lesion topography in dysarthria as derived from cer-
ebellar ischemia and functional magnetic resonance imaging. Archives of Neurology, 60,
965–972.
Urban, P. P., Rolke, R., Wicht, S., Keilmann, A., Stoeter, P., Hopf, H. C., & Dieterich, M. (2006).
Left-hemispheric dominance for articulation: A prospective study on acute ischaemic dys-
arthria at different localizations. Brain, 129, 767–777.
Vargha-Khadem, F., Gadian, D. G., Copp, A., & Mishkin, M. (2005). FOXP2 and the neuroa-
natomy of speech and language. Nature Reviews Neuroscience, 6(2), 131–138.
Wildgruber, D., Ackermann, H., Klose, U., Kardatzki, B., & Grodd, W. (1996). Functional lat-
eralization of speech production at primary motor cortex: A fMRI study. NeuroReport, 7,
2791–2795.
Wolpert, D. M., Miall, R. C., & Kawato, M. (1998). Internal models in the cerebellum. Trends in
Xu, J., Haith, A. M., & Krakauer, J. W. (2015). Motor control of the hand before and after stroke.
In K. Kansaku, L. G. Cohen, & N. Birbaumer (Eds.), Clinical systems neuroscience (pp. 271–
289). Tokyo: Springer.
Ziegler, W. (2003). Speech motor control is task-specific: Evidence from dysarthria and apraxia
of speech. Aphasiology, 17(1),  3–36.
Ziegler, W. (2008). Apraxia of speech. In G. Goldenberg & B. Miller (Eds.), Handbook of clinical
neurology (Vol. 88, pp. 269–285). London: Elsevier.
Ziegler, W. (2015). The phonetic cerebellum: Cerebellar involvement in speech sound pro-
duction. In P. Marien & M. Manto (Eds.), The linguistic cerebellum (pp. 1–32). New York:
Academic Press.
Ziegler, W. (2016). Phonology versus phonetics in speech sound disorders. In P. van Lieshout,
B. Maassen, & H. Terband (Eds.), Speech motor control in normal and disordered speech:
Future developments in theory and methodology (pp. 233–256). Rockville, MD: American
Speech Language and Hearing Association.
Ziegler, W., & Ackermann, H. (2013). Neuromotor speech impairment: It’s all in the talking.
Folia Phoniatrica et Logopaedica, 65(2),  55–67.
Ziegler, W., & Ackermann, H. (2017). Subcortical contributions to motor speech: Phylogenetic,
developmental, clinical. Trends in Neurosciences, 40, 458–468.
Ziegler, W., & Aichert, I. (2015). How much is a word? Predicting ease of articulation planning
from apraxic speech error patterns. Cortex, 69,  24–39.
Ziegler, W., Aichert, I., & Staiger, A. (2012). Apraxia of speech: Concepts and controversies.
Journal of Speech, Language, and Hearing Research, 55, S1485–S1501.
Ziegler, W., & Wessel, K. (1996). Speech timing in ataxic disorders: Sentence production and
rapid repetitive articulation. Neurology, 47, 208–214.
Chapter 19
I nvestigat i ng t h e
Spatial and T e mp ora l
C omp onents of Spe e c h
Produ c t i on
Greig I. de Zubicaray and Vitória Piai
Introduction
To date, two major behavioral approaches have been utilized in speech-production re-
search. The first involves characterizing speech errors or dysfluencies produced either
spontaneously, experimentally, and/or as a consequence of brain injury (i.e., aphasia).
The second entails experimental investigations of the time course of speech produc-
tion in different contexts by means of response-time analyses. These psycholinguistic
approaches have been complemented recently by a third, involving the application of
electrophysiological, neuroimaging, and brain stimulation technologies, following
from the realization that we require a better understanding of the neural mechanisms
underlying the behavioral data. Theoretical accounts of speech production have also
begun to incorporate neural mechanisms in their feature set, although most remain lim-
ited to explaining specific effects observed during performance of particular paradigms.
However, application of these technologies involves surmounting methodological
challenges not typically encountered in studies of language comprehension, largely due
to speech-related artifacts introduced by articulation and breathing.
In this chapter, we provide an overview of relevant spatial and temporal components
of language production. We then review findings from a range of studies investigating
language production in brain-lesion patients and healthy participants, with an emphasis
on evidence from context manipulations in picture-naming paradigms designed to
identify and characterize core processes and components of the network involved in
retrieving words from the mental lexicon. Boxes 19.1–19.4 address the methodological
the Spatial and Temporal Components of Speech Production 473
Box 19.1 Methodological Challenges for Investigations of Speech Production
From a methodological perspective, neurolinguistic investigations of speech production

are inherently more complicated to conduct than those targeting nonlanguage processes
and/or comprehension. Two of the most widely employed technologies, functional mag-
netic resonance imaging (fMRI; see Heim & Specht, Chapter 4 in this volume) and neuro-
physiological recordings via electroencephalography and magnetoencephalography (EEG
and MEG, respectively; see, in this volume, Leckey & Federmeier, Chapter 3, and Salmelin,
Kujala, & Liljeström, Chapter 6) are especially sensitive to speech-related signal artifacts
that compromise data quality and confound interpretations. Consequently, the over-
whelming majority of studies using these technologies have examined the production of
single words or short phrases. The nature of these speech-related artifacts and the appro-
priate data acquisition and analysis techniques to ameliorate them are briefly described in
Boxes 19.2 and 19.3.
The earliest, and perhaps most intuitive, approach to dealing with speech production
artifacts in fMRI and electrophysiological studies was to avoid them entirely. In the first
fMRI studies conducted in the 1990s, participants were typically instructed to respond co-
vertly (i.e., use inner speech). Aside from the obvious problem of having no behavioral data
to confirm the presence of a significant psycholinguistic effect, later studies comparing co-
vert and overt speech within the same paradigm revealed engagement of different cerebral
regions. However, the relative contributions of those areas are difficult to interpret (for ex-
ample, greater engagement of regions during overt production might represent syllabifica-
tion and articulatory-motor processes, while covert production might additionally engage
domain general processes such as response inhibition and self-monitoring while reducing
activity in production-related regions; see the review by Indefrey & Levelt, 2004). Thus, it
became clear that avoiding speech-related artifacts by requesting covert responses from
participants was not the optimal strategy.
challenges that need to be surmounted by speech-production researchers in order to

provide valid and reliable data.
Spatiotemporal Components
Relating to Processing Stages
in Speech-P roduction  Models
Based upon speech error and response time data, models of spoken-word produc-
tion have stipulated that the processing stages involved in producing words include
perceptual and conceptual preparation, lexical selection, word-form encoding, and ar-
ticulation. Indefrey and Levelt’s (2000, 2004) influential review and meta-analysis
represented the first serious attempt to relate neuroimaging and electrophysiolog-
ical data to these processing stages. In 2004, the electronencephalography (EEG) and
Box 19.2 Methodological Challenges for Neuroimaging Investigations
In fMRI, the physical act of producing speech introduces blood oxygenation level de-
pendent (BOLD) signal changes during continuous imaging, necessitating a different ap-
proach to data acquisition. These task-correlated, speech-induced signal changes are not a
consequence of neural processes, although they tend to mimic or mask them as they occur
over the perisylvian cortex, and are not limited to the speech act itself—they contaminate
subsequent images acquired in a functional time series. They also do not merely reflect
whole-head rigid body motion, and persist following the application of standard image-
realignment algorithms. However, it is not uncommon to read papers in which authors
claim to have inspected their realigned data and confirmed that it was free from speech re-
lated artifacts. Rather, signal changes from producing even a single word during continuous
imaging reflect movement-by-magnetic-susceptibility interactions. Areas of magnetic-
susceptibility-related signal variation are located in inferior temporal and frontal cortices,
where air-bone-tissue interfaces exist, and are problematic for both structural and func-
tional MRI. Producing speech worsens these susceptibility-related artifacts by increasing
geometric distortions, losing and introducing signal—the latter from regions within and
immediately outside the field of view linked to muscle movement. A relatively reliable
finding is that articulatory muscle movement produces a rapid signal decrease in and
around Broca’s area and its right hemisphere homologue, compromising any task-related
signal changes (see Mehta, Grabowski, Razavi, Eaton, & Bolinger, 2006). Consequently, the
BOLD signal in continuous fMRI acquisitions is unavoidably compromised during speech
production, and the simultaneous gradient noise often impedes recording and measuring
of accuracy and speech-onset latencies. It is therefore essential to adopt well-validated, al-
ternative approaches. As we will note later, some of the inconsistencies in reported fMRI
findings for speech production paradigms can be directly attributed to inappropriate acqui-
sition methods.
In the first decade of the 2000s, a concerted international research program demonstrated
the utility of a different image-acquisition method for speech production fMRI studies, now
broadly referred to as sparse temporal sampling. In brief, the technique involves acquiring
only a single fMRI volume after each utterance, timed to capture the estimated peak BOLD
signal response. As speech is produced during relative silence (i.e., without gradient noise
associated with image acquisition), both accuracy and latency are able to be measured re-
liably (see de Zubicaray, Wilson, McMahon, & Muthiah, 2001). Independent validation
studies subsequently confirmed that the technique is able to acquire artifact-free images
during spoken-word production (Gracco, Tremblay, & Pike, 2005), and showed that neural
activity patterns during non-speech paradigms are comparable for sparse and continuous
imaging acquisitions (Nebel et al., 2005). Note that the technique requires relatively long
inter-trial intervals, and BOLD signal time-course data are not available. However, hemo-
dynamic time-course information is rarely analyzed, even in fMRI studies using contin-
uous imaging whose primary purpose is to provide spatial information.
Another recently introduced fMRI acquisition method that shows promise for
investigating speech production is arterial spin labeling (ASL), which detects increases
in cerebral perfusion associated with task performance. Continuous perfusion imaging
acquisitions are demonstrably less sensitive to speech-related susceptibility artifacts than
BOLD fMRI (Kemeny, Ye, Birn, & Braun, 2005), and so may prove useful for sentence-
production paradigms. Although the sensitivity and temporal resolution of ASL are typi-
cally lower than BOLD acquisitions, there is less inter-individual variability in perfusion
signal changes compared to the BOLD signal, resulting in increased sensitivity to group-
level effects (Detre, Rao, Wang, Chen, & Wang, 2012).
Box 19.3 Methodological Challenges for Electrophysiological Investigations
Until recently, the dominant approach in electrophysiological (here collectively defined as

EEG and MEG; see, in this volume, Leckey & Federmeier, Chapter 3, and Salmelin, Kujala,
& Liljeström, Chapter 6) studies of speech production has been to analyze data time-locked
to stimulus onset (e.g., a picture to be named) up to an arbitrary point before articulation
(e.g., 400 ms; see Ganushchak, Christoffels, & Schiller, 2011). Muscle activity associated
with mouth, head, and eye movements are well-known sources of artifacts that confound
EEG and MEG recordings. Yet, myogenic and speech-related breathing artifacts can in fact
precede production by as much as 500–600 ms (e.g., Brooker & Donald, 1980; Galgano &
Froud, 2008). Several approaches to measuring and correcting for speech-related artifacts
have since been proposed and adopted, including using electromyogenic (EMG) recordings
from the mouth/lip (e.g., Ouyang et al., 2016; Porcaro, Meaglia, & Krott, 2015; De Vos et al.,
2010). In addition to stimulus-locked analyses, recent studies have also begun analyzing
ERPs locked to articulation onset (e.g., Riès, Xie, Haaland, Dronkers, & Knight, 2013).
However, it is important to note that response-locked ERPs are sensitive to onset differences
due to phonetic features (e.g., /b/and /p/onset words vary by about 40 ms; Fargier, Bürki,
Pinet, Alario, & Laganaro, 2017), so balanced designs are essential.
Additional sources of variability are encountered in intracranial EEG studies of speech
production (for a review, see Llorens, Trébuchon, Liégeois-Chauvel, & Alario, 2011). These
studies are typically conducted in candidates for epilepsy surgery who have intractable
seizures. This necessitates comprehensive language assessments for each patient to charac-
terize his or her cognitive function, and investigations to be performed in medication-and
seizure-free periods. Due to clinical constraints, electrode placement is often inconsistent
across patients within a study, a problem referred to as sparse sampling (not to be confused
with the fMRI acquisition technique).
As Piai, Riès, and Knight (2015) noted in their commentary, a major issue in the elec-
trophysiological literature on speech production has been variability in reporting anal-
ysis steps and suboptimal statistical testing. In particular, a number of studies appear to
lack procedures for correcting speech-related artifacts. Moreover, electrophysiological
data are multidimensional (i.e., many channels can be analyzed at multiple time points),
which requires analysis strategies that take into account this multiple-comparison problem.
However, a number of studies have adopted inappropriate statistical methods for dealing
with this problem (see Piai, Dahlslätt, & Maris, 2015). These factors have likely contributed
to the lack of consistency among some of the reported findings we discuss later.
magnetoencephalography (MEG) literature on picture naming and word production

was scant compared to neuroimaging studies (functional magnetic resonance imagining
[fMRI] and positron emission tomography [PET]), and the available data reflected a
mix of covert and overt production. In 2011, Indefrey reviewed the literature for new
studies and provided an updated version of the time course of these processing stages
and the corresponding brain regions. According to this updated version, and assuming
a response time of 600 milliseconds (ms), perceptual and conceptual preparation re-
liably involve the ventral surface of the temporal-occipital cortex. Perceptual prepara-
tion (given a visual or auditory stimulus) is completed around 100–150 ms after stimulus
476 Greig I. de Zubicaray and Vitória Piai
Box 19.4 Methodological Challenges for Brain Stimulation Techniques
Brain stimulation methods have also seen increasing application to investigate speech pro-
duction. Noninvasive techniques include transcranial magnetic stimulation (TMS; see
Schuhmann, Chapter 5 in this volume) and transcranial direct current stimulation (tDCS;
for review, see Hartwigsen, 2014), while direct electrical stimulation (DES) is invasive and
most frequently used intraoperatively for surgical planning (see Duffau, Chapter 8 in this
volume). Brain stimulation methods are useful for studies of language production because
researchers can modulate brain activity, and even temporarily disrupt the function of a
targeted region (see Schuhmann, Chapter 5 in this volume), and examine what impact this
has on task performance. Brain stimulation studies therefore provide an additional dimen-
sion of information regarding the necessity of a brain region for task performance. For both
tDCS and TMS, effective sham approaches are able to blind participants to the stimulation
conditions. Like TMS, online and offline stimulation protocols are possible with tDCS.
However, it has been proposed that they differ in terms of their effects on neural activity: the
former is thought to modulate a specific network involved in task performance, while the
latter likely produces modifications to a broader network of neural activity that lasts be-
yond the period of stimulation (Miniussi, Harris, & Ruzzoli, 2013). The latter approach may
therefore be less relevant to the study of psycholinguistic effects. The two modes of stimu-
lation present a challenge in terms of reporting and interpreting effects in neurolinguistic
studies. In particular, findings from offline and online studies should not be discussed syn-
onymously in terms of replications. Ideally, a comparison of offline and online protocols in
the one study would be most informative.
Of the three techniques, tDCS is less well established from a methodological perspective,
and the validation of specific protocols in both healthy participants and aphasic patients
requires further research attention. Inconsistencies in reported findings appear attributable
to variability in protocols applied across studies, including departures from standard ex-
perimental paradigms (e.g., Westwood, Olson, Miall, Nappo, & Romani, 2017; see Gauvin,
Meinzer, & de Zubicaray, 2017, for comment). The tDCS technique involves applying a weak
electrical current between scalp-affixed electrode pads to modulate resting membrane
potentials on the cortical surface. These pads are typically 5 x 7 cm, although high-density
electrode sets are now being marketed by some manufacturers. Localization of effects in be-
havioral studies is therefore inexact, dependent on electrode placement. Combined tDCS-
fMRI studies show promise in terms of localizing effects beneath electrodes (e.g., Meinzer,
Yetim, McMahon, & de Zubicaray, 2012). Unlike TMS, tDCS is not able to provide informa-
tion about the timing of functionally relevant brain regions. Likewise, in the case of DES,
the clinical setting does not easily afford timing investigations given that the stimulation
cannot be applied systematically at different time points during stimulus presentation.
onset, whereas conceptual preparation is completed around 200 ms. The mid-section
of the lateral middle temporal cortex is reliably involved in lexical-semantic (lemma)
processing, and the posterior superior and middle temporal cortex (Wernicke’s area) in
phonological word form (lexeme) retrieval. These processing stages span between 200–
290 ms and 290–370 ms, respectively. Correlates of post-lexical stages of processing,
including syllabification at 355–475 ms, phonetic encoding at 455–600 ms, followed by
motor articulation, were found to encompass both left inferior frontal gyrus (IFG) and
premotor cortices (for a review of these latter mechanisms, see Tremblay, Deschamps,
& Dick, Chapter 15 in this volume). Finally, Indefrey and Levelt (2004) ascribed the
monitoring of both internal and external speech to bilateral superior temporal gyrus
(STG), noting the prolonged activation observed in these structures. However, Indefrey
(2011) later acknowledged that self-monitoring processes likely engage a more com-
prehensive network involving the anterior cingulate cortex (ACC) and supplementary
motor area (SMA), perhaps reflecting more domain-general mechanisms, an issue we
will return to later.
This account of the relative timing of brain regions engaged during production has
been challenged by recent reviews. For example, Llorens, Trébuchon, Liégeois-Chauvel,
and Alario (2011) reviewed intracranial EEG studies, noting that the findings were not
always consistent with the earlier scalp-recorded EEG and MEG findings. In partic-
ular, they did not find consistent evidence for the involvement of the mid-portion of the
middle temporal gyrus (MTG) linked with the time window for lexical-semantic pro-
cessing, and noted much earlier engagement of motor cortex and IFG (from 200 ms),
challenging an interpretation of the latter regions’ roles in terms of only post-lexical
processes. Recently, Munding, Dubarry, and Alario (2016) reviewed 20 years of MEG
research on speech production, and concluded that while the data broadly supported
the conclusions of Indefrey and Levelt’s meta-analyses, the evidence was not consistent
with a serial processing architecture, but rather suggested parallel activation. In addi-
tion, they presented evidence of very early activation of IFG and motor cortex in some
studies, which they attributed to top-down control and selection processes. This review
provoked considerable debate (see responses by Piai, 2016; Riès, 2016; Strijkers, 2016).
A relatively consistent criticism was that Munding and colleagues reviewed studies with
different experimental manipulations that elicited much longer production latencies
than the 600 ms assumed by Indefrey and Levelt, blurring the temporal estimates for
the processes in question. Therefore, the study by Munding et al. (2016) is inconclusive
with respect to the question of serial versus parallel processing in spoken-word produc-
tion. Another recent critique by Strijkers and Costa (2016) likewise advocated parallel
over serial processing, and invoked top-down selection and control processes within the
framework of a neural assembly rather than a chronometric model (but see the response
by Indefrey, 2016).
Despite the emphasis on electrophysiological recordings to inform the tem-
poral components of speech production, research with single-and multiple-pulse
transcranial magnetic stimulation (TMS) has afforded additional information about the
critical or necessary timing of the brain regions in the production network. A number
of studies have consistently targeted left IFG/Broca’s area and have employed picture
naming (e.g., Chouinard, Whitwell, & Goodale, 2009; Schuhmann, Schiller, Goebel, &
Sack, 2009, 2012; Wheat et al., 2013). Across these studies, TMS reliably slowed naming
latencies when applied to the left IFG 300–400 ms following picture presentation, con-
sistent with Indefrey’s (2011) updated account. Critically, when TMS was applied to left
IFG at 100–200 ms following picture onset (i.e., the time window of early activation
reported in some MEG studies; see Munding et al., 2016; Strijkers & Costa, 2016),
naming latencies were not significantly affected (Schuhmann et al., 2009, 2012; Wheat
et al., 2013). Therefore, if early activation of IFG is attributable to top-down selection and
control mechanisms, as some authors propose, then the consistent finding from TMS
studies does not support the necessary early involvement of these processes in simple
picture naming. The same can be said for the early involvement of motor cortex based
upon some MEG results (e.g., Munding et al., 2016; Strijkers & Costa, 2016), with two
TMS studies consistently failing to observe effects of stimulation of the motor cortex on
naming latencies at 0, 100, or 300 ms after picture onset (Mottaghy, Sparing, & Töpper,
2006; Töpper, Mottaghy, Brugmann, Noth, & Huber, 1998).
To our knowledge, three TMS studies of picture naming have targeted Wernicke’s area
at the 200–300 ms time period identified by Indefrey (2011) as corresponding to phono-
logical word form retrieval (Mottaghy et al., 2006; Schuhmann et al., 2012; Töpper et al.,
1998). Surprisingly, none reported a significant effect on naming latencies, although one
observed an effect at 400 ms post–picture onset, interpreted as being consistent with
self-monitoring (Schuhmann et al., 2012). Note that these studies, while problematic for
Indefrey and Levelt’s account, likewise showed that earlier activation of posterior STG is
not necessary for successful picture naming, as TMS did not produce an effect at 0, 100,
or 200 ms (cf., Munding et al., 2016; Strijkers & Costa, 2016). Although it is tempting
to conclude that word-form retrieval might occur later than estimated by both chron-
ometric and electrophysiological methods, it is important to note that all three studies
applied TMS solely to the left posterior STG (Brodmann area 22), whereas Indefrey and
Levelt’s meta-analysis attributed word-form processing to cortical tissue encompassing
the posterior sections of both left MTG and STG. Schuhmann et al.’s (2012) study was
the only one to apply TMS to the mid-portion of the MTG, finding an effect at 225 ms
post–picture onset, consistent with Indefrey and Levelt’s attribution of lexical-semantic
processing. Consequently, mid-and posterior sections of MTG should be targets for fu-
ture TMS investigations concerning the time course of production.
Alternatively, Indefrey and Levelt’s attribution of posterior MTG/STG might be
wrong, and a different region could subserve phonological word form retrieval. For ex-
ample, using lesion-symptom mapping (LSM; see Wilson, Chapter 2 in this volume),
Schwartz, Faseyitan, Kim, & Coslett (2012) and Dell, Schwartz, Nozari, Faseyitan, &
Coslett (2013) analyzed speech error data from large cohorts of aphasic patients and
implicated a region adjacent to Wernicke’s area, the left inferior parietal lobe (IPL,
comprising supramarginal and angular gyri, and planum temporale), in phonological/
phonemic errors during picture naming. They interpreted their findings as consistent
with a dorsal stream, articulatory-based account of phonological processing in produc-
tion (see also Hickok, Chapter 20 in this volume). However, a reanalysis of Schwartz
et al.’s (2012) data examining phonological neighborhood density effects (as lexical
phonological processing involves activating not only the target word, but also phono-
logically related words) implicated both posterior STG and IPL (Mirman & Graziano,
2013). Both regions were also reported in a sparse-design fMRI study of phonolog-
ical neighborhood effects in production (Peramunage, Blumstein, Myers, Goldrick, &
Baese-Berk, 2011). Of note, other lesion symptom mapping studies have reported left
posterior MTG/STG rather than IPL, consistent with Indefrey and Levelt’s account
(e.g., Butler, Lambon Ralph, & Woollams, 2014). Future investigations using more fine-
grained analyses of aphasics’ speech errors differentiating lexical versus post-lexical
phonological representations are needed (see Goldrick & Rapp, 2007; e.g., Schwartz
et al., 2012 analyzed all nonword errors, regardless of their phonological overlap with the
target).
Investigating the Spatiotemporal
Components of Speech
Production Models
with Psycholinguistic Paradigms
In this next section, we summarize and discuss some key findings from neurolinguistic
investigations using context manipulations in picture-naming paradigms to charac-
terize processing stages in speech production. As the spatiotemporal components corre-
sponding to representational stages in production models are relatively well delineated
(see earlier discussion), neurolinguistic investigations are informative for testing rival
accounts that attribute manipulations of an experimental factor to different processing
stages (see de Zubicaray, 2012). However, context manipulations influencing the time
course of production in chronometric studies likewise influence the timing of evoked
responses in electrophysiological recordings, so should not be used for estimating tem-
poral signatures of specific processing stages during simple production tasks (see Piai,
Roelofs, Jensen, Schoffelen, & Bonnefond, 2014; cf. Munding et al., 2016). Rather, they
are useful for demonstrating the correlates of the cognitive effect of interest. Here, we
constrain our review to paradigms and context effects for which results from multiple
sources of evidence are available, that is, across functional neuroimaging, electrophysi-
ological, lesion-mapping, and/or brain simulation methods. We interpret only reliably
reported findings across multiple studies.
Picture-Word Interference
Over four decades, the picture-word interference (PWI) paradigm has been used to test
hypotheses about processing stages involved in spoken-word production (Rosinski,
Golinkoff, & Kukish, 1975). In brief, the PWI paradigm involves participants naming a
set of target pictures in context with written or auditorily presented distractor words. The
most commonly employed manipulations in neurolinguistic studies involve distractors
either semantically or phonologically related to the target picture name, compared
with unrelated words. The semantic interference effect refers to the finding that naming
latencies are significantly slower for pictures paired with categorically related versus un-
related words (e.g., a pictured “cow” with related distractor “pig” versus unrelated dis-
tractor “pin”). Conversely, phonologically related distractors (i.e., showing phoneme
overlap with the picture name, e.g., a pictured “cow” with related distractor “couch”)
have been shown to facilitate (speed) target naming compared to unrelated distractors
(e.g., “pin”). Both effects occur reliably within a small range of distractor stimulus onset
asynchronies (SOAs) around target picture presentation (–150 to 150 ms; i.e., the dis-
tractor word is presented up to 150 ms before the picture or after picture presentation),
with simultaneous distractor-target presentation usually employed. Virtually all pro-
duction accounts attribute the locus of the phonological facilitation effect to the word
form retrieval stage of processing. However, the locus of the semantic interference ef-
fect is a matter of debate between rival accounts. According to the lexical-selection-by-
competition account, lemmas compete for selection and semantically related distractors
increase this competition, delaying the selection process, which surfaces as longer
naming latencies for the related condition (e.g., Piai et al., 2014). Conversely, according
to post-lexical accounts, the semantic interference effect emerges in later stages, closer
to articulation onset (see Mahon, Costa, Peterson, Vargas, & Caramazza, 2007). In ad-
dition, recent accounts have increasingly proposed involvement of top-down selection
and control processes for resolving competition among lexical candidates.
Our review identified over 20 neurolinguistic studies of semantic interference and
phonological facilitation using the PWI paradigm with EEG, MEG, fMRI, transcranial
direct current stimulation (tDCS), and lesion patients. Of note, there is consider-
able variability in experimental designs across studies, including departures from the
standard PWI procedure. A critical design feature of the PWI paradigm is that the same
set of target pictures is used in each condition, with each picture paired with a different
distractor word, ensuring that target processing is identical. Distractors are matched
on a range of lexical variables, and the same words are often re-paired with different
pictures to create the unrelated condition. Distractors may also be target picture names
(i.e., members of the response set). Some studies include another condition, for ex-
ample, picture naming in the absence of a distractor, or the picture name itself, or a neu-
tral distractor condition (e.g., a row of Xs, nonwords, or symbols). Here, we restrict our
review to canonical (i.e., related versus unrelated) contrasts for the semantic interfer-
ence and phonological facilitation effects of interest. For both effects, we report differen-
tial activity associated with related compared to unrelated distractors (i.e., mean activity
increases for related versus unrelated distractors, and vice versa).
Seven fMRI studies, two lesion studies, one tDCS study, and one MEG study have
provided spatial location/source information for the semantic interference effect, all
of which (except two) employed written distractors (Abel et al., 2009; Abel, Dressel,
Weiller, & Huber, 2012; de Zubicaray & McMahon, 2009). Of the fMRI studies, four de-
parted from the traditional PWI design by using unique sets of target pictures per dis-
tractor condition, or a distractor SOA of 550 ms (i.e., distractor presented after picture
onset), and all used continuous rather than optimal sparse imaging designs (Abel et al.,
2009; Abel et al., 2012; Diaz et al., 2014; Rizio, Moyer, & Diaz, 2017; Spalek & Thompson-
Schill, 2008). Diaz et al. (2014) failed to observe any significant activity for the contrast
of related versus unrelated distractor conditions in either whole brain or Region of in-
terest (ROI) analyses. Abel et al.’s (2012) reanalysis of their 2009 study data reported
significantly increased activity in the left IFG and reduced activity in left lingual and
bilateral precentral gyri, left ACC, posterior STG, parietal operculum, and bilateral
cuneus. Conversely, Rizio et al. (2017) observed only significantly increased activity in
the bilateral middle frontal gyrus (MFG), bilateral MTG, and precuneus. Spalek and
Thompson-Schill (2008) did not observe a significant semantic interference effect in
naming latencies (likely due to the late SOA employed), and reported increased blood
oxygenation level dependent (BOLD) signal in the right posterior cerebellum and left
fusiform and parahippocampal gyri. The inconsistent results across the four studies
might therefore reflect the use of unique picture sets in each condition, written versus
auditory distractors, a late distractor SOA, speech-related artifacts in continuous im-
aging, or a combination of all these issues.
More consistent findings are provided by the one sparse and one continuous event-
related fMRI study and one MEG study using the standard PWI design with written
distractors (de Zubicaray, Hansen, & McMahon, 2013; Piai, Roelofs, Acheson, &
Takashima, 2013, Piai et al., 2014). A consistent finding across all studies is reduced ac-
tivity in left mid-to posterior MTG and STG for related versus unrelated distractors.
This pattern was also found in another sparse design fMRI study using auditory
distractors (de Zubicaray & McMahon, 2009). Piai and Knight (2018; see also Piai, Riès,
& Swick, 2016) also reported increased semantic interference in error rates for patients
with left lateral temporal cortex lesions, but not for patients with IFG lesions. However,
Henseler, Mädebach, Kotz, and Jescheniak (2014) failed to observe any modulation of
semantic interference with tDCS applied to the left posterior MTG or IFG. Notably,
across studies, semantic interference was not consistently associated with involvement
of cortical regions attributed to post-lexical processes or mechanisms proposed for re-
solving competition among semantic competitors (e.g., premotor cortex or IFG). Thus,
the available data support a lexical-level rather than post-lexical account of semantic
interference in PWI (cf., Mahon et al., 2007).
Of the six EEG studies and one MEG study attempting to characterize event-related
responses (i.e., event-related potentials [ERPs] and event-related fields [ERFs]) to se-
mantic interference using stimulus-locked analyses, one departed from the typical
PWI design by blocking (and cycling) category members (Aristei, Melinger, & Abdel
Rahman, 2011), while two others failed to observe any significant differences between
related and unrelated ERPs (Hirschfeld, Jansma, Bölte, & Zwitserlood, 2008; Piai,
Roelofs, & van der Meij, 2012). Across the remaining four studies, semantic interfer-
ence consistently modulated the event-related responses in the N400 time window
(around 250–550 ms), showing a more negative-going waveform or smaller amplitude
for related compared to unrelated distractors (i.e., N200–400; Blackford, Holcomb,
Grainger, & Kuperberg, 2012; Dell’Acqua et al., 2010; Piai et al., 2014; Zhu, Zhang, &
Damian, 2016). Given the relatively longer naming latencies observed in PWI, the
electrophysiological data are also consistent with a lexical-level account of the semantic
interference effect.
Phonological facilitation in PWI has been investigated by five fMRI studies. Two
of these used the standard paradigm, each with written and auditory distractors in
sparse designs (de Zubicaray, McMahon, Eastburn, & Wilson, 2002; de Zubicaray &
McMahon, 2009). Three further fMRI studies departed from the standard PWI design
and used continuous imaging. Surprisingly, all three of these latter studies failed to ob-
serve significant differences in naming latencies; that is, the experimental manipulation
did not produce the phonological facilitation effect of interest (Abel et al., 2009; Abel
et al., 2012; Diaz et al., 2014; Rizio et al., 2017). This might reflect the use of unique pic-
ture sets or gradient noise associated with continuous imaging that prevents accurate
measurements of response latency even with noise-canceling techniques (although Abel
and colleagues also failed to observe a significant difference in an identical behavioral
experiment conducted outside the scanner). Diaz et al. (2014) reported increased acti-
vation for phonologically related versus unrelated distractors in bilateral supramarginal
and angular gyri, whereas Rizio et al. (2017) reported signal increases in the right an-
gular gyrus and left superior parietal cortex, and Abel et al. reported increased activity
in left supramarginal gyrus and inferior parietal lobule (BA 40), accompanied by signal
reductions in bilateral visual cortex, left ACC, and parahippocampal gyrus. Ignoring the
absence of evidence for a significant phonological effect in the behavioral data across
studies, the fMRI data could be interpreted as reflecting relatively consistent involve-
ment of the left IPL, which might be considered to support a dorsal stream, articulatory-
based account of phonological processing in production (e.g., Dell et al., 2013; Schwartz
et al., 2012; but see further discussion in this chapter).
Across the two fMRI studies using sparse event-related designs, BOLD signal
decreases were observed consistently in the left posterior MTG/STG (de Zubicaray et al.,
2002; de Zubicaray & McMahon, 2009). Additional evidence for left posterior temporal
cortex involvement is provided by Pisoni and colleagues (Pisoni, Cerciello, Cattaneo,
& Papagno, 2017), using anodal tDCS that significantly reduced the magnitude of the
facilitation effect. The evidence for left IFG involvement is equivocal. Piai et al. (2016)
reported an increased facilitation effect in left IFG lesion patients compared to healthy
controls. Significantly reduced BOLD signal responses in left IFG were reported in one
auditory PWI study (de Zubicaray & McMahon, 2009). However, another fMRI study
did not observe differential left IFG activity for the phonological effect (de Zubicaray
et al., 2002), and Pisoni et al. (2017) reported tDCS applied to left IFG did not influence
the facilitation effect. Of the four EEG studies to investigate event-related responses to
phonological facilitation in PWI, two failed to observe any significant differences in
stimulus-locked waveforms (Blackford et al., 2012; Bürki, 2017), and the remaining two
reported different results, ranging from a less negative-going waveform between 250
and 450 ms (Dell’Acqua et al., 2010) to a more positive-going waveform between 450 and
600 ms (Zhu et al. 2016; in Chinese).
Figure 19.1 provides a summary of the spatiotemporal components associated with
the semantic (left) and phonological (right) effects in picture-word interference. Both
Semantic effect Phonological effect

LSM
Stimulation
fMRI
Electrophysiology
350–550 250–450
Negativity Rel < Unr Negativity Rel < Unr
250–450 450–600
Negativity Rel < Unr Positivity Rel < Unr
375–400
Amplitude Rel < Unr No effect
250–450
Negativity Rel < Unr
No effect
Figure 19.1. Schematic view of the evidence on the spatial (for the left mid-to posterior supe-
rior and middle temporal gyri and left IFG) and temporal components of semantic and phono-
logical context effects in picture-word interference. Only studies using the standard paradigm are
shown. Each method is color coded (light blue: fMRI; dark blue: electrophysiology; red: lesion-
symptom mapping, LSM; pink: noninvasive brain stimulation). Each colored circle represents
one study.
Abbreviations: interf = interference; rel = related; unr = unrelated.
significant and nonsignificant effects are shown for studies using the standard paradigm.
Each study is represented by a circle, which is color-coded according to the method em-
ployed. The most consistent pattern for the semantic effect seems to be decreased brain
activity in the left posterior temporal lobe (i.e., related < unrelated), in a window ranging
around 250–500 ms. For the phonological effect, decreased brain activity in the poste-
rior temporal lobe seems to be the only reproducible pattern, although not as consistent
as for the semantic effect.
Blocked Cyclic Naming
Another production paradigm with a relatively large neurolinguistic evidence base
is blocked cyclic naming. The most frequently implemented version of the paradigm
entails a small set of pictures that participants name repeatedly over several cycles, al-
ternating between categorically related (e.g., animals) versus unrelated contexts (ani-
mals, musical instruments, vehicles, vegetables). Like the PWI paradigm, categorically
related contexts elicit a semantic interference effect, typically observed only from the
second cycle onward (for a review, see Belke & Stielow, 2013). Theoretical accounts have
been devised to specifically explain semantic interference in this paradigm, with several
incorporating evidence from lesion, neuroimaging, and noninvasive brain stimulation
studies. All of these accounts propose roles for both the left posterior MTG/STG and left
IFG, in lexical-semantic processing and various control/selection-biasing mechanisms,
respectively (e.g., Belke & Stielow, 2013; Oppenheim, Dell, & Schwartz, 2010; Schnur
et al., 2009). An incremental learning mechanism has also been proposed to be respon-
sible for the persistence of interference across cycles but has received less research atten-
tion in terms of its neural correlates (Damian & Als, 2005; Oppenheim et al., 2010).
Neuroimaging, electrophysiological, brain stimulation, and LSM studies of semantic
interference in blocked cyclic naming mostly provide evidence for involvement of left
posterior MTG/STG. However, there is little consistency with respect to the direction
of the effect across studies. Two early fMRI studies involved nonstandard design and
analysis manipulations, making results difficult to interpret. For example, Hocking,
McMahon, & de Zubicaray (2009) manipulated visual feature overlap across blocks
and were unable to record naming latencies during perfusion fMRI, while Schnur et al.
(2009) reported findings for a contrast of semantic interference versus phonological fa-
cilitation (i.e., [semantic > unrelated] > (phonological > unrelated]). Both studies re-
ported signal increases. One perfusion fMRI experiment using the standard design
reported signal reductions (de Zubicaray, Johnson, Howard, & McMahon, 2014). Two
tDCS studies reported opposite effects for online and offline stimulation protocols
(Meinzer, Yetim, McMahon, & de Zubicaray, 2016; Pisoni, Papagno, & Cattaneo, 2012).
One LSM study reported a significant increase in error rates (Harvey & Schnur, 2015).
Interestingly, the peak spatial coordinates reported by Harvey and Schnur’s (2015) LSM
and de Zubicaray et al.’s (2014) perfusion fMRI studies in posterior MTG/STG were vir-
tually identical. An MEG study likewise reported differential MTG/STG activity (Maess,
Friederici, Damian, Meyer, & Levelt, 2002; note the direction of the effect could not be
interpreted given the principal component analysis approach). An intracranial EEG
study reported decreases in the evoked responses in the MTG and STG, but increases in
the inferior temporal lobe (Riès et al., 2017). Finally, a TMS study by Krieger-Redwood
and Jefferies (2014) reported an effect of stimulation over posterior MTG/STG only in
the first cycle of naming (i.e., prior to the emergence of the interference effect).
The evidence from a similar range of studies is mostly consistent with left IFG in-
volvement, although there are some notable exceptions. Whereas only one of three fMRI
studies reported significant IFG activity using a nonstandard comparison (Schnur et al.,
2009; cf. Hocking et al., 2009; de Zubicaray et al., 2014), two of three studies of aphasics
with IFG lesions noted significant effects in error rates (Riès, Greenhouse, Dronkers,
Haaland, & Knight, 2014; Schnur et al., 2009; cf. Harvey & Schnur, 2015). Two of three
tDCS studies reported an effect of stimulation to the left IFG, reducing the magnitude of
the interference effect using offline (Pisoni et al., 2012) and online stimulation protocols
(Meinzer et al., 2016; cf. Westwood, Olson, Miall, Nappo, & Romani, 2017), although
one TMS study observed an effect only in the first cycle of naming (i.e., prior to semantic
interference occurring; Krieger-Redwood & Jefferies, 2014). An additional tDCS study
reported a significant reduction in the magnitude of semantic interference following

online but not offline stimulation over the dorsal frontal cortex (Wirth et al., 2011).
Unfortunately, EEG and MEG estimates of stimulus-locked event-related responses
for the semantic interference effect vary considerably, both in terms of timing and po-
larity of waveforms: from 150–225 ms (Maess et al., 2002), 200–500 ms (Wang, Shao,
Chen & Schiller, 2017; smaller negativity; in Chinese), 220–450 ms (Janssen, Carreiras,
& Barber, 2011; smaller negativity), 270–315 ms (Python, Farghier, & Laganaro, 2017;
smaller positivity), to 500–750 ms (Janssen, Hernández-Cabrera, van der Meij, &
Barber, 2015; smaller positivity). Two other electrophysiological studies either used a
nonstandard design combining PWI and blocking (Aristei et al., 2011) or failed to ob-
serve any significant differences in event-related responses (Llorens, Trebuchon, Riès,
Alario, & Liegeois-Chauvel, 2014). The considerable variability in analysis techniques
across studies might explain the inconsistent findings, as might the authors’ choice
of interpretations for event-related responses in particular time windows. With re-
spect to the latter, some studies reported more than one time window for event-related
responses, permitting some flexibility in interpretation. For example, Maess et al. (2002)
reported a second evoked response around 450–475 ms interpreted as self-monitoring,
whereas Janssen et al. (2015) reported an effect in an earlier 250–400 ms time window,
which they interpreted in terms of conceptual processing.
Substantial evidence in animals and humans implicates the hippocampus in both im-
plicit (unconscious) and explicit retrieval of relational information (see Duss et al., 2014;
for review, see Henke, 2010), making it a plausible candidate for the implicit incremental
learning mechanism proposed in psycholinguistic accounts of semantic interference in
blocked cyclic naming (e.g., Damian & Als, 2005; Oppenheim et al., 2010). One perfu-
sion fMRI study and one intracranial EEG study explicitly targeted hippocampal ac-
tivity using the standard design, interpreting their findings as reflecting the operation
of an incremental learning mechanism (de Zubicaray et al., 2014; Llorens, Dubarry,
Trébuchon, Chauvel, Alario, & Liégeois-Chauvel, 2016). The perfusion fMRI study re-
vealed a reduction in activity during the related context. Using intracranial electrodes
implanted directly in bilateral hippocampus, Llorens et al. (2016) reported that the am-
plitude of the event-related responses (a negativity peaking around 600 ms) in the re-
lated blocks was smaller than in unrelated blocks. Crucially, this negative peak emerged
progressively from the second cycle of naming onward.
Figure 19.2 provides a summary of the spatiotemporal components associated with the
semantic effect in blocked cyclic naming. Both significant and nonsignificant effects are
shown for the studies using the standard paradigm. Each study is represented by a circle,
color-coded according to the method employed. The most consistent pattern for the se-
mantic effect seems to be an impact on the magnitude of the interference depending
on whether the left IFG is stimulated noninvasively or is damaged. Decreased brain ac-
tivity in the left posterior temporal lobe (i.e., related < unrelated) is also relatively con-
sistent, and again brain stimulation or damage in that area impact the magnitude of the
interference effect. Regarding the temporal component, modulations in the 200–450 ms
range seem to be the most consistent pattern, with the homogeneous condition showing
Electrophysiology Brain activity
fMRI No effect
Stimulation
LSM
Behavior
Reduced interf
Increased interf
Behavior
Reduced interf
No effect Increased interf
Brain activity
Hom < Het
150–225 450–475
Direction unknown
220–450
Negativity Hom < Het
270–315
Positivity Hom < Het
250–400 500–750
Negativity Hom < Het Positivity Hom < Het
200–500
Negativity Hom < Het
No effect
Figure 19.2. Schematic view of the evidence on the spatial (for the left superior and middle
temporal gyri and left IFG) and temporal components of the semantic context effect in blocked
cyclic naming. Only studies using the standard paradigm are shown. Each method is color-coded
(light blue: fMRI; dark blue: electrophysiology; red: lesion-symptom mapping, LSM; pink: non-
invasive brain stimulation). Each colored circle represents one study.
Abbreviations: het = heterogeneous; hom = homogeneous; interf = interference.
decreased amplitude relative to the heterogeneous condition, although as noted earlier,

authors tend to disagree on the interpretation of that modulation.
Continuous Naming
The continuous-naming paradigm introduced by Howard, Nickels, Coltheart, & Cole-
Virtue (2006) requires participants to name a pseudo-random series of pictures, and
likewise elicits a semantic interference effect. Within the series, exemplars from a range
of semantic categories are interspersed with filler items. The interval or lag between
each consecutive category exemplar is also varied. Semantic interference in this para-
digm manifests from the second ordinal position within a category and accumulates
linearly at ~30 ms for each successive categorically related picture. As with the semantic
interference effects in PWI and blocked cyclic naming tasks, the left lateral temporal
lobe is proposed to play a prominent role in lexical-semantic retrieval in continuous
naming (e.g., Belke, 2013). These accounts also tentatively ascribe a role for the left IFG
in either top-down, selection-biasing, or activation-boosting mechanisms (Belke, 2013;
Canini et al., 2016; Oppenheim et al., 2010). Further, at least one account presumes
that semantic interference in the continuous and blocked cyclic naming arises due to
a common mechanism(s) (Oppenheim et al., 2010), leading to the expectation that the
spatiotemporal mechanisms will bear at least a strong resemblance across paradigms.
A cumulative interference effect for phonologically related words has also been reported
in continuous reading aloud (Mulatti, Peresotti, Job, Saunders, & Coltheart, 2012), but is
yet to be subjected to neurolinguistic investigation.
Despite being introduced only a decade earlier, the continuous-naming paradigm
has been the subject of three fMRI studies, as many EEG studies, one lesion study, and
one tDCS investigation. Of the three fMRI studies, two employed continuous BOLD
acquisitions (Canini et al., 2016; Wilson, Isenberg, & Hickok, 2009), while the third
used perfusion imaging (de Zubicaray, McMahon, & Howard, 2015). Wilson et al.
(2009) employed Howard et al.’s (2006) stimuli, yet were unable to detect any significant
BOLD signal correlates of cumulative interference. Canini et al.’s (2016) design departed
from Howard et al.’s by presenting participants with two different experimental lists,
averaging the ordinal position data. They were unable to report naming latency data
due to the gradient noise accompanying continuous imaging, and did not find evidence
for cumulative interference in error rates. Their parametric fMRI analysis revealed a
linear increase in BOLD signal in the left IFG and caudate. However, this analysis in-
cluded the first ordinal position data, whereas the cumulative effect is calculated from
the second ordinal position onward. Using perfusion imaging with the original Howard
et al. experimental lists, de Zubicaray et al. (2015) reported a significant linear increase
in left mid-MTG and perirhinal cortex activity from the second ordinal position on-
ward. However, Westwood et al. (2017) reported that tDCS to left posterior MTG did
not modulate the interference effect compared to sham (but see Gauvin, Meinzer, & de
Zubicaray, 2017, for a critique of Westwood et al.’s methods). Thus, the evidence for left
lateral temporal lobe involvement in cumulative semantic interference is best described

as inconsistent and in need of further investigation.
The findings from lesion, EEG, and tDCS studies are more consistent with respect to
the left IFG. Riès, Karzmark, Navarrete, Dronkers, & Knight (2015) failed to observe an
effect of left IFG lesions on cumulative interference in either naming latencies or error
rates. Westwood et al. (2017) reported that tDCS to left IFG did not modulate the in-
terference effect compared to sham (but see Gauvin et al., 2017). Using a nonstandard
design omitting the lag manipulation, one study failed to observe any differences asso-
ciated with cumulative interference (Llorens et al., 2014). Both Costa et al. (2009) and
Rose and Abdel Rahman (2016) reported a linear modulation of positive waveforms
at around 200–400 ms post–picture onset over only posterior electrodes, in addition
to significant correlations with naming latencies between 208 and 388 ms, and 268 and
413 ms, respectively, noting that their findings were broadly consistent with the time
window suggested for lexical selection (see spatial and temporal components section
earlier in this chapter).
Figure 19.3 provides a summary of the spatiotemporal components associated with the
cumulative semantic effect in continuous naming. Both significant and nonsignificant
effects are shown for the studies using the standard paradigm. Each study is represented
by a circle, color-coded according to the method employed. The paucity of studies
contributes to an impression of a lack of consistency in findings. In the temporal do-
main, a linear modulation of positive-going waveforms in the 250–400 range seems to
be the consistent finding over two studies.
Are Top-D own Control/Selection

Biasing Mechanisms Really Required
to Resolve Semantic Interference?
Our review of findings from the PWI, blocked cyclic, and continuous- naming
paradigms is informative both from a neurolinguistic perspective and for constraining
cognitive accounts of production. All three paradigms were designed to promote activa-
tion of multiple lexical candidates, and observations of behavioral semantic interference
effects across paradigms are typically interpreted in terms of common mechanisms.
Recent psycholinguistic accounts of all three paradigms have begun to incorporate
neural data, with a particular emphasis on intervention by domain-general, top-down
cognitive control mechanisms to resolve competition during production. For the most
part, these accounts have proposed a prominent role for the left IFG in resolving compe-
tition among lexical-semantic competitors. Indeed, Belke and Stielow (2013) concluded,
“It appears that any future model of word production unavoidably faces the challenge
of specifying how left frontal mechanisms of domain-general cognitive control interact
Electrophysiology Brain activity

fMRI No effect
Stimulation
LSM
Behavior
No effect Behavior
No effect
Brain activity
Linear increase
250–400 450–600
Linear modulation of Linear modulation of
positive waveforms negative waveforms
200–400
Linear modulation of positive waveforms
No effect
Figure 19.3. Schematic view of the evidence on the spatial (for the left MTG and left IFG)
and temporal components of the cumulative semantic effect in continuous naming. Only studies
using the standard paradigm are shown. Each method is color-coded (light blue: fMRI; dark
blue: electrophysiology; red: lesion-symptom mapping, LSM; pink: noninvasive brain stimula-
tion). Each colored circle represents one study.
with paradigmatic interference during lexical- semantic encoding” (p. 23). Based
upon the evidence we have reviewed here, this conclusion appears to be a significant
overstatement.
We could find no reliable evidence for a role of the left IFG in semantic interference
in either PWI or continuous-naming paradigms. Although absence of evidence might
not be evidence of absence, the fact that left IFG involvement was observed only semi-
reliably in blocked cyclic naming studies of semantic interference suggests the need for
a reassessment of proposals concerning the ubiquitous involvement of domain-general,
top-down mechanisms in biasing or resolving competition during spoken word pro-
duction (subserved by the left IFG). At the very least, it demonstrates that semantic
interference across the three naming paradigms does not necessarily reflect iden-
tical mechanisms (cf. Oppenheim et al., 2010). Of the three paradigms, blocked cyclic
naming is the least akin to naturalistic speech, involving the massed repetition/cycling
of a small set of responses. The prominence afforded to this paradigm in speech pro-
duction accounts is therefore questionable. It is also worth emphasizing that semantic
interference effects may not necessarily reflect lexical-level processes either. In other
naming paradigms, semantic interference has been attributed to prelexical, concep-
tual processes, for example, post-cue naming (Dean, Bub, & Masson, 2001; Hocking,
McMahon, & de Zubicaray, 2010) and negative priming (de Zubicaray, McMahon,
Eastburn, Pringle, & Lorenz, 2006; Tipper, 1985).
As we have noted, proposals for top-down involvement have now extended to simple
production tasks such as basic picture naming (e.g., Munding et al., 2016; Strijkers &
Costa, 2016). Such proposals require theoretical motivation beyond the mere observa-
tion of neurophysiological responses. For example, when speakers can produce two to
four words per second, and produce errors no more than one to two times every 1,000
words during everyday speaking (Levelt, Roelofs, & Meyer, 1999), it would be useful to
explain precisely why top-down intervention in selecting lexical candidates for produc-
tion is so essential.
This is not to say that there is no evidence of left IFG or other control-related
mechanisms in the studies we reviewed. In neuroimaging studies of the PWI paradigm,
lateral and medial frontal (ACC or SMA) engagement was reported relatively consist-
ently for contrasts of related versus identity or neutral conditions (de Zubicaray, Wilson,
McMahon, & Muthiah, 2001; Piai et al., 2013; Piai et al., 2014), and patients with left
IFG lesions similarly showed increased interference for the contrast of a lexical dis-
tractor versus a neutral condition when compared to healthy controls (Piai et al., 2016).
This suggests a role for the left IFG in resolving competition introduced by competing
linguistic information, rather than selecting among semantic competitors per se. This
may reflect the operation of an early attention-blocking mechanism, as Piai et al. (2016)
suggested.
A View to the Future
In this chapter, we have examined a number of issues facing researchers when

investigating the spatiotemporal components of speech production. A relatively clear
impression from our review is that suboptimal methods have compromised a signif-
icant proportion of studies. Inconsistencies in BOLD fMRI study results were a reg-
ular outcome with standard continuous imaging acquisitions. We recommend that
continuous BOLD acquisitions be avoided in future fMRI studies of speech produc-
tion. As neuroimaging is an expensive enterprise, adherence to methodological best
practice is both a scientific and economic imperative. Similarly, findings from electro-
physiological recordings showed such variability that they were uninterpretable for
some psycholinguistic effects. A consensus approach to design and analysis with these
techniques is sorely needed. Nonstandard experimental designs were also frequently a
source of problems for interpretation. Throughout our review, we relied on converging
evidence from multiple sources of data to support our interpretations. Perhaps the only
relatively consistent finding across paradigms was for the 250–450 ms time window
and posterior temporal lobe (MTG/STG) involvement. Notably, many arguments that
engendered debate in the literature emphasized data from a single modality—proposals
concerning evidence for parallel rather than serial activation of processing stages being
a prominent example.
We reviewed findings from context manipulations in picture-naming paradigms,
as these reflect the most well-developed evidence base for investigating the spatial and
temporal components of processing stages in spoken-word production, and in par-
ticular the stage of retrieving words from the mental lexicon. The neurolinguistic lit-
erature currently reflects only a fraction of the context manipulations conducted in
psycholinguistic studies, and is strongly biased toward semantics and monolingual pro-
duction. This imbalance needs to be addressed. Outside of Stroop-like color-naming
paradigms that have been the topic of many reviews, there is a paucity of neurolinguistic
evidence concerning context manipulations in reading aloud. Recent sparse and perfu-
sion fMRI studies have successfully mapped the networks involved in sentence produc-
tion (e.g., Geranmayeh, Wise, Mehta, & Leech, 2014; Kemeny, Ye, Birn, & Braun, 2005;
Tremblay & Small, 2011), paving the way for more sophisticated manipulations. Finally,
compared to the psycholinguistic literature, neurolinguistic studies of self-monitoring
mechanisms are also scarce, and there is a clear tension between domain-general and
speech-perception-based accounts that should prove a fruitful area for inquiry.
References
Abel, S., Dressel, K., Bitzer, R., Kümmerer, D., Mader, I., Weiller, C., & Huber, W. (2009). The
separation of processing stages in a lexical interference fMRI-paradigm. NeuroImage, 44,
1113–1124.
Abel, S., Dressel, K., Weiller, C., & Huber, W. (2012). Enhancement and suppression in a lexical
interference fMRI-paradigm. Brain and Behavior, 2, 109–127.
Aristei, S., Melinger, A., & Abdel Rahman, R. (2011). Electrophysiological chronometry of
semantic context effects in language production. Journal of Cognitive Neuroscience, 23,
1567–1586.
Belke, E. (2013). Long-lasting inhibitory semantic context effects on object naming are neces-
sarily conceptually mediated: Implications for models of lexical-semantic encoding. Journal
of Memory and Language, 69, 228–256.
Belke, E., & Stielow, A. (2013). Cumulative and non-cumulative semantic interference in ob-
ject naming: Evidence from blocked and continuous manipulations of semantic context.
Quarterly Journal of Experimental Psychology, 66(11), 2135–2160.
Blackford, T., Holcomb, P. J., Grainger, J., & Kuperberg, G. R. (2012). A funny thing happened
on the way to articulation: N400 attenuation despite behavioral interference in picture
naming. Cognition, 123(1),  84–99.
Brooker, B. H., & Donald, M. W. (1980). Contribution of the speech musculature to apparent
human EEG asymmetries prior to vocalization. Brain & Language, 9, 226–245.
Bürki, A. (2017). Electrophysiological characterization of facilitation and interference in the
picture-word interference paradigm. Psychophysiology, 54(9), 1370–1392.
Butler, R. A., Ralph, M. A. L., & Woollams, A. M. (2014). Capturing multidimensionality in
stroke aphasia: Mapping principal behavioural components to neural structures. Brain,
137(12), 3248–3266.
Canini, M., Della Rosa P. A., Catricalà E., Strijkers K., Branzi F. M., Costa A., et al. (2016).
Semantic interference and its control: A functional neuroimaging and connectivity study.
Chouinard, P. A., Whitwell, R. L., & Goodale, M. A. (2009) The lateral-occipital and the
inferior-frontal cortex play different roles during the naming of visually-presented objects.
Costa, A., Strijkers, K., Martin, C. D., & Thierry, G. (2009). The time-course of word retrieval
revealed by event-related brain potentials during overt speech. Proceedings of the National
Damian, M. F., & Als, L. C. (2005). Long-lasting semantic context effects in the spoken produc-
tion of object names. Journal of Experimental Psychology: Learning, Memory, and Cognition,
31, 1372–1384.
Dean, M. P., Bub, D. N., & Masson, M. E. (2001). Interference from related items in object iden-
tification. Journal of Experimental Psychology: Learning, Memory, and Cognition, 27, 733–743.
De Vos, M., Riès, S., Vanderperren, K., Vanrumste, B., Alario, F.-X., Van Huffel, S., & Burle, B.
(2010). Removal of muscle artifacts from EEG recordings of spoken language production.
Neuroinformatics, 8, 135–150.
de Zubicaray, G. I. (2012). Strong inference in functional neuroimaging. Australian Journal of
de Zubicaray, G. I., Hansen, S., & McMahon, K. L. (2013). Differential processing of thematic
and categorical conceptual relations in spoken word production. Journal of Experimental
Psychology: General, 142, 131–142.
de Zubicaray, G., Johnson, K., Howard, D., & McMahon, K. (2014). A perfusion fMRI investi-
gation of thematic and categorical context effects in the spoken production of object names.
Cortex, 54, 135–149.
de Zubicaray, G. I., & McMahon, K. L. (2009). Auditory context effects in picture naming
investigated with event-related fMRI. Cognitive, Affective and Behavioral Neuroscience, 9(3),
260–269.
de Zubicaray, G., McMahon, K., Eastburn, M., Pringle, A., & Lorenz, L. (2006). Classic identity
negative priming involves accessing semantic representations in the left anterior temporal
cortex. NeuroImage, 33(1), 383–390.
de Zubicaray, G. I., McMahon, K. L., Eastburn, M. M., & Wilson, S. J. (2002). Orthographic-
phonological facilitation of naming responses in the picture-word task: An event-related
fMRI study using overt vocal responding. NeuroImage, 16(4), 1084–1093.
de Zubicaray, G. I., McMahon, K., & Howard, D. (2015). Perfusion fMRI evidence for priming
of shared feature-to lexical connections during cumulative semantic interference in spoken
word production. Language and Cognitive Neuroscience, 30(3), 261–272.
ference effect in the picture-word paradigm: An event-related fMRI study employing overt
responses. Human Brain Mapping, 14(4), 218–227.
Dell, G. S., Schwartz, M. F., Nozari, N., Faseyitan, O., & Coslett, H. B. (2013). Voxel-based
word production in aphasia. Cognition, 128, 380–396.
Dell’Acqua, R., Sessa, P., Peressotti, F., Mulatti, C., Navarrete, E., & Grainger, J. (2010). ERP
evidence for ultra-fast semantic processing in the picture-word interference paradigm.
Detre, J. A., Rao, H., Wang, D. J., Chen, Y. F., & Wang, Z. (2012). Applications of arterial spin la-
beled MRI in the brain. Journal of Magnetic Resonance Imaging, 35, 1026–1037.
Diaz, M. T., Hogstrom, L. J., Zhuang, J., Voyvodic, J. T., Johnson, M. J. & Camblin, C. C. (2014).
The influence of written distractor words on brain activity during overt picture naming.
Frontiers in Human Neuroscience, 8, 167.
Duss, S. B., Reber, T. P., Hänggi, J., Schwab, S., Wiest, R., Müri, R.M., et al. (2014). Unconscious
relational encoding depends on hippocampus. Brain, 27, 3355–3370.
Fargier, R., Bürki, A., Pinet, S., Alario, F.-X., & Laganaro, M. (2017). Word onset phonetic
properties and motor artifacts in speech production EEG recordings. Psychophysiology,
55(2), e12982. doi: 10.1111/psyp.12982
Galgano, J., & Froud, K. (2008). Evidence of the voice- related cortical potential: An
electroencephalographic study. NeuroImage, 41, 1313–1323.
Ganushchak, L. Y., Christoffels, I. K., & Schiller, N. O. (2011). The use of electroencephalog-
raphy in language production research: A review. Frontiers in Psychology, 2, 208.
Gauvin, H., Meinzer, M., & de Zubicaray, G. (2017). tDCS effects on word production: Limited
by design? Comment on Westwood et al. (2017). Cortex, 96, 137–142.
Geranmayeh, F., Wise, R. J., Mehta, A., & Leech, R. (2014). Overlapping networks engaged
during spoken language production and its cognitive control. Journal of Neuroscience, 34,
8728–8740.
spoken production. Cognition, 102, 219–260
Gracco, V. L., Tremblay, P, & Pike, G. B. (2005). Imaging speech production using fMRI.
Hartwigsen, G. (2014). The neurophysiology of language: Insights from non-invasive brain
stimulation in the healthy human brain. Brain & Language, 148,  81–94.
Harvey, D., & Schnur, T. T. (2015). Distinct loci of lexical and semantic access deficits in
aphasia: Evidence from voxel-based lesion-symptom mapping and diffusion tensor im-
aging. Cortex, 67,  37–58.
Henke, K. (2010). A model for memory systems based on processing modes rather than con-
sciousness. Nature Reviews Neuroscience, 11(7), 523–532.
Henseler, O., Mädebach, A., Kotz, S. A., & Jescheniak, J. D. (2014). Modulating brain
mechanisms resolving lexico-semantic interference during word production: A transcranial
direct current stimulation study. Journal of Cognitive Neuroscience, 26(7), 1403–1417.
Hirschfeld, G., Jansma, B., Bölte, J., & Zwitserlood, P. (2008). Interference and facilitation
in overt speech production investigated with event-related potentials. Neuroreport, 19,
1227–1230.
Hocking, J., McMahon, K. L., & de Zubicaray, G. I. (2009). Semantic context and visual feature
effects in object naming: An fMRI study using arterial spin labeling. Journal of Cognitive
Neuroscience, 21(8), 1571–1583.
Hocking, J., McMahon, K. L., & de Zubicaray, G. I. (2010). Semantic interference in object
naming: An fMRI study of the postcue naming paradigm. NeuroImage, 50(2), 796–801.
Howard, D., Nickels, L., Coltheart, M., & Cole-Virtue, J. (2006). Cumulative semantic inhibi-
tion in picture naming: Experimental and computational studies. Cognition, 100, 464–482.
Indefrey, P. (2011). The spatial and temporal signatures of word production components: A
critical update. Frontiers in Psychology, 2, 255.
Indefrey, P. (2016). On putative shortcomings and dangerous future avenues: Response to
Strijkers & Costa. Language, Cognition & Neuroscience, 31, 517–520.
Indefrey, P., & Levelt, W. J. M. (2000). The neural correlates of language production. In M.
S. Gazzaniga (Ed.), The new cognitive neurosciences (2nd ed., pp. 845–865). Cambridge,
MA: MIT Press.
components. Cognition, 92, 101–144.
Janssen, N., Carreiras, M., & Barber, H. A. (2011). Electrophysiological effects of semantic con-
text in picture and word naming. NeuroImage, 57, 1243–1250.
Janssen, N., Hernández-Cabrera, J. A., van der Meij, M., & Barber, H. A. (2015). Tracking the
time course of competition during word production: Evidence for a post-retrieval mech-
anism of conflict resolution. Cerebral Cortex, 25(9), 2960–2969.
Kemeny, S., Ye, F. Q., Birn, R., & Braun, A. R. (2005). Comparison of continuous overt speech
fMRI using BOLD and arterial spin labeling. Human Brain Mapping, 24, 173–183.
Krieger-Redwood, K., & Jefferies, E. (2014). TMS interferes with lexical-semantic retrieval in
left inferior frontal gyrus and posterior middle temporal gyrus: Evidence from cyclical pic-
ture naming. Neuropsychologia, 64,  24–32.
tion. Behavioral and Brain Sciences, 22(1),  1–38.
Llorens, A., Trébuchon, A., Liégeois-Chauvel, C., & Alario F-X. (2011). Intracranial recordings
of brain activity during language production. Frontiers in Psychology, 2, 375.
Llorens, A., Dubarry, A.-S., Trébuchon, A., Chauvel, P., Alario, F.-X., & Liégeois-Chauvel, C.
(2016). Contextual modulation of hippocampal activity during picture naming. Brain and
Language, 159, 92–101. doi: 10.1016/j.bandl.2016.05.011
Llorens, A., Trebuchon, A., Riès, S., Alario, F.-X., & Liegeois-Chauvel, C. (2014). How fa-
miliarization and repetition modulate the picture naming network. Brain & Language,
133,  47–58.
Maess, B., Friederici, A. D., Damian, M., Meyer, A. S., & Levelt, W. J. (2002). Semantic cate-
gory interference in overt picture naming: Sharpening current density localization by PCA.
Mahon, B. Z., Costa, A., Peterson, R., Vargas, K. A., & Caramazza, A. (2007). Lexical selection
is not by competition: A reinterpretation of semantic interference and facilitation effects
in the picture–word interference paradigm. Journal of Experimental Psychology: Learning,
Memory, & Cognition, 33, 503–535.
Mehta, S., Grabowski, T. J., Razavi, M., Eaton, B., & Bolinger, L. (2006). Analysis of speech-
related variance in rapid event- related fMRI using a time- aware acquisition system.
NeuroImage, 29, 1278–1293.
Meinzer, M., Antonenko, D., Lindenberg, R., Hetzer, S., Ulm, L., Avirame, K., . . . Flöel, A.
(2012). Electrical brain stimulation improves cognitive performance by modulating func-
tional connectivity and task-specific activation. Journal of Neuroscience, 32(5), 1859–1866.
Meinzer, M., Yetim, O., McMahon, K., & de Zubicaray, G. (2016). Brain mechanisms of se-
mantic interference in spoken word production: An anodal transcranial direct current stim-
ulation (atDCS) study. Brain & Language, 157–158,  72–80.
Miniussi, C., Harris, J. A., & Ruzzoli, M. (2013). Modelling non-invasive brain stimulation in
cognitive neuroscience. Neuroscience and Biobehavioral Reviews, 37(8), 1702–1712.
Mirman, D., & Graziano, K. M. (2013). The neural basis of inhibitory effects of semantic and
phonological neighbors in spoken word production. Journal of Cognitive Neuroscience,
25(9), 1504–1516.
Mottaghy, F. M., Sparing, R., & Topper, R. (2006). Enhancing picture naming with transcranial
magnetic stimulation. Behavioral Neurology, 17(3–4), 177–186.
Mulatti, C., Peresotti, F., Job, R., Saunders, S. & Coltheart, M. (2012). Reading aloud: The cumu-
lative lexical interference effect. Psychonomic Bulletin & Review, 19, 662–667.
Munding, D., Dubarry, A.-S., & Alario, F.-X. (2016). On the cortical dynamics of word produc-
tion: A review of the MEG evidence. Language, Cognition and Neuroscience, 31(4), 441–462.
doi:10.1080/23273798.2015.1071857
Nebel, K., Stude, P., Wiese, H., Müller, B., de Greiff, A., Forsteing, M., Diener, H.C., Keidel, M.
(2005): Sparse imaging and continuous event-related fMRI in the visual domain: A system-
atic comparison. Human Brain Mapping, 24, 130–143.
Oppenheim, G. M., Dell, G. S., & Schwartz, M. F. (2010). The dark side of incremental
learning: A model of cumulative semantic interference during lexical access in speech pro-
duction. Cognition, 114, 227–252.
Ouyang, G., Sommer, W., Zhou, C., Aristei, S., Pinkpank, T., & Abdel Rahman, R.
(2016). Articulation artifacts during overt language production in event-related brain
potentials: Description and correction. Brain Topography, 29(6), 791–813.
Peramunage, D., Blumstein, S. E., Myers, E., Goldrick, M., & Baese- Berk, M. (2011).
Piai, V. (2016). The role of electrophysiology in informing theories of word production: A crit-
ical standpoint. Language, Cognition and Neuroscience, 31, 471–473.
Piai, V., Dahlslätt, K., & Maris, E. (2015). Statistically comparing EEG/MEG waveforms through
successive significant univariate tests: How bad can it be? Psychophysiology, 52, 440–443.
Piai, V., & Knight, R. T. (2018). Lexical selection with competing distractors: Evidence from left
temporal lobe lesions. Psychonomic Bulletin & Review, 25, 710–7 17.
Piai, V., Riès, S. K., & Knight, R.T. (2015). The electrophysiology of language production: What
could be improved. Frontiers in Psychology, 5, 5160.
Piai, V., Riès, S. K., & Swick, D. (2016). Lesions to lateral prefrontal cortex impair interference
control in word production. Frontiers in Human Neuroscience, 9, 721.
Piai, V., Roelofs, A., Acheson, D. J., & Takashima, A. (2013). Attention for speaking: Domain-
general control from anterior cingulate cortex in spoken word production. Frontiers in
Human Neuroscience, 7, 832.
Piai, V., Roelofs, A., Jensen, O., Schoffelen, J.M., & Bonnefond, M. (2014). Distinct patterns of
brain activity characterise lexical activation and competition in spoken word production.
PLoS One, 9(2), e88674.
Piai, V., Roelofs, A., & Van der Meij, R. (2012). Event-related potentials and oscillatory brain
responses associated with semantic and Stroop-like interference effects in overt naming.
Brain Research, 1450, 87–101.
Pisoni, A., Cerciello, M., Cattaneo, Z., & Papagno, C. (2017). Phonological facilitation in pic-
ture naming: When and where? A tDCS study. Neuroscience, 352, 106–121.
Pisoni, A., Papagno, C., & Cattaneo, Z. (2012). Neural correlates of the semantic interference
effect: New evidence from transcranial direct current stimulation. Neuroscience, 223,  56–67.
Porcaro, C., Medaglia, M. T., & Krott, A. (2015). Removing speech artifacts from
electroencephalographic recordings during overt picture naming. NeuroImage, 105, 171–180.
Python, G., Fargier, R., & Laganaro, M. (2017). ERP evidence of distinct processes underlying
semantic facilitation and interference in word production. Cortex, 99, 1–12. https://doi.org/
10.1016/j.cortex.2017.09.008
Riès, S. (2016). Serial versus parallel neurobiological processes in language produc-
tion: Comment on Munding, Dubarry, and Alario, 2015. Language, Cognition, and
Neuroscience, 31(4), 476–479. doi: 10.1080/23273798.2015.1117644
Riès, S. K., Dhillon, R. K., Clarke, A., King-Stephens, D., Laxer, K.D., Weber, P.B., . . . & Lin, J.J.
(2017). Spatiotemporal dynamics of word retrieval in speech production revealed by cor-
tical high-frequency band activity. Proceedings of the National Academy of Sciences USA, 114,
E4530–E4538.
Riès, S. K., Greenhouse, I., Dronkers, N. F., Haaland, K. Y., & Knight, R. T. (2014). Double dis-
sociation of the roles of the left and right prefrontal cortices in anticipatory regulation of
action. Neuropsychologia, 63, 215–225.
Riès, S., Janssen, N., Burle, B., & Alario, F.-X. (2013). Response-locked brain dynamics of word
production. PLoS One, 8(3), e58197.
Riès, S., Karzmark, C., Navarrete, E., Dronkers, N., & Knight, R.T. (2015). Specifying the role of
the left prefrontal cortex in word selection. Brain & Language, 149, 135–147.
Riès, S., Xie, K., Haaland, K., Dronkers, N., & Knight, R. T. (2013). Role of the lateral prefrontal
cortex in speech monitoring. Frontiers in Human Neuroscience, 7, 703.
Rizio, A. A., Moyer, K. J., & Diaz, M. T. (2017). Neural evidence for phonologically-based lan-
guage production deficits in older adults: An fMRI investigation of age-related differences in
picture-word interference. Brain and Behavior, 7(4), e00660. doi: 10.1002/brb3.660
Rose, S. B., & Abdel Rahman, R. (2016). Semantic similarity promotes interference in the
continuous naming paradigm: Behavioural and electrophysiological evidence. Language,
Cognition and Neuroscience, 32,  55–68.
Rosinski, R. R., Golinkoff, R. M., & Kukish, K. S. (1975). Automatic semantic processing in a
picture-word interference task. Child Development, 46, 247–253.
Schnur, T. T., Schwartz, M. F., Kimberg, D. Y., Hirshorn, E., Coslett, H. B., & Thompson-Schill,
S. L. (2009). Localizing interference during naming: Convergent neuroimaging and neuro-
psychological evidence for the function of Broca’s area. Proceedings of the National Academy
of Sciences USA, 106, 322–327.
Schuhmann, T., Schiller, N. O., Goebel, R., & Sack, A. T. (2009). The temporal characteristics
of functional activation in Broca’s area during overt picture naming. Cortex, 45(9), 1111–1116.
Schuhmann, T., Schiller, N. O., Goebel, R., & Sack, A. T. (2012). Speaking of which: Dissecting
the neurocognitive network of language production in picture naming. Cerebral Cortex, 22,
701–709.
Schwartz, M. F., Faseyitan, O., Kim, J., & Coslett, H. B. (2012). The dorsal stream contribution
to phonological retrieval in object naming. Brain, 135, 3799–3814.
Spalek, K., & Thompson-Schill, S. L. (2008). Task-dependent semantic interference in lan-
guage production: An fMRI study. Brain & Language, 107, 220–228.
Strijkers, K. (2016). Can hierarchical models display parallel cortical dynamics? A non-
hierarchical alternative of brain language theory. Language, Cognition and Neuroscience, 31,
465–469.
Strijkers, K., & Costa, A. (2016). The cortical dynamics of speaking: Present shortcomings and
future avenues. Language, Cognition and Neuroscience, 31(4), 484–503.
Tipper, S. P. (1985). The negative priming effect: Inhibitory priming by ignored objects.
Quarterly Journal of Experimental Psychology, 37A, 571–590.
Töpper, R., Mottaghy, F. M., Brugmann, M., Noth, J., & Huber, W. (1998). Facilitation of picture
naming by focal transcranial magnetic stimulation of Wernicke’s area. Experimental Brain
Research, 121(4), 371–378.
Tremblay, P., & Small, S. L. (2011). Motor response selection in overt sentence production: A
functional MRI study. Frontiers in Psychology, 2, 253.
Wang, M., Shao, Z., Chen, Y., & Schiller, N. O. (2017). Neural correlates of spoken word pro-
duction in semantic and phonological blocked cyclic naming. Language, Cognition and
Neuroscience, 33, 575–586. doi: 10.1080/23273798.2017.1395467
Westwood, S. J., Olson, A., Miall, R. C., Nappo, R., & Romani, C. (2017). Limits to tDCS effects
in language: Failures to modulate word production in healthy participants with frontal or
temporal tDCS. Cortex, 86,  64–82.
Wheat, K. L., Cornelissen, P. L., Sack, A. T., Schuhmann, T., Goebel, R., & Blomert, L. (2013).
Charting the functional relevance of Broca’s area for visual word recognition and picture
naming in Dutch using fMRI-guided TMS. Brain & Language, 125(2), 223–230.
Wilson, S. M., Isenberg, A. L., & Hickok, G. (2009). Neural correlates of word production
stages delineated by parametric modulation of psycholinguistic variables. Human Brain
Mapping, 30, 3596–3608.
Wirth, M., Abdel Rahman, R., Kuenecke, J., Koeniga, T., Horn, H., Sommer, W., & Dierks, T.
(2011). Effects of transcranial direct current stimulation (tDCS) on behaviour and electro-
physiology of language production. Neuropsychologia, 49, 3989–3998.
Zhu, X., Zhang, Q., & Damian, M. F. (2016). Additivity of semantic and phonological
effects: Evidence from speech production in Mandarin. Quarterly Journal of Experimental
Psychology, 69, 2285–2304.
Chapter 20
The D orsa l St re a m
Au ditory-Motor
Interface for  Spe e c h
Gregory Hickok
As the title forecasts, this chapter is about the dorsal auditory-motor speech interface.
This network is part of a larger system involved in auditory, speech, and language pro-
cessing that is roughly organized into two main streams, referred to as the dorsal and
ventral streams, corresponding to their locations on the lateral cortex. We review the
organization of the broader dual-stream context, including its historical foundation.
We then turn to recent progress in mapping the dorsal stream and understanding its
computational basis. A current hypothesis is that the dual stream supports speech pro-
duction via a feedback-control mechanism. This line of investigation offers the hope of
integrating psycholinguistic, neurolinguistic, and motor control research.
Dual-Stream Models:
A Computational Necessity
The brain must perform at least two tasks with incoming sensory information: (1) map it
onto conceptual representations for the purpose of recognizing what is being perceived,
and (2) map it onto motor representations for the purpose of generating an appropriate
action in response to the information. This makes intuitive sense for visual perception,
where, for example, one needs to recognize what an object is and how to use infor-
mation about size, shape, orientation, and distance to control a limb for reaching and
grasping the object (Milner & Goodale, 1995). In the case of speech, these two kinds of
mappings are equally necessary, which led to the development of the dual-stream model
of speech processing (Hickok & Poeppel, 2000, 2004, 2007) (Figure 20.1). For example,
one needs to be able to recognize what words are being spoken, and one needs to be
The Dorsal Stream Auditory-Motor Interface for Speech 499
(a) Via higher-order frontal networks
Articulatory network Sensorimotor interface Input from

pIFG, PM, anterior insula Parietal–temporal Spt other sensory
(left dominant) Dorsal stream (left dominant) modalities
Spectrotemporal analysis Phonological network

Conceptual network
Dorsal STG Mid-post STS
Widely distributed
(bilateral) (bilateral)
Combinatorial network Ventral stream Lexical interface

aMTG, aITS pMTG, pITS
(left dominant?) (weak left-hemisphere bias)
(b)
Figure 20.1. (A) Schematic diagram of the dual-stream model. Phonological network diverges
into two streams: a dorsal sensorimotor stream supporting speech motor control and a ven-
tral sensory-conceptual stream supporting comprehension. The relation between auditory-
phonological and articulatory representations, on one hand, and auditory-phonological and
conceptual representations, on the other, is mediated by distinct interface systems, the “sen-
sorimotor interface” and the “lexical interface.” (B) Approximate anatomical locations of the
dual-stream model components. Regions shaded pink represent the more bilaterally organized
ventral stream. Regions shaded blue represent the dorsal stream, which is strongly left dominant.
Functional area Spt (Sylvian parietal temporal) is the posterior-most blue shaded area.
pIFG, posterior inferior frontal gyrus; PM, premotor; Spt, Sylvian parietal-temporal; STG, superior
temporal gyrus; STS, superior temporal sulcus; aMTG; anterior middle temporal gyrus; aITS, anterior
inferior temporal sulcus; pMTG, posterior middle temporal gyrus; pITS, posterior inferior temporal sulcus.
able to reproduce the words (and sentence patterns, etc.) of the language with one’s own
speech articulatory system.
The need to be able to reproduce perceived speech is most obvious in development,
where the young child must perceive the sound patterns in the surrounding linguistic
environment and learn to reproduce those patterns him-or herself (Doupe & Kuhl,
1999). But much evidence has shown that acoustic input plays a critical role in several
aspects of speech production throughout life, including maintenance and fine-tuning
of articulatory patterns (Waldstein, 1989), compensating for changing vocal tract state
or environmental perturbations either experimentally (Houde & Jordan, 1998; Larson,
Burnett, Bauer, Kiran, & Hain, 2001; Perkell, 2012) or in natural situations, such as
500 Gregory Hickok
talking with food the mouth, and in speech motor planning (Hickok, 2012; Hickok,
Houde, & Rong, 2011; Houde & Nagarajan, 2011), as we will see in the following sections.
Given that sensory information must be interfaced with conceptual systems, on one
hand, and motor systems, on the other, and given that these are largely distinct com-
putational tasks, it is no surprise that dual-stream models have been proposed in all
the major sensory modalities and have been proposed repeatedly over the last century
and a half (Dijkerman & de Haan, 2007; Hickok & Poeppel, 2004; Ingle, 1973; Milner &
Goodale, 1995; Poljak, 1926; Rauschecker & Scott, 2009; Wernicke, [1874] 1969).
Dual-Stream Models:
Historical Context
Although most students and practitioners of cognitive neuroscience today trace the
roots of dual-stream processing models to the pioneering work of Ungerleider and
Mishkin and their “two visual system” proposal—the what versus where distinction
(Ungerleider & Mishkin, 1982)—the idea was far from new. More than a century be-
fore Ungerleider and Mishkin’s influential paper, Wernicke proposed a similar func-
tional distinction for language: one stream that associates sound with concepts and
another that associates sound with the motor speech system. And it wasn’t just the lan-
guage scientists who were on to the idea in the nineteenth century. Research coming
out of Wilhelm Wundt’s lab in the 1880s clearly makes a distinction between “appercep-
tion,” the conscious recognition of the stimulus, and the ability of a stimulus to trigger
a motor response more directly (discussed in Neumann, 1990). Ludwig Lange, for ex-
ample, distinguished between muscular and sensorial reaction time in 1888, and Hugo
Munsterberg underlined this perspective, stating,
When we apperceive the stimulus, we have as a rule already started responding to

it. Our motor apparatus does not wait for our conscious awareness, but does rest-
lessly its duty, and our consciousness watches it and has no right to give it orders.
(Munsterberg, 1889, p. 173, cited and translated by Neumann, 1990)
Twentieth-century research on the visual systems of frogs (Ingle, 1973), rodents

(Schneider, 1969), and primates (Trevarthen, 1968) prior to Ungerleider and Mishkin’s
work also led to the conclusion that the visual system is split into two parallel streams.
And, as we will see, Poljak concluded in a 1926 publication that the auditory system has
a “double function.”
The take-home message here is that the concept of dual processing streams has been
arrived at from multiple perspectives (neuropsychology, psychophysics, neuroim-
aging), in multiple domains (notably language, vision, and audition), and repeatedly
throughout the course of history. This suggests that there is an important generalization
here. In fact, it has been suggested to be a fundamental principle of the cortical organi-
zation of sensory systems (Hickok & Poeppel, 2007), presumably due to the computa-
tional necessity of the distinct mapping functions.
We turn next to more specific dual-stream models in vision and audition.
Dual Streams in Vision
As noted earlier, the idea that the visual system is computationally bifurcated was well
established by the time Ungerleider and Mishkin published their hugely influential
chapter on the “Two Cortical Visual Systems” in 1982. They write,
It has been our working hypothesis (Mishkin 1972; Pohl 1973) that the ventral or
occipitotemporal pathway is specialized for object perception (identifying what an
object is) whereas the dorsal or occipitoparietal pathway is specialized for spatial
perception (locating where an object is). This distinction between the two types of
visual perception is not new (see, for example, Ingle 1967; Held 1968). (Ungerleider &
Mishkin, 1982, p. 549)
What the 1982 paper did was make the case that the two streams in the macaque monkey
were entirely cortical, that their origin was the striate visual cortex. There were two basic
observations: first, that damage to ventral temporal regions resulted in deficits in using
object identity to perform a task, whereas damage to parietal regions resulted in deficits
in using spatial cues to perform a task (Pohl, 1973); and second, that lesions of striate
cortex interacted with the temporal and parietal lesions, thus pointing to the source of
information flowing into the two streams (Ungerleider & Mishkin, 1982). It was noted
by Ungerleider and Mishkin that spatial discrimination was not the only symptom
observed in monkeys with parietal lesions; they also exhibited misreaching deficits,
along with other classic parietal symptoms such as contralateral neglect and tactile dis-
crimination. But it took another decade for the action-related deficits to surpass the
spatial deficits in theoretical prominence in the form of Milner and Goodale’s reformu-
lation of the functional role of the dorsal stream into a “how” (sensorimotor) stream
(Milner & Goodale, 1995; see also Neumann, 1990, who proposed a similar subdivision).
Evidence for the existence of what (recognition) versus how (sensorimotor) dual-
processing streams in vision comes from a variety of sources. Milner and Goodale’s book
is still a nice review of much of the foundational data (Milner & Goodale, 1995). Some of
the strongest arguments come from neuropsychology, where double dissociations can
be found. Patients with visual form agnosia have severe deficits in recognizing objects by
sight (tactile and auditory recognition is intact), yet are still able to interact with objects
motorically in an appropriate way. A dramatic, and now famous case, patient D. F., was
reported by Milner and colleagues, which clearly illustrates the dissociation (Milner
et al., 1991; see Karnath, Ruter, Mandler, & Himmelbach, 2009, for recent brief review).
D. F. was unable to consciously indicate the orientation of a slot, but was able to “post”
a card accurately through that same slot. Patients with visual form agnosia have ventral
occipito-temporal lobe damage consistent with the ventral stream claim. However, most
cases suffered brain damage due to carbon monoxide intoxication, which produces
diffuse damage, and this has raised concerns regarding the underlying neuroanatomy
(Karnath et al., 2009). But the functional dissociations remain clear, and a recent case of
a stroke patient with focal ventral temporal lesions has been reported who exhibits the
same pattern of dissociation (Karnath et al., 2009).
In contrast to visual agnosics, patients with optic ataxia are able to recognize objects
by sight but have substantial difficulty in generating accurate reaching for and grasping
of objects. They tend to grope for their targets instead of using visual guidance to de-
termine trajectory and anticipatory grasp shaping of the hand. Lesions associated with
optic ataxia have a posterior parietal lobe focus, consistent with the dorsal stream claim
(Perenin & Vighetto, 1988).
The deficit in optic ataxia is not absolute, however. It mostly emerges when reaching
for objects in the visual periphery, or when the reach requires rapid adjustments due
to changing conditions, or when the object is unfamiliar (Rossetti, Pisella, & Vighetto,
2003)—precisely those conditions when the particulars of the object, as opposed to the
what of the object, are critical for guiding action. This is an important point that has led
to some confusion in the literature and is worth spending a few lines on here. If asked to
demonstrate how to reach for a cup, one could do it in a fairly generic way. Most likely,
the reach would extend straight out in front, at an average height of a table and with an
average grip aperture. Presumably, we have a stored motor program for how to reach
for the average cup in its average location, and no visual information is needed to do
this. In a real reaching situation, where objects may not be of average size and in average
locations, one will have to modify the default reach, and for that one needs visual input.
Likewise, if a perturbation is introduced during the reach, a modification from the ge-
neric plan is required. These are the conditions that require analysis of visual details
about the particular shape and location of the object, as opposed to invariant object
properties, and these are the conditions that are most impaired in optic ataxia. The point
is that even though reaching deficits only show up under some conditions, this does not
dispute the existence of a sensorimotor system that is computationally tuned to guide
reaching as opposed to object recognition.
Additional evidence for a dual-stream model in vision comes from behavioral
experiments where it has been shown that psychophysical functions differ for
action-versus non-action perceptual tasks. One controversial demonstration is
that the Ebbinghaus illusion—object size perception that is dependent upon the
size of surrounding objects—does not fool the visuomotor system, or at least does
not fool it to the same degree (Aglioti, DeSouza, & Goodale, 1995; Haffenden &
Goodale, 1998).
The illusion-based findings have been critiqued quite heavily and remain contro-
versial (Franz & Gegenfurtner, 2008; Westwood & Goodale, 2011), but similar effects
have been reported in other psychophysical paradigms (for review, see Westwood &
Goodale, 2011), and thus behavioral data are, on balance, consistent with findings from
neuropsychology.
Dual Streams in Audition
Josef Rauschecker is often credited with idea that auditory cortex is functionally
subdivided into two processing streams, a dorsal “where” stream and a ventral “what”
stream (Rauschecker, 1998; Rauschecker & Scott, 2009), similar to previous claims in
the primate visual system (Ungerleider & Mishkin, 1982). However, the idea of dual au-
ditory streams predates Rauschecker’s influential papers by several decades. Deutsch
and Roll proposed separate “what” and “where” mechanisms for hearing in their 1976
report (Deutsch & Roll, 1976), citing then recent animal neurophysiological evidence
for the distinction (Evans & Nelson, 1973). And a historical precedent to a dual-stream
model of audition goes even farther back to the work of Poljak, who, in 1926, discussed
the various subdivisions in “the connections of the acoustic nerve” and came to a con-
clusion that foreshadowed current dual-stream ideas by the better part of a century:
The constituent parts of the central auditory system have mostly a double function—
viz. to conduct the peripheral auditory sensations to the prosencephalon [forebrain]
on the one hand, and on the other, to establish a reflex path for the cochlear stimuli to
the motor mechanisms of the brain stem. (Poljak, 1926, p. 468)
The data that drove Rauschecker’s claims, as well as those of Deutsch and Roll, con-
cerned the apparent separability of the neural response to the perception of auditory ob-
ject content versus location, hence the “what” versus “where” distinction. Rauschecker
noted that some cells were more responsive to the type of information presented (e.g.,
different monkey calls), while other cells were more responsive to the location of the
sound source (Tian, Reser, Durham, Kustov, & Rauschecker, 2001). Deutsch and Roll
came to their conclusion based on the observation that the dichotic fusion of the per-
ception of pitch dissociates from the perception of sound source.
While there is good agreement on the existence of a ventral “what” stream, evi-
dence for a dedicated “where” stream has been mixed (Bregman & Steiger, 1980; Lee &
Middlebrooks, 2013; Middlebrooks, 2002; Recanzone, Guard, Phan, & Su, 2000; Smith,
Hsieh, Saberi, & Hickok, 2010; Smith, Okada, Saberi, & Hickok, 2004; Smith, Saberi, &
Hickok, 2007; Warren & Griffiths, 2003; Zatorre, Bouffard, Ahad, & Belin, 2002). Most
of the dissenting voices have questioned the specificity of the spatial hearing system—
the idea that there is a dedicated cortical system for spatial hearing. For example, some
authors have argued that it is not a spatial computation per se that is carried out in the
purported “where” cortical stream, but rather a system that uses spatial information to
perform auditory scene segregation, which could easily be viewed as part of a “what”
stream (Smith et al., 2010; Zatorre et al., 2002). This debate will not concern us here.
Dual Streams for Speech
The historical origin of the dual-stream model for speech sits with Wernicke, as noted
earlier. Wernicke’s model has had its ups and downs since it was proposed in 1874. It
enjoyed great initial success with thoughtful elaborations by Lichtheim in 1885 and
extensions of the basic functional anatomic ideas into other domains such as apraxia
(Liepmann, 1908). But the model fell out of favor as the field shifted away from fairly
modular views of mental and neural function (Benton, 1991). Geschwind is credited
with reviving the classical model of the anatomy of language in the 1960s (Geschwind,
1965, 1971), but advances in linguistics (Chomsky, 1965, 1986) revealed the model’s
shortcomings as a complete explanation for the functional anatomy of language. Again,
Wernicke’s dual-stream model was largely abandoned, leaving the field without a
unifying framework.
The modern incarnation of the dual-stream model as proposed by Hickok and
Poeppel (2000, 2004, 2007) was inspired not so much by Wernicke (that connection was
noticed later), but by the Milner and Goodale dual-stream model in vision, which was
quite influential in the late 1990s. Just as Milner and Goodale had updated the “what”
versus “where” visual pathways to a “what” versus “how” model, Hickok and Poeppel
updated the Rauschecker “what” versus “where” auditory pathway model to a “what”
versus “how” framework for speech. Rauschecker and Scott (2009) have proposed their
own version of a dual-stream model, including a discussion of both “what” and “how”
functions. A recent lesion-based empirical test of the two models favors the Hickok and
Poeppel variant (Fridriksson et al., 2016).
Now we turn to a deeper discussion of the dorsal speech stream, including recent
developments concerning its anatomy and computational function.
Anatomy and Physiology of

the Auditory Dorsal Stream
The auditory dorsal stream for speech comprises a network of regions, identified prima-
rily on the basis of functional magnetic resonance imaging (fMRI) studies, that includes
auditory-related cortices in the superior temporal gyrus/sulcus, motor speech areas
in the posterior lateral frontal lobe, and a region in the posterior Sylvian fissure at the
parietal-temporal boundary (area Spt) (B. Buchsbaum, Hickok, & Humphries, 2001;
Hickok, Buchsbaum, Humphries, & Muftuler, 2003; Hickok, Okada, & Serences, 2009).
Spt is hypothesized to serve as the interface between auditory and motor representations
of speech (Hickok et al., 2011) in much the same way as regions of the posterior pari-
etal cortex of macaques and humans function as a visuomotor interface (Andersen,
1997; Gallese, Fadiga, Fogassi, Luppino, & Murata, 1997; Grefkes & Fink, 2005; Milner
& Goodale, 1995). Accordingly, Spt has been shown to exhibit auditory-motor response
properties, activating both during the perception and (covert) production of speech;
indeed, this is a defining property of functional circuit. Spt has also been found to be
relatively selective for vocal compared to manual auditory-motor actions. One fMRI
study compared Spt activation during listening to simple novel melodies and then either
silently humming them back or imagining playing them on a keyboard (participants
were skilled pianists): Spt activated more strongly during the humming condition de-
spite identical auditory input (Pa & Hickok, 2008). Spt also appears to be functionally
(B. Buchsbaum et al., 2001; B. R. Buchsbaum, Olsen, Koch, & Berman, 2005) and ana-
tomically (Isenberg, Vaden, Saberi, Muftuler, & Hickok, 2012) connected to premotor
regions involved in speech production, consistent with what is found for visuomotor
integration areas in the parietal lobe (Grefkes & Fink, 2005). It also exhibits distinct ac-
tivation patterns to auditory and motor phases of the task (Hickok et al., 2009), which
argues against a purely auditory or purely motor explanation for its activation pattern.
Damage to this region is associated with conduction aphasia, a syndrome
characterized by frequent sound-based errors, which are readily detected by the patient,
in otherwise fluent speech production, and difficulty with repeating speech verbatim,
particularly nonwords; auditory comprehension is largely preserved (Goodglass, 1992).
Functionally, one can conceptualize conduction aphasia as an auditory-motor integra-
tion deficit (B. R. Buchsbaum et al., 2011; Hickok et al., 2011). Patients are fluent because
frontal lobe motor speech networks are intact; they have good auditory comprehension
and can detect their own errors because the speech perception system is intact; what is
going wrong is the integration of the two, leading to phonemic errors and difficulty re-
peating speech. (We take up the details of how this works later in the chapter.) If it is true
that conduction aphasia is a deficit of auditory-motor integration and that Spt is a hub
in the auditory-motor system, then the lesions in conduction aphasia should involve
the Spt region. This is precisely what was found in a recent comparative analysis of Spt
functional activation data from more than 100 healthy participants and lesion data from
14 individuals with conduction aphasia (B. R. Buchsbaum et al., 2011). A more recent
large-scale lesion analysis of word and nonword repetition more specifically found that
repetition impairment is associated with the Spt region (Rogalsky et al., 2015).
As noted, Spt is located within the Sylvian fissure at its posterior-most extent, and
in individual subjects typically involves the posterior-medial portions of the planum
temporale and/or the parietal operculum, but which can extend laterally toward the
crown of the superior temporal and/or supramarginal gyri. Cytoarchitectonically, this
location corresponds to area Tpt, a region not considered part of unimodal auditory
cortex (Galaburda & Sanides, 1980; Smiley et al., 2007; Sweet, Dorph-Petersen, & Lewis,
2005) and therefore consistent with the claim that Spt is not solely auditory but rather
auditory-motor.
A recent large-scale lesion study questioned the role of Spt in speech motor planning,
suggesting that damage to the region did not predict increases in the number of phono-
logical speech errors in aphasia (Dell, Schwartz, Nozari, Faseyitan, & Branch Coslett,
2013). But this appears to be due to a misspecification of the Spt region of interest in their
analysis. An examination of their Figure 4 shows quite clearly that the posterior planum
temporale regions is indeed implicated in phonological-level speech errors in aphasia.
This study, therefore, provides additional evidence for the role of Spt is speech motor
planning at the phonological level.
The anatomical connection between Spt and posterior motor areas is presumed to
involve dorsal temporo-parietal-frontal white matter bundles, including the arcuate
fasciculus and the superior longitudinal fasciculus (Isenberg et al., 2012; Yagmurlu,
Middlebrooks, Tanriover, & Rhoton, 2016). Some models, and one in particular,
dubbed “Lichtheim 2,” have suggested a role for ventral temporo-frontal pathways,
such as the uncinate fasciculus, in speech production (Ueno, Saito, Rogers, & Lambon
Ralph, 2011). Although this proposal has been questioned empirically (Roelofs, 2014),
it is not inconsistent with the view that the dorsal speech stream and its dorsal white
matter connections are a critical part of the auditory-motor speech-production net-
work. Specifically, the dual-stream model has never assumed that the dorsal stream is
the only pathway for speech production (note the three separate inputs to the articu-
latory system in Figure 20.1 and see later in the chapter for computational arguments).
It is quite possible, therefore, that ventral stream temporo-frontal pathways play a role
speech planning, perhaps serving to activate high-level action plans via lexical-semantic
or combinatorial-semantic representations, which are then fine-tuned by lower-level
auditory-motor circuits (see following discussion).
Computational/Functional
Hypotheses
It has been repeatedly suggested that Spt is an auditory-motor integration area. But what
does that mean precisely? And why is auditory-motor integration useful for speech pro-
duction? Starting with the second question, it is becoming increasingly clear that audi-
tory representations serve as a critical target in speech planning (Guenther, Hampson,
& Johnson, 1998; Hickok, 2012, 2014). It is not the only target, as somatosensory
representations also play a demonstrable role (Hickok, 2012; Tremblay, Shiller, & Ostry,
2003). A useful way to think about the notion of auditory targets for speech planning
is to consider manual grasp planning. When reaching for a cup on a table, the target of
that motor gesture is essentially visual—the size, shape, location, and orientation of the
cup relative to the limb—and the task of the motor planning system is to configure the
limb such that it matches the visual features of the target. If there is no cup, there is no
visual representation and thus no target. Speech appears to be different in that there is
nothing overtly present in the acoustic environment that we direct our speech gestures
to “hit.” But consider a situation where you place your cup in the same location every
day while, say, working on your computer. You don’t need to look at the cup to reach for
it; in fact, you could do it with your eyes closed. What you do need, however, is a mental
representation of that target, which in the case of the cup is the same size, shape, orien-
tation, and in the same location every day. In this case, what we are reaching for as our
target is a stored representation of the cup’s location, and so on. This is what we are doing
in speech—“reaching” for a stored representation of the sound pattern of the words
we are trying to articulate. We know that this is the case because if one experimentally
manipulates the acoustic targets, for example using altered auditory feedback (Houde &
Jordan, 1998), this changes the form of the articulated speech.
The auditory dorsal stream, then, provides the neural means to translate auditory
speech targets into accurate motor speech plans. How does it do this? One emerging
hypothesis is that it does this via state feedback control (Hickok et al., 2011; Houde
& Nagarajan, 2011). Feedback-control models generally are those that use sensory
feedback to update motor plans. A limitation of models that rely only on overt sen-
sory feedback alone is that such feedback is delayed in time. State feedback-control
models add a further component, namely, an internal model or prediction of the
current dynamic state (position and trajectory) of the motor effector, in our case the
vocal tract. Overt sensory feedback is used to train the internal model such that it
can make predictions (often called “forward” predictions) about the current state of
the motor effector given its previous state and current motor commands. This allows
the system to make predictions regarding the sensory consequences of motor plans,
which facilitates error detection and correction. It has been argued further that state
feedback control in speech planning allows for error detection and correction even
prior to articulation (Hickok, 2012). The basic idea is that during speech planning, a
target word form is activated in sensory cortex and a motor plan is activated in motor
cortex; the sensory consequences of the motor plan can then be checked against the
target via forward prediction and corrected, if need be, prior to articulation. The basic
computational architecture has been extended in a further model called hierarchical
state feedback control that goes beyond auditory-motor interaction by including a
lower hierarchical planning level involving somatosensory targets in somatosensory
cortex, motor coding in primary or pre-motor cortex, and an interface involving the
cerebellum (Hickok, 2012).
Coming back to the role of Spt in this computational framework—our first question
at the beginning of this section—it has been argued that Spt is part of the internal model,
specifically, the component that computes a transform between auditory and motor
speech representations (Hickok, 2012; Hickok et al., 2011). Figure 20.2 presents a graph-
ical depiction of the proposed architecture (Hickok et al., 2011). The model assumes that
in the mature system (i.e., one in which the internal model has been robustly learned)
auditory target and motor plans can be activated in parallel via inputs from the lexical
system, which may be achieved anatomically via distinct white matter pathways. Motor
plans are then checked against their auditory targets via forward prediction, and cor-
rection signals are generated as needed. Anatomically, it is assumed that the auditory
phonological system localizes to the superior temporal sulcus, that the motor phono-
logical system localizes to premotor cortex, and the auditory-motor translation system
localizes to area Spt.
Articulatory
Vocal tract Speech
control
Motor Auditory
Phonological Auditory- Phonological
System motor System
Vocal tract state translation Sensory targets/
estimation prediction
Internal model
Predict
Lexical-Conceptual
Correct System
Figure 20.2. A state feedback-control model for speech processing.

Source: Reprinted with permission from Hickok et al. (2011).
A Computational Test of One

Component of the State
Feedback-C ontrol Architecture
From a psycholinguistic standpoint, the idea that the phonological system is split into
two components is not typical of most models of speech production, including the dom-
inant computational approaches (Dell, 1986; Dell, Schwartz, Martin, Saffran, & Gagnon,
1997; Levelt, 1999; Levelt, Roelofs, & Meyer, 1999). There is some precedent, however, in
the neuropsychological literature where patterns of dissociation have led to the claim
that there exist distinct phonological input and output lexicons (Jacquemot, Dupoux,
& Bachoud-Levi, 2007). Still, one would like to see additional psycholinguistic/com-
putational evidence for the proposed architecture. One approach is to modify existing
successful computational models of speech production so that they approximate the
architecture of the state feedback-control model (i.e., contain separate auditory-and
motor-phonological components), and then assess whether the modified model better
explains the data.
The recently developed SLAM (semantic lexical auditory motor) computational
model does just this (Walker & Hickok, 2016a Figure 20.3). SLAM is a modification of
the semantic-phonological (S-P) model, which has proven quite successful in accounting
for distributions of naming errors in aphasia (Dell, Martin, & Schwartz, 2007; Dell et al.,
S-P Model SLAM Model
... ...
... ...
Semantic-Lexical weight
Semantic-Lexical weight
MAT RAT CAT DOG FOG LOG

Lexical-Auditory weight Lexical-Motor weight
MAT RAT CAT DOG FOG LOG LM weight < LA weight
m r k d f l a o t g m r k d f l a o t g
Lexical-Phonological weight
m r k d f l a o t g
Auditory-Motor weight
Figure 20.3. Graphic depiction of the semantic-phonological (S-P) and semantic lexical auditory motor (SLAM) models.
1997; Nozari, Kittredge, Dell, & Schwartz, 2010). The S-P model is a two-step model of
word production where the first step involves lexical or lexeme selection and the second
step involves phonological selection. The model has been implemented in a connectionist
network comprising three layers: an input semantic feature layer, a middle lexical selec-
tion layer, and an output phonological layer. By adjusting only two connection strength
parameters, the semantic-lexical weight and the lexical-phonological weight, the model
can approximate the pattern of errors in individual aphasic patients with reasonably high
accuracy.
SLAM made one modification to the S-P model: it split (actually duplicated) the pho-
nological layer into an auditory-and motor-phonological layer, with the latter serving
as the output layer. The question that was asked is whether SLAM provided a better fit to
the aphasia naming data than S-P. The answer was yes. Overall, SLAM did a better job of
fitting the data than S-P and substantially outperformed S-P for patients with conduc-
tion aphasia, which fits nicely with the preceding discussion regarding the importance
of auditory-motor interaction is speech production and its link to conduction aphasia
(Walker & Hickok, 2016a). This finding provides an additional source of support for the
architectural assumptions of the state feedback-control framework.
As with any new theory, SLAM has its detractors. Goldrick (2016) argues that
SLAM’s improvement over SP is merely a result of approximating a different model
of production in which the phonological layer is broken down, not into auditory and
motor components, but into lexical (higher-level) and post-lexical (lower-level) layers
(Goldrick & Rapp, 2007). This is an important point that highlights the fact that SLAM
is not a complete model of speech production in that it does not model all of the hier-
archical complexity of the system. For example, SLAM does not even attempt to model
all the layers proposed in the hierarchical version of the state feedback-control model
(Hickok, 2012). That said, a computational evaluation of Goldrick’s claims showed that
simply adding an additional post-lexical layer to the SP model does not yield the same
fit improvement over SP that was found with SLAM (Walker & Hickok, 2016b); it is the
additional layer arranged in a particular architecture that provides the boost.
Conclusions
Dual-stream frameworks for sensory processing have proven to be highly successful

and long-lived in vision and audition, suggesting that this broad architecture is a general
principle of sensory neurocomputational design. The success of such frameworks has
been shown in speech and language, perhaps as a prototypical case. Much progress has
been made over the last decade and half in mapping the neuroanatomy and computa-
tional function of the dorsal, sensorimotor speech stream. The field has progressed from
the simple auditory-motor connection proposed by Wernicke in the 1870s to the specifi-
cation of several nodes in a cortical network including temporal, temporoparietal junc-
tion (Spt), and posterior frontal regions and with new hypothesized computational roles
and anatomical pathway connectivities. These networks are beginning to make serious
contact with psycholinguistics models and with analogous networks outside of speech
and language. Given the trajectory of recent progress, there is every reason to believe we
will see significant further advances in the coming years.
References
Aglioti, S., DeSouza, J. F., & Goodale, M. A. (1995). Size-contrast illusions deceive the eye but
not the hand. Current Biology, 5(6), 679–685.
Andersen, R. (1997). Multimodal integration for the representation of space in the poste-
rior parietal cortex. Philosophical Transactions of the Royal Society of London B Biological
Sciences, 352, 1421–1428.
Benton, A. (1991). Aphasia: Historical perspectives. In M. T. Sarno (Ed.), Acquired aphasia (2nd
ed., pp. 1–26). San Diego, CA: Academic Press.
Bregman, A. S., & Steiger, H. (1980). Auditory streaming and vertical localiza-
tion: Interdependence of “what” and “where” decisions in audition. Perception and
Psychophysics, 28(6), 539–546.
Buchsbaum, B., Hickok, G., & Humphries, C. (2001). Role of left posterior superior temporal
gyrus in phonological processing for speech perception and production. Cognitive Science,
25, 663–678.
Buchsbaum, B. R., Baldo, J., Okada, K., Berman, K. F., Dronkers, N., D’Esposito, M., et al.
(2011). Conduction aphasia, sensory- motor integration, and phonological short- term
memory: An aggregate analysis of lesion and fMRI data. Brain and Language, 119(3), 119–128.
Buchsbaum, B. R., Olsen, R. K., Koch, P., & Berman, K. F. (2005). Human dorsal and ventral
auditory streams subserve rehearsal-based and echoic processes during verbal working
memory. Neuron, 48(4), 687–697.
Chomsky, N. (1965). Aspects of the theory of syntax. Cambridge, MA: MIT Press.
Chomsky, N. (1986). Knowledge of language: Its nature, origin, and use. New York: Praeger.
Dell, G. S. (1986). A spreading- activation theory of retrieval in sentence production.
Dell, G. S., Martin, N., & Schwartz, M. F. (2007). A case-series test of the interactive two-step
model of lexical access: Predicting word repetition from picture naming. Journal of Memory
and Language, 56(4), 490–520.
Dell, G. S., Schwartz, M. F., Martin, N., Saffran, E. M., & Gagnon, D. A. (1997). Lexical access in
aphasic and nonaphasic speakers. Psychological Review, 104, 801–838.
Dell, G. S., Schwartz, M. F., Nozari, N., Faseyitan, O., & Branch Coslett, H. (2013). Voxel-based
word production. Cognition, 128(3), 380–396.
Deutsch, D., & Roll, P. L. (1976). Separate “what” and “where” decision mechanisms in pro-
cessing a dichotic tonal sequence. Journal of Experimental Psychology: Human Perception &
Performance, 2(1),  23–29.
Dijkerman, H. C., & de Haan, E. H. (2007). Somatosensory processes subserving perception
and action. Behavioral and Brain Sciences, 30(2), 189–201; discussion 201–139.
Doupe, A. J., & Kuhl, P. K. (1999). Birdsong and human speech: Common themes and
mechanisms. Annual Review of Neuroscience, 22, 567–631.
Evans, E. F., & Nelson, P. G. (1973). On the relationship between the dorsal and ventral cochlear
nucleus. Experimental Brain Research, 17, 428–442.
Franz, V. H., & Gegenfurtner, K. R. (2008). Grasping visual illusions: Consistent data and no
dissociation. Cognitive Neuropsychology, 25(7–8), 920–950.
Fridriksson, J., Yourganov, G., Bonilha, L., Basilakos, A., Den Ouden, D. B., & Rorden,
C. (2016). Revealing the dual streams of speech processing. Proceedings of the National
Galaburda, A., & Sanides, F. (1980). Cytoarchitectonic organization of the human auditory
cortex. Journal of Comparative Neurology, 190, 597–610.
Gallese, V., Fadiga, L., Fogassi, L., Luppino, G., & Murata, A. (1997). A parietal-frontal circuit
for hand and grasping movements in the monkey: Evidence from reversible inactivation
experiments. In P. Thier & H.-O. Karnath (Eds.), Parietal lobe contributions to orientation in
3D space (pp. 255–270). Heidelberg: Springer-Verlag.
Geschwind, N. (1965). Disconnexion syndromes in animals and man. Brain, 88, 237–294,
585–644.
Geschwind, N. (1971). Aphasia. New England Journal of Medicine, 284, 654–656.
Goldrick, M. (2016). Integrating SLAM with existing evidence: Comment on Walker and
Hickok (2015). Psychonomic Bulletin & Review, 23(2), 648–652.
spoken production. Cognition, 102(2), 219–260.
Goodglass, H. (1992). Diagnosis of conduction aphasia. In S. E. Kohn (Ed.), Conduction
aphasia (pp. 39–49). Hillsdale, NJ: Lawrence Erlbaum Associates.
Grefkes, C., & Fink, G. R. (2005). The functional organization of the intraparietal sulcus in
humans and monkeys. Journal of Anatomy, 207(1),  3–17.
Guenther, F. H., Hampson, M., & Johnson, D. (1998). A theoretical investigation of reference
frames for the planning of speech movements. Psychological Review, 105, 611–633.
Haffenden, A. M., & Goodale, M. A. (1998). The effect of pictorial illusion on prehension and
perception. Journal of Cognitive Neuroscience, 10(1), 122–136.
Hickok, G. (2014). The myth of mirror neurons: The real neuroscience of communication and cog-
nition. New York: W. W. Norton.
Hickok, G., Buchsbaum, B., Humphries, C., & Muftuler, T. (2003). Auditory-motor interac-
tion revealed by fMRI: Speech, music, and working memory in area Spt. Journal of Cognitive
Hickok, G., Houde, J., & Rong, F. (2011). Sensorimotor integration in speech pro-
cessing: Computational basis and neural organization. Neuron, 69(3), 407–422.
Hickok, G., Okada, K., & Serences, J. T. (2009). Area Spt in the human planum temporale
supports sensory-motor integration for speech processing. Journal of Neurophysiology,
101(5), 2725–2732.
Hickok, G., & Poeppel, D. (2000). Towards a functional neuroanatomy of speech perception.
Houde, J. F., & Jordan, M. I. (1998). Sensorimotor adaptation in speech production. Science,
279, 1213–1216.
Houde, J. F., & Nagarajan, S. S. (2011). Speech production as state feedback control. Frontiers in
Human Neuroscience, 5. https://doi.org/10.3389/fnhum.2011.00082
Ingle, D. (1973). Two visual systems in the frog. Science, 181(4104), 1053–1055.
Isenberg, A. L., Vaden, K. I., Jr., Saberi, K., Muftuler, L. T., & Hickok, G. (2012). Functionally dis-
tinct regions for spatial processing and sensory motor integration in the planum temporale.
Jacquemot, C., Dupoux, E., & Bachoud-Levi, A. C. (2007). Breaking the mirror: Asymmetrical
disconnection between the phonological input and output codes. Cognitive Neuropsychology,
24(1),  3–22.
Karnath, H. O., Ruter, J., Mandler, A., & Himmelbach, M. (2009). The anatomy of object
recognition: Visual form agnosia caused by medial occipitotemporal stroke. Journal of
Neuroscience, 29(18), 5854–5862.
Larson, C. R., Burnett, T. A., Bauer, J. J., Kiran, S., & Hain, T. C. (2001). Comparison of voice
F0 responses to pitch-shift onset and offset conditions. Journal of the Acoustical Society of
America, 110(6), 2845–2848.
Lee, C. C., & Middlebrooks, J. C. (2013). Specialization for sound localization in fields A1, DZ,
and PAF of cat auditory cortex. Journal of the Association for Research on Otolaryngology,
14(1),  61–82.
Levelt, W. J. M. (1999). Models of word production. Trends in Cognitive Sciences, 3, 223–232.
tion. Behavioral & Brain Sciences, 22(1),  1–75.
Liepmann, H. (1908). Drei Aufsätze aus dem Apraxiegebiet. Berlin: Karger.
Middlebrooks, J. C. (2002). Auditory space processing: Here, there or everywhere? Nature
Milner, A. D., & Goodale, M. A. (1995). The visual brain in action. Oxford: Oxford
University Press.
Milner, A. D., Perrett, D. I., Johnston, R. S., Benson, P. J., Jordan, T. R., Heeley, D. W., et al.
(1991). Perception and action in “visual form agnosia.” Brain, 114(Pt 1B), 405–428.
Neumann, O. (1990). Direct parameter specification and the concept of perception.
Psychological Research, 52(2–3), 207–215.
Nozari, N., Kittredge, A. K., Dell, G. S., & Schwartz, M. F. (2010). Naming and repetition in
aphasia: Steps, routes, and frequency effects. Journal of Memory and Language, 63(4),
541–559.
Pa, J., & Hickok, G. (2008). A parietal-temporal sensory-motor integration area for the human
vocal tract: Evidence from an fMRI study of skilled musicians. Neuropsychologia, 46, 362–368.
Perenin, M. T., & Vighetto, A. (1988). Optic ataxia: A specific disruption in visuomotor
mechanisms. I. Different aspects of the deficit in reaching for objects. Brain, 111(Pt 3),
643–674.
Perkell, J. S. (2012). Movement goals and feedback and feedforward control mechanisms in
speech production. Journal of Neurolinguistics, 25(5), 382–407.
Pohl, W. (1973). Dissociation of spatial discrimination deficits following frontal and parietal
lesions in monkeys. Journal of Comparative Physiological Psychology, 82(2), 227–239.
Poljak, S. (1926). The connections of the acoustic nerve. Journal of Anatomy, 60, 465–469.
Rauschecker, J. P. (1998). Cortical processing of complex sounds. Current Opinion in

primates illuminate human speech processing. Nature Neuroscience, 12(6), 718–724.
Recanzone, G. H., Guard, D. C., Phan, M. L., & Su, T. K. (2000). Correlation between the ac-
tivity of single auditory cortical neurons and sound-localization behavior in the macaque
monkey. Journal of Neurophysiology, 83(5), 2723–2739.
Roelofs, A. (2014). A dorsal- pathway account of aphasic language production: The
WEAVER++/ARC model. Cortex, 59,  33–48.
Rogalsky, C., Poppa, T., Chen, K. H., Anderson, S. W., Damasio, H., Love, T., et al. (2015).
Speech repetition as a window on the neurobiology of auditory-motor integration for
speech: A voxel-based lesion symptom mapping study. Neuropsychologia, 71,  18–27.
Rossetti, Y., Pisella, L., & Vighetto, A. (2003). Optic ataxia revisited: Visually guided action
versus immediate visuomotor control. Experimental Brain Research, 153(2), 171–179.
Schneider, G. E. (1969). Two visual systems. Science, 163(3870), 895–902.
Smiley, J. F., Hackett, T. A., Ulbert, I., Karmas, G., Lakatos, P., Javitt, D. C., et al. (2007).
Multisensory convergence in auditory cortex, I. Cortical connections of the caudal superior
temporal plane in macaque monkeys. Journal of Computational Neurology, 502(6), 894–923.
Smith, K. R., Hsieh, I. H., Saberi, K., & Hickok, G. (2010). Auditory spatial and object pro-
cessing in the human planum temporale: No evidence for selectivity. Journal of Cognitive
Smith, K. R., Okada, K., Saberi, K., & Hickok, G. (2004). Human cortical motion areas are not
motion selective. Neuroreport, 9, 1523–1526.
Smith, K. R., Saberi, K., & Hickok, G. (2007). An event-related fMRI study of auditory motion
perception: No evidence for a specialized cortical system. Brain Research 1150,  94–99.
Sweet, R. A., Dorph-Petersen, K. A., & Lewis, D. A. (2005). Mapping auditory core, lateral
belt, and parabelt cortices in the human superior temporal gyrus. Journal of Computational
Neurology, 491(3), 270–289.
Tian, B., Reser, D., Durham, A., Kustov, A., & Rauschecker, J. P. (2001). Functional specializa-
tion in rhesus monkey auditory cortex. Science, 292(5515), 290–293.
Tremblay, S., Shiller, D. M., & Ostry, D. J. (2003). Somatosensory basis of speech production.
Nature, 423(6942), 866–869.
Trevarthen, C. B. (1968). Two mechanisms of vision in primates. Psychologische Forschung,
31(4), 299–348.
ventral language pathways. Neuron, 72(2), 385–396.
Ungerleider, L. G., & Mishkin, M. (1982). Two cortical visual systems. In D. J. Ingle, M. A.
Goodale, & R. J. W. Mansfield (Eds.), Analysis of visual behavior (pp. 549–586). Cambridge,
MA: MIT Press.
Waldstein, R. S. (1989). Effects of postlingual deafness on speech production: Implications for
the role of auditory feedback. Journal of the Acoustical Society of America, 88, 2099–2144.
Walker, G. M., & Hickok, G. (2016a). Bridging computational approaches to speech produc-
tion: The semantic-lexical-auditory-motor model (SLAM). Psychonomic Bulletin & Review,
23(2), 339–352.
Walker, G. M., & Hickok, G. (2016b). Evaluating quantitative and conceptual models of speech
production: How does SLAM fare? Psychonomic Bulletin & Review, 23(2), 653–660.
Warren, J. D., & Griffiths, T. D. (2003). Distinct mechanisms for processing spatial sequences
and pitch sequences in the human auditory brain. The Journal of Neuroscience, 23, 5799–5804.
Wernicke, C. (1874/1969). The symptom complex of aphasia: A psychological study on an an-
atomical basis. In R. S. Cohen & M. W. Wartofsky (Eds.), Boston studies in the philosophy of
science (pp. 34–97). Dordrecht: D. Reidel.
Westwood, D. A., & Goodale, M. A. (2011). Converging evidence for diverging
pathways: Neuropsychology and psychophysics tell the same story. Vision Research, 51(8),
804–811.
Yagmurlu, K., Middlebrooks, E. H., Tanriover, N., & Rhoton, A. L., Jr. (2016). Fiber tracts of the
dorsal language stream in the human brain. Journal of Neurosurgery, 124(5), 1396–1405.
Zatorre, R. J., Bouffard, M., Ahad, P., & Belin, P. (2002). Where is “where” in the human audi-
tory cortex? Nature Neuroscience, 5(9), 905–909.
Pa rt   I V
C ON C E P T S A N D
C OM P R E H E N SION
Chapter 21
Neural Represe ntat i ons

of C oncep t Knowl e d g e
Andrew J. Bauer and Marcel A. Just
Introduction
A key goal of cognitive neuroscience is to delineate the nature, content, and neuroana-
tomical distribution of the neural representation of concept knowledge, which underlies
human thought, communication, and daily activities, from small talk about well-worn
topics to the learning of quantum physics. Accordingly, research that identifies the
neural systems that underlie different categories of concept knowledge (e.g., concepts
of animals, tools, and numbers) has made significant advances, particularly research on
object concepts (see Martin, 2007). Earlier methods that investigated the neural repre-
sentation of concept knowledge included the study of deficits in concept knowledge in
brain-damaged patients, as well as univariate analyses of activation in the healthy brain
using functional magnetic resonance imaging (fMRI).
More recent neuroimaging research is uncovering the fine-grained spatial patterns
of brain activation (e.g., multi-voxel patterns) evoked by individual concepts. These
brain-reading or neurosemantic studies have generally shown that the spatial pattern
of activation that is the neural signature of the concept is distributed across multiple
brain regions, where the regions are presumed to encode or otherwise process different
aspects of a concept. Conventional linear model-based univariate analyses of activa-
tion levels (e.g., statistical parametric mapping in fMRI; Friston et al., 1994), which do
not take account of multi-voxel patterns, have typically detected only a small number of
brain areas involved in concept representation. Although the idea of multivariate pat-
tern analysis of activation data is not new (Cox & Savoy, 2003), neurosemantic methods
have enabled a paradigm shift in studying how concepts are neurally represented.
This chapter summarizes some key research findings that have characterized where
and how different types of concept knowledge are represented in the brain. The focus
of this chapter is on studies of the neural representations of concepts, rather than on
520 Andrew J. Bauer and Marcel A. Just
the brain regions that support and mediate semantic processing (see Binder, Desai,
Graves, & Conant, 2009, for a meta-analytic review of the neural systems that underlie
semantic processing; for other approaches, see, in this volume, Musz & Thompson-
Schill, Chapter 22, and Garcea & Mahon, Chapter 23). Because most of the neuroim-
aging research reviewed here used blood oxygenation level-dependent (BOLD) fMRI
(see Heim & Specht, Chapter 4 in this volume), brain activation henceforth refers to data
collected using fMRI unless stated otherwise (e.g., magnetoencephalography, or MEG;
see Salmelin, Kujala, & Liljeström, Chapter 6 in this volume).
The majority of this chapter details how neurosemantic research has illuminated var-
ious prominent questions in ways that build on the results of conventional data analytic
methods. Some of these questions are the following: Do neural concept representations
evoked by pictures differ from those evoked by words? What types of informa-
tion are encoded in a given neural concept representation? To what extent are neural
representations common across different people? What are the differences between the
neural representations of abstract versus concrete concepts? The chapter includes a brief
survey of the neurosemantic methods that are used to anatomically localize and charac-
terize the kinds of information that are neurally represented.
The chapter ends by summarizing the results of neurosemantic studies that char-
acterize the changes in neural concept representations during the learning of new
concepts, a topic that has received little attention. The findings from these studies pro-
vide a foundation for cognitive neuroscience to trace how a new concept makes its way
from the words and pictures used to teach it, to a neural representation of that concept in
a learner’s brain. Monitoring the growth of a new neural concept representation has the
potential for further illuminating how concepts are stored and processed in the brain.
The Spatially Distributed Nature of

a Neural Concept Representation
Human beings are capable of thinking about a vast number of concepts at various levels
of abstraction. This variety of ideas and abstractions is reflected in everyday vocabu-
lary and technical terminology (although not all concepts are necessarily expressible
in language). One approach to characterizing concepts in terms of semantically related
words is to construct a lexical database, as the authors of WordNet have done. WordNet
is an English lexical database in which nouns, verbs, adjectives, and adverbs are grouped
into sets of synonyms, where each set constitutes a distinct individual concept. It is a
semantic network that consists of 117,659 concepts, each of which is connected to other
concepts through a chain of semantic relations (WordNet 3.1, http://wordnet.princeton.
edu). WordNet was originally created to be consistent with hierarchical propositional
theories of semantic memory, which postulate that concepts are organized hierar-
chically from general to specific concepts (e.g., Collins & Quillian, 1972). The most
Neural Representations of Concept Knowledge 521
common type of semantic connection between words is the hierarchical “is-a” relation.
The concept chair, for example, is related to furniture by an “is-a” connection. As indi-
cated in WordNet, concepts that human beings can think about range from thoughts of
physical objects, such as organisms and geological formations, to abstractions, such as
psychological states and mathematical entities.
The ability to study how this range of concepts is represented neurally has only re-
cently become possible since the development of data analytic methods that can detect
a correspondence between a distributed brain activation pattern and an individual con-
cept. Neurosemantic research initially focused on only a small fraction of this range of
concepts, namely animate and inanimate object concepts such as animals, faces, and
tools and other manmade objects (e.g., Haxby et al., 2001; Mitchell et al., 2003; for an
account of early neurosemantic research, see Haxby, 2012). These choices of concept
categories were motivated by previous clinical studies of object category-specific ag-
nosia, and also by univariate analysis-based neuroimaging findings that elaborated on
the clinical results. Clinical studies found that relatively selective cortical damage was
associated with a disproportionate deficit in concept knowledge for one of a small set
of categories (e.g., animals or tools; for a review of the clinical literature, see Capitani,
Laiacona, Mahon, & Caramazza, 2003). This body of work suggested that concept
categories were subserved by only a few brain regions. However, mapping large brain
areas to single-concept categories does not provide an account of neural concept
representations that scales to the huge number of concepts that must be represented.
A more efficient scheme that can accommodate vast numbers of concepts would be a
pattern-encoding scheme, such that the neural representation of a concept corresponds
to a spatial pattern of activation of many individual voxels, each displaying a level of
activation that is characteristic of the concept. Early neurosemantic research provided
the empirical basis for pattern encoding by indicating that concept knowledge might be
represented in neural populations distributed over a large number of brain areas.
Since the early fMRI research on concrete object concepts, neurosemantic research
has replicated the finding that neural concept representations span multiple brain re-
gions, and has revealed the activation patterns associated with other types of concept
knowledge, such as emotions (Baucom, Wedell, Wang, Blitzer, & Shinkareva, 2012;
Kassam, Markey, Cherkassky, Loewenstein, & Just, 2013), numbers (Damarla & Just,
2013; Eger et al., 2009), personality traits (Hassabis et al., 2013), and social interactions
(Just Cherkassky, Buchweitz, Keller, & Mitchell, 2014). In one study, close to 2,000 in-
dividual object and action concepts were each localized to multiple brain areas (Huth,
Nishimoto, Vu, & Gallant, 2012). Figure 21.1 shows that the neural representations of
these object and action concepts each reside in multiple areas distributed throughout
the brain. The figure contains color-coded mappings between various concepts and
their representations in various areas. For example, concepts related to communication
(cyan) were found to be represented in auditory sensory cortex in the temporal lobe and
a frontal area that includes Broca’s area, a canonical language region.
The main theoretical interpretation regarding spatially distributed neural
representations is that the multiple brain areas that conjointly represent a given concept
Figure 21.1. Neural concept representations are distributed throughout the brain. In Huth
et al. (2012), 1,705 individual object and action concepts that appeared in movies were each
found to be represented over multiple brain areas. Indicated by the three types of ellipses are the
major brain areas associated with some of the superordinate categories of these object and action
concepts. Auditory sensory cortex in the temporal lobe and a frontal language area were associ-
ated with communication; postcentral gyrus (sensation and movement) and occipitotemporal
cortex (visual) were associated with biological entities; and parietal (spatial) and occipital areas
were associated with buildings and shelter.
Source: The figure corresponds to one participant’s inflated brain, and was extracted
from http://gallantlab.org/semanticmovies.
correspond to the brain systems that are involved in the physical and mental interaction
with the concepts’ referents. For example, the concept of a knife entails what it looks like,
what it is used for, how one holds and wields it, and so on, resulting in a neural represen-
tation distributed over sensory, perceptual, motor, and association areas.
These findings from multivariate analyses build on and are consistent with past uni-
variate analysis-based research. For example, nouns that refer to physically manipulable
objects such as a knife have been shown to activate left premotor cortex in right-handers
(Lewis, 2006). In addition to left premotor cortex, activation has been observed in addi-
tional regions but at lower magnitudes, a result that hinted at the greater spatial distribu-
tion of concept knowledge in the brain (Chao, Weisberg, & Martin, 2002).
In behavioral cognitive science, a concept is often treated as a mental representation
that specifies some of the dimensions of a real-world phenomenon (e.g., visual or tac-
tile properties of an object), in addition to the relations among those dimensions (see
Barsalou, 1992, for a discussion of the nature of concept representation). Consistent
with this approach is the finding that multiple brain regions, which encode different
dimensions, collectively contain the information about a single concept. For example,
the concept cat might include dimensions of cats that are common across different
breeds, such as general body shape, locomotion, diet, temperament, and so on. These
properties should be detectable in the brain activation pattern associated with the con-
cept cat. Several studies have used regression models to predict the activation pattern of
a given object concept, based on how different voxels are tuned to various dimensions of
objects and on how important those dimensions are to defining a given object concept.
Accurate predictions have been made using properties of objects generated by human
participants (Chang, Mitchell, & Just, 2011) or extracted from text corpora such as web-
based articles (Mitchell et al., 2008; Pereira, Botvinick, & Detre, 2013). In addition, an
MEG study that used properties generated by participants predicted the spatial pattern
of evoked magnetic fields associated with an object concept (Sudre et al., 2012).
The discussion here has assumed that sensorimotor systems in the brain store or
otherwise process information that is integral to the comprehension of a concept, par-
ticularly object concepts. In this view, the representations of some concepts entail body-
object interaction information; that is, they are embodied representations (Barsalou,
Santos, Simmons, & Wilson, 2008). Some alternative theories hold that the brain activa-
tion observed in sensorimotor regions reflects imagery or simulated motion that occurs
only after conceptual processing, and that fundamental concept meaning is encoded
only in association areas such as the anterior medial temporal lobe (for a review of the
competing theories, see Mahon & Caramazza, 2008; Meteyard, Cuadrado, Bahrami, &
Vigliocco, 2010; and Kiefer & Pulvermüller, 2012). However, several studies using words
referring to concrete objects have shown that sensorimotor activity evoked by the words
occurs too early to originate from imagery explicitly generated by the participants (e.g.,
Kiefer, Sim, Herrnberger, Grothe, & Hoenig, 2008; see also Martin, 2007), providing
some evidence for the embodied view of the representations of certain concepts.
Characterizing the Semantic
Dimensions That Underlie Neural
Concept Representations
A central objective of neurosemantic research is to determine some of the main se-

mantic dimensions that underlie the neural representation of a given concept. This
objective can be reached by reducing an activation pattern’s dimensionality (often
consisting of the activation levels of tens to hundreds of voxels) to determine the main
factors underlying the representation. For example, in a study of emotions (e.g., anger,
disgust, envy, fear, happiness, lust, pride, sadness, and shame), factor analysis of the
activation data indicated that each emotion was represented with respect to four un-
derlying dimensions: valence, arousal, sociality, and lust (Kassam et al., 2013). Each of
these dimensions was further localized to plausible networks of brain regions. Arousal,
for example, was localized to basal ganglia and precentral gyrus, which have previously
been implicated in action preparation. The sociality dimension (which was not previ-
ously recognized as a core dimension of emotions) was traced to anterior and posterior
cingulate cortex, two default mode network regions previously shown to be involved
in social cognition. Although most of the dimensions were traced to brain areas pre-
viously implicated by univariate analysis-based research, it is notable that this single
neurosemantic study uncovered results comparable to the results of multiple conven-
tional neuroimaging studies.
A similar analysis of the activation patterns evoked by 60 object concepts identified
three key dimensions: manipulation (e.g., tools and other manipulable objects),
eating (e.g., vegetables, kitchen utensils), and shelter (e.g., dwellings, vehicles) (Just,
Cherkassky, Aryal, & Mitchell 2010). Manipulation was associated with left postcentral/
supramarginal gyrus and left inferior temporal gyrus, which have previously been
implicated in the processing of tool concepts (Lewis, 2006). The eating dimension
was traced to left inferior temporal gyrus and left inferior frontal gyrus, which re-
vealed a link between representations of tool concepts (namely kitchen utensils) and
representations of face-and jaw-related actions (Hauk, Johnsrude, & Pulvermüller,
2004). Finally, the shelter dimension was traced to bilateral parahippocampal gyrus and
bilateral precuneus. The parahippocampal gyrus is well known to activate in response
to information about dwellings and scenes (e.g., Epstein & Kanwisher, 1998), and the
precuneus areas were anatomically close to retrosplenial cortex, which is thought to be
involved in the comprehension of a scene within a larger environment (for a review on
retrosplenial cortex, see Vann, Aggleton, & Maguire, 2009).
One study greatly expanded the range of concepts whose neural representations
were uncovered by collecting activation data as participants watched several hours
of movies (Huth et al., 2012). The investigators used WordNet to label 1,364 common
objects (nouns) and actions (verbs) that appeared in the movies, and an additional 341
superordinate categories were inferred using the hierarchical relationships in WordNet
(e.g., canine and mammal were added if wolf was an object that appeared in a movie).
Data reduction yielded four dimensions that were interpretable: mobility/animacy, so-
ciality (e.g., words about people and communication), civilization (e.g., people, man-
made objects, vehicles), and biological entities. Interestingly, these dimensions partially
overlap with the dimensions revealed by the two other neurosemantic studies that sep-
arately investigated object and emotion concepts (described earlier). Thus, different
studies using different methodologies appear to be converging on a common set of un-
derlying neural dimensions of representation.
Integration among a Concept’s Semantic Dimensions

A central aspect of a concept is how the different semantic dimensions relate to each
other. Although some of the underlying dimensions of certain types of concept know-
ledge are being identified, much less is known about the relations among the dimensions.
For example, some of the key dimensions of concrete objects appear to be manipula-
tion, eating, and shelter. Is a neural representation of an object concept then anything
more than the sum of the representations of the concept’s individual dimensions? For
example, would there be some indication in a neural representation that a gingerbread
house is fundamentally different from a cafeteria building, even though both involve
information related to eating and shelter? It is unclear whether relations among the
dimensions of a concept are represented in brain areas that are spatially distinct from
the locations of the individual dimensions (e.g., convergence zones; Damasio, 1989), or
whether relational information is somehow encoded in a distributed way across the set
of areas that also represent the individual dimensions.
A multivariate analysis of the activation in a visual perception task investigated
which brain areas encode the conjunction of separate dimensions (Seymour, Clifford,
Logothetis, & Bartels, 2009). The dimensions that were combined were the color and di-
rection of motion of a set of dots, which were either green or red and rotated either clock-
wise or counterclockwise. The specific conjunction of color and direction of motion of a
given item was found to be represented in multiple areas of early visual cortex that also
encode the individual dimensions, indicating that these areas contain both a representa-
tion of the individual dimensions and the relation between the dimensions. This integra-
tion of information might be a critical aspect of the representation of a cohesive percept.
On the other hand, there is also evidence that the relations among a concept’s com-
ponent dimensions are represented exclusively within specific high-order brain areas or
convergence zones. An anterior temporal lobe region has been suggested as a site of di-
mension integration because it is innervated by different sensory modalities, and because
abnormal functioning of this region is associated with impairments to semantic pro-
cessing, but not to the performance of non-semantic cognitive tasks (Pobric, Jefferies, &
Ralph, 2007; Patterson, Nestor, & Rogers, 2007). Coutanche and Thompson-Schill (2014)
demonstrated that the conjunction of an object’s dimensions was encoded in a multi-
voxel pattern in anterior temporal lobe, but not in the areas that separately represent the
individual dimensions. Specifically, the depicted objects were fruits and vegetables, and
the dimensions were color and shape (whose representations were investigated in fusi-
form gyrus and occipitotemporal cortex, respectively). Furthermore, the representation
of the conjunction was detectable by the investigators only when each dimension’s repre-
sentation could be detected, strengthening the evidence for the conclusion that the ante-
rior temporal region represents the integration of individual components.
Thus the evidence is mixed as to whether the integration of information about sep-
arate dimensions is represented in high-order brain regions versus in the network of
areas that underlie the individual dimensions. It is also possible that integrated informa-
tion is encoded in both representational formats.
Neurosemantic Methodology
Neurosemantic data analyses attempt to detect multi-voxel patterns of brain activa-

tion that correspond to the thoughts of concepts. One virtue of this approach is that
it adheres to the fundamental principle that thinking is a network function involving
multiple brain systems, including thinking about a concept. A second advantage of this
approach is that it bestows a greater sensitivity for discovering the underlying phenom-
enon, by virtue of concurrently assessing the activations of many voxels with similar
activation patterns for the stimuli at hand, regardless of the voxels’ proximity to each
other. One phenomenon whose discovery has benefited from this greater sensitivity is
the representation of different concepts that are in the same superordinate semantic cat-
egory. The greater sensitivity of multi-voxel analysis enables researchers to distinguish
between the activation patterns of such concepts (e.g., distinguishing between different
animal concepts such as a primate and bird; Connolly et al., 2012); that is, the newer
methods can compare patterns of activation between individual concepts in spatially
distributed voxels. With the use of neurosemantic methods, a finding of various re-
lated concepts eliciting unique yet similar activation patterns over a set of brain areas
constitutes suggestive evidence that those brain areas store or otherwise process the
meanings of the concepts.
By contrast, conventional univariate analyses often report the magnitudes of activa-
tion of individual voxels that are averaged over a region of interest, requiring spatial
proximity among the voxels that are grouped together (see Poldrack, 2007). Univariate
analysis is sometimes not sensitive enough to distinguish between similar experimental
conditions because the mean activation level of a set of voxels is often equivalent be-
tween similar conditions. Univariate analysis is useful for identifying the brain areas
that are involved in the processing of some class of concepts, by determining whether an
area’s activation level is elevated. Figure 21.2 depicts a hypothetical scenario in which the
greater sensitivity of multivariate analysis enables distinguishing between two similar
conditions, whereas univariate analysis finds no difference between the conditions but
establishes for each condition an elevation in mean activation level (for a detailed com-
parison of the methods, see O’Toole et al., 2007, and Mur, Bandettini, & Kriegeskorte,
2009). Of course, “sensitivity” is assessed with respect to the phenomenon of interest,
and there are doubtless phenomena other than multi-voxel activation patterns corre-
sponding to concepts for which univariate analyses may be more sensitive (Coutanche,
2013; Davis et al., 2014).
One commonly used technique in neurosemantic studies is discriminative multivar-
iate pattern classification analysis (MVPA). A classifier is an algorithm that is trained
to associate an activation pattern with each of the stimuli (or classes of stimuli) and is
subsequently reiteratively tested (using a procedure called cross-validation) on an inde-
pendent data set (for a tutorial, see Pereira, Mitchell, & Botvinick, 2009, and Norman,
Polyn, Detre, & Haxby, 2006). Logistic regression is an example of a discriminative clas-
sifier. The main strength of MVPA is its concurrent consideration of the activations
of multiple voxels, regardless of their relative locations in the brain. MVPA has been
used to discover neural representations of various types of information, such as covert
intentions in prefrontal cortex (Haynes et al., 2007), visual imagery of simple shapes in
occipitotemporal cortex (Stokes, Thompson, Cusack, & Duncan, 2009), and episodic
memories in the hippocampus (Chadwick et al., 2010). The accuracy of the classifica-
tion is a measure of the discriminability of the stimuli (or classes of stimuli), sometimes
Figure 21.2. Comparison of univariate and multivariate data analysis. A hypothetical scenario

in which multivariate analysis (right) reveals that the multi-voxel pattern of activation levels in
left primary auditory cortex differs between two animal concepts (i.e., two animals that make
similar sounds). Univariate analysis (left) shows that the level of activation averaged over the set
of voxels is the same between the two concepts, but establishes an elevation in mean activation for
both concepts.
computed as the rank accuracy, or the normalized percentile rank of a correct stimulus
in the classifier’s ranked output (Mitchell et al., 2004). Here, chance level is a normalized
rank accuracy of 0.5, where the correct classification response occupies the middle rank
among all possible responses. The obtained accuracy can then be compared to a dis-
tribution of accuracies that would be obtained by chance (typically obtained by Monte
Carlo simulations).
Methods That Assess the Semantic Content

of Neural Representations
Importantly, what is neurally represented is not just the meaning of a concept, but also
the perceptual form of the word or picture that refers to the concept. The perceptual form
is represented in primary and secondary sensory brain regions. Thus, simply obtaining
accurate classification of an activation pattern does not ensure that the pattern encodes
concept meaning. To substantiate that a given activation pattern corresponds to the
meaning of a concept, it is sometimes useful to show that the set of correlations among
the activation patterns bears a clear relation to behavioral judgments of similarity
among the concepts. A statistically reliable correlation between the two sets of inter-
item similarities would provide converging evidence that concept meaning underlies
the systematic differences in the activation data. Another way to ensure that an MVPA
is identifying the representation of a concept—and not the representation of the picture
or word that evokes it—is to exclude sensory and early perceptual area voxels from the
analysis.
Although discriminative classifiers are extremely useful for associating brain areas
with stimuli, they do not easily lend themselves to predictive or generative modeling, as
a generative classifier can do. A generative classifier is useful if there is a need to predict
the activation that will be evoked by a new stimulus. The central property of a generative
classifier is its postulation of a set of intermediate variables between the stimulus and
activation that modulate the activation as a function of the properties of the stimulus.
The classic method for predictive modeling is regression, which can be used to predict
the activation of a yet unseen stimulus, based on how its properties modulated the acti-
vation of the stimuli on which the model was trained. Predictive regression models can
provide converging evidence for the neural representation of concept meaning in terms
of the postulated semantic dimensions that underpin the neural representation of a con-
cept. A model is first estimated of how a set of voxels is tuned to different dimensions
(e.g., the size or animacy of an object). A prediction is then made of a concept’s acti-
vation pattern based on the weighted importance of the dimensions in the represen-
tation of that concept. The generalizability of the model can be assessed by testing the
predicted activation pattern of any concept that is definable by the dimensions included
in the model (Naselaris, Kay, Nishimoto, & Gallant, 2011). As mentioned previously, it
is possible to accurately predict an object concept’s activation pattern using properties
of objects generated by human participants (Chang et al., 2011; Sudre et al., 2012). This
approach has also been used to investigate how different voxels in visual brain areas are
tuned to different visual features (Kay, Naselaris, Prenger, & Gallant, 2008). The general
goal of this approach is to relate concept properties to one or more areas of activation.
Another methodology that can help characterize the neural representation of a con-
cept is representational similarity analysis (RSA), which assesses the neural similarity
between all pairs of items and relates the resulting similarity structure to the activation
patterns (see Musz & Thompson-Schill, Chapter 22 in this volume). Some researchers
use the idea of an n-dimensional representational space, in which the distance between
a given pair of concepts approximates the degree of similarity between the concepts’ ac-
tivation patterns. If the multi-voxel activation pattern for two concepts is known, then
the similarity (or distance) between them can be computed and the full set of inter-item
distances can be used to specify the space. In this approach, the set of inter-concept
distances can reveal the kinds of information that underlie the representations (see
Kriegeskorte, Mur, & Bandettini, 2008, for a tutorial on RSA). For example, Figure 21.3
shows the similarity structure of six biological species concepts, corresponding to two
different brain areas. The similarity structure of the concepts differs between the two
areas, indicating that the information that is encoded or otherwise processed differs be-
tween the areas. The information represented in occipitotemporal cortex is organized
(a) (b)
Figure 21.3. Representational similarity structures in different brain areas reveal differences in

the types of information neurally represented therein. Shown for two different brain areas are the
dissimilarities between all pairs of biological concepts’ activation patterns, and the hierarchical
structure that emerges from these dissimilarities. The representational dissimilarity between any
two concepts was computed as 1 –the correlation between their activation patterns. (A) Lateral
occipital complex (LOC), a high-order visual brain area that here represents information corre-
sponding to species category. (B) Early visual cortex (EV), a collection of brain areas that process
low-level visual features, which seems to encode visual properties of the concepts that do not cor-
relate with species category.
Source: The figure was adapted with permission from Connolly et al. (2012).
with respect to species category, given that the neural dissimilarity is lowest between
the two primates, between the two birds, and between the two insects. On the other
hand, the information in early visual cortex seems to encode visual properties of the
concepts that do not correlate with species category. In this way, representational sim-
ilarity structures have the potential to reveal the underlying dimensions along which
concepts are organized in the brain.
The capacity of RSA to reveal the semantic content encoded in neural representations
depends in part on the measures of dissimilarity between a pair of vectors of activa-
tion levels, such as correlational measures (e.g., 1—the Pearson correlation between
multi-voxel activation patterns), or geometric distances (e.g., Euclidean or Mahalanobis
distance). Another type of measure of neural dissimilarity is the classification accu-
racy in a classifier’s confusion matrix. A comparison of these dissimilarity measures for
RSA has revealed that continuous distances (i.e., correlation and geometric distances)
produce more reliable results than classification accuracies, largely because classifi-
cation accuracies are obtained from binary decisions that discard continuous dissimi-
larity information (Walther et al., 2016). Furthermore, dissimilarity measures that are
cross-validated across subsets of activation data provide an interpretable zero point
against noise.
Apart from RSA, other data-driven, exploratory methods are used to characterize
the informational content contained in neural representations. To identify key under-
lying dimensions from a large set of voxels spanning multiple brain regions, dimension
reduction techniques, such as principal or independent components analysis or factor
analysis, are useful (see Heim & Specht, Chapter 4 in this volume). These dimension re-
duction methods can separate the activation patterns into smaller sets of voxels (which
maximize the amount of total or shared variance explained in the data), where each set
is associated with one or more of the dimensions (e.g., Just et al., 2010; Kassam et al.,
2013). If some of the voxels associated with one of the dimensions are localized primarily
in the motor cortex, for example, then it is likely that motor action constitutes part of the
semantic content of that dimension.
Even with the use of advanced neurosemantic methods, caution may be needed in
concluding that the activation pattern in a set of brain regions represents the meaning
of a particular concept because of the notorious difficulty in distinguishing represen-
tation from process (Anderson, 1978). It is sometimes difficult to distinguish whether
an activation pattern corresponds to where and how information is stored, versus cor-
responding to the processes that operate on the representation. Measurement of a
neural concept representation requires evoking an activation pattern, thus potentially
conflating representation and process; for example, the content of a neural concept
representation might be a facet of processing related to selective attention. Selective at-
tention has been shown to change the tuning characteristics of occipitotemporal and
frontoparietal cortex for the objects shown in a movie (Çukur, Nishimoto, Huth, &
Gallant, 2013). A way to address this type of difficulty might be to test whether char-
acteristics of the activation patterns vary as a function of the nature of the task that the
participants perform. It will be useful for future research to characterize neural concept
representations in a way that takes into account the nature of the processing that evokes
that concept, for example in sentence comprehension (Poeppel, 2012), story compre-
hension (Wehbe et al., 2014), and problem-solving (Anderson & Fincham, 2014).
Methods of Evoking a Neural Concept Representation

Various methods have been used to evoke the brain activation that underlies a concept.
They vary with respect to three characteristics: (1) the amount of time allotted for a par-
ticipant to process and think about a concept; (2) the nature of the task that the partic-
ipant is asked to perform; and (3) the modality of the stimulus used to evoke a concept.
Each method of evoking a concept has its own profile of advantages and disadvantages;
for example, allotting more time to think about a concept can yield more robust signal
in the activation data and thus greater classification accuracy. However, if there is no in-
struction to think about a concept in a certain way, greater thinking time may result in
variation in the activation across the repetitions of a concept, due to the different ways
in which a participant may think about a concept. Furthermore, an instruction to think
about a concept in a particular way or context may induce an unrepresentative instanti-
ation of the concept (e.g., thinking about dog as a participant in a race).
A study in which the participants were presented the same emotion concepts under
different task instructions provided evidence of the commonality of the neural represen-
tation across two very different task conditions (Kassam et al., 2013). In one condition,
participants were presented with emotion words (such as “anger”) and were instructed
to evoke thoughts and feelings associated with each emotion. In a different condition,
participants passively viewed pictures that evoked disgust. A classifier that was trained
on the activation evoked by the emotion words (which included “disgust”) was then able
to identify the disgust evoked by the pictures with good accuracy. This finding provided
evidence that, at least in this case, the brain activation patterns corresponding to disgust
in these two very different conditions were fairly similar to each other. It would be useful
to see many other concept representations compared, under many different conditions,
to determine which facets of a neural representation are always activated and which are
modulated by the nature of the evoking task.
Influences of Language on Neural

Concept Representations
One task effect of long-standing interest involves the difference in the content of a neural
concept representation depending on whether the evoking stimulus is a word versus a
picture. For example, is a picture of a screwdriver more likely than the word “screw-
driver” to evoke a specific, potentially unrepresentative instantiation of the concept
screwdriver, especially if the picture is richly detailed? A resolution of this issue would
provide a theoretical framework to account for the results of numerous studies that use
words, pictures, movies, or other stimuli to evoke a concept.
A neurosemantic study uncovered suggestive evidence that the central aspects of a
neural concept representation are to a large extent independent of the stimulus used
to evoke the concept (Shinkareva, Malave, Mason, Mitchell, & Just, 2011). In this study,
it was possible to classify the activation pattern of an object concept cued by the noun
naming the object with a classifier trained on the activation pattern of the same con-
cept evoked by a simple line drawing, and vice versa. Specifically, the classifier deter-
mined whether a given object concept referred to a tool or dwelling. Although this study
assessed only a small number of items from only two categories, it is suggestive of a
common core neural representation that is evoked regardless of the stimulus modality.
In Shinkareva et al. (2011), it was possible to classify the words or pictures using acti-
vation from the language system (left inferior frontal gyrus) and also from sensorimotor
brain regions. These results are consistent with the Language and Situated Simulation
theory of semantic processing (akin to the embodied cognition approach), which holds
that a concept activates both the language system and the same sensorimotor regions
that are active during actual interaction with the concepts’ referents (Barsalou et al.,
2008; Simmons, Hamann, Harenski, Hu, & Barsalou, 2008).
Despite there being a shared core of the neural representation between pictures and
words referring to a concept, there is evidence of differences in the neural concept
representations. A possible asymmetry between pictures and words is that pictures evoke
not only a concept’s core meaning, but also some detailed instantiation of the concept as
it is depicted in a picture. A picture generally contains a greater amount of information
about an object than does a word (e.g., the shape of a screwdriver’s handle and its tip).
Words, on the other hand, tend to evoke only the most generic properties of a concept.
In the study of the cross-stimulus modality classification described earlier (Shinkareva
et al., 2011), the classification accuracy was higher when the classifier was trained on
word-cued activation and tested on pictures, versus when it was trained on pictures and
tested on words. This result suggests that although the neural representations of words
and pictures are similar to each other, pictures activate additional information that is
specific to the picture. The classifier that was trained on pictures and tested on words
apparently extracted some information unique to the pictures, leading to a classification
accuracy that was lower than when the classifier was trained on words and extracted ge-
neric information common to the picture and word representations.
In sum, there is evidence of overlap in the semantic content between word-and
picture-cued neural representations. However, additional research is needed to char-
acterize the distinctions between the content of representations evoked by words versus
pictures. Can identical representations be evoked between words and pictures by
manipulating the information that is directly expressed in either presentation modality?
For example, would the addition of modifiers to a word increase the amount of informa-
tion about the evoked concept (e.g. “short, yellow Phillips screwdriver”)? Similarly, can
the neural representation of a concept evoked by a picture be made more similar to the
one evoked by a word by making the picture completely schematic, thereby removing
the extra information conveyed by the picture? Empirical studies that address such is-
sues would enable a refinement of theories of how concept knowledge is neurally stored
and activated (e.g., dual-coding theory; Paivio, 1986).
Neural Representations of Lexical

and Grammatical Categories
There has not yet been found a clear difference in activation patterns evoked by dif-
ferent word classes (e.g., verbs versus nouns). Studies that have addressed this question
have been hampered by the problem that concepts corresponding to different gram-
matical categories are inherently different. For example, verbs are typically associated
with concrete actions, whereas nouns typically refer to objects (see Vigliocco, Vinson,
Druks, Barber, & Cappa, 2011, for a review of the brain activation underlying nouns and
verbs). These grammatical categories also differ in terms of lexical stress and ortho-
phonological typicality, which complicate matters further (Arciuli, McMahon, & de
Zubicaray, 2012). Consequently, any differences in activation found between nouns
and verbs might plausibly be attributed to differences in semantic content, rather than
to (non-semantic) differences in lexical category or grammatical structure. One study
compared the activation evoked by abstract nouns and verbs (e.g., “idea” versus “think”)
because abstract nouns and verbs are not associated with concrete objects or concrete
actions and thus are not inherently different in the same way that concrete nouns and
verbs are (Moseley & Pulvermüller, 2014). This study found no activation location dif-
ference between abstract verbs and nouns, whereas there was a difference between
concrete verbs and nouns. The authors concluded that there are no word class-specific
processing centers in the brain. However, the analysis focused on only a small set of re-
gions of interest; a whole-brain comparison was not conducted, thus leaving the ques-
tion open to additional investigation.
Other studies have reported differences in the activation locations evoked by pseudo-
nouns versus pseudo-verbs (Shapiro et al., 2005). Pseudo-nouns (nonsense words with
morphological cues to lexical category, such as ending in –age, cuing a noun) elicited
greater activation than pseudo-verbs bilaterally in temporal regions, whereas pseudo-
verbs (e.g., those ending in –eve) evoked greater activity in left-lateralized frontal areas.
In another study, pseudo-verbs (e.g., ending in –eve) elicited greater activity in motor
cortex than pseudo-nouns (de Zubicaray, Arciuli, & McMahon, 2013). Such differential
activation evoked by stimuli that are devoid of meaning suggests that a word’s lexical
class is a possible dimension of lexical organization in the brain.
In a study using multivariate analysis, it was possible to distinguish between the ac-
tivation patterns associated with semantically equivalent but grammatically different
sentences (Allen, Pereira, Botvinick, & Goldberg, 2012). Specifically, a classifier could
determine whether a sentence was ditransitive (e.g., “Mike brought a book to Chris”)
or dative (e.g., “Mike brought Chris a book”), despite the fact that the two grammat-
ical constructions convey the same core information. The classifier used activation
from left-lateralized brain areas involved in language processing, such as left inferior
frontal gyrus. The result suggests that grammatical category is neurally represented in-
dependent of semantic meaning, but further research is needed that identifies the gram-
matical aspects of ditransitive and dative sentences that might be neurally represented.
At the same time, it remains unclear whether grammatically different sentences with the
same core meaning still have subtle differences in semantic content that can be detected
in activation patterns.
If nouns and verbs tend to evoke differing semantic content (i.e., object-and action-
related content, respectively), then some neurosemantic theories would posit that this
content is primarily represented in association areas that integrate among different sen-
sorimotor modalities, such as the anterior medial temporal lobe or angular gyrus (e.g.,
Patterson et al., 2007; for a review of these and other theories, see Mahon & Caramazza,
2008; Meteyard et al., 2010; and Kiefer & Pulvermüller, 2012). In support of this view,
abnormal functioning of these brain regions is associated with impairments to semantic
processing, but not to the performance of non-semantic cognitive tasks (e.g., Pobric
et al., 2007). Because many of the activation differences observed between nouns and
verbs are not constrained to these association areas, these theories might predict that
activation differences between lexical categories could instead reflect non-conceptual
linguistic processing. For example, greater activity observed in motor cortex in response
to verbs versus nouns could reflect verb-specific ortho-phonological properties (de
Zubicaray et al., 2013). However, there is also reason to believe that sensorimotor acti-
vation elicited by words instantiates the concepts to which the words refer, which may
constitute a form of semantic processing (Mahon & Caramazza, 2008). Thus, additional
research that demarcates the semantic system in the brain may provide some answers
regarding which brain areas underlie syntactic processing.
Finally, another prominent question addressed by the neurosemantic approach is
whether neural concept representations differ between different languages, assuming
that the words or phrases are good translation equivalents of each other. There is sugges-
tive evidence that the neural representation of a concept is largely the same, regardless of
which language is used to evoke it. In two neurosemantic studies of bilinguals, it was pos-
sible to identify the activation pattern associated with an object concept cued in one lan-
guage based on the activation pattern of the same concept denoted in another language
(Buchweitz, Shinkareva, Mason, Mitchell, & Just, 2012; Correia et al., 2014). However,
there are subtle clues that neural representations of word classes differ between speakers
of different languages due to differences in the semantic content associated with the
classes. For example, the most frequently used class of verbs in Spanish refers to the path
of an object (see Goldstone & Kersten, 2003), whereas English verbs often refer to the
manner of an object’s motion. Thus, neural concept representations might differ between
the two languages in this respect, despite a core commonality in the representations.
Commonality of Neural
Concept Representations across
Different Individuals
One of the most dramatic findings in neurosemantics is that the fine-grained activa-
tion pattern corresponding to a given concept is largely common across people. When
two people think about the concept apple, their activation patterns are distributed over
the same brain locations and are very similar. When a classifier is trained on the activa-
tion patterns from a set of participants (whose activation data were spatially aligned to a
common anatomical template), it can reliably predict which concept a left-out test par-
ticipant is contemplating. This phenomenon of commonality has been demonstrated
for the neural representations of concrete objects (Just et al., 2010), emotions (Kassam
et al., 2013), numbers (Damarla & Just, 2013), and social interactions (Just et al., 2014).
The ability to accurately classify concepts across people suggests that some of the
same properties of a given concept are evoked in many or most individuals. Behavioral
studies in which participants generate properties of objects report that there are some
properties that are commonly associated with a given object concept (e.g., Cree &
McRae, 2003; Nelson, McEvoy, & Schreiber, 1998). Moreover, a neurosemantic study
uncovered suggestive evidence that the most defining properties of a concept are auto-
matically activated during any instance of evoking that concept, even during tasks for
which that information is irrelevant (Hsu, Schlichting, & Thompson-Schill, 2014).
Accurate cross-individual classification is somewhat surprising given the uniqueness

and variety of personal experiences and associations that might underlie a concept, in
addition to the varied levels of experience with a concept. Although the commonality of
neural representations of concepts has been demonstrated, the unique aspects of neural
representations have yet to be characterized. The unique components could be similar
in kind to the common aspects (which constitute semantic memory), or they could be
tagged in some way as part of one’s autobiographical memory (Charest, Kievit, Schmitz,
Deca, & Kriegeskorte, 2014). It will be interesting to determine the characteristics of
concept knowledge that are unique, although to do so will be challenging precisely be-
cause any emerging pattern of results will be difficult to aggregate over participants. If
successful, such research may enable an understanding of how different properties of
a neural representation, such as its particular pattern of activity or its anatomical dis-
tribution, are shaped by individual factors (e.g., unique experience, genetic predis-
position) versus shared, cross-individual factors (e.g., cultural values, evolutionarily
conserved biases toward processing certain types of information, and inherent neural
constraints; Sadtler et al., 2014). Representational commonality might indicate that
there exist category-specific brain networks that process specific kinds of information
that are important to survival, such as information about food or shelter (Mahon &
Caramazza, 2003).
Methods That Assess Representational Commonality

by Abstracting Away from Person-Specific Patterns
of Neural Activity
There is another approach that makes it possible to compare neural representations
across people, namely cross-individual comparison of the similarity relations among
the concepts’ activation patterns (Raizada & Connolly, 2012). This approach uses RSA
(described earlier) to abstract the activation data away from voxel space (i.e., acti-
vation patterns corresponding to different brain locations) to representational sim-
ilarity space (i.e., correlations among the activation patterns). Thus, the method does
not need to warp different participants’ activations to a common anatomical template.
Classification is not performed directly on the activation patterns. Rather, the classi-
fier determines whether the similarity relations among the concepts’ activation patterns
are similar across individuals. Cross-individual classification in representational simi-
larity space may produce greater classification accuracy because the method does not
need to account for individual differences in brain anatomy. Thus, although there may
be slight differences in the precise brain locations of two participants’ representations
of the same concept, the neural similarity between the two representations is robust to
these differences.
Another cross-individual classification method is the mapping of each individual’s
activation data from original voxel space to a common, high-dimensional space over
all the participants (Haxby et al., 2011; Haxby, Connolly, & Guntupalli, 2014). The
dimensions in this new common space are not individual voxels, but rather distinct
response-tuning functions defined by their commonality across the different brains.
This method also results in greater cross-individual classification accuracy.
Yet another cross-individual classification method encodes both activation loca-
tion and magnitude in a graph structure and is robust to anatomical differences among
people (Takerkart, Auzias, Thirion, & Ralaivola, 2014). Thus, the warping of activation
data to align with a common anatomical template might lead to an underestimation of
the commonality of the semantic content in neural representations.
Intriguingly, one study that used RSA assessed commonality of neural representations
between human beings and macaque monkeys (Kriegeskorte, Mur, Ruff, et al., 2008;
Kriegeskorte, 2009). This study indicated that pictures of various animate and inani-
mate objects elicited activity patterns that had similar representational similarity
between the humans and monkeys in homologous inferotemporal cortex, a set of high-
order visual brain areas. The human data consisted of fMRI activation, and the monkey
data were single-neuron electrophysiological recordings from two macaque monkeys
(Kiani, Esteky, Mirpour, & Tanaka, 2007). The commonality between the neural
representations belonging to the two species is illustrated in Figure 21.4. To the extent
that the inferotemporal activation reflected semantic processing rather than perceptual
processing of the picture stimuli themselves, this result may bear on the profound ques-
tion of the nature of thought in other species. The result also motivates future research
that uses similar methods to compare neural representations between humans and
monkeys within other brain areas, and for other concept categories such as numbers
(e.g., Beran, Johnson-Pynn, & Ready, 2011).
Neural Representations of Abstract

and Concrete Concepts
Previous behavioral research suggests that how a concept is stored and otherwise
processed depends on how concrete or abstract it is. For example, words that refer to
concrete concepts (e.g., “ball”) are more quickly recognized (e.g., Schwanenflugel
& Harnishfeger, 1988), and knowledge of concrete concepts is more resistant to brain
damage (see Coltheart, Patterson, & Marshall, 1980). Several fMRI studies have shown
that words that refer to either abstract concepts (e.g., “blame”) or concrete concepts ac-
tivate overlapping but partially distinct brain networks, with abstract concepts eliciting
greater activation in the frontal language system (e.g., Binder, Westbury, McKiernan,
Possing, & Medler, 2005; Friederici, Opitz, & von Cramon, 2000; Noppeney & Price,
2004). In several studies, abstract words were defined as those with low imageability
and concreteness ratings, and vice versa for the concrete words. The overlapping por-
tion of activation between the two word classes consisted of left-lateralized areas that
Figure 21.4. Neural representations of concrete objects are similar between monkeys and
humans. Arrangements of the same picture stimuli separately for monkeys and humans, such
that the distance between any two pictures reflects the dissimilarity between their activity
patterns (1 –the spatial correlation) in IT (inferotemporal cortex), a set of high-order visual
brain areas.
Monkey data: 674 single-neuron electrophysiological recordings from two macaque monkeys (Kiani et al., 2007).
Human data: fMRI activation in 316 voxels (Kriegeskorte, Mur, Ruff, et al., 2008). Different categories: face (red),
body (magenta), natural object (blue), artificial object (cyan). The lines connect the same pictures between monkey
and human; thick lines indicate that the neural representations were dissimilar between monkey and human.
Source: The figure was adapted from Kriegeskorte (2009) freely under the terms of the
Creative Commons Attribution License.
receive inputs from multiple sensory modalities, such as angular gyrus. In one of the
studies, concrete concepts, in contrast to abstract concepts, elicited greater activation
in right-lateralized multimodal areas (Binder et al., 2005). Abstract concepts produced
greater activation primarily in left inferior frontal gyrus, an important area in the lan-
guage system. Thus, the findings from these univariate analysis-based studies suggest
that abstract (versus concrete) concepts evoke other verbal concepts and involve less
sensorimotor knowledge than concrete concepts.
The only multivariate analysis- based study to date that compares the neural
representations between abstract and concrete concepts documented findings similar
to those of the univariate analysis-based studies (Wang, Baucom, & Shinkareva, 2013).
For example, left inferior frontal gyrus was among a small set of regions that by it-
self enabled a classifier to recognize the activation patterns corresponding to abstract
concepts. Taken together, the results in this research area support the dual-coding
theory of semantic processing (Paivio, 1986), which postulates that abstract concepts are
neurally represented primarily as lexical items, whereas concrete concepts are addition-
ally stored as sensorimotor representations.
The findings of left inferior gyrus involvement in the representation of abstract
concepts leave open the question of what type of information is represented there. Apart
from its critical role in language processing, this region has been associated with phono-
logical working memory (Burton, 2001), and so activation patterns detected in this re-
gion might contain sustained phonological representations of words, while knowledge
related to the word meaning is being retrieved from other brain locations. Left inferior
frontal gyrus has also been suggested to mediate conflicts in the retrieval of knowledge
among competing alternatives (Thompson-Schill, D’Esposito, & Kan, 1999). Thus, ac-
tivation detected in this region might reflect various instances of mediation among
competing requests for the retrieval of knowledge related to the concept currently being
thought about. According to either interpretation, activation patterns in this region
would not appear to encode the knowledge per se associated with a concept.
The evidence uncovered thus far suggests that an abstract concept evokes a set
of verbal or lexical representations associated with that concept, more so than does a
concrete concept. This lexical information might also include concrete words, whose
meaning is neurally represented in sensorimotor brain areas. Regression models might
be used to discover sensorimotor activation patterns associated with abstract concepts
by accounting for any hidden concrete factors that underpin the representations.
Scientific concepts are a specific type of abstract concept learned only through formal
education. The neural signatures of scientific abstract physics concepts (e.g., gravity,
torque, frequency) can be decomposed into meaningful underlying neural and se-
mantic dimensions, despite their abstractness. Mason and Just (2016) used factor anal-
ysis to uncover the underlying dimensions of the neural representation of 30 physics
concepts. The four main dimensions underlying the neural representation of these ab-
stract concepts were causality, periodicity, algebraic representation (a sentence-like state-
ment of the quantitative relations among concepts), and energy flow, all of which are
dimensions that are used for representing familiar concrete concepts. For example, a
concept like frequency has a strong periodicity component. (The brain locations corre-
sponding to this factor included bilateral superior parietal gyrus, left postcentral sulcus,
left posterior superior frontal gyrus, and bilateral inferior temporal gyrus.) The appli-
cability of these underlying dimensions was assessed in terms of a classification model
that used the factor-related brain locations to accurately classify the 30 abstract concepts
based on their neural signature. The findings suggest that abstract scientific concepts are
represented by repurposing neural structures that originally evolved for more general
purposes. The underlying brain capabilities that form the basis for physics concepts
existed long before physics knowledge was developed.
Changes in Neural Concept

Representations with Learning
The ability to track the growth of a neural concept representation speaks to one of the
foundational goals of cognitive neuroscience research, namely to understand the neural
basis of knowledge acquisition. The study of concept learning also promises to enable a
greater understanding of how concept knowledge is represented and processed in the
brain. However, little is known about the changes that occur in a neural concept repre-
sentation as a new concept is being learned.
Much of the existing research on concept learning has focused on changes in which
brain regions show heightened activation between pre-and post-learning. For ex-
ample, after a session of learning how to manipulate novel tool-like objects, activation
to pictures of the objects was found to shift predominantly to motor cortex compared to
pre-learning (Weisberg, van Turennout, & Martin, 2007). Another study showed that
after participants were verbally instructed about the kind of motion or sound that was
associated with novel living objects, the activation elicited by the object pictures was
localized to motion-specific or auditory cortex (James & Gauthier, 2003). These studies
showed that the brain regions that became active after learning corresponded to the
kinds of information that were taught. However, the univariate analyses used in these
studies did not permit a determination of how each individual new concept became
encoded in a distributed neural representation within the new sites of activation.
A multivariate study of concept learning documented the emergence of the neural
representations of individual new concepts (Bauer & Just, 2015). Specifically, the growth
of the representations of new animal concepts was monitored as two properties of each
animal were taught, namely an animal’s habitat and its diet or eating habits. The learning
of information about each of these dimensions was demonstrated by an increase in
the accuracy of classifying the animal identities based on the brain areas associated
with the dimension that had been learned. For example, after participants had learned
about the habitats of some animals, it was possible to classify which animal they were
thinking about by training a classifier on the activation patterns in regions associated
with shelter information. This study provides a novel form of causal evidence that
newly acquired knowledge comes to reside in the brain regions previously shown to
underlie a particular type of concept knowledge.
Another neurosemantic study examined the changes in the neural representations
of complex mechanical concepts as they were being learned, and found that different
stages of learning are associated with different sets of brain regions that encode the
emerging knowledge (Mason & Just, 2015). Specifically, the study demonstrated
how incremental instruction about the workings of several mechanical concepts
(e.g., bathroom scale, automobile braking system) gradually changed the neural
representations of the systems. The representations progressed through different
states that reflected different learning stages, starting with the visual properties of
the concept encoded from the display, mental animation of mechanical components,
generation of causal hypotheses associated with the animation, and determination
of how a person would interact with the mechanical system. Research on interme-
diate stages of learning has lagged behind studies that focus only on final outcomes
of learning (Karuza, Emberson, & Aslin, 2014). The results in Mason and Just (2015)
raise the possibility that the neural representations of familiar concepts (which is the
only type of concept that most previous studies have investigated) may fail to reveal
the constructive processes by which the neural representations become established.
The constructive processes may reveal some fundamental properties of neural con-
cept representations.
The neurosemantic research on concept learning provides a foundation for brain re-
search to trace how new knowledge makes its way from the words and graphics used
to teach it, to a neural concept representation in a learner’s brain. It might foreshadow
an era in which brain imaging and neurosemantic methods are used to diagnose which
aspects of a concept a student misunderstands or lacks, in a way that might be more fun-
damental and accurate than conventional behavioral testing. An fMRI study in which
real-time measurement of brain activation identified mental states that were either “pre-
pared” or “unprepared” for encoding a new stimulus lends credence to this possibility
(Yoo et al., 2012).
The study of how the learning process changes neural concept representations
promises to enable a greater understanding of the kinds of information encoded in
neural representations. Just as neurosemantic methods have been useful in determining
where and how the different dimensions of a concept are encoded, these methods might
eventually be used to track the developmental trajectory of neural representations as
a function of various factors of interest, such as a person’s previous experience or
knowledge, or elapsed time between learning episodes. Perhaps a comparison of
representations at different stages of knowledge expertise would aid in deciphering the
kinds of information that are encoded in the representations. For example, chess experts
can remember large configurations of chess pieces on a board by representing various
relationships among the chess pieces (Gobet & Simon, 1996). Comparisons between the
information that is neurally encoded in a domain expert versus a novice might illumi-
nate the process of building complex neural representations.
Conclusion
Neurosemantic methods have enabled enormous advances in uncovering how var-

ious types of concept knowledge are neurally represented, and also in characterizing
the information contained in the representations. The ability to study how different
concepts are neurally represented has only recently become possible since the de-
velopment of data analytic methods that can detect a correspondence between a
distributed activation pattern and an individual concept. The key virtues of the
neurosemantic approach over older methods are that it generally permits greater
sensitivity to uncovering the underlying phenomenon, and it adheres to the fun-
damental principle that concept information is encoded in neural populations dis-
tributed throughout the brain. The approach promises to illuminate a number of
prominent questions; for example, the field is better equipped to determine whether
abstract concepts are neurally encoded as lexical representations, or whether ab-
stract thoughts are underpinned by sensorimotor factors as revealed by organized
patterns of activation in these brain regions. The neurosemantic paradigm provides
the tools for forging discoveries in areas of daunting complexity, such as how the re-
lations among a concept’s underlying semantic dimensions are neurally encoded and
thereby represent a cohesive concept, and how learning establishes and shapes new
representations.
Acknowledgments
This work was supported by the National Institute of Mental Health Grant MH029617 and the
Office of Naval Research Grant N00014-16-1-2694.
References
Allen, K., Pereira, F., Botvinick, M., & Goldberg, A. E. (2012). Distinguishing grammatical
constructions with fMRI pattern analysis. Brain and Language, 123(3), 174–182. doi: 10.1016/
j.bandl.2012.08.005
Anderson, J. R. (1978). Arguments concerning representations for mental imagery.
Psychological Review, 85, 249–277. doi: 10.1037/0033- 295X.85.4.249
Anderson, J. R., & Fincham, J. M. (2014). Discovering the sequential structure of thought.
Cognitive Science, 38(2), 322–352. doi: 10.1111/cogs.12068
Arciuli, J., McMahon, K., & de Zubicaray, G. (2012). Probabilistic orthographic cues to
grammatical category in the brain. Brain and Language, 123(3), 202–210. doi: 10.1016/
j.bandl.2012.09.009
Barsalou, L. W. (1992). Frames, concepts, and conceptual fields. In E. Kittay & A. Lehrer (Eds.),
Frames, fields, and contrasts: New essays in semantic and lexical organization (pp. 21–74).
Hillsdale, NJ: Lawrence Erlbaum Associates.
Barsalou, L. W., Santos, A., Simmons, W. K., & Wilson, C. D. (2008). Language and simulation
in conceptual processing. In M. De Vega, A. M. Glenberg, & A. C. Graesser (Eds.), Symbols,
embodiment, and meaning (pp. 245–283). Oxford: Oxford University Press.
Baucom, L. B., Wedell, D. H., Wang, J., Blitzer, D. N., & Shinkareva, S. V. (2012). Decoding
the neural representation of affective states. NeuroImage, 59(1), 718–727. doi: 10.1016/
Bauer, A. J., & Just, M. A. (2015). Monitoring the growth of the neural representations of new
animal concepts. Human Brain Mapping, 36(8), 3213–3226. doi: 10.1002/hbm.22842
Beran, M. J., Johnson-Pynn, J. S., & Ready, C. (2011). Comparing children’s Homo sapiens and
chimpanzees’ Pan troglodytes quantity judgments of sequentially presented sets of items.
Current Zoology, 57(4), 419–428.
19(12), 2767–2796. doi: 10.1093/cercor/bhp055
Binder, J. R., Westbury, C. F., McKiernan, K., Possing, E. T., & Medler, D. (2005). Distinct brain
systems for processing concrete and abstract concepts. Journal of Cognitive Neuroscience,
17(6), 905–917.
Buchweitz, A., Shinkareva, S. V, Mason, R., Mitchell, T. M., & Just, M. A. (2012). Identifying bi-
lingual semantic neural representations across languages. Brain and Language, 120(3), 282–
289. doi: 10.1016/j.bandl.2011.09.003
Burton, M. W. (2001). The role of inferior frontal cortex in phonological processing. Cognitive
Science, 25, 695–709.
Capitani, E., Laiacona, M., Mahon, B., & Caramazza, A. (2003). What are the facts of se-
mantic category-specific deficits? A critical review of the clinical evidence. Cognitive
Neuropsychology, 20, 213–261. doi: 10.1080/02643290244000266
Chadwick, M. J., Hassabis, D., Weiskopf, N., Maguire, E. A. (2010). Decoding individual epi-
sodic memory traces in the human hippocampus. Current Biology, 20, 544–547.
Chang, K. K., Mitchell, T., & Just, M. A. (2011). Quantitative modeling of the neural representa-
tion of objects: How semantic feature norms can account for fMRI activation. NeuroImage,
56(2), 716–727. doi: 10.1016/j.neuroimage.2010.04.271
Chao, L. L., Weisberg, J., & Martin, A. (2002). Experience-dependent modulation of category-
related cortical activity. Cerebral Cortex, 12(5), 545–551.
Charest, I., Kievit, R. A., Schmitz, T. W., Deca, D., & Kriegeskorte, N. (2014). Unique semantic
space in the brain of each beholder predicts perceived similarity. PNAS, 111(40), 14565–
14570. doi: 10.1073/pnas.1402594111
Collins, A., & Quillian, M. R. (1972). Experiments on semantic memory and language com-
prehension. In L. W. Gregg (Ed.), Cognition in learning and memory (pp. 117– 147).
New York: John Wiley & Sons.
Coltheart, M., Patterson, K., & Marshall, J. (1980). Deep dyslexia. London: Routledge &
Kegan Paul.
Connolly, A. C., Guntupalli, J. S., Gors, J., Hanke, M., Halchenko, Y. O., Wu, Y.-C., Abdi, H., &
Haxby, J. V. (2012). The representation of biological classes in the human brain. The Journal
of Neuroscience, 32(8), 2608–2618. doi: 10.1523/JNEUROSCI.5547-11.2012
Correia, J., Formisano, E., Valente, G., Hausfeld, L., Jansma, B., & Bonte, M. (2014). Brain-based
translation: fMRI decoding of spoken words in bilinguals reveals language-independent se-
mantic representations in anterior temporal lobe. Journal of Neuroscience, 34(1), 332–338.
doi:10.1523/JNEUROSCI.1302-13.2014
Coutanche, M. N. (2013). Distinguishing multi-voxel patterns and mean activation: Why,
how, and what does it tell us? Cognitive, Affective & Behavioral Neuroscience, 13(3), 667–73.
doi: 10.3758/s13415-013-0186-2
Coutanche, M. N., & Thompson-Schill, S. L. (2014). Creating concepts from converging
features in human cortex. Cerebral Cortex, 25, 2584–2593. doi: 10.1093/cercor/bhu057
Cox, D. D., & Savoy, R. L. (2003). Functional magnetic resonance imaging (fMRI) “brain
reading”: Detecting and classifying distributed patterns of fMRI activity in human visual
cortex. NeuroImage, 19(2), 261–270. doi: 10.1016/S1053-8119(03)00049-1
Cree, G. S., & McRae, K. (2003). Analyzing the factors underlying the structure and compu-
tation of the meaning of chipmunk, cherry, chisel, cheese, and cello (and many other such
concrete nouns). Journal of Experimental Psychology: General, 132(2), 163–201. doi: 10.1037/

0096-3445.132.2.163
Çukur, T., Nishimoto, S., Huth, A. G., & Gallant, J. L. (2013). Attention during natural vision
warps semantic representation across the human brain. Nature Neuroscience, 16(6), 763–
770. doi: 10.1038/nn.3381
Damarla, S. R., & Just, M. A. (2013). Decoding the representation of numerical values from brain
activation patterns. Human Brain Mapping, 34(10), 2624–2634. doi: 10.1002/hbm.22087
Damasio, A. R. (1989). Time-locked multiregional retroactivation: A systems-level proposal
for the neural substrates of recall and recognition. Cognition, 33(1–2),  25–62.
Davis, T., LaRocque, K. F., Mumford, J. A., Norman, K. A., Wagner, A. D., & Poldrack, R. A.
(2014). What do differences between multi-voxel and univariate analysis mean? How
subject-, voxel-, and trial-level variance impact fMRI analysis. NeuroImage, 97, 271–283.
doi: 10.1016/j.neuroimage.2014.04.037
de Zubicaray, G., Arciuli, J., & McMahon, K. (2013). Putting an “end” to the motor cortex
representations of action words. Journal of Cognitive Neuroscience, 25(11), 1957–1974.
Eger, E., Michel, V., Thirion, B., Amadon, A., Dehaene, S., & Kleinschmidt, A. (2009).
Deciphering cortical number coding from human brain activity patterns. Current Biology,
19(19), 1608–1615. doi: 10.1016/j.cub.2009.08.047
Epstein, R., & Kanwisher, N. (1998). A cortical representation of the local visual environment.
Nature, 392(6676), 598–601. doi: 10.1038/33402
Friederici, A. D., Opitz, B., & von Cramon, D. Y. (2000). Segregating semantic and syntactic
aspects of processing in the human brain: An fMRI investigation of different word types.
Friston, K. J., Holmes, A. P., Worsley, K. J., Poline, J.-P., Frith, C. D., & Frackowiak, R. S. J. (1994).
Statistical parametric maps in functional imaging: A general linear approach. Human Brain
Mapping, 2(4), 189–210.
Gobet, F., & Simon, H. A. (1996). Templates in chess memory: A mechanism for recalling sev-
eral boards. Cognitive Psychology, 31(1),  1–40.
Goldstone, R. L., & Kersten, A. (2003). Concepts and categorization. In A. F. Healy & R. W.
Proctor (Eds.), Comprehensive handbook of psychology, Vol. 4: Experimental psychology (pp.
591–621). New York: John Wiley & Sons.
Hassabis, D., Spreng, R. N., Rusu, A. A., Robbins, C. A., Mar, R. A., & Schacter, D. L. (2013).
Imagine all the people: How the brain creates and uses personality models to predict beha-
vior. Cerebral Cortex, 24, 1979–1987.
Hauk, O., Johnsrude, I., & Pulvermüller, F. (2004). Somatotopic representation of action words
in human motor and premotor cortex. Neuron, 41, 301–307.
Haxby, J. V. (2012). Multivariate pattern analysis of fMRI: The early beginnings. NeuroImage,
62(2), 852–855. doi: 10.1016/j.neuroimage.2012.03.016
Haxby, J. V, Connolly, A. C., & Guntupalli, J. S. (2014). Decoding neural representational spaces
using multivariate pattern analysis. Annual Review of Neuroscience, 37, 435–456. doi: 10.1146/
annurev-neuro-062012-170325
Haxby, J. V., Gobbini, M. I., Furey, M. L., Ishai, A., Schouten, J. L., & Pietrini, P. (2001).
Distributed and overlapping representations of faces and objects in ventral temporal cortex.
Science, 293, 2425–2430.
Haxby, J. V, Guntupalli, J. S., Connolly, A. C., Halchenko, Y. O., Conroy, B. R., Gobbini, M.
I., Hanke, M., & Ramadge, P. J. (2011). A common, high-dimensional model of the
representational space in human ventral temporal cortex. Neuron, 72(2), 404–416.

doi: 10.1016/j.neuron.2011.08.026
Haynes, J.-D., Sakai, K., Rees, G., Gilbert, S., Frith, C., & Passingham, R. E. (2007). Reading
hidden intentions in the human brain. Current Biology, 17(4), 323– 328. doi: 10.1016/
j.cub.2006.11.072
Hsu, N. S., Schlichting, M. L., & Thompson-Schill, S. L. (2014). Feature diagnosticity affects
representations of novel and familiar objects. Journal of Cognitive Neuroscience, 26, 2735–
2749. doi: 10.1162/jocn_a_00661
Huth, A. G., Nishimoto, S., Vu, A. T., and Gallant, J. L. (2012). A continuous semantic space
describes the representation of thousands of object and action categories across the human
brain. Neuron, 76, 1210–1224.
James, T. W., & Gauthier, I. (2003). Auditory and action semantic features activate sensory-
specific perceptual brain regions. Current Biology, 13, 1792–1796.
Just, M. A., Cherkassky, V. L., Aryal, S., & Mitchell, T. M. (2010). A neurosemantic theory of
concrete noun representation based on the underlying brain codes. PLOS One, 5(1), e8622.
doi: 10.1371/journal.pone.0008622
Just, M. A., Cherkassky, V. L., Buchweitz, A., Keller, T. A., & Mitchell, T. M. (2014). Identifying
autism from neural representations of social interactions: Neurocognitive markers of au-
tism. PLOS One, 9(12), e113879. doi: 10.1371/journal.pone.0113879
Karuza, E. A, Emberson, L. L., & Aslin, R. N. (2014). Combining fMRI and behavioral meas-
ures to examine the process of human learning. Neurobiology of Learning and Memory, 109,
193–206. doi: 10.1016/j.nlm.2013.09.012
Kassam, K. S., Markey, A. R., Cherkassky, V. L., Loewenstein, G., & Just, M. A. (2013).
Identifying emotions on the basis of neural activation. PLoS ONE, 8(6), e66032.
Kay, K. N., Naselaris, T., Prenger, R. J., & Gallant, J. L. (2008). Identifying natural images from
human brain activity. Nature, 452(7185), 352–355. doi: 10.1038/nature06713
Kiani, R., Esteky, H., Mirpour, K., & Tanaka, K. (2007). Object category structure in re-
sponse patterns of neuronal population in monkey inferior temporal cortex. Journal of
Neurophysiology, 97(6), 4296–4309. doi: 10.1152/jn.00024.2007
Kiefer, M., & Pulvermüller, F. (2012). Conceptual representations in mind and brain: Theoretical
developments, current evidence and future directions. Cortex, 48(7), 805–825. doi: 10.1016/
j.cortex.2011.04.006
Kiefer, M., Sim, E.-J., Herrnberger, B., Grothe, J., & Hoenig, K. (2008). The sound of
concepts: Four markers for a link between auditory and conceptual brain systems. The
Journal of Neuroscience, 28(47), 12224–12230. doi: 10.1523/JNEUROSCI.3579-08.2008
Kriegeskorte, N. (2009). Relating population-code representations between man, monkey,
and computational models. Frontiers in Neuroscience, 3(3), 363– 373. doi: 10.3389/
neuro.01.035.2009
Kriegeskorte, N., Mur, M., & Bandettini, P. (2008). Representational similarity anal-
ysis: Connecting the branches of systems neuroscience. Frontiers in Systems Neuroscience,
2(4). doi: 10.3389/neuro.06.004.200
Kriegeskorte, N., Mur, M., Ruff, D. A., Kiani, R., Bodurka, J., Esteky, H., Tanaka, K., &
Bandettini, P. A. (2008). Matching categorical object representations in inferior temporal
cortex of man and monkey. Neuron, 60(6), 1126–1141. doi: 10.1016/j.neuron.2008.10.043
Lewis, J. W. (2006). Cortical networks related to human use of tools. The Neuroscientist, 12(3),
211–231. doi: 10.1177/1073858406288327
Mahon, B. Z., & Caramazza, A. (2003). Constraining questions about the organisation and
representation of conceptual knowledge. Cognitive Neuropsychology, 20(3), 433–450.
doi: 10.1080/02643290342000014
Mahon, B. Z., & Caramazza, A. (2008). A critical look at the embodied cognition hypothesis
and a new proposal for grounding conceptual content. Journal of Physiology, Paris, 102(1–3),
59–70. doi: 10.1016/j.jphysparis.2008.03.004
Martin, A. (2007). The representation of object concepts in the brain. Annual Review of
Psychology, 58, 25–45. doi: 10.1146/annurev.psych.57.102904.190143
Mason, R. A., & Just, M. A. (2015). Physics instruction induces changes in neural knowledge
representation during successive stages of learning. NeuroImage, 111, 36–48. doi: 10.1016/
Mason, R. A., & Just, M. A. (2016). Neural representations of physics concepts. Psychological
Science, 27(6), 904–913. doi: 10.1177/0956797616641941
Meteyard, L., Cuadrado, S. R., Bahrami, B., & Vigliocco, G. (2010). Coming of age: A review
of embodiment and the neuroscience of semantics. Cortex, 48(7), 788–804. doi: 10.1016/
j.cortex.2010.11.002
Mitchell, T. M., Hutchinson, R., Just, M. A, Niculescu, R. S., Pereira, F., & Wang, X. (2003).
Classifying instantaneous cognitive states from fMRI data. AMIA Annual Symposium
Proceedings, 465–469.
Mitchell, T., Hutchinson, R., Niculescu, R. S., Pereira, F., Wang, X., Just, M. A., & Newman,
S. D. (2004). Learning to decode cognitive states from brain images. Machine Learning, 57,
145–175.
Mitchell, T. M., Shinkareva, S. V, Carlson, A., Chang, K.-M., Malave, V. L., Mason, R. A, &
Just, M. A. (2008). Predicting human brain activity associated with the meanings of nouns.
Science, 320(5880), 1191–1195. doi: 10.1126/science.1152876
Moseley, R. L., & Pulvermüller, F. (2014). Nouns, verbs, objects, actions, and abstractions: Local
fMRI activity indexes semantics, not lexical categories. Brain and Language, 132, 28–42.
doi:10.1016/j.bandl.2014.03.001
Mur, M., Bandettini, P. A, & Kriegeskorte, N. (2009). Revealing representational content
with pattern-information fMRI: An introductory guide. Social Cognitive and Affective
Neuroscience, 4(1), 101–109. doi:10.1093/scan/nsn044
Naselaris, T., Kay, K. N., Nishimoto, S., & Gallant, J. L. (2011). Encoding and decoding in fMRI.
Nelson, D. L., McEvoy, C. L., & Schreiber, T. A. (1998). The University of South Florida word as-
sociation, rhyme, and word fragment norms. http://w3.usf.edu/FreeAssociation
Noppeney, U., & Price, C. J. (2004). Retrieval of abstract semantics. NeuroImage, 22(1), 164–
170. doi: 10.1016/j.neuroimage.2003.12.010
Norman, K. A, Polyn, S. M., Detre, G. J., & Haxby, J. V. (2006). Beyond mind-reading: Multi-
voxel pattern analysis of fMRI data. Trends in Cognitive Sciences, 10(9), 424–430. doi: 10.1016/
j.tics.2006.07.005
O’Toole, A. J., Jiang, F., Abdi, H., Pénard, N., Dunlop, J. P., & Parent, M. A. (2007). Theoretical,
statistical, and practical perspectives on pattern-based classification approaches to the anal-
ysis of functional neuroimaging data. Journal of Cognitive Neuroscience, 19(11), 1735–1752.
doi: 10.1162/jocn.2007.19.11.1735
Paivio, A. (1986). Mental representations: A dual- coding approach. New York: Oxford
University Press.
Patterson, K., Nestor, P. J., & Rogers, T. T. (2007). Where do you know what you know? The rep-
resentation of semantic knowledge in the human brain. Nature Reviews Neuroscience, 8(12),
976–987. doi: 10.1038/nrn2277
Pereira, F., Botvinick, M., & Detre, G. (2013). Using Wikipedia to learn semantic feature
representations of concrete concepts in neuroimaging experiments. Artificial Intelligence,
194, 240–252. doi:10.1016/j.artint.2012.06.005
Pereira, F., Mitchell, T., & Botvinick, M. (2009). Machine learning classifiers and fMRI: A tu-
torial overview. NeuroImage, 45(1 Suppl), S199–S209. doi: 10.1016/j.neuroimage.2008.11.007
Princeton University. (2010). “About WordNet.” http://wordnet.princeton.edu.
Pobric, G., Jefferies, E., & Ralph, M. A. L. (2007). Anterior temporal lobes mediate semantic
representation: Mimicking semantic dementia by using rTMS in normal participants.
PNAS, 104(50), 20137–20141. doi: 10.1073/pnas.0707383104
Poeppel, D. (2012). The maps problem and the mapping problem: Two challenges for a cog-
nitive neuroscience of speech and language. Cognitive Neuropsychology, 29(1–2), 34–55.
doi: 10.1080/02643294.2012.710600
Poldrack, R. A. (2007). Region of interest analysis for fMRI. Social Cognitive and Affective
Neuroscience, 2(1), 67–70. doi: 10.1093/scan/nsm006
Raizada, R. D. S., & Connolly, A. C. (2012). What makes different peopleʼs representations
alike: Neural similarity space solves the problem of across-subject fMRI decoding. Journal of
Cognitive Neuroscience, 24(4), 868–877. doi: 10.1162/jocn_a_00189
Sadtler, P. T., Quick, K. M., Golub, M. D., Chase, S. M., Ryu, S. I., Tyler-Kabara, E. C., Yu,
B. M., & Batista, A. P. (2014). Neural constraints on learning. Nature, 512(7515), 423–426.
doi: 10.1038/nature13665
Schwanenflugel, P. J., & Harnishfeger, K. K. (1988). Context availability and lexical decisions for
abstract and concrete words. Journal of Memory and Language, 27(5), 499–520.
Seymour, K., Clifford, C. W. G., Logothetis, N. K., & Bartels, A. (2009). The coding of color,
motion, and their conjunction in the human visual cortex. Current Biology, 19(3), 177–183.
doi: 10.1016/j.cub.2008.12.050
Shapiro, K. A., Mottaghy, F. M., Schiller, N. O., Poeppel, T. D., Flüß, M. O., Müller, H.-
W., . . . Caramazza, A., & Krause, B. J. (2005). Dissociating neural correlates for nouns and
verbs. NeuroImage, 24(4), 1058–1067. doi: 10.1016/j.neuroimage.2004.10.015
Shinkareva, S. V, Malave, V. L., Mason, R., Mitchell, T. M., & Just, M. A. (2011). Commonality
of neural representations of words and pictures. NeuroImage, 54(3), 2418–2425. doi: 10.1016/
Simmons, W. K., Hamann, S. B., Harenski, C. L., Hu, X. P., & Barsalou, L. W. (2008). fMRI
evidence for word association and situated simulation in conceptual processing. Journal of
Physiology, Paris, 102(1–3), 106–119. doi: 10.1016/j.jphysparis.2008.03.014
Stokes, M., Thompson, R., Cusack, R., & Duncan, J. (2009). Top-down activation of shape-
specific population codes in visual cortex during mental imagery. Journal of Neuroscience,
29(5), 1565–1572. doi: 10.1523/JNEUROSCI.4657-08.2009
Sudre, G., Pomerleau, D., Palatucci, M., Wehbe, L., Fyshe, A., Salmelin, R., & Mitchell, T. (2012).
Tracking neural coding of perceptual and semantic features of concrete nouns. NeuroImage,
62(1), 451–463. doi: 10.1016/j.neuroimage.2012.04.048
Takerkart, S., Auzias, G., Thirion, B., & Ralaivola, L. (2014). Graph-based inter-subject pattern
analysis of fMRI data. PLOS One, 9(8), e104586. doi: 10.1371/journal.pone.0104586
Thompson-Schill, S. L., D’Esposito, M., & Kan, I. P. (1999). Effects of repetition and competi-
tion on activity in left prefrontal cortex during word generation. Neuron, 23(3), 513–522.
Vann, S. D., Aggleton, J. P., & Maguire, E. A. (2009). What does the retrosplenial cortex do?
Nature Reviews Neuroscience, 10(11), 792–802. doi: 10.1038/nrn2733
Vigliocco, G., Vinson, D. P., Druks, J., Barber, H., & Cappa, S. F. (2011). Nouns and verbs in
the brain: A review of behavioural, electrophysiological, neuropsychological and im-
aging studies. Neuroscience and Biobehavioral Reviews, 35(3), 407– 426. doi: 10.1016/
j.neubiorev.2010.04.007
Walther, A., Nili, H., Ejaz, N., Alink, A., Kriegeskorte, N., & Diedrichsen, J. (2016). Reliability
of dissimilarity measures for multi- voxel pattern analysis. NeuroImage, 137, 188–200.
doi: 10.1016/j.neuroimage.2015.12.012
Wang, J., Baucom, L. B., & Shinkareva, S. V. (2013). Decoding abstract and concrete concept
representations based on single-trial fMRI data. Human Brain Mapping, 34(5), 1133–1147.
doi: 10.1002/hbm.21498
Wehbe, L., Murphy, B., Talukdar, P., Fyshe, A., Ramdas, A., & Mitchell, T. (2014).
Simultaneously uncovering the patterns of brain regions involved in different story reading
subprocesses. PLOS One, 9(11), e112575. doi: 10.1371/journal.pone.0112575
Weisberg, J., van Turennout, M., & Martin, A. (2007). A neural system for learning about ob-
ject function. Cerebral Cortex, 17(3), 513–521. doi: 10.1093/cercor/bhj176
Yoo, J. J., Hinds, O., Ofen, N., Thompson, T. W., Whitfield-Gabrieli, S., Triantafyllou, C., &
Gabrieli, J. D. E. (2012). When the brain is prepared to learn: Enhancing human learning
using real-time fMRI. NeuroImage, 59(1), 846–852. doi: 10.1016/j.neuroimage.2011.07.063
Chapter 22
Finding C onc e p ts
i n Brain Pat t e rns
From Feature Lists to Similarity Spaces
Elizabeth Musz and

Sharon L. Thompson-S chill
Introduction
One likes to believe that the methods used by scientists are determined by the questions
that they aim to answer, and not the other way around. However, there are instances
when one can see that the development of a new method has had the effect of changing
the questions that are asked of it, and in so doing, changes the course of scientific in-
quiry. This chapter reviews one such change of course. The story begins with the ad-
vent of cognitive neuroimaging, and with a methodological insight by Michael Posner
and his colleagues at Washington University. Posner realized that the very same logic of
“cognitive subtraction”—which could be used to isolate a single mental operation (by
comparing reaction times to two tasks that differed only by the presence of absence of
that operation; Posner, 1978)—could also be applied to isolate the neural correlates of a
single mental event (by comparing brain images; Posner, Petersen, Fox, & Raichle, 1988;
see Heim & Specht, Chapter 4 in this volume). This insight arguably created the field of
cognitive neuroimaging—a field that immediately began to yield insights into the ways
in which any complex cognitive system could be carved into parts. One such system was
semantic memory, and the subtractive approach allowed cognitive neuroscientists to
identify regions of cortex whose activity increased or decreased as a function of which
“part” of the concept was isolated. In the years that followed, the study of concepts be-
came the study of concept-parts (i.e., features), the importance of which was confirmed
by activation difference maps. This shift likely surprised cognitive psychologists who
had witnessed a move away from thinking of concepts as feature lists, prior to the ad-
vent of functional neuroimaging. However, the method of image subtraction was well
Finding Concepts in Brain Patterns 549
suited to questions about parts. Fast forward from 1988 to 2001, when an analysis of
functional magnetic resonance imaging (fMRI) data that was not based on image sub-
traction shook up the world of cognitive neuroscience: Haxby et al. (2001) described a
methodology for characterizing what was called in their abstract a “pattern of response,”
and discussions of neural patterns have permeated all areas of cognitive neuroscience
ever since. Perhaps nowhere has the consequence of this innovation been as impactful
as in the study of concepts, which increasingly are described by their patterns and not
by their parts. In this chapter, we review studies of conceptual knowledge that illustrate
this course correction, and we describe some of the challenges that lay ahead. But first,
we review some foundational principles about concepts and provide a brief tutorial into
the methodology—generally called multi-voxel pattern analysis—that changes the way
concepts are studied today.
What Is an Object Concept?
How is it that we are able to recognize and interact with things in the world that we have
never encountered before? Although we come across many novel objects throughout
our lives, they often resemble other things that we already know about. If we have de-
veloped a concept (a mental representation) that corresponds to a category of objects
(a class of things in the world that elicit a common response), then our knowledge
of the concept will allow us to identify and appropriately respond to new instance of
that category (Murphy, 2002). For instance, we can figure out whether or not a newly
encountered object is a spoon by asking ourselves whether this new thing is similar to
our concept of spoon. If the object resembles the concept, then we can infer its identity
and its properties, and use it to eat our soup. In this way, concepts serve as information
structures that link our abstract, accumulated knowledge about various things in the
world to our present, novel interactions. The speed and ease with which we can identify
new instances of familiar objects belies the complexity and impressiveness of this cogni-
tive phenomenon (see Garcea & Mahon, Chapter 23 in this volume).
Inferring Concepts via Similarities
Our concepts contain much of our world knowledge, as they allow us to infer the iden-
tity of each unique entity that we encounter. Does this new item resemble something
we have seen before? To accomplish this feat, concepts must be broad enough to ab-
stract over some variation in object properties because not all instances of a concept will
have the exact same characteristics, yet narrow enough that new entities are not mis-
takenly classified, and different sorts of objects can be discriminated from one another
(e.g., a fork versus a spoon). Hence, some variation in object properties is tolerated (e.g.,
550 Elizabeth Musz and Sharon L. Thompson-Schill
a spoon can be made of plastic or metal), yet other variation is not (e.g., a rounded end
versus a pronged end).
How do we determine which features of a concept are free to vary, and which are the
defining characteristics of that concept? Given the enormous amount of complexity
and variation in the world, it is overly simplistic, if not impossible, to define concepts
by a finite set of necessary and sufficient features. In fact, the philosopher Wittgenstein
proposed that concepts cannot be defined by specific features, but rather by “family
resemblances,” that is, sets of overlapping similarities between members of a cate-
gory (Wittgenstein & Anscombe, 1953). Objects in a category resemble other category
members more than non-members, and instances of a concept are relatively more sim-
ilar to one another than they are to instances of other concepts. This theoretical frame-
work has long provided a useful approach for studying categorization in the domains of
both cognitive psychology (e.g., prototype and exemplar theories) and cognitive science
(e.g., applications in artificial intelligence).
As rich information structures, concepts allow us to generalize and discrimi-
nate among similar entities. One way, therefore, to understand the nature of these in-
formation structures is to characterize the similarity (and dissimilarity) between
concepts. However, in studying the neural representation of concepts, neuroscientists
have only recently started to adopt similarity-based approaches (see Bauer & Just,
Chapter 21 in this volume). Modern functional neuroimaging techniques have allowed
neuroscientists to measure brain activity evoked by thoughts about concepts, rendered
visible by fMRI. By studying the similarities among patterns evoked by various concepts,
researchers can investigate how and where conceptual knowledge is represented in the
brain. In this chapter, we will briefly describe how the neural similarities between mental
representations of objects (i.e., object concepts) are computed from spatially distributed
fMRI activity patterns. Then, we will review empirical findings that relate the observed
neural similarities to various models of semantic representation. Using this approach,
these studies have contributed to our understanding of how conceptual knowledge is
stored and organized in the brain.
From Describing Features

to Characterizing Similarities
Concepts endow us with the fundamental knowledge required to interact with all of the
things that surround us. This information store comprises our semantic memory: a di-
vision of long-term, declarative memory, in which our knowledge about people, places,
and things in the world is generalized and abstracted away from any specific experience,
and is therefore considered conceptual in nature. How is this information organized,
and which brain structures support this knowledge?
Feature-Based  Models
One possibility is that concepts are organized by their respective properties. According
to feature-based theories of semantic memory, the meanings of object concepts can be
described as patterns of activation that are distributed over a concept’s various visual
and nonvisual features, like its shape and function (e.g., Allport, 1985; Barsalou, 1999;
Tyler, Moss, Durrant-Peatfield, & Levy, 2000). These models predict that concepts that
have similar features will have overlapping representations. Feature-based models also
capture both category structure and within-category individuation because concepts
from the same category will have overlapping features, yet each concept within a cate-
gory is composed by its own unique set of properties.
Over the past few decades, most neuroimaging studies on concepts have fo-
cused on testing and finding support for these sorts of models. These reports
have described the neural bases of concepts’ features by identifying, for example,
dissociations between neural activity associated with the visual versus nonvisual
attributes of objects. For instance, Martin and colleagues (1995) found that re-
trieval of action-related information about a concept is associated with activation
in middle temporal and frontal cortex, whereas retrieving color knowledge about a
concept activates bilateral ventral temporal cortex (VTC) (Martin, Haxby, Lalonde,
Wiggs, & Ungerleider, 1995). Since this report, several other studies have found that
retrieving information about different object attributes (e.g., shape; color; motion)
activates distinct and spatially distributed cortical areas (for a review, see Martin,
2007; Thompson-Schill, 2003).
While fMRI studies of object knowledge offer support for a feature-based organiza-
tion of object concepts, they have mostly constrained their inquiries to descriptions of
the conditions under which neural activity increases during conceptual knowledge re-
trieval. Such comparisons can reveal the stimulus and task conditions that give rise to
dissociable patterns in brain activity. However, this research has not fully characterized
the representation of information within these activated regions.
Moreover, on theoretical grounds, descriptions of neural activity associated
with various object properties can contribute only part of the story of conceptual
representation. After all, concepts are more than just sets of features. They are ex-
perienced as wholes, such that their properties combine and interact to jointly rep-
resent a coherent entity. In other words, concepts are more than the sum of their
parts. Describing a concept’s constituent features in isolation can offer only limited
insight into how high-dimensional, information-rich concepts are represented.
Instead, a more fruitful approach may be to investigate the relationships between
whole concepts, and where and how these relationships are neurally represented. By
characterizing the similarities between concepts and the similarities between their
associated neural activity patterns, we can learn more about how concepts are struc-
tured in the brain.
Similarity-Based  Models
One way to study the underlying neural structure is to measure the similarity be-
tween neural responses to objects that differ from one another along various stimulus
dimensions. This approach is based on the premise that what we call a “representation”
comprises representations of similarities (Edelman, 1998; Shepard & Chipman, 1970);
that is, representations do not need to resemble the things that they represent—instead,
what is important is that the representations preserve the similarity relations between
the concepts that they represent.
With this approach, representations of concepts can be described in terms of the
tuning parameters of the neurons that respond to a concept’s various semantic features.
These tuning properties can be inferred by measuring the similarity between concepts
that share features. For example, a set of neurons that encode information about the
concept spoon might evoke a similar response for a plastic spoon and a metal spoon,
but a different response for a spoon versus a fork. Such neurons are tuned to (i.e., repre-
sent) specific object shape properties, but are not sensitive to differences in the objects’
materials. Beyond information about object shape and material, the high-dimensional
representation of a concept such as spoon would also comprise several additional sim-
ilarity spaces that reflect the concept’s other properties. Each unique similarity space
may be encoded in distinct (though perhaps overlapping) brain regions or networks,
such that each similarity space could be described as one dimension of a more complete,
high-dimensional representation.
In similarity models, data from direct (i.e., neural) and indirect (i.e., subjective, psy-
chological) measurements are interpreted as proximity data that provide information
about the distance between objects in an abstract, high-dimensional space. Concepts are
encoded as points in this conceptual space, where the semantic similarity between the
two concepts is measured by concepts’ proximity to one another. In a high-dimensional
space that represents knowledge about the shape of objects, the plastic spoon would be
situated closer to the metal spoon, and farther from the fork. Additionally, one could
conceive of another dimension of this high-dimensional space that represents know-
ledge of object materials, where a plastic spoon might be located closer to a plastic
fork than it is to a metal spoon. By observing these similarity spaces, and the relative
distances between different points in the space, one can infer the object properties that
determine the relative arrangement of the points.
The standard method for computing a neural similarity space is to measure the ac-
tivity evoked by each stimulus item (i.e., the response in a set of neurons while an ex-
perimental participant views a picture of a fork, or of a spoon), and then compute the
similarity between the neural responses for each possible pair of stimulus items. The
observed neural similarity space is then related to a model of semantic similarity space,
where the similarity between every possible pair is again computed, but this time ac-
cording to the concepts’ predicted proximities. The predictions regarding the rela-
tive similarities among the stimuli come from a theoretical model of how the invoked
concepts are semantically related (i.e., subjects’ subjective ratings of the strength of
similarity between fork and spoon). Here, the general question is whether concepts that
are judged to be similar in the world, according to the model based on semantic relat-
edness, are represented by neural states that are likewise similar. With this method, one
can test predictions against the data in a manner that abstracts away from the under-
lying representational substrate (that is, the stimulus attributes and the neural activity
values, respectively).1
By comparing the neural similarity space to predicted models of the underlying se-
mantic similarity space, researchers can identify where semantic content is encoded in
the brain, generally, but also, more specifically, which components of semantic know-
ledge are reflected in different neural similarity spaces. In this way, the neural similarity
space is observed and then compared to semantic models of the space that would be
predicted if a region were sensitive to a specific type of conceptual information. For in-
stance, does an observed neural similarity space correspond to the visual features of the
concepts, or more abstract semantic properties?
Note that the preceding approach describes a hypothesis-driven similarity analysis,
where neural similarities are compared to a predicted model of similarity. However,
instead of testing the correspondence between the neural similarity space and a
predicted similarity space, one could directly extract the dimensions of a neural simi-
larity space and infer the type of information that it carries, unconstrained by the the-
oretical assumptions of any specific model. In this data-driven approach, exploratory
visualizations are used to discover natural groupings between concepts in the neural
similarity space. This could be accomplished with a variety of exploratory analytic
techniques, which we will describe in later sections.
Computing Neural Similarity Spaces

with MVPA: A Brief Tutorial
Most neural similarity analyses typically begin with the same preprocessing and estima-
tion of hemodynamic activity performed in traditional univariate methods. A pattern of
activation for a given stimulus item is then identified as a vector of activity values, equal
to the length of the number of voxels selected for analysis. The voxels included in this
vector can be selected from a variety of criteria, including anatomical and functional
constraints in a specific regions of interest (ROIs). In more exploratory analyses, multi-
voxel patterns are measured and compared in roaming “searchlights,” which are local
neighborhoods of spatially contiguous voxels iteratively sampled throughout the brain
(Kriegeskorte, Goebel, & Bandettini, 2006).
1
The versatility in this method can also be leveraged to compare similarity spaces from brain data
measured with different neuroimaging techniques (e.g., the representational structure derived from
fMRI data versus magnetoencephalography data) (cf. Cichy et al., 2016).
0
compare patterns
(e.g. 1-corr)
...
0.2 0
N brain N
...
...
...
...
...
...
...
0.5 0.8 0
representational
brain-activity dissimilarity matrix
stimuli
patterns (RDM)
Figure 22.1. In similarity-based fMRI analyses, researchers expose subjects to a set of N stim-

ulus objects, each presented in isolation. Multi-voxel patterns of neural activity evoked by each
stimulus are measured in the same set of spatially distributed voxels. Each of these multi-voxel
patterns are then compared to one another. The computed similarity value is entered into a ma-
trix, where the value assigned to a matrix cell indicates the similarity between the pair of stimuli,
which corresponds to objects which label that row and column.
Source: From Nili et al. (2014). Reprinted with permission from the Creative Commons Attribution License.
Neural similarity between two patterns evoked by two different stimuli can be
computed using measures of vector proximity (e.g., Pearson or Spearman correlation;
cosine similarity; Euclidean distance) or linear separability (cf. Weber, Thompson-
Schill, Osherson, Haxby, & Parsons, 2009). The magnitude of similarities measured be-
tween every possible pairing is often illustrated as a matrix, in which the experimental
stimuli are indexed horizontally and vertically. Each cell of the matrix contains a sim-
ilarity value, which compares the two multi-voxel patterns associated with the stimuli
that label that row and column (Figure 22.1). By computing neural similarity spaces,
researchers can examine the relative similarities and dissimilarities in the neural activity
patterns evoked by the experimental stimuli. In the following sections, we will discuss
examples in which neuroscientists have utilized this approach to probe the neural bases
of conceptual representations.
How Does the Informational

Specificity of Concepts Vary
across Brain Regions?
In order for us to flexibly use our concepts, they must be represented at varying levels
of abstraction. Objects contain a wealth of information, and a set of objects can be rela-
tively similar or dissimilar to one another, depending on the features by which they are
being compared. For example, a beetle and a moth have distinct visual characteristics
that allow us to distinguish between them, but they are also similar to each other in
many ways, and hence are both grouped into the category insects.
Which brain regions represent various object dimensions and their varying levels
of specificity? In other words, which areas of the brain are sensitive to higher-order,
cross-category distinctions between concepts, versus item-specific, within-category
distinctions? To study the similarity spaces of concepts, researchers have varied the rela-
tive degrees of specificity and abstractness between object stimuli. We begin this section
by describing various models of similarity that predict how concepts are represented in
the brain, and by summarizing the neural evidence in support of each of these models.
Discrete Models of Similarity Spaces

One straightforward similarity-based model of semantic representation predicts a
category-level organization of concepts. According to this model, objects from the same
category should be similar to one another, while objects from two different categories
should be dissimilar. To construct these models, researchers have relied upon prede-
fined semantic categories based on conventional groupings of objects. The categories
could be broad, superordinate classifications (e.g., animals) or subordinate groupings
(e.g., cats). An example of a category-based model of similarity is depicted in matrix
form in Figure 22.2. To identify brain areas where neural responses are predicted by this
131 objects
Object Category:
1
Animals
Fruits
131 objects
Similarity
Vegetables
Tools
Vehicles
Musical
Instruments
0
Figure 22.2. A model of discrete category-level similarity, similar to the model employed in
Clarke and Tyler (2014). The stimulus objects are indexed in the same order along the rows and
columns of the matrix. The color bar to the right indicates the degree of similarity indicated by
each color label, where pairings in blue indicate maximal similarity and pairings in red indicate
maximal dissimilarity. In this model, all objects within a category are predicted to evoke similar
activity patterns, and all comparisons of objects from two different categories are predicted to ex-
hibit dissimilar activity patterns. Note that the predicted patterns are symmetric and redundant on
either side of the diagonal because the predicted similarity between two objects is symmetric (e.g.,
the similarity between object 1 and object 2 is equal to the similarity of object 2 versus object 1).
sort of model, researchers have measured the multi-voxel patterns evoked by pictures of
various real-world objects that span a number of object categories. A similarity matrix
is then constructed, to compare the pairwise similarity between responses evoked by
each possible pairing of two objects from the same category, versus the neural responses
evoked by pairs of two objects from two different categories.
Using this approach, researchers have found that neural activity in posterior and ven-
tral regions of temporal cortex (VTC) exhibit this coarse level of category-based sim-
ilarity. For example, Haxby et al. (2001) found that in object-selective regions of VTC
and ventrolateral occipital cortex, patterns evoked by the same subordinate object class
(e.g., one shoe versus another shoe) more frequently exhibited greater similarity, relative
to patterns evoked between categories (e.g., a shoe versus a bottle). Additionally, Clarke
and Tyler (2014) found that searchlight volumes in lateral occipital cortex (LOC), pos-
terior VTC (pVTC), and left perirhinal cortex exhibit more similar neural responses
for objects from the same superordinate category, relative to noncategory members. In
addition to category-level distinctions, Clarke and Tyler found that a model based on
domains of object animacy (nonbiological—plant—animal) predicted neural similarity
spaces in bilateral medial pVTC and right lateral pVTC. Taken together, these findings
indicate that subregions of VTC encode coarse distinctions between predefined object
classifications.
Continuous Measures of Similarity Spaces

Instead of comparing neural responses evoked by traditionally defined object categories,
it may be more informative to examine the neural similarity spaces evoked by individual
concepts, both within and between object categories. To characterize finer-grained
distinctions between real-world objects, researchers have begun to study the item-
specific signals within distributed patterns of neural activity. These studies employ par-
ametric analyses of similarity, where neural responses are related to descriptions of a
stimulus space that is more detailed and continuous than binary category membership
assignment.
The similarities between stimulus items can be measured with a variety of methods.
The main division is between hypothesis-driven versus data-driven similarity analyses.
In hypothesis-driven approaches, the data are used to test predefined, theoretically
motivated predictions. In contrast, in data-driven approaches, the results reflect the
data. The former method allows one to test specific models of similarity, whereas the
latter approach allows one to discover the similarities that are present in the neural data.
Below, we review the applications of each of these methods.
In VTC, a common pattern of results in emerges from these two types of analyses.
Several studies have identified a gradient of informational specificity in ventral brain
regions, whereby responses in more anterior regions increasingly reflect sensitivity
to higher-level and item-specific properties, such as object shape and category, while
more posterior regions in early visual cortex (EVC) are sensitive to lower-level visual
properties. The functional and anatomical dissociations between the neural similarity
spaces that are predicted by these divergent models indicate that different brain regions
are sensitive to different variations across stimulus features.
Predicting Item-Level Similarities

One way to investigate finer-grained, within-category similarity spaces is to compare
item-level neural similarity spaces to models that are based on continuous, rather than
categorical, dimensions. These models are constructed according to what one would
predict the similarity space would look like if it were sensitive to graded variations
among stimulus items.
To construct a model of item-level semantic similarity, researchers often rely upon
explicit, subjective assessments of the pairwise similarities between two stimuli. To col-
lect these judgments, researchers instruct participants to report the relative similarity
between each possible pairing of the experimental stimuli. Researchers then use these
models to examine, for example, whether participants’ judgments of the similarity be-
tween a bear and a zebra correspond to the similarity of their respective neural activity
patterns. Weber and colleagues (2009) used this approach to examine conceptual simi-
larity among various mammals (e.g., camels, hippos). The pairwise similarities between
the multi-voxel patterns evoked by mammal pictures were correlated with participants’
post-scan, pairwise rankings of conceptual similarity for the same set of mammals.
Neural similarity and the behavioral similarity measures were correlated in object-
selective functional regions of interest (fROIs) in bilateral LOC. In a similar study,
Connolly et al. (2012) examined the neural similarity space evoked by exemplars from a
wider range of animals. This study compared the neural responses evoked by six animal
species, from three biological classes (primates, birds, and insects). Participants’ subjec-
tive similarity ratings predicted the neural similarity spaces in searchlight clusters cen-
tered in bilateral LOC. Moreover, in both of these studies, the neural similarity spaces
observed in EVC did not match the subjective similarity ratings.
In contrast, the neural similarity spaces in EVC are sensitive to lower-level visual
features. Researchers develop image-based models of similarity, which are independent
of semantic information like object identity or category membership. These image-
based models are typically constructed by computing the pixel-wise similarity between
stimulus pictures (e.g., Weber et al., 2009) or by comparing the stimulus pictures with
a set of spatial filters and then simulating the responses of V1 complex cortical cells
(e.g., Connolly et al., 2012; Clarke & Tyler, 2014; cf. Serre, Wolf, Bileschi, Riesenhuber,
& Poggio, 2007). Importantly, these low-level, image-based models predict the neural
similarity spaces observed in EVC, but not VTC or LOC. These findings indicate an an-
atomical dissociation between more posterior, early visual regions, which are sensitive
to visual similarities like luminance and line orientation, and progressively anterior re-
gions in VTC and LOC, which correspond to subjective ratings of similarities between
pictures of objects.
Discovering Item-Level Similarities

Instead of comparing neural similarity spaces to predicted models, researchers have
also used exploratory, data-driven methods, such as multidimensional scaling (MDS)
and hierarchical clustering, to project higher-dimensional similarity spaces onto a more
visualizable, lower-dimension space. Researchers use these techniques to visualize the
similarities that emerge in a particular brain region, and to discover the aspects of rep-
resentational space that a given region encodes. Rather than testing hypotheses about
specific patterns in the neural data, this approach allows researchers to look for inter-
pretable relationships that are present within the observed neural patterns. Results from
these sorts of analyses have largely conformed to the findings from hypothesis-based
approaches, although they have extended some results as well.
Using data-driven analyses, Kriegeskorte and colleagues (2008) found that neural
similarity patterns in object-selective VTC generally conformed to conventional human
categories. Here, the authors measured the multi-voxel patterns evoked by 92 real-world
objects, and then used hierarchical clustering methods and MDS to interpret patterns
in the resulting neural similarity space. The resulting clusters of objects revealed a
broad distinction between neural patterns evoked by animate and inanimate objects,
as well as more fine-grained distinctions within categories, where faces and body parts
formed sub-clusters within the animate objects (Figure 22.3). Moreover, models based
on low-and intermediate-complexity stimulus features (e.g., luminance, silhouette) did
body natural obj.

face artificial obj.
Figure 22.3. Data-driven, multidimensional scaling applied to neural similarity data extracted

from human inferotemporal cortex in Kriegeskorte et al. (2008). The high- dimensional
similarities between neural activity evoked by a variety of real-world objects are projected in two-
dimensional space. This analysis reveals which objects tend to evoke more similar neural activity
patterns. The similarity space on the right illustrates that assigning these stimuli labels based on
their broad semantic category (e.g., body or face) reveals consistent distinctions between body
parts and objects.
Source: Kriegeskorte et al. (2008). Reprinted with permission from Elsevier via RightsLink.
not account for the observed structure. In contrast, neural patterns in EVC exhibited
weak category specificity, reflecting a broad distinction between animate and inanimate
objects, but lacked the fine-grained distinctions within categories that were observed in
VTC. This finding is consistent with a hierarchical organization in the ventral stream,
where regions anterior to EVC code more complex stimulus features, such as object
form and identity.
While Kriegeskorte et al. (2008) demonstrate how data- driven, bottom- up
methods can be employed to study the brain’s functional distinctions between broad
categories, these methods have also enabled researchers to study finer distinctions
between representations in greater detail. These analyses reveal differences in neural
similarity spaces that were not well captured by similarity ratings. For example, in ad-
dition to their model-based analysis that compared neural activity patterns evoked
by primates, birds, and insects, Connolly and colleagues (2012) also performed
a cross-participant clustering analysis, such that the most similar neural similarity
spaces across participants would cluster together. This analysis revealed the neural
similarities observed in LOC were highly consistent across participants. In fact, the
participants’ neural similarity spaces in this region were more similar to one an-
other than they were to the similarity space defined by pairwise similarity ratings of
the stimuli, suggesting that the neural activity here reflects information that is not
captured by the similarity model. Applying MDS to this region revealed that the most
prominent dimension in LOC neural data reflected a continuum of animacy, where
the most animate animals (primates) clustered at one end, and the least animate
animals (bugs) clustered at the other end, with birds in-between the two. A follow-
up study revealed a high degree of neural similarity between LOC activity patterns
evoked by low-animacy animals (e.g., lobsters and ladybugs) and inanimate objects
(e.g., tools and keys), despite the fact that participants’ behavioral judgments reflected
a dichotomous distinction between animate and inanimate objects (Sha et al., 2015).
Although the LOC data did not quite conform with expectations of semantic struc-
ture according to the behavioral judgments, the complementary, data-driven analyses
uncovered these compelling findings.
Perceptual or Semantic Similarity Spaces?

Which stimulus dimensions are driving the observed neural similarity spaces in ventral
temporal regions? Relatedly, what sort of information does a participant rely upon to ex-
plicitly assess the similarity between two stimulus objects? These questions highlight the
challenges in identifying the stimulus dimensions that underlie subjective ratings of se-
mantic similarity, and relatedly, the stimulus dimensions that best predict the observed
neural similarity space.
One possibility is that the subjective similarity ratings, and the corresponding neural
patterns observed in VTC and LOC, are largely driven by visual effects. This possi-
bility is supported by a couple of key points. For one, because the studies reviewed in
the preceding sections presented concepts in pictorial format, the perceptual attributes
of the concepts were more prominently featured than their nonvisual, more abstract
properties. Additionally, while viewing these pictures in the scanner, participants were
only required to retrieve minimal semantic information about the concept (e.g., ex-
emplar naming in Clarke & Tyler, 2014; exemplar repetition detection in Weber et al.,
2009), or no semantic information at all (e.g., detecting repetitions of stimulus pictures
or a fixation cross, as in Haxby et al., 2001; Kriegeskorte et al., 2008; Connolly et al.,
2012). Moreover, there is some evidence that these putatively within-category sensitive
regions (i.e., LOC) also reflect subjective ratings of shape similarity for artificial shape
stimuli, which presumably elicit minimal semantic information (Drucker & Aguirre,
2009; Op de Beeck, Torfs, & Wagemans, 2008).
Additionally, subjective assessments of pairwise similarity might also reflect the per-
ceptual relatedness of the stimulus pictures. Weber and colleagues (2009) collected sep-
arate ratings based on their stimuli’s conceptual similarity, biological similarity, and
perceptual similarity. The judgments obtained according to these three dimensions
were highly correlated with one another, such that it is not possible to determine the
unique and relative contributions of each dimension for predicting the observed neural
similarity structure. Likewise, in Connolly et al. (2012), participants rated the stimulus
pictures based on their general similarity to one another, rather than according to ex-
plicitly semantic relationships. Here, too, the participants may have based their ratings
on the perceptual similarity of the images, rather than semantic information about the
concepts per se.
These methodological issues reflect the challenge in determining the extent to
which the observed neural similarity spaces reflect both visual and nonvisual con-
tent, and the extent to which this information is conceptual in nature. One way to
address this issue is to employ carefully controlled experimental stimuli, such that the
dimensions of category membership and perceptual similarity are explicitly dissoci-
ated from one another; that is, stimuli must be equally perceptually similar to both
category members and to noncategory members. In a fully crossed design, Bracci
and Op de Beeck (2016) measured the neural similarities evoked by pictures of six
different object categories (e.g., animals, minerals), where each category included
an exemplar that had one of nine possible shape forms (e.g., roughly spherical, ver-
tical oblong). In a similar approach, Proklova, Kaiser, & Peelen (2016) investigated
the animate-inanimate distinction using pictorial stimuli where shape similarity was
equated across the two object categories (e.g., a snake versus a rope). Such designs
enable researchers to identify neural similarity spaces that correspond to the visual
versus categorical (e.g., conceptual) distinctions among the tested stimuli. These
studies revealed that shape and category information can independently and jointly
contribute to neural similarity spaces throughout VTC. In the coming pages, we will
discuss additional strategies that researchers have used to more directly target con-
ceptual information about objects, independent of perceptual and non-semantic
characteristics of the experimental stimuli.
Concepts as Feature Lists
To address some of the shortcomings of subjective similarity judgments, researchers
have constructed alternative models of item-level similarity. For example, one could first
catalog the visual and nonvisual semantic features that are typically associated with each
concept. Then, the list of features for each concept could be compared to one another, to
assess how well the two concepts align on their various features. This approach allows
researchers to characterize meaningful dimensions that are not well captured by explicit
similarity judgments of picture stimuli (i.e., more abstract and nonvisual features).
To construct these models, researchers first collect feature-norming data. In these
tasks, behavioral participants are presented with a concept name and instructed to de-
scribe as many of the concept’s descriptive features as they possibly can (McRae, Cree,
Seidenberg, & McNorgan, 2005). These features include a concept’s perceptual attributes
(e.g., “is round” for apples), as well as its more propositional properties (e.g., “eaten in
pies”). After collecting feature lists associated with each concept, researchers can code
the responses as a binary vector, indicating whether each potential feature is associated
with the concept. Then, the semantic similarity of individual objects can be compared
to one another by calculating the proximity between their respective feature vectors,
much in the same way that multi-voxel activity patterns are compared in neural simi-
larity analyses.
Whereas studies that applied models of explicit, item-level judgments observed
effects in pVTC and LOC, empirical work using models based on item-level fea-
ture vectors has found effects in perirhinal cortex. Clarke and Tyler (2014) scanned
participants while they performed a basic-level naming task (e.g., “apple”) for pictures
of 131 different objects from a variety of categories. The similarity model was created
from semantic feature norms, defined by lists of features associated with each concept.
This analysis yielded an average of 13 descriptive features per concept, which included
both visual and nonvisual properties.
Because variations in semantic features could be correlated with low-level visual
properties and with taxonomic category membership, the authors also computed sim-
ilarity structures generated according to these two additional spaces. In a whole-brain
searchlight analysis, the authors used a partial correlation analysis to fit all three models
at the same time. With this approach, they could examine the unique contribution of
each model in predicting the neural similarity structure.
In agreement with prior findings, Clarke and Tyler (2014) found that category-based
representations are most prominent in the posterior ventral stream, and that patterns
in early visual areas matched the models of lower-level visual features. However, un-
like previous work, the authors also detected more fine-grained similarity patterns
in regions anterior to LOC. The object-specific model based on semantic features
predicted the neural similarity space observed bilaterally in anterior medial temporal
lobe and perirhinal cortex. Critically, these findings remained significant, even after
controlling for neural similarity that tracked the category-level model and low-level
visual feature model. Taken together, the results indicate that there are coarse, cat-
egorical representations most prominently in the posterior ventral stream, and more
fine-grained similarity patterns in the anterior medial temporal lobe, which predict
object-specific semantic similarity above and beyond that which is explained by models
of categorical or visual similarity.
As reviewed earlier, prior studies have found that activation patterns in the poste-
rior ventral stream correlate with human ratings of pairwise similarity (e.g., Connolly
et al., 2012; Weber et al., 2009). In Clarke and Tyler’s (2014) study, semantic feature
effects were initially found in posterior VTC, but these effects became nonsignificant
once the authors accounted for the variance in the neural data explained by the models
of categorical and visual similarity. This finding leads to a couple of ways to interpret
object-specific effects that have previously been reported in VTC. One is that within-
category effects that have been previously reported in posterior VTC are driven by
visual similarity.
Alternatively, it is possible that VTC activity encodes conceptual information that is
not well captured by the object-specific model employed by Clarke and Tyler (2014).
Although participant-produced feature lists have some advantages over explicit sim-
ilarity judgments, there are also some drawbacks to this metric of similarity. Feature
norming data tend to underrepresent information that is obvious or highly shared
among the sampled concepts (e.g., “breathes”; “is solid”). Moreover, the produced
features are limited to those that participants can easily verbalize. It will be a chal-
lenge for future work to examine the correspondence between semantic similarity
models constructed from feature vectors and explicit pairwise ratings, and whether any
divergences between these models can further characterize how object concepts are
represented in the ventral stream.
Do Similarity Spaces
Encode Modality-Independent
Semantic Information?
Thus far, we have reviewed studies that examine the neural similarity spaces evoked by
pictures of real-world objects. As noted earlier, the use of visual stimuli makes it difficult
to distinguish between the contributions of perceptual versus semantic properties on
an observed pattern of activity. One strategy for avoiding stimulus-driven perceptual
effects is to measure multi-voxel patterns that are evoked when concepts are presented
as words, because the semantic similarity and orthographic similarity are orthogonal
stimulus dimensions.
In addition to using word stimuli, neuroscientists are interested in the similarities and
differences in multi-voxel patterns evoked by a concept when it is accessed using dif-
ferent stimulus modalities (e.g., in picture or word form). In order to understand the
meaning of a concept denoted by a word or a picture, we must retrieve the underlying

representation, and meaning is evoked regardless of the format in which the concept is
retrieved. For example, reading the word hairbrush should lead to the retrieval of con-
ceptual content that is generally similar to the representation that is activated when one
identifies a picture of a hairbrush. Brain areas that are involved in semantic retrieval are
interpreted as representing high-level object representations that can be accessed from
different stimulus modalities.
To what extent is meaning encoded in modality-independent systems? In attempts
to further target neural representations of conceptual information, researchers have
investigated the extent to which the multi-voxel patterns evoked by object concepts
are invariant to the stimulus format with which the representations are accessed.
These studies have implicated a number of regions, but researchers are still actively
investigating the extent to which these putatively modality-independent neural simi-
larity spaces represent conceptual information.
Cross-Modal Category Decoding

One way to identify regions that encode semantic information across different stimulus
modalities is to use MVPA decoding methods. In these experiments, researchers test
whether a classification algorithm trained on neural data from one modality can ac-
curately classify unseen neural data evoked by the stimuli when they are presented in
a different modality (Kaplan, Man, & Greening, 2015). Several studies have accurately
classified stimulus classes across two stimulus modalities (e.g., words and pictures of
tools versus dwellings in Shinkareva, Malave, Mason, Mitchell, & Just, 2011; spoken and
written words of animals versus tools in Akama, Murphy, Na, Shimizu, & Poesio, 2012),
and, in bilingual subjects, from concepts accessed in one language to another (e.g.,
Buchweitz, Shinkareva, Mason, Mitchell, & Just, 2012; Coreira et al., 2014). The brain
areas that are identified by this line of research are interpreted as carrying information
about the stimulus content, abstracted from the format in which the information is
presented.
In one application of this approach, researchers were able to classify objects according
to their semantic category across four different stimulus modalities. Subjects performed
a semantic categorization task on various animals and tools. These stimuli were sep-
arately presented as photographs, written names, spoken names, and natural sounds
(Simanova, Hagoort, Oostenveld, & van Gerven (2012). The authors used a classifier al-
gorithm to discriminate between the animals and tools by iteratively training the classi-
fier based on data from three of the modalities, and then testing the classifier’s decoding
accuracy on the fourth, excluded modality. The authors found that large portions of left
VTC, bilateral frontal gyrus, and posterior middle temporal gyrus (pMTG) exhibited
cross-modal classification of category membership. These results suggest that the neural
activity patterns evoked by animals versus tools are discriminable, regardless of whether
the stimulus objects are presented auditorily, visually, or verbally. The authors suggest
that these regions support semantic knowledge representation by integrating informa-

tion that originates from different input streams, and abstracting from low-level percep-
tual features to higher-level conceptual processing.
Cross-Modal Similarity Spaces
To what extent do regions implicated in modality-independent processing contribute
to the representation of conceptual content? Perhaps the brain regions that can de-
code object category across modalities perform distinctive mental operations for each
category, but these cognitive processes might not reflect semantic information. Some
researchers have suggested that in order for a region that exhibits cross-modal decoding
to be interpreted as representing conceptual content, it should play a role in distin-
guishing among representations of different objects and object categories; that is, the
activity patterns in such a region would correspond with a model of semantic similarity.
To addition to broad classification tests, researchers have also examined neural simi-
larity spaces within brain regions that exhibit cross-modal decoding.
Between-Category Cross-Modal Findings

To further gain traction on whether the neural activity in putatively modality-
independent brain regions reflects semantic information, Fairhall and Caramazza
(2013) conducted an MVPA study using word and picture stimuli of objects from five se-
mantic categories (fruits, clothes, tools, mammals, and birds). These authors proposed
that if a region exhibits cross-modal decoding, in order to represent conceptual content,
that region should support distinctions between different object categories. The authors
tested this prediction by searching for modality-independent brain regions whose
neural similarity structures matched category-defined semantic similarity structures.
During scanning, subjects made category typicality judgments in response to indi-
vidual exemplars from five different categories (e.g., how typical is an apple of the cate-
gory fruit?). The stimuli were presented as words in the first half of the experiment and
then as pictures in the second half, so that word presentations were not confounded by
prior exposure to particular images. The authors then trained a classifier to decode the
distinct neural activity associated with each of the five semantic categories, independ-
ently of the process through which these representations were accessed (i.e., the stim-
ulus modality). The classifier was then trained on the category-evoked patterns from
each modality for subsequent testing on the data from the other modality (e.g., train on
picture data; test on word data, and vice versa). Hence, only the category-specific infor-
mation that was general to both modalities was informative to the classifier.
In regions that exhibited cross-modal category decoding, the authors related the
observed category-level neural similarity space to the predicted model of category-based
similarity structure. Rather than predicting a binary model of category membership, the
authors quantified continuous gradations of similarity between each category pair. Such
a model quantifies the similarity space at the category level, where, for example, birds
are more similar to mammals than they are to fruits.
The cross-modal MVPA searchlight classifications revealed a network of six left-
lateralized regions, mostly outside of category-selective visual cortex, in which there
was overall cross-modal sensitivity to semantic category. The cross-modal decoding
was identified in VTC, which included fusiform gyrus, parahippocampal gyri, and
perirhinal cortex; pMTG; angular gyrus (AG); posterior cingulate and precuneus; and
lateral and dorsomedial prefrontal cortex. Additionally, cross-modal neural similarity
spaces in left VTC and left pMTG predicted semantic relationships among the object
categories. This correspondence supports the view that these regions encode modality-
independent conceptual information.
The identification of six regions that are sensitive to semantic category information
across modalities—but only two of which are sensitive to the semantic similarities
among categories—potentially indicates a functional dissociation between areas that
represent category-based semantic similarities, and areas that are engaged in con-
ceptual processing regardless of stimulus modality, but are not necessarily involved
in representing conceptual content. More specifically, a brain region implicated in
modality-independent conceptual processing might play a role in operating upon or
accessing semantic representations, without necessarily representing the content it-
self. These potential distinctions between cognitive operations and cognitive content
are entirely speculative; we will return to the issue of interpreting whether multi-voxel
patterns reflect semantic processing versus semantic representations in a later section.
Comparing Word and Picture Similarity Spaces

These studies demonstrate that classifier-based decoding methods can identify regions
involved in cross-modal semantic processing by training on data from one modality
and testing against data from the other modality. Additionally, the findings in Fairhall
and Caramazza (2013) suggest that some of these regions are also sensitive to category-
level similarities between objects. But do modality-independent regions represent the
same similarity relations among semantic representations, regardless of whether they
are accessed by word or picture format? A region might be involved in semantic pro-
cessing for both words and pictures, but that region might engage in distinct, modality-
specific networks for each modality. In such a case, the underlying neural similarity
spaces might differ across modality, even if the region encodes semantic content from
each of the modalities.
To determine the computational commonalities and differences for accessing
concepts in picture or word format, Devereux and colleagues (2013) compared neural
similarity spaces evoked by concepts first presented as words and then as pictures. The
multi-voxel patterns were computed from brain activity while subjects named the cate-
gory of the presented stimulus object (e.g., “clothes” for a sweater). The objects included
10 different exemplars from six common semantic categories (e.g., insects; vegetables).
A category-level similarity model was constructed, such that stimulus pairs from the
same category were predicted to evoke more similar patterns than pairs from different
categories (as in Figure 22.2).
The neural similarity structures yielded by the word data and the picture data
were then compared to one another. This analysis tests whether the pairwise neural
similarities observed in response to the word stimuli match the neural similarity space
observed from the picture stimuli. Further, to determine the extent to which these
neural similarity spaces reflected semantic content, the word-based and picture-based
neural similarity spaces were each compared to the model of category-level representa-
tion. The category-level model matched the picture-evoked data in VTC, and the word-
evoked data in anterior MTG. Additionally, several regions exhibited neural-semantic
correspondence for each separate stimulus modality, including the MTG, AG, and left
inferior parietal sulcus (IPS).
To explore the similarities between the neural similarity spaces across modalities,
the neural similarity spaces from the two modalities were compared in a data-driven
clustering analysis, such that similar spaces would cluster together. This method can
identify representational invariance both across modalities, and also across brain re-
gions, because the clustering algorithm is blind to whether similar searchlights spatially
correspond to one another. In contrast, cross-modal classifier methods, like those used
by Simanova et al. (2012) and Fairhall and Caramazza (2013), presuppose that common
representational content is only found in the same set of corresponding voxels across
modalities.
This analysis revealed that left IPS was relatively invariant to stimulus modality, as its
word and picture neural similarity spaces clustered together. The representational in-
variance observed here suggests that the semantic feature information required to per-
form the category-naming task does not differ as a function of stimulus modality in this
region. Additionally, though word-based and picture-based neural similarity structures
in left MTG each separately correlated with the semantic model, they did not cluster to-
gether. These findings suggest that the left MTG performs modality-specific functional
roles that yield distinct yet overlapping neural responses. Such a finding has impor-
tant consequences for research investigating modality-invariant semantics, because it
demonstrates that identifying regions involved in both word and picture processing—
and even discovering that they both correspond to a common semantic similarity
model—is insufficient to claim that they form part of a common modality-independent
semantic network.
The analyses in Devereux, Clarke, Marouchos, and Tyler. (2013) go beyond identifying
individual regions involved in modality-independent, category-sensitive processing,
and illustrate how the response properties of a region can vary as a function of the
modality-specific network in which it is engaged. Whereas findings in Fairhall and
Caramazza (2013) indicated that MTG encodes semantic content in both modalities,
Devereux and colleagues (2013) revealed that each modality evokes a distinct neural
similarity space in MTG. These advanced approaches are uniquely enabled by the
comparison of neural similarity spaces evoked by multi-voxel patterns, independent of

the modality in which they are sampled.
Within-Category Cross-Modal Findings

In contrast to reports of category-level distinctions and similarities across modalities,
Bruffaerts et al. (2013) and Liuzzi et al. (2015) limited their investigation of cross-modal
neural similarity spaces to concepts from a single superordinate category: animals.
Although these two studies employed the same stimuli and experimental task, they
probed neural responses using two different stimulus modalities: Bruffaerts et al. (2013)
visually presented the animal stimuli as words and pictures, and Liuzzi et al. (2015)
presented them as written words (i.e., visual presentation) and spoken words (i.e., au-
ditory presentation). In both cases, semantic clusters and concept similarities were de-
termined in a data-driven manner, on the basis of a concept-feature data matrix derived
from a behavioral study, similar to the feature-norming data used by Clarke and Tyler
(2014). During scanning, subjects performed a property verification task on the stimuli
(e.g., “has wings?”).
Bruffaerts et al. (2013) and Liuzzi et al. (2015) tested whether the average neural simi-
larity in response to stimuli from the same within-category semantic cluster was greater
than would be expected by chance. The analysis was performed regardless of stimulus
modality, and then again in each modality separately. Bruffaerts et al. (2013) found that
the left perirhinal cortex and left anteromedial fusiform gyrus exhibited above-chance
within-cluster similarity. They observed this pattern of results when word and picture
responses were pooled together; however, a follow-up analysis revealed that the neural
responses evoked by the word stimuli were driving these effects. Liuzzi et al. (2015)
replicated this correspondence between the neural and semantic similarity spaces in the
same area of left anteromedial fusiform gyrus. However, as in Bruffaerts et al. (2013),
this effect was limited to visually presented words. Moreover, Liuzzi et al. (2015) also
observed effects for written words in left perirhinal cortex: this region exhibited greater
neural similarity for stimuli from the same versus a different semantic cluster.
The results from Bruffaerts et al. (2013) and Liuzzi et al. (2015) indicate that, in re-
sponse to visually presented words, semantic similarity is reflected at a fine-grained
level in regions of left perirhinal cortex. This finding comports with Clark and Tyler’s
(2014) observed correlations between fine-grained measures of feature similarity and
object-specific responses to pictures in bilateral perirhinal cortex. Additionally, this re-
gion partly overlaps with the ventral temporal region that exhibited cross-modal cate-
gory selectivity in Fairhall and Caramazza (2013).
It is unclear why responses in left perirhinal cortex exhibited cross-modal effects in
Fairhall and Caramazza (2013), but not in Bruffaerts et al. (2013) or Liuzzi et al. (2015).
One possible explanation for null result in response to spoken-word presentations is that
the perirhinal cortex is specialized for visual input. Additionally, the authors suggest
that the degree to which perirhinal cortex reflects cross-modality semantic similarity
might vary as a function of task demands. For instance, perhaps subjects performed
some of the property verification tasks (e.g., “smooth?”; “exotic?”) on picture stimuli
in Bruffaerts et al. (2013) by relying upon the depicted perceptual features, and without
accessing semantic information. This raises the possibility that neural similarity spaces
are dynamically influenced by task-and stimulus-based factors. We discuss this topic in
the next section.
How Are Similarity Spaces Altered

by Experience?
Thus far, we have reviewed studies that examined correspondences between neural and
semantic similarity spaces. These studies also have explored how the neural similarity
structures vary across brain regions and across stimulus modalities. In addition to these
broad research questions, some studies have investigated how neural similarity spaces
might vary across tasks or individual experience. This line of research reveals that neural
activity evoked by objects—and consequently, the observed neural similarity space—
can flexibly adapt in response to changes in experimental task or context.
Manipulating Object Category

One potential way to alter a similarity space is to manipulate the salience of the
dimensions that distinguish between object categories. One might predict that if cat-
egory members are modified such that they receive more similar responses to one an-
other and more distinct responses relative to non-members, then these changes might
be evident in the resulting similarity space. A study by Dunsmoor and colleagues (2013)
tested how one such category-level manipulation might affect a neural similarity space.
These authors investigated how category-level neural similarity structures can be
modulated by aversive fear learning. In this study, fMRI subjects viewed images from
two object categories: animals and tools. During the scanning session, basic-level
exemplars were presented one at a time while subjects rated their expectancy of re-
ceiving a shock. Unbeknownst to the subjects, one group of subjects learned through
experience that presentations of animal stimuli were sporadically associated with an
aversive electrical shock and that images of tools were safe, and a separate group learned
the opposite contingencies. The authors predicted that multi-voxel patterns of activity
in object-selective cortex would exhibit enhanced representational similarity among
different category members from the feared category. Such a mechanism might facilitate
the transfer of affective learning between semantically related objects.
For this analysis, the authors examined patterns of neural activity within object-
selective regions of bilateral occipitotemporal cortex. The authors compared the average
pairwise neural similarity within and between object categories, to determine whether
the within-category neural similarity for the shocked category was different from the
within-category neural similarity for objects from the safe category. This analysis re-
vealed that in both subject groups, activity patterns in object-selective cortex were more
similar among exemplars from the threat versus safe category.
These results suggest that aversive learning selectively enhances representational sim-
ilarity among categorically related exemplars. The representational structure of these
categories was functionally altered in an experience-dependent fashion, as aversive
learning selectively enhanced the neural similarity space of the object category that
acquired threat value. The authors hypothesize that this effect might support within-
category generalization, such than an emotional experience with one instance of an
object category leads to generalizations about the properties of related objects through
induction (Murphy, 2002).
Manipulating Object Features

Rather than modifying the salience or the distinctiveness of the response required for
an entire object category, one could also modify the salience of a particular stimulus
dimension by manipulating whether or not that dimension was necessary to perform a
cognitive task. One might predict that if one stimulus dimension (e.g., object color) was
made relevant for one group of subjects but not for another, then the first group’s neural
similarity space might reflect sensitivity to this dimension, while the second group’s
would not.
This prediction was tested in a paper by Hsu and colleagues (2014). To examine
the impact of feature diagnosticity on concept representations, these authors taught
participants a set of novel objects that were each labeled with a novel name. In a
between-subjects design, the authors manipulated whether or not color knowledge was
necessary for identifying each novel object. Half of the subjects learned that shape was
sufficient to distinguish between objects in the stimulus set (the S subjects), while the
other half learned that the conjunction of shape and color was diagnostic of object iden-
tity (CS subjects). After subjects were trained to learn the object names, subjects’ neural
activity was measured while they performed a shape memory task on these objects.
The authors hypothesized that the retrieval of diagnostic feature information would
yield group-level differences in the extent to which pairwise similarity ratings of these
objects would predict the neural similarity space constructed from neural activity meas-
ured during the shape knowledge task. Specifically, the authors predicted that (1) ge-
neral similarity ratings and (2) ratings based on color similarity of the stimuli would
predict the neural similarity space observed in the CS subjects, but not in the S subjects.
Moreover, this effect should emerge in color-sensitive brain regions.
For this analysis, a color-sensitive region of left fusiform gyrus was used as an fROI.
The authors extracted the multi-voxel patterns evoked in response to shape retrieval
associated with each object and computed the resulting similarity space. They then
assessed whether this neural similarity space could be predicted by three sets of behav-
ioral similarity ratings: one previously obtained from the trained subjects in response
to the object names, and two obtained from untrained subjects, who rated the object
pictures based on their color similarity.
The results indicated that the color similarity ratings approached significance for
predicting the neural similarity space in the CS subjects, but not for the S subjects.
Further, the between-group correlations were reliably different from one another.
Because color did not provide the same diagnostic information for S subjects as it did
for the CS group, this may explain the lack of correlation between color-based similarity
and neural similarity for the S group. These findings demonstrate that the use of feature
knowledge affects conceptual representations, and that the learned context of an object
can influence its conceptual representation.
Manipulating Object Familiarity

To explore how object representations in VTC differ across individuals, Charest et al.
(Charest, Kievit, Schmitz, Deca, & Kriegeskorte, 2014) measured neural activity patterns
while subjects viewed two separate stimulus sets: one composed of pictures of faces,
places, bodies, and objects that were personally meaningful to the subject (e.g., a friend’s
face, his or her own backpack), and one set composed of pictures that were unfamiliar
to the subject (e.g., pictures that were personally meaningful for another, unknown
subject). For each separate stimulus set, subjects also judged the relative similarities
among the pictures. The authors found that the neural similarities in VTC more closely
matched each subject’s own similarity judgments (relative to other subjects’ judgments)
when the stimulus set was personally meaningful. Such a result suggests that the neural
similarity spaces in VTC reflect our unique, subjective perception of real-world objects
and their similarity relationships. Moreover, this finding illustrates that subtle indi-
vidual differences in neural representations of objects can be uncovered with similarity-
based approaches.
What Do Multi-Voxel
Patterns Measure?
Theoretical Challenges
As with any method, the results yielded by similarity-based analyses must be interpreted
with some caution. Here, we review some of the challenges in making sense of the infor-
mation reflected in the similarity between neural activity patterns elicited by thoughts
about concepts.
The dominant theory behind these investigations is that representations of concepts

can be conceived of as points in a high dimensional space. Testing hypotheses about
the content and structure of neural representational spaces requires measuring the
relationships between neural signals evoked by thoughts about various concepts. This
approach strongly depends on the assumption that the multi-voxel patterns evoked by
thinking about a concept constitute the neural representation of that concept. This as-
sumption is most notably evident in the term that Kriegeskorte and colleagues (2008)
coined to describe similarity-based analyses of multi-voxel patterns: “Representational
Similarity Analysis,” or RSA. This term helpfully provides a common name for studies to
invoke when they introduce their methodological approach. However, it is important to
point out that the “R” in RSA comes with a caveat.
The use of the term “representation” suggests a detectable and meaningful distinc-
tion between representations, which are codes that store informational content, and
processes, which are mechanisms that create and operate upon these representations
(cf. Davis & Poldrack, 2013). These terms are often rhetorically useful for developing
and testing hypotheses about various cognitive mechanisms and underlying represen-
tational structures. However, the distinction between the two is often uninterpretable,
because fMRI can only measure representations while they are in use, that is, while they
are being operated upon by some process. Hence, the multi-voxel patterns evoked by a
given representation are inseparable from the set of processes that are currently oper-
ating upon that representation. For the purposes of characterizing human cognition,
the distinction between process and representation might be inconsequential, because
a thought must be accessed in order to be known. It is a matter of philosophical debate
whether inaccessible thoughts contain meaningful content, and whether they even exist.
But, the lack of a distinction between process and representation potentially qualifies
the results yielded by neural similarity analyses. These analyses are incapable of targeting
the multi-voxel patterns that are exclusively evoked by conceptual representations. In
fact, so-called RSA methods are just as sensitive to process-level differences between
stimuli as they are to representation-level differences. As a consequence, any observed
relationships between the multi-voxel patterns evoked by two stimuli might occur be-
cause the stimuli are representationally similar, but they might also occur because the
two stimuli both require the engagement of a common cognitive process.
Methodological Challenges
While it is impossible to fully dissociate process-level and representational accounts,
researchers can design their studies such that the confounds between processes and
representations are minimized. For instance, to reduce the impact of processes on multi-
voxel patterns, the patterns could be sampled while subjects perform a secondary task
that is orthogonal to the representational aspects of the stimuli that are being studied.
Additionally, basic processes like familiarity and complexity should be uncorrelated
with the dimensions of the representational space. In a thoughtful commentary, Davis

and Poldrack (2013) propose additional experimental solutions to this issue.
These suggestions highlight the fact that multi-voxel patterns will likely reflect a
number of nonrepresentational signals that overlap with the stimulus-evoked responses.
These signals might reflect the engagement of a cognitive process, or other characteris-
tics that differ between stimulus conditions, including differences in the signal-to-noise
ratio between conditions, perhaps due to differences in memorial and attentional re-
sources that have been allocated to one condition versus the other (cf. Todd, Nystrom,
Cohen, 2013). These spurious sources of reliability in multi-voxel patterns could un-
derlie any observed differences in similarities between conditions. Likewise, any po-
tential downstream effects, such as differences between conditions in task difficulty or
response latencies, could contribute to the similarity of the observed neural patterns.
Future Directions and Summary
A growing number of studies have explored modality-specific and cross-modal concep-

tual representations of objects by highlighting the commonalities between the neural
similarity structures evoked with and between object categories. One potential avenue
for future research is to further characterize the semantic similarity spaces that best
predict modality-specific effects. Moreover, additional studies could explore how stim-
ulus modality, experimental task, and object-specific properties can each uniquely and
jointly shape the resulting neural similarity space.
Future investigations might benefit from constructing similarity spaces that are de-
fined by object relatedness according to specific dimensions (e.g., shape or color sim-
ilarity). With this approach, researchers can investigate the specific semantic features
that best predict the observed neural similarity spaces. The studies reviewed here have
quantified conceptual similarity at a far more general level: according to objects’ tax-
onomic category (as in Devereux et al., 2013; Fairhall & Caramazza, 2013; Haxby et al.,
2001); superordinate, within-category object class (e.g., Bruffaerts et al., 2013; Connolly
et al., 2012); or basic-level, within-category comparisons that are either based on
pairwise ratings of overall relatedness (e.g., Connolly et al., 2012; Weber et al., 2009);
or comparisons between object vectors that code over a dozen object features (e.g.,
Bruffaerts et al., 2013; Clarke & Tyler, 2014)
These studies indicate that semantically related concepts evoke similar representations.
But given how these studies measure similarity, they cannot fully characterize the
features (or feature combinations) that are responsible for the correspondence between
the neural and semantic similarity spaces. What information is driving the observed
neural representational structures? Using data-driven analyses, researchers can more
closely explore and observe neural similarity spaces and attempt to extract their most
informative dimensions. And in hypothesis-driven analyses, one can systematically vary
the dimensions that are predicted by a semantic model of the similarity space.
In sum, this line of research illustrates a progression in the types of questions that
researchers have asked, based on the complexity and sensitivity of the analysis
techniques that have become available. Traditional univariate-based analyses identified
regions that were “commonly activated” by pictures and words; MVPA decoding
methods identified regions where neural patterns could discriminate between broad
object classes across stimulus modalities; and similarity-based methods have since
identified whether these putatively modality-independent regions are sensitive to
more fine-grained similarities among concepts. The advent of MVPA techniques has
brought about a shift in how analyses of conceptual content are both conducted and
interpreted in the brain. These methods provide increased sensitivity for detecting the
representational structure that underlies concept-specific and even feature-specific
representations. Given this unique sensitivity and flexibility, neural similarity analyses
are an essential part of the neuroscientist’s toolkit.
References
Akama, H., Murphy, B., Na, L., Shimizu, Y., & Poesio, M. (2012). Decoding semantics across
fMRI sessions with different stimulus modalities: A practical MVPA study. Frontiers in
Neuroinformatics, 6(24), 1–10. doi:10.3389/fninf.2012.00024
Allport, D. A. (1985). Distributed memory, modular subsystems and dysphasia. In S. K. Newman
& R. Epstein (Eds.), Current perspectives in dysphasia (pp. 207–244). Edinburgh: Churchill
Livingstone.
Barsalou, L. W. (1999). Perceptual symbol systems. Behavioral Brain Science, 22, 577–660.
Bracci, S., & de Beeck, H. O. (2016). Dissociations and associations between shape and category
representations in the two visual pathways. Journal of Neuroscience, 36(2), 432–444.
Bruffaerts, R., Dupont, P., Peeters, R., De Deyne, S., Storms, G., & Vandenberghe, R.
(2013). Similarity of fMRI activity patterns in left perirhinal cortex reflects semantic
similarity between words. Journal of Neuroscience, 33(47), 18597– 18607. doi:10.1523/
JNEUROSCI.1548-13.2013
Buchweitz, A., Shinkareva, S. V., Mason, R. A., Mitchell, T. M., & Just, M. A. (2012). Identifying
bilingual semantic neural representations across languages. Brain and Language, 120(3),
282–289.
Charest, I., Kievit, R. A., Schmitz, T. W., Deca, D., & Kriegeskorte, N. (2014). Unique semantic
space in the brain of each beholder predicts perceived similarity. Proceedings of the National
Academy of Sciences, 111(40), 14565–14570.
Cichy, R. M., Pantazis, D., & Oliva, A. (2016). Similarity-based fusion of MEG and fMRI reveals
spatio-temporal dynamics in human cortex during visual object recognition. Cerebral
Cortex, 26, 3563–3579.
Clarke, A., & Tyler, L. K. (2014). Object-specific semantic coding in human perirhinal cortex.
Journal of Neuroscience, 34(14), 4766–4775. doi: 10.1523/JNEUROSCI.2828-13.2014
Connolly, A. C., Guntupalli, J. S., Gors, J., Hanke, M., Halchenko, Y. O., Wu, Y.-C., . . . Haxby, J. V.
(2012). The representation of biological classes in the human brain. Journal of Neuroscience,
32(8), 2608–2618. doi: 10.1523/JNEUROSCI.5547-11.2012
translation: fMRI decoding of spoken words in bilinguals reveals language-independent
semantic representations in anterior temporal lobe. Journal of Neuroscience, 34(1), 332–338.

doi:10.1523/JNEUROSCI.1302-13.2014
Davis, T., & Poldrack, R. A. (2013). Measuring neural representations with fMRI: Practices and
pitfalls: Representational analysis using fMRI. Annals of the New York Academy of Sciences,
1296(1), 108–134. doi: 10.1111/nyas.12156
Devereux, B. J., Clarke, A., Marouchos, A., & Tyler, L. K. (2013). Representational similarity
analysis reveals commonalities and differences in the semantic processing of words and
objects. Journal of Neuroscience, 33(48), 18906–18916. doi:10.1523/JNEUROSCI.3809-13.2013
Drucker, D. M., & Aguirre, G. K. (2009). Different spatial scales of shape similarity representa-
tion in lateral and ventral LOC. Cerebral Cortex, 19(10), 2269–2280.
Dunsmoor, J. E., Kragel, P. A., Martin, A., & LaBar, K. S. (2013). Aversive learning modulates
cortical representations of object categories. Cerebral Cortex, 24(11), 2859–2872. doi: 10.1093/
cercor/bht138
Edelman, S. (1998). Representation is representation of similarities. Behavior and Brain
Sciences, 21, 449–467.
Fairhall, S. L., & Caramazza, A. (2013). Brain regions that represent amodal conceptual know-
ledge. Journal of Neuroscience, 33(25), 10552–10558. doi: 10.1523/JNEUROSCI.0051-13.2013
Haxby, J. V., Gobbini, M. I., Furey, M. L., Ishai, A., Schouten, J. L., & Pietrini, P. (2001).
Distributed and overlapping representations of faces and objects in ventral temporal cortex.
Science, 293(5539), 2425–2430.
Hsu, N. S., Schlichting, M. L., & Thompson-Schill, S. L. (2014). Feature diagnosticity affects
representations of novel and familiar objects. Journal of Cognitive Neuroscience, 26(12),
2735–2749.
Kaplan, J. T., Man, K., & Greening, S. G. (2015). Multivariate cross-classification: Applying ma-
chine learning techniques to characterize abstraction in neural representations. Frontiers in
Kriegeskorte, N. (2008). Representational similarity analysis: Connecting the branches of sys-
tems neuroscience. Frontiers in Systems Neuroscience, 2, 4.
Kriegeskorte, N., Goebel, R., & Bandettini, P. (2006). Information-based functional brain
mapping. Proceedings of the National Academy of Sciences USA, 103(10), 3863–3868.
Kriegeskorte, N., Mur, M., Ruff, D. A., Kiani, R., Bodurka, J., Esteky, H., . . . Bandettini, P. A.
(2008). Matching categorical object representations in inferior temporal cortex of man and
monkey. Neuron, 60(6), 1126–1141. doi: 10.1016/j.neuron.2008.10.043
Liuzzi, A. G., Bruffaerts, R., Dupont, P., Adamczuk, K., Peeters, R., De Deyne, S., Storms, D.
& Vandenberghe, R. (2015). Left perirhinal cortex codes for similarity in meaning between
written words: Comparison with auditory word input. Neuropsychologia, 76,  4–16.
Psychology, 58(1), 25–45. doi:10.1146/annurev.psych.57.102904.190143
Martin, A., Haxby, J. V., Lalonde, F. M., Wiggs, C. L., & Ungerleider, L. G. (1995). Discrete
cortical regions associated with knowledge of color and knowledge of action. Science, 270,
102–105.
McRae, K., Cree, G. S., Seidenberg, M. S., & McNorgan, C. (2005). Semantic feature produc-
tion norms for a large set of living and nonliving things. Behavioral Research Methods, 37,
547–559.
Murphy, G. L. (2002). The big book of concepts. Cambridge, MA: MIT Press.
Op de Beeck, H. P., Torfs, K., & Wagemans, J. (2008). Perceived shape similarity among un-
familiar objects and the organization of the human object vision pathway. Journal of
Neuroscience, 28(40), 10111–10123.
Posner, M. I. (1978). Chronometric explorations of mind. Hillsdale, NJ: Lawrence Erlbaum.
Posner, M. I., Petersen, S. E., Fox, P. T., & Raichle, M. E. (1988). Localization of cognitive opera-
tions in the human brain. Science, 240(4859), 1627–1631.
Proklova, D., Kaiser, D., & Peelen, M. V. (2016). Disentangling representations of object shape
and object category in human visual cortex: The animate–inanimate distinction. Journal of
Serre, T., Wolf, L., Bileschi, S., Riesenhuber, M., & Poggio, T. (2007). Robust object recogni-
tion with cortex-like mechanisms. IEEE Transactions on Pattern Analysis and Machine
Intelligence, 29(3), 411–426.
Sha, L., Haxby, J. V., Abdi, H., Guntupalli, J. S., Oosterhof, N. N., Halchenko, Y. O., & Connolly,
A. C. (2015). The animacy continuum in the human ventral vision pathway. Journal of
Shepard, R. N., and Chipman, S. (1970). Second- order isomorphism of internal
representations: Shapes of states. Cognitive Psychology, 1,  1–17.
Shinkareva, S. V., Malave, V. L., Mason, R. A., Mitchell, T. M., & Just, M. A. (2011). Commonality
of neural representations of words and pictures. NeuroImage, 54(3), 2418–2425.
Simanova, I., Hagoort, P., Oostenveld, R., & van Gerven, M. A. J. (2012). Modality-independent
decoding of semantic information from the human brain. Cerebral Cortex, 24(2), 426–434.
doi: 10.1093/cercor/bhs324
Thompson-Schill, S. L. (2003). Neuroimaging studies of semantic memory: Inferring “how”
from “where.” Neuropsychologia, 41(3), 280–292.
Todd, M. T., Nystrom, L. E., Cohen, J. D. (2013). Confounds in multivariate pattern anal-
ysis: Theory and rule representation case study. NeuroImage, 77, 157–165.
Tyler, L. K., Moss, H. E., Durrant-Peatfield, M. R., & Levy, J. P. (2000). Conceptual structure
and the structure of concepts: A distributed account of category-specific deficits. Brain &
Language, 75, 195–231.
Weber, M., Thompson-Schill, S. L., Osherson, D., Haxby, J., & Parsons, L. (2009). Predicting
judged similarity of natural categories from their neural representations. Neuropsychologia,
47(3), 859–868. doi:10.1016/j.neuropsychologia.2008.12.029
Wittgenstein, L., & Anscombe, G. E. M. (1953). Philosophical investigations. London, Basic
Blackwell.
Chapter 23
T he How and W hat of

Object Knowl e d g e i n
the Hum a n  Bra i n
Frank E. Garcea and Bradford Z. Mahon
Introduction
The ability to manipulate objects in order to carry out complex tasks is a fundamental
cognitive ability that we utilize on a daily basis: we are constantly recognizing, grasping,
and manipulating objects (e.g., pliers, scissors, fork, etc.). Despite the indefinite number
of ways in which one could, in principle, interact with objects in the environment (Wu,
2008), we grasp and manipulate objects in a specific manner: in order to carry out their
function and satisfy behavioral intentions. Thus, everyday tool use requires the integra-
tion of action knowledge in the motor domain (i.e., knowing how to physically manipu-
late a pair of scissors) with abstract conceptual knowledge (i.e., knowing the function of
scissors) in order to manipulate an object according to the goal or purpose of use (cut-
ting a piece of paper). The objective of this chapter is to review the main elements that
must be satisfied by a cognitive model of tool processing, including object recognition
and object use, and to situate that model in the context of what we currently know about
the neural substrate of tool processing.
Scope of the Chapter
The goal of this review is to argue for two empirical hypotheses: (1) function and manip-
ulation knowledge are dissociable types of information about manipulable objects; and
(2) there are specific neural pathways and regions involved in integrating knowledge
The How and What of Object Knowledge in the Human Brain 577
of object function and object manipulation during tool use. We review three principal
sources of evidence: (1) functional neuroimaging studies measuring blood oxygena-
tion level dependent (BOLD; see Heim & Specht, Chapter 4 in this volume) signal when
healthy adults retrieve knowledge of object function and object manipulation; (2) cog-
nitive dissociations between object function and object manipulation in neuropsy-
chological patients; and (3) transcranial magnetic stimulation (TMS; see Schuhmann,
Chapter 5 in this volume) studies measuring the online retrieval of function and manip-
ulation knowledge.
More broadly, and to situate our review of tool processing alongside the other
contributions that form this volume, there are interesting parallels to be explored be-
tween function and manipulation knowledge, on the one hand, and between lexical-
semantic and lexical knowledge, on the other hand. In their seminal theoretical review,
Rothi, Ochipa, and Heilman (1991) distinguished between what they referred to as ‘ac-
tion semantics’ and the ‘Action output lexicon’ (for discussion and development, see also
Cubelli et al., 2000; Negri, Rumiati, et al., 2007). The distinction between semantic infor-
mation pertaining to objects and actions, and high-level descriptions of object-directed
actions, is important and warrants further close scrutiny. As will be described in the
following, a defining feature of “upper limb apraxia” is a deficit in the skilled use of the
hands that cannot be explained by elemental motor deficits; this may be akin to patients
with anomia, who do not have difficulty with articulation of words, or with accessing
word meaning, but rather with interfacing meaning with articulation. As Rothi and
colleagues, and a number of others, have emphasized, patients with apraxia can retain
the ability to imitate actions, much like patients with anomia can retain the ability to re-
peat words (e.g., see Tessari, Canessa, Ukmar, & Rumiati, 2007). We return later to con-
sider these potentially fruitful parallels between the praxis and the language systems.
Overview of Tool Processing in

the Human Brain:
The Tool-P rocessing Network
The Dorsal and Ventral Visual Pathways

An influential model of visual processing, motivated in large part by the work of
Melvyn Goodale and David Milner, distinguishes between a ventral visual pathway
and a dorsal visual pathway (e.g., see Goodale, Milner, Jakobson, & Carey, 1991;
Goodale & Milner, 1992; see also Livingstone & Hubel, 1988; Merigan & Maunsell,
1993; Ungerleider & Mishkin, 1982). The dorsal/ventral stream hypothesis argued that
visual information is processed by separate anatomical and functional pathways in the
visual system: The ventral visual pathway, which courses ventrally from primary visual
578 Frank E. Garcea and Bradford Z. Mahon
cortex to inferotemporal cortex, processes visual input in the service of object identifi-
cation and long-term memory retrieval and encoding; the dorsal visual pathway, which
projects subcortically and cortically, potentially via motion sensitive area MT/V5 as
well as striate cortex, to dorsal occipital and posterior parietal cortex (e.g., Almeida,
Mahon, Nakayama, & Caramazza, 2008; Culham et al., 2003; Fang & He, 2005; Gallivan,
McLean, Flanagan, & Culham, 2013; Kristensen, Garcea, Mahon, & Almeida, 2016; Lyon,
Nassi, & Callaway, 2010; Mahon, Kumar, & Almeida, 2013; Sincich, Park, Wohlgemuth,
& Horton, 2004), processes volumetric information about objects in egocentric frames
of reference in the service of online object-directed actions (for reviews, see Binkofski
& Buxbaum, 2013; Milner & Goodale, 2008; Pisella, Binkofski, Lasek, Toni, & Rossetti,
2006; Rossetti, Pisella, & Vighetto, 2003; see also de Haan & Cowey, 2011).
The Tool-Processing Network

During the same period in which the dorsal/ventral stream hypothesis was coming into
focus, Alex Martin and colleagues carried out a series of functional MRI experiments
on semantic category representation that permanently changed the theoretical land-
scape. In their early reports, Chao, Martin, and colleagues reported that viewing
pictures of tools compared to baseline categories (e.g., faces, vehicles, animals) elicited
differential BOLD contrast in the medial fusiform gyrus bilaterally and the left poste-
rior middle temporal gyrus (e.g., see Chao, Haxby, & Martin, 1999), as well as in the
inferior and superior parietal lobule, dorsal occipital cortex, and premotor cortex (e.g.,
see Chao & Martin, 2000; see also Almeida, Fintzi, & Mahon, 2013; Culham, Danckert,
DeSouza, Gati, et al., 2003; Fang & He, 2005; Garcea & Mahon, 2014; Mahon, Kumar,
& Almeida, 2013; Mahon, Milleville, Negri, Rumiati, Caramazza, & Martin, 2007;
Noppeney, Price, Penny, & Friston, 2006; Peeters, Rizzolatti, & Orban, 2013; Rumiati,
Weiss, Shallice, Ottoboni, Noth, et al., 2004; for reviews, see Lewis, 2006; Martin, 2007,
2009, 2016). Differential BOLD contrast for tools is generally left-lateralized, with
the exception of the medial fusiform gyrus, and superior/posterior parietal cortices.
Collectively, we refer to this entire brain network of regions as the “tool-processing
network” (see Figure 23.1).
Different regions within the network of regions shown in Figure 23.1 support different
aspects of tool processing. Regions within ventral temporal-occipital cortex process
visual features such as surface texture (e.g., see Cant & Goodale, 2007, 2011), color (e.g.,
see Miceli, Fouch, Capasso, Shelton, Tomaiuolo, & Caramazza, 2001; Stasenko, Garcea,
Dombovy, & Mahon, 2014), and object weight (e.g., see Gallivan, Cant, Goodale, &
Flanagan, 2014). Functional MRI studies have found that patterns of unarticulated mo-
tion associated with tools in use are represented in the left posterior middle temporal
gyrus (e.g., see Beauchamp, Lee, Haxby, & Martin, 2002, 2003). Additional neuropsy-
chological investigations have reported an association between lesions in the vicinity
of the left posterior middle temporal gyrus and impairments in tool recognition (e.g.,
see Brambati et al., 2006; Campanella, D’Agostini, Skrap, & Shallice, 2010; Mahon et al.,
Left ventral Left inferior

premotor cortex parietal lobule Left superior
parietal lobule
Left dorsal
occipital cortex
Left posterior
middle temporal gyrus
Right medial Left medial
fusiform gyrus fusiform gyrus
Figure 23.1. Cortical regions in the dorsal stream, the ventral stream, and frontal-motor
cortex that comprise the tool-processing network. Undergraduate participants viewed images of
tools and animals (among other stimuli). These data replicate a pattern originally reported by
Chao and colleagues (1999) and Chao and Martin (2000). Plotted in yellow are the regions of
cortex that express increased BOLD contrast for images of manipulable objects (contrast: tools >
animals). Specifically, viewing images of manipulable objects elicits increased BOLD contrast in
dorsal and ventral premotor cortex, left parietal cortex in the vicinity of the anterior intraparietal
sulcus, the left posterior middle/inferior temporal gyrus, bilateral posterior parietal/dorsal oc-
cipital cortex, and medial fusiform gyrus bilaterally (for details, see Garcea and Mahon, 2014;
Chen et al., 2016; Chen et al., 2018).
2007; Tranel, Damasio, & Damasio, 1997), and in the retrieval of conceptual knowledge
associated with actions (e.g., see Buxbaum, Shapiro, & Coslett, 2014; Tranel, Kemmerer,
Adolphs, Damasio, & Damasio, 2003). The left ventral premotor cortex (e.g., see Chao
& Martin, 2000) and left dorsal premotor cortex (e.g., see Grafton, Fadiga, & Rizzolatti,
1997) have been argued to support the planning and sequencing of complex actions. The
left dorsal occipital cortex, in the vicinity of left posterior parietal cortex, is hypothesized
to process volumetric and spatial information, likely in body-centered coordinates, nec-
essary for accurate reaching and grasping. Lesions to posterior parietal and/or dorsal
occipital cortex are associated with optic ataxia, a visuomotor impairment for reaching
and/or grasping in peripersonal space (Desmurget & Sirigu, 2009; Jeannerod, Arbib,
Rizzolatti, & Sakata, 1995; Jeannerod, Decety, & Michel, 1994; Karnath & Perenin, 2005;
Pisella, Gréa, Tilikete, & Vighetto, 2000).
In order to use objects to satisfy behavioral goals, it is necessary to access stored infor-
mation about how to manipulate the object according to its function—for instance, to
use a hammer to pound a nail, the hammer must be gripped off of its center of mass, and
swung in a particular manner, ensuring that a particular aspect of the head of the hammer
makes contact with the nail. A long tradition of neuropsychological research has described
a level of action representation that corresponds to the “knowledge” of how to manipu-
late an object according to its function—we term this manipulation knowledge. Patients
with limb apraxia have impairments when performing skilled actions, which manifests
as an impairment in using objects according to their function. Limb apraxia is associ-
ated with lesions to the left inferior parietal lobule, in the vicinity of the supramarginal
gyrus (e.g., see Liepmann, 1905; see also Bartolo, Cubelli, Della Sala, Drei, & Marchetti,
2001; Buxbaum, Veramonti, & Schwartz, 2000; Halsband, Schmitt, Weyers, Binkofski,
Grutzner, & Freund, 2001; Mahon et al., 2007; Negri, Rumiati, Zadini, Ukmar, Mahon,
& Caramazza, 2007; Garcea, Dombovy, & Mahon, 2013; Ochipa, Rothi, & Heilman, 1989;
Rapcsak, Ochipa, Anderson, & Poizner, 1995; Rumiati, Zanini, Vorano, & Shallice, 2001;
for reviews, see Binkofski & Buxbaum, 2013; Buxbaum, 2017; Cubelli, Marchetti, Boscolo,
& Della Sala, 2000; Goldenberg, 2009; Johnson-Frey, 2004; Mahon & Caramazza, 2005;
Osiurak & Badets, 2016; Rothi, Ochipa, & Heilman, 1991). Functional neuroimaging
studies converge with the view that the left inferior parietal lobule represents complex
manipulation knowledge, as there is increased BOLD contrast in the left inferior parietal
lobule when healthy adults view images of manipulable objects (e.g., Almeida et al., 2013;
Chao & Martin, 2000; Garcea, Kristensen, Almeida, & Mahon, 2016; Kristensen, Garcea,
Mahon, & Almeida, 2016; Garcea & Mahon, 2014; Mahon et al., 2007; Mahon et al., 2013;
for reviews, see Lewis, 2006; Martin, 2007) or pantomime object use while in the scanner
(Chen, Garcea, & Mahon 2016; Choi et al., 2001; Moll et al., 2000; Rumiati et al., 2004).
Neuroimaging work focusing on the reaching and grasping components of object use
have dissociated subregions within the left superior parietal lobule where BOLD signal
is maximal for reaching, while BOLD signal in regions of the left anterior intraparietal
sulcus (aIPS) is maximal for grasping (e.g., see Cavina-Pratesi, Goodale, & Culham,
2007; Culham, Danckert, DeSouza, Gati, et al., 2003; Konen, Mruczek, Montoya, &
Kastner, 2013; Rossit, McAdam, Mclean, Goodale, & Culham, 2013). These human im-
aging studies parallel neurophysiological studies in macaques (e.g., see Galletti, Fattori,
Kutz, & Gamberini, 1999; Murata, Gallese, Luppino, Kaseda, & Sakata, 2000; Sakata,
Taira, Mine, & Murata, 1992).
Dissociations Between Manipulation Knowledge

and Function Knowledge: Evidence from Cognitive
Neuropsychological Studies
Selective Impairment to Manipulation Knowledge
Cognitive neuropsychological evaluations of patients with limb apraxia have pointed
to a distinction between conceptual knowledge of objects and the ability to manipulate
objects correctly according to their function. For instance, Ochipa, Rothi, and Heilman
(1989) reported the performance of an individual who was able to identify (17/20) and
point to objects from verbal command (19/20) that she could not use (2/20; see Figure
23.2 B, left inset; see also Garcea et al., 2013; Negri, Rumiati, et al., 2007; Rapcsak et al.,
1995). In a series of studies, Laurel Buxbaum and her colleagues established key aspects
of what we now understand about how manipulation and function knowledge are or-
ganized in the brain. Buxbaum, Veramonti, and Schwartz (2000) asked two limb
apraxic patients to carry out a series of action production and semantic judgment tasks
with manipulable objects. Those patients’ abilities to use objects, complete multistep
actions (e.g., making a sandwich), and make manipulation-based declarative judgments
were grossly impaired; however, their ability to make function judgments over those
same items was relatively spared (see Figure 23.2 A; see also Buxbaum & Saffran, 2002).
Together, these reports demonstrate that knowledge of the identity and function of
objects is dissociable from the ability to manipulate the object correctly after brain in-
jury (for review, see Mahon & Caramazza, 2005).
Additional evidence for that empirical generalization is provided by the findings of
Rosci, Chiesa, Laiacona, and Capitani (2003). Those authors carried out a case series
analysis with individuals who presented with and without limb apraxia after left brain
damage; the participants were asked to point to pictures from verbal command, to imi-
tate meaningful and meaningless gestures produced by the experimenter, to pantomime
object use from the visual presentation of common manipulable objects, and to com-
plete picture-word matching and picture-naming tasks. Rosci and colleagues reported
that individuals with limb apraxia tended to have more severe naming deficits than
those without apraxia, but at the single-case level, patients with severe apraxia remained
able to name pictures and match pictures to their corresponding names (see also Negri,
Rumiati, et al., 2007).
Somewhat tangentially, those neuropsychological dissociations between impaired
use and spared general (including function) knowledge of objects has important
implications for theories of the format of concept representation (for review and discus-
sion, see Mahon & Caramazza, 2005, 2008; Mahon & Hickok, 2016). Specifically, the fact
that high-level motor-relevant information about objects can be impaired while sparing
other forms of semantic knowledge rules out strong forms of the so-called embodied
cognition view, which argues that motor information is constitutively involved in the
representation of semantic knowledge. We have further argued that weaker forms of the
embodied view that drop the proposal that concepts are sensorimotor in their format do
not fare better. Weakening embodied theories so that the core claim is no longer about
the format of concept representation, but rather about conceptual content, renders the
theory indistinguishable from the putative alternative theory, namely that concepts
are represented as “abstract symbols” (for discussion, see Caramazza, Hillis, Rapp, &
Romani, 1990; Mahon, 2015).
Selective Impairment to Function Knowledge

Several case studies and group-level analyses have demonstrated the opposite pattern of
performance: following damage to the temporal lobes, patients can be impaired when
(a) The dissociation between function and manipulation knowledge.
100
80
% Correct Performance
60
40
20
0
W.C. J.D.
Buxbaum et al. (2000)
Matching pictures by manner of manipulation

(e.g., pliers, scissors, knife).
Matching pictures by manner of function

(e.g., pliers, scissors, knife).
(b). The double dissociation between object use and object naming.
100
80
% Correct Performance
60
40
20
0
Ochipa et al. A.M.
(1989) Negri et al. (2007)
Object Use Object Naming
Figure 23.2. Neuropsychological dissociation between conceptual knowledge and praxis.

(A) The dissociation between function and manipulation knowledge. Buxbaum, Veramonti, and
Schwartz (2000) reported the performance of two individuals (W.C. and J.D.) in which object-
associated manipulation knowledge was impaired relative to object-associated function know-
ledge. (B) The double dissociation between object use and object naming. Ochipa, Rothi, and
Heilman (1989) reported the performance of an individual who was impaired with using objects
in the face of spared object naming; in contrast, Negri, Lunardelli, and colleagues (2007) reported
the performance of an individual (D.L.) who was within control range when using objects despite
presenting with an impairment for naming objects, which worsened when tested two years later.
retrieving conceptual knowledge of objects that they can nonetheless successfully ma-
nipulate (e.g., see Negri, Lunardelli, Reverberi, Gigli, & Rumiati, 2007). While there
are reports of patients with impairments to function knowledge due to viral infections
affecting the anterior temporal lobes (e.g., see Sirigu, Duhamel, & Poncet, 1991), other
investigations have been carried out with individuals with degenerative diseases like se-
mantic dementia (SD), a disease that typically affects the anterior temporal lobes bi-
laterally, but asymmetrically (Negri, Lunardelli, et al., 2007). In the early stages of SD,
individuals are typically impaired in expressive and receptive language, and in retrieval
of conceptual knowledge of objects; as the disease progresses, their ability to retrieve
conceptual knowledge of objects worsens, affecting their knowledge of people, common
objects, and words. Despite these deficits, individuals with SD typically have spared vis-
uospatial processing, intact phonological and syntactic processing, and intact executive
control.
Because the effects of SD progress over time, researchers have focused on capturing
the progression of SD and its longitudinal effect on conceptual processing and object-
use abilities. Negri, Lunardelli, and colleagues (2007) performed a case-series analysis of
two individuals, one with suspected SD, and another with Alzheimer’s disease. Despite
a gradual degradation of lexical-semantic knowledge of objects (especially tools) over
a two-year period, Negri and colleagues showed that the two individuals were none-
theless able to successfully manipulate those objects, and that their object-use abilities,
while also declining, did not decline as dramatically (see Figure 23.2 B, right inset).
Those data present a puzzle: How are SD patients able to successfully manipulate an
object when they have lost knowledge of what it is used for, or can no longer verbally
describe its use? Hodges, Spatt, and Patterson (1999) addressed this question when they
tested three participants; two of the participants had SD, while the third participant had
been diagnosed with corticobasal degeneration (CBD); CBD is characterized by severe
limb apraxic symptoms without ataxia or elemental sensory/motor dysfunction after
damage to the basal ganglia, parietal lobes, and in some instances, frontal lobes. Hodges
and colleagues showed that both SD participants were unable to name objects, and were
at chance when judging the functions of objects. When contrasting the SD participants
with the performance of the CBD participant, Hodges and colleagues reported that the
CBD individual’s naming performance and knowledge of object function, while mar-
ginally outside of control range, was markedly better. Interestingly, while the three
participants were impaired when using objects, the SD participants were at ceiling when
asked to carry out a novel tool-selection task, while the CBD participant was no dif-
ferent than chance. The novel tool-selection task probed mechanical problem-solving
abilities by requiring participants to decide which of three novel tools would best fit into
a socket. Taken together, Hodges and colleagues argued that SD participants were able
to do well on the novel object-use task because mechanical problem-solving calls upon a
system of reasoning that is independent of object knowledge (for recent discussion, see
Buxbaum, 2017; Osiurak & Badets, 2016).
In a group- level analysis of SD participants, Hodges, Bozeat, Lambon Ralph,
Patterson, and Spatt (2000) reported that SD participants were deficient when asked
to use everyday objects, as well as retrieve conceptual/functional information about

those same objects. Interestingly, there was a strong correlation between object use and
object knowledge: as the degree of semantic impairment increased, the ability to use
those objects decreased (see also Negri, Rumiati, et al., 2007). None of the participants
presented with limb apraxia, and in subsequent tests, the participants performed within
control range when asked to select and use novel tools (for similar results with a more
comprehensive battery of tests, see Bozeat, Lambon Ralph, Patterson, & Hodges, 2002).
Hodges and colleagues speculated that the participants with SD were able to succeed in
using novel tools for a number of reasons. One factor is mechanical problem-solving, or
the ability to reason about the nature of the task on the basis of visual or somatosensory
features (but not learned and stored semantic information). It is not surprising that the
participants with SD were able to do well in the novel tool-use task given that patients
who fail the novel tool-use task tend to have lesions to fronto-parietal structures (e.g.,
see Hodges et al., 1999).
The Opacity of Object Affordances

A number of researchers have studied the influence of visual affordances in mediating
object-use abilities in patients (e.g., see Buxbaum, Schwartz, & Carew, 1997; Hodges
et al., 2000). We define affordances as the three-dimensional visual structure of the
object that is interpreted in terms of motor action. Take, for example, a hammer—a
hammer has a long handle that is grasped when swinging the object in order to pound
nails. The handle of the hammer serves as a strong cue as to which end of the object
“should” be grasped in order to functionally manipulate the tool. Thus, on the one hand,
one might conclude that deriving manipulation information from the visual input of a
hammer is facilitated because a hammer’s correct manipulation is transparently “given”
by the visual input. By contrast, the affordances of a corkscrew are perhaps more opaque
in that the appropriate manipulation information is not available from the visual input
alone; proper functional manipulation of a corkscrew requires the retrieval of complex
manipulation knowledge from long-term memory. On the other hand, we know that
even for what seems to be an object with “transparent” affordance, the manipulation
knowledge is not given “bottom-up” simply from the visual input—we know this be-
cause patients with lesions to ventral stream structures (e.g., patient D.F.) have difficulty
grasping objects in a functionally appropriate manner (Carey, Harvey & Milner, 1996;
see also Goodale, Jakobsen, & Keillor, 1994). The fact that patient D.F. was not able to
grasp objects in a functionally appropriate manner suggests that in order to appreciate
even what seem to be transparent object affordances, it is necessary that the object is
recognized as such.
There are also data suggesting that contextual familiarity can modulate patient per-
formance. Snowden, Griffiths, and Neary (1994) performed a case series study of
patients with SD to understand if object familiarity would modulate the patients’ object-
and place-naming performance. One case, K.E., could recognize and name objects that
belonged to her, discussing how to manipulate the objects and where they are found in
her house; however, when given the same types of objects, but exemplars that did not
belong to her, K.E.’s naming performance dropped markedly, and she could not describe
how to properly manipulate those objects (see also Snowden, Griffiths, & Neary, 1996).
An analogous set of issues is present in the domain of visual word reading. For ex-
ample, when one reads the word yacht, one cannot read the letters that constitute the
word yacht and appropriately read it as “/jɔt/”; rather, successful mapping of the letters
yacht to the sound “/jɔt/” requires access to lexical-semantic information and subse-
quent lexical access (on the production side). Other words could, in theory, be read
on the basis of orthography if there is transparency in the mapping of orthographic
representations to phonological representations of the word, and of course languages
differ dramatically in the transparency of their orthography.
The key issue for tools and novel tool-selection (or mechanical problem-solving)
tasks is this: Are there familiar objects that have a truly transparent affordance struc-
ture? Or is it the case that novel tool-selection tasks simply place qualitatively different
computational demands on the system than actual object use? In other words, it could
be that novel tool-selection tasks are akin to developing a set of pseudoword stimuli in
the context of a language with a completely opaque orthography.

and Function Knowledge in the Healthy Brain:
Evidence from Neuroimaging
Functional neuroimaging studies in healthy adults have sought to identify which regions
of cortex are involved in the online computation of object function and object manipu-
lation. Kellenbach, Brett, and Patterson (2003) asked participants to decide if a visually
presented picture satisfied a manipulation-based probe question (e.g. “does using the
object involve squeezing or pinching?”) or a function-based probe question (e.g., “is
the object used to cut?”); they also had a control condition in which participants were
instructed to pay attention to scrambled photographs. Kellenbach and colleagues found
that relative to the baseline condition, the left posterior middle temporal gyrus, left ven-
tral premotor cortex, and left inferior and posterior parietal cortices were more strongly
activated when making manipulation-based judgments than function-based judgments
(for similar results, see Boronat et al., 2005). Canessa and colleagues (2008) replicated
the pattern reported by Kellenbach and colleagues (2003) and Boronat and colleagues
(2005), and in addition found that retrieval of object-based function knowledge en-
gaged the retrosplenial cortex and lateral anterior inferotemporal cortex more so than
manipulation judgments (see also Leshinskaya & Caramazza, 2015). Interestingly, the
lateral and anterior portions of inferotemporal cortex are the regions of cortex that,
when damaged, have been associated with impairments to function knowledge (e.g., see
Hodges et al., 1999; Sirigu et al., 1991).
In two recent reports, we studied manipulation and function knowledge using
multivoxel pattern analysis (Chen et al., 2016; Chen, Garcea, Jacobs, & Mahon, 2018;
see also, in this volume, Bauer & Just, Chapter 21, and Musz & Thompson-Schill,
Chapter 22). In Chen and colleagues (2018), healthy adults pantomimed object use in
response to word stimuli or, in separate runs, performed a difficult n-back perceptual
matching task over gray-scale images of tools. Items were selected so as to be analyzable
in sets of triads; for instance, one triad was “scissors, knife, pliers.” Within each triad, two
of the three items were related by manner of manipulation (e.g., scissors and pliers are
manipulated similarly), and two of the three items were functionally related (e.g., scis-
sors and knife are used for the same function, to cut; see Boronat et al., 2005, for prec-
edent on this approach to structuring items experimentally). Separately, the functional
magnetic resonance imaging (fMRI) volunteers also participated in a series of func-
tional localizer scans that independently identified tool-preferring regions of interest
(ROIs). Those regions, plotted in Figure 23.1, included the left inferior parietal lobule,
the left and right medial fusiform gyrus, and the left posterior middle temporal gyrus.
Chen and colleagues (2016, 2018) trained a multivoxel pattern classifier (binary
linear support vector machine, SVM) to discriminate the pantomime of using, for in-
stance, a screwdriver, from the pantomiming of using scissors. The binary classifier
was then tested on a new pair of items in which the manner of manipulation was sim-
ilar between objects (training data: screwdriver vs. scissors; testing data: corkscrew
vs. pliers). Thus, successful transfer from training to test implies decoding of manip-
ulation information, over and above the objects themselves. Separately, the same
analysis was carried out with pantomimes in which the functional properties among
items were similar (training data: corkscrew vs. scissors; testing data: bottle opener vs.
knife). Finally, the same types of analyses were also carried out using the n-back per-
ceptual matching task data. Above-chance discrimination of manipulation information
was observed in left motor cortex, left somatosensory cortex, and in the left anterior
intraparietal sulcus. By contrast, function relations among objects could be decoded
in temporal lobe regions (see also Anzellotti, Mahon, Schwarzbach, & Caramazza,
2011; Yee, Drucker, & Thompson-Schill, 2010). Perhaps the most important finding
that emerged from Chen and colleagues (2018) was that the left supramarginal gyrus
contained neural representations of object manipulation that transferred across object
pairs (training data: screwdriver vs. scissors; testing data: corkscrew vs. pliers), stim-
ulus format (training on words, testing on pictures, and vice versa), and task (training
on pantomime testing on perceptual matching, and vice versa). These findings indicate
that neural activity in the left supramarginal gyrus when participants perform tasks over
tools indicate compulsory access to abstract representations of object manipulation.

and Function Knowledge in the Healthy Brain:
Evidence from Transcranial Magnetic Stimulation Studies
Recent evidence from repetitive transcranial magnetic stimulation (rTMS) studies has
lent support to the hypothesis that function and manipulation knowledge constitute
dissociable types of object knowledge. rTMS is a form of TMS in which high-frequency

magnetic pulses are delivered to a region of cortex, extracranially, yielding a “virtual”
lesion (see Schuhmann, Chapter 6 in this volume). Pelgrims, Olivier, and Andres
(2011) carried out an rTMS experiment in which participants were required to de-
cide if two visually presented objects were compatible or incompatible across four dif-
ferent dimensions. Pelgrims and colleagues probed manipulation knowledge by asking
participants to decide if two visually presented manipulable objects were similar in the
configuration of the hand when using the items (hand-configuration task), or to decide
if a visually presented hand, in a specific posture (e.g., gripping), was compatible with a
visually presented object (e.g., scissors; object-hand interaction task). The participants
were also required to make compatibility judgments over two visually presented objects
that could be used in a given context (e.g., scissors, stapler; contextual task), or for the
same function (e.g., scissors, knife; function task). Pelgrims and colleagues found that
rTMS to the left inferior parietal lobule (in the vicinity of the supramarginal gyrus) se-
lectively slowed down hand-configuration judgments relative to rTMS applied to the
right inferior parietal lobule, or a control site (i.e., the vertex); no differences in response
time were found among the three stimulation sites when the three other tasks were
carried out. Pelgrims and colleagues argued that their findings support the hypothesis
that motor-based knowledge of manipulable objects is not necessary in order to access
contextual/functional information about object use (for a similar argument on the basis
of behavioral studies in healthy subjects, see Garcea & Mahon, 2012, and on the basis of
patient evidence, see Mahon & Caramazza, 2005, 2008).
Andres, Pelgrims, and Olivier (2013) expanded on their initial rTMS findings by
measuring contextual/functional and manipulation knowledge after stimulating the
left inferior parietal lobule, and the left middle temporal gyrus. Andres and colleagues
found that rTMS to the left inferior parietal lobule once again selectively slowed down
judgments in a hand-object configuration task. Interestingly, stimulating the left middle
temporal gyrus selectively slowed down judgments in the context task (e.g., are scissors
and knife used in the same context?). This led Andres and colleagues to argue that the
left supramarginal gyrus processes information about tools necessary to shape the hand
appropriately, whereas the left middle temporal gyrus processes information relevant to
the context in which the tool is typically found and used.
Ishibashi, Lambon Ralph, Saito, and Pobric (2011) combined an rTMS stimulation
protocol with a semantic matching paradigm in order to understand the neural areas
that contribute to function and manipulation knowledge. Participants were presented
with a target word, and below that target word were three alternate words. Participants
were instructed to decide which of the three alternates best matched the target word. In
the function condition, the target word (e.g., scissors) matched the correct choice (e.g.,
knife) by function; in the manipulation condition, the target word (e.g., scissors) matched
the correct choice (e.g., pliers) by manner of manipulation; the two foils were always un-
related to the target word (e.g., whisk). A control task employed a similar design, but re-
quired participants to make decisions about the visual similarity of scrambled pictures
of words. Ishibashi and colleagues found that relative to the control task, rTMS over
the left anterior temporal lobe selectively slowed down function judgments, but not
manipulation judgments; in contrast, rTMS over the left inferior parietal lobule selec-
tively slowed down manipulation judgments relative to the control task (see also Pobric,
Jefferies, & Lambon Ralph, 2010). Ishibashi and colleagues argued that their results are
consistent with the performance of SD patients in that rTMS to the left anterior tem-
poral lobe slowed down judgments of tool function; and furthermore, that rTMS to the
left inferior parietal lobule selectively slowed down manipulation judgments, which
mirrors the performance of apraxic individuals with left parietal damage.
Function and Manipulation

Knowledge in the Human
Brain: Interim Summary
Taken together, there is remarkable convergence among the neuropsychological, func-

tional neuroimaging, and TMS studies that have investigated the neural substrates of
manipulation and function knowledge in the human brain. Neuropsychological studies
indicate that patients with lesions to the inferior parietal lobule can be impaired for
manipulating objects correctly, but spared for knowledge of objects function; in con-
trast, patients with temporal lobe damage can be impaired for knowledge of object
function but spared for manipulating objects. After rTMS to the left inferior parietal
lobule, there is interference with manipulation judgments, while function/conceptual
judgments are not affected; in contrast, after rTMS applied to lateral and anterior tem-
poral lobes, there is selective interference with conceptual/function judgments, while
manipulation judgments are not affected. Figure 23.3 presents a summary of the peak
Talairach coordinates of regions in the left hemisphere reported during fMRI or TMS
experiments probing function and manipulation knowledge. The emerging pattern is
that manipulation judgments maximally drive activity in the left parietal cortex, in the
vicinity of the left inferior parietal lobule (left supramarginal gyrus, left anterior IPS);
furthermore, TMS-induced transient deficits for manipulation knowledge overlap
with the regions involved in processing object-associated manipulation knowledge
(see Figure 23.3 A). In the context of function knowledge, online retrieval of function
knowledge is associated with increased neural activity in the left ventrolateral and an-
terior temporal cortex, which overlaps with the regions that, when stimulated with
rTMS, were associated with relative performance decrements in tasks tapping function
knowledge.
The question then becomes: How do we translate this understanding of the brain
regions that support manipulation and function knowledge into an understanding
of how the brain deploys the right action to the right object? Imagine that your
friend asks you to pass her a hammer. There is an indefinite number of ways in
which you can grasp the object to pass to her—whatever is biomechanically com-
fortable given the physical constraints of the environment, the weight distribution
(a) (b)
ATL
Central sulcus
Calcarine
IPS sulcus
Central sulcus
IPS
Collateral
sulcus STS
Manipulation > Function Function > Manipulation

Reference Talairach (X Y Z) Reference Talairach (X Y Z)
Chen et al. (2016) −37, −44, 51 Chen et al. (2016) −19 −38 −12
Kellenbach et al. (2003) −38, −41, 45 Yee et al. (2010) −34 −28 −9
Canessa et al. (2008) −49, −28, 38 −43 −54 −2
Ishibashi et al. (2011) * −48 −12 −15
Boronat et al. (2005) −37, −57, 38
Canessa et al. (2008) −52 −6 −30
Ishibashi et al. (2011) * −38, −41, 45
−5 −57 −29
Pelgrims et al. (2011) * −58, −30, 43
Andres et al. (2013) * −57 −50 4
Andres et al. (2013) * −59, −32, 43
Yee et al. (2010) −27, −66, 23
Figure 23.3. Meta-analysis showing regions representing manipulation and function know-
ledge. (A) Peak Talairach coordinates of regions were maximally responsive when participants
were asked to retrieve knowledge of object manipulation. )B) Peak Talairach coordinates of re-
gions were maximally responsive when participants retrieved knowledge of object function.
Asterisks denote studies that used TMS to measure function and manipulation knowledge.
Coordinates originally published in MNI space were converted to Talairach space in order to plot
all regions in a common stereotactic space.
Abbreviations: IPS: intraparietal sulcus; ATL: anterior temporal lobe; STS: superior temporal sulcus.
Note that major sulci are demarcated with white lines.
of the object, and the state of your hand. However, when you pick up a hammer to
use it, the reach-to-grasp interaction with the object is informed by the goal—to
use the hammer according to its function—and you would grasp the hammer with
a functionally appropriate grasp that anticipates its use. Consider the different ways
in which one might pick up a hammer if the goal is to hammer a nail versus pull a
nail out by using the claw of the hammer. Decisions that must be made over motor
or visual information are causally constrained by the current behavioral goals. For
instance, the retrieval of functional or manipulation knowledge is not relevant
when you simply need to pass the hammer to your friend (e.g., see Creem & Proffitt,
2001; Jax & Buxbaum, 2010); however, successful use of a hammer to pound a nail
requires that function knowledge is integrated with manipulation knowledge in

order to satisfy the behavioral goal, and in a way that implements the designed func-
tion of the object.
It is important to emphasize that the proposal that access to object manipulation
is contingent on access to object identity is not in conflict with the dorsal/ventral
stream hypothesis of visual processing. The proposal that there is a dorsal visual
pathway that processes online, visual information in the service of action never
claimed that access to complex object-associated manipulation is given by the dorsal
stream. In other words, manipulation knowledge is not given by a “bottom-up” anal-
ysis of the visual input devoid of semantic interpretation; rather, volumetric analysis
of an object in a semantically uninterpreted manner is the provenance of the dorsal
visual pathway (e.g., see Almeida et al., 2008; Almeida et al., 2014). These issues are
at times run together in the literature, where “parietal” activity is considered to be
“activity in the dorsal stream,” rather than simply access to action information poten-
tially entirely contingent on prior retrieval of object-identity information in the ven-
tral visual pathway. Where the current proposal does substantively interface with the
dorsal/ventral stream hypothesis is in the context of functionally appropriate object
grasping. The dorsal stream would not know, on its own, how to grasp an object in
a functionally appropriate manner; in order to grasp an object in a functionally ap-
propriate manner, it is necessary to retrieve cognitive or semantic information about
what the object is and what its function is, in a manner that is integrated with current
behavioral goals. What this suggests is that the dorsal stream may provide a “space”
of possible object-directed grasps and that parietal regions then look to information
yielded via a ventral stream analysis in order to winnow that space down to the space
of functionally appropriate grasps (for discussion, see Mahon and Wu, 2015; Garcea
et al., in press).
The data reviewed to this point converge on the idea that computation of object iden-
tity and retrieval of object function occur via ventral stream pathways, and that access
to manipulation knowledge in the left inferior parietal lobule is contingent on prior
analysis of the object by the ventral visual pathway (for relevant findings, see Almeida
et al., 2013; Garcea et al., 2016; Kristensen et al., 2016; Mahon et al., 2013; for discussion,
see Binkofski & Buxbaum, 2013). Furthermore, in order to deploy functionally appro-
priate grasps to objects, it is necessary to take into account the eventual manipulation to
be applied to the object once it is in hand, and that retrieval of manipulation informa-
tion presupposes the retrieval of information about object identity and function. Thus,
understanding how functionally appropriate grasps are deployed to objects may point
toward a deeper understanding of interactions between the ventral and dorsal visual
pathways (e.g., Creem-Regher & Lee, 2005; Goodale et al., 1994; Wu 2008). An impor-
tant first step in addressing that hypothesis is to identify the structural and functional
connectivity between regions of cortex that could, in principle, integrate action-relevant
processing in parietal cortex with cognitive or semantic analysis of objects in the ventral
stream.
Anatomical and Functional

Connectivity in the Tool-P rocessing
Network: Toward a New Framework
for Understanding Impairments
for Object Use
There is a rich literature parcellating parietal cortex on the basis of sulcul and gyral struc-
tural similarity (e.g., Caspers, Geyer, Schleicher, Mohlberg, Amunts, & Zilles, 2006),
nonhuman primate tract tracing (e.g., see Borra, Belmalih, Calzarava, Gerbella, Murata,
Rozzi, et al., 2008; Borra, Ichinohe, Sato, Tanifuji, & Rockland, 2010), autoradiographic
labeling (e.g., see Pandya & Seltzer, 1982), neurotransmitter receptor density (e.g., see
Caspers, Schleicher, Bacha-Trams, Palomero-Gallagher, Amunts, et al., 2013), white
matter connectivity (e.g., see Caspers, Eickhoff, Rick, von Kapri, Kuhlen, et al., 2011;
Mars, Jbabdi, Sallet, O’Reilly, Croxson, Olivier, et al., 2011; Ruschel, Knösche, Friederici,
Turner, Geyer, & Anwander, 2014; Rushworth, Behrens, & Johansen-Berg, 2006), and
more recently, functional connectivity (e.g., see Garcea & Mahon, 2014). Studies have
focused on parcellating the inferior parietal lobule (Caspers et al., 2006; Caspers et al.,
2011; Caspers et al., 2013; Ruschel et al., 2014; Zhong & Rockland, 2003), the superior
parietal lobule (Zhang, Fan, Zhang, Wang, Zhu, et al., 2014), or the entire parietal lobe
(Durand, Nelissen, Joly, Wardak, Todd, et al., 2007; Mars et al., 2011; Nelson, Cohen,
Power, Wig, Miezin, et al., 2010; Orban, Claeys, Nelissen, Smans, Sunaert, et al., 2006;
Rushworth et al., 2006; for review, see Kravitz, Saleem, Baker, & Mishkin, 2011).
Caspers, Schleicher, Bacha-Trams, Palomero-Gallagher, Amunts, and Zilles (2013)
have recently argued for a parcellation scheme on the basis of autoradiographic labeling
of neurotransmitter receptors carried out in post-mortem brains. Caspers and colleagues
showed that the inferior parietal lobule could be parcellated into three clusters: the first
cluster was positioned in the rostral portion of the inferior parietal lobule (areas PFop,
PFcm, and PFt); the second cluster was positioned in an intermediate area between
the supramarginal gyrus and angular gyrus (areas PF and PFm); the third cluster was
positioned in the lateral and posterior portion of the inferior parietal lobule (posterior
PG [PGa and PGp]; see Caspers et al., 2013, Figure 8 D therein). This parcellation of the
inferior parietal lobule aligns well with previous parcellations of the inferior parietal
lobule (e.g., see Caspers et al., 2006; Caspers et al., 2011), and with human and nonhuman
primate studies parcellating parietal cortex by its white matter connectivity and resting-
state connectivity (e.g., see Mars et al., 2011; Rushworth et al., 2006).
While some studies have focused on the organization of the subregions within the
inferior parietal lobule (e.g., see Caspers et al., 2006; Caspers et al., 2011; Caspers et al.,
2013; Mars et al., 2011; Rushworth et al., 2006), others have measured the whole-brain
cortical connections of the inferior parietal subregions. For example, Borra, Belmalih,
Calzarava, Gerbella, Murata, Rozzi, et al. (2008) used tracer injections in the macaque
brain to measure the connections of the anterior intraparietal sulcus (aIPS). The ante-
rior intraparietal sulcus processes visuomotor information in order to shape the hand to
grasp objects in the environment (e.g., see Binkofski, Buccino, Posse, Seitz, Rizzolatti, &
Freund, 1999; Binkofski, Dohle, Posse, Stephan, Hefter, Seitz, et al., 1998; Sakata, Taira,
Murata, & Mine, 1995). Borra and colleagues found that regions of premotor cortex
(F5) in the macaque brain receive inputs from the anterior intraparietal sulcus, in the
vicinity of area PF (see also Rizzolatti & Matelli, 2003); they also found that the ante-
rior intraparietal sulcus receives inputs from the lateral superior temporal sulcus and
regions within the middle temporal gyrus and ventral temporal cortex (see also Borra
et al., 2010; Zhong & Rockland, 2003).
Garcea and Mahon (2014; see Figure 23.4) sought to integrate what is known about
the anatomical connections of the inferior parietal lobule with an analysis of functional
Complex object-associated manipulation (praxis)
Hand-shaping of object grasping Surface-texture and material properties

Volumetric analysis and online
Visual form
visuomotor processing
Motor processing Privileged functional connectivity
Figure 23.4. A network perspective on tool processing in the human brain. The key finding
is that left parietal tool preferring voxels can be triply dissociated (using k-means clustering)
based on their functional connectivity to frontal motor areas, temporal lobe object-processing
areas, and dorsal occipital regions involved in object-directed reaching. Privileged functional
connectivity among tool representations in the brain supports the integration of complex object-
associated manipulation knowledge with representation of object identify and form, which is
sent to the frontal-motor cortex for subsequent motor output.
Source: Derived from Garcea and Mahon (2014) with permission.
connectivity among parietal, temporal, and frontal-motor tool-preferring regions in

the human brain. In their analysis, Garcea and Mahon parcellated the tool-selective
left parietal cortex into three clusters on the basis of differential functional connec-
tivity to tool-selective regions in the motor system (left ventral premotor cortex), tem-
poral lobe (left medial fusiform gyrus, left posterior middle temporal gyrus), and dorsal
stream (left dorsal occipital cortex). Ventral premotor tool-preferring areas expressed
privileged functional connectivity the lateral and inferior parietal cortex (for similar
results using white matter connectivity, see Caspers et al., 2011; Mars et al., 2011; Ruschel
et al., 2014; Rushworth et al., 2006); the left medial fusiform gyrus and left posterior
middle temporal gyrus expressed privileged functional connectivity with the anterior
intraparietal sulcus (for convergent findings using functional connectivity, see Almeida
et al., 2013; Mahon et al., 2007; Mahon et al., 2013; Stevens, Tessler, Peng, Martin, 2015;
for review of white matter connectivity results, see Kravitz et al., 2011); the left dorsal oc-
cipital cortex expressed privileged functional connectivity to the superior parietal tool
preferring area.
As noted earlier, neuropsychological evidence indicates that damage to the left infe-
rior parietal lobule is associated with limb apraxia, while damage to the superior and
posterior parietal lobule is associated with optic ataxia. One speculation that may be
extrapolated from prior research that sought to parcellate parietal cortex based on ana-
tomical and functional connectivity is that some forms of limb apraxia may be attribut-
able to a disconnection deficit in which parietal action representations are de-afferented
of inputs from temporal lobe representations of object concepts or from frontal-motor
areas. Geschwind (1965) proposed a similar approach to understanding the causes of
apraxia. In his model, Geschwind suggested that the variants of limb apraxic errors
could be traced back to a differential disconnection among visual, motor, and left hemi-
sphere language areas that process information relevant for action production.
This framework predicts that disconnection of parietal action representations from
temporal lobe representations of object concepts could result in errors of content in
object use. In so-called ideational apraxia, patients’ movements are kinematically
and spatially coherent, but are carried out with the incorrect objects—for instance,
using a toothbrush as if it were a knife. In contrast, disconnecting left parietal action
representations from frontal motor areas would, by hypothesis, give rise to actions
that are conceptually correct, but spatially and kinematically incorrect—for instance, a
toothbrush would be used in the correct context, but the organization of the fingers and
joints would be spatially incoherent and kinematically incorrect (see also Binkofski &
Buxbaum, 2013).
Conclusion
On a daily basis we perceive, recognize, grasp, and use tools—we do this effortlessly
and fluidly in the service of behavioral goals. The “ability to recognize and use tools”
is a complex set of processes spread across many domains of cognition, and tool use in
humans has a widely distributed neural basis. This means that in order to understand
how the brain accesses action information from visual input, it is necessary to under-
stand what types of information about objects are computed by each of the distinct
pathways that “connect the eyes to the hands” and the “eyes to the mouth.” The goal of
this short review has been to review evidence from three sources—functional neuroim-
aging, neuropsychological studies of brain damaged patients, and TMS perturbations
of the healthy brain—that collectively bear on the issue of how manipulation and func-
tional knowledge about objects are organized and represented in the brain. As we have
briefly reviewed, a tremendous amount of progress has been made in advancing our un-
derstanding of how object-directed actions in the parietal lobe interface with high-level
visual representations in the temporal lobe and with the frontal-motor system.
We would suggest that many of the key issues that lay ahead of us, as a field, are
summarized by this question: How does the brain figure out how to deploy the correct
actions to the correct objects? We don’t pick up a banana to call our friend, or use a
toothbrush to clean the sink—why not? How can the errors exhibited after focal brain
injury constrain our understanding of the structure and dynamics of the system that
interfaces semantic and cognitive representations of objects and actions with action-
relevant representations that mediate our physical interaction with the world? Progress
on these questions requires a computational theory (in the sense of Marr, 1982), and
such a computational theory requires explicit bridging hypotheses about how it may be
implemented in neural architecture (for cogent discussion in the domain of language,
see Poeppel, 2012).
One exciting and new direction for the field is to analogize more aggressively from
models of lexical access to models of action processing; the theoretical contribution of
Rothi, Ochipa, and Heilman (1991) was foundational in establishing a cognitive frame-
work within which to think about how the correct actions are deployed to the correct
objects. In the more than 25 years since their proposal, we would suggest that while
much progress has been made in terms of understanding the neural correlates of object-
directed action processing, the basic parameters of a cognitive model proposed by Rothi
and colleagues remain the state of the art. This may of course be because the model
works and there have been no findings that dramatically challenge its core components
(for an extension on the original proposal by Rothi et al., 1991, see Cubelli et al., 2000).
Perhaps further progress could be spurred by taking a more granular approach to under-
standing the basic building blocks of complex object-associated actions. For instance, it
is an open empirical question whether a complex action such as using a corkscrew could
be decomposed into elemental components, such that there are shared “primitives”
with other actions (many with the action of using a screwdriver, fewer with the action of
turning a doorknob, fewer still with the action of brushing one’s teeth). In other words,
are “units” of action-representations recombined to form larger complex actions, or
are complex actions represented holistically and, as such, non-decomposable? If ac-
tion representations were shown to be componentially built from smaller units, then
an exciting direction would be to begin to ask what the “syntactic” computations are
that operate over those granular representations, and whether fine-grained connec-
tivity can relate specific object affordances to those action primitives. In summary,
we suggest that a fruitful direction may be to adopt the approach taken by models of
lexical access, in which there has been concerted focus on understanding the granu-
larity of representations at each level of processing, the computations applied to those
representations, the connectivity of those representations across and within levels, and
the processing dynamics that mediate system-level processing.
Acknowledgments
We would like to thank Jorge Almeida and Alfonso Caramazza for their discussion of these issues,
and Robert Jacobs and Michael Tanenhaus for feedback on an earlier version of this manuscript,
portions of which were prepared in partial satisfaction of the requirements of the PhD program
in the Department of Brain and Cognitive Sciences at the University of Rochester. We are grateful
to Niels Schiller and Greig de Zubicaray for their feedback on an earlier draft of this chapter.
Preparation of this chapter was supported by NIH grants R21 NS076176 and R01 NS089069,
and NSF grant 1349042, to B. Z. M., by a Center for Visual Science pre-doctoral training fellow-
ship (NIH training grant 5T32EY007125-24) to F. E. G., and by a Moss Rehabilitation Research
Institute postdoctoral training fellowship (NIH T32HD007425) to F. E. G.
References
Almeida, J., Fintzi, A. R., & Mahon, B. Z. (2013). Tool manipulation knowledge is retrieved by
way of the ventral visual object processing pathway. Cortex, 49, 2334–2344.
Almeida, J., Mahon, B. Z., Nakayama, K., & Caramazza, A. (2008). Unconscious processing
dissociates along categorical lines. Proceedings of the National Academy of Sciences, 105(39),
15214–15218.
Almeida, J., Mahon, B. Z., Zapater-Raberov, V., Dziuba, A., Cabaço, T., Marques, J. F., &
Caramazza, A. (2014). Grasping with the eyes: The role of elongation in visual recognition of
manipulable objects. Cognitive, Affective, & Behavioral Neuroscience, 14(1), 319–335.
Andres, M., Pelgrims, B., & Olivier, E. (2013). Distinct contribution of the parietal and temporal
cortex to hand configuration and contextual judgements about tools. Cortex, 49, 2097–2105.
Anzellotti, S., Mahon, B. Z., Schwarzbach, J., & Caramazza, A. (2011). Differential activity
for animals and manipulable objects in the anterior temporal lobes. Journal of Cognitive
Neuroscience, 23(8), 2059–2067.
Bartolo, A., Cubelli, R., Della Sala, S., Drei, S., & Marchetti, C. (2001). Double dissociation be-
tween meaningful and meaningless gesture reproduction in apraxia. Cortex, 37, 696–699.
Beauchamp, M. S., Lee, K. E., Haxby, J. V., & Martin, A. (2002). Parallel visual motion pro-
cessing streams for manipulable objects and human movements. Neuron, 34(1), 149–159.
Beauchamp, M. S., Lee, K. E., Haxby, J. V., & Martin, A. (2003). FMRI responses to video
and point-light displays of moving humans and manipulable objects. Journal of Cognitive
Binkofski, F., & Buxbaum, L. J., (2013). Two action systems in the human brain. Brain and
Language, 127, 222–229.
Binkofski, F., Buccino, G., Posse, S., Seitz, R. J., Rizzolatti, G., & Freund, H. J. (1999). A fronto‐
parietal circuit for object manipulation in man: Evidence from an fMRI‐study. European
Binkofski, F., Dohle, C., Posse, S., Stephan, K. M., et al. (1998). Human anterior intraparietal
area subserves prehension: A combined lesion and functional MRI activation study.
Neurology, 50, 1253–1259.
Boronat, C. B., Buxbaum, L. J., Coslett, H. B., Tang, K., Saffran, E. M., Kimberg, D. Y., et al.
(2005). Distinctions between manipulation and function knowledge of objects: Evidence
from functional magnetic resonance imaging. Cognitive Brain Research, 23, 361–373.
Borra, E., Belmalih, A., Calzavara, R., Gerbella, M., Murata, A., Rozzi, S., & Luppino, G. (2008).
Cortical connections of the macaque anterior intraparietal (AIP) area. Cerebral Cortex, 18,
1094–1111.
Borra, E., Ichinohe, N., Sato, T., Tanifuji, M., & Rockland, K. S. (2010). Cortical connections
to area TE in monkey: Hybrid modular and distributed organization. Cerebral Cortex, 20,
257–270.
Bozeat, S., Lambon Ralph, M. A., Patterson, K., & Hodges, J. R. (2002). When objects lose their
meaning: What happens to their use? Cognitive, Affective, and Behavioral Neuroscience, 2,
236–251.
Brambati, S. M., Myers, D., Wilson, A., Rankin, K. P., Allison, S. C., Rosen, H. J., . . . Gorno-
Tempini, M. L. (2006). The anatomy of category-specific object naming in neurodegenera-
tive diseases. Journal of Cognitive Neuroscience, 18(10), 1644–1653.
Buxbaum, L. J. (2017). Learning, remembering, and predicting how to use tools: Distributed
neurocognitive mechanisms: Comment on Osiurak and Badets (2016). Psychological
Review, 124, 346–360.
Buxbaum, L. J., & Saffran, E. M. (2002). Knowledge of object manipulation and object func-
tion: Dissociations in apraxic and nonapraxic subjects. Brain and Language, 82, 79–199.
Buxbaum, L. J., Schwartz, M. F., & Carew, T. G. (1997). The role of semantic memory in object
use. Cognitive Neuropsychology, 14(2), 219–254.
Buxbaum, L. J., Shapiro, A. D., Coslett, H. B. (2014). Critical brain regions for tool-related and
imitative actions: A componential analysis. Brain, 137, 1971–1985.
Buxbaum, L. J., Veramonti, T., & Schwartz, M. F. (2000). Function and manipulation tool
knowledge in apraxia: Knowing “what for” but not “how.” Neurocase, 6,  83–97.
Campanella, F., D’Agostini, S., Skrap, M., & Shallice, T. (2010). Naming manipulable
objects: Anatomy of a category specific effect in left temporal lobe tumors. Neuropsychologia,
48, 1583–1597.
Canessa, N., Borgo, F., Cappa, S. F., Perani, D., Falini, A., Buccino, G., et al. (2008). The different
neural correlates of action and functional knowledge in semantic memory: An fMRI study.
Cant, J. S., & Goodale, M. A. (2007). Attention to form or surface properties modulates dif-
ferent regions of human occipitotemporal cortex. Cerebral Cortex, 17(3), 713–731.
Cant, J. S., & Goodale, M. A. (2011). Scratching beneath the surface: New insights into the func-
tional properties of the lateral occipital area and parahippocampal place area. Journal of
Caramazza, A., Hillis, A. E., Rapp, B. C., & Romani, C. (1990). The multiple semantics hypo-
thesis: Multiple confusions? Cognitive Neuropsychology, 7(3), 161–189.
Carey, D. P., Harvey, M., & Milner, A. D. (1996). Visuomotor sensitivity for shape and orienta-
tion in a patient with visual form agnosia. Neuropsychologia, 34(5), 329e337.
Caspers, S., Eickhoff, S. B., Rick, T., von Kapri, A., Kuhlen, T., Huang, R., et al. (2011).
Probabilistic fibre tract analysis of cytoarchitectonically defined human inferior parietal
lobule areas reveals similarities to macaques. NeuroImage, 58, 362–380.
Caspers, S., Geyer, S., Schleicher, A., Mohlberg, H., Amunts, K., & Zilles, K. (2006). The human
inferior parietal cortex: Cytoarchitectonic parcellation and interindividual variability.
Caspers, S., Schleicher, A., Bacha-Trams, M., Palomero-Gallagher, N., Amunts, K., & Zilles,
K. (2013). Organization of human inferior parietal lobule based on receptor architectonics.
Cavina-Pratesi C., Goodale, M. A., & Culham, J. C. (2007). FMRI Reveals a dissociation be-
tween grasping and perceiving the size of real 3D objects. PLoS One, 2,  1–14.
Chao, L. L., Haxby, J. V., & Martin, A. (1999). Attribute-based neural substrates in temporal
cortex for perceiving and knowing about object. Nature Neuroscience, 2, 913–919.
Chao, L. L., & Martin, A. (2000). Representation of manipulable man-made objects in the
dorsal stream. NeuroImage, 12, 478–484.
Chen, Q., Garcea, F. E., Jacobs, R. A., & Mahon, B. Z. (2018). Decoding object knowledge in the
human brain during tool pantomiming and viewing. Cerebral Cortex, 28, 2162–2174.
Chen, Q., Garcea, F. E., & Mahon, B. Z. (2016). The representation of object-directed action and
function knowledge in the human brain. Cerebral Cortex, 26, 1609–1618.
Choi, S. H., Na, D. L., Kang, E., Lee, K. M., Lee, S. W., Na, D. G. (2001). Functional magnetic
resonance imaging during pantomiming tool-use gestures. Experimental Brain Research,
139, 311–317.
Creem, S. H., & Proffitt, D. R. (2001). Grasping objects by their handles: A necessary interac-
tion between cognition and action. Journal of Experimental Psychology: Human Perception
and Performance, 27(1), 218.
Creem-Regehr, S. H., & Lee, J. N. (2005). Neural representations of graspable objects: Are tools
special? Cognitive Brain Research, 22(3), 457–469.
Cubelli, R., Marchetti, C., Boscolo, G., & Della Salla, S. (2000). Cognition in action: Testing a
model of limb apraxia. Brain and Cognition, 44, 144–165.
Culham, J. C., Danckert, S. L., DeSouza, J. F. X., Gati, J. S., et al., (2003). Visually guided grasping
produces fMRI activation in dorsal but not ventral stream brain areas. Experimental Brain
Research, 153, 180–189.
de Haan, E. H. F., & Cowey, A. (2011). On the usefulness of “what” and “where” pathways in vi-
sion. Trends in Cognitive Sciences, 15, 460–466.
Desmurget, M., & Sirigu, A. (2009). A parietal-premotor network for movement intention and
motor awareness. Trends in Cognitive Sciences, 13, 411–419.
Durand, J-B., Nelissen, K., Joly, O., Wardak, C., Todd, J. T., et al. (2007). Anterior regions of
monkey parietal cortex process visual 3D shape. Neuron, 55, 493–505.
Fang, F., & He, S. (2005). Cortical responses to invisible objects in the human dorsal and ven-
tral pathways. Nature Neuroscience, 8(10), 1380–1385.
Galletti, C., Fattori, P., Kutz, D. F., & Gamberini M. (1999). Brain location and visual topog-
raphy of cortical area V6A in the macaque monkey. European Journal of Neuroscience, 11,
575–582.
Gallivan, J. P., Cant, J. S., Goodale, M. A. & Flanagan, J. R. (2014) Representation of object
weight in human ventral visual cortex. Current Biology, 24(16), 1866–1873.
Gallivan, J. P., McLean, D. A., Flanagan, J. R., & Culham, J. C. (2013). Where one hand
meets the other: Limb-specific and action-dependent movement plans decoded from
preparatory signals in single human frontoparietal brain areas. Journal of Neuroscience,

33, 1991–2008.
Garcea, F. E., Almeida, J., Sims, M. H., Nunno, A., Meyers, S. P., Li, Y. M., Walter, K., Pilcher, W.
H., & Mahon, B. Z. (in press). Domain-specific diaschisis: Lesions to parietal action areas
modulate neural responses to tools in the ventral stream. Cerebral Cortex. doi: 10.1093/
cercor/bhy183
Garcea, F. E., Dombovy, M., & Mahon, B. Z. (2013). Preserved tool knowledge in the context
of impaired action knowledge: Implications for models of semantic memory. Frontiers in
Human Neuroscience, 7,  1–18.
Garcea, F. E., Kristensen, S., Almeida, J., & Mahon, B. Z. (2016). Resilience to the contralateral
visual field bias as a window into object representations. Cortex, 81,  14–23.
Garcea F. E., & Mahon B. Z. (2012). What is in a tool concept? Dissociating manipulation
knowledge from function knowledge. Memory and Cognition, 40, 1303–1313.
Garcea, F. E., & Mahon, B. Z. (2014). Parcellation of left parietal tool representations by func-
tional connectivity. Neuropsychologia, 60, 131–143.
Geschwind, N. (1965). Disconnexion syndromes in animals and man. Part II. Brain, 8, 585–644.
Goldenberg, G. (2009). Apraxia and the parietal lobes. Neuropsychologia, 47, 1449–1459.
Goodale, M. A., Jakobson, L. S., & Keillor, J. M. (1994). Differences in the visual control of
pantomimed and natural grasping movements. Neuropsychologia, 32, 1159–1178.
Goodale, M. A., & Milner, A. D. (1992). Separate visual pathways for perception and action.
Trends in Neurosciences, 15,  20–25.
Goodale, M. A., Milner, A. D., Jakobson, L. S., & Carey, D. P. (1991) A neurological dissociation
between perceiving objects and grasping them. Nature, 349, 154–156.
Grafton, S. T., Fadiga, L., Arbib, M. A. & Rizzolatti, G. (1997). Premotor cortex activation
during observation of familiar tools. NeuroImage, 6, 231–236.
Halsband, U., Schmitt, J., Weyers, M., Binkofski, F., Grützner, G., & Freund, H. J. (2001).
Recognition and imitation of pantomimed motor acts after unilateral parietal and premotor
lesions: A perspective on apraxia. Neuropsychologia, 39(2), 200–216.
Hodges, J. R., Bozeat, S., Lambon Ralph, M. A., Patterson, K., & Spatt, J. (2000). The role
of conceptual knowledge in object use: Evidence from semantic dementia. Brain, 123,
1913–1925.
Hodges, J. R., Spatt, J., & Patterson, K. (1999). “What” and “how”: Evidence for the dissociation
of object knowledge and mechanical problem-solving skills in the human brain. Proceedings
of the National Academy of Sciences USA, 96, 9444–9448.
Ishibashi, R., Lambon Ralph, M.A., Saito, S., & Pobric, G. (2011). Different roles of lateral ante-
rior temporal and inferior parietal lobule in coding function and manipulation tool know-
ledge: Evidence from an rTMS study. Neuropsychologia, 49, 1128–1135.
Jax, S. A., & Buxbaum, L. J. (2010). Response interference between functional and structural
actions linked to the same familiar object. Cognition, 115(2), 350–355.
Jeannerod, M., Arbib, M. A., Rizzolatti, G., & Sakata, H. (1995). Grasping objects: The cortical
mechanisms of visuomotor transformation. Trends in Neuroscience, 18, 314–320.
Jeannerod, M., Decety, J., & Michel, F. (1994). Impairment of grasping movements following a
bilateral posterior parietal lesion. Neuropsychologia, 32, 369–380.
Johnson-Frey, S. (2004). The neural bases of complex tool use in humans. Trends in Cognitive
Sciences, 8,  71–78.
Karnath, H. O., & Perenin, M. T. (2005). Cortical control of visually guided reaching: Evidence
from patients with optic ataxia. Cerebral Cortex, 15, 1561–1569.
Kellenbach, M. L., Brett, M., & Patterson, K. (2003). Actions speak louder than functions: The
importance of manipulability and action in tool representation. Journal of Cognitive
Konen, C. S., Mruczek, R. E. B., Montoya, J. L., & Kastner, S. (2013). Functional organization of
human posterior parietal cortex: Grasping-and reaching-related activations relative to top-
ographically organized cortex. Journal of Neurophysiology, 109, 2897–1908.
Kravitz, D. J., Saleem, K. S., Baker, C. I., & Mishkin, M. (2011). A new neural framework for vis-
uospatial processing. Nature Reviews Neuroscience, 12(4), 217–230.
Kristensen, S., Garcea, F. E., Mahon, B. Z., & Almeida, J. (2016). Temporal frequency tuning
reveals interactions between the dorsal and ventral visual streams. Journal of Cognitive
Leshinskaya, A., & Caramazza, A. (2015). Abstract categories of functions in anterior parietal
lobe. Neuropsychologia, 76,  27–40.
Lewis, J. (2006). Cortical networks related to human use of tools. The Neuroscientist, 12,
211–231.
Liepmann, H. (1905). The left hemisphere and action. (Translation from Munch. Med. Wschr.
48–49). (Translations from Liepmann’s essays on apraxia. In Research Bulletin (vol. 506).
Department of Psychology, University of Western Ontario, London, ON; 1980).
Livingstone, M., & Hubel, D. (1988). Segregation of form, color, movement, and
depth: Anatomy, physiology, and perception. Science, 240, 740–749, 1988.
Lyon, D. C., Nassi, J. J., & Callaway, E. M. (2010). A disynaptic relay from superior colliculus to
dorsal stream visual cortex in macaque monkey. Neuron, 65, 270–279.
Mahon, B. Z. (2015). What is embodied about cognition? Language, Cognition and Neuroscience,
30, 420–429.
Mahon, B. Z., & Caramazza, A. (2005). The orchestration of the sensory-motor systems: Clues
from neuropsychology. Cognitive Neuropsychology, 22, 480–494.
Mahon, B. Z. & Caramazza, A. (2008). A critical look at the embodied cognition hypothesis and
a new proposal for grounding conceptual content. Journal of Physiology—Paris, 102,  59–70.
Mahon, B. Z., & Hickok, G. (2016). Arguments about the nature of concepts: Symbols, embodi-
ment, and beyond. Psychonomic Bulletin & Review, 23, 941–958.
Mahon, B. Z., Kumar, N., & Almeida, J. (2013). Spatial frequency tuning reveals interactions
between the dorsal and ventral visual systems. Journal of Cognitive Neuroscience, 25,
862–871.
Mahon, B. Z., Milleville, S., Negri, G. A. L., Rumiati, R. I., Caramazza, A, & Martin, A. (2007).
Action-related properties of objects shape object representations in the ventral stream.
Neuron, 55, 507–520.
Mahon, B. Z., & Wu, W. (2015). Cognitive penetration of the dorsal visual stream? In Zeimbekis,
J., & Athanasios, R (Eds.), The cognitive penetrability of perception: New philosophical
perspectives. Oxford: Oxford University Press.
Marr, D. (1982). Vision: A Computational Investigation into the Human Representation and
Processing of Visual Information. Cambridge: MIT Press.
Mars, R. B., Jbabdi, S., Sallet, J., O’Reilly, J. X., Croxson, P. L., et al. (2011). Diffusion-weighted
imaging tractography-based parcellation of the human parietal cortex and comparison with
human and macaque resting-state functional connectivity. The Journal of Neuroscience, 31,
4087–4100.
Martin, A. (2009). Circuits in mind: The neural foundations for object concepts. In M.
S. Gazzaniga (Ed.), The cognitive neurosciences (4th ed., pp. 1031– 1046). Cambridge,
MA: MIT Press.
Martin, A. (2016). GRAPES—Grounding Representations in Action, Perception, and Emotion
Systems: How object properties and categories are represented in the human brain.
Psychonomic Bulletin & Review, 23, 979–990,
Merigan, W. H., & Maunsell, J. H. R. (1993). How parallel are the primate visual pathways?
Annual Review of Neuroscience, 16, 369–402.
Miceli, G., Fouch, E., Capasso, R., Shelton, J. R., Tomaiuolo, F., & Caramazza, A. (2001).
The dissociation of color from form and function knowledge. Nature Neuroscience, 4,
662–667.
Milner, A. D., & Goodale, M. A. (2008). Two visual systems re-viewed. Neuropsychologia, 76,
774–785.
Moll, J., de Oliveira-Souza, R., Passman, L. J., Cimini Cunha, F., Souza-Lima, F., & Andreiuolo,
P. A. (2000). Functional MRI correlates of real and imagined tool-use pantomimes.
Neurology, 54, 1331–1336.
Murata, A., Gallese, V., Luppino, G., Kaseda, M., & Sakata, H. (2000). Selectivity for the shape,
size, and orientation of objects for grasping in neurons of monkey parietal area AIP. Journal
of Neurophysiology, 83, 2580–2601.
Negri, G. A., Lunardelli, A., Reverberi, C., Gigli, G. L., & Rumiati, R. I. (2007). Degraded se-
mantic knowledge and accurate object use. Cortex, 43, 376–388.
Negri, G. A. L., Rumiati, R. I., Zadini, A., Ukmar, M., Mahon, B. Z., & Caramazza, A. (2007).
What is the role of motor simulation in action and object recognition? Evidence from
apraxia. Cognitive Neuropsychology, 24, 795–816.
Nelson, S. M., Cohen, A. L., Power, J. D., Wig, G. S., Miezin, F. M., et al. (2010). A parcellation
scheme for human left lateral parietal cortex. Neuron, 67, 156–170.
Noppeney, U., Price, C. J., Penny, W. D., & Friston, K. J. (2006). Two distinct neural mechanisms
for category-selective responses. Cerebral Cortex, 16, 437–445.
Ochipa, C., Rothi, L. J. G., & Heilman, K. M. (1989). Ideational apraxia: A deficit in tool selec-
tion and use. Annals of Neurology, 25, 190–193.
Orban, G. A., Claeys, K., Nelissen, K., Smans, R., Sunaert, S., et al. (2006). Mapping the parietal
cortex of human and non-human primates. Neuropsychology, 44, 2647–2667.
Osiurak, F., & Badets, A. (2016). Tool use and affordance: Manipulation- based versus
reasoning-based approaches. Psychological Review, 123(5), 534–568.
Pandya, D. N., & Seltzer, B. (1982). Intrinsic connections and architectonics of posterior pari-
etal cortex in the rhesus monkey. Journal of Comparative Neurology, 204, 196–210.
Peeters, R. R., Rizzolatti, G., & Orban, G.A. (2013). Functional properties of the left parietal
tool use region. NeuroImage, 78,  83–93.
Pelgrims, B., Olivier, E., & Andres, M. (2011). Dissociation between manipulation and con-
ceptual knowledge of object use in supramarginalis gyrus. Human Brain Mapping, 32,
1802–1810.
Pisella, L., Binkofski, F., Lasek, K., Toni, I., & Rossetti, Y. (2006). No double-dissociation be-
tween optic ataxia and visual agnosia: Multiple sub-strams for multiple visuo-manual
integrations. Neuropsychologia, 44, 2734–3748.
Pisella, L., Gréa, H., Tilikete, C., Vighetto, A., Desmurget, M., et al. (2000). An “automatic
pilot” for the hand in human posterior parietal cortex: Toward reinterpretation of optic
ataxia. Nature Neuroscience, 3, 729–736.
Pobric, G., Jeffries, E., & Lambon Ralph, A. (2010). Amodal semantic representations depend
on both anterior temporal lobes: Evidence from repetitive transcranial magnetic stimula-
tion. Neuropsychologia, 48, 1336–1342.
Pobric, G., Jefferies, E., & Ralph, M. A. L. (2010). Category-specific versus category-general
semantic impairment induced by transcranial magnetic stimulation. Current Biology, 20,
964–968.
Poeppel, D. (2012): The maps problem and the mapping problem: Two challenges for a cogni-
tive neuroscience of speech and language. Cognitive Neuropsychology, 29,  34–55.
Rapcsak, S. Z., Ochipa, C., Anderson, K. C., & Poizner, H. (1995). Progressive ideomotor
apraxia: Evidence for a selective impairment of the action production system. Brain and
Cognition, 27, 213–236.
Rizzolatti, G., & Matelli, M. (2003). Two different streams form the dorsal visual
system: Anatomy and functions. Experimental Brain Research, 153, 146–157.
Rosci, C., Chiesa, V., Laiacona, M., & Capitani, E. (2003). Apraxia is not associated to a dispro-
portionate naming impairment for manipulable objects. Brain and Cognition, 53, 412–415.
Rossetti, Y., Pisella, L., & Vighetto, A. (2003). Optic ataxia revisited. Experimental Brain
Research, 153, 171–179.
Rossit, S., McAdam, T., Mclean, D. A., Goodale, M. A., & Culham, J. C. (2013). fMRI reveals
a lower visual field preference for hand actions in human superior parieto-occipital cortex
(SPOC) and precuneus, Cortex, 49, 2525–2541.
Rothi, L. J. G., Ochipa, C., & Heilman, K. M. (1991). A cognitive neuropsychological model of
limb praxis. Cognitive Neuropsychology, 8, 443–458.
Rumiati, R. I., Weiss, P. H., Shallice, T., Ottoboni, G., Noth, J., Zilles, K., & Fink, G. R. (2004).
Neural basis of pantomiming the use of visually presented objects. NeuroImage, 21,
1224–1231.
Rumiati, R. I., Zanini, S., Vorano, L., and Shallice, T. (2001). A form of ideational apraxia as a
selective deficit of contention scheduling. Cognitive Neuropsychology, 18, 617–642.
Ruschel, M., Knösche, T. R., Friederici, A. D., Turner, R., Geyer, S., & Anwander, A. (2014).
Connectivity architecture and subdivision of the human inferior parietal cortex revealed by
diffusion MRI. Cerebral Cortex, 24(9), 2436–2448.
Rushworth, M. F. S., Behrens, T. E. J., & Johansen-Berg, H. (2006). Connection patterns distin-
guish 3 regions of human parietal cortex. Cerebral Cortex, 16, 1418–1430.
Sakata, H., Taira, M., Mine, S., & Murata, A. (1992). The hand-movement-related neurons
of the posterior parietal cortex of the monkey: Their role in the visual guidance of hand
movement. In R. Caminiti, P. B. Johnson, Y. Bumod (Eds.), Control of arm movement in
space: Neurophysical and computational approaches (pp. 185–198). Berlin: Springer.
Sakata, H., Taira, M., Murata, A., & Mine, S. (1995). Neural mechanisms of visual guidance of
hand action in the parietal cortex of the monkey. Cerebral Cortex, 5, 429–438.
Sincich, L. C., Park, K. F., Wohlgemuth, M. J., & Horton, J. C. (2004) Bypassing V1: A direct ge-
niculate input to area MT. Nature Neuroscience, 7, 1123–1128.
Sirigu, A., J. Duhamel, J. R., & Poncet, M. (1991). The role of sensorimotor experience in object
recognition. Brain, 114, 2555–2573.
Snowden, J., Griffiths, H., & Neary, D. (1994). Semantic dementia: Autobiographical contribu-
tion to preservation of meaning. Cognitive Neuropsychology, 11, 265–288.
Snowden, J. S., Griffiths, H. L., & Neary, D. (1996). Semantic-episodic memory interactions
in semantic dementia: Implications for retrograde memory function. Cognitive
Stasenko, A., Garcea, F. E., Dombovy, M., & Mahon, B. Z. (2014). When concepts lose their
color: A case of object-color knowledge impairment. Cortex, 58, 217–238.
Stevens, W. D., Tessler, M. H., Peng, C. S., Martin, A. (2015). Functional connectivity constrains
the category-related organization of human ventral occipitotemporal cortex. Human Brain
Mapping, 36, 2187–2206.
Tessari, A., Canessa, N., Ukmar, M., & Rumiati, R. I. (2007). Neuropsychological evidence for a
strategic control of multiple routes in imitation. Brain, 130(4), 1111–1126.
Tranel, D., Damasio, H., & Damasio, A. R. (1997). A neural basis for the retrieval of conceptual
knowledge. Neuropsychologia, 35, 1319–1327.
Tranel, D., Kemmerer, D., Adolphs, R., Damasio, H., & Damasio, A. R. (2003). Neural correlates
of conceptual knowledge for actions. Cognitive Neuropsychology, 20, 409–432.
Ungerleider, L. G., & Mishkin, M. (1982). Two cortical visual systems. In D. J. Ingle, M. A.
Goodale, & R. J. W. Mansfield (Eds.), Analysis of visual behavior (pp. 549–586). Cambridge,
MA: MIT Press.
Wu, W. (2008). Visual attention, conceptual content, and doing it right. Mind, 117, 1003–1033.
Yee, E., Drucker, D. M., & Thompson-Schill, S. L. (2010). fMRI-adaptation evidence of
overlapping neural representations for objects related in function or manipulation.
NeuroImage, 50(2), 753–763.
Zhang, Y., Fan, L., Zhang, Y., Wang, J., Zhu, M., Zhang, Y., et al. (2014). Connectivity-based
parcellation of the human posteromedial cortex. Cerebral Cortex, 24, 719–727.
Zhong Y., & Rockland, K. S. (2003). Inferior parietal lobule projections to anterior
inferotemporal cortex (Area TE) in macaque monkey. Cerebral Cortex, 13, 527–540.
Chapter 24
Neu ral Basi s of

Monolingua l a nd
Bilingual Re a di ng
Pedro M. Paz-A lonso, Myriam Oliver,

Ileana Quiñones, and Manuel Carreiras
The use of printed symbols to represent the world is a unique human activity. Compared
to oral communication, writing and reading are very recent abilities in the history of
human evolution, emerging approximately about 4,000–5,000 years ago. The first
known systems of writing were pictographs and counting methods used in commerce in
3,200 b.c.e. (Spar & Lambert, 2005). In addition to the evolution of written symbols, so-
cial expectations and assumptions about reading correspondingly evolved (Finkelstein
& McCleery, 2012). In this regard, those individuals who could read held considerable
economic and social importance (Moorhead, 2011). This made reading not only a very
useful activity, but also an important social ability that probably contributed to its ex-
pansion. Currently, reading has become an elementary part of our modern society, so
that we are continually exposed to written messages in media (e.g., television, adver-
tising messages on the street, etc.) and also in technologies (e.g., computers, mobile
phones).
Nonetheless, reading is not a simple process; it involves a series of sequential/parallel
and mutually dependent cascades of neurocognitive operations. Reading requires first
the visual recognition of a word or a symbol and the correspondence between this visual
form and its phonology. Furthermore, alphabetic writing systems require an aware-
ness of the phonological constituents that make up the word. This awareness allows the
reader to connect letter strings (i.e., orthography) to the corresponding units of speech
(i.e., phonology). Moreover, reading requires accessing the meaning of those words (i.e.,
semantics). Thus, typical reading relies on a progressive interaction between visual, or-
thographic, phonological, and semantic systems. The use of advanced neuroimaging
techniques (see the chapters in Part I of this volume) allows researchers to examine
604 Paz-Alonso, Oliver, Quiñones, and Carreiras
in vivo the neural correlates underlying these operations. Better understanding of the
neural dynamics supporting reading is a priority for cognitive neuroscience, with pro-
found implications for elucidation of the brain mechanisms supporting typical and
atypical reading (i.e., reading disorders), as well as to contribute to improving educa-
tional policies.
The amount of research dedicated to understanding the neural correlates of reading
has steadily increased over the last two decades. A PubMed search using “reading” and
“MRI” as combined terms reveals over 3,100 peer-reviewed journal articles published
between 1996 and 2016. A detailed analysis indicates that the main goals of these studies
were to examine brain processes associated with orthographic, phonological, and/or se-
mantic systems embedded in written material. However, whereas some of these studies
focused on regional specialization or examined the contribution of specific regions,
more recent studies have followed a network-based approach or combined regional-
and network-based approaches.
The present chapter is aimed at reviewing what is known about the neural bases of
reading in monolingual and bilingual populations. Specifically, the first section reviews
empirical evidence on the functional specialization of left perisylvian reading regions
and their participation in orthographic, phonological, and semantic reading systems,
paying special attention to current theoretical debates on the functional role of these
main reading nodes. Second, we present an overview of the ventral and dorsal reading
networks and the factors that seem to modulate them, including the type of stimuli,
reading demands, and language orthography. Finally, the third section of the chapter
pays particular attention to the neural correlates underlying bilingual reading and
reviews research evidence showing that age of acquisition, language proficiency, lan-
guage exposure, and language orthography modulate the engagement of reading re-
gions and networks in bilinguals.
Functional Specialization of Left

Perisylvian Reading Regions
Empirical evidence has demonstrated that reading is mediated by a dedicated cortical

network, located along the perisylvian areas of the left hemisphere (e.g., Carreiras,
Mechelli, Estévez, & Price, 2007; Jobard, Crivello, & Tzourio-Mazoyer, 2003). As illus-
trated in Figure 24.1, four different large brain areas can be identified within this net-
work: the inferior frontal gyrus (including pars opercularis, triangularis, and orbitalis),
middle and superior temporal gyrus, parietal cortex, and ventral occipito-temporal
cortex (Jobard et al., 2003; Sandak et al., 2004). Researchers have long debated the role
played by these key language-related regions in reading tasks involving orthographic,
phonological, and semantic processing (Schlaggar & McCandliss, 2007), and despite
some differences in terms of results and theoretical views about the roles of some of
Neural Basis of Monolingual and Bilingual Reading 605
Figure 24.1. Meta-analysis of fMRI activations associated with the term “reading” in a total
of 427 studies. Regions in blue represent areas that are reported more selectively with the term
reading (reverse inference, Z = 1.96).
Abbreviations: IFG = inferior frontal gyrus; MTG/STG = middle/superior temporal gyrus; PC = parietal cortex;
vOT = ventral occipito-temporal cortex; L = left hemisphere; R = right hemisphere.
Source: Results from Neurosynth.org (Yarkoni, Poldrack, Nichols, Van Essen & Wager, 2011).
these regions, neuroimaging research has been seminal in understanding their overall
contribution to reading processes at regional and network levels.
Inferior Frontal Gyrus
The left inferior frontal gyrus (IFG) has been indicated as a key component of reading
(Price, 2012). Empirical evidence has shown the engagement of this region in word
reading (Mechelli, Price, Friston, & Ishai, 2004), lexical and semantic retrieval (Binder,
Desai, Graves, & Conant, 2009; Carreiras, Mechelli, & Price, 2006), and mapping or-
thography to semantics (Jobard et al., 2003).
The IFG can be divided into its anterior-ventral (pars orbitalis and pars triangularis),
and posterior-dorsal parts (pars opercularis). These ventral and dorsal subdivisions are
mainly based on cytoarchitectonic and neuroanatomical studies conducted over the
twentieth century. One of the first cytoarchitectonic differentiations of the anterior
and posterior parts of the human IFG was proposed by Brodmann (1909). Specifically,
there are variations between anterior and posterior parts of the IFG in size and packing
density of cell bodies over the layers of the cortical sheet. Brodmann’s initial division of
the IFG was further refined by comparative neuroanatomical studies between monkey
and human brains (Economo & Koskinas, 1925; Petrides & Pandya, 1994, 2002; Walker,
1940; see Petrides, Tomaiuolo, Yeterian, & Pandya, 2012, for a review). Moreover, other
parameters were added successively to this initial topological classification, such as the
distribution and amount of intra-cortical myelinated fibers (Vogt, 1910), precise ax-
onal terminations in IFG parts in the macaque brain (Petrides & Pandya, 2009), den-
sity of certain neurotransmitters receptors (Zilles, Palomero‐Gallagher, & Schleicher,
2004) and resting-state functional connectivity in humans based on the macaque
monkey data (Kelly et al., 2010).
Consistently, recent neuroimaging studies on reading have found a differential func-

tional involvement of anterior-ventral and posterior-dorsal IFG regions (e.g., Badre &
Wagner, 2007; Price, 2012). Specifically, the anterior IFG is more strongly engaged in
studies using semantic reading tasks and when word retrieval during the reading task
is semantically demanding, such as retrieving narratives (Badre & Wagner, 2002, 2004;
De Zubicaray, Wilson, McMahon, & Muthiah, 2001; Wagner, Paré-Blagoev, Clark, &
Poldrack, 2001). In contrast, when stronger phonological demands are required by the
task, for example during a rhyming task, stronger posterior IFG engagement is typi-
cally found (Bitan et al., 2005; Gabrieli, Poldrack, & Desmond, 1998; Poldrack et al.,
1999). Structural and functional neuroimaging evidence has also highlighted the crit-
ical role of the interactions between IFG and lateral temporal cortex during reading,
with different parts of the left IFG orchestrating controlled retrieval of lexical/semantic
representations from lateral posterior temporal cortex via top-down processes (e.g.,
Badre & Wagner, 2007; Hagoort, 2013; Thompson-Schill, D’Esposito, Aguirre, & Farah,
1997; see also Molinaro, Paz-Alonso, Duñabeitia, & Carreiras, 2015).
Middle/Superior Temporal Gyrus
Neuroscientific reading models have agreed on the participation and roles of the IFG
and posterior middle/superior temporal gyrus (MTG/STG) in reading processes (e.g.,
Friederici, 2012; Hagoort, 2013; Lau, Phillips, & Poeppel, 2008; Snijders, Petersson, &
Hagoort, 2010; see also Brouwer & Hoeks, 2013; Jefferies, 2013). In most of these models,
accessing lexical/semantic information related to single words is associated with pos-
terior MTG/STG activation. Neuroimaging and neuropsychological research have
also shown the involvement of these posterior temporal regions in tasks that require
judgments on word semantic properties, semantic categorizations, or lexical semantic
processes (e.g., Cappa, Perani, Schnur, Tettamanti, & Fazio, 1998; Price et al., 1994;
Pugh et al., 1996). Furthermore, aphasic patients with lesions in these posterior tem-
poral regions have difficulties performing semantic tasks that require access to lexical
representations (e.g., Dronkers, Wilkins, Van Valin, Redfern, & Jaeger, 2004; Hart &
Gordon, 1990).
Parietal Cortex
The specific role of the parietal cortex in reading processes has long been debated.
Empirical evidence has shown that different regions within the parietal cortex are as-
sociated with different language functions. Whereas some studies have highlighted
the overall involvement of parietal regions in phonological processing, others have
pointed out that certain regions within the parietal cortex are involved in semantic
processes. One the one hand, previous studies have suggested the involvement of the
supramarginal gyrus (SMG) in phonological reading processes (Sliwinska, Khadilkar,
Campbell-Ratcliffe, Quevenco, & Devlin, 2012), such as assembled phonology or

grapheme-to-phoneme mapping (Graves, Desai, Humphries, Seidenberg, & Binder
2010; Mei et al., 2014; Mei et al., 2015), processing phonological versus processing se-
mantic information (e.g., Devlin et al., 2003; Seghier et al., 2004), and reading aloud
pseudowords versus words (e.g., Binder, Medler, Desai, Conant, & Liebenthal, 2005;
Carreiras, Duñabeitia, & Perea, 2007; Vigneau, Jobard, Mazoyer, & Tzourio-Mazoyer,
2005). On the other hand, the angular gyrus is typically involved in semantic processing
of auditory and visual stimuli (Demonet et al., 1992; Molinaro et al., 2015; Vandenberghe,
Price, Wise, Josephs, & Frackowiak, 1996), a finding that has been extensively and reli-
ably replicated across multiple studies with different semantic tasks and stimuli (e.g.,
Binder et al., 2009; Cabeza & Nyberg, 2000; Quiñones et al., 2014; Vigneau et al., 2006).
Importantly, the left angular gyrus and SMG can be parcellated into multiple func-
tionally different subregions. For instance, Seghier et al.’s (Seghier, Fagan, & Price,
2010) study characterized three functional subdivisions in the left angular gyrus dif-
ferentially related to semantic processing and processes related to the default-network.
Similarly, Oberhuber et al.’s (2016) study found four functionally distinct regions, all
within left SMG, that have implications for differentiating between different types of
phonological processing operations, including articulatory sequencing, auditory pro-
cessing, word production, and executive functioning. Thus, even within these pari-
etal cortex regions, there are relevant differences with regard to the specific functional
contributions of their subregions.
Interestingly, some recent findings have also highlighted the role of parietal regions
in the early stages of visual word recognition, suggesting that these areas contribute to
letter-identity and letter-position processing. For instance, Reilhac, Peyrin, Démonet,
and Valdois (2013) reported increased activity in the left superior and inferior pa-
rietal cortex during processing of two consecutive five-letter strings that could differ
in the replacement of some internal characters (e.g., TSHFL-TSHFL versus TSHFL-
TGHML). In addition, Carreiras, Quiñones, Hernádez- Cabrera, and Duñabeitia
(2014) manipulated character-position and character-identity through the transpo-
sition or substitution of two internal elements within strings of four letters, digits or
symbols (e.g., letters: NDTF-NTDF vs. NDTF-NKSF; digits: 1754-1574 vs. 1754-1684;
symbols: &$?<-&?$< vs. &$?<-&%€<). In this study, participants were asked to decide
whether the two subsequent strings were the same or different. Critically, these authors
showed a dissociation effect for letters versus digits and symbols, both relative to the
letter-identity and letter-position coding processes. In particular, the left inferior and
superior parietal gyri showed increased activation for letter strings as compared to the
other types of characters. Moreover, common effects of transpositions for all types of
characters were found in the right inferior and superior parietal and in the right an-
gular gyrus, but at the same time differential effects for letters as compared to digits and
symbols were found, with letter-specific effects in the left superior and inferior parietal
cortex. These studies highlight the importance of the parietal cortex in perceptual tasks
that involve letter identification, and suggest that parietal regions may be involved in the
early stages of visual processing.
Ventral Occipito-Temporal Cortex
The left ventral occipito-temporal cortex (vOT) plays a crucial role in reading and has
been proposed as a core area for orthographic processing during visual word recog-
nition. This region has been proposed as the gateway from vision to language, where
visual word recognition occurs prior to accessing nonvisual information (Twomey,
Kawabata Duncan, Price, & Devlin, 2011). This region exhibits a marked preference for
letter strings with orthographic information, showing higher activation for words and
pseudowords relative to false fonts or checkerboards (e.g., Baker et al., 2007; Cohen
et al., 2002; Binder, Medler, Westbury, Liebenthal, & Buchanan, 2006). However, crit-
ically, the preferential left vOT engagement for visual orthographic processing, its
functional selectivity, as well as its functional role during reading, are still subjected to
important debates (e.g., Carreiras, Armstrong, Perea, & Frost, 2014; Dehaene & Cohen,
2011; Price & Devlin, 2011).
Some theoretical accounts highlight left vOT involvement in bottom-up pre-lexical
computation processes of visual word forms (Dehaene & Cohen, 2011; Cohen et al.,
2000), while others underline its implication in integrating visuospatial features ab-
stracted from sensory inputs with higher-level associations via bottom-up and top-
down connections (i.e., interactive account; Price & Devlin, 2011). Nevertheless,
differences in terms of reading paradigms and analytical approaches can potentially
contribute to these divergences with regard to the putative functional role of this region
in reading (Glezer & Riesenhuber, 2013). First, whereas studies supporting a pre-lexical
computational role of the vOT have mainly used low-level reading tasks (e.g., passive
silent reading; Cohen et al., 2002; Dehaene et al., 2010; Pegado, Nakamura, Cohen, &
Dehaene, 2011), most of the studies favoring the interactive account of the vOT have
typically utilized high-level reading tasks (e.g., lexical decision; Seghier & Price, 2013;
Twomey et al., 2011; Woollams, Silani, Okada, Patterson, & Price, 2011). Second, there is
a strong inter-subject variability in the location of the left vOT, probably due to its rapid
readaptation to support a phylogenetically new skill (Bouhali et al., 2014; Dehaene &
Cohen, 2011), and differences in how this region is neuroanatomically identified in dif-
ferent studies can also determine different results and interpretations about its role. For
instance, Glezer and Riesenhuber’s (2013) study revealed that the left vOT showed word
selectivity when it was identified at the individual-subject level, but inter-subject varia-
bility when the vOT was identified at the group level or was based on coordinates from
previous studies caused this selectivity to be washed out.
A recent functional magnetic resonance imaging (fMRI) study (Oliver, Carreiras, &
Paz-Alonso, 2017) investigated differences in left vOT as a function of perceptual (i.e.,
to identify words that contained a colored letter) versus semantic (i.e., to identify any
words that were the name of an animal) demands on reading, additionally using dif-
ferent strategies to identify this region: individual-subject level, group level, and coor-
dinates from previous studies (see Figure 24.2). Results revealed that semantic reading
demands relative to perceptual reading demands elicited stronger recruitment of the left
litROI gROIs iROIs
Left vOT
ROIs
2
Perceptual task
1.8
Semantic task
1.6
1.4
% Signal Change
1.2
1
0.8
0.6
0.4
0.2
0
Words Pseudow. Consts. Words Pseudow. Consts. Words Pseudow. Consts.
litROI gROIs iROIs
Figure 24.2. Left vOT ROI analyses based on three different approaches: literature-based ROI
(litROI), group-based ROIs (gROIs), and individually defined ROIs (iROIs). Bar graphs show
averaged parameter estimates (% signal change) for these three left vOT ROI definitions as a
function of words, pseudowords, and consonant string stimuli.
Abbreviations: Pseudow. = pseudowords; Consts. = consonant strings.
Source: Modified with permission from Oliver et al. (2017).
vOT. Thus, low-versus high-level reading demands may modulate differences typically
observed with regard to the functional role of left vOT in previous studies.
Dorsal and Ventral Reading Networks
Based on behavioral evidence, two main theoretical models have been proposed to
explain reading processes: the parallel distributed processing (PDP) model (Harm &
Seidenberg, 1999; Plaut, McClelland, Seidenberg, & Patterson, 1996; Seidenberg &
McClelland, 1989), and the dual route cascaded (DRC) model (Coltheart, Rastle, Perry,
Langdon & Ziegler, 2001). Although these models have put forward relevant hypotheses
about how the orthographic, phonological, and semantic systems are divided and
interrelated to each other, they did not provide specific predictions in relation to the
neural computations carried out by the left perisylvian reading regions. However, in
line with these models, neuroimaging research has been important in elucidating a di-
vision of labor between brain regions, organized into ventral and dorsal reading neural
networks. Research evidence has shown that the ventral reading network, including
left vOT and anterior IFG (pars orbitalis and pars triangularis), supports mapping of
orthographic-lexical stimuli onto semantic representations (Sandak et al., 2004). On the
other hand, the dorsal reading network, encompassing parietal cortex, STG, and pos-
terior IFG (pars opercularis), is thought to subserve the link between the orthographic
form of words and semantics via phonological processing (Pugh et al., 2001; Schlaggar
& McCandliss, 2007).
Investigating visual word recognition involves establishing links between the pro-
cessing of visual features, and semantic and phonological processing on these ventral
and dorsal streams. Evidence of semantic and phonological reliance on ventral and
dorsal pathway regions comes from studies revealing a specific modulation of these
two pathways based on the type of stimuli and reading demands. Regarding the type
of stimuli, ventral regions typically show greater engagement for high-frequency words
than for pseudowords (Jobard et al., 2003; Vigneau et al., 2005), which suggests that
the ventral pathway is recruited for well-learned words and is involved in more auto-
matic word identification (Jamal, Piche, Napoliello, Perfetti & Eden, 2012). In contrast,
dorsal regions typically show stronger activation during reading pseudowords and
low-frequency words (Borowsky & Besner, 2006; Mechelli, Gorno-Tempini, & Price,
2003; Tagamets, Novick, Chalmers, & Friedman, 2000). The increased activation for
pseudowords in regions along the dorsal stream may reflect demands on accessing
phonology from sub-lexical orthographic codes (Price, 2012). Concerning reading
demands, ventral regions appear to be more strongly activated by semantic than phono-
logical reading tasks (Price & Mechelli, 2005). In contrast, the opposite pattern is found
in dorsal regions, which are typically more activated for phonological than semantic
reading-related decisions (Paulesu, Frith, & Frackowiak, 1993; Zatorre, Evans, Meyer, &
Gjedde, 1992).
Importantly, different languages present different writing systems, and therefore
reading demands may differ depending on the characteristics of a given language.
Neuroimaging studies have revealed a different modulation of these ventral and dorsal
reading pathways depending on the orthography of the language (Oliver et al., 2017;
Paulesu et al., 2000; Rueckl et al., 2015). Languages can be classified according to their
grapheme-to-phoneme mapping or language orthography (Frost, Katz, & Bentin,
1987). Transparent orthographies, such as Spanish or Italian, have a shallow letter-to-
sound correspondence. Opaque orthographies, such as English, have a more complex
grapheme-to-phoneme mapping. Given that transparent orthographies have relatively
simple, consistent, and complete connections between letters and phonemes, it is easier
for readers to recover the phonology of the printed words pre-lexically, by assembling
grapheme-to-phoneme correspondences. In other words, the phonology of transparent
orthographies directly supports word-recognition processes. In contrast, in opaque

orthographies, the reader must process printed words by referring to their morphology
via the visual-orthographic structure of the printed words (Frost et al., 1987; Pugh
et al., 2001).
Neuroimaging evidence has shown that whereas transparent languages rely more
on phonological areas within the dorsal pathway, opaque orthographies rely more
on ventral pathway regions—consistent with graphemic analysis, to resolve possible
ambiguities with pronunciation (Meschyan & Hernandez, 2006; Paulesu et al., 2000).
However, the opposite pattern was also found in another study (Jamal et al., 2012),
suggesting that if less phonological processing is needed, as would be expected in trans-
parent languages, there may be less use of the phonological (dorsal) pathway and greater
use of the semantic (ventral) pathway. In contrast, in opaque languages, greater activa-
tion of the dorsal route would be expected. Further studies on language orthography are
needed to resolve these mixed findings.
In a recent fMRI study intended to examine the overlap between speech and reading
networks across languages varying in orthographic depth and in writing system (i.e.,
Spanish, English, Hebrew, Chinese), Rueckl et al. (2015) found that a striking convergence
emerged as a common brain signature across the wide spectrum of selected languages.
This speech-print convergence may reflect a universal principle of brain organization
resulting from the biological constraints imposed by initial perisylvian specialization for
speech and natural-language processing, and the cognitive imperative to leverage these
biologically specialized systems for supporting reading comprehension across highly
contrastive languages such as Spanish and Chinese. Importantly, Rueckl et al.’s (2015)
study also revealed a higher speech-print correlation among regions along the dorsal
reading network for transparent (i.e., Spanish) versus opaque orthographies (i.e., English
and Hebrew), including left SMG/inferior parietal lobe (IPL) and left IFG. In contrast,
speech-print correlations were higher for these opaque orthographies compared with
transparent orthography in left angular gyrus, left posterior MTG, left vOT, and left pars
triangularis in the IFG (see Figure 24.3). Results from this study are especially relevant to
the debate about language orthography effects in ventral and dorsal reading networks.
Although there has been extensive discussion about how orthographic depth modulates
the reading networks, most of this previous work comes from behavioral research
(Caravolas et al., 2012; Caravolas, Lervåg, Defior, Málková, & Hulme, 2013; Frost et al.,
1987; Ziegler et al., 2010) and the few available functional neuroimaging studies on this
topic have been typically limited to two-language comparisons (e.g., Cherodath & Singh,
2015). Moreover, Rueckl et al.’s (2015) work constitutes the first study that has specifically
examined speech-print convergence, showing that sites with stronger speech-print cou-
pling in orthographically transparent systems such as Spanish are generally associated
with phonological processing, whereas the sites with stronger speech-print coupling in
orthographically deep systems such as English and Hebrew (including angular gyrus,
MTG, and ITG) are more closely associated with semantic processing.
Additional support showing the different types of decoding required during the
reading process as found in fMRI studies on the ventral and dorsal functional pathways
(a)
Meta-analyses
(reading-related)
Transparent>
Opaque
Opaque>
Transparent
(b) LIPL/Supramarginal gyrus L Angular gyrus

Spanish > English : Spanish > Hebrew English > Spanish : Hebrew > Spanish
Spanish
4 English 8
3 Hebrew
6
2
1 4
Speech
0
–4 –2 2 4 2
–1 0
–2 0
–6 –4 –2 0 2 4 6 8
–3 –2
–4
–4
–5
–6 Print –6
Figure 24.3. Speech-print convergence as a factor of orthographic depth in three contras-

tive alphabetic languages. (A) Areas in yellow show the reading circuit identified by published
meta-analyses. Speech-print correlations were higher for transparent (Spanish) than opaque
orthographies (English and Hebrew compared independently, P <0.05 for each comparison)
in areas shown in pink. Areas shown in light blue show higher correlations for the opaque
orthographies compared with the transparent orthography, P <0.05. (B) Scatter plots of the cor-
relation between print and speech activation in representative areas showing greater convergence
for more transparent languages in left inferior parietal lobe, supramarginal gyrus (left), and left
angular gyrus for opaque languages (right).
Source: Adapted with permission from Rueckl et al. (2015).
has been provided by studies using diffusion-weighted imaging (DWI; see Catani
& Forkel, Chapter 9 in this volume). This technique allows the identification of white
matter micro-structure properties of the fiber bundles connecting distant cortical re-
gions. Structural connections between temporal and frontal cortical regions are
mediated by different ventral and dorsal fiber tracts (Friederici, 2012; Saur et al., 2008).
For instance, using a deterministic fiber-tracking approach with the two end points of
the connections predefined on the basis of functional data, Saur et al. (2008) showed
that within the ventral pathway, the extreme fiber capsule system (connecting temporal
cortex with pars triangularis and pars orbitalis) supported sound-to-meaning mapping.
On the other hand, the structural dorsal pathway (going from the temporal lobe to the
premotor cortex and continuing to the pars opercularis, via the arcuate and superior lon-
gitudinal fascicle) supported sensorimotor mapping of sound-to-articulation.
Reading in Bilinguals
A relevant demand in today’s world is the need to speak more than one language. The
rapid need for bilingualism and multilingualism in modern societies encourages
a significant proportion of individuals to learn new languages over childhood and
adult years. How does the brain accomplish such a feat? How is language organized in
individuals who know more than one language? The following section provides a de-
scription of the neural mechanisms underlying bilingual reading.
Bilingualism is increasingly common in most of the world today. In many coun-
tries, bilingualism is the norm, while in predominantly monolingual countries (e.g.,
the United States) the size of the bilingual population has been steadily growing over
the last two decades. Thus, it is not surprising that study of the neural correlates un-
derlying bilingualism is also on the increase. For people who speak two languages,
understanding tens of thousands of words in each language is an exceptional accom-
plishment of their cognitive and neural systems. One of the main issues in this area that
is generating heated debate is to what extent bilinguals share or do not share the same
neural networks for their L1 and L2 (Parker Jones et al., 2012; Price, 2012; Schlaggar &
McCandliss, 2007). Neuropsychological evidence has shown that injuries to Broca’s area
in bilinguals can produce deficits in language production (i.e., expressive aphasia) in
their L1, but not necessarily in their L2 (Wolf et al., 1999). In contrast, neuroimaging re-
search has yielded mixed results in this regard, with some studies revealing differential
regional activation in L1 relative to L2 processing (Kim, Relkin, Lee, & Hirsch, 1997) and
other studies showing extensive overlap among the regions recruited for L1 and L2 pro-
cessing (Friederici, Steinhauer, & Pfeifer, 2002; Musso et al., 2003).
Nevertheless, there are some relevant factors that are known to influence the func-
tional and structural correlates of reading networks in bilinguals, such as the age of ac-
quisition (AoA) of the L2, the proficiency level, exposure, and language orthography
(Jamal et al., 2012; Mechelli et al., 2004; Perani et al., 2003; Wartenburger et al., 2003).
These factors can explain these and other inconsistencies in neuroimaging research in
regard to the overlap or the lack of overlap between L1 and L2 regions and networks in-
volved in bilingual reading. A description of the functional and structural findings re-
garding these factors is discussed next.
Age of Acquisition
The fact that children appear to acquire an L2 faster and more efficiently than adults
suggests the existence of a temporal window for optimal L2 learning (see Lennenberg,
1967, for the concept of critical period). Evidence supporting the existence of an op-
timal temporal window for L2 learning is supported by neuropsychological studies
of bilinguals with different L2 AoA. Findings from these studies have revealed that
language recovery in aphasics is considered to be excellent if occurring in early in-

fancy, while older children show more adult-like aphasic patterns and poorer recovery
(Bates et al., 2001; see Reilly & Polse, Chapter 10 in this volume). While neuroimaging
studies of bilingual reading are still relatively scarce, extensive overlap in perisylvian re-
gions for L1 and L2 reading has been typically found in early bilinguals, but not in late
bilinguals (Kim et al., 1997; Wartenburger et al., 2003). In particular, Kim et al.’s (1997)
study revealed that separate but adjacent tissue of Broca´s area (encompassing IFG
pars opercularis and triangularis) was activated for L1 and L2 in late bilinguals, whereas
stronger overlap within this region was found in early bilinguals for L1 and L2 pro-
cessing. More recently, similar results were observed in reading studies with bilinguals
(Wartenburger et al., 2003), and also in studies focused on different language-related
processing, such as syntax (Mahendra, Plante, Magloire, Milman, & Trouard, 2003) and
lexical aspects (Isel & Shen, 2011) This evidence indicates that the later an L2 is acquired,
the less the overlap observed in cortical representations for L1 and L2. This means that
additional resources are required in late bilingual L2 processing to achieve native-like
performance. In contrast, early bilinguals learn both languages in parallel and appear to
show stronger overlap in brain regions for processing L1 and L2 (Hernandez & Li, 2007;
Perani & Abutalebi, 2005).
The effects of L2 AoA have been also investigated using structural measures. Empirical
evidence suggests that the earlier bilinguals begin L2 learning, the more gray-matter
density is found in the left inferior parietal cortex (Mechelli et al., 2004). The structural
changes in this region are described as reflecting explicit learning strategy for linking
new words to concepts (Richardson, Seghier, Leff, Thomas, & Price, 2011), a process that
is implicit when learning languages from birth. Furthermore, differences in white-matter
microstructure also have been found as a function of the AoA. Early bilinguals relative to
late bilinguals exhibited higher fractional anisotropy (FA; a measure of the coherence of
directionality of white-matter fibers) values in the left inferior frontal occipital fasciculus
(IFOF), which has been suggested to play a role in semantic processing (Mohades et al.,
2012). Higher FA values may be related to increased structural connectivity between
reading related regions, such as among the regions that the IFOF connects, due to greater
language experience or automaticity (Mohades et al., 2012; Schlegel, Rudelson, & Peter,
2012). Recently, one study suggested that later acquisition of L2 is associated with thicker
cortex in the left IFG, a crucial region within the reading network (Klein, Mok, Chen, &
Watkins, 2014), interpreted by the authors to be due to less synaptic pruning in this region
as a result of acquiring L2 later in development. Overall, the bulk of evidence suggests
that acquiring an L2 as a new skill earlier or later in life induces different functional and
anatomical changes in reading-and language-related regions.
Language Proficiency
Some studies have pointed to proficiency as the main factor determining differences in
the recruitment of neural regions for reading in L1 and L2. Neuroimaging studies have
shown that lower proficiency in the L2 is associated with greater IFG pars orbitalis acti-
vation during reading (Chee, Hon, Lee, & Soon, 2001). Convergent evidence showing
hyperactivation in participants with lower versus high proficiency was found in studies
examining other language-related tasks, such as verbal fluency (Vingerhoets, 2003),
semantic and syntactic judgments (Wartenburger et al., 2003), and naming tasks
(Hernandez, Dapretto, Mazziotta, & Bookheimer, 2001). This hyperactivation during L2
reading and other L2 language-related processes may be due to a lower degree of auto-
maticity because of the lower proficiency in L2. In line with this interpretation, when L2
and L1 proficiency are equated, early and late bilinguals tend to exhibit strikingly similar
neural responses for processing both the L1 and L2 (Abutalebi, Cappa, & Perani, 2001;
Chee et al., 2001; Frenck-Mestre, Anton, Roth, Vaid, & Viallet, 2005; Klein, Watkins,
Zatorre, & Milner, 2006). Thus, as L2 proficiency increases, differences in the neural
correlates of reading in L1 versus L2 decrease. AoA effects might therefore be dimin-
ished when early and late learners are equated on proficiency.
Furthermore, L2 proficiency also seems to induce changes in structural measures.
L2 proficiency correlates with increased gray-matter density in a variety of reading-
related regions, such as left inferior parietal cortex (IPC) (Mechelli et al., 2004), left IFG
(Grogan, Green, Ali, Crinion, & Price, 2009; Stein et al., 2012) and left STG (Martensson
et al., 2012). Additionally, increases in WM connectivity between left-hemispheric
perisylvian regions and their right homolog regions (IFG, caudate nucleus, STG) are
related to the level of L2 proficiency in English learners of Chinese (Schlegel et al., 2012).
In sum, higher L2 proficiency is associated with increased gray-matter density or struc-
tural increases in gray matter and also with increased white-matter connectivity in
reading-related fiber tracts.
Language Exposure
The level of exposure to L1 and L2 has also been implicated in modulation of the neural
correlates of reading processes in bilinguals (e.g., Tu et al., 2015). Even when the degree
of proficiency is kept constant, differential exposure seems to be associated with differ-
ential engagement of reading-related regions (Vingerhoets et al., 2013). Similar results
have been observed in studies using language-related tasks other than reading. For ex-
ample, Perani et al.’s (2003) study compared two groups of highly proficient bilinguals,
differing in their L2 exposure, on a word-generation task. Participants with high L2 ex-
posure exhibited reduced activation in left prefrontal cortex relative to those with lower
L2 exposure. Thus, experience and practice in the L2 may result in a decrease of neural
activation, suggesting more automaticity and less effort or resources needed for L2 pro-
cessing (Abutalebi, 2008; see Green & Kroll, Chapter 11 in this volume). Recent evidence
also suggests that even brief 30-day periods of differential exposure to a given language
may induce significant neural changes in areas responsible for language processing (Tu
et al., 2015). Furthermore, neuroimaging studies have proposed that L2 exposure can
even replace L1 when bilinguals are no longer exposed to their native language (Pallier
et al., 2003). The role of L2 exposure was also investigated in a few structural studies that
found correlations between L2 exposure and gray-matter density in language-related
regions, such as the bilateral posterior putamen (Pliatsikas, Johnstone, & Marinis, 2014).
Based on these neuroimaging findings, the amount of language exposure seems to affect
the functional and structural neural responses involved in L2 processing.
Language Orthography
Orthographies vary across languages in their grapheme-to-phoneme correspondence.
Orthography differences modulate the involvement of the functional reading networks
(Paulesu et al., 2000; Rueckl et al., 2015). Disparities in the activation of reading networks
may be also visible in bilinguals with two different language orthographies (e.g., Spanish-
English bilinguals). In fact, Meschyan and Hernandez (2006) reported differences in
activation profiles for early Spanish-English bilinguals during L1 and L2 reading. More
specifically, in Meschyan and Hernandez’s (2006) study, Spanish reading elicited greater
activation in the right supplementary motor area, right putamen, right insula, and left
STG. In contrast, English reading elicited greater activation in the IPC and along regions
in the occipito-parietal vicinity. However, participants in this study differed in proficiency,
and so it is not clear if these differences were due to proficiency or to language orthography
per se. A more recent study of Spanish-English late bilinguals matched in proficiency re-
vealed functional differences in the reading network as a function of language orthography
(Jamal et al., 2012). Interestingly, one study that tested simultaneous and sequential Hindi-
English bilinguals showed a division of labor with more ventral activation for English (an
opaque orthography) and more dorsal activation for Hindi (a transparent orthography),
as is the case for each language in English and Hindi monolinguals. However, sequential
bilinguals with late exposure to English showed more dorsal activation when reading both
in English and in Hindi (Das, Padakannaya, Pugh, & Singh, 2011). The findings of this
study suggest that early, but not late, exposure to reading distinct orthographies results in
orthography-specific plasticity that persists through adulthood. Altogether, this evidence
suggests that bilinguals show changes to the typical neural representations of reading,
depending upon language orthography. However, it is still not clear what specific neural
representations are expected as a function of the orthography and if this divergent activa-
tion is confounded with other variables, such as AoA.
Importantly, a recent study (Oliver et al., 2017) examining the functional dynamics
of ventral and dorsal reading networks in bilinguals with the same L1 (i.e., Spanish) and
either an opaque (i.e., English) or transparent (i.e., Basque) L2 showed that differences
in orthographic depth of the L2 were associated with strikingly segregated profiles in the
coupling of left vOT with regions along the ventral network in the group with an opaque
L2, on the one hand, and with regions along the dorsal network for the group with a
transparent L2, on the other (Figure 24.4). These results were observed during word
reading, with further analysis indicating that this pattern of co-activation with left vOT
holds for word reading in L2, but not for word reading in L1. Oliver et al.’s (2017) findings
Spanish-Basque Spanish-Basque
Spanish-English
Spanish-English
Figure 24.4. Left-hemisphere brain renderings showing whole-brain functional connectivity

maps with left vOT for the contrast Words > Null separately for the Spanish-Basque (in green) and
Spanish-English (in yellow) groups at a voxel-wise FDR-corrected statistical threshold of q < .0001.
Source: Modified with permission from Oliver et al. (2017).
are consistent with evidence suggesting that transparent orthographies with strong
letter-to-sound mapping rely more on dorsal regions and that opaque orthographies
rely more on lexico-semantic processes carried out by regions along the ventral pathway
(Das et al. 2011; Paulesu et al., 2000; Rueckl et al., 2015).
In sum, based on the research evidence reviewed in this section, it is possible to con-
clude that the brain has an extraordinary ability to functionally and structurally change
in response to environmental stimuli, cognitive demands, and experiences. Specifically,
reading demands, type of stimuli, AoA, language proficiency and exposure, and lan-
guage orthography are crucial factors that can affect the brain representations of each
language in bilingual reading. Despite recent advances in cognitive neuroscience re-
garding bilingual reading, results from functional and structural studies with bilingual
populations are still scarce and inconsistent. Future studies should take into account
the influence of the variables reviewed here (i.e., AoA, proficiency, exposure, language
orthography) and the importance of controlling or manipulating these within the
same study. Importantly, there is also increasing evidence that individual differences
in functional activation are related to underlying brain structure. Therefore, future
investigations can shed further light on the neural bases of bilingual reading by com-
bining structural and functional data in the same study.
Conclusions
Neuroimaging research on the neurobiology of reading has provided critical insights

about how monolingual and bilingual brains achieve reading. In the present chapter, we
have reviewed well-established findings and current debates on the (1) functional spe-
cialization of left perisylvian reading regions and their participation in orthographic,
phonological, and sematic reading systems; (2) the division of labor between ventral
and dorsal reading networks and the factors that modulate them; and (3) the neural
correlates supporting bilingual reading and the impact of AoA, language proficiency,
language exposure, and language orthography in the modulation of bilingual reading
regions and networks. Additional research is still needed to further inform ongoing
debates, particularly with regard to the factors that modulate bilingual reading. This
area is of growing theoretical and practical interest due to the fact that more than half of
the world’s population is bilingual, but little is still known about what the most benefi-
cial and efficient educational practices are for reading acquisition in bilingual and mul-
tilingual populations.
Acknowledgments
Supported by grants (RYC- 2014-
15440, PSI2015- 65696) from the Spanish Ministry of
Economy and Competitiveness (MINECO) and a grant (PI2016-12) from the Gobierno Vasco,
departamento de educación, política lingüística y cultura (P.M.P-A.); a pre-doctoral grant from
the Department of Education, Universities and Research from the Basque Government (M.O.);
grants (PSI2015-67353-R) from the MINECO, and a grant (ERC-2011-ADG-295362) from the
European Research Council (M.C.). The Basque Center on Cognition, Brain and Language
acknowledges funding from Ayuda Centro de Excelencia Severo Ochoa SEV-2015-0490 from
the MINECO. We thank Margaret Gillon-Dowens for helpful editing and comments.
References
Abutalebi, J. (2008). Neural aspects of second language representation and language control.
Acta Psychologica, 128(3), 466–478.
Abutalebi, J., Cappa, S. F., & Perani, D. (2001). The bilingual brain as revealed by functional
neuroimaging. Bilingualism: Language and Cognition, 4(2), 179–190.
Badre, D., & Wagner, A. D. (2002). Semantic retrieval, mnemonic control, and prefrontal
cortex. Behavioral and Cognitive Neuroscience Reviews, 1(3), 206–218.
Badre, D., & Wagner, A. D. (2004). Selection, integration, and conflict monitoring: Assessing
the nature and generality of prefrontal cognitive control mechanisms. Neuron, 41(3),
473–487.
trol of memory. Neuropsychologia, 45(13), 2883–2901.
Baker, C. I., Liu, J., Wald, L. L., Kwong, K. K., Benner, T., & Kanwisher, N. (2007). Visual word
processing and experiential origins of functional selectivity in human extrastriate cortex.
Bates, E., Reilly, J., Wulfeck, B., Dronkers, N., Opie, M., Fenson, J., . . . Herbst, K. (2001).
Differential effects of unilateral lesions on language production in children and adults. Brain
and Language, 79(2), 223–265.
19(12), 2767–2796.
Binder, J. R., Medler, D. A., Desai, R., Conant, L. L., & Liebenthal, E. (2005). Some neurophysi-
ological constraints on models of word naming. NeuroImage, 27(3), 677–693.
Binder, J. R., Medler, D. A., Westbury, C. F., Liebenthal, E., & Buchanan, L. (2006). Tuning
of the human left fusiform gyrus to sublexical orthographic structure. NeuroImage, 33(2),
739–748.
Bitan, T., Booth, J. R., Choy, J., Burman, D. D., Gitelman, D. R., & Mesulam, M. M. (2005). Shifts
of effective connectivity within a language network during rhyming and spelling. Journal of
Neuroscience, 25(22), 5397–5403.
Borowsky, R., & Besner, D. (2006). Parallel distributed processing and lexical-semantic effects
in visual word recognition: Are a few stages necessary? Psychologival Review, 113(1), 181–195.
Bouhali, F., Thiebaut de Schotten, M., Pinel, P., Poupon, C., Mangin, J. F., Dehaene, S., & Cohen,
L. (2014). Anatomical connections of the visual word form area. Journal of Neuroscience,
34(46), 15402–15414.
Brodmann, K. (1909). Vergleichende Localisationslehre der Grosshirnrinde in ihren Prinzipien
dargestellt auf Grund des Zellenbaues (J. A. Barth). Translated by Laurence Garey as
Localisation in the cerebral cortex (1994). London: Smith-Gordon, Imperial College Press.
Brouwer, H., & Hoeks, J. C. (2013). A time and place for language comprehension: Mapping the
N400 and the P600 to a minimal cortical network. Frontiers in Human Neuroscience, 7, 758.
Cabeza, R., & Nyberg, L. (2000). Neural bases of learning and memory: Functional neuroim-
aging evidence. Current Opinion in Neurology, 13(4), 415–421.
Cappa, S. F., Perani, D., Schnur, T., Tettamanti, M., & Fazio, F. (1998). The effects of semantic
category and knowledge type on lexical-semantic access: A PET study. NeuroImage, 8(4),
350–359.
Caravolas, M., Lervåg, A., Defior, S., Málková, G. S., & Hulme, C. (2013). Different patterns, but
equivalent predictors, of growth in reading in consistent and inconsistent orthographies.
Psychological Science, 24(8), 1398–1407.
Caravolas, M., Lervåg, A., Mousikou, P., Efrim, C., Litavský, M., Onochie-Quintanilla, E., . . . &
Seidlová-Málková, G. (2012). Common patterns of prediction of literacy development in dif-
ferent alphabetic orthographies. Psychological Science, 23(6), 678–686.
Carreiras, M., Duñabeitia, J. A., & Perea, M. (2007). Reading words, NUMB3R5 and $
YMßOL$. Trends in Cognitive Sciences, 11(11), 454–455.
Carreiras, M., Armstrong, B. C., Perea, M., & Frost, R. (2014) The what, when, where, and how
of visual word recognition. Trends in Cognitive Sciences, 18(2),  90–98.
Carreiras, M., Mechelli, A., Estevez, A., & Price, C. (2007). Brain activation for lexical deci-
sion and reading aloud: Two sides of the same coin? Journal of Cognitive Neuroscience, 19(3),
433–444.
Carreiras, M., Mechelli, A., & Price, C. (2006). The effect of word and syllable frequency on ac-
tivation during lexical decision and reading aloud. Human Brain Mapping, 27(12), 963–972.
Carreiras, M., Quiñones, I., Hernández-Cabrera, J. A., & Duñabeitia, J. A. (2014). Orthographic
coding: Brain activation for letters, symbols, and digits. Cerebral Cortex, 25(12), 4748–4760.
Chee, M. W., Hon, N., Lee, H. L., & Soon, C. S. (2001). Relative language proficiency modulates
BOLD signal change when bilinguals perform semantic judgments: Blood oxygen level de-
pendent. NeuroImage, 13, 1155–1163.
Cherodath, S., & Singh, N. C. (2015). The influence of orthographic depth on reading networks
in simultaneous biliterate children. Brain and Language, 143,  42–51.
Cohen, L., Dehaene, S., Naccache, L., Lehericy, S., Dehaene-Lambertz, G., Henaff, M. A., &
Michel, F. (2000). The visual word form area: Spatial and temporal characterization of an ini-
tial stage of reading in normal subjects and posterior split-brain patients. Brain, 123, 291–307.
Cohen, L., Lehericy, S., Chochon, F., Lemer, C., Rivaud, S., & Dehaene, S. (2002). Language-
specific tuning of visual cortex? Functional properties of the Visual Word Form Area. Brain,
125, 1054–1069.
Coltheart, M., Rastle, K., Perry, C., Langdon, R., & Ziegler, J. (2001). DRC: A dual route
cascaded model of visual word recognition and reading aloud. Psychological Review,
108(1), 204.
Das, T., Padakannaya, P., Pugh, K. R., & Singh, N. C. (2011). Neuroimaging reveals dual routes
to reading in simultaneous proficient readers of two orthographies. NeuroImage, 54(2),
1476–1487.
ference effect in the picture‐word paradigm: An event‐related fMRI study employing overt
responses. Human Brain Mapping, 14(4), 218–227.
Dehaene, S., & Cohen, L. (2011). The unique role of the visual word form area in reading. Trends
in Cognitive Sciences, 15(6), 254–262.
Dehaene, S., Pegado, F., Braga, L. W., Ventura, P., Nunes Filho, G., Jobert, A., . . . Cohen, L.
(2010). How learning to read changes the cortical networks for vision and language. Science,
330(6009), 1359–1364.
Demonet, J. F., Chollet, F., Ramsay, S., Cardebat, D., Nespoulous, J. L., Wise, R., . . . &
Frackowiak, R. (1992). The anatomy of phonological and semantic processing in normal
subjects. Brain, 115(6), 1753–1768.
Devlin, J. T., Raley, J., Tunbridge, E., Lanary, K., Floyer-Lea, A., Narain, C., . . . & Moore, D. R.
(2003). Functional asymmetry for auditory processing in human primary auditory cortex.
Dronkers, N. F., Wilkins, D. P., Van Valin, R. D., Redfern, B. B., & Jaeger, J. J. (2004). Lesion
analysis of the brain areas involved in language comprehension. Cognition, 92(1), 145–177.
Finkelstein, D., & McCleery, A. (2012). Introduction to book history. New York: Routledge.
Frenck-Mestre, C., Anton, J. L., Roth, M., Vaid, J., & Viallet, F. (2005). Articulation in early
and late bilinguals’ two languages: Evidence from functional magnetic resonance imaging.
NeuroReport, 16(7), 761–765.
Friederici, A. D. (2012). The cortical language circuit: From auditory perception to sentence
comprehension. Trends in Cognitive Science, 16(5), 262–268.
Friederici, A. D., Steinhauer, K., & Pfeifer, E. (2002). Brain signatures of artificial language
processing: Evidence challenging the critical period hypothesis. Proceedings of the National
Frost, R., Katz, L., & Bentin, S. (1987). Strategies for visual word recognition and orthographical
depth: A multilingual comparison. Journal of Experimental Psychology: Human Perception
and Performance, 13(1), 104.
Gabrieli, J. D., Poldrack, R. A., & Desmond, J. E. (1998). The role of left prefrontal cortex in lan-
guage and memory. Proceedings of the National Academy of Sciences USA, 95(3), 906–913.
Glezer, L. S., & Riesenhuber, M. (2013). Individual variability in location impacts orthographic
selectivity in the “visual word form area.” Journal of Neuroscience, 33(27), 11221–11226.
Graves, W. W., Desai, R., Humphries, C., Seidenberg, M. S., & Binder, J. R. (2010). Neural sys-
tems for reading aloud: A multiparametric approach. Cerebral Cortex, 20(8), 1799–1815.
Grogan, A., Green, D. W., Ali, N., Crinion, J. T., & Price, C. J. (2009). Structural correlates of
semantic and phonemic fluency ability in first and second languages. Cerebral Cortex, 19(11),
2690–2698.
Hagoort, P. (2013). MUC (memory, unification, control) and beyond. Frontiers in Psychology,
4, 416.
Hart, J., & Gordon, B. (1990). Delineation of single‐word semantic comprehension deficits in
aphasia, with anatomical correlation. Annals of Neurology, 27(3), 226–231.
Hernandez, A. E., Dapretto, M., Mazziotta, J., & Bookheimer, S. (2001). Language switching
and language representation in Spanish–English bilinguals: An fMRI study. NeuroImage,
14(2), 510–520.
Hernandez, A. E., & Li, P. (2007). Age of acquisition: Its neural and computational mechanisms.
Psychological Bulletin, 133(4), 638–650.
Isel, F., & Shen, W. (2011). Modulation of semantic integration as a function of syntactic expec-
tations: Event-related brain potential evidence. NeuroReport, 22(5), 195–199.
Jamal, N. I., Piche, A. W., Napoliello, E. M., Perfetti, C. A., & Eden, G. F. (2012). Neural basis of
single-word reading in Spanish-English bilinguals. Human Brain Mapping, 33(1), 235–245
Jefferies, E. (2013). The neural basis of semantic cognition: Converging evidence from neuro-
psychology, neuroimaging and TMS. Cortex, 49(3), 611–625.
Jobard, G., Crivello, F., & Tzourio-Mazoyer, N. (2003). Evaluation of the dual route theory of
reading: A metanalysis of 35 neuroimaging studies. NeuroImage, 20(2), 693–7 12.
Kelly, C., Uddin, L. Q., Shehzad, Z., Margulies, D. S., Castellanos, F. X., Milham, M. P., &
Petrides, M. (2010). Broca’s region: Linking human brain functional connectivity data and
non‐human primate tracing anatomy studies. European Journal of Neuroscience, 32(3),
383–398.
Kim, K. H., Relkin, N. R., Lee, K. M., & Hirsch, J. (1997). Distinct cortical areas associated with
native and second languages. Nature, 388(6638), 171–174.
Klein, D., Mok, K., Chen, J. K., & Watkins, K. E. (2014). Age of language learning shapes brain
structure: A cortical thickness study of bilingual and monolingual individuals. Brain &
Language, 131,  20–24.
Klein, D., Watkins, K. E., Zatorre, R. J., & Milner, B. (2006). Word and nonword repetition in
bilingual subjects: A PET study. Human Brain Mapping, 27(2), 153–161.
Lau, E. F., Phillips, C., & Poeppel, D. (2008). A cortical network for semantics: (de)constructing
the N400. Nature Reviews Neuroscience, 9(12), 920–933.
Lenneberg, E. (1967). Biological foundations of language. New York: John Wiley & Sons.
Mahendra, N., Plante, E., Magloire, J., Milman, L., & Trouard, T. P. (2003). fMRI variability and
the localization of languages in the bilingual brain. NeuroReport, 14(9), 1225–1228.
Martensson, J., Eriksson, J., Bodammer, N. C., Lindgren, M., Johansson, M., Nyberg, L., &
Lovden, M. (2012). Growth of language-related brain areas after foreign language learning.
NeuroImage, 63(1), 240–244.
Mechelli, A., Gorno-Tempini, M. L., & Price, C. J. (2003). Neuroimaging studies of word and
pseudoword reading: Consistencies, inconsistencies, and limitations. Journal of Cognitive
Mechelli, A., Price, C. J., Friston, K. J., & Ishai, A. (2004). Where bottom-up meets top-down:
Neuronal interactions during perception and imagery. Cerebral Cortex, 14(11), 1256–1265.
Mei, L., Xue, G., Lu, Z. L., He, Q., Wei, M., Zhang, M., . . . & Chen, C. (2015). Native language
experience shapes neural basis of addressed and assembled phonologies. NeuroImage,
114,  38–48.
Mei, L., Xue, G., Lu, Z. L., He, Q., Zhang, M., Wei, M., . . . & Dong, Q. (2014). Artificial language
training reveals the neural substrates underlying addressed and assembled phonologies.
PLoS One, 9(3), e93548.
Meschyan, G., & Hernandez, A. E. (2006). Impact of language proficiency and orthographic
transparency on bilingual word reading: An fMRI investigation. NeuroImage, 29(4), 1135–1140.
Mohades, S. G., Struys, E., Van Schuerbeek, P., Mondt, K., Van De Craen, P., & Luypaert, R.
(2012). DTI reveals structural differences in white matter tracts between bilingual and mon-
olingual children. Brain Research, 1435,  72–80.
Molinaro, N., Paz-Alonso, P. M., Duñabeitia, J. A., & Carreiras, M. (2015). Combinatorial se-
mantics strengthens angular-anterior temporal coupling. Cortex, 65, 113–127.
Moorhead, J. (2011). Reading in late antiquity. In S. Towheed, R. Crone, & K. Halsey (Eds.), The
history of reading (pp. 52–65). London: Taylor and Francis.
Musso, M., Moro, A., Glauche, V., Rijntjes, M., Reichenbach, J., Büchel, C., & Weiller, C. (2003).
Broca’s area and the language instinct. Nature Neuroscience, 6(7), 774–781.
Oberhuber, M., Hope, T. M. H., Seghier, M. L., Jones, O. P., Prejawa, S., Green, D. W., & Price,
C. J. (2016). Four functionally distinct regions in the left supramarginal gyrus support word
processing. Cerebral Cortex, 26(11), 4212–4226.
Oliver, M., Carreiras, C., & Paz-Alonso, P. M. (2017). Functional dynamics of dorsal and ven-
tral reading networks in bilinguals. Cerebral Cortex, 27(12), 5431–5443.
Pallier, C., Dehaene, S., Poline, J.-B., LeBihan, D., Argenti, A.-M., Dupoux, E., & Mehler, J.
(2003). Brain imaging of language plasticity in adopted adults: Can a second language re-
place the first? Cerebral Cortex, 13(2), 155–161.
Parker Jones, O., Green, D. W., Grogan, A., Pliatsikas, C., Filippopolitis, K., Ali, N., . . . Price,
during picture naming and reading aloud. Cerebral Cortex, 22(4), 892–902.
Paulesu, E., Frith, C. D., & Frackowiak, R. S. (1993). The neural correlates of the verbal compo-
nent of working memory. Nature, 362(6418), 342–345.
Paulesu, E., McCrory, E., Fazio, F., Menoncello, L., Brunswick, N., Cappa, S. F., . . . Frith, U.
(2000). A cultural effect on brain function. Nature Neuroscience, 3(1),  91–96.
Pegado, F., Nakamura, K., Cohen, L., & Dehaene, S. (2011). Breaking the symmetry: Mirror dis-
crimination for single letters but not for pictures in the Visual Word Form Area. NeuroImage,
55(2), 742–749.
Perani, D., & Abutalebi, J. (2005). The neural basis of first and second language processing.
Current Opinion in Neurobiology, 15(2), 202–206.
Perani, D., Abutalebi, J., Paulesu, E., Brambati, S., Scifo, P., Cappa, S. F., & Fazio, F. (2003). The
role of age of acquisition and language usage in early, high-proficient bilinguals: An fMRI
study during verbal fluency. Human Brain Mapping, 19(3), 170–182.
Petrides, M., & Pandya, D. N. (1994). Comparative architectonic analysis of the human and the
macaque frontal cortex. Handbook of Neuropsychology, 9,  17–58.
Petrides, M., & Pandya, D. N. (2002). Comparative cytoarchitectonic analysis of the human
and the macaque ventrolateral prefrontal cortex and corticocortical connection patterns in
the monkey. European Journal of Neuroscience, 16(2), 291–310.
Petrides, M., & Pandya, D. N. (2009). Distinct parietal and temporal pathways to the
homologues of Broca’s area in the monkey. PLoS Biology, 7(8), e1000170.
Petrides, M., Tomaiuolo, F., Yeterian, E. H., & Pandya, D. N. (2012). The prefrontal
cortex: Comparative architectonic organization in the human and the macaque monkey
brains. Cortex, 48(1),  46–57.
Plaut, D. C., McClelland, J. L., Seidenberg, M. S., & Patterson, K. (1996). Understanding
Pliatsikas, C., Johnstone, T., & Marinis, T. (2014). Grey matter volume in the cerebellum is re-
lated to the processing of grammatical rules in a second language: A structural voxel-based
morphometry study. The Cerebellum, 13(1),  55–63.
Poldrack, R. A., Wagner, A. D., Prull, M. W., Desmond, J. E., Glover, G. H., & Gabrieli, J. D.
(1999). Functional specialization for semantic and phonological processing in the left infe-
rior prefrontal cortex. NeuroImage, 10(1),  15–35.
Price, C. J., & Devlin, J. T. (2011). The interactive account of ventral occipitotemporal
contributions to reading. Trends in Cognitive Science, 15(6), 246–253.
Price, C. J., & Mechelli, A. (2005). Reading and reading disturbance. Current Opinion in
Price, C. J., Wise, R. J. S., Watson, J. D., Patterson, K., Howard, D., & Frackowiak, R. S. J. (1994).
Brain activity during reading: The effects of exposure duration and task. Brain, 117(6),
1255–1269.
Pugh, K. R., Mencl, W. E., Jenner, A. R., Katz, L., Frost, S. J., Lee, J. R., . . . Shaywitz, B. A. (2001).
Neurobiological studies of reading and reading disability. Journal of Communication
Disorders, 34(6), 479–492
Pugh, K. R., Shaywitz, B. A., Shaywitz, S. E., Constable, R. T., Skudlarski, P., Fulbright, R.
K., . . . & Gore, J. C. (1996). Cerebral organization of component processes in reading. Brain,
119(4), 1221–1238.
Quiñones, I., Molinaro, N., Mancini, S., Hernández-Cabrera, J. A., & Carreiras, M. (2014).
Where agreement merges with disagreement: fMRI evidence of subject-verb integration.
NeuroImage, 88, 188–201
Reilhac, C., Peyrin, C., Démonet, J. F., & Valdois, S. (2013). Role of the superior parietal lobules
in letter-identity processing within strings: fMRI evidence from skilled and dyslexic readers.
Neuropsychologia, 51(4), 601–612.
Richardson, F. M., Seghier, M. L., Leff, A. P., Thomas, M. S., & Price, C. J. (2011). Multiple
routes from occipital to temporal cortices during reading. Journal of Neuroscience, 31(22),
8239–8247.
Rueckl, J. G., Paz-Alonso, P. M., Molfese, P. J., Kuo, W. J., Bick, A., Frost, S. J., . . . Lee, J. R. (2015).
Universal brain signature of proficient reading: Evidence from four contrasting languages.
Sandak, R., Mencl, W. E., Frost, S. J., Rueckl, J. G., Katz, L., Moore, D. L., . . . Pugh, K. R. (2004).
The neurobiology of adaptive learning in reading: A contrast of different training conditions.
Cognitive, Affective, & Behavioral Neuroscience, 4(1),  67–88.
Saur, D., Kreher, B. W., Schnell, S., Kummerer, D., Kellmeyer, P., Vry, M. S., . . . Weiller, C.
(2008). Ventral and dorsal pathways for language. Proceedings of the National Academy of
Sciences USA, 105(46), 18035–18040.
Schlaggar, B. L., & McCandliss, B. D. (2007). Development of neural systems for reading.
Annual Review of Neuroscience, 30, 475–503.
Schlegel, A. A., Rudelson, J. J., & Peter, U. T. (2012). White matter structure changes as adults
learn a second language. Journal of Cognitive Neuroscience, 24(8), 1664–1670.
Seghier, M. L., Fagan, E., & Price, C. J. (2010). Functional subdivisions in the left angular
gyrus where the semantic system meets and diverges from the default network. Journal of
Neuroscience, 30(50), 16809–16817.
Seghier, M. L., Lazeyras, F., Pegna, A. J., Annoni, J. M., Zimine, I., Mayer, E., . . . Khateb, A.
(2004). Variability of fMRI activation during a phonological and semantic language task in
healthy subjects. Human Brain Mapping, 23(3), 140–155.
Seghier, M. L., & Price, C. J. (2013). Dissociating frontal regions that co-lateralize with dif-
ferent ventral occipitotemporal regions during word processing. Brain & Language, 126(2),
133–140.
Seidenberg, M. S., & McClelland, J. L. (1989). A distributed, developmental model of word rec-
ognition and naming. Psychological Review, 96(4), 523–568.
Sliwinska, M. W., Khadilkar, M., Campbell-Ratcliffe, J., Quevenco, F., & Devlin J. T. (2012).
Early and sustained supramarginal gyrus contributions to phonological processing.
Snijders, T. M., Petersson, K. M., & Hagoort, P. (2010). Effective connectivity of cortical and
subcortical regions during unification of sentence structure. NeuroImage, 52(4), 1633–1644.
Spar, I., & Lambert, W. G. (2005). Cuneiform texts in the Metropolitan Museum of Art: Literary
and scholastic texts of the first millennium BC (Vol. 2). New York: Metropolitan Museum
of Art.
Stein, M., Federspiel, A., Koenig, T., Wirth, M., Strik, W., Wiest, R., . . . Dierks, T. (2012).
Structural plasticity in the language system related to increased second language profi-
ciency. Cortex, 48(4), 458–465.
Tagamets, M.-A., Novick, J. M., Chalmers, M. L., & Friedman, R. B. (2000). A parametric
approach to orthographic processing in the brain: An fMRI study. Journal of Cognitive
National Academy of Sciences USA, 94(26), 14792–14797.
Tu, L., Wang, J., Abutalebi, J., Jiang, B., Pan, X., Li, M., . . . Huang, R. (2015). Language exposure
induced neuroplasticity in the bilingual brain: A follow-up fMRI study. Cortex, 64,  8–19.
Twomey, T., Kawabata Duncan, K. J., Price, C. J., & Devlin, J. T. (2011). Top-down modulation
of ventral occipito-temporal responses during visual word recognition. NeuroImage, 55(3),
1242–1251.
Vandenberghe, R., Price, C., Wise, R., Josephs, O., & Frackowiak, R. S. (1996). Functional
anatomy of a common semantic system for words and pictures. Nature, 383(6597), 254–256.
and sentence processing. NeuroImage, 30(4), 1414–1432.
Vigneau, M., Jobard, G., Mazoyer, B., & Tzourio-Mazoyer, N. (2005). Word and non-word
reading: What role for the Visual Word Form Area? NeuroImage, 27(3), 694–705.
Vingerhoets, G. (2003). Multilingualism: An fMRI study. NeuroImage, 20(4), 2181–2196.
Vingerhoets, G., Alderweireldt, A. S., Vandemaele, P., Cai, Q., Van der Haegen, L., Brysbaert,
M., & Achten, E. (2013). Praxis and language are linked: Evidence from co-lateralization in
individuals with atypical language dominance. Cortex, 49(1), 172–183.
Vogt, O. (1910). Die myeloarchitektonische Felderung des menschlichen Stirnhirns. Journal of
Psychological Neurology, 15(4/5), 221–232.
von Economo, C. F., & Koskinas, G. N. (1925). Die cytoarchitektonik der hirnrinde des
erwachsenen menschen. Berlin: Springer.
Wagner, A. D., Paré-Blagoev, E. J., Clark, J., & Poldrack, R. A. (2001). Recovering meaning: Left
prefrontal cortex guides controlled semantic retrieval. Neuron, 31(2), 329–338.
Walker, A. E. (1940). A cytoarchitectural study of the prefrontal area of the macaque monkey.
Journal of Comparative Neurology, 73(1),  59–86.
Wartenburger, I., Heekeren, R. H., Abutalebi, J., Cappa, S. F., Villringer, A., & Perani, D. (2003).
Early setting of grammatical processing in the bilingual brain. Neuron, 37(1), 159–170.
Wolf, D. S., Gearing, M., Snowdon, D. A., Mori, H., Markesbery, W. R., & Mirra, S. S. (1999).
Progression of regional neuropathology in Alzheimer disease and normal elderly: Findings
from the Nun study. Alzheimer Disease & Associated Disorders, 13(4), 226–231.
Woollams, A. M., Silani, G., Okada, K., Patterson, K., & Price, C. J. (2011). Word or word-like?
Dissociating orthographic typicality from lexicality in the left occipito-temporal cortex.
Yarkoni, T., Poldrack, R. A., Nichols, T. E., Van Essen, D. C., & Wager, T. D. (2011). Large-
scale automated synthesis of human functional neuroimaging data. Nature Methods, 8(8),
665–670.
Zatorre, R. J., Evans, A. C., Meyer, E., & Gjedde, A. (1992). Lateralization of phonetic and pitch
discrimination in speech processing. Science, 256(5058), 846–849.
Ziegler, J. C., Bertrand, D., Toth, D., Csepe, V., Reis, A., Faisca, L., . . . Blomert, L. (2010).
Orthographic depth and its impact on universal predictors of reading: A cross-language in-
vestigation. Psychological Science, 21(4), 551–559.
Zilles, K., Palomero‐Gallagher, N., & Schleicher, A. (2004). Transmitter receptors and func-
tional anatomy of the cerebral cortex. Journal of Anatomy, 205(6), 417–432.
Chapter 25
Dyslexia a nd I ts
Neurobiol o g i c a l   Basi s
Kaja Jasińska and Nicole Landi
Introduction
Developmental dyslexia (DD) is one of the most prevalent developmental learning

disabilities, affecting approximately 7%–10% of school-age children (Lyon, Shaywitz, &
Shaywitz, 2003). Extensive research in the areas of education, psychology, linguistics,
and cognitive science conducted in the past several decades has identified a core set of
reading behaviors that are characteristic of children with DD, including poor decoding
and spelling, despite typical levels of intelligence. Although DD is a complex, polyge-
netic disorder it is widely accepted that many children with dyslexia have a primary im-
pairment in phonological processing (Harm & Seidenberg, 1999; Katz, 1986; Kovelman
et al., 2012; Tanaka et al., 2011; Vellutino, Pruzek, Steger, & Meshoulam, 1973; Wagner &
Torgesen, 1987; Ziegler, Perry, Ma-Wyatt, Ladner, & Schulte-Korne, 2003). This hypo-
thesis is supported by consistent findings that early deficits in phonological awareness
predict poor reading ability (Foy & Mann, 2006; Goswami & Bryant, 1990; Hulme, 2002;
Hulme, Caravolas, Málkova, & Brigstocke, 2005; Penke & Schrader, 2008; Scarborough,
1990; Snowling, 1987; Snowling, Van Wagtendonk, & Stafford, 1988; Stanovich, 1988;
Wagner & Torgesen, 1987). However, an observed deficit in phonological processing
could be caused by one or more underlying factors, including deficits in sensory pro-
cessing (Stein & Walsh, 1997; Tallal, 2004); visual attention (Bosse, Tainturier, & Valdois,
2007); sensitivity to amplitude rise times or temporal processing (Goswami et al., 2002);
access to phonological representations (Boets, 2014; Boets et al., 2013); or sensorimotor
integration (Kuhl, 2004; Preston et al., 2010; Pugh et al., 2013).
Although phonological deficits are typically observed in DD, some children with pho-
nological deficits do not have dyslexia, and the severity of phonological deficits alone
does not predict the severity of dyslexia (Peterson, Pennington, Shriberg, & Boada, 2009;
Snowling, 2000; Valdois et al., 2011). Given the considerable heterogeneity associated
Dyslexia and Its Neurobiological Basis 627
with DD and the complex profile of deficits observed across children with DD, it is be-
coming increasingly clear that dyslexia is unlikely to be reduced to one casual mech-
anism (Pennington, 2006; van Bergen, van der Leij, & de Jong, 2014; Wolf & Bowers,
1999). The multiple deficit model put forth by Pennington (2006) and elaborated by van
Bergen and colleagues (2014) proposes that multiple genetic factors act probabilistically,
in combination with environmental factors, to produce a continuum of liability for dys-
lexia and other developmental disorders. This model has particular power for explaining
the high comorbidity observed between DD and other neurodevelopmental disorders,
including specific language impairment (SLI)/Developmental Language Disorder
(DLD), attention-deficit hyperactivity disorder (ADHD), dyscalculia, and speech-
sound disorder. Elevated incidence of one of these disorders given the presence of an-
other suggests that they are not etiologically independent. Neurobiological evidence,
including both neuroimaging and genetic studies of DD, can inform theories of reading
that seek to better understand both the observed heterogeneity in DD and the high
rate of comorbidity between DD and other developmental disorders (see Paz-Alonso,
Oliver, Quiñones, & Carreiras, Chapter 24 in this volume). The following sections re-
view profiles of perceptual and cognitive performance associated with dyslexia, as well
as its genetic and neurobiological bases, including recent contributions from imaging
genetic approaches.
Phonological Deficit Hypothesis
The phonological deficit hypothesis assumes that the underlying cause of dyslexia is pri-
marily a deficit in representation of phonological information, access to phonological
information, or manipulation of phonological information. This hypothesis has been
widely supported because the primary impairments observed in dyslexia involve pho-
nology, including poor phonological awareness, poor phonological memory, and poor
grapheme to phoneme mapping (Joanisse et al., 2000; Snowling, 2000). The pivotal
place of phonological processing deficits in dyslexia has been further supported by
results from neuroimaging studies that highlighted the central role of phonological ac-
tivation in fluent word reading (see Pugh et al., 2000; Norton et al., 2015; Sandak, Mencl,
Frost, & Pugh, 2004).
Phonological awareness (PA), a meta-phonological skill that requires both the aware-
ness of individual speech sounds that make up words and the ability to manipulate them,
has been identified as a particularly strong predictor of reading skill (Foy & Mann, 2006;
Goswami & Bryant, 1990; Hulme, 2002; Hulme et al., 2005; Wagner & Torgesen, 1987).
Children with better phonological awareness perform better on tasks such as indicating
the number of syllables or phonemes in a word, naming a word that differs from other
words in onset, rime, or vowel (such as toy, boy, bay), repeating a word while omitting a
phoneme, or matching words that contain the same sound, and they are more likely to
become better readers. The strong relationship between PA and reading is linked that
628 Kaja Jasińska and Nicole Landi
children’s underlying understanding that individual speech sounds map onto letters,
and the proper representation of these mappings underlie successful decoding of new
words (Share, 1995; Share & Gur, 1999).
The ability to store phonological codes in short-term memory (phonological loop)
and maintain information about words and their meanings as lexical access occurs is
also necessary for skilled reading (Alloway, Gathercole, Willis, & Adams, 2004; Chein
& Schneider, 2005). Phonological short-term memory, which involves preserving
new incoming information while simultaneously being able to process this informa-
tion and integrate it with existing information (Baddeley, 1987), is also predictive of
reading decoding and fluency (McCallum et al., 2006). Further, children with poor
short-term working memory capacity, particularly in the verbal storage system, are
more likely to be poor readers (Gathercole, Tiffany, Briscoe, & Thorn, 2005; Swanson &
Jerman, 2007).
Impaired phonological representations may arise from speech perception problems
that compromise phonological representations, which manifest as higher-order phono-
logical processing and reading impairments. Phonological representations in dyslexia
may be underspecified (e.g., Elbro, Nielsen, & Petersen, 1994), causing confusion be-
tween similar speech sounds, and in turn, phonological processing difficulties. Indeed,
several studies find support for impaired phonological representations that correspond
to higher-level phonological processing, including impaired categorical perception
(Godfrey, Syrdal-Lasky, Millay, & Knox, 1981; Messaoud-Galusi, Hazan, & Rosen, 2011),
and others have revealed significant associations between early production delays and
reading difficulties (Preston et al., 2010).
Alternatively, some have suggested that children with dyslexia may have over-
specified representations at the allophonic (sub-phonemic variations within pho-
neme categories) level rather than phonemic level of representation, which may
be problematic for acquiring stable mappings between phonemes and graphemes
(Bogliotti, Serniclaes, Messaoud- Galusi, & Sprenger-
Charolles, 2008; Serniclaes,
Collet, & Sprenger-Charolles, 2015; Serniclaes, Van Heghe, Mousty, Carre, & Sprenger-
Charolles, 2004). Indeed, individuals with dyslexia have been shown to discriminate
between within-category allophonic variants of the same phoneme, whereas non-
dyslexic readers perceive allophonic variants as the same phoneme (Serniclaes, Van
Heghe, Mousty, Carre, & Sprenger-Charolles, 2004). Moreover, behavioral studies
of Dutch children at risk for dyslexia showed that these children demonstrate a shift
during kindergarten from allophonic perception to phonemic perception in first grade,
whereas non-dyslexic children already demonstrate developed phonemic percep-
tion in kindergarten (Noordenbos, Segers, Serniclaes, Mitterer, & Verhoeven, 2012a).
Further, mismatch negativity response (MMN, an electrophysiological measure of dis-
crimination; see Leckey & Federmeier, Chapter 3 in this volume) to both phonemic
and allophonic contrasts have been observed in adults with dyslexia, but only for pho-
nemic contrasts in adults without dyslexia (Noordenbos, Segers, Serniclaes, Mitterer,
& Verhoeven, 2012b). Finally, yet another possibility is that access to phonological in-
formation, rather than representation per se, may be the source of the phonological
deficit in dyslexia, discussed further under neurobiological bases (Boets, 2014; Boets
et al., 2013).
Double-Deficit Hypothesis
Impairments in the processes that underlie rapid word recognition have been detected
in children with dyslexia, and have been shown to be at least partially independent of
impairments in phonology, indicating that other aspects of language and cognition
contribute to the development of reading. As mentioned earlier, not all children with
poor phonological processing skills (broadly construed) will have reading difficulty
(Snowling, 2008), and not all children with reading difficulty necessarily show poor
phonological skills (Pennington et al., 2012; Valdois et al., 2011; van Bergen et al., 2014).
Thus, impairments in phonological processing may represent only a portion of the def-
icit in dyslexia, and deficits in processing speed have been hypothesized to contribute to
dyslexia profiles as well. Wolf and Bowers (1999) offered a double-deficit hypothesis for
reading impairment that implicated both phonological and speed of naming processes
as separate sources of reading difficulty (Katzir, Kim, Wolf, Morris, & Lovett, 2008; Wolf,
Bowers, & Biddle, 2000). Children with DD often present with a discrete deficit in ei-
ther phonological processing or processing speed, or a combination of both (McCardle,
Scarborough, & Catts, 2001). However, naming speed is not completely independent
from phonological processing, given that a word’s phonological form is accessed while
reading.
A key finding supporting the double deficit hypothesis is that children’s performance
on the Rapid Automatized Naming (RAN) task has been shown to be predictive of
fluent word reading (Wolf & Bowers, 1999). Typical RAN tasks involve speeded naming
of familiar stimuli (e.g., digits, letters, colors), and poor performance on this task is as-
sociated with dyslexia (Bowers & Swanson, 1991; Denckla & Rudel, 1974). Importantly,
RAN makes independent contributions to reading ability (Bowers & Swanson, 1991;
Wolf et al., 2002). That is, when phonological awareness ability has been accounted for,
RAN scores still significantly predict reading (Wolf et al., 2002). Speed of processing
during lexical access and the accurate retrieval of information that underlie RAN are
also components of multiple other cognitive abilities, and successful RAN perfor-
mance depends on other cognitive processes and abilities as well (e.g., attention; Arnell,
Joanisse, Klein, Busseri, & Tannock, 2009). For example, children with ADHD show
poorer performance on RAN measures than typically developing children (Waber,
Wolff, Forbes, & Weiler, 2000), as good performance on RAN tasks requires sustained
attention and the ability to exhibit inhibition of previous and upcoming responses while
the current response is executed (Wolf & Bowers, 1999). Thus RAN impairments sug-
gest deficits in the ability to process information accurately and quickly that go beyond
phonological processing. In the following, we review other processing deficits that are
hypothesized to contribute to dyslexia.
Sensory-P rocessing Deficits
As described earlier, phonological processing is a critical component of reading.

Importantly, the development of robust phonological representations depends on intact
sensory processing involved in auditory learning that is supported by multisensory in-
formation, including motor information from producing speech and visual information
from the face (Kuhl, 2004). Following on this argument, several theoretical approaches
have put forth low-level visual-and auditory-processing deficits as having a possible
causal role in the development and manifestation of dyslexia.
The magnocellular theory posits deficits in the dorsal visual pathway (Stein & Walsh,
1997) as a causal mechanism for dyslexia. For example, deficits have been observed
in the ability of children with dyslexia to effectively parse the visual field (detect mo-
tion, process spatial position)—the domain of magnocellular visual processing (Stein
& Walsh, 1997), However, other research suggests that visual processing that depends
on the magnocellular system is intact in individuals with dyslexia, or that deficits are
only observed when information is presented in noisy conditions (i.e., under increased
cognitive and attentional load). Sperling, Lu, Manis, and Seidenberg (2005) found
that individuals with dyslexia perform more poorly than individuals without dys-
lexia when identifying visual patterns presented in noisy conditions (akin to television
static). However, individuals with and without dyslexia perform alike when identifying
visual patterns presented in non-noisy conditions, suggesting that the deficit lies in
signal-noise discrimination rather than magnocellular processing (Sperling et al.,
2005). Moreover, compelling data suggest that the magnocellular visual-processing
impairments may not be a cause of dyslexia, but rather a consequence of limited reading
experience itself (Amitay, Ben-Yehudah, Banai, & Ahissar, 2002; Olulade, Napoliello, &
Eden, 2013). Specifically, in one functional magnetic resonance imaging (fMRI) study,
Olulade and colleagues (2013) found reduced neural activation in the V5/MT brain re-
gions that support magnocellular visual processing in children with dyslexia, compared
to children without dyslexia; however, no differences in neural activation in V5/MT
were observed between children with dyslexia and younger children matched for
reading ability. Additionally, when children with dyslexia took part in a phonologically
based reading intervention, activation in the V5/MT region also increased, suggesting
a causal relationship between reading experience and magnocellular visual processing
(Olulade et al., 2013).
Impaired visual attention (the amount of information that can be maintained with
interrupted input, such as blink or saccades) and impaired visuospatial attention (the
ability to orient spatial attention) can also limit reading ability (Bosse et al., 2007;
Facoetti et al., 2003). Indeed, individuals with dyslexia show longer (i.e., slower) re-
sponse times to visual and auditory stimuli in cued-detection task (whereby a cue
directs attention toward a response), indicating slower attentional processing (Facoetti
et al., 2010). Further, differences in the time course of attentional processing accounts for
variance in children’s nonword reading performance above and beyond the contribu-
tion of phonological ability (Facoetti et al., 2010); note, however, that children with dys-
lexia in this study also had poor pre-reading phonological awareness. Children with
dyslexia also showed gains in spatial attention, phonological recoding, and reading
speed after receiving active video-game training intervention (Franceschini et al., 2013).
One notable difficulty with studies of visual attention in dyslexia is that their designs
limit the ability to infer causal relations (e.g., to date there are no studies including chil-
dren with dyslexia as well as younger reading-matched controls); as such, like deficits
in magnocellular visual processing, the deficits observed in visual attention may be the
product of reduced reading experience, which could train up visual attention, rather
than a causal mechanism of dyslexia (Facoetti et al., 2003; 2006).
In addition to visual processing and visual/visuospatial attention-deficit hypotheses,
impaired auditory processing hypotheses postulate a causal connection between audi-
tory dysfunction and phonological deficits, frequently observed in individuals with
dyslexia. Thus, the inability to perceptually organize the auditory input that makes up
speech can manifest in impaired phonological processing ability. The rapid auditory
processing (RAP) deficit hypothesis proposes that a specific deficit in the ability to pro-
cess rapidly and sequentially presented auditory information causes dyslexia (Tallal,
2004). The premise of the hypothesis lies in the fact that spectral changes that are rele-
vant for speech (e.g., frequency variation in transitions from consonants to vowels) are
quite rapid (around 40 ms). Children with language impairment show poorer ability to
process rapid auditory information; this deficit was suggested to also account for the
phonological deficits observed in children with dyslexia (Tallal, 1980, 2004). However,
empirical evidence with respect to this hypothesis has been mixed: only some children
with dyslexia show this processing difficulty (Beaton & Davies, 2007; Boyer & Ehri, 2011;
Goswami, Gerson, & Astruc, 2010); thus, the deficit might have a more prominent role
in SLI than dyslexia. Moreover, if processing rapid temporal information of speech is
the critical deficit in poor phonological performance in children with dyslexia, then
slowing down the temporal information should improve phonological performance in
this group; however, that is not the case (McAnally, Hansen, Cornelissen, & Stein, 1997).
Another sensory-based theory put forth by Goswami and colleagues (2002), the rise
time theory, instead focuses on slower auditory processing and/or impaired discrimina-
tion of amplitude (or intensity) in speech in children with dyslexia. According to this ac-
count, the deficits observed in dyslexia might stem from impairments in discriminating
rise time, a measure of the rate of change in amplitude at the onset of a speech segment,
which affects the detection of speech rhythms that correspond to prosodic and syllabic
processing rates. Evidence for this theory comes from findings of impaired rise time dis-
crimination at slow temporal rates in children with dyslexia compared to age-matched
typically developing readers and reading-matched younger readers across multiple
languages and orthographies (Goswami, Fosker, Huss, Mead, & Szucs, 2011; Poelmans
et al., 2011). Moreover, children and adults with dyslexia have been shown to exhibit pro-
sodic deficits or reduced prosodic awareness (Goswami, Huss, Mead, Fosker, & Verney,
2013; Kitzen, 2001), and children with dyslexia are poorer than age-matched controls
and younger reading age-matched controls at judging musical rhythm with similar rise
times (Goswami et al., 2013). Finally, at the neural level (see also the following discussion
of the neural basis of dyslexia), children with dyslexia show atypical neural entrainment
(captured by EEG phase alignment) to syllable streams at 2 Hz (Power, Mead, Barnes, &
Goswami, 2013).
Neural Basis of Dyslexia
Neuroimaging studies of dyslexia have been instrumental in informing our knowledge

of the biological bases of dyslexia. What has emerged from findings using fMRI (see
Heim & Specht, Chapter 4 in this volume) and positron emission tomography (PET) is
that children and adults with dyslexia have atypical functional brain activation (during
reading and reading related tasks) in a network of left hemisphere language regions
(Landi et al., 2010; Maisog, Einbinder, Flowers, Turkeltaub, & Eden, 2008; Paulesu et al.,
2001; Price & Devlin, 2011; Pugh et al., 2001; Richlan, Kronbichler, & Wimmer, 2009,
2012; Rumsey et al., 1997; Salmelin, Service, Kiesila, Uutela, & Salonen, 1996; Shaywitz
et al., 2002; Temple et al., 2003). This atypical activation most commonly manifests as
hypoactivation in several regions, including left temporo-parietal (superior temporal
gyrus [STG], angular gyrus [AG], inferior parietal lobe [IPL]), occipito-temporal gyrus
(OT), and inferior frontal gyrus (IFG), during reading. In some studies, individuals
with dyslexia also showed compensatory right hemisphere activity and/or increased
left IFG or other frontal lobe activation; however, some evidence suggests that these
differences may be age-dependent, and thus a byproduct of reading skill (Sarkari et al.,
2002; Shaywitz et al., 1998; Shaywitz et al., 2000; Shaywitz et al., 2002; Simos et al., 2002).
In addition to reduced activation, several reports indicate reduced functional connec-
tivity between a number of the aforementioned left hemisphere regions during reading
tasks (e.g., Hampson, Olson, Leung, Skudlarski, & Gore, 2004; Pugh et al., 2000).
Recent findings further suggest that the degree to which individuals use the same left
hemisphere regions (including STG, IFG, and inferior parietal cortex) for both print
and speech processing is associated with reading skill, and is predictive of later reading
success in young readers (Frost et al., 2009; Preston et al., 2016). Although this research
did not include a large number of clinically defined readers with dyslexia, continuous
approaches to examining brain-behavior relations have successfully replicated group
comparisons (e.g., Pugh et al., 2013; 2014). Other recent findings highlight a potential
role for subcortical structures, including the basal ganglia and thalamus, in reading and
reading disorder, suggesting a possible link to some of the observed procedural learning
deficits observed in dyslexia (Preston et al., 2010; Pugh et al., 2013).
In addition to these functional neuroimaging differences, structural differences have
also been observed in the form of smaller gray matter volume in a number of the left
hemisphere regions that show functional differences in individuals with dyslexia (e.g.,
Brambati et al., 2004; Kronbichler et al., 2008). Further, atypical white matter integrity
has also been observed in tracts that connect these same language and reading critical
regions (Boets et al., 2013; Niogi & McCandliss 2006; Vandermosten, Boets, Wouters, &
Ghesquière, 2012).
In order to disentangle cause and consequence of dyslexia, researchers have begun
to explore neural circuitry in pre-reading children and infants at increased familial risk
for dyslexia. Taking this approach, Raschle, Chang, and Gaab (2011) found reduced
gray matter volume (GMV) in temporo-parietal and occipito-temporal regions in at-
risk pre-reading children relative to controls who were not at risk; further, GMV was
correlated with RAN in these regions, and phonological processing was correlated
with a region in the left occipito-temporal cortex (Raschle et al., 2011). At the level of
function, at-risk, pre-reading children showed hypoactivation during a phonological-
awareness task in key left hemisphere reading-associated regions, including the left lin-
gual gyrus, left STG/MTG, and this activation also correlated with performance in a
PA task (Raschle, Zuk, & Gaab, 2012). Further, at-risk pre-reading children also showed
positive correlations between their phonological awareness ability and both the volume
and fractional anisotropy (FA) of the left arcuate fasciculus, a prominent white matter
fiber tract (Saygin et al., 2013; see Catani & Forkel, Chapter 9 in this volume).
Recent evidence from electroencephalography (EEG) and event-related potential
(ERP) research (see Leckey & Federmeier, Chapter 3 in this volume) has also contributed
to the neurobiological profile of dyslexia. Differences in the MMN component to speech
tokens have been found between infants with and without family risk who go on to have
dyslexia (Leppanen et al., 2012; Molfese, 2000; Van Zuijen, Plakas, Maassen, Maurits, &
van der Leij, 2013), and between young children who have dyslexia and their siblings who
are at heightened genetic risk but do not have dyslexia (Neuhoff et al., 2012). Further,
there is evidence that, unlike controls, individuals with dyslexia do not show inter-trial
delta phase (2 Hz) coherence (the temporal rate with which stressed syllables occur in
speech), and that individuals with dyslexia show reduced expectancy-based response
(contingent negative variation, CNV) to these rhythms (Hamalainen, Rupp, Soltesz,
Szucs, & Goswami, 2012; Soltész, Szűcs, Leong, White, & Goswami, 2013). Individuals
with dyslexia, unlike controls, also do not show left dominant auditory entrainment
to phoneme level (25–35 Hz) modulation in auditory cortex, and present with bilat-
eral abnormal responses at a higher rate (>50 Hz), which could indicate that they have
finer grained phonemic representations than controls (Lehongre, Ramus, Villiermet,
Schwartz, & Giraud, 2011); that is, as discussed earlier in the chapter, individuals with
dyslexia may represent subphonemes, which could present as an impairment in phono-
logical awareness and memory if they are holding on to and manipulating more infor-
mation than required (e.g., Dufor, Serniclaes, Sprenger-Charolles, & Demonet, 2009).
In contrast to these auditory EEG and ERP findings, recent fMRI findings suggest
the possibility that phonological representations in individuals with dyslexia in the
temporal cortex are in fact intact, but that access to them may be impaired (Boets,
2014; Boets et al., 2013). Specifically, Boets and colleagues (2013) examined patterns
of neural activation between adults with and without dyslexia while discriminating
vowels and stop-consonants. Both dyslexic and nondyslexic adult readers showed dis-
tinct neural activation patterns for all speech sounds that differentially rely on spec-
tral and spectrotemporal acoustic feature processing, suggesting robust phonetic
representations. However, the functional and structural connectivity between au-
ditory cortices and the left IFG was reduced in individuals with dyslexia, but not in
individuals without dyslexia, suggesting that poor access to otherwise intact phonolog-
ical representations is the core feature of the phonological deficit in dyslexia. Additional
support for this hypothesis comes from several studies mentioned previously that have
documented abnormalities in the arcuate fasciculus in older individuals with dyslexia
and in pre-reading children with low PA (Saygin et al., 2013; Vandermosten, Wouters, &
Ghesquière, 2014).
Genetic and Imaging Genetic

Approaches
Twin and family studies have established the high heritability of reading ability and
reading problems; depending on variables such as socioeconomic status (SES), age,
gender, and IQ, genetic effects account for 40%–80% of the variance in different reading
and reading-related skills (Byrne et al., 2009; Grigorenko, 2004; Taylor, Roehrig, Soden
Hensler, Connor, & Schatschneider, 2010). With respect to molecular genetic studies of
dyslexia, nine candidate dyslexia genetic loci, which have been labeled DYX1–DYX9,
have been proposed. Fine-mapping of these regions has identified several candidate
genes that play a role in neuronal migration or axon guidance during development, in-
cluding DYX1C1 (Taipale et al., 2003), DCDC2 (Meng et al., 2005), KIAA0319 (Cope et al.,
2005), and ROBO1 (Hannula-Jouppi et al., 2005). In addition to their association with
dyslexia, these genes have been associated with behaviors and endophenotypes related
to dyslexia; for example, DYX1C1 has been associated with deficits in memory, phono-
logical decoding, sight word reading, orthographic decoding, RAN, and spelling (Bates
et al., 2010; Dahdouh et al., 2009; Marino et al., 2007; Taipale et al., 2003); KIAA0319
and DCDC2 have been associated with phonological awareness, spelling, phonemic
decoding, and sight word reading (Eicher et al., 2015; Francks et al., 2004; Kaplan et al.,
2002.; Meng et al., 2005; Scerri et al., 2011; Turic et al., 2003); and, ROBO1 has been as-
sociated with phonological memory, word reading, and spelling (Bates et al., 2011; Tran
et al., 2014).
Given that complex neurodevelopmental disorders that are heterogeneous with com-
pound phenotypes are likely to be regulated by multiple genes and gene-gene as well as
gene-environment interactions, recent studies have taken a network or path approach to
studying dyslexia and other complex disorders (e.g., Biernacka, Jenkins, Wang, Moyer,
& Fridley, 2012). These approaches move away from focusing on the primary associa-
tion between phenotype and single nucleotide polymorphisms (SNPs) individually to
include path-level analyses and/or analyses that specifically look for gene-gene or gene-
environment interactions based on hypothesized relationships (Winham & Biernacka,
2013). One study that has utilized this approach to study dyslexia risk found that patterns
of combined genotypes across multiple SNPs on KIAA0319 (i.e., haplotypes) conferred
specific levels of risk for dyslexia (Shao et al., 2016). A second example comes from a
recent study that found gene-gene interactions between multiple markers in DYX1C1,
KIAA0319, and GRIN2B—genes associated with worse performance on memory for
letters in a sample of families with dyslexia (Mascheretti, Bureau, Trezzi, Giorda, &
Marino, 2015).
Another approach that yields promise for understanding the mechanisms by which
genes lead to behavioral change includes neuro-endophenotypes measured with MRI or
EEG/ERP or MEG (see Salmelin, Kujala, & Liljeström, Chapter 6 in this volume). This
approach builds on prior knowledge from early behavioral neuroscience approaches
that identified abnormal neuronal migration and ectopias in the temporal lobes of
rodents whose genome had been modified to knock out (or knock down) a dyslexia
candidate gene; similar abnormalities in neuronal migration were also identified in post
mortem studies of several individuals with a history of dyslexia (Galaburda Sherman,
Rosen, Aboitiz, & Geschwind, 1985). Findings such as these have begun to reveal a po-
tential mechanistic link between gene and behavior. For example, one proposal, put
forth by Giraud and Ramus (2013), posited that abnormal neuronal migration patterns
in auditory cortex may alter oscillatory activity in the delta, theta, and low gamma
ranges, which correspond to the time scales associated with processing of syllabic stress
and may thus play an important role in phonological segmentation, manipulation, and
awareness. Two ERP studies of speech processing provide neurogenetic evidence for
impaired phonetic processing. Czamara and colleagues (2011) found that one SNP on
region on DCDC2 and two SNPs in regions between DCDC2 and KIAA0319 were as-
sociated with MMN amplitude (effect size) in a population of German children with
dyslexia (Czamara et al., 2011). Further, another SNP in SLC2A3 (the predominant facil-
itative glucose transporter not previously associated with dyslexia) was also found to be
associated with MMN amplitude in children with dyslexia (Roeske et al., 2011).
A few examples of gene-brain-behavior approaches come from studies of so called
generalist genes (i.e., genes proposed to impact a multitude of cognitive systems)
(Haworth & Plomin, 2010) in our lab. The first example focused on a well-studied SNP
on the COMT gene, the Val158Met polymorphism at rs4680, which has been associated
with multiple aspects of cognitive function, including auditory function, memory, and
attention (e.g., Chen et al., 2004; Egan et al., 2001; Lebedeva et al., 2009). Our studies,
which included measuring fMRI activation during reading, and multiple assessments
of reading and language skills, found a direct relationship between the COMT genotype
at this locus, patterns of activity in the brain during reading, and PA skill. Specifically,
children with at least one Met allele (Met carriers) outperformed homozygous Val al-
lele carriers on assessments of PA and showed patterns of activation in brain that have
previously been associated with better readers (Landi et al., 2013). Follow-up haplotype
analyses further confirmed the relationship between COMT and PA by identifying
haplotypes across two and three SNP combinations of SNPs near rs4680 within the same
gene, which were associated with additional reductions in PA, beyond those observed
for Val carriers at rs4680 alone (Landi, 2015).
We have also examined the BDNF gene, which has an established role in brain matu-
ration and plasticity, specifically, the Val66Met polymorphism in the BDNF gene, which
affects neural events related to brain plasticity (Cotman & Berchtold, 2002), particu-
larly long-term potentiation (LTP) in the hippocampus. Moreover, BDNF gene 13q21
susceptibility alleles have been associated with language impairment (Simmons et al.,
2010). We compared reading behavior and functional neural activation in children who
were either homozygous carriers of the Val allele (Val/Val; higher levels of BDNF pro-
tein) or carriers of the Met allele (Val/Met or Met/Mat; lower levels of BDNF protein).
Children who were Met carriers showed poorer behavioral performance on measures
of phonological memory and reading comprehension. Further, these children showed
increased memory-associated processing (in the hippocampus and parahippocampal
cortex) during reading, suggesting greater effort in retrieval of grapheme phoneme
correspondences (Jasińska et al., 2016). Taken together, these two studies of so-called
generalist genes provide additional evidence for considering a multiple deficit approach
that considers the contribution of many neurobiological systems to dyslexia.
Advances in whole exome and whole genome sequencing, in combination with
increasing use of network-based approaches to study gene-gene and gene-environment
interactions as well as gene-brain–behavior relations are expected to dramatically im-
prove our understanding of causal mechanisms that link genes to brain and behavior
across development of this complex disorder. Additionally, whole-systems approaches
that include measurement of neurometabolites, in conjunction with traditional imaging
approaches and behavioral phenotyping, may help bridge the gap between the levels
of gene-brain and behavior. For example, a recent paper from our group reported ele-
vated levels of glutamate and choline, measured with magnetic resonance spectroscopy
(MRS) in children with dyslexia (Pugh et al., 2014), providing a potential biomarker for
the observed variability and hyperexcitability observed in dyslexic systems. This type
of finding can generate new hypotheses for exploring previously unexplored candidate
genes and gene-brain-behavior relations.
Conclusions
The current chapter presents an increasingly accepted view of dyslexia as a heritable,

neurobiologically based deficit that is heterogeneous and thus unlikely to be easily
reduced to one causal mechanism at the level of gene, brain, or behavior. The majority
of individuals with dyslexia have deficits in skills that rely on phonological processing,
though the underlying basis of those deficits remains debated. Indeed, some studies
have identified deficits at the level of representation (e.g., abnormal categorical percep-
tion; Godfrey et al., 1981), and others have found only higher-level deficits (phonological
awareness, memory: Ramus & Szenkovits, 2008). Further, some studies have suggested
representation-level deficits in processing of amplitude rise times that may manifest as
auditory temporal-processing deficits, and give rise to impaired syllable-level processing
(Goswami, 2015). Still other lines of research have identified deficits in visual processing
or visual attention (Facoetti et al., 2010; Stein & Walsh, 1997), though existing evidence
suggests that these deficits are most likely due to reading experience. Moreover, children
with multiple deficits (e.g., deficits in both phonological awareness and rapid naming/
processing speed), appear to be more severely impaired, and recent neuroimaging work
suggests unique neural dysfunction patterns for these two impairments (Norton, Beach,
& Gabrieli, 2015).
Neuroimaging studies have identified consistent functional and structural anomaly
in left hemisphere brain systems for reading and language in individuals with dys-
lexia in both the dorsal and ventral language-processing streams (Price & Devlin, 2011;
Pugh et al., 2000; Sandak et al., 2004). Further, studies of pre-readers at risk for dyslexia
who go on to develop reading problems show similar anomalies in both structure and
function (Raschle et al., 2011; Raschle et al., 2012) The most robustly observed neural
anomaly at the level of function is hypoactivation in left hemisphere regions, including
the fusiform gyrus/occipitotemporal junction, and perisylvian regions during reading
(Landi et al., 2010; Pugh et al., 2000). At the level of structure, reduced gray matter
volumes in these regions has been routinely observed (Kronbichler et al., 2008). Recent
neuroimaging data also strongly implicate abnormal integrity in the arcuate fasciculus
in dyslexia (Raschle et al., 2011; Vandermosten et al., 2012), which, combined with intact
representation-level phonological processing in left temporal cortex, suggests that the
phonological deficits observed in dyslexia may be about access rather than representa-
tion (Boets, 2014).
Evidence from behavioral genetic research indicates high levels of heritability for
dyslexia, and molecular genetic studies of humans and animals have identified several
candidate genes that likely contribute to the biological and behavioral anomalies asso-
ciated with dyslexia (Grigorenko, 2004). New approaches in molecular genetic analyses
are further refining our understanding of the role of particular gene, gene-gene, and
gene-environment interactions in dyslexia (Biernacka et al., 2012). And neurogenetic
approaches and imaging of neurometabolites have been helpful for identifying how genes
may alter neural circuits that result in different dyslexia phenotypes, and in identifying
possible new candidate pathways (Landi et al., 2013; Pugh et al., 2014; Roeske et al., 2011).
References
Alloway, T. P., Gathercole, S. E., Willis, C., & Adams, A. M. (2004). A structural analysis of
working memory and related cognitive skills in young children. Journal of Experimental
Child Psychology, 87(2), 85–106.
Amitay, S., Ben-Yehudah, G., Banai, K., & Ahissar, M. (2002). Disabled readers suffer from
visual and auditory impairments but not from a specific magnocellular deficit. Brain,
125(10), 2272–2285.
Arnell, K. M., Joanisse, M. F., Klein, R. M., Busseri, M. A., & Tannock, R. (2009). Decomposing
the relation between Rapid Automatized Naming (RAN) and reading ability. Canadian
Journal of Experimental Psychology/Revue canadienne de psychologie expérimentale, 63(3),
173–184.
Baddeley, A. D. (1987). Working memory. Oxford: Clarendon Press.
Bates, T. C., Lind, P. A., Luciano, M., Montgomery, G. W., Martin, N. G., & Wright, M. J. (2010).
Dyslexia and DYX1C1: Deficits in reading and spelling associated with a missense mutation.
Molecular Psychiatry, 15(12), 1190–1196.
Bates, T. C., Luciano, M., Medland, S. E., Montgomery, G. W., Wright, M. J., & Martin, N.
G. (2011). Genetic variance in a component of the language acquisition device: ROBO1
polymorphisms associated with phonological buffer deficits. Behavior Genetics, 41(1),  50–57.
Beaton, A. A., & Davies, N. W. (2007). Semantic errors in deep dyslexia: Does orthographic
depth matter? Cognitive Neuropsychology, 24(3), 312–323.
Biernacka, J. M., Jenkins, G. D., Wang, L., Moyer, A. M., & Fridley, B. L. (2012). Use of the
gamma method for self-contained gene-set analysis of SNP data. European Journal of
Human Genetics, 20(5) 565–571.
Boets, B. (2014). Dyslexia: Reconciling controversies within an integrative developmental per-
spective. Trends in Cognitive Science, 18(10), 501–503.
Boets, B., Op de Beeck, H. P., Vandermosten, M., Scott, S. K., Gillebert, C. R., Mantini,
D., . . . Ghesquière, P. (2013). Intact but less accessible phonetic representations in adults with
dysFlexia. Science, 342(6163), 1251–1254.
Bogliotti, C., Serniclaes, W., Messaoud- Galusi, S., & Sprenger- Charolles, L. (2008).
Discrimination of speech sounds by children with dyslexia: Comparisons with chronolog-
ical age and reading level controls. Journal of Experimental Child Psychology, 101(2), 137–155.
Bosse, M. L., Tainturier, M. J., & Valdois, S. (2007). Developmental dyslexia: The visual atten-
tion span deficit hypothesis. Cognition, 104(2), 198–230.
Bowers, P. G., & Swanson, L. B. (1991). Naming speed deficits in reading disability: Multiple
measures of a singular process. Journal of Experimental Child Psychology, 51(2), 195–219.
Boyer, N., & Ehri, L. C. (2011). Contribution of phonemic segmentation instruction with let-
ters and articulation pictures to word reading and spelling in beginners. Scientific Studies of
Reading, 15(5), 440–470.
Brambati, S. M., Termine, C., Ruffino, M., Stella, G., Fazio, F., Cappa, S. F., & Perani, D. (2004).
Regional reductions of gray matter volume in familial dyslexia. Neurology, 63(4), 742–745.
Byrne, B., Coventry, W. L., Olson, R. K., Samuelsson, S., Corley, R., Willcutt, E. G., . . . DeFries, J.
C. (2009). Genetic and environmental influences on aspects of literacy and language in early
childhood: Continuity and change from preschool to Grade 2. Journal of Neurolinguistics,
22(3), 219–236.
Chein, J. M., & Schneider, W. (2005). Neuroimaging studies of practice-related change: fMRI
and meta-analytic evidence of a domain-general control network for learning. Cognitive
Brain Research, 25(3), 607–623.
Chen, J., Lipska, B., Halim, N., Quang, D. M., Matsumoto, M., Melhem, S., . . . Weinberger,
D. R. (2004). Functional analysis of genetic variation in catecholomethyltransferase
(COMT): Effects of mRNA, protein, and enzyme activity in postmortem human brain.
American Journal of Human Genetics, 75(5), 807–821.
Cope, N., Harold, D., Hill, G., Moskvina, V., Stevenson, J., Holmans, P., . . . Williams, J. (2005).
Strong evidence that KIAA0319 on chromosome 6p is a susceptibility gene for develop-
mental dyslexia. The American Journal of Human Genetics, 76(4), 581–591.
Cotman, C. W., & Berchtold, N. C. (2002). Exercise: A behavioral intervention to enhance

brain health and plasticity. Trends in Neurosciences, 25(6), 295–301.
Czamara, D., Bruder, J., Becker, J., Bartling, J., Hoffmann, P., Ludwig, K. U., Muller-Myhsok, B.,
& Schulte-Korne, G. (2011). Association of a rare variant with mismatch negativity in a re-
gion between KIAA0319 and DCDC2 in dyslexia. Behavioral Genetics, 41(1), 110–119.
Dahdouh, F., Anthoni, H., Tapia-Páez, I., Peyrard-Janvid, M., Schulte-Körne, G., Warnke,
A., . . . Nöthen, M. M. (2009). Further evidence for DYX1C1 as a susceptibility factor for dys-
lexia. Psychiatric Genetics, 19(2),  59–63.
Denckla, M. B., & Rudel, R. (1974). Rapid “automatized” naming of pictured objects, colors, let-
ters and numbers by normal children. Cortex, 10(2), 186–202.
Dufor, O., Serniclaes, W., Sprenger-Charolles, L., & Demonet, J.-F. (2009). Left premotor cortex
and allophonic speech perception in dyslexia: A PET study. NeuroImage, 46(1), 241–248.
Egan, M. F., Goldberg, T. E., Kolachana, B. S., Callicott, J. H., Mazzanti, C. M., Straub, R. E.,
Goldman, D., & Weinberger, D. R. (2001). Effect of COMT Val108/158 Met genotype on
frontal lobe function and risk for schizophrenia. Proceedings of the National Academy of
Sciences, 98(12), 6917–6922.
Eicher, J. D., Stein, C. M., Deng, F., Ciesla, A. A., Powers, N. R., Boada, R., . . . Gruen, J. R. (2015).
The DYX2 locus and neurochemical signaling genes contribute to speech sound disorder
and related neurocognitive domains. Genes, Brain and Behavior, 14(4), 377–385.
Elbro, C., Nielsen, I., & Petersen, D. K. (1994). Dyslexia in adults: Evidence for deficits in non-
word reading and in the phonological representation of lexical items. Annals of Dyslexia,
44(1), 203–226.
Facoetti, A., Lorusso, M. L., Paganoni, P., Cattaneo, C., Galli, R., & Mascetti, G. G. (2003). The
time course of attentional focusing in dyslexic and normally reading children. Brain and
Cognition, 53(2), 181–184.
Facoetti, A., Trussardi, A. N., Ruffino, M., Lorusso, M. L., Cattaneo, C., Galli, R., . . . Zorzi, M.
(2010). Multisensory spatial attention deficits are predictive of phonological decoding skills
in developmental dyslexia. Journal of Cognitive Neuroscience, 22(5), 1011–1025.
Facoetti, A., Zorzi, M., Cestnick, L., Lorusso, M. L., Molteni, M., Paganoni, P., . . . Mascetti, G.
G. (2006). The relationship between visuo-spatial attention and nonword reading in devel-
opmental dyslexia. Cognitive Neuropsychology, 23(6), 841–855.
Foy, J. G., & Mann, V. (2006). Changes in letter sound knowledge are associated with development
of phonological awareness in pre-school children. Journal of Research in Reading, 29(2), 143–161.
Franceschini, S., Gori, S., Ruffino, M., Viola, S., Molteni, M., & Facoetti, A. (2013). Action video
games make dyslexic children read better. Current Biology, 23(6), 462–466.
Francks, C., Paracchini, S., Smith, S. D., Richardson, A. J., Scerri, T. S., Cardon, L. R., . . . Fisher,
S. E. (2004). A 77-kilobase region of chromosome 6p22. 2 is associated with dyslexia in
families from the United Kingdom and from the United States. The American Journal of
Human Genetics, 75(6), 1046–1058.
Frost, R. (1998). Toward a strong phonological theory of visual word recognition: True issues
and false trails. Psychological Bulletin, 123(1),  71–99.
Frost, S. J., Landi, N., Mencl, W. E., Sandak, R., Fulbright, R. K., Tejada, E. T., . . . Pugh, K. R.
(2009). Phonological awareness predicts activation patterns for print and speech. Annals of
Dyslexia, 59(1),  78–97.
Galaburda, A. M., Sherman, G. F., Rosen, G. D., Aboitiz, F., & Geschwind, N. (1985).
Developmental dyslexia: Four consecutive patients with cortical anomalies. Annals of
Neurology, 18(2), 222–233.
Gathercole, S. E., Tiffany, C., Briscoe, J., & Thorn, A. (2005). Developmental consequences of
poor phonological short-term memory function in childhood: A longitudinal study. Journal
of Child Psychology and Psychiatry, 46(6), 598–611.
Giraud, A. L., & Ramus F. (2013). Neurogenetics and auditory processing in developmental
dyslexia. Current Opinion in Neurobiology, 23, 37–42. doi: 10.1016/j.conb.2012.09.003
Godfrey, J. J., Syrdal-Lasky, K., Millay, K. K., & Knox, C. M. (1981). Performance of dyslexic chil-
dren on speech perception tests. Journal of Experimental Child Psychology, 32(3), 401–424.
Goswami, U. (2015). Sensory theories of developmental dyslexia: Three challenges for research.
Nature Reviews Neuroscience, 16(1),  43–54.
Goswami, U., & Bryant, P. (1990). Phonological skills and learning to read (p. 1). London:
Lawrence Erlbaum.
Goswami, U., Fosker, T., Huss, M., Mead, N., & Szucs, D. (2011). Rise time and formant tran-
sition duration in the discrimination of speech sounds: The Ba–Wa distinction in develop-
mental dyslexia. Developmental Science, 14(1),  34–43.
Goswami, U., Gerson, D., & Astruc, L. (2010). Amplitude envelope perception, phonology and
prosodic sensitivity in children with developmental dyslexia. Reading and Writing, 23(8),
995–1019.
Goswami, U., Huss, M., Mead, N., Fosker, T., & Verney, J. P. (2013). Perception of patterns of
musical beat distribution in phonological developmental dyslexia: Significant longitudinal
relations with word reading and reading comprehension. Cortex, 49(5), 1363–1376.
Goswami, U., Thomson, J., Richardson, U., Stainthorp, R., Hughes, D., Rosen, S., & Scott, S.
K. (2002). Amplitude envelope onsets and developmental dyslexia: A new hypothesis.
Proceedings of the National Academy of Sciences, 99(16), 10911–10916.
Grigorenko, E. L. (2004). Genetic bases of developmental dyslexia: A capsule review of herita-
bility estimates. Enfance, 56(3), 273–288.
Hamalainen, J. A., Rupp, A., Soltesz, F., Szucs, D., & Goswami, U. (2012). Reduced phase
locking to slow amplitude modulation in adults with dyslexia: An MEG study. NeuroImage,
59(3), 2952–2961.
Hampson, M., Olson, I. R., Leung, H. C., Skudlarski, P., & Gore, J. C. (2004). Changes in
functional connectivity of human MT/V5 with visual motion input. NeuroReport, 15(8),
1315–1319.
Hannula- Jouppi, K., Kaminen- Ahola, N., Taipale, M., Eklund, R., Nopola- Hemmi, J.,
Kääriäinen, H., & Kere, J. (2005). The axon guidance receptor gene ROBO1 is a candidate
gene for developmental dyslexia. PLoS Genetics, 1(4), e50.
Haworth, C. M., & Plomin, R. (2010). Quantitative genetics in the era of molecular ge-
netics: Learning abilities and disabilities as an example. Journal of the American Academy of
Child & Adolescent Psychiatry, 49(8), 783–793.
Hulme, C. (2002). Phonemes, rimes, and the mechanisms of early reading development.
Journal of Experimental Child Psychology, 82(1),  58–64.
Hulme, C., Caravolas, M., Málkova, G., & Brigstocke, S. (2005). Phoneme isolation ability is
not simply a consequence of letter-sound knowledge. Cognition, 97(1), B1–B11.
Jasińska, K., Kornilov, S., Molfese, P., Frost, S., Lee, M., Mencl, W. E., Pugh, K. R., Grigorenko,
E., & Landi, N. (2016). The BDNF Val66Met polymorphism influences reading ability and
patterns of neural activation in children. PLoS One, 11(8), e0157449.
Joanisse, M. F., Manis, F. R., Keating, P., & Seidenberg, M. S. (2000). Language deficits in dys-
lexic children: Speech perception, phonology, and morphology. Journal of experimental
child psychology, 77(1),  30–60.
Kaplan, D. E., Gayan, J., Ahn, J., Won, T. W., Pauls, D., Olson, R. K., . . . Smith, S. D. (2002).
Evidence for linkage and association with reading disability, on 6p21. 3–22. The American
Journal of Human Genetics, 70(5), 1287–1298.
Katz, R. B. (1986). Phonological deficiencies in children with reading disability: Evidence from
an object-naming task. Cognition, 22(3), 225–257.
Katzir, T., Kim, Y. S., Wolf, M., Morris, R., & Lovett, M. W. (2008). The varieties of pathways
to dysfluent reading: Comparing subtypes of children with dyslexia at letter, word, and
connected text levels of reading. Journal of Learning Disabilities, 41(1),  47–66.
Kitzen, K. R. (2001). Prosodic sensitivity, morphological ability, and reading ability in young
adults with and without childhood histories of reading difficulty. Dissertation Abstracts
International Section A: Humanities and Social Sciences, 62(2-A),  460.
Kovelman, I., Norton, E. S., Christodoulou, J. A., Gaab, N., Lieberman, D. A., Triantafyllou,
C., . . . Gabrieli, J. D. E. (2012). Brain basis of phonological awareness for spoken language in
children and its disruption in dyslexia. Cerebral Cortex, 22(4), 754–764.
Kronbichler, M., Wimmer, H., Staffen W., Hutzler, F., Mair, A. & Ladurner, G. (2008).
Developmental dyslexia: Gray matter abnormalities in the occipitotemporal cortex. Human
Brain Mapping, 29(5), 613–625.
Landi, N. (2015, November). Variation in BDNF and COMT is associated with neural struc-
ture and function in and children’s language and reading skills. Talk presented at the Boston
University Communication Disorders Colloquium, Boston, MA.
Landi N., Frost, S. J., Mencl, W. E., Preston, J. L., Jacobsen, L. K., Lee, M., . . . Grigorenko, E. L.
(2013). The COMT Val/Met polymorphism is associated with reading‐related skills and con-
sistent patterns of functional neural activation. Developmental Science, 16(1),  13–23.
Landi, N., Mencl, W. E., Frost, S. J., Sandak, R., Chen, H., & Pugh, K. R. (2010). An fMRI study
of multimodal semantic and phonological processing in reading disabled adolescents.
Annals of Dyslexia, 60(1), 102–121.
Lebedeva, L. S., Korovaitseva, G. I., Lezheiko, T. V., Kaleda, V. G., Abramova, L. I., Barkhatova,
A. N., & Golimbet, V. E. (2009). Influence of genetic variants modulating dopamine ac-
tivity on brain processing of auditory information (the P300 paradigm). Human Physiology,
35(1),  21–24.
Lehongre, K., Ramus, F., Villiermet, N., Schwartz, D., & Giraud, A. L. (2011). Altered low-
gamma sampling in auditory cortex accounts for the three main facets of dyslexia. Neuron,
72(6), 1080–1090.
Leppanen, P. H., Hamalainen, J. A., Guttorm, T. K., Eklund, K. M., Salminen, H.,
Tanskanen, . . . Lyytinen, H. (2012). Infant brain responses associated with reading-related
skills before school and at school age. Clinical Neurophysiology, 42(1),  35–41.
Lyon, G. R., Shaywitz, S. E., & Shaywitz, B. A. (2003). A definition of dyslexia. Annals of
Dyslexia, 53,  1–14.
Maisog, J. M., Einbinder, E. R., Flowers, D. L., Turkeltaub, P. E., & Eden, G. F. (2008), A meta-
analysis of functional neuroimaging studies of dyslexia. Annals of the New York Academy of
Sciences, 1145(1), 237–259.
Marino, C., Citterio, A., Giorda, R., Facoetti, A., Menozzi, G., Vanzin, L., . . . Molteni, M.
(2007). Association of short-term memory with a variant within DYX1C1 in developmental
dyslexia. Genes, Brain and Behavior, 6(7), 640–646.
Mascheretti, S., Bureau, A., Trezzi, V., Giorda, R., & Marino, C. (2015). An assessment of gene-
by-gene interactions as a tool to unfold missing heritability in dyslexia. Human Genetics,
134(7), 749–760.
McAnally, K. I., Hansen, P. C., Cornelissen, P. L., & Stein, J. F. (1997). Effect of time and fre-
quency manipulation on syllable perception in developmental dyslexics. Journal of Speech,
Language, and Hearing Research, 40(4), 912–924.
McCallum, R. S., Bell, S. M., Wood, M. S., Below, J. L., Choate, S. M., & McCane, S. J. (2006). What
is the role of working memory in reading relative to the big three processing variables (orthog-
raphy, phonology, and rapid naming)? Journal of Psychoeducational Assessment, 24(3), 243–259.
McCardle, P., Scarborough, H. S., & Catts, H. W. (2001). Predicting, explaining, and preventing
children’s reading difficulties. Learning Disabilities Research & Practice, 16(4), 230–239.
Meng, H., Smith, S. D., Hager, K., Held, M., Liu, J., Olsen, R. K., . . . Somlo, S. (2005). DCDC2
is associated with reading disability and modulates neuronal development in the brain.
Messaoud-Galusi, S., Hazan, V., & Rosen, S. (2011). Investigating speech perception in chil-
dren with dyslexia: Is there evidence of a consistent deficit in individuals? Journal of Speech,
Language, and Hearing Research, 54(6), 1682–1701.
Molfese, D. L. (2000). Predicting dyslexia at 8 years of age using neonatal brain responses.
Brain and Language, 72(3), 238–245.
Neuhoff, N., Bruder, J., Bartling, J., Warnke, A., Remschmidt, H., Müller-Myhsok, B., Schulte-
Korne, G. (2012). Evidence for the late MMN as a neurophysiological endophenotype for
dyslexia. PLoS One, 7(5), e34909.
Niogi, S. N., & McCandliss, B. D. (2006). Left lateralized white matter microstructure ac-
counts for individual differences in reading ability and disability. Neuropsychologia, 44(11),
2178–2188.
Noordenbos, M. W., Segers, E., Serniclaes, W., Mitterer, H., & Verhoeven, L. (2012a).
Allophonic mode of speech perception in Dutch children at risk for dyslexia: A longitudinal
study. Research in Developmental Disabilities, 33(5), 1469–1483.
Noordenbos, M. W., Segers, E., Serniclaes, W., Mitterer, H., & Verhoeven, L. (2012b). Neural
evidence of allophonic perception in children at risk for dyslexia. Neuropsychologia, 50(8),
2010–2017.
Norton, E. S., Beach, S. D., & Gabrieli, J. D. (2015). Neurobiology of dyslexia. Current Opinion
in Neurobiology, 30,  73–78.
Olulade, O. A., Napoliello, E. M., & Eden, G. F. (2013). Abnormal visual motion processing is
not a cause of dyslexia. Neuron, 79(1), 180–190.
Paulesu, E., Démonet, J. F., Fazio, F., McCrory, E., Chanoine, V., Brunswick, N., . . . Frith, U.
(2001). Dyslexia: Cultural diversity and biological unity. Science, 291(5511), 2165–2167.
Penke, M., & Schrader, K. (2008). The role of phonology in visual word recognition: Reading
acquisition vs. skilled reading. Written Language & Literacy, 11(2), 167–190.
Pennington, B. F. (2006). From single to multiple deficit models of developmental disorders.
Cognition, 101(2), 385–413.
Pennington, B. F., Santerre-Lemmon, L., Rosenberg, J., MacDonald, B., Boada, R., Friend,
A., . . . Olson, R. K. (2012). Individual prediction of dyslexia by single versus multiple deficit
models. Journal of Abnormal Psychology, 121(1), 212–224.
Peterson, R. L., Pennington, B. F., Shriberg, L. D., & Boada, R. (2009). What influences literacy
outcome in children with speech sound disorder? Journal of Speech, Language, and Hearing
Research, 52(5), 1175–1188.
Poelmans, H., Luts, H., Vandermosten, M., Boets, B., Ghesquière, P., & Wouters, J. (2011).
Reduced sensitivity to slow-rate dynamic auditory information in children with dyslexia.
Research in Developmental Disabilities, 32(6), 2810–2819.
Power, A. J., Mead, N., Barnes, L., & Goswami, U. (2013). Neural entrainment to rhythmic
speech in children with developmental dyslexia. Frontiers in Human Neuroscience, 7, 777.
doi: 10.3389/fnhum.2013.00777
Preston, J. L., Frost, S. J., Mencl, W. E., Fulbright, R. K., Landi, N., Grigorenko, E., . . . Pugh, K. R.
(2010). Early and late talkers: School-age language, literacy and neurolinguistic differences.
Brain, 133(8), 2185–2195.
Preston, J. L., Molfese, P. J., Frost, S. J., Mencl, W. E., Fulbright, R. K., Hoeft, F., . . . Pugh, K.
R. (2016). Print-speech convergence predicts future reading outcomes in early readers.
Psychological Science, 27(1),  75–84.
contributions to reading. Trends in Cognitive Sciences, 15(6), 246–253.
Pugh, K. R., Frost, S. J., Rothman, D. L., Mason, G. M., Del Tufo, S., Molfese, P. J., . . . Fulbright,
R. K. (2014). Glutamate and choline levels predict individual differences in reading ability in
emergent readers. Journal of Neuroscience, 34(11), 4082–4089.
Pugh, K. R., Landi, N., Preston, J. L., Mencl, W. E., Austin, A. C., Sibley, D., . . . Frost, S. J. (2013).
The relationship between phonological and auditory processing and brain organization in
beginning readers. Brain and Language, 125(2), 173–183.
Functional neuroimaging studies of reading and reading disability (developmental dys-
lexia). Mental retardation and developmental disabilities research reviews, 6(3), 207–213.
Neurobiological studies of reading and reading disability. Journal of Communication
Disorders, 34(6), 479–492.
Pugh, K., Mencl, W. E.., Shaywitz, B. A., Shaywitz, S. E., Fulbright, R. K., Skudlarski, P.,Constable,
R. T., . . . Gore, J. C. (2000). The angular gyrus in developmental dyslexia: Task-specific
differences in functional connectivity in posterior cortex. Psychological Science, 11(1),  51–56.
Ramus, F., & Szenkovits, G. (2008). What phonological deficit? The Quarterly Journal of
Experimental Psychology, 61(1), 129–141.
Raschle, N., Chang, M., & Gaab, N. (2011) Structural brain alterations associated with dyslexia
predate reading onset. NeuroImage, 57(3), 742–749.
Raschle, N. M., Zuk, J., & Gaab, N. (2012). Functional characteristics of developmental dys-
lexia in left-hemispheric posterior brain regions predate reading onset. Proceedings of the
National Academy of Sciences, 109(6), 2156–2161.
Richlan, F., Kronbichler, M., & Wimmer, H. (2009). Functional abnormalities in the dyslexic
brain: A quantitative meta‐analysis of neuroimaging studies. Human Brain Mapping, 30(10),
3299–3308.
Richlan, F., Kronbichler, M., & Wimmer, H. (2012). Structural abnormalities in the dyslexic
brain: A meta-analysis of voxel-based morphometry studies. Human Brain Mapping, 34(11),
3055–3065.
Roeske, D., Ludwig, K. U., Neuhoff, N., Becker, J., Bartling, J., Bruder, J., . . . Müller-Myhsok, B.
(2011). First genome-wide association scan on neurophysiological endophenotypes points
to trans-regulation effects on SLC2A3 in dyslexic children. Molecular Psychiatry, 16(1),

97–107.
Rumsey, J. M., Nace, K., Donohue, B., Wise, D., Maisog, J. M., & Andreason, P. (1997). A pos-
itron emission tomographic study of impaired word recognition and phonological pro-
cessing in dyslexic men. Archives of Neurology, 54(5), 562–573.
Salmelin, R., Service, E., Kiesila, P., Uutela, K., & Salonen, O. (1996). Impaired visual word
processing in dyslexia revealed with magnetoencephalography. Annals of Neurology, 40(2),
157–162.
Sandak, R., Mencl, W. E., Frost, S., & Pugh, K. R. (2004). The neurobiological basis of skilled
and impaired reading: Recent findings and new directions. Scientific Studies of Reading, 8(3),
273–292.
Sarkari, S., Simos, P. G., Fletcher, J. M., Castillo, E. M., Breier, J. I., & Papanicolaou, A. C. (2002).
The emergence and treatment of developmental reading disability: Contributions of func-
tional brain imaging. Seminars in Pediatric Neurology, 9(3), 227–236.
Saygin, Z. M., Norton, E. S., Osher, D. E., Beach, S. D., Cyr, A. B., Ozernov-Palchik,
O., . . . Gabrieli, J. D. E (2013). Tracking the roots of reading ability: White matter volume and
integrity correlate with phonological awareness in pre-reading and early-reading kinder-
garten children. Journal of Neuroscience, 33(33), 13251–13258.
Scarborough, H. S. (1990). Very early language deficits in dyslexic children. Child Development,
61(6), 1728–1743.
Scerri, T. S., Morris, A. P., Buckingham, L. L., Newbury, D. F., Miller, L. L., Monaco, A.
P., . . . Paracchini, S. (2011). DCDC2, KIAA0319 and CMIP are associated with reading-
related traits. Biological Psychiatry, 70(3), 237–245.
Serniclaes, W., Collet, G., & Sprenger-Charolles, L. (2015). Review of neural rehabilitation
programs for dyslexia: How can an allophonic system be changed into a phonemic one?
Serniclaes, W., Van Heghe, S., Mousty, P., Carre, R., & Sprenger- Charolles, L. (2004).
Allophonic mode of speech perception in dyslexia. Journal of Experimental Child Psychology,
87(4), 336–361.
Shao, S., Kong, R., Zou, L., Zhong, R., Lou, J., Zhou, J., . . . Song, R. (2016). The roles of genes in
the neuronal migration and neurite outgrowth network in developmental dyslexia: Single-
and multiple-risk genetic variants. Molecular Neurobiology, 53(6), 3967–3975.
Share, D. L. (1995). Phonological recoding and self-teaching: Sine qua non of reading acquisi-
tion. Cognition, 55(2), 151–218; discussion 219–126.
Share, D. L., & Gur, T. (1999). How reading begins: A study of preschoolers’ print identification
strategies. Cognition and Instruction, 17(2), 177–213.
Shaywitz, B. A., Pugh, K. R., Jenner, A. R., Fulbright, R. K., Fletcher, J. M., Gore, J. C., &
Shaywitz, S. E. (2000). The neurobiology of reading and reading disability (dyslexia).
Handbook of Reading Research, 3, 229–249.
Shaywitz, B. A., Shaywitz, S. E., Pugh, K. R., Mencl, W. E., Fulbright, R. K., Skudlarski,
P., . . . Gore, J. C. (2002). Disruption of posterior brain systems for reading in children with
developmental dyslexia. Biological Psychiatry, 52(2), 101–110.
Shaywitz, S. E., Shaywitz, B. A., Pugh, K. R., Fulbright, R. K., Constable, R. T., Mencl, W.
E., . . . Katz, L. (1998). Functional disruption in the organization of the brain for reading in
dyslexia. Proceedings of the National Academy of Sciences, 95(5), 2636–2641.
Simmons, T. R., Flax, J. F., Azaro, M. A., Hayter, J. E., Justice, L. M., Petrill, S. A., . . . Bartlett,
C. W. (2010). Increasing genotype-phenotype model determinism: Application to bivariate
reading/language traits and epistatic interactions in language-impaired families. Human

Heredity, 70(4), 232–244.
Simos, P. G., Fletcher, J. M., Bergman, E., Breier, J. I., Foorman, B. R., Castillo, E.
M., . . . Papanicolaou, A. C. (2002). Dyslexia-specific brain activation profile becomes
normal following successful remedial training. Neurology, 58(8), 1203–1213.
Snowling, M. (1987, February). Developmental dyslexia: A cognitive-developmental perspec-
tive. Bulletin of the British Psychological Society, 40, A23.
Snowling, M. J. (2000). Dyslexia. Oxford: Blackwell.
Snowling, M. J. (2008). Specific disorders and broader phenotypes: The case of dyslexia. The
Quarterly Journal of Experimental Psychology, 61(1), 142–156.
Snowling, M. J., Van Wagtendonk, B., & Stafford, C. (1988). Object-naming deficits in develop-
mental dyslexia. Journal of Research in Reading, 11(2),  67–85.
Soltész, F., Szűcs, D., Leong, V., White, S., & Goswami, U. (2013). Differential entrainment of
neuroelectric delta oscillations in developmental dyslexia. PloS One, 8(10), e76608.
Sperling, A. J., Lu, Z. L., Manis, F. R., & Seidenberg, M. S. (2005). Deficits in perceptual noise
exclusion in developmental dyslexia. Nature Neuroscience, 8(7), 862–863.
Stanovich, K. E. (1988). Explaining the differences between the dyslexic and the garden-
variety poor reader: The phonological-core variable-difference model. Journal of Learning
Disabilities, 21(10), 590–604, 612.
Stein, J., & Walsh, V. (1997). To see but not to read: The magnocellular theory of dyslexia. Trends
in Neurosciences, 20(4), 147–152.
Swanson, H. L., & Jerman, O. (2007). The influence of working memory on reading growth in
subgroups of children with reading disabilities. Journal of Experimental Child Psychology,
96(4), 249–283.
Taipale, M., Kaminen, N., Nopola-Hemmi, J., Haltia, T., Myllyluoma, B., Lyytinen, H., . . . Kere,
J. (2003). A candidate gene for developmental dyslexia encodes a nuclear tetratricopeptide
repeat domain protein dynamically regulated in brain. Proceedings of the National Academy
of Sciences, 100(20), 11553–11558.
Tallal, P. (1980). Auditory temporal perception, phonics, and reading disabilities in children.
Tallal, P. (2004). Improving language and literacy is a matter of time. Nature Reviews
Tanaka, H., Black, J. M., Hulme, C., Stanley, L. M., Kesler, S. R., Whitfield-Gabrieli, S., . . . &
Hoeft, F. (2011). The brain basis of the phonological deficit in dyslexia is independent of IQ.
Psychological Science, 22(11), 1442–1451.
Taylor, J., Roehrig, A. D., Hensler, B. S., Connor, C. M., & Schatschneider, C. (2010). Teacher
quality moderates the genetic effects on early reading. Science, 328(5977), 512–514.
Temple, E., Deutsch, G. K., Poldrack, R. A., Miller, S. L., Tallal, P., Merzenich, M. M., & Gabrieli,
J. D. E. (2003). Neural deficits in children with dyslexia ameliorated by behavioral remedia-
tion: Evidence from functional MRI. Proceedings of the National Academy of Sciences, 100(5),
2860–2865.
Tran, C., Wigg, K. G., Zhang, K., Cate-Carter, T. D., Kerr, E., Field, L. L., . . . Barr, C. L. (2014).
Association of the ROBO1 gene with reading disabilities in a family-based analysis. Genes,
Brain and Behavior, 13(4), 430–438.
Turic, D., Robinson, L., Duke, M., Morris, D. W., Webb, V., Hamshere, M., . . . Grierson, A.
(2003). Linkage disequilibrium mapping provides further evidence of a gene for reading dis-
ability on chromosome 6p21.3–22. Molecular Psychiatry, 8(2), 176–185.
Valdois, S., Bidet-Ildei, C., Lassus-Sangosse, D., Reilhac, C., N’Guyen-Morel, M. A., Guinet,
E., & Orliaguet, J. P. (2011). A visual processing but no phonological disorder in a child with
mixed dyslexia. Cortex, 47(10), 1197–1218.
Vandermosten, M., Boets, B., Wouters, J., & Ghesquière, P. (2012). A qualitative and quanti-
tative review of diffusion tensor imaging studies in reading and dyslexia. Neuroscience &
Biobehavioral Reviews, 36(6), 1532–1552.
Vandermosten, M., Wouters, J., & Ghesquière, P. (2014). In search for the neuroanatomical
network underlying phonological aspects of reading: Studies in typical and dyslexic readers
across development (Invited Lecture).
van Bergen, E., van der Leij, A., & de Jong, P. F. (2014). The intergenerational multiple deficit
model and the case of dyslexia. Frontiers in Human Neuroscience, 8, 346.
van Zuijen, T. L., Plakas, A., Maassen, B. A, Maurits, N. M., & van der Leij, A. (2013). Infant
ERPs separate children at risk of dyslexia who become good readers from those who become
poor readers. Developmental Science, 16(4), 554–563.
Vellutino, F. R., Pruzek, R. M., Steger, J. A., & Meshoulam, U. (1973). Immediate visual recall in
poor and normal readers as a function of orthographic-linguistic familiarity. Cortex, 9(4),
370–386.
Waber, D., Wolff, P. H., Forbes, P. W., & Weiler, M. D. (2000). Rapid automatized naming in
children referred for evaluation of heterogeneous learning problems: How specific are
naming speed deficits to reading disability? Child Neuropsychology, 6(4), 251–261.
Wagner, R. K., & Torgesen, J. K. (1987). The nature of phonological processing and its causal
role in the acquisition of reading skills. Psychological Bulletin, 101(2), 192–212.
Winham, S. J., & Biernacka, J. M. (2013). Gene-environment interactions in genome-wide as-
sociation studies: Current approaches and new directions. Journal of Child and Psychology
and Psychiatry, 54(10), 1120–1134.
Wolf, M., & Bowers, P. G. (1999). The double-deficit hypothesis for the developmental
dyslexias. Journal of Educational Psychology, 91(3), 415–438.
Wolf, M., Bowers, P. G., & Biddle, K. (2000). Naming-speed processes, timing, and reading: A
conceptual review. Journal of Learning Disabilities, 33(4), 387–407.
Wolf, M., O’Rourke, A. G., Gidney, C., Lovett, M., Cirino, P., & Morris, R. (2002). The second
deficit: An investigation of the independence of phonological and naming-speed deficits in
developmental dyslexia. Reading and Writing, 15(1),  43–72.
Ziegler, J. C., Perry, C., Ma-Wyatt, A., Ladner, D., & Schulte-Korne, G. (2003). Developmental
dyslexia in different languages: Language-specific or universal? Journal of Experimental
Child Psychology, 86(3), 169–193.
Chapter 26
Speech Perc e p t i on
A Perspective from Lateralization, Motorization,
and Oscillation
David Poeppel, Gregory B. Cogan,

Ido Davidesco, and Adeen Flinker
Definitional Preliminaries
Necessary Distinctions
In research on the neurobiological foundations of speech perception, terminology has
on occasion been used rather loosely, leading to unnecessary, terminology-induced
confusion. Most problematically, the term speech perception has regrettably sometimes
been used interchangeably with language comprehension. We suggest that the expres-
sion language comprehension ought best be reserved for the sequence of processing
sub-routines that occur subsequent to the initial perceptual analyses of some linguistic
input. In particular, language comprehension can be subserved by ear, eye, or touch.
That is to say, the linguistic system underlying comprehension can be engaged by audi-
tory input (speech), visual input (text or sign), and tactile input (Braille). The processes
that form the basis for language comprehension proper therefore build on sensory-input
processes that appear to be, at least in part, segregated and independent. The systems
and cognitive neuroscience literature contains a number of reports that do not carefully
distinguish between these concepts—and therefore conflate the neurocomputational
operations responsible for the distinct aspects of perception and comprehension. The
focus of the present chapter is, principally, speech perception proper, the perceptual
analysis of auditory input. We will provide a brief outline of what we consider a serv-
iceable definition for the cognitive neuroscience of speech perception that links various
interrelated questions from acoustics to phonology to lexical access.
648 David Poeppel, Gregory B. Cogan, Ido Davidesco, and Adeen Flinker
Different Experimental Approaches to Speech Perception

and Their Consequences
Before turning to a brief definition, and subsequently to the three larger current
perspectives on speech perception—the role of hemispheric asymmetry, the role of au-
ditory motor interactions, and the role of neural oscillations—we discuss a few common
experimental approaches. We raise these approaches here because several distinctions
that become germane in the context of different studies should, in our view, not be
overlooked. Broadly speaking, there are at least four experimental approaches that
one can group under the rubric speech perception. Critically, because the approaches
differ substantially with regard to (1) the structure of the input used, (2) the perceptual
subroutines under investigation, and (3) the hypothesized end point of the processes, it
is important to be sensitive to key differences. First, many experiments use tasks that are
executed on individual speech sounds (e.g., single vowels or single syllables); this will
be elaborated later. A second type of study exploits lower-level acoustic properties by
using non-speech materials that approximate (aspects of) speech, for example, sinewave
speech (Remez, Rubin, Pisoni, & Carrell, 1981), noise-vocoded speech (Shannon, Zeng,
Kamath, & Ekelid, 1995), frequency-modulated (FM) sweeps (Boemio, Fromm, Braun,
& Poeppel, 2005), and other suitable low-level stimuli. Third, speech perception is often
investigated in the context of spoken-word recognition using lexical level tasks. This,
too, will be expanded later in the chapter. Finally, sentential materials are used to probe
speech processing; the view on speech perception from a naturalistic signal perspective
has gained popularity in recent years. We will discuss this further, as well.
It should be noted that the experimental approaches and distinctions we point out
in the preceding cut across the neural methods that are employed. Whether exploiting
the high-spatial resolution of functional magnetic resonance imaging (fMRI; see
Heim & Specht, Chapter 4 in this volume), the high temporal resolution of elec-
troencephalography (EEG; see Leckey & Federmeier, Chapter 3 in this volume),
magnetoencephalography (MEG; see Salmelin, Kujala, & Liljeström, Chapter 6 in this
volume), and electrocorticography (ECoG), the dissociation logic of neuropsycholog-
ical deficit-lesion studies, or the causal manipulations investigated with transcranial
magnetic stimulation (TMS; see Schuhmann, Chapter 5 in this volume), the questions
surrounding speech perception have been addressed with each technique and at each
level of analysis, from single vowels to single words to connected spoken sentences. The
diversity of findings requires that one be especially careful about the decomposition of
the tasks and the materials and the putative representations and computations that are
in fact implicated in a given study.
A large body of research on speech perception works with specific contrasts across
individual speech sounds (i.e. sub-lexical/pre-lexical units of speech). Participants
are typically presented with single vowels or single syllables and are asked to execute
some particular task (e.g., discrimination or identification). In a typical study de-
sign, participants listen to consonant-vowel (CV) syllables drawn from some acoustic
Speech Perception 649
continuum—for example, stimuli drawn from the /ra/-/la/tongue-shape contrast or the

/bi/-/pi/voicing contrast—and are asked upon presentation of a single stimulus to iden-
tify the category. These questions focus on sub-lexical properties of speech and examine,
for example, questions concerning categorical perception in speech, the phonemic in-
ventory of speakers/listeners of different languages, perceptual magnet effects, the
changes associated with first-and second-language learning, phonotactic constraints,
the role of distinctive features, and other issues reasonably addressed at the pre-lexical
level of analysis.
This avenue of research has been very productive in the behavioral literature and
is also prominent in the cognitive neurosciences. Using fMRI, for example, several
labs have examined regionally specific hemodynamic effects when participants make
judgments on categorically varying stimuli (Blumstein, Myers, & Rissman, 2005; Chang
et al., 2010; Liebenthal, Binder, Spitzer, Possing, & Medler, 2005; Raizada & Poldrack,
2007;). These studies show that there are cortical regions responding differentially to
signals that belong to different categories, or that are speech versus non-speech. It is
noteworthy that no simple answer has resulted from a group of rather similar studies (at
least at the surface). Temporal, parietal, and frontal brain areas have all been implicated.
Similarly, electrophysiological techniques (EEG, MEG) have been used to probe the
phonemic inventories of speakers of different languages. Näätänen et al. (1997) were
able to show subtle neurophysiological distinctions that reflect the vowel inventories
of Finnish versus Estonian speakers. Kazanina, Phillips, and Idsardi (2006) used MEG
recordings to illustrate how language-specific contrasts (here Russian vs. Korean), in-
cluding allophonic distinctions, can be measured neurophysiologically.
This research program has had considerable influence, but it also has important lim-
itations. For example, a very large number of experiments test categorical perception—
as well as the notion of “rapid temporal processing.” These are typically based on plosive
contrasts (especially voice-onset time, VOT). While it is true that syllables with plosive
onsets are fascinating in their acoustic complexity (and VOT is easily manipulated),
a variety of other phenomena at the pre-lexical level have not been explored very
well. Also, these types of studies are, in some sense, “maximally ecologically invalid.”
Experimenters present single, sub-lexical segments of speech in the context of exper-
imental tasks that require attention to some particular feature. The tasks by and large
engage no (or only very limited) further linguistic processing. The results from such
experiments are in danger of capturing the processes underlying ecologically natural
speech perception in a very limited way, at best. Listeners do not (consciously) attend to
sub-lexical material; therefore, the interpretation of such results requires a great deal of
caution because task effects are known to modulate neural reactivity in very dramatic
ways, modulating, for example, response amplitudes and even hemispheric asymmetry
(e.g., Poeppel et al., 1996).
Another line of research tests speech perception through the lens of spoken word
recognition. There exists a range of lexical access models (e.g., lexical access from
spectra, Klatt, 1979, 1989; TRACE, McClelland & Elman, 1986; instantiations of the co-
hort model, e.g., Gaskell & Marslen-Wilson, 2002; the neighborhood activation model,
Luce & Pisoni, 1998; continuous mapping models, Allopenna, Magnuson, & Tanenhaus,
1998, and others). These have provided critical information regarding the structure of
the mental/neural representations of, informally speaking, words. The most typical
experimental manipulations include lexical decision, naming, gating, and priming.
Recognizing single spoken words is more natural than performing atypical task
demands on sub-lexical material.
Influential cognitive neuroscience contributions to this area have been made
by Blumstein and colleagues, who have examined spoken-word recognition using
deficit-lesion approaches and imaging data (e.g., Utman, Blumstein, & Sullivan, 2001;
Misiurski, Blumstein, Rissman, & Berman, 2005; Prabhakaran, Blumstein, Myers,
Hutchison, & Britton, 2006; see Blumstein, Chapter 1 in this volume). They support a
model in which superior temporal cortex areas mediate acoustic-phonetic analyses,
temporo-parietal areas perform the “mapping” from acoustics to phonological-lexical
representations, and frontal areas (specifically the inferior frontal gyrus) play a role in
resolving competition (i.e., deciding between alternatives when listeners are exposed to
noisy or underspecified input). In addition, the effect of lexical status on speech-sound
categorization has been investigated in the behavioral literature (typically in the context
of evaluating top-down effects), and Myers and Blumstein, using voicing continua with
word or nonword end points, have extended this work using fMRI (Myers & Blumstein,
2008). They demonstrate fMRI- based dissociations between functionally earlier
effects in the temporal lobes (arguably related to perceptual analyses) and later, down-
stream decision processes implicating frontal lobe structures. One behavioral task that
has been used productively in studies of lexical representation is repetition priming.
Gagnepain et al. (2008) used word and nonword repetition priming to elucidate which
cortical structures are specifically sensitive to the activation of lexical entries (for an
MEG implementation of this design, see Almeida & Poeppel, 2013). Bilateral superior
temporal sulcus (STS) and superior temporal gyrus (STG) were particularly prominent,
suggesting that the mapping to lexical information occurs in cortical regions slightly
more ventral than perceptual computations (cf. Hickok & Poeppel, 2000, 2004, 2007).
A third way in which speech perception is increasingly examined is to use natural
spoken sentences, sometimes explicitly to assess their intelligibility if a task is desired,
and sometimes under conditions of passive listening (the most naturalistic condition).
In such experiments, participants are presented with sentences (sometimes having
undergone acoustic manipulations) and are asked to provide some estimate of intelli-
gibility (e.g., by reporting keywords). Understanding spoken sentences is an important
goal because it is the perceptual situation we most want to explain. However, there is
a big price to pay for using ecologically natural materials: it becomes quite difficult to
isolate input-related perceptual processes per se, because the presentation of sentences
necessarily entails lexical access processes, syntactic processes, both lexical semantic
and compositional semantic processes, and so on. Numerous “top-down” factors—that
play a critical role in the overall analysis of spoken input—are all concurrently engaged.
Virtually all cognitive neuroscience recording techniques have been used to test
intelligibility at the sentence level (see Bornkessel- Schlesewsky & Schlesewesky,
Chapter 27 in this volume). Based on a series of positron emission tomography (PET)

and fMRI studies, Scott and colleagues show that anterior temporal lobe structures,
especially anterior STS, play a privileged role in mediating intelligibility (e.g., Scott
Blank, Rosen, & Wise, 2000). Electrophysiological techniques have also been used to
study sentence-level intelligibility. Luo and Poeppel (2007), for example, argue that
phase information in the cortical signal of a particular frequency, the theta band, is
closely related to and modulated by the acoustics of sentences (this is discussed in more
detail later in the chapter). A nice link between spoken-sentence processing and low-
level speech analysis was recently demonstrated by the electrophysiological experi-
ment of Mesgarani, Cheung, Johnson, and Chang (2014). Pre-surgical epilepsy patients
with implanted grids listened to spoken sentences. The electrocorticographic data were
analyzed, with special attention to the local speech segments (i.e., the phonemic level).
By way of innovative analysis approaches, the authors found support for the relevance
of low-level features: the data from STG were consistent with the view that neural ac-
tivation patterns reflect an encoding of distinctive features as the building blocks of
phonemic processing.
In summary, the expression speech perception is used in at least three ways. Important
aspects of the neurocognitive system underlying speech and language have been discov-
ered using all three approaches. This brief perspective serves to remind the reader that it
is challenging to isolate the relevant perceptual computations. Clearly, we need to turn
to all forms of experimental approaches to obtain a comprehensive characterization.
For example, to understand the nature of distinctive features for perception and repre-
sentation, experimentation at the level of sub-phonemic, phonemic, or syllabic levels
is critical; to elucidate how words are represented and accessed, research on spoken-
word recognition is required; and it goes without saying that we cannot do so without
investigating the comprehension of spoken sentences.
Attempt at a Definition of Speech Perception Linking

across Levels
The perspective summarized in brief here has been developed more fully elsewhere
(Poeppel & Hackl, 2008; Poeppel, Idsardi, & van Wass, 2008; Poeppel & Monahan,
2008). Figure 26.1, adapted from Poeppel et al. (2008), summarizes what we take to be
the perceptual challenge. We take speech perception to refer to a specific set of computa-
tional sub-routines: speech perception comprises the set of operations that take as input
continuously varying acoustic waveforms made available at the auditory periphery and
that generate as output those representations (morphemic, lexical) that constitute the
data structures for subsequent operations mediating language comprehension. More
colloquially speaking, this view can be caricatured as “the collection of operations that
lead from vibrations in the ear to abstractions in head” (or from “Sounds in the Cochlea
to Morphemes in Cortex”—SoCoMoCo).
(A) (B)
(D) c a t
x x x
[+cons, -son] [-cons, +son] [+cons, -son]

(C)
[-cont] [-cont]
LAR/PHAR PLACE LAR/PHAR PLACE LAR/PHAR PLACE
GLOT DORSAL [-ATR] DORSAL GLOT CORONAL

phonological
promal sketch
[-voice] [-back, -high, +low] [-voice] [+ant]
Figure 26.1. From waveforms to words. Continuously varying acoustic signals (A) are analyzed
in the afferent auditory system. The analysis streams yield “neural versions” of spectrograms
(spectrotemporally specified representations) in bilateral auditory cortex (B) Based on this high-
resolution cochleogram representation, we hypothesize that a “primal sketch”—based on multi-
time resolution analysis of the information contained in (B)—is constructed (C). The perceptual
goal or end game is the identification of words, which we take to be represented in memory as
sequences of segments that themselves are composed of bundles of distinctive features, here
exemplified as the word [cat] (D).
Source: From Poeppel, Idsardi, and van Wassenhove (2008).
The starting point, the acoustic signal, is a continuously varying waveform that
encodes information on different timescales (Figure 26.1 A). For example, the ampli-
tude envelope of a signal (which can be investigated as a broadband signal or by coch-
lear channels or critical bands) correlates well with properties of the syllabic sequence
of an utterance. The fine structure of a signal, in contrast, carries information over
shorter time scales (including features and segments). This input data stream must, in
the end, be transformed into a sequence of discrete segments that constitute a stored
morpheme (or informally, a word). Because we believe the central goal to be the iden-
tification of spoken words, specifying the format of the stored lexical representation
becomes necessary. In particular, the morphemes/words must be stored in a format
that permits them to enter into linguistic computation (including, most critically, the
combinatoric operations that underlie language comprehension). Identifying a word is
obviously not enough—the listener must (be able to) connect it formally (in terms of
representational specifications, i.e., noun, determiner, etc., in whatever neural code that
is specified in) to its neighboring environment. This is required to perform the phono-
logical, morphological, syntactic, or semantic operations the situation demands—and
which we know take place. We adopt the view here, developed in linguistic research over
the last decades—and implicit since the Phoenicians invented an alphabet—that lexical
representations are represented as a series of segments; segments themselves are made
up of bundles of features (Figure 26.1 D). We explicitly will also allow other, parallel
representations (e.g., the notion of a syllable).
The initial input waveform is analyzed by the auditory periphery and presum-
ably represented in auditory cortex by neurons with spectrotemporal receptive field
properties (STRFs) covering a range of relevant sound modulations. This can be
thought of as a neural version of a spectrogram, although a version composed of nu-
merous mini-spectrograms, encompassing specializations for certain spectrotemporal
patterns (Figure 26.1 B). (Given the highly conserved nature of mammalian auditory
cortex, these representations are likely shared with other species, and consequently
can be investigated fruitfully using animal models.) Based on such initial (rather high
spectrotemporal resolution) auditory cortical patterns, multiple representations on dif-
ferent time and frequency scales are constructed, in parallel. In such a next step, “au-
ditory primitives” are built out of early auditory cortical elements, with a key feature
being the time scale of the new representations. This third type of representation must
be of a granularity that permits mappings (linking procedures) from simple acoustic
properties in early auditory cortical areas to speech primitives in more downstream
areas (arguably STG and STS). We hypothesize that these intermediate representations
encompass at least two subtypes, commensurate with syllabic and segmental durations,
respectively (Boemio et al., 2005; Giraud et al., 2007; Poeppel, 2001, 2003; Poeppel et al.,
2008). The initial cortical representation is segregated into (at least) two parallel pro-
cessing streams, and concurrent multi-time resolution analysis then lies at the basis of
subsequent steps. The specific nature of this intermediate representation is a critical re-
search question; one might characterize the question as arriving at a “primal sketch” for
speech perception (Figure 26.1 C), related to David Marr’s famous hypothesis about in-
termediate representations for visual object recognition.
One specific possibility for a primal sketch is a PFNA (plosive-fricative-nasal-
approximant) coarse coding. Let us hypothesize that this code is just the speech
stream roughly encoded into the four categories of PFNA. Preliminary modeling of
English-like lexica shows that this coarse coding yields pools of words of approxi-
mately the same size as the typical lexical neighborhoods, with a fair degree of overlap
between neighborhoods. We now stipulate a directed left-to-right search, using con-
textually defined featural definitions. Importantly, this search—after having found
broad landmarks—can be guided by the differences among the words in the active
neighborhood by using analysis-by-synthesis and/or Bayesian inference. Finally,
once the best-available target has been selected, the content of that lexical item is
examined, compared to the trace of the incoming signal, and verified to be the word
being sought.
The final, featurally specified representation is by hypothesis the format that is both
the end point of speech perception, which is also the set of instructions for possible
articulation. The loop between perception, memory, and action is enabled in such
a scheme because the representational format used for words in memory, distinctive
features, allows the mapping both from the input waveform to words and from words to
motoric action (features are in motor coordinates). In conclusion, we envision a three-
step process that offers a computational locus for the kinds of findings typical of speech
perception research: from early encoding and sensitivity to acoustics, to various lexical
effects, to links to the motor system. We now turn to broader issues that have received a
great deal of attention in the cognitive neuroscience literature.
Hemispheric Asymmetries
One of the most striking examples of functional lateralization in cortex is that damage
to the left hemisphere quite often produces a severe impairment to language processing,
while damage to the right hemisphere does not. The lateralization of language to the left
hemisphere has been consistently reported ever since the pioneering work of Paul Broca
and Carl Wernicke in the nineteenth century (Broca, 1861; Wernicke, 1874; see Van der
Haegen & Cai, Chapter 34 in this volume). Despite 150 years of research, elucidating
hemispheric dominance in healthy participants is surprisingly difficult and, in clinical
contexts, still requires an intra-carotid injection of sodium amytal to each hemisphere
separately (Wada, 1949). Laterality of function has been extensively demonstrated
using psychophysical measures, electrophysiology, and neuroimaging—but the exact
underpinnings of the cortical architecture and its asymmetric nature remain elusive.
Behaviorally, participants are more accurate in reporting stimuli presented to the right
ear compared with the left ear in a dichotic listening paradigm (Kimura, 1967). This
right ear advantage (REA) is highly modulated by attention and is only partially con-
cordant with Wada-test results (Hugdahl, 2011). There is an ongoing effort to introduce
neuroimaging as part of preoperative language mapping; nevertheless, studies report
that 15%–20% of patients show discordant results with the Wada test (Bauer, Reitsma,
Houweling, Ferrier, & Ramsey, 2014; Janecek et al., 2013), and it remains controver-
sial whether noninvasive measures are sufficient for mapping language dominance
(McDonald, Kean, & Saykin, 2011; Papanicolaou et al., 2014).
Evidence for hemispheric specialization exists early in development. Near-infrared
spectroscopy in neonates demonstrates an asymmetric hemodynamic response profile
to repetition-based structures in syllable sequences (Gervain, Macagno, Cogoi, Peña,
& Mehler, 2008), to phonemic and prosodic cues in word stimuli (Arimitsu et al., 2011),
as well as to temporal modulations in speech-inspired non-speech stimuli (Telkemeyer
et al., 2009). It is debated to what degree these putative specializations change across
development. For example, Telkemeyer et al. (2011) argue that left and right temporal
cortex sensitivity to acoustic temporal modulations remains stable as infants mature
from neonates (Telkemeyer et al., 2009) to infants (Telkemeyer et al., 2011), while data
from Arimitsu et al. provide evidence that temporal cortex sensitivity to phonemic
cues is symmetrically bilateral and an asymmetric shift toward increased left sensitivity
occurs at age 11–12 months (Arimitsu et al., 2011).
Since the seminal findings of Carl Wernicke, localizing language comprehension

deficits to the left posterior temporal cortex, the perceptual analysis of language has
been mostly considered a left hemisphere capacity (Geschwind, 1970; Wernicke, 1874).
Observations starting in the second half of the twentieth century have shed light on
the contributions of different features, painting a more complicated picture of the un-
derlying cortical structures involved in processing speech. Studies in aphasic patients
implicated the left temporal lobe in processing temporal order (Efron, 1963) as well as
specific temporal cues (Carpenter & Rutherford, 1973; Schwartz & Tallal, 1980), formant
transitions (Tallal & Piercy, 1975), and voice onset time (Blumstein, Cooper, Zurif, &
Caramazza, 1977). These deficits in temporal processing were often contrasted with the
spared ability to discriminate spectral cues (Carpenter & Rutherford, 1973). Around
the same time, evidence emerged that focal damage to the right hemisphere produced
impairments in processing prosodic aspects of speech (Ross & Mesulam, 1979; Tucker,
Watson, Heilman, 1977), and later studies also implicated the right hemisphere in pro-
cessing spectral cues (Robin, Tranel, & Damasio, 1990; Zatorre, 1988).
Based on these findings, as well as early neuroimaging studies (Belin et al., 1998;
Zatorre, Evans, Meyer, & Gjedde, 1992), an influential framework emerged whereby
the left hemisphere is more sensitive to temporal cues, whereas the right hemisphere
is more sensitive to spectral cues (Zatorre, Belin, & Penhune, 2002). A complemen-
tary framework explains the differences in auditory sensitivity as a function of tem-
poral integration windows, whereby the left hemisphere preferentially integrates over
shorter timescales (20–80 ms) and the right hemisphere preferentially integrates longer
timescales (150–300 ms) (Poeppel, 2003). Both the Zatorre and Poeppel views, which
are very similar, have been debated and contested; for example, Scott et al. argue that
while the right temporal cortex is highly selective to acoustic features and is domain ge-
neral, the acoustic selectivity of the left hemisphere is weak at best and is rather driven
by speech sounds, which are domain-specific and have a more complex acoustic charac-
terization (McGettigan & Scott, 2012; Scott & McGettigan, 2013).
The advent of neuroimaging techniques gave rise to numerous studies trying to elu-
cidate the functional anatomy of auditory, speech, and language processing. A recur-
ring finding has been the bilateral involvement of the STG in processing speech sounds
(Binder et al., 1997; Hirano et al., 1997; Mazoyer et al., 1993; Petersen, Fox, Posner,
Mintun, & Raichle, 1988; Wise et al., 1991), with preferential recruitment of left hemi-
sphere structures depending on how stimuli were controlled and contrasted, and the
demands of the tasks employed (Binder et al., 2000; Hickok & Poeppel, 2007; Price,
2012; Scott, Blank, Rosen, & Wise, 2000). Listening to non-speech stimuli has consist-
ently shown bilateral responses characterized by an asymmetry in response to stimuli,
varying along spectral and temporal dimensions. Neuroimaging studies have reported
a greater sensitivity in the left hemisphere to a temporal gradient (the rate of alternating
two pure tones), in contrast to a greater sensitivity in the right hemisphere to a spectral
gradient (the number of alternating pure tones varying in frequency) (Jamison, Watkins,
Bishop, & Matthews, 2006; Zatorre & Belin, 2001). Similar lateralized responses have
been observed in neurophysiological data in response to spectral change (stimulus
changes in frequency) or temporal change (stimulus changes in amplitude modulation)

(Okamoto & Kakigi, 2013; Okamoto, Stracke, Draganova, & Pantev, 2009).
Evidence from passive listening paradigms using both neuroimaging (Boemio et al.,
2005; Telkemeyer et al., 2009) and neurophysiology (Luo & Poeppel, 2012) shows asym-
metric responses as a function of a temporal axis. Both left and right temporal cortices
responded similarly to stimuli consisting of concatenated short segments (~25–50 ms),
while a preferential response in right temporal cortex was reported for stimuli consisting
of long segments (~150–300 ms). Similarly, studies have shown enhanced right hemi-
sphere sensitivity to features such as spectral correlation (Overath, Kumar, Kriegstein
von, & Griffiths, 2008), where increased correlations across frequency components
are equivalent to longer temporal windows, as well as pitch discrimination (Hyde,
Peretz, & Zatorre, 2008). Systematically varying the temporal modulations of acoustic
stimuli has shown a low-pass characteristic bilaterally (Liégeois-Chauvel, Lorenzi,
Trébuchon, Régis, & Chauvel, 2004; Overath, Zhang, Sanes, & Poeppel, 2012; Wang
et al., 2012), with reports of the left hemisphere covarying with temporal modulations
(Schönwiesner, Rübsamen, & Cramon Von, 2005), as well as left hemisphere increased
sensitivity to slower temporal modulations (Liégeois-Chauvel et al., 2004; Wang et al.,
2012). Recently, studies have reported a spatial topography of auditory cortex sensitivity
to temporal and spectral modulations (Barton, Venezia, Saberi, Hickok, & Brewer, 2012;
Herdener et al., 2013; Joanisse & DeSouza, 2014; Santoro et al., 2014; Schönwiesner &
Zatorre, 2009). While these studies have reported mostly bilateral symmetries, modu-
lation topographies show conflicting results (Herdener et al., 2013; Santoro et al., 2014;
Schönwiesner & Zatorre, 2009), as well as conflicting reports of frequency modulation
encoding, which in some cases was only revealed using multi-voxel pattern analysis
(Hsieh, Fillmore, Rong, Hickok, & Saberi, 2012; Joanisse & DeSouza, 2014). The exact
spatial topographic distribution of combined spectral-temporal modulations remains
to be clarified—as well as how these maps relate to functional hemispheric asymmetries.
The acoustic spectrotemporal features that drive hemispheric responses are closely
related to speech-specific features that have been reported in the left hemisphere, such as
voicing (Liégeois-Chauvel, de Graaf, Laguitton, & Chauvel, 1999; Steinschneider et al.,
2005; Trébuchon-Da Fonseca, Giraud, Badier, Chauvel, & Liégeois-Chauvel, 2005) and
place of articulation (Arsenault & Buchsbaum, 2015; Chang et al., 2010; Kilian-Hütten,
Valente, Vroomen, & Formisano, 2011). In the case of speech, these acoustic and pho-
netic features do not exist in a vacuum, but are rather an integral part of connected
speech and the basis for engaging semantic and syntactic systems. In that context,
Obleser et al. report that degrading speech intelligibility along temporal (reducing tem-
poral modulations) and spectral (reducing the number of spectral bands) axes drives the
left and right superior temporal sulcus, respectively, but the left hemisphere was equally
affected by the temporal and spectral manipulations (Obleser, Eisner, & Kotz, 2008).
Similarly, neuroimaging studies have consistently reported that degrading the intelligi-
bility of speech stimuli shows a strong left hemisphere bias (Obleser, Meyer, & Friederici,
2011; Obleser, Wise, Dresner, & Scott, 2007; Rosen, Wise, Chadha, Conway, & Scott,
2011; Scott et al., 2000), although the right hemisphere sensitivity to spectral features
is debated (McGettigan & Scott, 2012; Zatorre & Gandour, 2008). Electrophysiological
findings have shown that the phase of cortical theta oscillations (~4–8 Hz) reliably
tracks and discriminates sentences (Luo & Poeppel, 2007), is modulated by attention
(Kerlin, Shahin, & Miller, 2010), and shows enhanced phase-locking to the envelope
(modulation of signal energy over time) of intelligible sentences (Peelle, Gross, & Davis,
2013). Phase-locking occurs bilaterally, and while intelligible sentences showed left
hemisphere enhanced locking (Peelle et al., 2013), sentence-discrimination effects were
driven by the right hemisphere (Kerlin et al., 2010; Luo & Poeppel, 2007). This right
hemisphere theta effect is consistent with a literature supporting its role in segmenting
the speech signal with a window of ~200 ms, while the increased effect of intelligibility
provides evidence for interaction with other linguistic systems in the left hemisphere
(Giraud & Poeppel, 2012; Millman, Woods, & Quinlan, 2011; Morillon et al., 2010).
It is now well established that speech is processed bilaterally but becomes increasingly
lateralized to the left with higher-order linguistic input (Binder et al., 2000; Friederici,
2011; Giraud & Poeppel, 2012; Hickok & Poeppel, 2007; Rauschecker & Scott, 2009).
Nevertheless, whether or not this asymmetry is driven by acoustic features in addition
to linguistic factors—as well as the exact computational framework—remains vigor-
ously debated (Giraud & Poeppel, 2012; Obleser, Herrmann, & Henry, 2012; Peelle, 2012;
Scott & McGettigan, 2013; Zatorre & Gandour, 2008).
The Role of the Motor System
The two primary modalities for speech are auditory input for perception (listening)
and motor output for production (speaking). Early work by neurologists in the nine-
teenth century proposed that these two processes were also separate neuroanatomically
(Broca, 1861; Lichtheim, 1885; Wernicke, 1874). The neural substrate for production was
proposed to be in the inferior frontal cortex, whereas perception was localized to the
posterior STG and STS. The major assumptions were therefore twofold: first, that there
were clearly definable and separate areas for production and perception; and second,
that the representations underlying these processes were also separable.
This latter proposed segregation of function was challenged by proposals for a strong
role for production-related processes during speech perception. This was the posi-
tion put forth by Liberman and colleagues with the motor theory of speech percep-
tion (Liberman, Cooper, Shankweiler, & Studdert-Kennedy, 1967; Liberman, Cooper,
Studdert-Kennedy, Harris, & Shankweiler, 1968; Liberman & Mattingly, 1985). They
argued that the underlying perceptual speech representations were in fact “intended ar-
ticulatory gestures” for speaking. Liberman and colleagues found that when presenting a
noise burst at a particular frequency, it was perceived as either a /k/or a /p/depending on
the subsequent vowel (Cooper, Delattre, Liberman, Borst, & Gerstman, 1952; Liberman,
Delattre, & Cooper, 1952). They argued that this constituted evidence against a purely
acoustic theory of perception, and supported a model that involved reconstructing the
intended outgoing motoric gesture (Liberman & Mattingly, 1985; Liberman et al., 1968).
These gestures were proposed to be the elementary units of speech, and consequently,
defining their feature space was seen as a central goal of studying the neural representa-
tion of speech.
The focus of the motor theory shifted from a purely motor involvement in perception
to a unified “phonetic” module for speech, in line with proposals for the modularity
of mind (Fodor, 1983). The “phonetic” module was argued to be responsible for pro-
cessing speech-specific information, which differed from purely acoustic information
(Liberman, Cooper, Shankweiler, & Studdert-Kennedy, 1967). Through this proposal,
representational processes that underlie both perception and production were united.
This contrasted with theories based on earlier lesion work by the nineteenth-century
neuroanatomists that argued for a stricter separation.
Initial evidence for this lack of perception/ production separation comes from
the work of Sheila Blumstein and colleagues starting in the 1970s. Contrary to what a
strong anterior/ posterior production/ perception split would predict, both Broca’s
and Wernicke’s aphasics display production errors (Blumstein, 1994). Both classes of
aphasics also demonstrate perceptual deficits, specifically on discriminating phonolog-
ical contrasts (Blumstein, Cooper, Zurif, & Caramazza, 1977). Furthermore, both classes
of aphasics make articulatory errors (Blumstein, 1994).
Further evidence for this unified theory has since been added from two other sources.
The first comes from perceptual difficulties in instances where the articulatory system is
compromised. The second is neuroimaging evidence that seeks to relate the anatomy as-
sociated with production during perceptual processing. With respect to the first source
of evidence, there have been several reports that perceptual deficits follow articula-
tory ones. Bridgeman and Snowling (1988) found that when comparing children with
dyspraxia and their age-and reading-ability-matched controls, the dyspraxic group
was worse at discriminating nonwords in a same/different task, while no such differ-
ence was found for real words. The authors argue that this is evidence for a dual route
of processing in which a lexical route (intact in both groups) processes lexical input,
and a non-lexical phonetic stream (deficient in the dyspraxic children) processes novel/
nonword input. Using a series of rhyme-based tasks, Marion Sussman, and Marquardt
(1993) found that dyspraxic children had difficulty producing and perceiving rhymes
compared to controls. Specifically, they produced far fewer rhyming words when
prompted with a token, were less able to recognize rhyme pairs, and were also worse
at judging vowel similarity (a result that was also echoed by Maassen Groenen, & Crul,
2003). The authors conclude that this is evidence for a deficit in a higher-order pho-
nemic mechanism that is shared between perception and production. Nijland (2009)
found that, like the previously mentioned studies, children with apraxia were worse
at nonword discrimination and rhyming judgments. She also found that these tasks
demonstrated a significant link to production via a positive correlation with nonword
repetition. In sum, behavioral perceptual deficits associated with clinical articulatory
deficits demonstrate that there is a tight link between processes that underlie perception
and production. The locus of these behavioral deficits seems to be focused on the role of
(faulty) articulation and its role in the development of phonetic/phonemic (sub-lexical)

processing.
From a neurophysiological standpoint, there have been several demonstrations of
activation of the premotor/motor cortex during passive listening to speech using both
ECoG and fMRI (Cogan et al., 2014; Pulvermüller et al., 2006; Wilson, Saygin, Sereno,
& Iacoboni, 2004). Du et al. (Du, Buchsbaum, Grady, & Alain, 2014) used fMRI to test
the activation of motor areas during speech perception. Specifically, they found that the
BOLD response in the premotor cortex exhibited a negative correlation with behavioral
accuracy of a phoneme identification task in increasing amounts of background noise,
demonstrating greater premotor cortex activation in noisier conditions. Furthermore,
using multivoxel pattern analysis (MVPA), they found that the identity of the pho-
neme could be determined in lower signal-to-noise ratios (SNR) in the premotor cortex
compared to a speech-sensitive auditory area (STS).
There have also been TMS studies that argue for a causal role of motor areas during
perception. Meister, Wilson, Deblieck, Wu, and Iacoboni (2007) used TMS over the left
premotor cortex to demonstrate that stimulation resulted in poorer phoneme discrimi-
nation in noise. A widely cited example of linkage between neural motor processes and
perception is from a study by D’Ausilio et al. (2009) in which TMS was applied to both
the lip and tongue areas of the primary motor cortex (M1). Their results suggested a
double disassociation during identification of syllables starting with one of four plosives
(/b/, /p/, /d/, /t/) in which stimulation to the tongue area of M1 disrupted /d/and /t/
identifications, and stimulation to the lip region disrupted identification of /b/and /p/
, suggesting a link between the perception of phonemes and the articulatory apparatus
that specifically produces them. It is worth noting that the stimuli in this study were also
presented in noise. This result was supported in a TMS study by Fadiga et al. (Fadiga,
Craighero, Buccino, & Rizzolatti, 2002) in which motor-evoked potentials (MEP)
were measured in the tongue during speech perception. Results implied an increase
in amplitude of the tongue MEPs during the passive listening of disyllabic words and
pseudowords containing either /f.f/(labiodental fricative—slight tongue tip movement)
or /r.r/(trill—more tongue tip movement). There were higher amplitude MEPs on the
tongue during TMS stimulation for words and pseudowords containing /r.r/compared
to /f.f/. This further suggests a link between the perception of phonemes and activation
of the corresponding articulatory mechanisms that produce them.
Cumulatively, these results argue for a potential role for the motor system during
speech-perception tasks, in particular focused on sub-lexical phonetic processing.
Phonetic task differences in children with dyspraxia provide behavioral evidence for a
motor involvement in speech perception, and various imaging (fMRI, ECoG) and stim-
ulation (TMS) results demonstrate that the motor system affects perceptual abilities.
The most likely candidate for the locus of this linkage is sub-lexical. This would pro-
vide a plausible basis for linking motor gestures with perception (Liberman, Cooper,
Shankweiler, & Studdert-Kennedy, 1967). What remains unclear is the necessity of the
motor system in perception. Given that the majority of neuroimaging and stimulation
results were presented in noise, it could be that the motor system is recruited specifically
in instances in which some form of supplementary processing is required. The role

of the motor system could therefore be more modulatory rather than causal (Hickok,
2009). The motor system could, for example, be recruited via a non-lexical route to aid
perception during noisy or ambiguous situations.
A different class of proposals argues against the sub-lexical perception/production
link and instead posits that the role of the motor system in speech perception relates
to the meaning of words. Specifically, it is argued that the motor system represents
words that are semantically congruent with the actions performed by the particular
effectors controlled by the motor system (e.g., lick: tongue; kick: leg). This approach
is based heavily in the broader theory of embodied cognition (Rosch, Thompson,
& Valera, 1992) where complex cognition is carried out through the same processes
that underlie action. Through this approach, action and perception are linked not
at the implementational level (i.e., through a link with the articulatory system), but
rather at the abstract level. The motor system’s role in speech, therefore, would be to
provide a causal instantiation of the semantic space of words. In other words, con-
ceptual knowledge about actions shares the same circuits as the motor programs for
these actions.
Linkage at this level was suggested by Hauk, Johnsrude, and Pulvermüller(2004),
who found that when subjects were visually presented with action words that were re-
lated to movements associated with specific effectors (e.g., leg, arm), they demonstrated
higher BOLD activation in their correspondent motor region in the motor cortex and
prefrontal cortex. This basic finding has also been explored using EEG (Shtyrov, Hauk,
& Pulvermüller, 2004) and MEG (Pulvermüller, Shtyrov, & Ilmoniemi, 2005). A more
recent study, however, using ECoG did not replicate this effect (Canolty et al., 2007). The
main drawback of such theories is that they fail to account for all but a very restricted
class of linguistic materials. Concepts that are not directly related to human action (or
their figurative counterparts (Fernandino et al., 2013) are not easily accounted for using
this model.
Neurophysiological arguments for a motor-centric view comes from the more re-
cent mirror neuron findings in humans. Mirror neurons fire both when an action is
perceived and when an action is being executed. They were first demonstrated in the
macaque area F5 (di Pellegrino, Fadiga, Fogassi, Gallese, & Rizzolatti, 1992). Similar
response profiles have since been identified in humans in the inferior frontal gyrus
(Kilner, Neal, Weiskopf, Friston, & Frith, 2009) using fMRI and in the supplementary
motor area (SMA) using single-unit recordings (Mukamel, Ekstrom, Kaplan, Iacoboni,
& Fried, 2010). These mirror neurons also respond to the sound of actions as well as
their sight (Kohler et al., 2002), suggesting a higher-order level of abstraction. It is worth
noting that these neurons respond also when a goal-directed action is taking place and is
not visually present, suggesting that the representation is not purely due to the external
stimulus (Umiltà et al., 2001). Furthermore, different populations of mirror neurons in
the inferior parietal lobule (IPL) respond differently to the same movement depending
on the context of the action, suggesting that this system could be used to differentiate
between higher-order actions (Fogassi et al., 2005).
On the face of it, the mirror neuron system seems to be a plausible substrate for
higher-order semantic-based representations in the motor system. The higher-order
representational properties of such cells could support a system that operates at the level
of conceptual (or, in humans, even lexical) semantics. However, the locus of represen-
tation could in theory be any complex directed representation. This is especially impor-
tant given that the initial proposed representation of the motor theory of speech was
claimed to be the intended motor gesture (Liberman, Cooper, Shankweiler, & Studdert-
Kennedy, 1967; Liberman & Mattingly, 1985). It could therefore be—assuming there is
a relationship between the mirror system and speech—that the representational sub-
strate that it operates on is at the level of phonetics rather than semantics. The mirror
system therefore, provides at best an ambiguous neural substrate for a semantic role for
the motor system in speech perception.
There does exist convincing evidence for a linkage between motor output and sensory
input systems for speech, certainly under particular processing circumstances. Mapping
between the sensory system and the motor system could operate to form a unified sub-
lexical representation for the speech system (Cogan et al., 2014). The engagement of
the motor system would therefore reflect part of a unified phonetic system (Liberman,
Cooper, Shankweiler, & Studdert-Kennedy, 1967; Liberman & Mattingly, 1985). In sum,
there is evidence that the motor system is active during speech perception. While there
is some evidence that the role of the motor system in speech perception is to represent,
in semantic terms, action words (Pulvermüller, 2005), there is stronger evidence that
the locus is the phonetic level. Cumulatively, the behavioral, lesion, and neurophysio-
logical evidence suggests that at the phonetic level, the original strong segregation of
function between production and perception in speech processing is no longer tenable.
That being said, a comprehensive, theoretically persuasive, and mechanistic model does
not yet exist.
Cortical Oscillations
in Speech Perception
There has been a substantial interest in the phenomenon of neural oscillations and
their potential role in attention (Lakatos, Karmos, Mehta, Ulbert, & Schroeder, 2008),
memory (Gruber, Tsivilis, Giabbiconi, & Muller, 2008; Palva, Palva, & Kaila, 2005),
decision-making (Guggisberg, Dalal, Findlay, & Nagarajan, 2007), and speech per-
ception (Ahissar et al., 2001; Lalor & Foxe, 2010; Luo & Poeppel, 2007; Peelle, Gross,
& Davis, 2013). Neural oscillations reflect physiological excitability cycles and occur in
multiple time scales that emerge from the biophysical properties of neural ensembles.
These time scales extend over a wide range, from very slow fluctuations (<1 Hz) to very
fast ones (>100 Hz) (Buzsáki & Draguhn, 2004; Fries, 2005). One hypothesis is that these
time scales relate to exogenous phenomena that occur at corresponding time frames
(Giraud & Poeppel, 2012; Poeppel, 2003). In the case of speech, there is a striking cor-
respondence between the time scales of phonemic, syllabic, and phrasal processing, on
the one hand, and low gamma (30–50 Hz), theta (4–8 Hz), and delta (<3 Hz) oscillations
in auditory cortex, on the other. This correspondence has led to the hypothesis that
cortical oscillations play an active role in “sampling” (or “parsing” or “segmenting” or
“chunking”) the continuous speech signal into meaningful linguistic units (Ghitza, 2011;
Ghitza, Giraud, & Poeppel, 2012; Giraud & Poeppel, 2012).
Giraud and Poeppel (2012) have suggested an oscillation-based processing model
that hypothesizes a cascade of processes leading from connected speech to a discrete
code (Figure 26.2). In this model, intrinsic slow (<10 Hz) oscillations in auditory cortex
phase-lock to incoming speech such that the activity in the theta band, in particular,
tracks the envelope (energy fluctuations over time) of speech. Next, through a mech-
anism of cross-frequency coupling, the phase of theta oscillations determines the am-
plitude of gamma oscillations. Finally, gamma oscillations regulate spiking activity such
that neural excitability is aligned with speech modulations (see Figure 26.2).
Speech waveform
Spectro-temporal
encoding
5. Alignment of neuronal excitability

with acoustic structure
Stimulus-driven spike train
(input layer IV)
Temporally organized spike train
(output layers II/III)
High excitability Low excitability
Strong spiking 4. Modulation of neuronal excitability
1. Phase reset and output discretization
Stimulus-induced theta-modulated gamma
oscillations (25–35 Hz)
3. Theta/gamma nesting
Stimulus-induced theta oscillations

(1–8 Hz; LFP, EEG, MEG)
2. Stimulus
envelope tracking
Figure 2 6.2. Proposedoscillation-basedoperationsinspeechperception.A sequenceofoperations

allows connected speech to be segmented by cortical theta and gamma oscillations. A high-
resolution spectrotemporal representation of speech in primary auditory cortex is assumed.
A typical afferent spike train activates layer IV cortical neurons. Many of these neurons phase-
lock to speech amplitude modulations. Response onset elicit theta oscillations reset in superficial
layers (step 1), where auditory cortex output is generated. By way of phase reset, theta oscillations
track the speech envelope (step 2). Theta reset induces a transient break in gamma activity and a
subsequent reset of gamma oscillations. Theta and gamma generators, weakly coupled at rest, be-
come more strongly coupled and nested (step 3). Gamma power, in turn, controls the excitability
of neurons generating the feedforward signal from A1 to higher order areas (step 4). Neuronal
excitability phase now aligns to speech modulations (step 5).
Ghitza (2011) has proposed a parallel phenomenological model (Tempo) in which

auditory information is processed simultaneously along two paths: a parsing (or seg-
mentation) path and a decoding path. The parsing path, in alignment with the Giraud
and Poeppel model, is composed of an array of cascaded oscillators in which the theta
oscillator serves as the master by providing syllabic level parsing. The theta oscillator
sets the boundary conditions for higher-frequency oscillatory activity, which in turn is
argued to regulate finer-grained within-syllable fragmentation and ultimately diphone-
based, phoneme-based, or feature-based fractionation. The assumption in this model is
that these oscillators can phase-lock to the rhythms of the input within a given range of
frequencies. The output of these oscillators is fed to the decoding path, which performs
fine-grained pattern recognition at the level of smaller speech units.
We first review some of the psychophysical evidence supporting the role of theta
oscillations in syllabic parsing, as proposed by the models described in the preceding.
Decomposing speech into spectral and temporal modulations reveals that most of the
power of the signal is concentrated in the low frequency range (2–10 Hz) (Greenberg,
2006; Greenberg, Carvey, Hitchcock, & Chang, 2003; Houtgast & Steeneken, 1985).
Psychophysical studies suggest that modulations in this low range are critical for speech
intelligibility (Drullman, Festen, & Houtgast, 1996; Drullman, Festen, & Plomp, 1994a,
1994b; Ghitza, 2012; Ghitza et al., 2012).
Ghitza and Greenberg (2009) used a novel method to restore intelligibility of time-
compressed speech. Speeding up speech by a factor of 3 or more severely impairs in-
telligibility; but, surprisingly, insertion of silent gaps in between successive intervals of
the compressed speech can substantially improve intelligibility. By manipulating the
duration of the silent gaps (while holding the length of the presented acoustic chunks
constant at 40 ms), Ghitza and Greenberg (2009) showed that the relationship between
intelligibility and silent gap duration has a U-shape, where intelligibility peaks for a gap
of 80 ms long. These results are compatible with the Tempo model (Ghitza, 2011): at
a compression rate of 3, the theta oscillator hits its boundary and can no longer track
speech, and therefore intelligibility is impaired. The insertion of silent gaps is interpreted
as a repackaging of the speech signal in a way that allows the (syllable-sized) theta os-
cillator to once again lock again to the input rhythm, allowing parsing and therefore
decoding.
In a later study, Ghitza (2012) examined the intelligibility of speech using a
manipulated modulation spectrum. Two classes of stimuli were synthesized: one class
carried acoustic-phonetic information sufficient for the function of the decoding path
(as per the Tempo model) but lacked the input syllabic rhythm (needed for the operation
of the parsing mechanism of the Tempo model). This was achieved by filtering out 2–9
Hz modulation information from speech, thus flattening the temporal envelope of the
signal. As expected, the intelligibility of this class of stimuli was low. The second class of
stimuli was generated by adding to the first stimulus type a signal that carried informa-
tion restricted to the input rhythm alone (i.e., without any acoustic-phonetic content).
This was achieved by placing clicks at the mid-vowel locations (that were eliminated
by the original filtering). This surprisingly simple manipulation resulted in significant
improvement in intelligibility. In other words, reinstating the input rhythm with an au-
ditory cue is argued to restore theta-based parsing, thus allowing the decoding path to
extract information from the flat modulation spectrum.
In the first MEG study that addressed the relationship between phase patterns of
neural responses and speech perception (Luo & Poeppel, 2007), participants were asked
to listen to naturally spoken sentences. An inter-trial coherence analysis revealed that
the phase, but not the power, of theta-band (4–8 Hz) oscillations tracks the dynamics of
speech and supports the reliable discrimination of different sentences. In a later study
(Luo, Liu, & Poeppel, 2010), these findings were extended to the multi-modal case. In
this study, participants were presented with short audiovisual movie clips while MEG
responses were recorded from auditory and visual cortices. Low-frequency (delta and
theta) phase patterns in early visual and auditory cortices were shown to track natu-
ralistic visual and auditory stimuli, respectively. In addition, the low-frequency phase
pattern in one modality could track, to some extent, the stimulus structure of the other
modality.
To what extent does theta-phase tracking reflect speech intelligibility? According
to the models described here, tracking via entrainment of neural theta oscillations
provides syllabic level parsing that is crucial for subsequent proper decoding of speech.
In the original Luo and Poeppel study (2007), discriminability between different speech
tokens was much lower for acoustically degraded sentences, which are also less intel-
ligible, suggesting that theta phase is linked directly to speech intelligibility. However,
a later study (Howard & Poeppel, 2010) showed that theta phase allows equally strong
discrimination between time-inverted sentences, which are unintelligible. This suggests
that theta-phase tracking depends on acoustic features of the speech signal, not compre-
hension. Theta-based tracking—and therefore syllabic level segmentation—might be
necessary for comprehension but is clearly not sufficient.
A neurophysiological study examined the link between theta-phase tracking and
intelligibility in a more direct way (Peelle et al., 2013). These authors varied the intel-
ligibility of sentences using a noise-vocoding technique. Specifically, the number of fre-
quency channels used in vocoding was manipulated, thus affecting intelligibility level
while preserving the overall amplitude envelope. Even in the case of completely unin-
telligible speech, bilateral low-frequency MEG responses were phase-locked to the
envelope of the auditory signal. However, phase-locking was enhanced for intelligible
speech, especially in left temporal cortex. The authors further showed that this result
cannot be attributed to changes in spectral complexity. This was done by comparing
responses with four-channel-vocoded speech (partially intelligible) to spectrally rotated
four-channel-vocoded speech (unintelligible). Even when spectral complexity was held
constant, phase-locking was stronger for intelligible speech. Thus, these results demon-
strate that phase-locking is not driven solely by acoustic factors.
So far, we have focused on theta phase- locking. As proposed by the models
described at the beginning of this section, speech perception is governed by a hier-
archy of oscillators in which the theta oscillator is the master (Ghitza, 2011; Giraud &
Poeppel, 2012). This conjecture is based on findings in both human and nonhuman
primates showing that the phase of theta (4–8 Hz) oscillations can determine the am-
plitude of gamma (30–90 Hz) oscillations via a mechanism called phase-amplitude
coupling (Canolty et al., 2006; Canolty & Knight, 2010; Lakatos et al., 2005). Are cor-
tical oscillations hierarchically coupled during speech perception? A recent study used
MEG and information theory to shed light on that important issue (Gross et al., 2013).
Participants listened to a 7-minute long audio story, as well as its backward version. The
authors used mutual information to assess the dependencies between the speech signal
and cortical oscillations. As demonstrated by previous studies (Cogan & Poeppel, 2011),
it was found that low-frequency speech modulations entrain delta and theta oscillations
in auditory cortex. Importantly, it was also shown that low-frequency phase is coupled
to the amplitude of broadband cortical gamma (Gross et al., 2013). It was demonstrated
that speech tracking (theta-band phase-locking) as well as cross-frequency coupling
were enhanced following temporal edges in speech. These effects outlasted the typical
evoked response to edge onset (Gross et al., 2013).
The role of acoustic edges in speech entrainment has been the focus of another re-
cent investigation (Doelling, Arnal, Ghitza, & Poeppel, 2014). This study used the stim-
ulus set developed by Ghitza (2012) that was described earlier. Recall that Ghitza (2012)
showed that reinstating temporal cues by using transient edges (e.g., clicks) at landmark
positions (e.g., “missing” vowel peaks due to filtering procedures) enhanced intelligi-
bility. Doelling et al. (2014) illuminated the neural basis of this psychophysical effect
by showing that adding temporal cues to the filtered stimuli improves speech tracking.
Moreover, there was a robust correlation between speech tracking and intelligibility.
Taken together these findings suggest (1) that sharp fluctuations in the speech signal
(e.g., edges) allow slow oscillations in auditory cortex to entrain, and (2) that this en-
trainment is necessary (although not sufficient) for speech comprehension.
There is accumulating evidence supporting the notion that cortical oscillations pro-
vide a way to temporally organize the incoming connected speech signal. In accordance
with theoretical models (Ghitza, 2011; Giraud & Poeppel, 2012), psychophysical and
neural evidence suggests that theta-band oscillations track the envelope of speech and
provide a means for syllabic-based parsing. Theta-tracking of speech is likely driven by
acoustic features (such as edges), but nonetheless it also reflects speech intelligibility.
Recent studies also support the hypothesis that cortical oscillations are organized hier-
archically, such that the phase of lower-frequency oscillations is coupled to the ampli-
tude of broadband gamma oscillations. That coupling might provide a mechanism to
align neural excitability with speech modulations.
What is the appeal of bringing neuronal oscillations to the domain of speech percep-
tion? One compelling argument is that oscillations form part of a linking hypothesis
between the physics of speech, phonological attributes of speech, and the neurophys-
iological foundations of speech perception. When analyzing, cross-linguistically, the
physical amplitude modulations of speech (i.e. what arrives at the ear), there is remark-
able consistency: the modulation spectrum peaks between 4 and 5 Hz (Ding et al.,
2017). This physical attribute of the speech signal, very robust, is actually what neural
oscillations can “lock” onto, specifically when one speaks of phase-locking. The large
amplitude fluctuations provide a physics-based scaffolding to which the neurophysio-

logical machinery can connect. And it is no accident that it is theta band oscillations,
between 4 and 8 Hz, that are precisely the physiological mechanisms linking to the phys-
ical signal. The third component of this linking hypothesis is that the mean duration of
syllables, across languages, is between 150 and 300 ms. Therefore, from the point of view
of language, as well, the same time scale is foundational. In summary, syllable duration
(from linguistics), the modulation spectrum (from physics), and low-frequency brain
rhythms (from neuroscience) provide the infrastructure for a plausible theory of speech
segmentation.
References
Ahissar, E., Nagarajan, S., Ahissar, M., Protopapas, A., Mahncke, H., & Merzenich, M. M.
(2001). Speech comprehension is correlated with temporal response patterns recorded from
auditory cortex. Proceedings of the National Academy of Sciences USA, 98(23): 13367–13372.
doi: 10.1073/pnas.201400998
Allopenna, P., Magnuson, J., & Tanenhaus, M. (1998). Tracking the time course of spoken word
recognition using eye movements: Evidence for continuous mapping models. Journal of
Almeida, D., & Poeppel, D. (2013) Word-specific repetition effects revealed by MEG and the
implications for lexical access. Brain and Language, 12:497–509.
Arimitsu T., Uchida-Ota M., Yagihashi, T., Kojima, S., Watanabe, S., Hokuto, I., Ikeda, K.,
Takahashi, T., & Minagawa-Kawai, Y. (2011). Functional hemispheric specialization in
processing phonemic and prosodic auditory changes in neonates. Frontiers in Psychology,
2(202),  1–10.
Arsenault, J. S., & Buchsbaum, B. R. (2015). Distributed neural representations of phonological
features during speech perception. Journal of Neuroscience, 35(2), 634–642.
Barton, B., Venezia, J. H., Saberi, K., Hickok, G., & Brewer, A. A. (2012). Orthogonal acoustic
dimensions define auditory field maps in human cortex. Proceedings of the National
Bauer, P. R., Reitsma, J. B., Houweling, B. M., Ferrier, C. H., & Ramsey, N. F. (2014). Can fMRI safely
replace the Wada test for preoperative assessment of language lateralisation? A meta-analysis
and systematic review. Journal of Neurology, Neurosurgery, and Psychiatry, 85(5), 581–588.
Belin, P., Zilbovicius, M., Crozier, S., Thivard, L., Fontaine, A., Masure, M. C., & Samson, Y.
(1998). Lateralization of speech and auditory temporal processing. Journal of Cognitive
Binder, J. R., Frost, J. A., Hammeke, T. A., Cox, R. W., Rao, S. M., & Prieto, T. (1997). Human
brain language areas identified by functional magnetic resonance imaging. Journal of
Blumstein, S. E. (1994). Impairments of speech production and speech perception in
aphasia. Philosophical Transactions of the Royal Society of London B Biological Sciences,
346(1315),  29–36.
Blumstein, S. E., Baker, E., & Goodglass, H. (1977). Phonological factors in auditory compre-
hension in aphasia. Neuropsychologia, 15(1),  19–30.
Blumstein, S. E., Cooper, W. E., Zurif, E. B., & Caramazza, A. (1977). The perception and pro-
duction of voice-onset time in aphasia. Neuropsychologia, 15, 371–383.
Blumstein, S. E., Myers, E., & Rissman, J. (2005). The perception of voice onset time: An
fMRI investigation of phonetic category structure. Journal of Cognitive Neuroscience, 17(9),
1353–1366.
Boemio, A., Fromm, S., Braun, A., & Poeppel, D. (2005). Hierarchical and asymmetric tem-
poral sensitivity in human auditory cortices. Nature Neuroscience, 8, 389–395.
Bridgeman, E., & Snowling, M. (1988). The perception of phoneme sequence: A comparison
of dyspraxic and normal children. International Journal of Language & Communication
Disorders, 23, 245–252.
Broca, P. (1861). Remarques sur le siege de la faculté du langage articulé, suivies d’une observa-
tion d’aphémie (perte de la parole). Bulletins Et Mémoires de la Société Médicale Des Hôpitaux
de Paris, 36, 330–356.
Buzsáki, G., & Draguhn, A. (2004). Neuronal oscillations in cortical networks. Science,
304(5679), 1926–1929.
Canolty, R. T., Edwards, E., Dalal, S. S., Soltani, M., Nagarajan, S. S., Kirsch, H. E., & Knight,
R. T. (2006). High gamma power is phase-locked to theta oscillations in human neocortex.
Science, 313(5793), 1626–1628. doi: 10.1126/science.1128115
Canolty, R. T., & Knight, R. T. (2010). The functional role of cross-frequency coupling. Trends
in Cognitive Science, 14(11), 506–515. doi: 10.1016/j.tics.2010.09.001
Canolty, R. T., Soltani, M., Dalal, S. S., Edwards, E., Dronkers, N. F., Nagarajan, S. S., Kirsch, H.
E., Barbaro, N. M., & Knight, R. T. (2007). Spatiotemporal dynamics of word processing in
the human brain. Frontiers in Neuroscience, 1(1), 185–196.
Carpenter, R. L., & Rutherford, D. R. (1973). Acoustic cue discrimination in adult aphasia.
Journal of Speech and Hearing Research, 16, 534–544.
13(11), 1428–1432.
Cogan, G. B., & Poeppel, D. (2011). A mutual information analysis of neural coding of speech
by low-frequency MEG phase information. Journal of Neurophysiology, 106(2), 554–563.
Cogan, G. B., Thesen, T., Carlson, C., Doyle, W., Devinsky, O., & Pesaran, B. (2014). Sensory-
motor transformations for speech occur bilaterally. Nature, 507(7490),  94–98.
Cooper, F. S., Delattre, P., Liberman, A. M., Borst, J. M., & Gerstman, L. J. (1952). Some
experiments on the perception of synthetic speech sounds. Journal of the Acoustical Society
of America, 24(6), 597–606.
D’Ausilio, A., Pulvermüller, F., Salmas, P., Bufalari, I., Begliomini, C., & Fadiga, L. (2009). The
motor somatotopy of speech perception. Current Biology, 19, 381–385.
di Pellegrino, G., Fadiga, L., Fogassi, L., Gallese, V., & Rizzolatti, G. (1992). Understanding
motor events: A neurophysiological study. Experimental Brain Research, 91, 176–180.
Ding, N., Patel, A. D., Chen, L., Butler, L. C., & Poeppel, D. (2017) Temporal
modulations in speech and music. Neuroscience & Biobehavioral Reviews, 81, 181–187.
doi: S0149-7634(16)30566-8
Doelling, K. B., Arnal, L. H., Ghitza, O., & Poeppel, D. (2014). Acoustic landmarks drive
delta-theta oscillations to enable speech comprehension by facilitating perceptual parsing.
Drullman, R., Festen, J. M., & Houtgast, T. (1996). Effect of temporal modulation reduction on
spectral contrasts in speech. Journal of the Acoustical Society of America, 99(4), 2358–2364.
Drullman, R., Festen, J. M., & Plomp, R. (1994a). Effect of reducing slow temporal modulations
on speech reception. Journal of the Acoustical Society of America, 95(5), 2670–2680.
Drullman, R., Festen, J. M., & Plomp, R. (1994b). Effect of temporal envelope smearing on
speech reception. Journal of the Acoustical Society of America, 95(2), 1053–1064.
Du, Y., Buchsbaum, B. R., Grady, C. L., & Alain, C. (2014). Noise differentially impacts pho-
neme representations in the auditory and speech motor systems. Proceedings of the National
Academy of Sciences USA, 111(19), 7126–7 131.
Efron, R. (1963). Temporal perception, aphasia and déjà vu. Brain, 86, 403–424.
Fernandino, L., Conant, L. L., Binder, J. R., Blindauer, K., Hiner, B., Spangler, K., & Desai,
R. H. (2013). Where is the action? Action sentence processing in Parkinson’s disease.
Fodor, J. A. (1983). The modularity of mind. Cambridge, MA: MIT Press.
Fogassi, L., Ferrari, P. F., Gesierich, B., Rozzi, S., Chersi, F., & Rizzolatti, G. (2005). Parietal
lobe: From action organization to intention understanding. Science, 308(5722), 662–667.
Friederici, A. D. (2011). The brain basis of language processing: From structure to function.
Physiological Reviews, 91, 1357–1392.
Fries, P. (2005). A mechanism for cognitive dynamics: Neuronal communication through neu-
ronal coherence. Trends in Cognitive Science, 9(10), 474–480.
Gagnepain, P., Chetelat, G., Landeau, B., Dayan, J., Eustache, F. & Lebreton, K. (2008). Spoken
word memory traces within the human auditory cortex revealed by repetition priming and
functional magnetic resonance imaging. Journal of Neuroscience, 28(20), 5281–5289.
Gaskell, G., & Marslen-Wilson, W. (2002). Representation and competition in the perception
of spoken words. Cognitive Psychology, 45(2), 220–266.
Gervain, J., Macagno, F., Cogoi, S., Peña, M., & Mehler, J. (2008). The neonate brain detects
speech structure. Proceedings of the National Academy of Sciences USA, 105(37), 14222–14227.
Geschwind, N. (1970). The organization of language and the brain. Science, 170(3961), 940–944.
Ghitza, O. (2011). Linking speech perception and neurophysiology: Speech decoding guided
by cascaded oscillators locked to the input rhythm. Frontiers in Psychology, 2(130), 1–13.
doi: 10.3389/fpsyg.2011.00130
Ghitza, O. (2012). On the role of theta-driven syllabic parsing in decoding speech: Intelligibility
of speech with a manipulated modulation spectrum. Frontiers in Psychology, 3(238), 1–12.
doi: 10.3389/fpsyg.2012.00238
Ghitza, O., Giraud, A. L., & Poeppel, D. (2012). Neuronal oscillations and speech percep-
tion: Critical-band temporal envelopes are the essence. Frontiers in Human Neuroscience,
6(340), 1–4. doi: 10.3389/fnhum.2012.00340
Ghitza, O., & Greenberg, S. (2009). On the possible role of brain rhythms in speech percep-
tion: Intelligibility of time-compressed speech with periodic and aperiodic insertions of si-
lence. Phonetica, 66(1–2), 113–126.
Giraud, A. L., Kleinschmidt, A., Poeppel, D., Lund, T., Frackowiak, R., & Laufs, H. (2007).
Endogenous cortical rhythms determine cerebral specialization for speech perception and
production. Neuron, 56(6), 1127–1134.
Giraud, A. L., & Poeppel, D. (2012). Cortical oscillations and speech processing: Emerging
computational principles and operations. Nature Neuroscience, 15(4), 511–517. doi: 10.1038/
nn.3063
Greenberg, S. (2006). A multi-tier framework for understanding spoken language. In S.
Greenberg & W. Ainsworth (Eds.), Listening to speech: An auditory perspective (Vol. 1, pp.
411–433). New Jersey: Psychology Press.
Greenberg, S., Carvey, H., Hitchcock, L., & Chang, S. Y. (2003). Temporal properties of spon-
taneous speech: A syllable-centric perspective. Journal of Phonetics, 31(3–4), 465–485.
doi: 10.1016/J.Wocn.2003.09.005
Gross, J., Hoogenboom, N., Thut, G., Schyns, P., Panzeri, S., Belin, P., & Garrod, S. (2013).
Speech rhythms and multiplexed oscillatory sensory coding in the human brain. PLoS
Biology, 11(12), e1001752. doi: 10.1371/journal.pbio.1001752
Gruber, T., Tsivilis, D., Giabbiconi, C. M., & Muller, M. M. (2008). Induced electroencephalo-
gram oscillations during source memory: Familiarity is reflected in the gamma band, recol-
lection in the theta band. Journal of Cognitive Neuroscience, 20(6), 1043–1053. doi: 10.1162/
jocn.2008.20068
Guggisberg, A. G., Dalal, S. S., Findlay, A. M., & Nagarajan, S. S. (2007). High-frequency
oscillations in distributed neural networks reveal the dynamics of human decision making.
Frontiers in Human Neuroscience, 1(14), 1–11. doi: 10.3389/neuro.09.014.2007
Hauk, O., Johnsrude, I., & Pulvermüller, F. (2004). Somatotopic representations of action
words in human motor and premotor cortex. Neuron, 41, 301–307.
Herdener, M., Esposito, F., Scheffler, K., Schneider, P., Logothetis, N.K., Uludag, K., & Kayser,
C. (2013). Spatial representations of temporal and spectral sound cues in human auditory
cortex. Cortex, 49, 2822–2833.
Hickok, G. (2009). Eight problems for the mirror neuron theory of action: Understanding in
monkeys and humans. Journal of Cognitive Neuroscience, 21(7), 1229–1243.
Trends in Cognitive Science, 4(4), 131–138.
Hirano, S., Kojima, H., Naito, Y., Honjo, I., Kamoto, Y., Okazawa, H., Ishizu, K., Yonekura, Y.,
Nagahama, Y., Fukuyama, H., & Konishi, J. (1997). Cortical processing mechanism for vo-
calization with auditory verbal feedback. Neuroreport, 8, 2379–2382.
Houtgast, T., & Steeneken, H. J. M. (1985). A review of the Mtf concept in room acoustics and
its use for estimating speech-intelligibility in auditoria. Journal of the Acoustical Society of
America, 77(3), 1069–1077. doi: 10.1121/1.392224
Howard, M. F., & Poeppel, D. (2010). Discrimination of speech stimuli based on neuronal re-
sponse phase patterns depends on acoustics but not comprehension. Journal Neurophysioly,
104(5), 2500–2511. doi: 10.1152/jn.00251.2010
Hsieh, I. H., Fillmore, P., Rong, F., Hickok, G., & Saberi, K. (2012). FM-selective networks in
human auditory cortex revealed using fMRI and multivariate pattern classification. Journal
of Cognitive Neuroscience, 24, 1896–1907.
Hugdahl, K. (2011) Fifty years of dichotic listening research: Still going and going and . . . . Brain
and Cognition, 76(2), 211–213.
Hyde, K. L., Peretz, I., & Zatorre, R. J. (2008). Evidence for the role of the right auditory cortex
in fine pitch resolution. Neuropsychologia, 46, 632–639.
Jamison, H. L., Watkins, K. E., Bishop, D. V., & Matthews, P. M. (2006). Hemispheric speciali-
zation for processing auditory nonspeech stimuli. Cerebral Cortex, 16(9), 1266–1275.
Janecek, J. K., Swanson, S. J., Sabsevitz, D. S., Hammeke, T. A., Raghavan, M. E., Rozman, M.,
& Binder, J. R. (2013). Language lateralization by fMRI and Wada testing in 229 patients with
epilepsy: Rates and predictors of discordance. Epilepsia, 54(2), 314–322.
Joanisse, M. F., & DeSouza, D. D. (2014) Sensitivity of human auditory cortex to rapid fre-
quency modulation revealed by multivariate representational similarity analysis. Frontiers
in Neuroscience, 8(306),  1–10.
Kazanina, N., Phillips, C., & Idsardi, W. (2006). The influence of meaning on the perception of
speech sounds. Proceedings of the National Academy of Sciences USA, 103(30), 11381–11386.
Kerlin, J. R., Shahin, A. J., & Miller, L. M. (2010). Attentional gain control of ongoing cortical
speech representations in a “cocktail party.” Journal of Neuroscience, 30(2), 620–628.
Kilian-Hütten, N., Valente, G., Vroomen, J., & Formisano, E. (2011). Auditory cortex encodes
the perceptual interpretation of ambiguous sound. Journal of Neuroscience, 31(5), 1715–1720.
Kilner, J. M., Neal, A., Weiskopf, N., Friston, K. J., & Frith, C.D. (2009). Evidence of mirror
neurons in human inferior frontal gyrus. Journal of Neuroscience, 29(32), 10153–10159.
Kimura, D. (1967). Functional asymmetry of the brain in dichotic listening. Cortex, 3, 163–178.
Klatt, D. (1979). Speech perception: A model of acoustic phonetic analysis and lexical access.
Journal of Phonetics, 7, 279–342.
Klatt, D. (1989). Review of selected models of speech perception. In W. Marslen-Wilson (Ed.),
Lexical representation and process (pp. 169–226). Cambridge, MA: MIT Press.
Kohler, E., Keysers, C., Umiltà, M. A., Fogassi, L., Gallese, V., & Rizzolatti, G. (2002). Hearing
sounds, understanding actions: Action representation in mirror neurons. Science, 297(5582),
846–848.
Lakatos, P., Karmos, G., Mehta, A. D., Ulbert, I., & Schroeder, C. E. (2008). Entrainment of
neuronal oscillations as a mechanism of attentional selection. Science, 320(5872), 110–113.
doi: 10.1126/science.1154735
Lakatos, P., Shah, A. S., Knuth, K. H., Ulbert, I., Karmos, G., & Schroeder, C. E. (2005). An os-
cillatory hierarchy controlling neuronal excitability and stimulus processing in the auditory
cortex. Journal of Neurophysiology, 94(3), 1904–1911. doi: 10.1152/jn.00263.2005
Lalor, E. C., & Foxe, J. J. (2010). Neural responses to uninterrupted natural speech can be
extracted with precise temporal resolution. European Journal of Neuroscience, 31(1), 189–193.
doi: 10.1111/j.1460-9568.2009.07055.x
Liberman, A. M., Cooper, F. S., Shankweiler, D. P., & Studdert-Kennedy, M. (1967). Perception
of the speech code. Psychological Review, 74(6), 431–461.
Liberman, A. M., Cooper, F. S., Studdert-Kennedy, M., Harris, K. S., & Shankweiler, D. P.
(1968). On the efficiency of speech sounds. Zeitschrift Für Phonetik Sprachwiss. Kommun,
21,  21–32.
Liberman, A. M., Delattre, P., & Cooper, F. S. (1952). The role of selected stimulus-variables
in the perception of the unvoiced stop consonants. American Journal of Psychology, 65(4),
497–516.
Liberman, A. M., & Mattingly, I. G. (1985). The motor theory of speech perception revised.
Cognition, 21,  1–36.
Liebenthal, E., Binder, J. R., Spitzer, S. M., Possing, E. T., & Medler, D. A. (2005). Neural
substrates of phonemic perception. Cerebral Cortex, 15(10), 1621–1631.
Liégeois-Chauvel, C., de Graaf, J. B., Laguitton, V., & Chauvel, P. (1999). Specialization of left
auditory cortex for speech perception in man depends on temporal coding. Cerebral Cortex,
9(5), 484–496.
Liégeois-Chauvel, C., Lorenzi, C., Trébuchon, A., Régis, J., & Chauvel, P. (2004). Temporal
envelope processing in the human left and right auditory cortices. Cerebral Cortex, 14(7),
731–740.
Luce, P., & Pisoni, D. (1998). Recognizing spoken words: The neighborhood activation model.
Ear and Hearing, 19(1),  1–36.
Luo, H., Liu, Z., & Poeppel, D. (2010). Auditory cortex tracks both auditory and visual stimulus
dynamics using low-frequency neuronal phase modulation. PLoS Biology, 8(8), e1000445.
doi: 10.1371/journal.pbio.1000445
Luo, H., & Poeppel, D. (2007). Phase patterns of neuronal responses reliably discrim-
inate speech in human auditory cortex. Neuron, 54(6), 1001– 1010. doi: 10.1016/
j.neuron.2007.06.004
Luo, H., & Poeppel, D. (2012). Cortical oscillations in auditory perception and speech: Evidence
for two temporal windows in human auditory cortex. Frontiers in Psychology, 3, 170.
Maassen, B., Groenen, P., & Crul, T. (2003). Auditory and phonetic perception of vowels in
children with apraxic speech disorders. Clinical Linguistics & Phonetics, 17(6), 447–467.
Marion, M. J., Sussman, H. M., & Marquardt, T. P. (1993). The perception and production
of rhyme in normal and developmentally apraxic children. Journal of Communication
Disordorders, 26(3), 129–160.
Mazoyer, B. M., Tzourio, N., Frak, V., Syrota, A., Murayama, N., Levrier, O., Salamon, G.,
Dehaene, S., Cohen, L., & Mehler, J. (1993). The cortical representation of speech. Journal of
McClelland, J. L. & Elman, J. (1986). The TRACE model of speech perception. Cognitive
McDonald, B. C., Kean, J., & Saykin, A. J. (2011). fMRI Wada test: Prospects for presurgical
mapping of language and memory. In S. H. Faro, F. B. Mohamed, M. Law, & J. T. Ulmer
(Eds.), Functional neuroradiology (pp. 455–484). Boston, MA: Springer US.
McGettigan, C., & Scott, S.K. (2012). Cortical asymmetries in speech perception: What’s
wrong, what’s right and what’s left? Trends in Cognitive Sciences, 16(5), 269–276.
Meister, I. G., Wilson, S. M., Deblieck, C., Wu, A. D., & Iacoboni, M. (2007). The essential role
of premotor cortex in speech perception. Current Biology, 17, 1692–1696.
Mesgarani, N., Cheung, C., Johnson, K., & Chang, E.F. (2014). Phonetic feature encoding in
human superior temporal gyrus. Science, 343(6174), 1006–1010.
Millman, R. E., Woods, W. P., & Quinlan, P. T. (2011). Functional asymmetries in the represen-
tation of noise-vocoded speech. NeuroImage, 54(3), 2364–2373.
Misiurski, C., Blumstein, S., Rissman, J., & Berman, D. (2005). The role of lexical competition
and acoustic-phonetic structure in lexical processing: Evidence from normal subjects and
aphasic patients. Brain and Language, 93(1).  64–78.
Morillon, B., Lehongre, K., Frackowiak, R. S., Ducorps, A., Kleinschmidt, A., Poeppel,
D., & Giraud, A. L. (2010). Neurophysiological origin of human brain asymmetry for
speech and language. Proceedings of the National Academy of Sciences USA, 107(43),
18688–18693.
Mukamel, R., Ekstrom, A. D., Kaplan, J., Iacoboni, M., & Fried, I. (2010). Single-neuron
responses in humans during execution and observation of actions. Current Biology, 20(8),
750–756.
Myers, E., & Blumstein, S. (2008). The neural bases of the lexical effect: An fMRI investigation.
Näätänen, R., Lehtokoski, A., Lennes, M., Cheour, M., Huotilainen, M., Iivonen, A., Vainio,
M., Alku, P., Ilmoniemi, R. J., Luuk, A., Allik, J., Sinkkonen, J., & Alho, K. (1997). Language-
specific phoneme representations revealed by electric and magnetic brain responses. Nature,
385(6615), 432–434.
Nijland, L. (2009). Speech perception in children with speech output disorders. Clinical
Linguistics & Phonetics, 23(3), 222–239.
Obleser, J., Eisner, F., & Kotz, S. A. (2008) Bilateral speech comprehension reflects differential
sensitivity to spectral and temporal features. Journal of Neuroscience, 28(32), 8116–8123.
Obleser, J., Herrmann, B., & Henry, M. J. (2012). Neural oscillations in speech: Don’t be
enslaved by the envelope. Frontiers in Human Neuroscience, 6, 250.
Obleser, J., Meyer, L., & Friederici, A. D. (2011). Dynamic assignment of neural resources in au-
ditory comprehension of complex sentences. NeuroImage, 56(4), 2310–2320.
Obleser, J., Wise, R. J. S., Dresner, M. A., & Scott, S. K. (2007) Functional integration across
brain regions improves speech perception under adverse listening conditions. Journal of
Neuroscience, 27(9), 2283–2289.
Okamoto, H., & Kakigi, R. (2013). Hemispheric asymmetry of auditory mismatch nega-
tivity elicited by spectral and temporal deviants: A magnetoencephalographic study. Brain
Topography, 28(3), 471–478.
Okamoto, H., Stracke, H., Draganova, R., & Pantev, C. (2009). Hemispheric asymmetry of au-
ditory evoked fields elicited by spectral versus temporal stimulus change. Cerebral Cortex,
19(10), 2290–2297.
Overath, T., Kumar, S., Kriegstein von, K., & Griffiths, T. D. (2008) Encoding of spectral corre-
lation over time in auditory cortex. Journal of Neuroscience, 28(49), 13268–13273.
Overath, T., Zhang, Y., Sanes, D. H., & Poeppel, D. (2012). Sensitivity to temporal modula-
tion rate and spectral bandwidth in the human auditory system: fMRI evidence. Journal of
Palva, J. M., Palva, S., & Kaila, K. (2005). Phase synchrony among neuronal oscillations
in the human cortex. Journal of Neuroscience, 25(15), 3962– 3972. doi: 10.1523/
JNEUROSCI.4250-04.2005
Papanicolaou, A. C., Rezaie, R., Narayana, S., Choudhri, A. F., Wheless, J. W., Castillo, E. M.,
Baumgartner, J. E., & Boop, F. A. (2014). Is it time to replace the Wada test and put awake cra-
niotomy to sleep? Epilepsia, 55(5), 629–632.
Peelle, J. E. (2012). The hemispheric lateralization of speech processing depends on what
“speech” is: A hierarchical perspective. Frontiers in Human Neuroscience, 6(309),  1–4.
Peelle, J. E., Gross, J., & Davis, M. H. (2013). Phase-locked responses to speech in human au-
ditory cortex are enhanced during comprehension. Cerebral Cortex, 23(6), 1378–1387.
doi: 10.1093/cercor/bhs118
Petersen, S. E., Fox, P. T., Posner, M. I., Mintun, M., & Raichle, M. E. (1988). Positron emission
tomographic studies of the cortical anatomy of single-word processing. Nature, 331(6157),
585–589.
Poeppel, D. (2001). Pure word deafness and the bilateral processing of the speech code.
Cognitive Science, 21(5), 679–693.

Poeppel, D., & Hackl, M. (2008). The architecture of speech perception. In J. Pomerantz
(Ed.), Topics in integrative neuroscience: From cells to cognition (pp. 154– 180).
Cambridge: Cambridge University Press.
Poeppel, D., Idsardi, W., & van Wassenhove, V. (2008). Speech perception at the interface
of neurobiology and linguistics. Philosophical Transactions of the Royal Society of London
Biological Sciences, 363(1493), 1071–1086.
Poeppel, D., & Monahan, P. (2008). Speech perception: Cognitive foundations and cortical im-
plementation. Current Directions in Psychological Science, 17(2),  80–85.
Poeppel, D., Yellin, E., Phillips, C., Roberts, T. P. L., Rowley, H. A., Wexler, K., & Marantz, A.
(1996). Task-induced asymmetry of the auditory evoked M100 neuromagnetic field elicited
by speech sounds. Cognitive Brain Research, 4(4), 231–242.
Prabhakaran, R., Blumstein, S., Myers, E., Hutchison, E., & Britton, B. (2006). An event-
related fMRI investigation of phonological-lexical competition. Neuropsychologia, 44(12),
2209–2221.
Price, C. J. (2012) A review and synthesis of the first 20 years of PET and fMRI studies of heard
Pulvermüller, F. (2005). Brain mechanisms linking language and action. Nature Reviews
Pulvermüller, F., Huss, M., Kherif, F., Moscoso del Prado Martin, F., Hauk, O., & Shtyrov,
Y. (2006). Motor cortex maps articulatory features of speech sounds. Proceedings of the
Nationall Academy of Sciences USA, 103(20), 7865–7870.
Pulvermüller, F., Shtyrov, Y., & Ilmoniemi, R. (2005). Brain signatures of meaning access in ac-
tion word recognition. Journal of Cognitive Neuroscience, 17(6), 884–892.
Raizada, R., & Poldrack, R. (2007). Selective amplification of stimulus differences during cate-
gorical processing of speech. Neuron, 56(4), 726–740.
primates illuminate human speech processing. Nature Neuroscience, 12(6):718–724.
Remez, R. E., Rubin, P. E., Pisoni, D. B., & Carrell, T. D. (1981). Speech perception without tradi-
tional speech cues. Science, 212(4497), 947–949.
Robin, D. A., Tranel, D., & Damasio, H. (1990). Auditory perception of temporal and spec-
tral events in patients with focal left and right cerebral lesions. Brain and Language, 39(4),
539–555.
Rosch, E., Thompson, E., & Valera, F. J. (1992). The embodied mind: Cognitive science and
human experience. Cambridge, MA: MIT Press.
Rosen, S., Wise, R. J. S., Chadha, S., Conway, E. J., & Scott, S. K. (2011). Hemispheric
asymmetries in speech perception: Sense, nonsense and modulations. PLoS One, 6,
e24672.
Ross, E. D., & Mesulam, M.-M. (1979). Dominant language functions of the right hemisphere?
Prosody and emotional gesturing. Archives of Neurology, 36(3), 144–148.
Santoro, R., Moerel, M., De Martino, F., Goebel, R., Ugurbil, K., Yacoub, E., & Formisano, E.
(2014). Encoding of natural sounds at multiple spectral and temporal resolutions in the
human auditory cortex. PLoS Computational Biology, 10, e1003412.
Schönwiesner, M., Rübsamen, R., & Cramon Von, D. Y. (2005). Hemispheric asymmetry for
spectral and temporal processing in the human antero‐lateral auditory belt cortex. European
Journal of Neuroscience, 22, 1521–1528.
Schönwiesner, M., & Zatorre, R. J. (2009). Spectro-temporal modulation transfer func-

tion of single voxels in the human auditory cortex measured with high-resolution fMRI.
Schwartz, J., & Tallal, P. (1980). Rate of acoustic change may underlie hemispheric specializa-
tion for speech perception. Science, 207(4437), 1380–1381.
Scott, S. K., Blank, C. C., Rosen, S., & Wise, R. J. (2000), Identification of a pathway for intelli-
gible speech in the left temporal lobe. Brain, 123(12), 2400–2406.
Scott, S. K., & McGettigan, C. (2013). Do temporal processes underlie left hemisphere domi-
nance in speech perception? Brain and Language, 127(1),  36–45.
Shannon, R. V., Zeng, F. G., Kamath, V., & Ekelid, M. (1995). Speech recognition with primarily
temporal cues. Science, 270(5234), 303–304.
Shtyrov, Y., Hauk, O., & Pulvermüller, F. (2004). Distributed neuronal networks for encoding
category-specific semantic information: The mismatch negativity to action words. European
Steinschneider, M., Volkov, I. O., Fishman, Y. I., Oya, H., Arezzo, J. C., & Howard, M. A. (2005).
Intracortical responses in human and monkey primary auditory cortex support a temporal
processing mechanism for encoding of the voice onset time phonetic parameter. Cerebral
Cortex, 15(2), 170–186.
Tallal, P., & Piercy, M. (1975). Developmental aphasia: The perception of brief vowels and ex-
tended stop consonants. Neuropsychologia, 13,  69–74.
Telkemeyer, S., Rossi, S., Koch, S. P., Nierhaus, T., Steinbrink, J., Poeppel, D., Obrig, H., &
Wartenburger, I. (2009). Sensitivity of newborn auditory cortex to the temporal structure of
sounds. Journal of Neuroscience, 29(47), 14726–14733.
Telkemeyer, S., Rossi, S., Nierhaus, T., Steinbrink, J., Obrig, H., & Wartenburger I (2011).
Acoustic processing of temporally modulated sounds in infants: Evidence from a combined
near-infrared spectroscopy and EEG study. Frontiers in Psychology, 2, 62.
Trébuchon-Da Fonseca, A., Giraud, K., Badier, J. M., Chauvel, P., & Liégeois-Chauvel, C.
(2005). Hemispheric lateralization of voice onset time (VOT) comparison between depth
and scalp EEG recordings. NeuroImage, 27(1),  1–14.
Tucker, D. M., Watson, R. T., & Heilman, K. M. (1977). Discrimination and evocation of affec-
tively intoned speech in patients with right parietal disease. Neurology, 27(10), 947–947.
Umiltà, M. A., Kohler, E., Gallese, V., Fogassi, L., Fadiga, L., Keysers, C., & Rizzolatti, G. (2001).
I know what you are doing a neurophysiological study. Neuron, 31, 155–165.
Utman, J., Blumstein, S., & Sullivan, K. (2001). Mapping from sound to meaning: Reduced lex-
ical activation in Broca’s aphasics. Brain and Language, 79(3), 444–472.
Wada, J. (1949). A new method for the determination of the side of cerebral speech domi-
nance: A preliminary report of the intra-carotid injection of sodium amytal in man. Igaku to
Seibutsugaki, 14, 221–222.
Wang, Y., Ding, N., Ahmar, N., Xiang, J., Poeppel, D., & Simon, J. Z. (2012). Sensitivity to tem-
poral modulation rate and spectral bandwidth in the human auditory system: MEG evi-
dence. Journal of Neurophysioly, 107(8), 2033–2041.
Wernicke, C. (1874). Symptomenkomplex. Eine psychologische Studie auf anatomischer Basis.
Breslau: Cohn und Weigert.
Wilson, S. M., Saygin, A. P., Sereno, M. I., & Iacoboni, M. (2004). Listening to speech activates
motor areas involved in speech production. Nature Reviews Neuroscience, 7(7), 701–702.
Wise, R., Chollet, F., Hadar, U., Friston, K., Hoffner, E., & Frackowiak, R. (1991). Distribution of
cortical neural networks involved in word comprehension and word retrieval. Brain, 114 (Pt
4), 1803–1817.
Zatorre, R. J. (1988). Pitch perception of complex tones and human temporal‐lobe function.
The Journal of the Acoustical Society of America, 84, 566–572.
Zatorre, R. J., Belin, P., & Penhune, V. B. (2002). Structure and function of auditory
cortex: Music and speech. Trends in Cognitive Sciences, 6(1),  37–46.
Zatorre, R. J., Evans, A. C., Meyer, E., & Gjedde, A. (1992). Lateralization of phonetic and pitch
discrimination in speech processing. Science, 256(5058), 846–849.
Zatorre, R. J., & Gandour, J. T. (2008). Neural specializations for speech and pitch: Moving be-
yond the dichotomies. Philosophical Transactions of the Royal Society of London B Biological
Sciences, 363(1493), 1087–1104.
Chapter 27
Sentence Pro c e s si ng
Toward a Neurobiological Approach
Ina Bornkessel-S chlesewsky

and Matthias Schlesewsky
Introduction: Neurolinguistics
Meets Neurobiology
The past two decades have seen major changes in the neurolinguistic perspective
on sentence processing. Following an earlier shift, in which “neurolinguistic” re-
search broadened beyond its original focus on patient studies, more recent efforts
have attempted to move the field toward a neurobiologically plausible perspective.
Accordingly, neurolinguistic models of sentence processing must go beyond traditional
cognitive approaches, and instead take into account how mechanisms of language pro-
cessing are implemented by the brain (Rösler, 2011; Small, 2008). This is a challenging
enterprise: language is notoriously difficult to study from a biological perspective,
since—prima facie—it defies analysis based on detailed animal models. The challenge is
heightened even further when considering the sentence level, with its complex combi-
natory meanings.
A true neurobiology of sentence processing thus currently lies beyond our achievable
grasp. Nevertheless, substantial progress has been made in moving toward a perspective
that is informed by biological design principles and insights from basic neuroscience.
Notable developments include the following:
• Language—including sentence processing—as embedded within the broader

perception-action cycle implemented by the brain. As articulated perhaps most
clearly by motor neuroscientist Daniel Wolpert and his colleagues, the purpose
of the human brain is to allow us to effectively interact with the world around us.
In their words, which they explicitly characterize as “a motor chauvinist’s point of
Sentence Processing: Toward a Neurobiological Approach 677
view”: “the entire purpose of the human brain is to produce movement” (Wolpert,

Ghahramani, & Flanagan, 2001, p. 487). At first glance, this perspective may appear
odd when applied to language. However, as we discuss in more detail later in this
chapter, sensorimotor integration and the brain circuits that perform it have played
a major role in advancing theories of sentence processing toward a more biologi-
cally informed perspective.
• A move away from abstract representations. In accordance with the observation that
any neurobiologically plausible theory of language must be embedded in a holistic
theory of brain function, there is a clear trend away from viewing the generation
of abstract representations as the be-all and end-all of language processing (and
particularly language comprehension) in the brain. The degree to which abstract
representations—for example, an abstract conceptual representation of sentence
meaning—are needed, what they might look like, and how they are represented/
processed by the brain remains a matter of vigorous debate (see Meteyard, Cuadrado,
Bahrami, & Vigliocco, 2012, for a review). Nevertheless, many researchers in the
field would agree that the purpose of such representations—whatever their level of
embodiment or abstraction—is to inform appropriate action.
• Dual-stream anatomical models of language processing. In accordance with the ge-
neral recognition in cognitive neuroscience that brain networks, rather than iso-
lated regions, provide an appropriate level at which to study cognitive functions,
there is now a strong consensus that language processing in the brain must be
studied at the systems level, taking into account anatomical and functional connec-
tivity (Behrens & Sporns, 2012). For language—including sentence—processing,
dual-stream anatomical models are now widely recognized (Bornkessel-
Schlesewsky, Schlesewsky, Small, & Rauschecker, 2015a; Friederici, 2011; Hagoort,
2014; Hickok & Poeppel, 2007; Rauschecker, 1998; Rauschecker & Scott, 2009; see
also Hickok, Chapter 20 in this volume). However, the various accounts make dif-
ferent claims about the nature of the computational mechanisms performed by the
dual streams and the division of labor between streams (see also Skipper, Devlin, &
Lametti, 2017, for a critical view).
In the following, we first outline the anatomical foundations of current dual-stream

models of sentence processing, before turning to candidate computational mechanisms
for sentence processing in the brain. We then discuss the current state of knowledge in
regard to the time course of sentence processing in the brain, before offering some brief
conclusions and providing an outlook.
Anatomical Foundations
There is now a broad consensus that sentence processing draws upon dual cortical
streams of information processing (Bornkessel-Schlesewsky & Schlesewsky, 2013;
678 Ina Bornkessel-Schlesewsky and Matthias Schlesewsky
PMC
CS
6
IPL
40
CORE BELT
44 41 42
IFC
45
AC
STS 22
Dorsal
Anterior Posterior
Ventral
Figure 27.1. Schematic illustration of the dorsal (red) and ventral (green) speech-and
language-processing streams Numbers denote Brodmann areas and arrows depict anatomical
connectivity (black dashed arrows denote feedback connections).
Abbreviations: AC = auditory cortex; CS = central sulcus; IFC = inferior frontal cortex; IPL = inferior parietal lobule;
PMC = premotor cortex; STS = superior temporal sulcus.
Source: Reproduced, with permission, from Bornkessel-Schlesewsky et al. (2015a).
Bornkessel-Schlesewsky et al., 2015a; Friederici, 2009, 2011, 2012; Hagoort, 2014),
building on models of nonhuman primate audition (Rauschecker, 1998; Rauschecker
& Tian, 2000) and human speech processing (Hickok & Poeppel, 2000, 2004, 2007;
Rauschecker & Scott, 2009; see, in this volume, Peelle, Chapter 13, and Poeppel, Cogan,
Davidesco, & Flinker, Chapter 26). The (postero-)dorsal and (antero-)ventral streams
(see Figure 27.1) both serve to connect the auditory cortex (AC) in the superior temporal
lobe with the prefrontal cortex, showing clear homologies to the nonhuman primate
brain (for further anatomical details, see Catani & Forkel, Chapter 9 in this volume).
In nonhuman primates, the ventral and dorsal streams have been associated with au-
ditory object recognition and spatial auditory processing, respectively (Rauschecker,
1998; Rauschecker & Tian, 2000; Rauschecker, Tian, & Hauser, 1995). The ventral stream
shows a hierarchical organization, with lower levels (more proximate to core regions of
AC) tuned to basic auditory features (e.g., frequency-modulated [FM] sweeps, band-
pass noise bursts) and higher levels tuned to complex spectrotemporal patterns via
sensitivity to a combination of lower-level features (Kikuchi, Horwitz, & Mishkin, 2010;

Rauschecker et al., 1995). Such patterns can be associated with specific meanings, for
example in species-specific vocalizations (“monkey calls”) (see Rauschecker & Scott,
2009, for detailed discussion). Recent neuroimaging evidence in humans suggests a
comparable hierarchical organization of the dorsal stream, with higher levels displaying
temporal integration windows of increasing length (Hasson, Chen, & Honey, 2015;
Lerner, Honey, Silbert, & Hasson, 2011; for sentence processing, see also Bornkessel-
Schlesewsky & Schlesewsky, 2013; Bornkessel-Schlesewsky et al., 2015a).
Anatomical Models of
the Neurobiology of Sentence
Processing and Candidate
Computational Mechanisms
Beyond the acknowledgment of a dorsal-ventral division of labor, there is currently no

consensus regarding the function that each stream performs in sentence processing.
There are several competing accounts, as outlined in the following. We will first consider
general accounts of dorsal and ventral stream functionality in speech and language, be-
fore turning to proposals that focus specifically on mechanisms of sentence processing
Extrapolation from Speech Processing

By building on insights on the primate auditory system, dual-stream models of speech
processing constitute an important first step in moving toward neurobiologically plau-
sible models of speech and language. They thus provide a crucial basis for neurobiolog-
ical models of sentence processing.
The model by Rauschecker and Scott (2009) is grounded in the functions of the dual
streams in nonhuman primate audition: auditory object recognition in the ventral
stream (Rauschecker, 1998; Rauschecker & Tian, 2000; Rauschecker et al., 1995; for a
recent review of auditory object processing, see Bizley & Cohen, 2013), and auditory
spatial processing in the dorsal stream (Rauschecker & Tian, 2000; see Rauschecker,
2011, for a review). The relevance of auditory object processing for speech and language
perception appears straightforward: units such as phonemes, words, and phrases con-
stitute linguistic auditory objects, and hence their perception plausibly draws on similar
mechanisms as the perception of auditory objects in nonhuman primate audition (see
earlier discussion). Indeed, auditory object recognition in nonhuman primates shares
important characteristics with human speech processing, such as categorical perception
and categorization into abstract, meaning-based units (Tsunada, Lee, & Cohen, 2011,
2012). As in auditory object recognition, the ventral stream shows hierarchical organiza-
tion in speech perception (DeWitt & Rauschecker, 2012), and plays an important role in
the sound-to-meaning mapping (Saur et al., 2008; Scott, Blank, Rosen, & Wise, 2000).
Further converging evidence for the ventral stream’s role in conceptual processing in
humans stems from the deficits shown by patients suffering from the semantic variant
of primary progressive aphasia, which involves atrophy of the anterior temporal lobes
(e.g., Gorno-Tempini et al., 2011; Mesulam, Grossman, Hillis, Kertesz, & Weintraub,
2003), as well as from neuroimaging (see Visser, Jefferies, & Lambon Ralph, 2010, for a
recent meta-analysis), transcranial magnetic stimulation (Chiou, Sowman, Etchell, &
Rich, 2014; Pobric, Jefferies, & Lambon Ralph, 2007; see Schuhmann, Chapter 5 in this
volume), and computational modeling (Ueno, Saito, Rogers, & Lambon Ralph, 2011).
The spatial function of the dorsal stream, by contrast, seems less readily associable
with speech-and language-related functions at first glance. As suggested by Rauschecker
and Scott (2009) and Rauschecker (2011), internal (forward and inverse) models may
provide a common computational denominator between the dorsal stream’s spatial and
speech-related functions. First posited for motor control (Wolpert, Ghahramani, &
Jordan, 1995) and extended to neural models of the body and the environment (Grush,
2004), internal models are now assumed in a wide range of areas in cognitive neurosci-
ence (e.g., to account for one’s sense of agency: Blakemore, Wolpert, & Frith, 1998; theory
of mind: Koster-Hale & Saxe, 2013; language processing: Pickering & Garrod, 2007, 2013;
audiovisual integration during speech processing: van Wassenhove, Grant, & Poeppel,
2005; and social interactions: Wolpert, Doya, & Kawato, 2003). They are well suited
to explaining rapid state transitions in the world, such as the sensory consequences of
actions. According to Rauschecker and Scott (2009, p. 722), “spatial transformations
may be one example of fast adaptations used by ‘internal models,’ ” thus explaining the
commonality between the spatial function of the dorsal stream and its role in auditory-
motor mappings (mapping sounds to articulations; see Tremblay, Deschamps, & Dick,
The assumed auditory-motor function of the dorsal stream also plays a central role
in the prominent dual-stream model of speech perception developed by Hickok and
Poeppel (2000, 2004, 2007; see also Hickok, Chapter 20 in this volume). In their ap-
proach, the auditory-motor integrations performed by the dorsal stream render it
important for speech production, as well as for language learning (acquiring these
mappings). The ventral stream, by contrast, is crucial for speech perception, via its pre-
viously mentioned involvement in mapping sound to meaning. Prima facie, Hickok and
Poeppel’s approach is thus highly similar to Rauschecker and Scott’s (2009) proposal with
respect to the function of the dual auditory streams. There are, however, at least three
important differences between the two models. First, Hickok and Poeppel’s assumptions
about the anatomy of the streams is motivated functionally (i.e., by neuroimaging evi-
dence in humans), rather than based on anatomical insights from nonhuman primates.
Thus, the assumed mapping between cortical regions and streams differs from that
assumed by Rauscher and Scott: following an initial, unified cortical stage of speech pro-
cessing in AC and the middle and posterior superior temporal sulcus (STS), the ventral
stream projects via posterior middle and inferior temporal cortex (deemed the lexical
interface that maps sound onto meaning) to the anterior temporal lobe (deemed the com-
binatorial network, which allows for a predictable meaning to emerge when smaller lin-
guistic units are combined into larger ones). The dorsal stream, by contrast, projects via
the temporo-parietal junction and a region located there (area Spt, an assumed sensori-
motor interface) to posterior inferior frontal and premotor regions (the articulatory net-
work). The second substantive difference between the Hickok and Poeppel model, on the
one hand, and the Rauschecker and Scott model, on the other, concerns the role of the
dorsal stream, which is primarily conceived of as auditory-motor mapping by Hickok
and Poeppel (in contrast to Rauschecker and Scott’s broader functional view; see earlier
discussion). Third, Hickok and Poeppel’s model focuses more strongly on hemispheric
asymmetries between the two streams, positing a bilateral organization of the ventral
stream, while the dorsal stream is strongly left-lateralized.
Clearly, the functional and anatomical assumptions made by current neurobiological
models of speech processing are also potentially relevant for neurobiological models of
sentence processing. However, they do not provide a mechanistic account of sentence
processing in the brain, that is, they do not explain the neurobiological mechanisms
by which the brain combines words and morphemes to form sentences and how it
supports the complex form-to-meaning mapping at this level. Accordingly, several neu-
robiological models have been proposed that directly address language processing at
the sentence level (and beyond, in some cases). These will be the focus of the following
subsections.
The Auditory Sentence Comprehension Model

One of the first dual-streams models to make specific assumptions about sentence-level
processing was proposed by Friederici (2009), and was elaborated upon in subsequent
publications (Friederici, 2011, 2012). This model of auditory sentence comprehension
posits an additional structural-functional layer that complements the assumptions
of the previously mentioned speech-processing models. In other words, Friederici
assumes that the dorsal and ventral streams both comprise multiple anatomical sub-
pathways with dissociable functions. Let us turn first to the dorsal stream. In addition
to its role in auditory-motor mappings, it fulfills the second function of processing com-
plex syntactic structures in Friederici’s model. However, while auditory-motor integra-
tion is thought to be subserved by a projection from posterior superior temporal (pST)
regions to the inferior parietal lobule (IPL) and premotor cortex (PMC) via the superior
longitudinal fasciculus (SLF), complex syntax processing draws upon direct temporo-
frontal connectivity from pST cortex to the pars opercularis (POp) of the left inferior
frontal gyrus (BA 44) via the arcuate fasciculus (AF). Similarly, the ventral stream is
assumed to support not only sound-to-meaning mappings (via the extreme capsule,
EC), but also the processing of simple syntactic information (via the uncinate fasciculus,
UF) (for further details, see Friederici, 2012).
The dissociation between simple and complex syntax is central to Friederici’s ap-
proach and thus requires some elaboration. She uses simple syntax to refer to the local
(i.e., essentially phrase-internal) combination of word categories (e.g., determiners and
nouns to form a noun phrase such as “the bridge,” or prepositions and noun phrases
to form a prepositional phrase such as “under the bridge”). Mechanisms of complex
syntax processing, by contrast, are required in the case of word order permutations (e.g.,
object-initial sentences in languages that allow them) and self-embeddings (i.e., recur-
sion, for example in sentences including relative clauses).
Evidence for involvement of the dorsal stream in complex syntax processing stems
from neuroimaging studies showing increased, mostly left-lateralized activation of
the posterior superior temporal sulcus and/or gyrus (pSTS/STG) and of the POp for
object-versus subject-initial sentences (e.g., Ben-Shachar, Palti, & Grodzinsky, 2004;
Bornkessel, Zysset, Friederici, von Cramon, & Schlesewsky, 2005; Burholt Kristensen,
Engberg-Pedersen, Højlund Nielsen, & Wallentin, 2013; Christensen & Wallentin, 2011;
Fiebach, Schlesewsky, Lohmann, von Cramon, & Friederici, 2005; Friederici et al.,
2006; Kinno, Kawamura, Shioda, & Sakai, 2008; Röder, Stock, Neville, Bien, & Rösler,
2002) and sentences with increasing depths of self-embedding (Makuuchi, Bahlmann,
Anwander, & Friederici, 2009). The results by Makuuchi and colleagues were addition-
ally interpreted as dissociating the processing of hierarchical structures—as required
in self-embeddings—from working memory requirements (implemented via the inser-
tion of linear dependencies without any embedded clauses; see Buchsbaum, Chapter 32
in this volume). While embedded (hierarchical) dependency processing engendered
increased activation in the POp, linear dependency processing correlated with an an-
atomically separable activation increase in the inferior frontal sulcus (IFS). Within
the dorsal sub-pathway that engages in the processing of complex syntax, Friederici
envisages the division of labor among regions as follows: “BA 44 is the core region of
syntactic processes across different languages (Friederici, 2011) and tasks (Caplan,
Chen, & Waters, 2008), whereas the posterior temporal region comes into play when
argument assignment in the service of sentence interpretation is required (Bornkessel
et al., 2005; Hickok & Poeppel, 2007; Kuperberg et al., 2000; Newman, Ikuta, & Burns,
2010; Snijders et al., 2009)” (Friederici, 2012, p. 265).
The claim that complex syntax processing is specifically tied to a direct dorsal sub-
pathway that connects pST to the POp stems mainly from Friederici and colleagues’
work on language acquisition. Diffusion tensor imaging (DTI; see Catani & Forkel,
Chapter 9 in this volume) suggests that this dorsal sub-pathway, which is deemed re-
sponsible for complex syntax, is not present at birth and matures slowly (Perani et al.,
2011). This contrasts with ventral fiber tracts and the second dorsal sub-pathway, which
is deemed to be responsible for auditory-motor mapping. Consequently, until the di-
rect dorsal sub-pathway is fully matured—which Friederici and colleagues assume is
not the case until after 7 years of age—children are unable to process object-initial word
orders very accurately (cf. Dittmar, Abbot-Smith, Lieven, & Tomasello, 2008) and show
(accuracy-dependent) inferior frontal activation differences from adults during the pro-
cessing of word-order variations (Knoll, Obleser, Schipke, Friederici, & Brauer, 2012).
Younger children are therefore thought to rely more strongly on the ventral pathway
during sentence processing (Brauer, Anwander, & Friederici, 2011). A similar claim has
been made for elderly adults, based on functional magnetic resonance imaging (fMRI)
and DTI findings from sentences involving embeddings (Antonenko et al., 2013).
The Memory, Unification, and Control Model

In his neurocognitive model of language, Hagoort conceives of sentence processing
as supported by three general computational components: memory, unification, and
control (MUC; Hagoort, 2005). The memory and unification components subserve
the storage of linguistic information in long-term memory and its combination into
more complex units, respectively. They thus correspond roughly to the assumed dis-
sociation between lexicon and grammar in linguistics (Chomsky, 1965), or between an
associative memory system and a rule-based computational component in cognitive
science (Pinker, 2009). Note, however, that Hagoort subscribes to a lexicalist grammar
(cf. MacDonald, Pearlmutter, & Seidenberg, 1994; Vosse & Kempen, 2000) and
thus assumes that syntactic structures are associated with lexical items and stored in
memory as templates. Structure building therefore involves the combination (“uni-
fication”) of these templates to form larger structures and is not directly compatible
with Chomskyan notions of grammar. The third (control) component of the MUC ar-
chitecture performs a regulatory role and thus corresponds to the notion of executive
functions in cognitive psychology (see Baddeley, 2012, for a review) or cognitive control
in cognitive neuroscience (e.g., Kimberg & Farah, 1993; Miller & Cohen, 2001; for cogni-
tive control and language processing, see Novick, Trueswell, & Thompson-Schill, 2005;
Thompson-Schill, Bedny, & Goldberg, 2005; Thompson-Schill, D’Esposito, Aguirre,
& Farah, 1997). According to the MUC model, the memory component is associated
with the left posterior temporal lobe (cf. Hickok and Poeppel’s assumption of posterior
temporal areas as a lexical interface) and adjacent inferior parietal regions (Hagoort,
2013, 2014). The left inferior frontal gyrus (IFG), by contrast, is envisaged as the neural
“unification space.” It is organized along a unification gradient from antero-ventral to
postero-dorsal parts of the left IFG, with semantic unification associated with the pars
orbitalis (POr) and pars triangularis (PTr) (BA 47/45), syntactic unification with the
PTr and POp, and phonological unification with the POp and adjacent ventral PMC.
In some sense, the MUC model is thus in agreement with Friederici’s (2012) assump-
tion that the POp should be viewed as the core syntax region in the brain. However, it
contrasts with Friederici’s model in assuming a lexicalist view of syntax as described in
the preceding and in viewing syntactic unification as only one of the various types of
unification performed by the left IFG. Finally, control is thought to engage parts of dor-
solateral prefrontal cortex (DLPFC), as well as the anterior cingulate cortex (ACC) and
attention-related parietal regions.
The MUC model has not, to date, made any strong claims about a separable func-
tional significance of the dorsal and ventral streams for language processing, beyond
their dual role in connecting the memory-and unification-related nodes of the overall
language network. However, based on functional connectivity measures obtained from
resting-state fMRI (Xiang, Fonteijn, Norris, & Hagoort, 2010), Hagoort (2013) recently
argued that phonological, syntactic, and semantic processes show dissociable connec-
tivity patterns within the overall language network. Though these functional patterns
are orthogonal to—and, in the case of Xiang et al.’s (2010) findings, do not straightfor-
wardly correspond to—anatomical connectivity, Hagoort concludes from these data
that “[t]‌he dorsal pathways might be most relevant for phonological and syntactic pro-
cessing, while the ventral pathways seem to be strongly, but presumably not exclusively,
involved in connecting regions for semantic processing” (Hagoort, 2013, p. 4).
Key evidence in favor of the MUC’s signature division of labor between its tem-
poral memory component and its inferior frontal unification component stems from
an fMRI study by Snijders et al. (2009). These authors investigated the processing of
category-ambiguous words (i.e., words that can be used as a noun or a verb), either as
part of a word list or in a sentence context. While the left posterior middle temporal
gyrus (pMTG) showed increased activation for ambiguous versus unambiguous words
in word lists and sentences, the left IFG showed an ambiguity effect only for sentences.
This was viewed as evidence for the lexical role of the pMTG due to the ambiguity effect
observed at the word level, and for a role of the IFG in combinatorics, due to the ambi-
guity effect requiring a sentence context (i.e., multiple unification options, in Hagoort’s
terminology). In support of the assumed unification gradient between syntactic and
semantic information, Hagoort and Indefrey (2014) report a meta-analysis of over 100
neuroimaging (fMRI and PET) studies, which involved a manipulation of syntactic and/
or semantic processing demands at the sentence level. The results of this study suggested
that syntactic processing difficulty led to a reliable activation of BA 44/45, while se-
mantic processing difficulty was more typically associated with activation of BA 45/47.
A similar gradient was observed for the posterior temporal lobe.
A final hallmark of the MUC is that it explicitly acknowledges the need to understand
how the neurobiology of language processing interacts with broader brain systems, such
as those involved in attention (see the earlier discussion of the control component) and
theory of mind (Hagoort, 2013). The MUC thereby aims to provide a neurobiological
framework for sentence processing as embedded in a broader discourse and in relation
pragmatic demands.
The Bi-hemispheric Model
Tyler and Marslen- Wilson’s bi-
hemispheric model of language comprehension is
based on the assumption of two separable brain systems for sentence processing: a left-
lateralized fronto-temporal system for syntactic processing, and a bilateral temporo-
parietal system for semantic/pragmatic processing (Tyler & Marslen-Wilson, 2008).
Thus, like the approaches by Friederici and Hagoort, this model posits that traditional
linguistic subdomains such as syntax, semantics, and phonology provide appropriate
levels of representation for characterizing the neurobiology of language. It differs from

these, however, in its characterization of the circuits involved in the processing of each
information type, as well as in its stronger focus on hemispheric asymmetries.
Early characterizations of the bi-hemispheric model emphasized the separability of
syntactic and semantic information processing within the broad perisylvian language
network (Tyler & Marslen-Wilson, 2008). Based primarily on studies of syntactic and
semantic ambiguity processing at the sentence level (Rodd, Longe, Randall, & Tyler,
2010), semantic processing was assumed to correlate with activation in left lateral infe-
rior frontal and left anterior superior temporal regions, while syntactic processing en-
gaged both of these regions, as well as pST and inferior parietal areas. The dissociability
of these networks was supported by functional connectivity analyses. In addition, only
activations in response to syntactic ambiguity (a word category ambiguity, as in “he
noticed that landing planes frightens/frighten some new pilots”; ambiguous region in
italics, disambiguating region in bold) were modulated by dominance (i.e., the degree
to which one reading is preferred over the other). By contrast, activations in response
to semantic ambiguities (homonyms, as in “in Australia, strange barks grow on many
trees”) were unaffected by dominance. From these findings, Tyler and Marslen-Wilson
concluded that semantic and syntactic processing is accomplished by separable fronto-
temporal systems, which differ not only with regard to the regions involved and their
connectivity, but also mechanistically (i.e., while syntactic processing appears to be
preference-driven, such that the dominant reading is adopted quickly and incremen-
tally, semantic processing appears to sustain multiple readings until disambiguating
information is encountered). With reference to their extensive previous work on mor-
phological processing, Tyler and Marslen-Wilson (2008) additionally argued that
the network for syntactic processing is more strongly comparable to that observed
for the processing of regular inflectional morphology (or phonological information
resembling regular inflectional morphemes), rather than to the network for semantic
processing. Nevertheless, the syntactic and morphological networks do not appear to
be identical, as suggested by separable loci within the pMTG identified via functional
connectivity analyses seeded in the left IFG. Evidence for the left lateralization of the
networks for morphological/syntactic processing and stronger bilaterality of the net-
work for semantic processing stems mainly from patient studies, with patients with left-
hemispheric, perisylvian lesions able to show normal performance on semantic tasks
(accompanied by increased recruitment of right-hemispheric regions), but not on syn-
tactic or morphological processing tasks.
Subsequent work within the scope of the bi-hemispheric model further fleshed out
the assumed contribution of dorsal and ventral fiber tracts to the picture outlined in
the preceding. Rolheiser, Stamatakis, and Tyler (2011) examined chronic stroke patients
using DTI and employed a battery of tests designed to measure phonological, morpho-
logical, syntactic, and semantic processing. All patients had left-hemispheric lesions, but
these differed in location and extent. Voxel-by-voxel correlations between white matter
integrity and behavioral performance suggested that semantic and phonological pro-
cessing relied on ventral and dorsal fiber systems, respectively, while morphological and
syntactic processing correlated with both streams. In another patient study (Papoutsi,
Stamatakis, Griffiths, Marslen-Wilson, & Tyler, 2011), effective connectivity between
the left IFG and the left pMTG was shown to correlate with syntactic processing per-
formance. From these results, the authors concluded that a division of labor between
the dorsal and ventral streams only holds for phonology versus semantics, while mor-
phology and syntax draw on both streams.
In summary, the bi-hemispheric model resembles the MUC model in emphasizing
the importance of a network comprising the left IFG and left pMTG for syntactic pro-
cessing, with the left pMTG engaged in lexical processing, and the left IFG responsible
for sentence-level processes such as ambiguity resolution (for recent supporting evi-
dence from multivariate pattern analysis, see Tyler, Cheung, Devereux, & Clarke, 2013).
The Extended Argument Dependency Model

The (extended) Argument Dependency Model (eADM) was originally conceived as a
neurocognitive model of sentence processing that aspired toward cross-linguistic plau-
sibility (Bornkessel, 2002; Bornkessel & Schlesewsky, 2006; Schlesewsky & Bornkessel,
2004). In other words, the aim of the model was to account for the similarities and
differences in sentence processing across the world’s 7,000 languages, both from a cog-
nitive and from a brain-based perspective. More recent versions of the model have
focused more strongly on a neurobiological grounding, based on a dual-stream archi-
tecture (Bornkessel-Schlesewsky & Schlesewsky, 2013; Bornkessel-Schlesewsky et al.,
2015a). In its most recent instantiation as a neurobiological model of sentence pro-
cessing, the eADM differs from the models described in the preceding subsections in at
least two important respects:
• Neurobiological grounding of language in primate audition. The most recent

versions of the eADM posit that higher-order language processing can be grounded
in basic computational mechanisms that are already evident in (nonhuman) pri-
mate audition. In essence, this amounts to the claim that the step from nonhuman
primate audition to human language is quantitative, rather than qualitative. This
assumption is in direct disagreement with Friederici’s (2011, 2012) auditory sen-
tence comprehension model, both anatomically and functionally. In anatomical
terms, Friederici and colleagues argue that the direct dorsal pathway from pST
to the POp via the AF is not present in monkeys (based on work by Rilling et al.,
2008) and, as described earlier, matures slowly in children. The eADM, by con-
trast, questions the validity of this conclusion based on anatomical tracer studies
in monkeys (Frey et al., 2014; Romanski et al., 1999; for discussion, see Bornkessel-
Schlesewsky, Schlesewsky, Small, & Rauschecker, 2015b). Functionally, the eADM
posits that nonhuman primates are, in principle, able to process hierarchically
structured sequences, but at a lower level of sophistication (i.e., hierarchical depth)
than humans (Bornkessel-Schlesewsky et al., 2015a).
• Basic information-processing mechanisms rather than linguistic subdomains. In con-

trast to all of the models discussed in the preceding subsections, the eADM does not
assume that traditional linguistic subdomains such as syntax and semantics provide
an adequate level of description for how sentences are represented in and processed by
the brain. Rather, the model focuses on basic mechanisms of information processing
inspired by cross-linguistic research. Specifically, it posits that both the dorsal and ven-
tral streams are crucial for combining linguistic units into more complex units (e.g.,
words and morphemes into sentences), but that the two streams differ in the nature
of the combinatoric operations that they support (see later in the chapter for further
details).
The signature claim of the eADM is that the mechanisms supporting higher-order lan-
guage combinatorics are grounded in the computational properties of the dual auditory
streams: (1) an antero-ventral stream that recognizes successively more complex (feature-
rich) auditory objects and thereby performs a mapping between spectrotemporal (audi-
tory) input patterns and conceptual representations; and (2) a postero-dorsal stream that
performs sequence processing at multiple levels based on a hierarchically organized pre-
dictive coding architecture. Both mechanisms are congruent with findings on (nonhuman)
primate audition and the speech-processing model by Rauschecker and Scott (2009).
Prefrontal cortex subserves both a regulatory function (Thompson-Schill et al., 2005) and
integrates the information processed by the two streams.
In this framework, the dorsal stream performs sequence-based, non-commutative
combinatorics (i.e., AB ≠ BA). This type of combinatorics corresponds to the neces-
sity of keeping track of linear order during language comprehension, a property that is
easily illustrated in English (i.e., the sentences “Tony helped Jim” and “Jim helped Tony”
don’t mean the same thing). The assumption that the dorsal stream plays an important
role in sequencing is supported by results implicating the dorsal stream in the pro-
cessing of word order (i.e., sentence-level sequencing; see the discussion of Friederici’s
model). Evidence from Primary Progressive Aphasia (PPA) patients further supports
these claims: as shown by Wilson et al. (2011) using a sentence-picture-matching task,
structural integrity of dorsal fiber tracts in these patients correlates with their ability
to comprehend non-canonical1 English sentences (in this study: passives such as “the
boy is kicked by the girl”). Several further studies using voxel-based lesion-symptom
mapping (VLSM; see Wilson, Chapter 2 in this volume) also observed a correlation be-
tween damage in dorsal stream regions in the posterior temporal and parietal lobes and
impaired non-canonical (passive and object relative/cleft) sentence comprehension in
English (Newhart et al., 2012; Thothathiri, Kimberg, & Schwartz, 2012). Importantly,
the eADM maintains that findings such as these do not implicate the dorsal stream
as a syntax stream per se, since the activations shown by the neuroimaging studies in
1
We use the term non-canonical to describe sentences that do not adhere to the typical agent-action-
object template for English, as first described by Bever (1970).
question also rely on non-syntactic (e.g., semantic) sequencing parameters (Bornkessel

et al., 2005; Bornkessel-Schlesewsky, Schlesewsky, & von Cramon, 2009; Grewe, et al,
2005, 2006). It also posits that the mechanism in question is one of sequence processing
at multiple time scales, rather than of complex syntax processing, as dorsal stream re-
gions are also implicated in the processing of sound sequences (e.g. Scott & Wise, 2004;
Wise et al., 2001) and speech rhythm (Geiser, Zaehle, Jancke, & Meyer, 2008), as well as
sequences at the discourse level (Kandylaki et al., 2016). A parsimonious explanation
for these findings is that the dorsal stream processes sequence information in temporal
receptive windows2 of increasing length (Hasson et al., 2015; Lerner et al., 2011), thus
demonstrating that, like the ventral stream, the dorsal stream is organized hierarchically
(Bornkessel-Schlesewsky & Schlesewsky, 2013; Bornkessel-Schlesewsky et al., 2015a).
The ventral stream, by contrast, is assumed to perform non-sequence-based, commuta-
tive combinatorics (i.e., AB = BA). This assumption is motivated by several observations.
First, it builds on the observation that non-sequence-based interpretation is crucially
important in many languages of the world, as illustrated aptly by example (1) from
the Indigenous Australian language Jiwarli. The example includes a discontinuous
noun phrase, with the demonstrative “this” which specifies “pipe of mine” occurring
at the beginning of the sentence, while the remainder of the noun phrase is placed
sentence-finally.
(1) Jiwarli, Pama-Nyungan, Western Australia (Austin, 2001)

Yinha nhurra parlura-rni-nma payipa nganaju.
this.acc 2sg.erg full-caus-imper pipe.acc 1sg.dat.acc
“You fill up this pipe of mine!”
At a first glance, sentences such as (1) may appear almost impossible to process—
particularly from the perspective of a native speaker of English. However, flexible word
order is a characteristic of many of the world’s approximately 7,000 languages and, ac-
cordingly, there must be neural mechanisms available to process it. In this example,
the noun phrase “this pipe of mine” can be identified as a constituent via case marking
information, that is, all three words comprising the noun phrase bear accusative (acc)
marking.3 The eADM proposes that non-sequential information types, such as case
marking, serve as grouping cues in the identification of linguistic auditory objects, in
much the same way that acoustic cues such as harmonicity subserve the identification of
non-linguistic auditory objects (Bizley & Cohen, 2013). Accordingly, the composition of
larger linguistic units such as phrases—and likely even some highly frequent, templatic
sentence forms—can be viewed as a more complex form of auditory object recognition.
2 Temporal receptive windows correspond to receptive fields in the visual system, but, as their
name suggests, are defined in terms of time rather than space. They define the time scales within which
auditory information is integrated (see also Poeppel, 2003).
3 Note, however, that not all free word order languages have morphological case marking.
The eADM’s conceptualization of the ventral stream thus provides a parsimonious ex-
planation for this stream’s involvement in conceptual processing (see earlier discussion)
and in basic combinatorics (e.g., Bemis & Pylkkänen, 2011, 2013a; Brennan et al., 2012).
The studies by Bemis and Pylkkänen demonstrate left aTL involvement in the composi-
tion of minimal phrases such as “red boat”; they further support the eADM’s notion of
commutative ventral stream combinatorics, by showing that—at least in the context of a
task requiring compositional processing—left aTL activation is observable irrespective
of noun-adjective order (i.e., for both “red boat” and “boat red”; Bemis & Pylkkänen,
2013b). Likewise, findings from semantic PPA patients indicate that aTL combinatorics
are non-sequence-based: recall from the earlier discussion that these patients show
pronounced anterior temporal lobe atrophy; nevertheless, they show a high accuracy
in the comprehension of non-canonically ordered sentences such as passives (Wilson
et al., 2014). This contrasts markedly with the previously mentioned result that PPA
patients with damage to dorsal fiber tracts are impaired in their comprehension of such
sentences (Wilson et al., 2011). Strikingly, recent VLSM results from Icelandic, a lan-
guage with a rich case-marking system, showed that lesions in the left aTL correlated
with deficits in the processing of non-canonically ordered sentences (passives, object
clefts, and object wh-questions) (Magnusdottir et al., 2013; see also Kim, Jeon, & Lee,
2010, for similar results from Korean, another case-marking language). Taken together,
the neuroimaging results and patient data support the eADM’s view of commutative
combinatorics in the ventral stream.
Extension to Reading
The reader will have noted that the discussion in the preceding subsections has focused
almost exclusively on auditory sentence processing. At present, an extension of these
insights to reading is—unfortunately—not possible. Though a number of neurobio-
logical models of reading have been put forward (for recent discussions, see Carreiras,
Armstrong, Perea, & Frost, 2014; Dehaene & Cohen, 2011; Price & Devlin, 2011; see also
Paz-Alonso, Oliver, Quiñones, & Carreiras, Chapter 24 in this volume), our under-
standing of reading in the brain is currently limited to the word level. A challenge for
future research thus lies in synthesizing insights on the neurobiology of visual word rec-
ognition with those on the neurobiology of sentence processing.
The Time Course of Sentence

Processing in the Brain
Time course considerations have been the bread and butter of sentence processing
theories for over three decades. They defined foundational debates in the field, such as
the question of how and when different information sources are utilized during the com-
prehension of a sentence (modularity versus interactivity; see, e.g., Frazier, 1987; Frazier
& Clifton, 1996; MacDonald et al., 1994; Rayner, Carlson, & Frazier, 1983; Trueswell &
Tanenhaus, 1994). The advent of neurobiological models, however, has raised the ques-
tion of how cognitive questions such as “is syntactic information initially prioritized
over semantic information?” or “is information source X processed pre-or post-
lexically (i.e., prior to or following lexical access)” can be framed in neurobiologically
plausible terms (for discussion in relation to the time course of sentence processing, see
Bornkessel-Schlesewsky, Staub, & Schlesewsky, 2016; for discussion of the notion of a
mental lexicon, see Kutas & Federmeier, 2011).
Challenges to the development of a truly neurobiological perspective on the time
course of sentence processing are manyfold. They include the following:
• Difficulties in mapping cognitive and neural architectures onto one another. In neu-
robiological terms, neither the strictly serial and encapsulated perspective of mod-
ular models nor the “everything is connected to everything” perspective of strongly
interactive models (often with connectionist implementations) is plausible. Rather,
neural streams of information processing (see earlier discussion) show a hierar-
chical organization, encompassing properties of both serial and parallel organization
(Rauschecker & Scott, 2009). Ultimately, however, connectivity is considerably more
complicated, involving both feedforward and feedback projections, and many, but
not all connections are bidirectional (for detailed discussion in relation to the visual
processing streams, see Kravitz, Saleem, Baker, & Mishkin, 2011; Kravitz, Saleem,
Baker, Ungerleider, & Mishkin, 2013). Moreover, hierarchical organization does not
imply “serial” connectivity in the sense that a level n of the hierarchy is only connected
to levels n + 1 and n – 1. Rather, long-range (feedforward and feedback) connections
can “skip” individual levels of the hierarchy (for possible implications, see Bar, 2003).
• Methods currently available to study the time course of language processing in the
brain. Electroencephalography (EEG; see Leckey & Federmeier, Chapter 3 in this
volume) and Magnetoencephalography (MEG; see Salmelin, Kujala, & Liljeström,
Chapter 6 in this volume) are typically used for this purpose in view of their ex-
quisite temporal resolution. However, they provide rather macroscopic meas-
ures of brain activity. Functional magnetic resonance imaging (fMRI; see Heim &
Specht, Chapter 4 in this volume), by contrast, does not offer the temporal resolu-
tion required for the question under consideration. Electrocorticography (ECoG;
i.e., the recording of electrophysiological activity directly from the cortex using
subdural electrodes; see, e.g., Chang et al., 2010; Martin, Millán, Knight, & Pasley,
in press, for applications to language), provides high resolution in both the tem-
poral and the spatial domains. However, ECoG studies can only be performed with
patients, and the electrode grids used typically only offer limited coverage as elec-
trode placement is determined by clinical motivations.
• Discrepancies between time course estimates obtained from neurophysiological
and behavioral measures. As has been observed repeatedly (e.g., Bornkessel &
Schlesewsky, 2006; Kretzschmar, Schlesewsky, & Staub, 2015; Rayner & Clifton,
2009; Sereno & Rayner, 2003), time course estimates differ between event-related
brain potentials (ERPs) and eye movements during reading, the two techniques
that have long been the methods of choice for examining timing information in the
psycholinguistic literature on sentence processing. For a recent discussion in the
context of the neurobiology of sentence processing, see Bornkessel-Schlesewsky
et al. (2016).
Differing perspectives
The time course of sentence processing in the brain is subject to substantial debate,
without a consensus view yet having emerged. In the following, we will thus discuss a
variety of perspectives. These will be based primarily on ERP data, which is still the most
widely used neurocognitive method for studying the time course of sentence processing.
Component Mapping and Absolute Timing

Originally, ERP-based research into the time course of sentence processing closely
mirrored the cognitive perspective that had been developed on the basis of behavioral
results. Thus, ERP components such as the N400 and P600 were viewed as correlates
of linguistic domains (semantic and syntactic processing, respectively) and the rela-
tive timing of these components, as well as their interaction or mutual independence,
was thus thought to shed light on the organization of the language architecture (e.g.
Friederici, Pfeifer, & Hahne, 1993; Hagoort, Brown, & Osterhout, 1999; Osterhout &
Nicol, 1999; among many others).
This perspective forms part of timing assumptions set out as part of Friederici’s
neurocognitive model of auditory sentence processing (Friederici, 1995, 1999, 2002,
2009, 2011) and Hagoort’s MUC model (Hagoort, 2003, 2005, 2013) (see earlier dis-
cussion for the anatomical assumptions of these models). In the spirit of the classic
modularity versus interactivity debate, the neurocognitive model posits that different
information sources are processed in several successive processing stages, which are
organized serially, while the MUC model advocates an interactive processing archi-
tecture. In each case, evidence for the respective view was provided by an assessment
of ERP component timing. Both models subscribe to the view that the N400 and
left-anterior negativity (LAN) effects are related to semantic and (morpho-)syntactic
aspects of processing, respectively. The MUC model also posits a syntactic role for the
P600, while the neurocognitive model assumes that the P600 reflects the late integra-
tion of syntactic and semantic information that was processed separately in preceding
phases. By contrast, the MUC model assumes that there is no priority for any one infor-
mation source over another during sentence interpretation (“one-step” interpretation)
and that the relative timing of when a particular information source is used depends
on its availability in the input signal (Hagoort, 2008). The one-step-interpretation as-
sumption of the MUC model is supported by results showing that sentence-internal and
sentence-external information all modulate the N400 component, with no evidence for
a different time course depending on the source of the information (see Hagoort & van
Berkum, 2007, for a detailed review of the relevant studies).
As in the cognitive domain, the debate between modular and interactive
neurocognitive models of sentence processing remains unresolved, even after several
decades of work on this subject. Furthermore, continuing progress in the field has re-
vealed challenges to some of the basic assumptions underlying the component mapping
approach:
Absolute timing. Several observations have called into question the assumption
(a)
that ERP component latencies provide absolute estimates of timing informa-
tion. The first stems from measures of eye movements during reading, which, as
briefly noted earlier, sometimes provide different timing estimates to ERPs. This
is perhaps most readily apparent with regard to the N400 component, which typ-
ically shows an onset latency of a little more than 200 milliseconds (ms) in young
adults (Kutas & Iragui, 1998, report an average onset latency of 236 ms for adults
in their twenties) and is highly stable in terms of its timing (e.g., Federmeier &
Laszlo, 2009). The N400 is reliably modulated by word frequency (at least for
words occurring early on in a sentence; Van Petten & Kutas, 1990), a factor that
is also known to affect the duration of the first fixation on a word during reading
(e.g., Inhoff & Rayner, 1986). Since fixations last approximately 200–250 ms on av-
erage for English (Rayner, 1998) and saccade programming is thought to require
approximately 150–175 ms (Rayner, Slowiaczek, Clifton, & Bertera, 1983), word
frequency appears to influence eye movements at a point in time that precedes the
onset of the N400 (for discussion, see Bornkessel & Schlesewsky, 2006; Rayner &
Clifton, 2009; Sereno & Rayner, 2003). This suggests that latencies of components
such as the N400 and P600 may not be reliable indicators of when a particular in-
formation source first becomes available or when it is first utilized during sentence
processing.
A second and related observation stems from the ERP domain itself. If com
ponents such as the N400 do not provide absolute timing information, perhaps
markers of this information should be sought earlier on in the ERP record. Indeed,
there have been a number of reports of language-related ERP effects “before the
N400” (e.g., Dien, 2009). These include word frequency-based modulations of
the N100 (Sereno, Rayner, & Posner, 1998) and modulations of the mismatch neg-
ativity (MMN), which peaks between approximately 100 and 250 ms post-stim-
ulus-onset, by morphosyntactic violations presented as part of an oddball design
(e.g., Hasting, Kotz, & Friederici, 2007). Recent MEG studies have also provided
converging evidence for early effects, for example by demonstrating a dissocia-
tion of brain responses to words and pseudowords from as early as 50 ms onward
(MacGregor, Pulvermüller, van Casteren, & Shtyrov, 2012) or a modulation of the

visual M100 by syntactic (Dikker, Rabagliati, & Pylkkänen, 2009) and lexical-
semantic violations (Dikker & Pylkkänen, 2011) in highly predictive contexts.
Possibly, findings such as these should be taken to indicate that ERPs and event-
related magnetic fields (ERFs) do provide absolute timing information once early
components—rather than “late” components such as the N400—are taken into
account. We shall return to this perspective later.
1:1 component mapping. Though highly appealing, the notion that language-
(b)
related ERP components can be used as diagnostic indicators of particular lin-
guistic or cognitive domains (e.g., the N400 as a correlate of lexical-semantic
processing or declarative memory contributions to language [Ullman, 2001]
versus the LAN/P600 as correlates of morphosyntactic processing or procedural
memory) is now challenged by a continually increasing number of findings. On
the one hand, the occurrence of monophasic P600 effects for certain types of se-
mantic violations—particularly violations in which a plausible sentence would
obtain if thematic role assignments to arguments were reversed (e.g., “The hearty
meals were devouring . . .”; Kim & Osterhout, 2005)—has been discussed in-
tensely in the literature on the electrophysiology of language over the past decade
(e.g., Hoeks, Stowe, & Doedens, 2004; Kim & Osterhout, 2005; Kolk, Chwilla,
van Herten, & Oor, 2003; Kuperberg, Sitnikova, Caplan, & Holcomb, 2003;
among many others; for reviews, see Bornkessel-Schlesewsky & Schlesewsky,
2008; Brouwer, Fitz, & Hoeks, 2012; van de Meerendonk, Kolk, Chwilla, &
Vissers, 2009). In addition, many studies employing “traditional” semantic
violations in the style of Kutas and Hillyard’s famous “He spread the warm bread
with socks” (Kutas & Hillyard, 1980) have observed N400s followed by P600
effects, and P600 effects have also been observed in response to other types of
incongruencies such as orthographic violations (Vissers, Chwilla, & Kolk, 2006),
thus leading several groups to suggest more general interpretations of the P600
(e.g., as a correlate of “conflict monitoring”: Kolk et al., 2003; van de Meerendonk
et al., 2009) or an instance of the domain-general P300 (e.g., Coulson, King,
& Kutas, 1998; Gunter, Stowe, & Mulder, 1997; Sassenhagen, Schlesewsky, &
Bornkessel-Schlesewsky,  2014).
While less well-known than semantic P600s, syntax-or morphosyntax-related
N400 effects have also been reported in a range of studies, for example for case
violations (Choudhary, Schlesewsky, Roehm, & Bornkessel-Schlesewsky, 2009;
Frisch & Schlesewsky, 2001, 2005; Mueller, Hahne, Fujii, & Friederici, 2005;
Mueller, Hirotani, & Friederici, 2007), gender incongruencies (Wicha, Moreno,
& Kutas, 2004), and word order variations (Bornkessel, McElree, Schlesewsky,
& Friederici, 2004; Haupt, Schlesewsky, Roehm, Friederici, & Bornkessel-
Schlesewsky, 2008). In summary, all of these results suggest that ERP components
do not reflect a clear-cut functional divide between linguistic subdomains and
that somewhat more general explanations may rather be required.
Problems with Absolute Timing: MMN Evidence

for Early, Cascaded Information Processing
As already discussed in some detail, there is considerable evidence to suggest that the
latency of classic language-related ERP components such as the N400, LAN, and P600
does not provide an adequate estimate of absolute time-course information. Within
the neurocognitive domain, this perspective has been advocated for some time by
Pulvermüller and colleagues, drawing primarily on experimental paradigms using
the mismatch negativity (MMN). The MMN typically occurs in response to deviant
stimuli in oddball paradigms (i.e., stimuli that occur infrequently within sequences of
high probability “standards”; for reviews, see Garrido, Kilner, Stephan, & Friston, 2009;
Kujala, Tervaniemi, & Schröger, 2007). Traditionally, MMN effects are elicited using
relatively simple stimuli (e.g., tone sequences), but Pulvermüller and colleagues have
extended this line of research to considerably more complex stimuli, including min-
imal phrases such as “we come” or, as a violation condition, “we comes” (Pulvermüller
& Shtyrov, 2003; Shtyrov & Pulvermüller, 2002). In comparison to the same word
presented out of context (i.e., “comes” preceded by a non-linguistic noise mimicking the
spectral envelope of the word “we”), the ungrammatical string elicited a larger MMN
than the grammatical string between 100 and 150 ms after the acoustic divergence point
between the two strings (for morphosyntactic MMN effects in German, see Hasting
et al., 2007). Semantic effects have also been observed (Menning et al., 2005; Shtyrov &
Pulvermüller, 2007).
Studies such as these have been taken to suggest that linguistic information sources
such as syntax and semantics are processed in “near simultaneity” and in considerable
earlier time windows than traditional language-related ERP components such as the
N400 suggest (Pulvermüller, Shtyrov, & Hauk, 2009). The supporting evidence from
MMN paradigms is sometimes viewed as problematic due to the highly repetitive na-
ture of the stimulation and the lack of lexical variation (i.e., an experiment may con-
sist simply of a large number of repetitions of “he walks” and “he walk,” with one the
standard and one the deviant). However, since syntactic or semantic violation deviants
are always contrasted against grammatical deviants in these paradigms, the linguistic
properties of the stimulus obviously play at least some role in engendering the larger
MMN response to incongruous stimuli. This could be explained by assuming that the
first word in a minimal phrase such as “he walks” sets up certain expectations with re-
gard to possible continuations that are commensurate with the prior experience of the
language-processing system. Clearly, this does not preclude additional expectations
or predictions being established by the intra-experimental probability of the different
stimuli (i.e. that a standard is more likely to occur than a deviant), but it appears entirely
plausible as an additional source of prediction (for an account of the MMN within a
predictive-coding-based framework, see Friston, 2005; Garrido et al., 2009).
In summary, a perspective advanced most prominently by Pulvermüller and
colleagues (among other researchers) assumes that ERP/ MEG components are
informative with regard to absolute time course information, but proposes that the
relevant effects occur in a time range prior to 200 ms and, therefore, precede typical
language-related components such as the N400. Since these early effects are thought
to arise from the near-simultaneous activation of word-specific cortical circuits, the
differing conduction delays within which account for the apparent cascade of effects,
this account essentially abolishes the idea of component mapping. As we have suggested
here, however, predictive processing may also play an important role in engendering—
or at least modulating—the effects under discussion, and this aspect may help to link the
relevant explanations to sentence-level processing.
Predictions, Predictions, Predictions

The importance of top-down predictions is emphasized more strongly in an ap-
proach that has been championed by Dikker, Pylkkänen, and colleagues. Using MEG,
these researchers have repeatedly shown that sentence-level (syntactic and semantic)
incongruities engender modulations of the visual M100, a component that is typi-
cally localized to visual cortex (Dikker & Pylkkänen, 2011; Dikker et al., 2009; Dikker,
Rabagliati, Farmer, & Pylkkänen, 2010). In the syntactic domain, these early effects have
been observed for word category violations (e.g., “the boys heard Joe’s about . . .” or “the
tastelessly soda . . .”). In the semantic domain, they have been demonstrated using a
picture-word matching paradigm (e.g., the noun phrase “the apple” following an image
of an apple (+predictive, +match), a picture of a banana (+predictive, –match), a picture
of a shopping bag with many items (–predictive, +match) or a picture of Noah’s ark with
many animals (–predictive, –match).
The early latency of the M100, as well as its likely source localization, suggests that
the effects in question may be indexing prediction mismatches at a rather early, sensory
level. In other words, if the predictive context is strong enough, predictions extend not
only to abstract features of an upcoming linguistic item (e.g.. word class, semantic cate-
gory, etc.), but also to its sensory properties (i.e., a concrete word form). Similar effects
have been observed in the auditory domain, as demonstrated by the so-called phonolog-
ical mapping negativity (PMN). The PMN, previously labeled phonological mismatch
negativity and analyzed by some as an instance of the N200 component (van den Brink,
Brown, & Hagoort, 2001, 2006), is a negativity with a peak latency of approximately
250–300 ms, which occurs in response to phonologically unexpected input (Connolly &
Phillips, 1994). As recently demonstrated by Newman and Connolly (2009), the PMN is
independent of whether the current input item is a word or a nonword, thus suggesting
that it reflects a purely form-based expectation mismatch. At least to some degree, an ac-
count of language-related ERP components that focuses on sensory predictability shifts
the burden of explanation for observed ERP or ERF effects from the current input item
to the preceding linguistic context. In other words, when evaluating time-course infor-
mation, the properties of the current stimulus item may be less important than the pre-
dictive capacity of the context in which it is encountered.
This perspective is highly compatible with a hierarchically organized predictive

coding architecture (Friston, 2005, 2010), in which predictions are passed from higher
to lower areas within the cortical hierarchy and, depending on their specificity, can lead
to prediction errors at any hierarchical level. Thus, the prediction error signal itself is
not necessarily diagnostic of any particular functional interpretation, but rather varies
as a function of where within the overall architecture the prediction error is registered.
Multimodal Processing and the Integration of Multiple

Information Sources
The notion of prediction also plays a crucial role in an account of language-related ERP
components, and specifically the N400, which abstracts away from time-course infor-
mation to a relatively large degree (Kutas & Federmeier, 2011). This perspective builds
on the observation that (a) the N400 is not parsimoniously described in relation to lex-
ical (and pre-lexical, post-lexical, etc.) processing, since, for example, it also responds to
nonword stimuli as long as they are meaningful in a given context (Federmeier & Laszlo,
2009, for an overview); and (b) the N400 is remarkably stable with respect to its latency
(see earlier discussion). Accordingly, Federmeier, Kutas, and colleagues argue that the
N400 should be viewed as indexing a period of convergence between multiple, multi-
modal information sources, during which meaning is constructed in a dynamic and
context-dependent manner. In other words, the N400 indexes the “transition from uni-
modal to multimodal processing (e.g., from word form to the concept that word brings
to mind)” and “reflect[s]‌the activity in a multimodal long-term memory system that
is induced by a given input stimulus during a delimited time window as meaning is dy-
namically constructed” (Kutas & Federmeier, 2011, p. 640).
This perspective, too, emphasizes the crucial role of prediction during language pro-
cessing, since linguistic context is assumed to pre-activate particular linguistic features
as well as their combination. This pre-activation process modulates the dynamic con-
struction of meaning, as described earlier, and hence impacts on N400 amplitude. Like
the perspectives discussed in the preceding sections, it thus de-emphasizes the notion
of component mapping and, though it does not focus on early language-related ERP
components, it shares with Pylkkänen and colleagues the assumption that predic-
tive processing plays a crucial role in language comprehension. In view of the focus
on the N400 and the fixed latency of this component, however, detailed time-course
considerations do not feature as part of this account.
The Interplay Between Top-Down and Bottom-Up

Information Sources
In addition to the focus on prediction that has been central to the discussion in
the preceding subsections, a range of findings now demonstrate that top- down
predictability or “pre-activation” is not the sole determinant of neurophysiological

effects during sentence comprehension. Rather, the “balance” between top-down and
bottom-up influences needs to be taken into account. Once again, a great deal of the
relevant evidence stems from the N400 ERP component. A recent finding which shows
that N400 amplitude is modulated by properties of the current stimulus, rather than
the preceding context, was reported by Laszlo and Federmeier (2011). They showed
that N400 amplitudes are higher for input items—both words and letter strings—with
more or higher frequency orthographic neighbors and lexical associates. This result also
provides compelling evidence against a view of the N400 as a correlate of lexical access
or post-lexical processing, since accounts along these lines should predict differences
between word and nonword stimuli. The need to consider the integration of top-down
and bottom-up information was further emphasized by Lotze, Tune, Schlesewsky, and
Bornkessel-Schlesewsky (2011), who found that an unpredictable physical change from
normal orthography to a capitalized sentence-final word almost completely neutralized
an implausibility-based N400 effect on that word, while a change from all capitalized
input to a lower-case sentence-final word had no effect on the N400. Clearly, the atten-
uation of the N400 must be attributable to bottom-up stimulus features in this study,
since top-down factors did not change between conditions. A somewhat similar result
was reported by Delaney-Busch and Kuperberg (2013), who found that N400 effects for
incongruous scenarios were neutralized when the critical word was an emotion word
(of positive or negative valence). A possible explanation for these results is that, in line
with Bar’s (2003) assumptions concerning visual object recognition (see also earlier dis-
cussion in this chapter), very basic and salient stimulus features (e.g., sudden capitaliza-
tion as a visual “pop out” or high emotional valence) are rapidly projected to prefrontal
cortex (or other higher-level regions within the cortical hierarchy) via long-range feed-
forward connections, thus enabling these features to lead to a top-down modulation of
plausibility or congruity effects.
If this proposal indeed turns out to be viable, it could provide the basis for a new ap-
proach for inferring the organization of the neural language architecture. Rather than
basing arguments on temporal precedence, investigations should focus more closely on
functional precedence: if an information source A can “block” the effects typically asso-
ciated with a second information source B, this suggests that A takes functional (hierar-
chical) precedence over B. Depending on the particular type of manipulation examined,
further inferences can be drawn regarding the likely nature of the blocking effect (i.e.,
whether it results from a feedforward influence, as is likely the case for concurrent word
category and semantic violations, or from a combination of long-range feedforward and
feedback connections, as may be the case for the orthographic and emotional influences
on congruity effects).
The Emerging Picture

As noted earlier, our understanding of the time course of sentence processing in the
brain is still woefully incomplete and, in particular, we have a long way to go in advancing
toward a truly neurobiologically grounded perspective in this respect. Nevertheless, the

preceding discussion shows that substantial progress has been made. Here, we attempt
to summarize some of the insights emerging from this overall picture:
(a) ERP components appear to be less specific for particular cognitive—and specif-

ically, particular linguistic—representations and/or operations than was once
thought. Accordingly, more and more accounts are adopting more general mech-
anistic interpretations of the components and no longer are emphasizing the dis-
tinction between them, as was previously the norm.
(b) The importance of top-down versus bottom-up information has emerged as
a major topic in the literature on sentence processing (e.g., Federmeier, 2007;
Federmeier & Laszlo, 2009; Kutas & Federmeier, 2011; Lotze et al., 2011; Newman
& Connolly, 2009; Tune et al., 2014). This view includes a critical role for predic-
tion (DeLong, Urbach, & Kutas, 2005; Dikker & Pylkkänen, 2011; Dikker et al.,
2010; Federmeier, Kutas, & Schul, 2010) and is also highly compatible with the
assumption of a hierarchically organized cortical architecture with predictive
coding and asymmetric forward and backward connections (Friston, 2005, 2010)
What are the consequences of these insights for our understanding of the time course of
sentence processing in the brain? While the question remains difficult, we can draw at
least the following minimal conclusion. If we want to think in cognitive terms, there is a
high likelihood of a cascaded architecture (though an “imperfect” one, due to the feed-
back connections and long-range feedforward connections described throughout this
chapter). This also means that specific information sources are less important than was
long thought for the processing of the current input item in comparison to their pre-
dictive capacity for upcoming input. This view allows for a new synthesis of the classic
modular and interactive views: it suggests that modular models placed too much em-
phasis on bottom-up information and a feedforward information flow, while interactive
models placed too much emphasis on top-down information and a parallel information
flow between all available information sources or representations.
Summary and Outlook
In this chapter, we have attempted to summarize the current state of the art with regard
to sentence processing in the brain. As will be readily apparent to the reader, there is
much left to do in this domain. Nevertheless, it is hopefully also clear that the field of
sentence processing has come a long way since its inception. While early approaches
focused mainly on testing the assumptions of linguistic theories (e.g., Clifton, Kurcz,
& Jenkins, 1965; Mehler, 1963; Miller, 1962), later perspectives were concerned with
uncovering more general cognitive-processing mechanisms (e.g., Frazier, 1987; Gibson,
1998; Lewis, 1996; Lewis, Vasishth, & Van Dyke, 2006; MacDonald et al., 1994; McElree,
1993, 2006; Rayner et al., 1983; Trueswell & Tanenhaus, 1994). For neurolinguistics, the
challenge now lies in developing true neurobiological models of sentence processing
that will explain how it emerges from basic principles of neural information processing
in both time and space.
References
Antonenko, D., Brauer, J., Meinzer, M., Fengler, A., Kerti, L., Friederici, A. D., & Flöel, A. (2013).
Functional and structural syntax networks in aging. NeuroImage, 83, 513–523.
Austin, P. (2001). Word order in a free word order language: The case of Jiwarli. In J. Simpson,
D. Nash, M. Laughren, P. Austin, & B. Alpher (Eds.), Forty years on: Ken Hale and Australian
languages (pp. 205–323). Canberra: The Australian National University (Pacific Linguistics).
Baddeley, A. (2012). Working memory: Theories, models, and controversies. Annual Review of
Bar, M. (2003). A cortical mechanism for triggering top-down facilitation in visual object rec-
ognition. Journal of Cognitive Neuroscience, 15, 600–609.
Behrens, T. E., & Sporns, O. (2012). Human connectomics. Current Opinion in Neurobiology,
22, 144–153.
Bemis, D. K., & Pylkkänen, L. (2011). Simple composition: A magnetoencephalography inves-
tigation into the comprehension of minimal linguistic phrases. Journal of Neuroscience, 31,
2801–2814.
Bemis, D. K., & Pylkkänen, L. (2013a). Basic linguistic composition recruits the left anterior
temporal lobe and left angular gyrus during both listening and reading. Cerebral Cortex, 23,
1859–1873.
Bemis, D. K., & Pylkkänen, L. (2013b). Flexible composition: MEG evidence for the deploy-
ment of basic combinatorial linguistic mechanisms in response to task demands. PLoS One,
8, e73949.
Ben-Shachar, M., Palti, D., & Grodzinsky, Y. (2004). Neural correlates of syntactic move-
ment: Converging evidence from two fMRI experiments. NeuroImage, 21, 1320–1336.
Bever, T. G. (1970). The cognitive basis for linguistic structures. In R. Hayes (Ed.), Cognition
and Language Development (pp. 279–362). New York: John Wiley & Sons.
Bizley, J. K., & Cohen, Y. E. (2013). The what, where and how of auditory-object perception.
Nature Reviews Neuroscience, 14, 693–707.
Blakemore, S.-J., Wolpert, D. M., & Frith, C. D. (1998). Central cancellation of self-produced
tickle sensation. Nature Neuroscience, 1, 635–640.
Bornkessel, I. (2002). The argument dependency model: A neurocognitive approach to incre-
mental interpretation. Leipzig: Max Planck Institute of Cognitive Neuroscience Leipzig.
Bornkessel, I., McElree, B., Schlesewsky, M., & Friederici, A. D. (2004). Multi-dimensional
contributions to garden path strength: Dissociating phrase structure from case marking.
Journal of Memory and Language, 51, 495–522.
Bornkessel, I., & Schlesewsky, M. (2006). The extended argument dependency model: A
neurocognitive approach to sentence comprehension across languages. Psychological
Review, 113, 787–821.
Bornkessel, I., Zysset, S., Friederici, A. D., von Cramon, D. Y., & Schlesewsky, M. (2005). Who
did what to whom? The neural basis of argument hierarchies during language comprehen-
sion. NeuroImage, 26, 221–233.
Bornkessel-Schlesewsky, I., & Schlesewsky, M. (2008). An alternative perspective on “semantic

P600” effects in language comprehension. Brain Research Reviews, 59,  55–73.
Bornkessel-Schlesewsky, I., & Schlesewsky, M. (2013). Reconciling time, space and func-
tion: A new dorsalventral stream model of sentence comprehension. Brain and Language,
125,  60–76.
Bornkessel-Schlesewsky, I., Schlesewsky, M., & von Cramon, D. Y. (2009). Word order and
Broca’s region: Evidence for a supra-syntactic perspective. Brain and Language, 111, 125–139.
Bornkessel- Schlesewsky, I., Schlesewsky, M., Small, S. L., & Rauschecker, J. P. (2015a).
Neurobiological roots of language in primate audition: Common computational properties.
Bornkessel-Schlesewsky, I., Schlesewsky, M., Small, S. L., & Rauschecker, J. P. (2015b). Response
to Skeide and Friederici: The myth of the uniquely human “direct” dorsal pathway. Trends in
Bornkessel-Schlesewsky, I., Staub, A., & Schlesewsky, M. (2016). The timecourse of sentence
processing in the brain. In G. Hickok & S. L. Small (Eds.), Neurobiology of language (pp. 607–
620). Amsterdam: Elsevier.
Brauer, J., Anwander, A., & Friederici, A. D. (2011). Neuroanatomical prerequisites for lan-
guage functions in the maturing brain. Cerebral Cortex, 21, 459–466.
Brennan, J., Nir, Y., Hasson, U., Malach, R., Heeger, D. J., & Pylkkänen, L. (2012). Syntactic
structure building in the anterior temporal lobe during natural story listening. Brain and
Language, 120, 163–173.
Brouwer, H., Fitz, H., & Hoeks, J. C. (2012). Getting real about semantic illusions: Rethinking
the functional role of the P600 in language comprehension. Brain Research, 1446, 127–143.
Burholt Kristensen, L., Engberg-Pedersen, E., Højlund Nielsen, A., & Wallentin, M. (2013). The
influence of context on word order processing: An fMRI study. Journal of Neurolinguistics,
26,  73–88.
Caplan, D., Chen, E., & Waters, G. (2008). Task- dependent and task- independent
neurovascular responses to syntactic processing. Cortex, 44, 257–275.
Carreiras, M., Armstrong, B. C., Perea, M., & Frost, R. (2014). The what, when, where, and how
of visual word recognition. Trends in Cognitive Sciences, 18,  90–98.
13, 1428–1432.
Chiou, R., Sowman, P. F., Etchell, A. C., & Rich, A. N. (2014). A conceptual lemon: Theta burst
stimulation to the left anterior temporal lobe untangles object representation and its canon-
ical color. Journal of Cognitive Neuroscience, 26, 1066–1074.
Chomsky, N. (1965). Aspects of the theory of syntax. Cambridge, MA: MIT Press.
Choudhary, K. K., Schlesewsky, M., Roehm, D., & Bornkessel-Schlesewsky, I. (2009). The
N400 as a correlate of interpretively- relevant linguistic rules: Evidence from Hindi.
Christensen, K. R., & Wallentin, M. (2011). The locative alternation: Distinguishing linguistic
processing cost from error signals in Broca’s region. NeuroImage, 56, 1622–1631.
Clifton, C. J., Kurcz, I., & Jenkins, J. J. (1965). Grammatical relations as determinants of sen-
tence similarity. Journal of Verbal Learning and Verbal Behavior, 4, 112–117.
Connolly, J. F., & Phillips, N. A. (1994). Event-related potential components reflect phonolog-
ical and semantic processing of the terminal word of spoken sentences. Journal of Cognitive
Coulson, S., King, J. W., & Kutas, M. (1998). Expect the unexpected: Event-related brain re-
sponse to morphosyntactic violations. Language and Cognitive Processes, 13,  21–58.
in Cognitive Sciences, 15, 254–262.
Delaney-Busch, N., & Kuperberg, G. R. (2013). Friendly drug-dealers and terrifying pup-
pies: Affective primacy can attenuate the N400 effect in emotional discourse contexts.
Cognitive, Affective and Behavioral Neuroscience, 13, 473–490.
DeLong, K. A., Urbach, T. P., & Kutas, M. (2005). Probabilistic word pre-activation during lan-
guage comprehension inferred from electrical brain activity. Nature Neuroscience, 8, 1117–1121.
DeWitt, I., & Rauschecker, J. (2012). Phoneme and word recognition in the auditory ventral
stream. Proceedings of the National Academy of Sciences, 109, E505–E514.
Dien, J. (2009). Foreword to the special issue “Before the N400: Early Latency Language ERPs.”
Biological Psychology, 80,  1–3.
Dikker, S., & Pylkkänen, L. (2011). Before the N400: Effects of lexical–semantic violations in
visual cortex. Brain and Language, 118,  23–28.
Dikker, S., Rabagliati, H., Farmer, T. A., & Pylkkänen, L. (2010). Early occipital sensitivity to
syntactic category is based on form typicality. Psychological Science, 21, 629–634.
Dikker, S., Rabagliati, H., & Pylkkänen, L. (2009). Sensitivity to syntax in visual cortex.
Cognition, 110, 293–321.
Dittmar, M., Abbot-Smith, K., Lieven, E., & Tomasello, M. (2008). German children’s com-
prehension of word order and case marking in causative sentences. Child Development, 79,
1152–1167.
prehension. Psychophysiology, 44, 491–505.
Federmeier, K., Kutas, M., & Schul, R. (2010). Age-related and individual differences in the use
of prediction during language comprehension. Brain and Language, 115, 149–161.
Federmeier, K., & Laszlo, S. (2009). Time for meaning: Electrophysiology provides insights
into the dynamics of representation and processing in semantic memory. In B. Ross (Ed.),
Psychology of learning and motivation (Vol. 51, pp. 1–44). Burlington: Academic Press.
Fiebach, C., Schlesewsky, M., Lohmann, G., von Cramon, D., & Friederici, A. (2005). Revisiting
the role of Broca’s area in sentence processing: Syntactic integration versus syntactic working
memory. Human Brain Mapping, 24,  79–91.
Frazier, L. (1987). Sentence processing: A tutorial review. In M. Coltheart (Ed.), Attention and
performance, Vol. 12: The psychology of reading (pp. 559–586). Hove: Erlbaum.
Frazier, L., & Clifton, C. J. (1996). Construal. Cambridge, MA: MIT Press.
Frey, S., Mackey, S., & Petrides, M. (2014). Cortico-cortical connections of areas 44 and 45B in
the macaque monkey. Brain and Language, 131,  36–55.
Friederici, A. D. (1995). The time course of syntactic activation during language processing: A
model based on neuropsychological and neurophysiological data. Brain and Language, 50,
259–281.
Friederici, A. D. (1999). The neurobiology of language comprehension. In A. D. Friederici
(Ed.), Language comprehension: A biological perspective (pp. 263–301). Berlin; Heidelberg;
New York: Springer.
Friederici, A. D. (2002). Towards a neural basis of auditory sentence processing. Trends in
Cognitive Sciences, 6,  78–84.
Friederici, A. D. (2009). Pathways to language: Fiber tracts in the human brain. Trends in
Friederici, A. (2011). The brain basis of language processing: From structure to function.
Physiological Reviews, 91, 1357–1392.
comprehension. Trends in Cognitive Sciences, 16, 262–268.
Friederici, A. D., Fiebach, C. J., Schlesewsky, M., Bornkessel, I., & von Cramon, D. Y. (2006).
Processing linguistic complexity and grammaticality in the left frontal cortex. Cerebral
Cortex, 16, 1709–1717.
Friederici, A. D., Pfeifer, E., & Hahne, A. (1993). Event-related brain potentials during natural
speech processing: Effects of semantic, morphological, and syntactic violations. Cognitive
Brain Research, 1, 183–192.
Frisch, S., & Schlesewsky, M. (2001). The N400 indicates problems of thematic hierarchizing.
Frisch, S., & Schlesewsky, M. (2005). The resolution of case conflicts from a neurophysiological
perspective. Cognitive Brain Research, 25, 484–498.
Friston, K. (2005). A theory of cortical responses. Philosophical Transactions of the Royal
Society B: Biological Sciences, 360, 815–836.
Friston, K. (2010). The free- energy principle: A unified brain theory? Nature Reviews
Garrido, M. I., Kilner, J. M., Stephan, K. E., & Friston, K. J. (2009). The mismatch negativity: A
review of underlying mechanisms. Clinical Neurophysiology, 120, 453–463.
Geiser, E., Zaehle, T., Jancke, L., & Meyer, M. (2008). The neural correlate of speech rhythm as
evidenced by metrical speech processing. Journal of Cognitive Neuroscience, 20, 541–552.
Gibson, E. (1998). Linguistic complexity: Locality of syntactic dependencies. Cognition,
68,  1–76.
Neurology, 76, 1006–1014.
Grewe, T., Bornkessel, I., Zysset, S., Wiese, R., von Cramon, D. Y., & Schlesewsky, M. (2005).
The emergence of the unmarked: A new perspective on the language-specific function of
Broca’s area. Human Brain Mapping, 26, 178–190.
Grewe, T., Bornkessel, I., Zysset, S., Wiese, R., von Cramon, D. Y., & Schlesewsky, M. (2006).
Linguistic prominence and Broca’s area: The influence of animacy as a linearization prin-
ciple. NeuroImage, 32, 1395–1402.
Grush, R. (2004). The emulation theory of representation: Motor control, imagery, and percep-
tion. Behavioral and Brain Sciences, 27, 377–396.
Gunter, T. C., Stowe, L. A., & Mulder, G. (1997). When syntax meets semantics.
Psychophysiology, 34, 660–676.
Hagoort, P. (2003). How the brain solves the binding problem for language: A
neurocomputational model of syntactic processing. NeuroImage, 20, S18–S29.
Hagoort, P. (2005). On Broca, brain, and binding: A new framework. Trends in Cognitive
Sciences, 9, 416–423.
Hagoort, P. (2008). The fractionation of spoken language understanding by measuring elec-
trical and magnetic brain signals. Philosophical Transactions of the Royal Society B, 363,
1055–1069.
Hagoort, P. (2013). MUC (memory, unification, control) and beyond. Frontiers in Psychology, 4.
doi: 10.3389/fpsyg.2013.00416
Hagoort, P. (2014). Nodes and networks in the neural architecture for language: Broca’s region
and beyond. Current Opinion in Neurobiology, 28, 136–141.
Hagoort, P., Brown, C., & Osterhout, L. (1999). The neurocognition of syntactic processing. In
C. Brown & P. Hagoort (Eds.), The neurocognition of language (pp. 273–316). Oxford: Oxford
University Press.
Hagoort, P., & Indefrey, P. (2014). The neurobiology of language beyond single words. Annual
Review of Neuroscience, 37, 347–362.
Hagoort, P., & van Berkum, J. J. (2007). Beyond the sentence given. Philosophical Transactions
of the Royal Society B, 362, 801–811.
Hasson, U., Chen, J., & Honey, C. J. (2015). Hierarchical process memory: Memory as an inte-
gral component of information processing. Trends in Cognitive Sciences, 19, 304–313.
Hasting, A. S., Kotz, S. A., & Friederici, A. (2007). Setting the stage for automatic syntax pro-
cessing: The mismatch negativity as an indicator of syntactic priming. Journal of Cognitive
Haupt, F. S., Schlesewsky, M., Roehm, D., Friederici, A. D., & Bornkessel-Schlesewsky, I.
(2008). The status of subject-object reanalyses in the language comprehension architecture.
Journal of Memory and Language, 59,  54–96.
Hoeks, J. C., Stowe, L. A., & Doedens, G. (2004). Seeing words in context: The interac-
tion of lexical and sentence level information during reading. Cognitive Brain Research,
19,  59–73.
Inhoff, W., & Rayner, K. (1986). Parafoveal word processing during eye fixations in reading:
Effects of word frequency. Perception & Psychophysics, 40, 431–439.
Kandylaki, K. D., Nagels, A., Tune, S., Kircher, T., Wiese, R., Schlesewsky, M., & Bornkessel-
Schlesewsky, I. (2016). Predicting “when” in discourse engages the human dorsal audi-
tory stream: An fMRI study using naturalistic stories. Journal of Neuroscience, 36(48),
12180–12191.
Kikuchi, Y., Horwitz, B., & Mishkin, M. (2010). Hierarchical auditory processing directed ros-
trally along the monkey’s supratemporal plane. Journal of Neuroscience, 30, 13021–13030.
Kim, A., & Osterhout, L. (2005). The independence of combinatory semantic pro-
cessing: Evidence from event-related potentials. Journal of Memory and Language, 52,
205–225.
Kim, M.-J., Jeon, H. A., & Lee, K.-M. (2010). Impairments of syntactic comprehension in
Korean and the location of ischemic stroke lesions: A voxel-based lesion-symptom mapping
study. Behavioural Neurology, 22,  3–10.
Kimberg, D. Y., & Farah, M. J. (1993). A unified account of cognitive impairments following
frontal lobe damage: The role of working memory in complex, organized behavior. Journal
of Experimental Psychology: General, 122, 411–428.
Kinno, R., Kawamura, M., Shioda, S., & Sakai, K. L. (2008). Neural correlates of noncanonical
syntactic processing revealed by a picture-sentence matching task. Human Brain Mapping,
29, 1015–1027.
Knoll, L., Obleser, J., Schipke, C., Friederici, A., & Brauer, J. (2012). Left prefrontal cortex acti-
vation during sentence comprehension covaries with grammatical knowledge in children.
Kolk, H. H., Chwilla, D. J., van Herten, M., & Oor, P. (2003). Structure and limited capacity
in verbal working memory: A study with event-related potentials. Brain and Language,
85,  1–36.
Koster-Hale, J., & Saxe, R. (2013). Theory of mind: A neural prediction problem. Neuron, 79,
836–848.
Kravitz, D. J., Saleem, K. S., Baker, C. I., & Mishkin, M. (2011). A new neural framework for vis-
uospatial processing. Nature Reviews Neuroscience, 12, 217–230.
Kravitz, D., Saleem, K., Baker, C., Ungerleider, L., & Mishkin, M. (2013). The ventral visual
pathway: An expanded neural framework for the processing of object quality. Trends in
Cognitive Sciences, 17,  26–49.
Kretzschmar, F., Schlesewsky, M., & Staub, A. (2015). Dissociating word frequency and predict-
ability effects in reading: Evidence from coregistration of eye movements and EEG. Journal
of Experimental Psychology: Learning, Memory, and Cognition, 41(6), 1648–1662.
Kujala, T., Tervaniemi, M., & Schröger, E. (2007). The mismatch negativity in cognitive and
clinical neuroscience: Theoretical and methodological considerations. Biological Psychology,
74,  1–19.
Kuperberg, G., McGuire, P. K., Bullmore, E. T., Brammer, M., Rabe-Hesketh, S., Wright,
I. C., . . . David, A. S. (2000). Common and distinct neural substrates for pragmatic, se-
mantic, and syntactic processing of spoken sentences: An fMRI study. Journal of Cognitive
Kuperberg, G. R., Sitnikova, T., Caplan, D., & Holcomb, P. (2003). Electrophysiological
distinctions in processing conceptual relationships within simple sentences. Cognitive Brain
Research, 17, 117–129.
Kutas, M., & Federmeier, K. D. (2011). Thirty years and counting: Finding meaning in the n400
component of the event-related brain potential (ERP). Annual Review of Psychology, 62,
621–647.
Kutas, M., & Hillyard, S. A. (1980). Reading senseless sentences: Brain potentials reflect se-
mantic incongruity. Science, 207, 203–205.
Kutas, M., & Iragui, V. (1998). The N400 in a semantic categorization task across 6 decades.
Electroencephalography and Clinical Neurophysiology, 108, 456–471.
Laszlo, S., & Federmeier, K. D. (2011). The N400 as a snapshot of interactive pro-
cessing: Evidence from regression analyses of orthographic neighbor and lexical associate
effects. Psychophysiology, 48, 176–186.
Lerner, Y., Honey, C. J., Silbert, L. J., & Hasson, U. (2011). Topographic mapping of a hierarchy
of temporal receptive windows using a narrated story. Journal of Neuroscience, 31, 2906–2915.
Lewis, R. L. (1996). Interference in short-term memory: The magical number two (or three) in
sentence processing. Journal of Psycholinguistic Research, 25, 93–115.
Lewis, R. L., Vasishth, S., & Van Dyke, J. A. (2006). Computational principles of working
memory in sentence comprehension. Trends in Cognitive Sciences, 10, 447–454.
Lotze, N., Tune, S., Schlesewsky, M., & Bornkessel-Schlesewsky, I. (2011). Meaningful physical
changes mediate lexical-semantic integration: Top-down and form-based bottom-up infor-
mation sources interact in the N400. Neuropsychologia, 49, 3573–3582.
MacDonald, M. C., Pearlmutter, N. J., & Seidenberg, M. S. (1994). The lexical nature of syn-
tactic ambiguity resolution. Psychological Review, 101, 676–703.
MacGregor, L., Pulvermüller, F., van Casteren, M., & Shtyrov, Y. (2012). Ultra-rapid access to
words in the brain. Nature Communications, 3, 711.
Magnusdottir, S., Fillmore, P., den Ouden, D., Hjaltason, H., Rorden, C., Kjartansson,
O., . . . Fridriksson, J. (2013). Damage to left anterior temporal cortex predicts impairment
of complex syntactic processing: A lesion-symptom mapping study: ATL damage impairs
complex syntax processing. Human Brain Mapping, 34, 2715–2723.
Makuuchi, M., Bahlmann, J., Anwander, A., & Friederici, A. D. (2009). Segregating the
core computational faculty of human language from working memory. Proceedings of the
Martin, S., Millán, J. del R., Knight, R. T., & Pasley, B. N. (in press). The use of intracranial
recordings to decode human language: Challenges and opportunities. Brain and Language.
https://doi.org/10.1016/j.bandl.2016.06.003
McElree, B. (1993). The locus of lexical preference effects in sentence comprehension. Journal
of Memory and Language, 32, 536–571.
McElree, B. (2006). Accessing recent events. In B. Ross (Ed.), The psychology of learning and
motivation (pp. 155–200). San Diego, CA: Academic Press.
Mehler, J. (1963). Some effects of grammatical transformations on the recall of English
sentences. Journal of Verbal Learning and Verbal Behavior, 2, 346–351.
Menning, H., Zwitserlood, P., Schöning, S., Hihn, H., Bölte, J., Dobel, C., . . . Lütkenhöner,
B. (2005). Pre- attentive detection of syntactic and semantic errors. Neuroreport,
16,  77–80.
Mesulam, M.-M., Grossman, M., Hillis, A., Kertesz, A., & Weintraub, S. (2003). The core
and halo of primary progressive aphasia and semantic dementia. Annals of Neurology, 54,
S11–S14.
Meteyard, L., Cuadrado, S. R., Bahrami, B., & Vigliocco, G. (2012). Coming of age: A review of
embodiment and the neuroscience of semantics. Cortex, 48, 788–804.
Miller, E. K., & Cohen, J. D. (2001). An integrative theory of prefrontal cortex function. Annual
Miller, G. A. (1962). Some psychological studies of grammar. American Psychologist, 17,
748–762.
Mueller, J. L., Hahne, A., Fujii, Y., & Friederici, A. (2005). Native and nonnative speakers’
processing of a miniature version of Japanese as revealed by ERPs. Journal of Cognitive
Mueller, J. L., Hirotani, M., & Friederici, A. D. (2007). ERP evidence for different strategies
in the processing of case markers in native speakers and non-native learners. BMC
Newhart, M., Trupe, L. A., Gomez, Y., Cloutman, L., Molitoris, J. J., Davis, C., . . . Hillis, A.
E. (2012). Asyntactic comprehension, working memory, and acute ischemia in Broca’s area
versus angular gyrus. Cortex, 48, 1288–1297.
Newman, R., & Connolly, J. F. (2009). Electrophysiological markers of pre-lexical speech
processing: Evidence for bottom-up and top-down effects on spoken word processing.
Biological Psychology, 80, 114–121.
Newman, S. D., Ikuta, T., & Burns, T. (2010). The effect of semantic relatedness on syntactic
analysis: An fMRI study. Brain and Language, 113,  51–58.
Novick, J. M., Trueswell, J. C., & Thompson-Schill, S. L. (2005). Cognitive control and
parsing: Reexamining the role of Broca’s area in sentence comprehension. Cognitive,
Affective, & Behavioral Neuroscience, 5, 263–281.
Osterhout, L., & Nicol, J. (1999). On the distinctiveness, independence, and time course of the
brain response to syntactic and semantic anomalies. Language and Cognitive Processes, 14,
283–317.
Papoutsi, M., Stamatakis, E. A., Griffiths, J., Marslen-Wilson, W. D., & Tyler, L. K. (2011). Is left
fronto-temporal connectivity essential for syntax? Effective connectivity, tractography and
performance in left-hemisphere damaged patients. NeuroImage, 58, 656–664.
Perani, D., Saccuman, M. C., Scifo, P., Anwander, A., Spada, D., Baldoli, C., . . . Friederici, A. D.
(2011). Neural language networks at birth. Proceedings of the National Academy of Sciences,
108, 16056–16061.
Pickering, M. J., & Garrod, S. (2007). Do people use language production to make predictions
during comprehension? Trends in Cognitive Sciences, 11, 105–110.
Pickering, M. J., & Garrod, S. (2013). An integrated theory of language production and compre-
hension. Behavioral and Brain Sciences, 36, 329–347.
Pinker, S. (2009). Language learnability and language development, with new commentary by
the author. Cambridge, MA: Harvard University Press.
Pobric, G., Jefferies, E., & Lambon Ralph, M. A. (2007). Anterior temporal lobes me-
diate semantic representation: Mimicking semantic dementia by using rTMS in normal
participants. Proceedings of the National Academy of Sciences, 104, 20137–20141.
lateralization as “asymmetric sampling in time.” Speech Communication, 41, 245–255.
contributions to reading. Trends in Cognitive Sciences, 15, 246–253.
Pulvermüller, F., & Shtyrov, Y. (2003). Automatic processing of grammar in the human brain as
revealed by the mismatch negativity. NeuroImage, 20, 159–172.
Pulvermüller, F., Shtyrov, Y., & Hauk, O. (2009). Understanding in an instant:
Neurophysiological evidence for mechanistic language circuits in the brain. Brain and
Language, 110,  81–94.
Rauschecker, J. P. (1998). Cortical processing of complex sounds. Current Opinion in
Neurobiology, 8, 516–521.
control and integration. Hearing Research, 271,  16–25.
Rauschecker, J. P., & Tian, B. (2000). Mechanisms and streams for processing of what
and where in auditory cortex. Proceedings of the National Academy of Sciences, 97,
11800–11806.
Rauschecker, J. P., Tian, B., & Hauser, M. (1995). Processing of complex sounds in the macaque
nonprimary auditory cortex. Science, 268, 111–114.
Rauschecker, J., & Scott, S. (2009). Maps and streams in the auditory cortex: Nonhuman
primates illuminate human speech processing. Nature Neuroscience, 12, 718–724.
Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research.
Psychological Bulletin, 124, 372–422.
Rayner, K., Carlson, G., & Frazier, L. (1983). The interaction of syntax and semantics during
sentence processing: Eye movements in the analysis of semantically biased sentences.
Journal of Verbal Learning and Verbal Behavior, 22, 657–673.
Rayner, K., & Clifton, C. J. (2009). Language processing in reading and speech perception is fast
and incremental: Implications for event-related potential research. Biological Psychology,
80,  4–9.
Rayner, K., Slowiaczek, M. L., Clifton, C., & Bertera, J. H. (1983). Latency of sequential
eye movements: Implications for reading. Journal of Experimental Psychology: Human
Perception and Performance, 9, 912.
Rilling, J. K., Glasser, M. F., Preuss, T. M., Ma, X., Zhao, T., Hu, X., & Behrens, T. E. J. (2008).
The evolution of the arcuate fasciculus revealed with comparative DTI. Nature Neuroscience,
11, 426–428.
Rodd, J. M., Longe, O. A., Randall, B., & Tyler, L. K. (2010). The functional organisation of
the fronto-temporal language system: Evidence from syntactic and semantic ambiguity.
Rolheiser, T., Stamatakis, E. A., & Tyler, L. K. (2011). Dynamic processing in the human lan-
guage system: Synergy between the arcuate fascicle and extreme capsule. Journal of
Romanski, L. M., Tian, B., Fritz, J., Mishkin, M., Goldman-Rakic, P. S., & Rauschecker, J. P.
(1999). Dual streams of auditory afferents target multiple domains in the primate prefrontal
cortex. Nature Neuroscience, 2, 1131–1136.
Röder, B., Stock, O., Neville, H., Bien, S., & Rösler, F. (2002). Brain activation modulated by the
comprehension of normal and pseudo-word sentences of different processing demands: A
functional magnetic resonance imaging study. NeuroImage, 15, 1003–1014.
Rösler, F. (2011). Psychophysiologie der Kognition: Eine Einführung in die kognitive
Neurowissenschaft [Psychophysiology of cognition: An introduction to cognitive neuroscience].
Heidelberg: Springer-Verlag.
Sassenhagen, J., Schlesewsky, M., & Bornkessel-Schlesewsky, I. (2014). The P600-as-P3 hypo-
thesis revisited: Single-trial analyses reveal that the late EEG positivity following linguisti-
cally deviant material is reaction time aligned. Brain and Language, 137,  29–39.
Saur, D., Kreher, B. W., Schnell, S., Kümmerer, D., Kellmeyer, P., Vry, M.-S., et al. (2008).
Ventral and dorsal pathways for language. Proceedings of the National Academy of Sciences,
105, 18035–18040.
Schlesewsky, M., & Bornkessel, I. (2004). On incremental interpretation: Degrees of meaning
accessed during sentence comprehension. Lingua, 114, 1213–1234.
Scott, S. K., Blank, C. C., Rosen, S., & Wise, R. J. (2000). Identification of a pathway for intelli-
gible speech in the left temporal lobe. Brain, 123, 2400–2406.
Scott, S. K., & Wise, R. J. (2004). The functional neuroanatomy of prelexical processing in
speech perception. Cognition, 92,  13–45.
Sereno, S. C., & Rayner, K. (2003). Measuring word recognition in reading: Eye movements
and event-related potentials. Trends in Cognitive Sciences, 7, 489–493.
Sereno, S. C., Rayner, K., & Posner, M. I. (1998). Establishing a time-line of word recogni-
tion: Evidence from eye movements and event-related potentials. Neuroreport, 9, 2195–2200.
Shtyrov, Y., & Pulvermüller, F. (2002). Memory traces for inflectional affixes as shown by mis-
match negativity. European Journal of Neuroscience, 15, 1085–1091.
Shtyrov, Y., & Pulvermüller, F. (2007). Early MEG activation dynamics in the left tem-
poral and inferior frontal cortex reflect semantic context integration. Journal of Cognitive
Skipper, J. I., Devlin, J. T., & Lametti, D. R. (2017). The hearing ear is always found close to the
speaking tongue: Review of the role of the motor system in speech perception. Brain and
Language, 164, 77–105.
Small, S. L. (2008). The neuroscience of language. Brain and Language, 106,  1–3.
Snijders, T. M., Vosse, T., Kempen, G., Berkum, J. J. A. V., Petersson, K. M., & Hagoort, P.
(2009). Retrieval and unification of syntactic structure in sentence comprehension: An
fMRI study using word-category ambiguity. Cerebral Cortex, 19, 1493–1503.
Thompson-Schill, S. L., Bedny, M., & Goldberg, R. F. (2005). The frontal lobes and the regula-
tion of mental activity. Current Opinion in Neurobiology, 15, 219–224.
Thothathiri, M., Kimberg, D. Y., & Schwartz, M. F. (2012). The neural basis of reversible sen-
tence comprehension: Evidence from voxel-based lesion-symptom mapping in aphasia.
Trueswell, J. C., & Tanenhaus, M. K. (1994). Toward a lexicalist framework for constraint-based
syntactic ambiguity resolution. In J. Clifton, C. L. Frazier, & K. Rayner (Eds.), Perspectives in
sentence processing. Hillsdale, NJ: Lawrence Erlbaum.
Tsunada, J., Lee, J. H., & Cohen, Y. E. (2011). Representation of speech categories in the primate
auditory cortex. Journal of Neurophysiology, 105, 2634–2646.
Tsunada, J., Lee, J. H., & Cohen, Y. E. (2012). Differential representation of auditory categories
between cell classes in primate auditory cortex. The Journal of Physiology, 590, 3129–3139.
Tune, S., Schlesewsky, M., Small, S. L., Sanford, A. J., Bohan, J., Sassenhagen, J., & Bornkessel-
Schlesewsky, I. (2014). Cross- linguistic variation in the neurophysiological response
to semantic processing: Evidence from anomalies at the borderline of awareness.
Tyler, L. K., & Marslen-Wilson, W. (2008). Fronto-temporal brain systems supporting spoken
language comprehension. Philosophical Transactions of the Royal Society B: Biological
Sciences, 363, 1037–1054.
Tyler, L. K., Cheung, T. P. L., Devereux, B. J., & Clarke, A. (2013). Syntactic computations in the
language network: Characterizing dynamic network properties using representational simi-
larity analysis. Frontiers in Psychology, 4. doi: 10.3389/fpsyg.2013.00271
ventral language pathways. Neuron, 72, 385–396.
Ullman, M. T. (2001). A neurocognitive perspective on language: The declarative/procedural
model. Nature Reviews Neuroscience, 2, 717–726.
van de Meerendonk, N., Kolk, H. H., Chwilla, D. J., & Vissers, C. T. W. M. (2009). Monitoring
in language perception. Language and Linguistics Compass, 3, 1211–1224.
van den Brink, D., Brown, C., & Hagoort, P. (2001). Electrophysiological evidence for early
contextual influences during spoken-word recognition: N200 versus N400 effects. Journal
van den Brink, D., Brown, C., & Hagoort, P. (2006). The cascaded nature of lexical selection and
integration in auditory sentence processing. Journal of Experimental Psychology: Learning,
Memory & Cognition, 32, 364–372.
Van Petten, C., & Kutas, M. (1990). Interactions between sentence context and word frequency
in event-related brain potentials. Memory & Cognition, 18, 380–393.
van Wassenhove, V., Grant, K. W., & Poeppel, D. (2005). Visual speech speeds up the neural
processing of auditory speech. Proceedings of the National Academy of Sciences USA, 102,
1181–1186.
Visser, M., Jefferies, E., & Lambon Ralph, M. (2010). Semantic processing in the anterior tem-
poral lobes: A meta-analysis of the functional neuroimaging literature. Journal of Cognitive
Neuroscience, 22(6), 1083–1094.
Vissers, C. T. W. M., Chwilla, D. J., & Kolk, H. H. (2006). Monitoring in language percep-
tion: The effect of misspellings of words in highly constrained sentences. Brain Research,
1106, 150–163.
Vosse, T., & Kempen, G. (2000). Syntactic structure assembly in human parsing: A compu-
tational model based on competitive inhibition and a lexicalist grammar. Cognition, 75,
105–143.
Wicha, N. Y. Y., Moreno, E. M., & Kutas, M. (2004). Anticipating words and their gender: An
event-related brain potential study of semantic integration, gender expectancy, and gender
agreement in Spanish sentence reading. Journal of Cognitive Neuroscience, 16, 1271–1288.
Wilson, S. M., DeMarco, A. T., Henry, M. L., Gesierich, B., Babiak, M., Mandelli, M.
L., . . . Gorno-Tempini, M. L. (2014). What role does the anterior temporal lobe play in
sentence-level processing? Neural correlates of syntactic processing in semantic variant pri-
mary progressive aphasia. Journal of Cognitive Neuroscience, 26, 970–985.
Neuron, 72, 397–403.
Wise, R. J., Scott, S. K., Blank, S. C., Mummery, C. J., Murphy, K., & Warburton, E. (2001).
Separate neural subsystems within “Wernicke’s area.” Brain, 124,  83–95.
Wolpert, D. M., Doya, K., & Kawato, M. (2003). A unifying computational framework for
motor control and social interaction. Philosophical Transactions of the Royal Society
B: Biological Sciences, 358, 593–602.
Wolpert, D. M., Ghahramani, Z., & Flanagan, J. R. (2001). Perspectives and problems in motor
learning. Trends in Cognitive Sciences, 5, 487–494.
Wolpert, D. M., Ghahramani, Z., & Jordan, M. I. (1995). An internal model for sensorimotor
integration. Science, 269, 1880–1882.
Xiang, H.-D., Fonteijn, H. M., Norris, D. G., & Hagoort, P. (2010). Topographical functional
connectivity pattern in the perisylvian language networks. Cerebral Cortex, 20, 549–560.
Chapter 28
C omprehensi on of
Metaphors and   I di oms
An Updated Meta-analysis of Functional Magnetic
Resonance Imaging Studies
Alexander Michael Rapp
Everyday language in both speech and written language is abundant in expressions

that, when taken literally, are wrong (at least in the aforementioned context). These
expressions are referred to as nonliteral language. Two important forms of nonliteral
constructions are metaphors (such as “neuroimaging is a gold mine”) and idioms (such
as “it’s raining cats and dogs”). Other types include ironic expressions, proverbs, and
metonymy. For linguists, nonliteral language (or “figurative” language, a term used more
often in the context of research, although presumably less exact) is interesting because
of its sheer frequency of occurrence alone. While corpus linguists still debate whether
metaphors (Cameron & Deignan, 2006; Stefanowitsch & Gries, 2007) or idioms
(Simpson & Mendis, 2003; Van Lancker-Sidtis & Rallon, 2004) are more frequent in
everyday speech, there is consensus that both forms are essential parts of everyday
speech (Lakoff & Johnson, 2008). At first glance that seems inappropriate. Nonliteral
language violates the maxim to speak in the utmost understandable form while avoiding
ambiguity. Nevertheless, it is often used and therefore obviously serves important com-
municative functions (Gibbs Jr. & Colston, 2012; Lakoff & Johnson, 2008), presumably
including being polite, of higher emotional precision, better remembered or—often
enough—the only way the speaker has to express an issue (Colston & Katz, 2004).
This chapter covers the underlying neurobiology of metaphor and idiom compre-
hension. The topic has gained interest in many different research fields. For decades,
it has been claimed that clinical populations may have a selective deficit in metaphor
comprehension, including patients with damage to the right hemisphere of the brain
(Brownell, Simpson, Bihrle, Potter, & Gardner, 1990; Winner & Gardner, 1977) and those
with disorders such as schizophrenia (Chapman, 1960; Rapp, 2009; Rapp, Langohr,
Comprehension of Metaphors and Idioms 711
Mutschler, & Wild, 2014) and autism (Gold, Faust, & Goldstein, 2010). The majority of
research has been focused on these disorders, but some studies have investigated other
conditions, including dementias (Rapp & Wild, 2011), William’s syndrome (Annaz
et al., 2009), depression (Rapp & Schmierer, 2010), traumatic brain injury (Martin &
McDonald, 2005), relational aggression (Blasko & Kazmerski, 2006), schizotypal per-
sonality traits (Ettinger et al., 2015), Parkinson’s disease (Gutmann, 2009), amyotrophic
lateral sclerosis (ALS; Bambini et al., 2016), and other developmental disorders (Rapp
& Wild, 2011). Paradigms on metaphor and idiom comprehension are also used as clin-
ical and research tools to investigate embodiment (Denke, Rotte, Heinze, & Schaefer,
2014; Gibbs, Lima, & Francozo, 2004) and motor language (Raposo, Moss, Stamatakis,
& Tyler, 2009), social cognition (Landau, Meier, & Keefer, 2010; Langdon, Coltheart,
Ward, & Catts, 2002), aphasia (Papagno & Caporali, 2006), intelligence (Jäger & Althoff,
1994), and coverbal gestures (Straube, Green, Sass, & Kircher, 2014). It is therefore not
surprising that the number of studies on the neural correlates of these forms of com-
munication has consistently increased, especially within the last 10 years. This chapter
reviews the functional magnetic resonance imaging studies (fMRI) on metaphor and
idioms.
A Meta-analysis on Metaphor
Comprehension
Findings of fMRI studies (see Heim & Specht, Chapter 4 in this volume) can be
summarized using coordinate-based meta-analyses (activation likelihood estimation
[ALE]; see BrainMapDevelopmentTeam, 2013; Eickhoff et al., 2009). In this type of anal-
ysis, the coordinates reported in the studies are used and, if necessary, transformed into
one coordinate system: either Talairach space (Talairach & Tournoux, 1988) or Montreal
Neurological Institute (MNI) space (Tzourio-Mazoyer et al., 2002). Briefly, this analysis
involves fitting the coordinate data to a Gaussian curve in order to identify any activation
clusters in the brain. The research tool identifies consistent regions of activation across a
collection of fMRI studies and shows where the reported activations converge (Eickhoff
et al., 2009). Meta-analyses of fMRI data have advantages and disadvantages. A disad-
vantage of such meta-analyses is that their interpretations can be limited when only a
small number of foci have been reported in the available literature, and that it might
overestimate the effect of studies with low or “liberate” chosen activation thresholds.
So far, five meta-analyses for metaphors (Bohrn, Altmann, & Jacobs, 2012; Rapp,
Mutschler, & Erb, 2012; Vartanian, 2012; Yang, 2014; Yang & Shu, 2016) and one for
idioms (Rapp et al., 2012) have been published. The meta-analyses show comparable
results, which is not astonishing, given that they rely on overlapping studies; however,
they slightly differ in their interpretation. Here, we present an updated version of our
previous meta-analysis (Rapp et al., 2012), which includes more recent data.
712 Alexander Michael Rapp
fMRI Literature on Metaphor
First, our previous systematic literature search (Rapp et al., 2012) was retrofitted with the
new publications since 2012. Studies were identified by PUBMED and “Web of Science”
(2016) databases, including targeted search for important researchers in the field and
search within citations of previous studies. Our meta-analysis of 2012 (Rapp et al.,
2012) for metaphors included 16 studies.
Because of our methodological criteria, this meta-analysis includes not all avail-
able work. We did not include studies using positron emission tomography (PET),
near-infrared spectroscopy (NIRS; see Minagawa & Cristia, Chapter 7 in this volume)
or electroencephalograph (EEG) technology (see Leckey & Federmeier, Chapter 3 in
this volume), although some of these studies report activation maxima (Bottini et al.,
1994; Proverbio, Crotti, Zani, & Adorni, 2009; Schneider et al., 2014; Sotillo et al.,
2005). Our rationale for this was that these research methods may have higher varia-
tion in the localization of maxima (BrainMapDevelopmentTeam, 2013). Further, we in-
cluded only reported whole brain analyses, while region of interest (ROI) data were not
recommended. This followed the rationale that ROI analyses might bias results toward
the same ROIs. Studies that did not report the contrast for metaphors > literal stimuli
were not included. Further, in the analysis presented here, activations for metaphors >
meaningless word combinations or metaphors > meaningless sentences were not
considered. The differences in the inclusion criteria and in the number of included
studies could explain the slight differences in the results between the other published
meta-analyses and this study.
The systematic literature search since 2012 yielded significant additional results. Table
28.1 shows the available studies until December 2015. Twenty-seven studies were in-
cluded in the meta-analysis for metaphors (see Rapp, 2012, for a narrative review of the
older studies and brain lesion research). In recent years, there has been a significant in-
crease in the quantity of literature, and the methodological quality of these studies has
further improved. For example, the average number of metaphoric stimuli has (again)
increased across all studies to 50.7 (see Table 28.1).
All metaphor and idiom studies together contained 875 study subjects, while the
meta-analysis on metaphors consists of data from only 460 subjects. These studies
investigated diverse aspects and several languages. There are now investigations in eight
languages. Shibata et al. (Shibata, 2011; Shibata et al., 2012) reported new fMRI data in
the Japanese language, while Obert et al. (2014) first reported fMRI data in the French
and Dutch (Samur, Lai, Hagoort, & Willems, 2015) languages. New comparisons for
different types of non-literal language were provided by Prat and Just (Prat, Mason, &
Just, 2012) with ironic stimuli, while two other studies, one in English (Desai, Conant,
Binder, Park, & Seidenberg, 2013) and one in Italian (Romero Lauro, Mattavelli,
Papagno, & Tettamanti, 2013), have idioms as stimuli in addition to metaphors. New
data with German metaphors was presented by Forgacs et al. (2012); Bohrn, Altmann,
Table 28.1 Overview of functional magnetic resonance imaging (fMRI) studies on metaphor and idiom comprehension: Studies
Included in the Updated fMRI Meta-analysis.
Number of
Non-literal Number of Non-literal
Year Language Type Language Subjects Stimulus Level Novel Stimuli Salient Stimuli Stimuli
1 Rapp et al., 2004 2004 Metaphor German 15 Sentence x 30

2 Mashal et al., 2005 2005 Metaphor Hebrew 15 Word x x 2 x 24
3 Eviatar and Just, 2006 2006 Metaphor English 16 Text vignette x 2x9
4 Lee and Dapretto, 2006 2006 Metaphor English 12 Word x 16
5 Stringaris et al., 2006 2006 Metaphor English 12 Sentence x ?
6 Aziz-Zadeh et al., 2006 2006 Metaphor English 12 Sentence x 15
7 Wakusawa et al., 2007 2007 Metaphor Japanese 38 Sentence x 14
8 Ahrens et al., 2007 2007 Metaphor Chinese 8 Sentence x x 2 x 14
9 Stringaris et al., 2007 2007 Metaphor English 11 Sentence x 25
11 Rapp et al., 2007 2007 Metaphor German 17 Sentence x 60
12 Zempleni et al., 2007 2007 Idiom Dutch 15 Sentence x 64 + 32
13 Shibata et al., 2007 2007 Metaphor Japanese 13 Sentence x 21
14 Mashal et al., 2008 2008 Idiom Hebrew 14 Sentence x 50
15 Lauro et al., 2008 2008 Idiom Italian 22 Sentence x 62
16 Boulenger et al., 2009 2009 Idiom English 18 Sentence x 26
17 Chen et al., 2008 2008 Metaphor English 14 Sentence x 35
(continued)
Table 28.1 Continued
Number of
Non-literal Number of Non-literal
Year Language Type Language Subjects Stimulus Level Novel Stimuli Salient Stimuli Stimuli
18 Mashal et al., 2009 2009 Metaphor Hebrew 15 Sentence x 25
19 Hillert and Buracas, 2009 2009 Idiom English 10 Sentence x 100
20 Raposo et al., 2009 2009 Idiom English 22 Sentence x 56
21 Yang et al., 2009 2009 Metaphor English 18 Sentence x x 100
22 Schmidt and Seger, 2009 2009 Metaphor English 10 Sentence x x 3 x 24
23 Yang et al., 2010 2010 Metaphor English 18 Sentence x 2 x 23
24 Mashal and Faust, 2010 2010 Metaphor Hebrew 10 Text x 28 1
25 Mejía-Constaín et al., 2010 Metaphor French 2 x 10 Word x 25
2010
26 Desai et al., 2011 2011 Metaphor English 22 Sentence x x 81
27 Diaz et al., 2011 2011 Metaphor English 16 Sentence x x 2 x40
28 Bambini et al., 2011 2011 Metaphor Italian 10 Two-sentence x x 40
passage
29 Diaz and Hogstrom, 2011 2011 Metaphor English 16 Two-sentence x x 2 x 40
passage
30 Uchiyama et al., 2012 2012 Metaphor Japanese 20 Text vignette X 2 x 20
31 Shibata, 2011 2011 Metaphor Japanese 12 Sentence x 10
32 Prat et al., 2012 2012 Metaphor Irony English 24 Text vignette x 2 x 10
33 Cardillo et al., 2012 2012 Metaphor English 20 Sentence x x 2 x 60
34 Subramaniam et al., 2012 2012 Metaphor English 11 Word x x 2 x 68
35 Forgacs et al., 2012 2012 Metaphor German 40 Word x x 2 x 40
36 Shibata et al., 2012 2012 Metaphor Japanese 24 Sentence x 2 x 20
37 Lacey et al., 2012 2012 Metaphor English 7 Sentence x 54
38 Kana et al., 2012 2012 Idiom English 36 Sentence x 9
39 Subramaniam et al., 2013 2013 Metaphor English 14 Word x x 2 x 68
40 Desai et al., 2013 2013 Idiom Metaphor English 27 Sentence x 2 x 40
41 Bohrn et al., 2012b 2012 Proverb German 26 Sentence x x 4 x 40
43 Romero Lauro et al., 2013 2013 Idiom Metaphor Italian 24 Sentence x 2 x 21
44 Schuil et al., 2013 2013 Idiom Dutch 20 Sentence x 100
45 Klepousniotou et al., 2014 2014 Metaphor English 15 Word x 30
46 Benedek et al., 2014 2014 Metaphor German 28 Production 24
48 Obert et al., 2014 2014 Metaphor French 19 Sentence x 24
context
49 Citron and Goldberg, 2014 2014 Metaphor German 26 Sentence x 37
50 Samur et al., 2015 2015 Metaphor Dutch 20 Text vignette x 60
51 Lai et al., 2015 2015 Metaphor English 22 Sentence x x 27
52 Yang et al., 2016 2016 Idiom Chinese 20 Sentence X 96
Lubrich, Menninghaus, and Jacobs (2012); and by Citron and Goldberg (2014). Of spe-
cial interest is the study by Benedek et al. (2014), which first investigated the produc-
tion of metaphors in the German language using fMRI. Another study in Hebrew was
presented by Mashal et al. (Mashal, Vishne, Laor, & Titone, 2013), while eight additional
studies in English (Cardillo, Watson, Schmidt, Kranjec, & Chatterjee, 2012; Desai et al.,
2013; Klepousniotou, Gracco, & Pike, 2014; Lacey, Stilla, & Sathian, 2012; Lai, van Dam,
Conant, Binder, & Desai, 2015; Prat et al., 2012; Subramaniam, Beeman, Faust, & Mashal,
2013; Subramaniam, Faust, Beeman, & Mashal, 2012) showed that it is the most frequent
language. This makes clear as well that the non-English studies, with 49% of the total
studies and 45% of the trial participants, play a significant role. To date, metaphors in
a participant’s second language (L2) have not yet been studied using fMRI, although
there are numerous “offline” (without fMRI) studies on this subject that have presented
experimental data (Charteris-Black, 2002; Heredia & Cieslicka, 2015; Littlemore, 2001;
Mashal, Borodkin, Maliniak, & Faust, 2015; Tuerker, 2016).
Table 28.1 shows whether salient or non-salient stimuli were used because this
distinction plays an important role in the literature (Bowdle & Gentner, 2005;
Desai, Binder, Conant, Mano, & Seidenberg, 2011; Giora, 1997; Rapp et al., 2012).
Classification of salient stimuli followed the classification scheme given by the
authors’ published classification. If the authors provided no description, classification
was based on the stimulus examples provided using the Google corpus (Rapp et al.,
2012). Although familiarity with metaphors is not identical to their salience (Giora,
1999; Rapp & Wild, 2011), familiar stimuli in this case were counted as salient. The
procedure for data analysis in this follow-up meta-analysis was identical to Rapp et al.
(2012). ALE analysis was implemented using the software update GingerALE 2.3 for
Windows (Eickhoff et al., 2009; www.brainmap.org). After identification of the rel-
evant studies, the reported activation maxima were extracted from the publications.
Coordinates reported in Talairach space were transformed into MNI coordinates
using the “Talairach to MNI (SPM)” tool implemented in GingerALE (Fox et al.,
2013) 2.3 (tal2icbm_spm.m).
Results
The 27 studies using metaphors reported 271 foci in 460 study participants. Figure 28.1
shows a projection of these coordinates onto a brain surface. Each dot indicates a
stronger reported activation for metaphors > literal control stimuli. An important lim-
itation of this type of analysis is that it—like perhaps fMRI activation itself—gives lim-
ited information about its functional importance. Nevertheless, several things become
clear when we look at the distribution; the pattern of activation points strongly against
a “metaphor center” in the brain. Instead, a distributed network is responsible for the
understanding of metaphors. Figure 28.1 shows significant similarity to that found in
Figure 28.1. Projection onto a brain surface: 271 foci reported in 27 studies for differential
contrasts metaphor > literal stimuli; 86 (31%) are located in the right hemisphere.
comparable studies on the semantic comprehension of literal language (Binder, Desai,

Graves, & Conant, 2009). In addition, its lateralization is similar to the pattern found
generally in semantic comprehension (Binder et al., 2009; Rapp et al., 2012). Laterality
has traditionally been a focus of non-literal language research (Burgess & Chiarello,
1996; Lindell, 2006; Zaidel, Kasher, Soroker, & Batori, 2002). Out of the 271 foci, 86 (31%)
are located in the right hemisphere. Compared to our previous meta-analysis (2012), the
relative proportion of right hemispheric foci has decreased slightly (from 35%), which
contradicts early fMRI studies that may have underestimated the proportion in the right
hemisphere. Table 28.2 shows the results of the meta-analysis.
Table 28.2 Activation likelihood estimation (ALE) meta-analysis of functional

magnetic resonance imaging (fMRI) studies with metaphoric stimuli
(metaphor > literal, whole brain analysis).
Brodmann Volume
Cluster # Hemisphere Region Area x y z (mm3)
1 LH Inferior frontal gyrus 45 –46 22 14 2,336

LH Middle frontal gyrus 9 –52 22 28
2 LH Superior temporal gyrus 22 –56 –40 8 1,368
3 LH Parahippocampal gyrus 36 –34 –34 –16 960
LH Parahippocampal gyrus 36 –30 –38 –10
4 LH Inferior frontal gyrus 47 –46 30 –8 712
5 LH Superior frontal gyrus 6 –4 16 50 656
Meta-analysis metaphor
Brain Regions Involved

in Metaphor Comprehension
The strongest cluster can be found in the left inferior frontal gyrus (IFG) in Brodmann
area 45, with expansion into BA 9 and the middle frontal gyrus. Numerous studies con-
tribute to this cluster (Ahrens et al., 2007; Forgacs et al., 2012; Lee & Dapretto, 2006;
Mashal, Faust, Hendler, & Jung-Beeman, 2007, 2009; Mashal et al., 2013; Shibata et al.,
2012). In our previous study (Rapp, Leube, Erb, Grodd, & Kircher, 2004), we argued
that the anterior-inferior part of the IFG could bring together the two semantic entities
within a metaphor, which are the tenor and the vehicle. This brain region could be re-
sponsible for the “mapping” process during metaphor comprehension. However, this
view is controversial. Other authors attributed this process to the right hemisphere
(Bottini et al., 1994; Burgess & Chiarello, 1996; Toga & Thompson, 2003; Yang, 2014). It
was recently noted that the process of bringing together semantic entities might better
be called “binding” during metaphor comprehension, instead of “mapping” (Strack,
2016). In our original papers (Kircher, Leube, Erb, Grodd, & Rapp, 2007; Rapp, Erb,
Grodd, Bartels, & Markert, 2011; Rapp et al., 2004; Rapp, Leube, Erb, Grodd, & Kircher,
2007), we suggested specificity for the anterior-inferior part of the left IFG. However,
the interim literature reveals a more distributed involvement within the left IFG (Figure
28.1). In addition to the ALE analysis in Table 28.2, we anatomically classified (Rapp, un-
published data) the data of today’s literature. This includes a spatial uncertainty because
some of the foci were converted from Tailairach into MNI coordinate systems (e.g.,
Lancaster et al., 2007). The localization of each of the foci reported in the literature was
then assigned to an anatomical region. Automated anatomical labeling (AAL; Tzourio-
Mazoyer et al., 2002) and xjview (Cui, Li, & Song, 2011) software were used for this pro-
cess. Currently, 36 foci have been reported within the inferior frontal region, with 25 in
the trigeminal part, nine in the orbital part, and two in the opercular part of the IFG. The
anterior-inferior part of the left IFG is believed to play a significant role in integrating
words into meaningful sentences (Badre & Wagner, 2007; Bookheimer, 2002; Menenti,
Petersson, Scheeringa, & Hagoort, 2009). This activation might reflect elevated cogni-
tive demands required to integrate non-literal meanings, as opposed to literal ones, into
a sentence context (Bambini, Gentili, Ricciardi, Bertinetto, & Pietrini, 2011; Rapp et al.,
2004; Rapp et al., 2007; Rapp et al., 2011). This region is, however, also involved when
two-word metaphors instead of sentences are used as stimuli, so it certainly does not
reflect sentence context integration alone (Lee & Dapretto, 2006; Mashal et al., 2007;
Mashal et al., 2013). Other possible roles for the left IFG in understanding metaphors
are meaning selection and evaluation, since a distinction must be made whether a
word is intended metaphorically or literally. Research on literal language suggests that
Brodmann area (BA) 45/47 may regulate the selection of multiple competing responses
during sentence comprehension (Petrides, 2005; Rapp et al., 2012; Turken & Dronkers,
2011). We recently argued (Rapp et al., 2012) that reciprocal interactions between BA 47
Table 28.3 Activation likelihood estimation (ALE) meta-analysis of functional

magnetic resonance imaging (fMRI) studies with idioms (idiom >
literal, whole brain analysis).
Brodmann Volume
Cluster Hemisphere Region Area x y z (mm3)
1 LH Inferior frontal gyrus 45 –52 26 0 4,392

LH Inferior frontal gyrus 46 –48 30 8
2 LH Middle temporal gyrus 21 –60 –58 4 448
3 RH Precuneus 7 16 –72 48 272
4 LH Fusiform gyrus 20 –38 –10 –28 224
LH Parahippocampal gyrus –36 –10 –24
and the left middle temporal gyrus (Turken & Dronkers, 2011) might contribute to the
selection between literal and non-literal meanings. Another possible role for the left IFG
is the integration of world knowledge into a context (Rapp et al., 2011; Rapp et al., 2012),
and that lower cloze probability of metaphoric stimuli (Rapp et al., 2004; Schneider
et al., 2014) contributes to activation.
Another significant area in the meta-analysis is the left superior temporal gyrus.
This brain region is located near Wernicke’s speech area and maintains anatomical
connections to numerous other semantic association centers (Turken & Dronkers, 2011),
like BA 45/47 (Bahlmann, Mueller, Makuuchi, & Friederici, 2011; Dronkers, Wilkins,
Van Valin, Redfern, & Jaeger, 2004; Weiller, Bormann, Saur, Musso, & Rijntjes, 2011), to
the more posterior superior part of the left inferior frontal gyrus (Broca’s area) (Weiller
et al., 2011), to other parts of the left lateral prefrontal cortex including Brodmann areas
6 and 46 (Turken & Dronkers, 2011; Weiller et al., 2011) and to the inferior parietal lobule
(BA 39) (Turken & Dronkers, 2011). Recent research combining high temporal resolu-
tion imaging procedures with imaging methods in metaphor comprehension highlights
a temporally early role in posterior brain regions (Schneider et al., 2014; Schneider et al.,
2015). The left middle and superior temporal gyri may play such a role. A key role for
this brain area in understanding non-literal language is compatible with lesion studies
(Gagnon, Goulet, Giroux, & Joanette, 2003; Winner & Gardner, 1977). Further analysis
of functional connectivity fMRI in this region would be quite interesting, especially
since the first such study found sufficient evidence for such involvement (Mashal, Faust,
& Hendler, 2005). The possible functional role also includes the setting up of cohesion
(Ferstl, Neumann, Bogler, & von Cramon, 2008; Rapp et al., 2012), a function that can
be attributed not only to the left hemisphere (LH), but also to the right hemisphere (RH)
homologue (e.g., Burgess & Chiarello, 1996; Kircher, Brammer, Tous Andreu, Williams,
& McGuire, 2001; Mashal et al., 2005). With only 23% of the reported foci (60 foci re-
ported in the literature are in the temporal lobe, 14 of them in the RH), the temporal
lobe has the least pronounced left lateralization. In particular, the frontal lobes (35% of
the reported foci) and the sub-lobar regions (33% of reported foci) have higher values.
However, we must again emphasize that this type of analysis is not qualified to itemize
the functional significance of reported foci. For example, functional connectivity
analyses with Hebrew two-word metaphors (Mashal et al., 2005) indicate that the role
of the right hemisphere homologue is substantial. The role of task instruction in fMRI
has probably been investigated too little, since lesion studies indicate that the role of the
right hemisphere homologue is critically dependent upon task instruction (Rapp, 2012;
Rinaldi, Marangolo, & Baldassarri, 2004; Winner & Gardner, 1977).
Another significant cluster of the meta- analysis can be found in the left
parahippocampal gyrus. This brain region has received minor attention in the lesion
studies. Foci from Yang, Edens, Simpson, and Krawczyk (2009) and Forgacs et al. (2012)
contribute most to the significance of this cluster in this ALE-analysis, but activations in
the left hemisphere parahippocampus are also reported by others (Citron & Goldberg,
2014; Desai et al., 2013; Diaz, Barrett, & Hogstrom, 2011; Mashal et al., 2005; Mashal
et al., 2013; Schmidt & Seger, 2009). It further plays a role in processing other types of
non-literal language like irony (Akimoto et al., 2014) and idioms (Desai et al., 2013).
Despite an apparent robustness of activations, the role of the parahippocampus has pre-
viously been discussed very little (see Rapp et al., 2012). The parahippocampal gyrus is
part of the limbic system and contributes to linguistic ambiguity resolution on the sen-
tence level (Hoenig & Scheef, 2005; Schmolck, Stefanacci, & Squire, 2000; Snijders et al.,
2009), which could well explain its functional role. However, the brain region has multi-
modal functions (Aminoff, Kveraga, & Bar, 2013), and the parahippocampal gyrus may
also contribute to contextual or emotion processing.
It is interesting to note that clinical populations, such as patients with schizophrenia,
show activation differences in the left parahippocampal gyrus during the processing
of metaphors (Mashal et al., 2013) and irony (Rapp et al., 2013). Although not signif-
icant in our meta-analysis, the right parahippocampal gyrus is also utilized during
the processing of non-literal language. For example, it is activated when metaphors
are produced (Benedek et al., 2014). On the basis of their meta-analysis, Yang and Shu
(2016) suggested that the parahippocampal gyrus is used in the simulation of spatial
information (see also Grill-Spector & Weiner, 2014). Proverbio et al. (2009) noted that
it could be responsible for “providing emotional connotation” during non-literal lan-
guage comprehension. The parahippocampal gyrus has extensive anatomical crosslinks
to the aforementioned regions, such as the left IFG/BA 47 (Petrides, 2005), and func-
tional connectivity to Wernicke’s area has been shown using fMRI (Mashal et al., 2005).
Entrenched in linguistic literature is the idea that thinking processes include em-
bodiment as a central element (Gallese & Lakoff, 2005; Lakoff, 2014; Lakoff & Johnson,
2008). The idea is that fictive motion is grounded in the sensorimotor system (Zhong
& Liljenquist, 2006). What has received much attention recently is the involvement
of the motor cortices and brain regions presumably associated with embodiment.
Currently, the topic remains controversial in studies of non-literal as well as literal lan-
guage (de Zubicaray, Arciuli, & McMahon, 2013; Watson, Cardillo, Ianni, & Chatterjee,
2013). However, there is an increasing number of studies concentrating on the use of
metaphors (Aziz- Zadeh, Wilson, Rizzolatti, & Iacoboni, 2006; Chen, Widick, &
Chatterjee, 2008; Desai et al., 2013; Romero Lauro et al., 2013) and idioms (Boulenger,
Hauk, & Pulvermueller, 2009; Desai et al., 2013; Raposo et al., 2009; Schuil, Smits, &
Zwaan, 2013). Several studies have demonstrated activation in the motor cortices during
the processing of metaphors. A meta-analysis conducted by Yang and Shu (2016) used
fictive motion sentences (such as “the road runs along the coast”) and metaphoric
actions. Fictive motion sentences contain a word that describes motion (in this case,
“runs”), and regions for spatial encoding and recognition were expected in their meta-
analysis, which were indeed found. However, this type of metaphor did not exhibit acti-
vation in the left premotor cortex. Fictive motion sentences bring together the concepts
of conceptual metaphors and embodiment. Perhaps the right parahippocampal
gyrus is involved in simulating information spatially (Bellmund, Deuker, Schroder, &
Doeller, 2016; Benoit & Schacter, 2015). Seen separately, but also grounded in the no-
tion of embodied cognition, are metaphoric action sentences (such as “grasp the idea”).
These sentences are sometimes said to activate the premotor and motor cortices, and
are often used to describe mental states (for example, “Matilde throws her sadness far
away”; Romero Lauro et al., 2013). Some studies aimed to find a “somatotopy” for ac-
tion metaphors, proverbs, and disentangled activation for “upper and lower limb
metaphors.” Yang and Shu (2016) found that metaphoric action lateralized to the left.
However, some studies included negative findings. Aziz-Zadeh et al. (2006) studied the
comprehension of metaphor and literal phrases containing hand, mouth, and foot ac-
tion verbs. Their results showed that within the premotor cortex in the left hemisphere,
a somatotopic activation pattern was found for literal phrases but not for metaphoric
phrases. From that perspective, it would be logical to expect that regions of higher-order
action intention (such as the inferior parietal lobule; Sowden & Catmur, 2015) also play
a role in figurative language comprehension. For example, in the metaphor “to grasp the
idea,” knowing the intention of the action grasp (i.e., to get something that is needed)
might be important for assessing the figurative meaning (i.e., to understand the idea).
Figure 28.1 demonstrates that the network involved for metaphor comprehension is
not limited to the regions described (see Rapp et al., 2012, and Rapp, 2013, for further
discussion of other brain regions involved). Other participating regions are the left su-
perior fontal gyrus (with a speculative role in the uncertainty dissolve; Bach & Dolan,
2012), the medial prefrontal cortex (perhaps for suppression of the literal meaning;
Cacciari & Papagno, 2012; Papagno & Romero-Lauro, 2010), the temporoparietal junc-
tion (for semantic and context integration), the cerebellum (which has a proven func-
tion in non-literal language comprehension; Cook, Murdoch, Cahill, & Whelan, 2004;
Murdoch, 2010; Rapp et al., 2012), the precuneus (perhaps for mental imagery; Mashal,
Vishne, & Laor, 2014), and the thalamus (suggested for identifying attributive categories
and establishing semantic associations; Stringaris, Medford, Giampietro, Brammer, &
David, 2007; or for ambiguity resolution; Uchiyama et al., 2012).
Brain Correlates of Idiom Processing
The next section of this chapter is on brain processing of idioms. Unlike metaphors,
idioms have a unique meaning. Several definitions exist for idioms (Nunberg, Sag,
& Wasow, 1994); idioms are a type of non-literal language; however—in contrast to
metaphors—they do not necessarily have a wrong literal meaning.1 It is prevalently
assumed that idiomatic meaning, assumed unlike metaphoric meaning, is stored
in the semantic memory as fixed “multiword strings,” like other multiword strings,
such as lines of poetry and popular song titles (Cacciari & Papagno, 2012). It has been
assumed that such multiword strings may be stored as one unit in semantic memory.
Consequently, in a recent overview on hemispheric processing of idioms, Cacciari and
Papagno argue that the cognitive processes for the two may differ: “Metaphors have
to do with categorization processes, while idioms have to do with retrieval from se-
mantic memory” (Cacciari & Papagno, 2012, p. 370). A practical definition for research
on their neuroanatomy is stated by Cacciari and Papagno (2012, p. 369): “Idiomatic is
a fixed string of constituents whose meaning is not necessarily derived from that of its
constituents.” Further, because of the fixed meaning, one cannot suddenly “invent” an
idiom, while this is possible for metaphors. The discussion of which are more frequent
in everyday speech samples is ongoing.
A systematic search of the present literature yielded 11 fMRI studies on idioms and one
for proverbs, which is lower than the number of studies on brain lesions and idioms (14).
Table 28.1 lists the fMRI studies. The first study was published in 2007 (Zempleni et al.,
2007) and used Dutch idioms. The main finding of both studies was bilateral activation
in the IFG and middle temporal gyrus for idioms > literal sentences. The reverse con-
trast (literal > figurative) also showed bilateral activations. Zempleni also investigated
a small number of patients with schizophrenia and found aberrant activation in them
(Zempleni et al., 2006).
In 2008, three studies were published. In a study on Hebrew, Mashal et al. (Mashal,
Faust, Hendler, & Jung-Beeman, 2008) investigated comprehension of idioms. Their
main findings were that brain activation for comprehension of the (correct) figura-
tive meaning of the idiom not only contrasted that for comprehension of literal control
sentences, but also differed from that for processing the literal meaning of the idiom.
The paper showed literal interpretation of idioms was associated with “increased ac-
tivity in right brain regions including the right precuneus, right middle frontal gyrus
1
For example, the idiom “it takes two to tango” has a literally correct meaning.
(MFG), right posterior middle temporal gyrus (MTG), and right anterior superior tem-
poral gyrus” (p. 848).
Lauro, Tettamanti, Cappa, and Papagno (2008) investigated the comprehension of
idioms versus that of literal sentences in Italian. Their participants performed a picture-
matching task, a task type that is often used in brain lesion studies. They also applied a
dynamic causal modelling (DCM) analysis. Again, bilateral inferior frontal and right
temporal activation were found. In addition, in accordance with previous findings, the
opposite contrast activates right temporal regions.
The first study on English idioms was published by Boulenger et al. (2009), who
investigated the role of the motor system in idiom comprehension. Again, left IFG ac-
tivation was observed. In contrast to previous studies, no right cerebral activation was
found. In a study from the United Kingdom, Raposo et al. (2009) investigated idiomatic
action sentences. Although they included 56 stimuli and 22 participants, no contrast
for idiomatic > literal sentences was detectable in this study. The study with the highest
number of subjects (n = 37) (Kana, Murdaugh, Wolfe, & Kumar, 2012) also yielded a
negative result. In this study, no activation was detected in the direct comparison be-
tween idioms > literal sentences; instead, the opposite contrast showed several activa-
tion clusters, mainly in the right hemisphere.
Another English-language study (Hillert & Buracas, 2009) used two types of id-
iomatic expressions, explicit and implicit, and compared them with literal sentences.
Explicit idiomatic sentences activated the left IFG, while the contrast ambiguous idio-
matic sentences > literal showed activation mainly in the medial prefrontal regions.
In a recent study with Chinese idioms, (Yang et al., 2016) investigated both opaque
and transparent idioms. While opaque idioms activated left hemisphere limbic and
right hemisphere temporoparietal regions, transparent idioms elicited only activation
of the left hemisphere.
Two studies investigated both idioms and metaphors (Desai et al., 2013; Romero
Lauro et al., 2013) and provided evidence for differences in brain processing between
idioms and metaphors. Desai et al. (2013) reported differential activation in temporal
and posterior brain regions, but less in the prefrontal regions.
The studies available on idiom comprehension offer the possibility of a coordinate-
based meta-analysis. However, the current follow-up analysis, like the previous one
(Rapp et al., 2012), has the severe limitation of having a very low number of reported
foci. The 12 published studies report 74 exploitable foci from 265 participants, 48 foci in
the left hemisphere and 25 in the right hemisphere. Schuil et al. (2013) reported only one
focus in the midline posterior cingulate. Figure 28.2 shows their projections onto a brain
surface. A meta-analysis (GingerALE 2.5, liberal threshold of 0.001 uncorrected, chosen
minimum cluster size = 100 mm3; see table 28.3) showed significant clusters in the left BA
45, left middle temporal gyrus, precuneus, and left hemisphere parahippocampal gyrus.
Numerous studies reported left inferior frontal gyrus activation (Boulenger et al., 2009;
Fernandino et al., 2013; Hillert & Buracas, 2009; Lauro et al., 2008; Mashal et al., 2008;
Romero Lauro et al., 2013; Zempleni, Haverkort, Renken, & Stowe, 2007; Zempleni et al.,
2006). The number of reported foci in the temporal lobes is equal for both hemispheres
Figure 28.2. Projection onto a brain surface: foci for fMRI-activation idioms > literal stimuli.
The 12 published studies report 74 exploitable foci from 265 study subjects, 48 foci in the left hem-
isphere and 25 in the right hemisphere.
(eight each), but more spatially distinct in the right hemisphere (see Rapp et al., 2012, for
a discussion on this topic). Altogether, coordinate-based analysis clearly indicates that
both hemispheres contribute to idiom processing.
Worth mentioning is the role of the medial prefrontal cortex in idiom compre-
hension, which is also supported by brain lesion research (Cacciari & Papagno, 2012;
Cacciari et al., 2006; Papagno & Caporali, 2007; Papagno, Curti, Rizzo, Crippa, &
Colombo, 2006, Papagno, Tabossi, Colombo, & Zampetti, 2004). According to the
model of idiom comprehension put forward by Papagno and Lauro (Lauro et al., 2008;
Papagno & Romero-Lauro, 2010), the medial prefrontal regions are responsible for re-
sponse monitoring during idiom comprehension and suppression of alternative (lit-
eral) interpretations. Beyond that, “theory of mind” is essential for the comprehension
of many non-literal stimuli, so that the medial prefrontal contribution to theory of
mind (Amodio & Frith, 2006) also represents a functional role for this brain region.
Contributions from the medial prefrontal cortex are also found in other types of non-
literal language, such as irony (see Rapp et al., 2010, for review), metonymy (Rapp et al.,
2011), and metaphors (Subramaniam et al., 2013).
Implications for Further Research
Although knowledge and data have substantially increased during the past few years,
several issues make further research worthwhile. One field is the developmental per-
spective. Metaphor comprehension has a rather late developmental biology contin-
uing during childhood (Nippold & Taylor, 1995; Nippold, Uhden, & Schwarz, 1997),
so from its timing it possibly interrelates with critical maturation and lateralization
processes, which has been postulated to be of interest for understanding impairment
in diseases (Sakai, 2005; Sommer & Kahn, 2009). To better understand the functional
neuroanatomy, further studies on the interplay between brain regions (e.g., connectivity
analyses) are worthwhile. The same is true for a more integrative perspective from fMRI
to comprehensive research using techniques with better temporal resolution. It is now
possible, for example, to combine electrophysiology and blood oxygenation level de-
pendent (BOLD)–based techniques (Schneider et al., 2014), or fMRI and eye-movement
tracking (Bonhage, Mueller, Friederici, & Fiebach, 2015). However, in the case of idioms
and metaphors, such combinations are only in their infancy. fMRI research in clinical
populations, such as patients with schizophrenia, is increasing (Kircher et al., 2007;
Mashal et al., 2013; Mashal et al., 2014; Schneider et al., 2015), while autism and other
diseases are less well studied. Metaphor aptness (Chiappe, Kennedy, & Smykowski,
2003; Giora, 2002) and personality traits (Rapp et al., 2010) have also been studied very
little using today´s imaging research techniques.
Emotion (see van Berkum, Chapter 29 in this volume) is a potential factor in the pro-
cess of understanding the neural correlates of metaphors (Citron & Goldberg, 2014;
Kövecses, 2003; Mirous & Beeman, 2012), as metaphors are often used to express emo-
tional content (Fainsilber & Ortony, 1987; Gibbs & Leggitt, 2002). Metaphors showed
different fMRI activation in an emotional valuation paradigm than in a metaphoricity
judgment paradigm in our previous study (Rapp et al., 2007), while there were no
differences found between emotionally positive and negative metaphors (unpub-
lished data). In contrast, Forgacs et al. (2014) found a stronger activation of emotionally
negative-colored metaphors in the same brain region. Overall, emotional content in the
more recent studies seems to attract more consideration. A comprehensive fMRI study
(Subramaniam et al., 2013) examined the effects of the emotional content of metaphors
on brain activation. These confirmed a specific role of the left IFG. The medial pre-
frontal cortex seems to play an important role in affect recognition during metaphor
comprehension. The authors suggested a connection with processes of cognitive control
processes (Subramaniam et al., 2013). In a recently published study, Samur et al. (2015)
showed that the emotional context in metaphorical interpretation of a sentence affected
brain activation in the areas of movement (visual motion areas).
It is always subjective to suggest what research needs to address most urgently, but
a rather neglected area is the explanation of brain activation for literal sentences >
metaphors. Recall that metaphors, even novel ones, are not necessarily more difficult
to process and do not necessarily require longer reaction times (Blasko & Connine,
1993; Glucksberg, 2003). There is frequently reported activation for literal > non-literal
stimuli in fMRI investigations, but it is rarely discussed. To use idioms as an example, all
nine studies analyzed the contrast for literal > idiomatic stimuli (Figure 28.3); however,
only Boulenger et al. (2009) and Yang et al. (2016) reported a negative result.
For example, Kana et al. (2012) reported, “Idiomatic sentences showed greater acti-
vation in the left fusiform gyrus, right parahippocampal gyrus, bilateral STG, LMTG,
and RMFG” (p. 20). All the other studies reported the coordinates of their results; these
studies together reported 54 foci, which is a comparable magnitude to the foci reported
in the “other direction.” Their projection onto a brain surface is shown in Figure 28.3,
Figure 28.3. Projection onto a brain surface: 37 foci reported in 8 studies for differential
contrasts literal stimuli > idioms.
and makes clear that they are not restricted to areas classically attributed to deactivation
phenomena. It is still a question for further research to offer a better explanation.
References
Ahrens, K., Liu, H. L., Lee, C. Y., Gong, S. P., Fang, S. Y., & Hsu, Y. Y. (2007). Functional MRI
of conventional and anomalous metaphors in Mandarin Chinese. Brain and Language, 100,
163–171.
Akimoto, Y., Sugiura, M., Yomogida, Y., Miyauchi, C. M., Miyazawa, S., & Kawashima, R.
(2014). Irony comprehension: Social conceptual knowledge and emotional response.
Aminoff, E. M., Kveraga, K., & Bar, M. (2013). The role of the parahippocampal cortex in cogni-
tion. Trends in Cognitive Sciences, 17, 379–390.
Amodio, D. M., & Frith, C. D. (2006). Meeting of minds: The medial frontal cortex and social
cognition. Nature Reviews Neuroscience, 7, 268–277.
Annaz, D., Van Herwegen, J., Thomas, M., Fishman, R., Karmiloff-Smith, A. & Rundblad, G.
(2009). Comprehension of metaphor and metonymy in children with Williams syndrome.
International Journal of Language & Communication Disorders, 44, 962–978.
Aziz-Zadeh, L., Wilson, S. M., Rizzolatti, G., & Iacoboni, M. (2006). Congruent embodied
representations for visually presented actions and linguistic phrases describing actions.
Current Biology, 16, 1818–1823.
Bach, D. R., & Dolan, R. J. (2012). Knowing how much you don’t know: A neural organization
of uncertainty estimates. Nature Reviews Neuroscience, 13, 572–586.
trol of memory. Neuropsychologia, 45, 2883–2901.
Bahlmann, J., Mueller, J. L., Makuuchi, M., & Friederici, A. D. (2011). Perisylvian functional
connectivity during processing of sentential negation. Frontiers in Psychology, 2, 104.
doi: 10.3389/fpsyg.2011.00104.
Bambini, V., Arcara, G., Martinelli, I., Bernini, S., Alvisi, E., Moro, A., Cappa, S. F., & Ceroni,
M. (2016). Communication and pragmatic breakdowns in amyotrophic lateral sclerosis
patients. Brain and Language 153, 1–12.
Bambini, V., Gentili, C., Ricciardi, E., Bertinetto, P. M., & Pietrini, P. (2011). Decomposing met-
aphor processing at the cognitive and neural level through functional magnetic resonance
imaging. Brain Research Bulletin, 86, 203–216.
Bellmund, J. L. S., Deuker, L., Schroder, T. N., & Doeller, C. F. (2016). Grid-cell representations
in mental simulation. Elife. 2016 Aug 30;5. pii: e17089.
Benedek, M., Beaty, R., Jauk, E., Koschutnig, K., Fink, A., Silvia, P. J., Dunst, B., & Neubauer,
A. C. (2014). Creating metaphors: The neural basis of figurative language production.
Benoit, R. G., & Schacter, D. L. (2015). Specifying the core network supporting episodic sim-
ulation and episodic memory by activation likelihood estimation. Neuropsychologia, 75,
450–457.
19, 2767–2796.
Blasko, D. G., & Connine, C. M. (1993). Effects of familiarity and aptness on metaphor pro-
cessing. Journal of Experimental Psychology-Learning Memory and Cognition, 19, 295–308.
Blasko, D. G., & Kazmerski, V. A. (2006). ERP correlates of individual differences in the com-
prehension of nonliteral language. Metaphor and Symbol, 21, 267–284.
Bohrn, I. C., Altmann, U., & Jacobs, A. M. (2012). Looking at the brains behind figurative lan-
guage: A quantitative meta-analysis of neuroimaging studies on metaphor, idiom, and irony
processing. Neuropsychologia, 50, 2669–2683.
Bohrn, I. C., Altmann, U., Lubrich, O., Menninghaus, W., & Jacobs, A. M. (2012). Old prov-
erbs in new skins: An fMRI study on defamiliarization. Frontiers in Psychology 2012,3, 204.
doi: 10.3389/fpsyg.2012.00204.
Bonhage, C. E., Mueller, J. L., Friederici, A. D., & Fiebach, C. J. (2015). Combined eye tracking
and fMRI reveals neural basis of linguistic predictions during sentence comprehension.
Cortex, 68,  33–47.
Bookheimer, S. (2002). Functional MRI of language: New approaches to understanding the
cortical organization of semantic processing. Annual Review of Neuroscience, 25, 151–188.
Bottini, G., Corcoran, R., Sterzi, R., Paulesu, E., Schenone, P., Scarpa, P., Frackowiak, R. S., &
Frith, C. D. (1994). The role of the right hemisphere in the interpretation of figurative aspects
of language: A positron emission tomography activation study. Brain, 117(Pt 6), 1241–1253.
Boulenger, V., Hauk, O., & Pulvermueller, F. (2009). Grasping ideas with the motor
system: Semantic somatotopy in idiom comprehension. Cerebral Cortex, 19, 1905–1914.
Bowdle, B. F., & Gentner, D. (2005). The career of metaphor. Psychological Review, 112, 193–216.
BrainMapDevelopmentTeam. (2013). User manual for GingerALE 2.3. P. Fox, A. Laird, S.
Eickhoff, J. Lancaster, M. Fox, A. Uecker, M. Robertson, and K. Ray, Eds. San Antonio,
TX: Research Imaging Institute, UT Health Science Center San Antonio.
Brownell, H. H., Simpson, T. L., Bihrle, A. M., Potter, H. H., & Gardner, H. (1990). Appreciation
of metaphoric alternative word meanings by left and right brain- damaged patients.
Burgess, C., & Chiarello, C. (1996). Neurocognitive mechanisms underlying metaphor com-
prehension and other figurative language. Metaphor and Symbolic Activity, 11,  67–84.
Cacciari, C., & Papagno, C. (2012). Neuropsychological and neurophysiological correlates

of idiom understanding: How many hemispheres are involved? In M. Faust (Ed.),
The handbook of the neuropsychology of language (pp. 368– 385). Chichester, West
Sussex: Wiley-Blackwell.
Cacciari, C., Reati, F., Colombo, M. R., Padovani, R., Rizzo, S., & Papagno, C. (2006). The com-
prehension of ambiguous idioms in aphasic patients. Neuropsychologia, 44, 1305–1314.
Cameron, L., & Deignan, A. (2006). The emergence of metaphor in discourse. Applied
Linguistics, 27, 671–690.
Cardillo, E. R., Watson, C. E., Schmidt, G. L., Kranjec, A., & Chatterjee, A. (2012). From novel
to familiar: Tuning the brain for metaphors. NeuroImage, 59, 3212–3221.
Chapman, L. J. (1960). Confusion of figurative and literal usages of words by schizophrenics
and brain damaged patients. Journal of Abnormal and Social Psychology, 60, 412–416.
Charteris-Black, J. (2002). Second language figurative proficiency: A comparative study of
Malay and English. Applied Linguistics, 23, 104–133.
Chen, E., Widick, P., & Chatterjee, A. (2008). Functional-anatomical organization of predicate
metaphor processing. Brain and Language, 107, 194–202.
Chiappe, D., Kennedy, J. M., & Smykowski, T. (2003). Reversibility, aptness, and the conven-
tionality of metaphors and similes. Metaphor and Symbol, 18, 85–105.
Citron, F. M. M., & Goldberg, A. E. (2014). Metaphorical sentences are more emotionally en-
gaging than their literal counterparts. Journal of Cognitive Neuroscience, 26, 2585–2595.
Colston, H. L., & Katz, A. N. (2004). Figurative language comprehension: Social and cultural
influences. New York: Routledge.
Cook, M., Murdoch, B., Cahill, L., & Whelan, B. M. (2004). Higher-level language deficits
resulting from left primary cerebellar lesions. Aphasiology, 18, 771–784.
Cui, X., Li, J., & Song, X. (2011). xjview: A viewing program for SPM. http://www.alivelearn.
net/xjview.
de Zubicaray, G., Arciuli, J., & McMahon, K. (2013). Putting an “end” to the motor cortex
representations of action words. Journal of Cognitive Neuroscience, 25, 1957–1974.
Denke, C., Rotte, M., Heinze, H.-J., & Schaefer, M. (2014). Lying and the subsequent desire for
toothpaste: Activity in the somatosensory cortex predicts embodiment of the moral-purity
metaphor. Cerebral Cortex (2016) 26(2):477–484.
Desai, R. H., Binder, J. R., Conant, L. L., Mano, Q. R., & Seidenberg, M. S. (2011). The neural ca-
reer of sensory-motor metaphors. Journal of Cognitive Neuroscience, 23, 2376–2386.
Desai, R. H., Conant, L. L., Binder, J. R., Park, H., & Seidenberg, M. S. (2013). A piece of the ac-
tion: Modulation of sensory-motor regions by action idioms and metaphors. NeuroImage,
83, 862–869.
Diaz, M. T., Barrett, K. T., & Hogstrom, L. J. (2011). The influence of sentence novelty and figur-
ativeness on brain activity. Neuropsychologia, 49, 320–330.
Diaz, M. T., & Hogstrom, L. J. (2011). The influence of context on hemispheric recruitment
during metaphor processing. Journal of Cognitive Neuroscience, 23, 3586–3597.
Dronkers, N. F., Wilkins, D. P., Van Valin, R. D., Redfern, B. B., & Jaeger, J. J. (2004). Lesion
analysis of the brain areas involved in language comprehension. Cognition, 92, 145–177.
Eickhoff, S. B., Laird, A. R., Grefkes, C., Wang, L. E., Zilles, K., & Fox, P. T. (2009). Coordinate‐
based activation likelihood estimation meta‐analysis of neuroimaging data: A random‐
effects approach based on empirical estimates of spatial uncertainty. Human Brain Mapping,
30, 2907–2926.
Ettinger, U., Mohr, C., Gooding, D. C., Cohen, A. S., Rapp, A., Haenschel, C., & Park, S. (2015).
Cognition and brain function in schizotypy: A selective review. Schizophrenia Bulletin,
41(Suppl 2), S417–S426.
Eviatar, Z. & Just, M. A. (2006). Brain correlates of discourse processing: An fMRI investiga-
tion of irony and conventional metaphor comprehension. Neuropsychologia, 44, 2348–2359.
Fainsilber, L., & Ortony, A. (1987). Metaphorical uses of language in the expression of emotions.
Metaphor and Symbolic Activity, 2, 239–250.
Fernandino, L., Conant, L. L., Binder, J. R., Blindauer, K., Hiner, B., Spangler, K., & Desai,
R. H. (2013). Where is the action? Action sentence processing in Parkinson’s disease.
Ferstl, E. C., Neumann, J., Bogler, C., & von Cramon, D. Y. (2008). The extended language
network: A meta-analysis of neuroimaging studies on text comprehension. Human Brain
Mapping, 29, 581–593.
Forgacs, B., Bohrn, I., Baudewig, J., Hofmann, M. J., Pleh, C., & Jacobs, A. M. (2012). Neural
correlates of combinatorial semantic processing of literal and figurative noun noun com-
pound words. NeuroImage, 63, 1432–1442.
Forgacs, B., Lukacs, A., & Pleh, C. (2014). Lateralized processing of novel metaphors:
Disentangling figurativeness and novelty. Neuropsychologia, 56, 101–109.
Fox, P., Eickhoff, S., Lancaster, J., Fox, M., Uecker, A., Robertson, M., & Ray, K. (2013).
GingerALE 2.3. San Antonio, TX: Brainmap Development Team.
Gagnon, L., Goulet, P., Giroux, F., & Joanette, Y. (2003). Processing of metaphoric and non-
metaphoric alternative meanings of words after right-and left-hemispheric lesion. Brain
Gallese, V., & Lakoff, G. (2005). The brain’s concepts: The role of the sensory-motor system in
conceptual knowledge. Cognitive Neuropsychology, 22, 455–479.
Gibbs, R. W., Jr., & Colston, H. L. (2012). Interpreting figurative meaning. Cambridge: Cambridge
University Press.
Gibbs, R. W., & Leggitt, J. S. (2002). What’s special about figurative language in emotional com-
munication. In E. A. Turner (Ed.), The verbal communication of emotions: Interdisciplinary
perspectives (pp. 125–149). New York: Lawrence Erlbaum.
Gibbs, R. W., Lima, P. L. C., & Francozo, E. (2004). Metaphor is grounded in embodied experi-
ence. Journal of Pragmatics, 36, 1189–1210.
Giora, R. (1997). Understanding figurative and literal language: The graded salience hypothesis
(psycholinguistics). Cognitive Linguistics, 8, 183–206.
Giora, R. (1999). On the priority of salient meanings: Studies of literal and figurative language.
Journal of Pragmatics, 31, 919–929.
Giora, R. (2002). Literal vs. figurative language: Different or equal? Journal of Pragmatics, 34,
487–506.
Glucksberg, S. (2003). The psycholinguistics of metaphor. Trends in Cognitive Sciences,
7,  92–96.
Gold, R., Faust, M., & Goldstein, A. (2010). Semantic integration during metaphor compre-
hension in Asperger syndrome. Brain and Language, 113, 124–134.
Grill-Spector, K., & Weiner, K. S. (2014). The functional architecture of the ventral temporal
cortex and its role in categorization. Nature Reviews Neuroscience, 15, 536–548.
Gutmann, M. L. (2009). The effect of frontal lobe function on proverb interpretation in
Parkinson’s disease. Tucson: University of Arizona.
Heredia, R., & Cieslicka, A. (2015). Bilingual figurative language processing.

Hillert, D. G., & Buracas, G. T. (2009). The neural substrates of spoken idiom comprehension.
Language and Cognitive Processes, 24, 1370–1391.
Hoenig, K., & Scheef, L. (2005). Mediotemporal contributions to semantic processing: fMRI
evidence from ambiguity processing during semantic context verification. Hippocampus, 15,
597–609.
Jäger, A. O., & Althoff, K. (1994). Der WILDE-Intelligenz-Test:(WIT); ein Strukturdiagnostikum.
Göttingen: Hogrefe, Verlag für Psychologie.
Kana, R. K., Murdaugh, D. L., Wolfe, K. R., & Kumar, S. L. (2012). Brain responses mediating
idiom comprehension: Gender and hemispheric differences. Brain Research, 1467,  18–26.
Kircher, T. T., Brammer, M., Tous Andreu, N., Williams, S. C., & McGuire, P. K. (2001).
Engagement of right temporal cortex during processing of linguistic context.
Kircher, T. T., Leube, D. T., Erb, M., Grodd, W., & Rapp, A. M. (2007). Neural correlates of met-
aphor processing in schizophrenia. NeuroImage, 34, 281–289.
Klepousniotou, E., Gracco, V. L., & Pike, G. B. (2014). Pathways to lexical ambiguity: fMRI evi-
dence for bilateral fronto-parietal involvement in language processing. Brain and Language,
131,  56–64.
Kövecses, Z. (2003). Metaphor and emotion: Language, culture, and body in human feeling.
Lacey, S., Stilla, R., & Sathian, K. (2012). Metaphorically feeling: Comprehending textural
metaphors activates somatosensory cortex. Brain and Language, 120, 416–421.
Lai, V. T., van Dam, W., Conant, L. L., Binder, J. R., & Desai, R. H. (2015). Familiarity differen-
tially affects right hemisphere contributions to processing metaphors and literals. Frontiers
in Human Neuroscience, 9, 44. doi: 10.3389/fnhum.2015.00044.
Lakoff, G. (2014). Mapping the brain’s metaphor circuitry: Metaphorical thought in everyday
reason. Front Hum Neurosci. 2014 Dec 16;8: 958. doi: 10.3389/fnhum.2014.00958.
Lakoff, G., & Johnson, M. (2008). Metaphors we live by. Chicago: University of Chicago Press.
Lancaster, J. L., Tordesillas-Gutierrez, D., Martinez, M., Salinas, F., Evans, A., Zilles, K.,
Mazziotta, J. C., & Fox, P. T. (2007). Bias between MNI and talairach coordinates analyzed
using the ICBM-152 brain template. Human Brain Mapping, 28, 1194–1205.
Landau, M. J., Meier, B. P., & Keefer, L. A. (2010). A metaphor-enriched social cognition.
Psychological Bulletin, 136, 1045.
Langdon, R., Coltheart, M., Ward, P. B., & Catts, S. V. (2002). Disturbed communication in
schizophrenia: The role of poor pragmatics and poor mind-reading. Psychological Medicine,
32, 1273–1284.
Lauro, L. J., Tettamanti, M., Cappa, S. F., & Papagno, C. (2008). Idiom comprehension: A pre-
frontal task? Cerebral Cortex, 18, 162–170.
Lee, S. S., & Dapretto, M. (2006). Metaphorical vs. literal word meanings: fMRI evidence
against a selective role of the right hemisphere. NeuroImage, 29, 536–544.
Lindell, A. K. (2006). In your right mind: Right hemisphere contributions to language pro-
cessing and production. Neuropsychology Review, 16, 131–148.
Littlemore, J. (2001). Metaphoric competence: A language learning strength of students with a
holistic cognitive style? TESOL Quarterly, 35, 459–491.
Martin, I., & McDonald, S. (2005). Exploring the causes of pragmatic language deficits fol-
lowing traumatic brain injury. Aphasiology, 19, 712–730.
Mashal, N., Borodkin, K., Maliniak, O., & Faust, M. (2015). Hemispheric involvement in native
and non-native comprehension of conventional metaphors. Journal of Neurolinguistics, 35,
96–108.
Mashal, N., & Faust, M. (2010). The effects of metaphoricity and presentation style on brain ac-
tivation during text comprehension. Metaphor and Symbol, 25,  19–33.
Mashal, N., Faust, M., & Hendler, T. (2005). The role of the right hemisphere in processing
nonsalient metaphorical meanings: Application of principal components analysis to fMRI
data. Neuropsychologia, 43, 2084–2100.
Mashal, N., Faust, M., Hendler, T., & Jung-Beeman, M. (2007). An fMRI investigation of the
neural correlates underlying the processing of novel metaphoric expressions. Brain and
Language, 100, 115–126.
Mashal, N., Faust, M., Hendler, T., & Jung-Beeman, M. (2008). Hemispheric differences in pro-
cessing the literal interpretation of idioms: Converging evidence from behavioral and fMRI
studies. Cortex, 44, 848–860.
Mashal, N., Faust, M., Hendler, T., & Jung-Beeman, M. (2009). An fMRI study of processing
novel metaphoric sentences. Laterality, 14,  30–54.
Mashal, N., Vishne, T., & Laor, N. (2014). The role of the precuneus in metaphor comprehen-
sion: Evidence from an fMRI study in people with schizophrenia and healthy participants.
Frontiers in Human Neuroscience, 8, 818. doi: 10.3389/fnhum.2014.00818.
Mashal, N., Vishne, T., Laor, N., & Titone, D. (2013). Enhanced left frontal involvement during
novel metaphor comprehension in schizophrenia: Evidence from functional neuroimaging.
Brain and Language, 124,  66–74.
Mejía-Constaín, B., Monchi, O., Walter, N., Arsenault, M., Senhadji, N., & Joanette, Y. (2010).
When metaphors go literally beyond their territories: The impact of age on figurative lan-
guage. Italian Journal of Linguistics, 22,  41–60.
Menenti, L., Petersson, K. M., Scheeringa, R., & Hagoort, P. (2009). When elephants
fly: Differential sensitivity of right and left inferior frontal gyri to discourse and world know-
ledge. Journal of Cognitive Neuroscience, 21, 2358–2368.
Mirous, H. J., & Beeman, M. (2012). Bilateral processing and affect in creative language com-
prehension. In M. Faust (Ed.), The handbook of the neuropsychology of language (pp. 319–
341). Chichester, West Sussex: Wiley-Blackwell.
Murdoch, B. E. (2010). The cerebellum and language: Historical perspective and review.
Cortex, 46, 858–868.
Nippold, M. A., & Taylor, C. L. (1995). Idiom understanding in youth: Further examination of
familiarity and transparency. Journal of Speech and Hearing Research, 38, 426–433.
Nippold, M. A., Uhden, L. D., & Schwarz, I. E. (1997). Proverb explanation through the
lifespan: A developmental study of adolescents and adults. Journal of Speech Language and
Hearing Research, 40, 245–253.
Nunberg, G., Sag, I. A., & Wasow, T. (1994). Idioms. Language, 70, 491–538.
Obert, A., Gierski, F., Calmus, A., Portefaix, C., Declercq, C., Pierot, L., & Caillies, S. (2014).
Differential bilateral involvement of the parietal gyrus during predicative metaphor pro-
cessing: An auditory fMRI study. Brain and Language, 137, 112–119.
Papagno, C., & Caporali, A. (2007). Testing idiom comprehension in aphasic patients: The
effects of task and idiom type. Brain and Language, 100, 208–220.
Papagno, C., Curti, R., Rizzo, S., Crippa, F., & Colombo, M. R. (2006). Is the right hemisphere
involved in idiom comprehension? A neuropsychological study. Neuropsychology, 20,
598–606.
Papagno, C., & Romero- Lauro, L. (2010). The neural basis of idiom processing:
Neuropsychological, neurophysiological and neuroimaging evidence. Italian Journal of
Linguistics, 22,  21–40.
Papagno, C., Tabossi, P., Colombo, M. R., & Zampetti, P. (2004). Idiom comprehension in
aphasic patients. Brain and Language, 89, 226–234.
Petrides, M. (2005). Lateral prefrontal cortex: Architectonic and functional organization.
Philosophical Transactions of the Royal Society B-Biological Sciences, 360, 781–795.
Prat, C. S., Mason, R. A., & Just, M. A. (2012). An fMRI investigation of analogical mapping in
metaphor comprehension: The influence of context and individual cognitive capacities on
processing demands. Journal of Experimental Psychology: Learning, Memory and Cognition,
38, 282–294.
Proverbio, A. M., Crotti, N., Zani, A., & Adorni, R. (2009). The role of left and right
hemispheres in the comprehension of idiomatic language: An electrical neuroimaging
study. BMC Neuroscience, 10, 116. doi: 10.1186/1471-2202-10-116.
Raposo, A., Moss, H. E., Stamatakis, E. A., & Tyler, L. K. (2009). Modulation of motor and pre-
motor cortices by actions, action words and action sentences. Neuropsychologia, 47, 388–396.
Rapp, A. (2009). The role of the right hemisphere for language in schizophrenia. In I.
E. Sommer & R. S. Kahn (Eds.), Language lateralization in psychosis (pp. 147– 156).
Rapp, A. M. (2012). The brain behind nonliteral language: Insights from brain imaging. In M.
Faust (Ed.), The handbook of the neuropsychology of language (pp. 406–424). Chichester,
West Sussex:Wiley-Blackwell.
Rapp, A. M. (2013). Where in the brain do metaphors become metaphoric? Research on the
functional neuroanatomy of nonliteral language. In M. Połczyńska, L. P. Pakuła, & D.
Jaworska (Eds.), Young linguists’ insights: Taking interdisciplinary approaches to the fore (pp.
153–165). Poznań: Wydział Anglistyki UAM.
Rapp, A. M., Erb, M., Grodd, W., Bartels, M., & Markert, K. (2011). Neural correlates of me-
tonymy resolution. Brain and Language, 119, 196–205.
Rapp, A. M., Langohr, K., Mutschler, D. E., Klingberg, S., Wild, B., & Erb, M. (2013). Isn’t it
ironic? Neural correlates of irony comprehension in schizophrenia. PLoS One, 8, e74224.
Rapp, A. M., Langohr, K., Mutschler, D. E., & Wild, B. (2014). Irony and proverb comprehen-
sion in schizophrenia: Do female patients “dislike” ironic remarks? Schizophrenia Research
and Treatment, 2014, 841086.
Rapp, A. M., Leube, D. T., Erb, M., Grodd, W., & Kircher, T. T. (2004). Neural correlates of met-
aphor processing. Cognitive Brain Research, 20, 395–402.
Rapp, A. M., Leube, D. T., Erb, M., Grodd, W. & Kircher, T. T. J. (2007). Laterality in meta-
phor processing: Lack of evidence from functional magnetic resonance imaging for the right
hemisphere theory. Brain and Language, 100, 142–149.
Rapp, A. M., Mutschler, D. E. & Erb, M. (2012). Where in the brain is nonliteral language?
A coordinate-based meta-analysis of functional magnetic resonance imaging studies.
Rapp, A. M., Mutschler, D. E., Wild, B., Erb, M., Lengsfeld, I., Saur, R., & Grodd, W. (2010).
Neural correlates of irony comprehension: The role of schizotypal personality traits. Brain
and Language, 113,  1–12.
Rapp, A., & Schmierer, P. (2010). Proverbs and nonliteral language in schizophrenia: A sys-
tematic methodological review of all studies published 1931–2010. Schizophrenia Research,
117, 422.
Rapp, A. M., & Wild, B. (2011). Nonliteral language in Alzheimer dementia: A review. Journal of
the International Neuropsychological Society, 17, 207–218.
Rinaldi, M. C., Marangolo, P., & Baldassarri, F. (2004). Metaphor comprehension in right
brain-damaged patients with visuo- verbal and verbal material: A dissociation (re)
considered. Cortex, 40, 479–490.
Romero Lauro, L. J., Mattavelli, G., Papagno, C., & Tettamanti, M. (2013). She runs, the road
runs, my mind runs, bad blood runs between us: Literal and figurative motion verbs: An
fMRI study. NeuroImage, 83, 361–371.
Sakai, K. L. (2005). Language acquisition and brain development. Science, 310, 815–819.
Samur, D., Lai, V. T., Hagoort, P. & Willems, R. M. (2015). Emotional context modulates
embodied metaphor comprehension. Neuropsychologia, 78, 108–114.
Schmidt, G. L., & Seger, C. A. (2009). Neural correlates of metaphor processing: The roles of
figurativeness, familiarity and difficulty. Brain and Cognition, 71, 375–386.
Schmolck, H., Stefanacci, L., & Squire, L. R. (2000). Detection and explanation of sentence
ambiguity are unaffected by hippocampal lesions but are impaired by larger temporal lobe
lesions. Hippocampus, 10, 759–770.
Schneider, S., Rapp, A. M., Haeussinger, F. B., Ernst, L. H., Hamm, F., Fallgatter, A. J., & Ehlis,
A. C. (2014). Beyond the N400: Complementary access to early neural correlates of novel
metaphor comprehension using combined electrophysiological and haemodynamic
measurements. Cortex, 53,  45–59.
Schneider, S., Wagels, L., Haeussinger, F. B., Fallgatter, A. J., Ehlis, A. C., & Rapp, A. M. (2015).
Haemodynamic and electrophysiological markers of pragmatic language comprehension in
schizophrenia. World Journal of Biological Psychiatry, 16 (6): 1–13.
Schuil, K. D. I., Smits, M., & Zwaan, R. A. (2013). Sentential context modulates the involve-
ment of the motor cortex in action language processing: An fMRI study. Frontiers in Human
Neuroscience, 7, 100. doi: 10.3389/fnhum.2013.00100.
Shibata, M. (2011). What is the difference between metaphor and simile? fMRI study (pp.
101–109). Centre for Advanced Research on Logic and Sensibility. The Global Centers of
Excellence Program, Keio University.
Shibata, M., Abe, J.-I., Terao, A., & Miyamoto, T. (2007). Neural mechanisms involved in the
comprehension of metaphoric and literal sentences: An fMRI study. Brain Research, 1166,
92–102.
Shibata, M., Toyomura, A., Motoyama, H., Itoh, H., Kawabata, Y., & Abe, J.-I. (2012). Does
simile comprehension differ from metaphor comprehension? A functional MRI study. Brain
Simpson, R., & Mendis, D. (2003). A corpus-based study of idioms in academic speech. TESOL
Quarterly, 37 (3): 419–441.
Snijders, T. M., Vosse, T., Kempen, G., Van Berkum, J. J. A., Petersson, K. M., & Hagoort, P.
(2009). Retrieval and unification of syntactic structure in sentence comprehension: An
fMRI study using word-category ambiguity. Cerebral Cortex, 19, 1493–1503.
Sommer, I. E., & Kahn, R. S. (2009). Language lateralization and psychosis.
Sotillo, M., Carretie, L., Hinojosa, J. A., Tapia, M., Mercado, F., Lopez-Martin, S., & Albert,
J. (2005). Neural activity associated with metaphor comprehension: Spatial analysis.
Neuroscience Letters, 373,  5–9.
Sowden, S., & Catmur, C. (2015). The role of the right temporoparietal junction in the control of
imitation. Cerebral Cortex, 25, 1107–1113.
Stefanowitsch, A., & Gries, S. T. (2007). Corpus-based approaches to metaphor and metonymy.
Berlin: Walter de Gruyter.
Strack, D. C. (2016). Solving metaphor theory’s binding problem: An examination of “map-
ping” and its theoretical implications. Metaphor and Symbol, 31,  1–10.
Straube, B., Green, A., Sass, K. & Kircher, T. (2014). Superior temporal sulcus disconnectivity
during processing of metaphoric gestures in schizophrenia. Schizophrenia Bulletin, 40,
936–944.
Stringaris, A. K., Medford, N., Giora, R., Giampletro, V. C., Brammer, M. J., & David, A. S.
(2006). How metaphors influence semantic relatedness judgments: The role of the right
frontal cortex. NeuroImage, 33, 784–793.
Stringaris, A. K., Medford, N. C., Giampietro, V., Brammer, M. J., & David, A. S. (2007).
Deriving meaning: Distinct neural mechanisms for metaphoric, literal, and non-meaningful
sentences. Brain and Language, 100, 150–162.
Subramaniam, K., Beeman, M., Faust, M., & Mashal, N. (2013). Positively valenced stimuli fa-
cilitate creative novel metaphoric processes by enhancing medial prefrontal cortical activa-
tion. Frontiers in Psychology, 4.
Subramaniam, K., Faust, M., Beeman, M., & Mashal, N. (2012). The repetition para-
digm: Enhancement of novel metaphors and suppression of conventional metaphors in the
left inferior parietal lobe. Neuropsychologia, 50, 2705–2719.
Talairach, J., & Tournoux, P. (1988). Co-planar stereotaxic atlas of the human brain: 3-
dimensional proportional system: An approach to cerebral imaging. Stuttgart: Thieme.
Toga, A. W., & Thompson, P. M. (2003). Mapping brain asymmetry. Nature Reviews
Tuerker, E. (2016). The role of L1 conceptual and linguistic knowledge and frequency in the ac-
quisition of L2 metaphorical expressions. Second Language Research, 32,  25–48.
Turken, A. U., & Dronkers, N. F. (2011). The neural architecture of the language comprehension
network: Converging evidence from lesion and connectivity analyses. Frontiers in Systems
Tzourio-Mazoyer, N., Landeau, B., Papathanassiou, D., Crivello, F., Etard, O., Delcroix, N.,
& Mazoyer, B. (2002). Automated anatomical labelling of activations in SPM using a mac-
roscopic anatomical parcellation of the MNI MRI single-subject brain. NeuroImage, 15,
273–289.
Uchiyama, H. T., Saito, D. N., Tanabe, H. C., Harada, T., Seki, A., Ohno, K., Koeda, T., & Sadato,
N. (2012). Distinction between the literal and intended meanings of sentences: A functional
magnetic resonance imaging study of metaphor and sarcasm. Cortex, 48, 563–583.
Van Lancker-Sidtis, D., & Rallon, G. (2004). Tracking the incidence of formulaic expressions in
everyday speech: Methods for classification and verification. Language & Communication,
24, 207–240.
Vartanian, O. (2012). Dissociable neural systems for analogy and metaphor: Implications for
the neuroscience of creativity. British Journal of Psychology, 103, 302–316.
Wakusawa, K., Sugiura, M., Sassa, Y., Jeong, H., Horie, K., Sato, S., Yokoyama, H., Tsuchiya,
S., Inuma, K., & Kawashima, R. (2007). Comprehension of implicit meanings in social
situations involving irony: A functional MRI study. NeuroImage, 37, 1417–1426.
Watson, C. E., Cardillo, E. R., Ianni, G. R., & Chatterjee, A. (2013). Action concepts in the
brain: An activation likelihood estimation meta-analysis. Journal of Cognitive Neuroscience,
25, 1191–1205.
Web of Science. (2016). Web of science. New York; Toronto: Thomson Reuters.
Weiller, C., Bormann, T., Saur, D., Musso, M. & Rijntjes, M. (2011). How the ventral pathway
got lost: And what its recovery might mean. Brain and Language, 118,  29–39.
Winner, E., & Gardner, H. (1977). The comprehension of metaphor in brain-damaged patients.
Brain, 100, 717–729.
Yang, F. G., Edens, J., Simpson, C., & Krawczyk, D. C. (2009). Differences in task demands
influence the hemispheric lateralization and neural correlates of metaphor. Brain and
Language, 111, 114–124.
Yang, F. G., Fuller, J., Khodaparast, N., & Krawczyk, D. C. (2010). Figurative language pro-
cessing after traumatic brain injury in adults: A preliminary study. Neuropsychologia, 48,
1923–1929.
Yang, J. (2014). The role of the right hemisphere in metaphor comprehension: A meta-analysis
of functional magnetic resonance imaging studies. Human Brain Mapping, 35, 107–122.
Yang, J., Li, P., Fang, X., Shu, H., Liu, Y., & Chen, L. (2016). Hemispheric involvement in the
processing of Chinese idioms: An fMRI study. Neuropsychologia, 87,  12–24.
Yang, J., & Shu, H. (2016). Involvement of the motor system in comprehension of non-literal
action language: A meta-analysis study. Brain Topography, 29, 94–107.
Zaidel, E., Kasher, A., Soroker, N., & Batori, G. (2002). Effects of right and left hemisphere
damage on performance of the “right hemisphere communication battery.” Brain and
Language, 80, 510–535.
Zempleni, M.-Z., Haverkort, M., Renken, R., & Stowe, L. A. (2007). Evidence for bilateral in-
volvement in idiom comprehension: An fMRI study. NeuroImage, 34, 1280–1291.
Zempleni, M. Z., Bruggeman, R., Knegtering, H., Van Velzen, M., Van Den Bosch, R. J., &
Stowe, L. A. (2006). Comprehension of idioms in schizophrenia: Preliminary fMRI results.
Schizophrenia Research, 81, 154–154.
Zhong, C.-B., & Liljenquist, K. (2006). Washing away your sins: Threatened morality and phys-
ical cleansing. Science, 313, 1451–1452.
Chapter 29
L anguage C om pre h e nsi on

and Emot i on
Where Are the Interfaces, and Who Cares?
Jos J. A. van Berkum
Introduction
When you hear somebody speak, or read a bit of text, you are somehow assigning
meaning to an unfolding sequence of signs. Because of the representational and compu-
tational complexity involved, this process of language interpretation is considered to be
one of the major feats of human cognition. However, you also happen to be just another
mammal, and as such, you are biologically predisposed to have emotions, evaluations,
and moods (i.e., to feel certain things about your environment). How do these two acts
of assigning meaning relate to one another? And what are the implications for neurolin-
guistics, the endeavor to understand how the brain realizes language use? These are the
central questions addressed in this chapter.
Over the last few decades, interest in the role of emotion in cognition has sharply
increased, and a substantial part of current cognitive neuroscience research is about how
affective factors mesh with cognition. With some delay, this affective turn in research on
mind and brain has also reached the language sciences (e.g., Corver, 2014; Jensen, 2014;
Majid, 2012; Peräkylä & Sorjonen, 2012; Van Berkum, 2010). In neurolinguistics, for ex-
ample, an older strand of research on the processing of emotional prosody (e.g., Pell,
1999) is now joined by research on such topics as the impact of emotional state on lan-
guage comprehension (e.g., Egidi & Carramazza, 2014; Van Berkum, De Goede, Van
Alphen, Mulder, & Kerstholt, 2013), the processing of “emotion words and sentences” (e.g.,
Hoffmann, Mothes‐Lasch, Miltner, & Straube, 2015; Ponz, Montant, Liegeois-Chauvel,
Language Comprehension and Emotion 737
Silva, Braun, Jacobs, & Ziegler, 2014), and the brain’s response to swear words and other
morally loaded language (Leuthold, Kunkel, Mackenzie, & Filik, 2015; Van Berkum,
Holleman, Nieuwland, Otten, & Murre, 2009).
But what is the status of such research in the language sciences? When discussing such
work with students in linguistics programs, the response is often mixed, in a way that
may well be indicative of a wider attitude in the field. Many find the topics quite inter-
esting. Emotion is “catchy,” and discussing its interface with language sometimes offers
a welcome change from such topics as predicate logic, minimalist syntax, or combina-
torial symbol processing in the brain. Also, many phenomena are saliently connected to
the students’ personal lives, from the reduced effectiveness of using a non-native swear
word to the painful sting of sarcastic prosody or a hesitant reply. At the same time, these
students often feel that research on language and emotion is not really “at the heart of the
matter.” The reasoning seems to be something like this:
1. Language is a code via which we communicate about everything, from muffin

recipes to our deepest fears, for a principally infinite number of reasons, and to a
principally unlimited number of effects.
2. Psycholinguistics and the associated cognitive neuroscience research endeavor
should study the generic mechanisms via which people acquire and use that code.
3. Other disciplines, like emotion science or social psychology, should study what
happens when people communicate about the specific things they do, and why
they choose to do so.
4. Although psycholinguistics is connected to those other disciplines in virtue of
people using language for everything, there is nothing about the interface that is
really of relevance to the task of understanding the generic mechanisms via which
people acquire and use language.
The reasoning is intuitively compelling, for muffin recipes, but also for our fears and
other emotions. Indeed, if human emotion is just a topic, a cause, or a consequence of
particular instances of language use, cleanly separated from the machinery that does
the language processing, psycholinguistics can just focus on the processing regardless
of emotion. So, is it this simple? In this chapter, I argue that it is not. The processing of
language and emotion is intricately intertwined, in ways that psycholinguistics and the
associated cognitive neuroscience enterprise cannot afford to ignore.
The analysis begins by examining why emotion is not naturally foregrounded in lan-
guage processing research. Because many readers will not be familiar with current views
on emotion, I subsequently review some basic insights, covering short-lived salient
emotions as well as other affective phenomena. After that, I make explicit the various
types of representations that people compute as they use language, ask where emo-
tion might kick in, and apply the resulting Affective Language Comprehension model to
738 Jos J. A. van Berkum
several neurolinguistics studies—this is the heart of the chapter. Finally, I explore how
the model can contribute to neurolinguistics and other fields.1
A terminological note: just as in emotion science, I will use “emotion” in two dif-
ferent ways in this chapter. The narrow meaning is that of the event-driven short-lived
phenomena that immediately come to mind when thinking about emotion: fear, joy,
anger, pride, disgust, and so on. More broadly construed, the term “emotion” (or “af-
fect”) covers emotions in this narrow sense, but also other affective phenomena, such as
affective evaluations and moods. Definitions of these various phenomena will be given
later in the chapter.
The Standard Approach

to Language Processing
Attention to emotion in psycholinguistics and the associated cognitive neurosci-

ence research is relatively recent, and current major textbooks and handbooks still
reveal a thoroughly “cold,” non-affective perspective on language processing that has
characterized the field for decades. The roots of this cold perspective can be found in
several important historical developments in the field, each of which led to a particular
bias (see van Berkum, 2018, for more extensive discussion).
Technological Systems Focus
Just like other disciplines within, or overlapping with, cognitive psychology, psycho-
linguistics has been heavily shaped by the technology-driven digital information pro-
cessing perspective in that larger field. In psycholinguistics, this technology frame has
inspired people to ask about such things as how comprehenders decode noisy acoustic
signals, store and retrieve lexical representations, recover syntactic structure, derive a
proposition, compute reference, update the situation model, and code their own ideas
for subsequent transmission—all questions about retrieving, manipulating, and storing
information. As might be expected, though, the technology frame did not readily lead to
questions about emotions, evaluations, and moods, or the needs of real living organisms
that give rise to these affective phenomena.
1
This chapter has overlap with Van Berkum (2018), particularly in the second and third sections.
However, while in the latter paper, I explore the interfaces between language and emotion with swear
words and present a model-driven discussion of the multifaceted nature of word valence, the current
chapter has a somewhat stronger cognitive neuroscience orientation, discusses a wider range of
examples, and applies the proposed model to specific neurolinguistics studies.
Code-Cracking  Focus
Psycholinguists have always enjoyed the luxury of being able to work from whatever
linguists had discovered about the nature of language. But with that luxury also came sub-
ject matter biases operating in linguistics itself. Mainstream linguistics in the 1970s–1990s
focused on language as a generative coding system, and abstracted away from actual usage.
As such, it has inspired a lot of psycholinguistic research on how people crack the linguistic
code (cf. all the research on lexical retrieval, syntactic parsing, anaphoric reference, and
ambiguity resolution) and how they acquire or lose their code-cracking competence, but it
has not inspired psycholinguists to study how the code actually gets to affect people.
Modularity Focus
Third, even for psycholinguists who did acknowledge the importance of emotion
to mental life, nothing of importance seemed to follow for their everyday scientific
concerns. After all, was the language system, or at least the most interesting bit of it,
not “informationally encapsulated” from the rest of mental life anyway? The idea that
language was an independent “module” in the mind (Fodor, 1983) paved the way for
thinking about language comprehension as computing what is said and implied before,
and cleanly separate from, computing the affective significance for the reader or listener.
Uniqueness Focus
As scientists carve up the world between them, it is only natural that people in different
disciplines tend to focus on what is unique to “their” chunk of the world. Language is a
discrete combinatorial system for very precise reference, unique in the animal kingdom.
However, psycholinguistics cannot focus only on the unique. To understand how the
system actually works in practice, you also need to look at the parts that may not be so
unique for Homo sapiens, but are critical just the same—such as memory, or emotion.
For example, although the observation that learning principles studied by behaviorists
could not easily account for the complexity of linguistic behavior was critical in the de-
velopment of the language sciences, this observation does not imply that as language
users, people are free from the standard effects of classic emotional conditioning.
The previously mentioned biases are to a large degree responsible for the dominant,
standard perspective on language use in psycho-and neurolinguistics, a perspective one
might call the TCP/IP approach to language use. In the TCP/IP approach, language users
are reduced to computational devices that exchange information via a fixed commu-
nication protocol (a human TCP/IP2), coding ideas into utterances and transmitting
2
TCP/IP is a communication protocol that regulates information exchange between computers
them for subsequent decoding at the other end, with the conversion to or from the
code carried out by special language “modems.” The research agenda of this approach
can be extracted from any recent psycholinguistics textbook or handbook, as well as
from programs of major psycho-or neurolinguistics conferences. Most of that agenda
is about storing, retrieving, manipulating, and transmitting data, about how listeners
figure out the bits of information that speakers want to pass on to them, and about how
speakers figure out what listeners already know, so that fewer bits need to be coded and
transferred.
Now, human language is a code for communication, and language users do need to
master that code to be able to profit from the additional precision and expressivity that
language provides. Research on the nature of the code, and on how language users ac-
quire, crack, and generate bits of this code, is therefore crucial to understanding the
human mind and brain. Having said that, it is clear that the TCP/IP approach cannot be
the whole story. Most obviously, language users are not dispassionate, immobile infor-
mation systems representing and exchanging information; they are animals with things
at stake, and with situations to cherish or best avoid. They care about things. Moreover,
they care enough about things to want to use language to inform, manipulate, or deeply
connect with other people (cf. Tomasello, 2008). They do things with words (Austin,
1962), to each other, and sometimes also to themselves. Emotion is at the heart of all that.
Hence, if we really want to understand the neural mechanisms that allow language to be
useful, we need to ask about emotion.
What Is Emotion? A Primer

for Language Researchers
Emotion is what has kept you alive so far—although details may vary, emotion may
have saved you from drowning, being run over by a car, losing sight of your primary
caretakers in a large crowd, or losing the means to sustain yourself. The affective systems
responsible for emotions, evaluations, and moods are at the core of how brains control
adaptive behavior in a complex environment (Damasio, 1994; Davidson, 2012; Frijda,
2008; Ledoux, 1996; Panksepp & Biven, 2012; Scherer, 2005)—not just in humans, but
in all mammals. Emotion science is a huge area of research, with branches reaching into
such disciplines as evolutionary biology, neuroscience, psychology, ethnography, and
philosophy (for various broad displays of this vast area, see Barrett, Lewis, & Haviland-
Jones, 2016; Davidson, 2012; Nussbaum, 2003; Prinz, 2004; Sander & Scherer, 2009;
Wetherell, 2012). There are countless fundamental debates, on such things as what
counts as emotion, on whether we have basic emotions, on the relative contribution of
biology and culture, and on how emotion relates to cognition (see Barrett et al., 2016,
for an extensive overview). Here, I focus on several key ideas and distinctions that have
generally proved useful to the field and are important when addressing the relation be-
tween emotion and language.
The starting point is a working definition of emotion that is suitable for current
purposes:
An emotion is a package of relatively reflex-like synchronized motivational, physi-

ological, cognitive, and behavioral changes, triggered by the appraisal of an external
or internal stimulus event as relevant to the interests (concerns, needs, values) of the
organism, and aimed at generating a prioritized functional response to that stimulus
event. The changes involved need not emerge in consciousness, but to the extent that
they do, they give rise to feeling.
This definition (which largely follows Scherer, 2005, but also incorporates aspects of
other proposals, notably Adolphs, 2017; Damasio, 2010; Frijda, 2008; Lazarus, 1991;
Panksepp & Biven, 2012), highlights several core properties of emotion that I will un-
pack in the following.
(1) Emotions are triggered by the appraisal of something as relevant to our concerns.
Emotions emerge when something about a stimulus is appraised as relevant to one’s
interests, either positively (such as when you win a contest, or see your child do well in
a school performance), or negatively (such as when you are insulted, find a huge spider
in the crib of your two-month-old baby, or drop your smartphone on the floor). An
emotion is referential (i.e., about something). What it is about might be “out there,” as
in all the preceding examples, or inside your head, as when you remember or imagine
any of the preceding, or mentally represent these scenarios in response to language; that
is, although examples in the emotion literature are often about concrete events, objects,
or situations in our environment, thoughts (consciously as well as unconsciously
entertained) can just as easily trigger emotion. Following Damasio (2010), I will use the
term emotionally competent stimulus, or ECS, to cover all of this. Appraisal can to some
extent be deliberate (i.e., under slow conscious control), but in line with what emotion
is supposed to do for us, it is usually fast, automatic, and unconscious (Adolphs, 2017;
Frijda, 2008; Prinz, 2004; Scherer, 2005; Zajonc, 1980)—as every psychotherapist or
coach will know, people often don’t know what aspect of a situation, person, or event
exactly triggered their emotion, or for what reason. Also, as illustrated by research on
olfactory and visual perception (e.g., Li, Moallem, Paller, & Gottfried, 2007; Tamietto,
Castelli, Vighetti, Perozzo, Geminiani, Weiskrantz, & de Gelder, 2009), people can re-
spond affectively without having consciously perceived the stimulus at all.
(2) Emotions involve a “package” of relatively automatic, short-lived, synchronized changes

in multiple systems. Emotion is not just about appraising something as relevant to
your interests, but also about doing something about it. For example, when something
makes you angry, your heart beats faster, you sweat a little more, and stress hormones
are released, as your body is preparing itself for “combat.” You will momentarily feel a
strong urge to act, and perhaps you will strike or yell at something, or someone. Your
face will have an angry expression. Attentional focus will briefly narrow, such that you
are no longer able to attend to other things in the environment. And finally, you may
become very aware of all of this, giving you the typical “feel” of anger. These specific
changes make up the average “package” for anger. Qualitatively different emotions, such
as anger and fear, have different action packages, with some shared ingredients (e.g.,
both increase sweating), but also some major differences (e.g., in contrast to anger, fear
increases the probability of retreat and avoidance). Specific instances of anger may also
differ somewhat in their exact “mix” of ingredients, and some mixes will be more pro-
totypical than others. The key observation, however, is that emotions involve relatively
automatic, short-lived, and synchronized changes along several different dimensions: (a)
motivational changes or action tendencies, the readiness to engage in, or disengage from,
particular behavior; (b) physiological changes that prepare the body for action or impact;
(c) cognitive changes, such as increased attention and better memorization; and (d) be-
havioral changes, involving approach or avoidance, as well as more specific actions such
as smiling, frowning, shouting, crying, changing posture, stroking, exploring, or playing.
(3) Emotions briefly take control. Emotion emerges when something is deemed sufficiently
important to relatively automatically engage multiple systems simultaneously, to have “all
hands on deck.” It is also about doing something now. Frijda (2008, p. 72) characterizes
emotion as “event-or object-instigated states of action readiness with control precedence”;
that is, you really have an urge to do something right now: strike out or yell at the intruder,
or write that email now. And that makes sense; after all, emotions are designed to watch
over your interests, directly or indirectly rooted in core biological values shaped by evolu-
tion. Although culturally conditioned and other personal life experiences construct addi-
tional layers of emotional complexity that are unique to humans (Barrett, 2014), emotion
is first and foremost about “biological homeostasis,” about regulating life within survival-
promoting and agreeable ranges (Damasio, 2010; Panksepp & Biven, 2012). Emotions
are bits of rapid biological intelligence that have proved useful in the past—reflex-like
solutions to recurring problems in the life of the species (and its ancestors), briefly taking
control, but also open to various forms of regulation (Adolphs, 2017).
(4) Emotions are not necessarily conscious. A crucial insight in emotion science is that
emotion doesn’t need to be conscious (Damasio, 2010; Frijda, 2008; Panksepp & Biven,
2012; Scherer, 2005); that is, one can have all of the ingredients (a) to (d) mentioned
earlier without actually being aware of them (i.e., of feeling them). This may be coun-
terintuitive, because in daily life we use “emotion” and “feeling” interchangeably. When
strong emotions are elicited, we will certainly “feel” them. But what holds for other
aspects of brain function also holds for emotion: most of the computations are done
without us being aware of the process and its results (Adolphs, 2017); that is, weak
emotions may unfold and affect our thoughts and behavior without any subjective
awareness. If this is hard to imagine, think about moments in life when you suddenly
became aware that you have been avoiding someone, or something, or that in particular
situations, your neck muscles tend to tighten up. Or about the effort that is sometimes
needed to make the relevant appraisals involved in your emotional life explicit, so that
you can reflect upon them.
(5) Emotions have ancient triggers but can hook up to new ones via learning. For
psycholinguists, a particularly critical observation is that there seem to be no limits on
the types of stimuli that can become emotionally competent. For a limited class of bio-
logically significant stimuli (e.g., pain, an unexpected loud noise, signs of decay, being
bodily restricted, the anticipation of sex or food, being stroked or otherwise cared for,
the loss of social bonds, a helpless baby, and the basic emotional displays of conspecifics,
such as smiles and frowns, aggression, or playful movement; Panksepp & Biven, 2012),
that competence is simply hardwired into your brain. Via “emotional conditioning,”
however, an infinite number of other stimuli can also become emotionally compe-
tent (De Houwer, Thomas, & Baeyens, 2001; Hofman, De Houwer, Perugini, Baeyens,
& Crombez, 2010; Ledoux, 1996; Panksepp & Biven, 2012), as generic categories, or
as specific tokens. The amygdalae are believed to be crucial to such emotional condi-
tioning, and they are capable of forging emotional associations without any awareness
or episodic recollection of the coupling (Janak & Tye; 2015; Ledoux, 1996; Phelps, 2006).
However, depending on the specific emotion involved, many other emotion-relevant
neural systems can also be involved, as generators of the affective brain state (i.e., the
unconditioned response, or UCR) that is now associatively connected with something
new (the conditioned stimulus, or CS), but also by realizing brain states that enhance the
formation of new memory (e.g., via arousal; Panksepp & Biven, 2012).
Crucially , as an unavoidable consequence of the generic mechanisms of associative
learning in the brain, the non-natural signs studied by semiotics and linguistics (e.g.,
a brand logo, a word, a particular linguistic construction) can also become emotion-
ally competent (e.g., Fritsch & Kuchinke, 2013; Hofmann et al., 2010; Jaanus, Defares,
& Zwaan, 1990; Kuchinke, Fritsch, & Müller, 2015; Keuper, Zwanzger, Nordt, Eden,
Laeger, Zwitserlood, Kissler, Junghöfer, & Dobel, 2014; Ortigue, Michel, Murray, Mohr,
Carbonnel, & Landis, 2004; Pülvermüller, 2012; Schacht, Adler, Chen, Guo, & Sommer,
2012; Silva, Montant, Ponz, & Ziegler, 2012). Such conditioning occurs automatically
whenever a particular sign is sufficiently reliably (or sufficiently strongly) paired with
affective responses, either in actual experience, or when such experience is sufficiently
imagined (as when we read a novel). Of course, the emotional conditioning process
must always bootstrap from something. But as the advertisement industry shows, this
is not hard at all: companies effectively associate their car, coffee, and ice cream brand
names or logos with positive emotions, simply via systematically pairing the initially
neutral stimulus with something that already is a highly competent ECS (e.g., an attrac-
tive man or woman, a scene with friendly people having fun). Although emotional con-
ditioning can lead to the transfer of strong and very salient emotions (as with the fear
conditioning that underlies PTSD or phobia), it usually affects us in much subtler ways,
via sometimes fully unconscious affective evaluations and the associated preferences
(see Hofmann et al., 2010, for a meta-analysis with verbal and nonverbal stimuli). In all,
emotions are sticky little things, value-relevant response packages that can attach them-
selves to anything without you noticing, and with the appraisal that is needed to elicit
them consisting of little more than the automatic retrieval of an acquired association
from long-term memory.
(6) Affective evaluation is low-intensity emotion. In a wide variety of fields, ranging from
social psychology (e.g., Zajonc, 1980) to the neuroscience of visual perception (e.g.,
Barrett & Bar, 2009), research has shown that we hardly ever see things in a neutral
way: affective evaluation is part and parcel of how we perceive the world. In the words of
Zajonc (1980, p. 154):
One cannot be introduced to a person without experiencing some immediate feeling

of attraction or repulsion and without gauging such feelings on the part of the other.
[ . . . ] Nor is the presence of affect confined to social perception. [ . . . ] We do not just
see “a house”: we see “a handsome house,” “an ugly house,” or “a pretentious house.”
We do not just read an article on attitude change, on cognitive dissonance, or on
herbicides. We read an “exciting” article on attitude change, an “important” article
on cognitive dissonance, or a “trivial” article on herbicides. And the same goes for a
sunset, a lightning flash, a flower, a dimple, a hangnail, a cockroach, the taste of qui-
nine, Saumur, the color of earth in Umbria, the sound of traffic on 42nd Street, and
equally for the sound of a 1,000-Hz tone and the sight of the letter Q.
Such automatic affective evaluations of the world around us build on the same affective
systems that generate salient emotions like anger, fear, disgust, pride, or joy. With evalu-
ation, however, the intensity of the emotion is so low that the response feels like a quality
of the stimulus (“an ugly house”), rather than like a particular state that we are in (“that
house made me feel disgusted”; see Barrett & Bar, 2009, for this distinction). Importantly,
just like more salient emotions, evaluations have an action component (emphasized by
the term “preference”): a more positive evaluation is associated with approach motivation,
with—consciously or unconsciously—preferring the evaluated item over something else.
Furthermore, these affective evaluations are by no means necessarily “post-perceptual,” or
“post-conceptual” (i.e., are not necessary generated only after something has been fully
identified or conceptualized in cognitive terms). In vision, for example, affect can be part
of the initial response to low-resolution, “coarse” aspects of an image, either because of
some evolutionary hardwiring (e.g., jagged contours, or the outline of what might be a
snake), or because of the associative conditioning brought about by real or vicarious expe-
rience (e.g., the contours of a gun; see Barrett & Bar, 2009). Echoing the classic psycholog-
ical notion of subjective perception, there is growing evidence in cognitive neuroscience
that what something is can often not be meaningfully separated from what it means to
me—perceptions are not objective, and affect can be an intrinsic part of perception
(Barrett & Bar, 2009; Gantman & Van Bavel, 2015; Lebrecht, Bar, Barrett, & Tarr, 2012).
(7) Mood. Mood differs from short-lived emotion in that it involves a relatively slow-
changing affective background state that is not really about something (i.e., is not
“referential”; Forgas, 1995; Scherer, 2005). Also, whereas short-lived emotions play their
role via unique prioritized action packages, mood is believed to play a functional role in
signaling the amount of resources available for exploration of the environment (Zadra
& Clore, 2011), and/or for signaling that the current course of action is working out well
(Clore & Huntsinger, 2007). The effects of this show up in differential patterns of action
and cognition. For example, in a bad mood we are not only less inclined to climb a steep
hill, but also inclined to overestimate the steepness of that hill (Zadra & Clore, 2011).
Furthermore, a bad mood narrows the spotlight of visual attention (Rowe, Hirsch, &
Anderson, 2007), and reduces such things as the width of associative memory retrieval
(Rowe et al., 2007), the use of scripts in episodic memory retrieval (Bless, Schwarz,
Clore, Golisano, Rabe, & Wölk, 1996), or the sensitivity to social stereotypes in person
judgment (Park & Banaji, 2000). In all, mood tunes cognitive processing in a variety of
interesting ways, again without us being aware of it.
(8) Emotions, evaluations, and moods recruit special neural circuity. Emotion is important
enough to warrant biologically evolved special neural and neuro-endocrine machinery,
partially or fully emotion-dedicated systems that we share with many other animals
(Adolphs, 2017; Panksepp & Biven, 2012; see also various chapters in Barrett et al., 2016,
for review). Many of those are subcortical structures (e.g., amygdala, hypothalamus, nu-
cleus accumbens, ventral tegmental area (VTA), periaqueductal grey (PAG)), but var-
ious regions of the neocortex (e.g., insula, anterior cingulate cortex (ACC), ventromedial
prefrontal cortex (vmPFC)) are also involved. Some of the emotion-relevant neural
structures are responsible for generating the physiological component of emotion (e.g.,
the hypothalamus, which controls much of the body’s internal milieu via direct neural
innervation, as well as a wide array of hormones released by the pituitary gland). Others
play a crucial role in supporting the subjective feeling of an emotion, such as the anterior
insula, which provides a map of visceral sensation (Craig, 2009), or the PAG, which has
been argued to underlie aspects of subjective core affect (Panksepp & Biven, 2012; see also
Satpute, Wager, Cohen-Adad, Bianciardi, Choi, Buhle, Wald, & Barrett, 2013). The degree
to which specific emotions have their own dedicated, non-overlapping bits of the brain
is heavily debated, and the most plausible model is one in which emotionally critical
structures like the amygdala play a—potentially different—role in different emotions as
a function of being recruited in a different wider network (Adolphs 2017; Hamann, 2012;
Kragel & LaBar, 2016; Pessoa, 2017). In any case, careful cross-species studies of systems
involved in fear, rage, care, or reward (reviewed in Panksepp & Biven, 2012) unequivo-
cally show that nature did not leave emotion entirely up to chance.
(9) The utility of emotion. Our emotional life covers a vast range of phenomena, in-
tense and subtle, consciously experienced or unconsciously nudging us, experienced as
strong emotion “in us,” or leading us to simply and sometimes imperceptibly “prefer”
particular things—people, objects, signs, ideas, actions—over others, or to refrain from
exploration at all. The point of all this, of course, is that our emotional life controls our
behavior. Emotions and evaluations are “motive states” (Frijda, 2008, 2013), urging or
nudging us to approach or avoid, prefer, attend to, explore, grab, attack, submit to, care
for, play with, or protect oneself from entities or events out there in the world, all be-
cause of how those entities or events relate to our interests (Damasio, 1994; Frijda, 2008;
Panksepp & Biven, 2012). And emotion does so right here, in your life. Emotional con-
trol is not just something that was vital when humans were hunter-gatherers, and ob-
solete in this age of food counters, gadgets, and the Internet. The motive states that are
part and parcel of emotions, evaluations, and moods control much of your everyday
behavior, from the supermarket you go to and the things you buy there, to the people
you seek out to chat and perhaps live with. They also determine whether you read on
or whether you cast this chapter aside, and whether you mentally explore certain ideas
or not. Emotions, evaluations, and moods need not be very strong to exert this control,
and we may not be aware of how they tug at us at all; our decisions to pursue some things
over others can be controlled by very subtle valence differences (cf. micro-valence;
Lebrecht et al., 2012). But they do guide us in our actions.
Those actions can be overt behavior, but also acts of thinking. For example, emotions
and evaluations play a crucial role in what we often experience as “rational” reasoning and
decision-making (e.g., Bechara, 2009; Damasio, 1994; Gigerenzer, 2007; Phelps, Lempert,
& Sokol-Hessner, 2014), when people are, for example, considering consumer products or
medical treatments (Kahneman, 2011), or thinking about a morally responsible course of
action (Greene, 2014; Haidt, 2012). Emotions and evaluations also influence attention (e.g.,
Harmon-Jones, Gable, & Price, 2012; Vuilleumier & Huang, 2009), memory encoding and
retrieval (e.g., Adolphs, Denburg, & Tranel, 2001), and reasoning and decision-making
(e.g., Damasio, 1994), and the specific beliefs that people are inclined to commit them-
selves to (Frijda, 2008) (for reviews, see Dolcos & Denkova, 2014; Dolcos, Iordan, &
Dolcos, 2011; Pessoa, 2008, 2010; Phelps, 2006; Phelps et al., 2014; Zadra & Clore, 2011).
Most of this affective control over our thinking occurs without our being aware of it.
Just like in other mammals, our affective system is thus key to the control of adap-
tive behavior in a complex environment (Panksepp & Biven, 2012). And just like other
mammals, such control is greatly enhanced by our capability for associative and other
forms of learning. What is special about us, Homo sapiens, is that our brain is capable
of constructing a much wider and more diverse range of representations of that envi-
ronment, as well as of ourselves, such that there is much more to have emotions about
and evaluations of, and such that we can influence our and other people’s behavior in
much more sophisticated ways. At the pinnacle of that sophistication is our talent for
language, and the inferential communication skills upon which that talent rests.
The Affective Language

Comprehension Model
So, how does the affective control system that we have just examined mesh with lan-
guage processing? In the context of this Handbook, it may seem obvious to address this
question by (a) delineating the sets of neural structures involved in emotion and lan-
guage processing, as well as the structural and functional connectivity between those
sets; and/or (b) simply reviewing all the empirical cognitive neuroscience research (with
electroencephalography [EEG], magnetoencephalography [MEG], functional mag-
netic resonance imaging [fMRI], etc.) on specific interactions between language and
emotion and inductively infer generic insights from that. However, these are not the
approaches taken here. As for the first, the set of neural structures involved in emotion is
very large, and there is much debate on the precise functional characterization of those
structures, as well as increasing awareness of the importance of dynamically configured
networks and the different roles that a particular node can play as a function of the net-
work it is in (Hamann, 2012; Pessoa, 2017). The same holds for language processing (see
the many chapters in this Handbook). This makes the hypothesis space for a bottom-
up connectivity-based approach rather large (but see Koelsch, Jacobs, Menninghaus,
Liebal, Klann-Delius, von Scheve, & Gebauer, 2015).
As for the second approach, reviews of concrete cognitive neuroscience experiments
that explore the interface between language and emotion are extremely useful (e.g.,
Citron, 2012). At the same time, I think they should be complemented by a theoretical
perspective. As reflected in rather loosely used expressions like “emotion sentences,”
much of the cognitive neuroscience research on language and emotion operates
with a relatively crude, non-articulated model of language processing—usually one
that focuses on context-free lexical or sentence meaning, at the expense of context-
dependent pragmatic levels of interpretation. If we are to make progress on how emotion
and language processing interact, however, we must begin by honoring the real com-
plexity of language processing. We know from pragmatics and psycholinguistics that
language comprehension is a highly complex business that extends beyond the single ut-
terance, involves several layers of interpretation, and is heavily context-dependent. We
also know that language is just one of many simultaneous “channels” or sign systems via
which we communicate, and that as we speak or write, such things as a flat voice, raising
an eyebrow, a well-chosen emoji, or slightly turning away can make all the difference.
What would be helpful is a wide-scope functional (“algorithm-level”; Marr, 1982) model
that pulls these various things together, and that systematically explores the functional
interfaces with emotion. A model like that can support researchers in orienting them-
selves, and in asking more refined questions about how language and emotion interface
in the brain (see also Willems, 2011, for the importance of a top-down approach in cog-
nitive neuroscience research).
In the remainder of this chapter, I describe and discuss such a blueprint for language
comprehension: the Affective Language Comprehension, or ALC, model. The model was
developed in a simple, two-step fashion, by first making explicit the various types of
representations that listeners or readers compute as they process language, and by subse-
quently asking where emotion might kick in. The original description of the model (van
Berkum, 2018) features an analysis of a verbal insult with a swear word, and provides a
related ALC-based analysis of the concept of word valence. Here, I expand the scope of
the model by showing that is also applies to several apparently much less “emotional”
examples (see the following section), and by subsequently illustrating the utility of the
model in interpreting the results of a few example cognitive neuroscience studies.
A Blueprint for Affective Language Comprehension

So what types of representations do language users compute when they comprehend
a spoken or written utterance? Drawing upon central ideas in psycholinguistics and
pragmatics (e.g., Clark, 1996; Enfield, 2013; Jackendoff, 2007; Kintsch, 1998; Levinson,
2006; Trueswell & Tanenhaus, 2005; Tomasello, 2008; Zwaan, 1999), as well as on what
we know about representation and processing from cognitive science and neurosci-
ence, Figure 29.1 represents a reasonable claim about the types of representation being
computed and the subprocesses involved in computing them.
To see the model at work, I will discuss three different example utterances throughout:
(1) a relative uttering, “Even John thinks euthanasia is acceptable in this case”; (2) a
spouse uttering, “We’ve run out of dog food”; and (3) a teacher uttering, “The number
7 is also a prime number.” The question I ask is: What impact can these communica-
tive moves have on addressee Y at that point in the exchange? In particular, what
representations might addressee Y compute, consciously as well as unconsciously,
and which of those representations can in principle be emotionally competent stimuli
(ECSs) for this addressee?
The Input: Multimodal, Composite Signs

In face-to-face conversation, conversational moves are always implemented as multi-
modal, composite signs, which include not just words arranged in a certain way, but a
wide variety of nonverbal signs as well (Clark, 1996; Enfield, 2013; Goodwin, Cekaite, &
Goodwin, 2012; Jensen, 2014). And in writing, people try to replace some of those signs
(e.g., emoji, exclamation marks). As for our examples, speaker X will inevitably utter
these sentences in a specific manner, such as with an annoyed, a pleading, or a relaxed
and patient voice, and with a certain expression and posture—nonverbal aspects that, as
will be seen in the following, are critical to interpretation.
Recognizing/Parsing the Signs Presented by the Speaker

The conventionalized ingredients of the composite sign will cue representations in long-
term memory (LTM), traces of stable practices of sign use tracked by an ever-learning
brain. For example, words like “euthanasia,” “dog,” or “number” will cue (retrieve, acti-
vate) whatever stable memory traces addressee Y has stored for those signs in the mental
lexicon, including their phonological and/or orthographic form properties, their syn-
tactic properties, and their conceptual properties, all of which will be brought to bear on
how the sentence will be parsed (Jackendoff, 2007). Specific constellations of words, such
as idiomatic expressions, or other stable constructions (Fillmore, Kay, & O’Connor,
1988; Lakoff, 1987), will likewise cue such representations in LTM (Jackendoff, 2007).
And particular gestures, facial expressions, or emoji will do so as well.
Y’s affective state
Interpret the communicative move
Bonus meaning
Infer bonus meaning
What else does this tell me about X or the world?
X’s social intention X’s com.

Infer social intention
What does X want me to do, know or feel? intention
Infer stance X’s stance

The situation x refers to
Infer referential intention
Recognize &
parse signs
Semantic - Gaze
parsing - Pointing gesture Signs
- Other gestures
- Affective prosody
Syntactic Phon/ortho
- Facial expression
parsing parsing
- Posture movement
Social intention - Other (e.g., emoji)
Recognize words and Recognize non-

Communicative Active representations
patterns over words verbal signs
intention
Stance LTM
Referential
intention
Y
Multimodal, composite signs
X (communicative + unintended) = (potential) ECS for Y
Figure 29.1. The affective language comprehension (ALC) model. Mental processes and the associated retrieved or computed representations are expanded for addressee
Y only. Y’s computational processes draw upon (and add to) long-term memory traces, and involve currently active dynamic representations that reflect what is currently
retrieved from LTM, composed from elements thereof, and/or inferred from context, in response to the current communicative move. Y’s active representations can be con-
scious or unconscious. Bonus meaning can be inferred from (or cued by) all other active dynamic representations, and Y’s current affective state (e.g., mood) can influence
all ongoing computational processes (arrows for these aspects not shown). The basic processing cascade is upward and incremental, starting from the signs, but small down-
ward or sideways arrows between component processes indicate top-down or sideways prediction or constraint satisfaction. Within each of the delineated representational
types, one or more ECSs can trigger an emotional processing cascade that affects Y’s motivational inclinations, physiology, cognitive processing, and actual behavior, plus
possibly Y’s conscious feeling.
Abbreviations: ECS = emotionally competent stimulus; LTM = long-term memory; Phon/ortho parsing = phonological/orthographic parsing;
X’s com. intention = X’s communicative intention.
Importantly, individual words and other “atomic” signs can themselves be ECSs,
(i.e., trigger a bit of emotion independent of the wider utterance and its pragmatic
implications). Models of how the brain represents word meaning have been shifting
away from amodal feature lists and directed graphs, toward a more modal view in which
lexical meaning is grounded in actual experience (e.g., Barsalou, 2008; Pülvermüller,
2012). Some psycho-and neurolinguists have begun to explore this for words that
refer to emotions or evaluations and the associated behavior (e.g., “smile,” “annoying”;
Foroni & Semin, 2009; Künecke, Sommer, Schacht, & Palazova, 2015; ‘t Hart, Struiksma,
Van Boxtel, & Van Berkum, 2018a, 2018b). But given what we know about associative
learning in the brain, and of emotional conditioning as a special case of that (see pre-
vious discussion in this chapter), the potential for grounding lexical meaning in emo-
tion is much wider than that (for evidence, see, e.g., Fritsch & Kuchinke, 2013; Hofmann
et al., 2010; Jaanus et al., 1990; Kuchinke et al., 2015; Keuper et al., 2014; Ortigue et al.,
2004; Pülvermüller, 2012; Schacht et al., 2012; Silva et al., 2012).
For example, if you have been raised with dogs, your personal concept “dog” will not
just include how they (can and tend to) look, sound, smell, and feel when touched, but
inevitably also how you relate to them affectively, with good or bad experiences leading
to traces of positive or negative emotion, respectively. Growing up in an environment
where euthanasia is considered pure evil will inevitably add traces of negative affect to
that concept. And if you have been raised in a family culture that placed a strict ban on
the use of swear words (e.g., you would be forced to wash your mouth with soap when-
ever you used one), this is bound to add some traces of affect to your representation of
the consequences of their use (see Jay, 2009). The same associative learning will inev-
itably shape the meaning of such things as emojis, intonation contours, or particular
constructions (e.g., “surely you know that . . .”): to the extent that their usage reliably
correlates with affective experiences, memory traces will simply be formed (see earlier
discussion in this chapter; and see van Berkum, 2018, for a more detailed ALC analysis
of word valence). Crucially, when the sign at hand is encountered again, these affective
memory traces will be retrieved early in processing (see Citron, 2012, for neurolinguis-
tics evidence).
Interpreting the Speaker’s Communicative Move

The goal of language comprehension, however, is not to retrieve the stable meaning
of words (and other signs) and combine those meanings into a “sentence meaning” in
a way that respects the rules of grammar. The goal is to work out the contextualized
“speaker meaning”: What does X mean, intend, by presenting this composite sign to Y
here and now? As indicated in Figure 29.1, these processes can take their cue from lan-
guage, but also, and in principle no less powerfully, from other types of signs, such as
a pointing gesture, a particular glance, or an emoji. And, as forcefully argued by prag-
matics researchers (Clark, 1996; Levinson, 2006; Scott-Phillips, 2015; Sperber & Wilson,
1995; Tomasello, 2008), the processes involved do not just tie up a few loose ends after
syntactic and semantic processes have done all of the serious work—they are a crucial
part of why our species has such powers of communication. In the subsequent sections,
I discuss the main types of inferential processes involved, primarily based on Tomasello’s
(2008) analysis.
Inferring the Speaker’s Referential Intention. One important ingredient of interpreting a

communicative move is to infer the speaker’s referential intention, i.e. to work out what
concrete situation the speaker is talking about exactly, and to build a situation model
that adequately reflects this (Johnson-Laird, 1983; Zwaan, 1999). With “Even John
thinks euthanasia is acceptable in this case,” for example, the addressee needs to work
out who is referred to by “John,” what is being asserted about this person, and, as part
of that, what “this case” refers to. Because situation models are always complex multi-
component structures, there may be multiple ECSs triggering an affective response.
In the case at hand, for example, the entire situation described (i.e., the fact that even
John thinks that such-and-such is OK) can be an ECS for the addressee, but the ref-
erent of “John” can also itself trigger emotions (e.g., when the addressee is not on good
terms with this person), and the composite “euthanasia is acceptable” (a statement that
might itself clash with moral values of the addressee) can do so, too. With “We’ve run
out of dog food,” the situation model computed by the addressee will depict, in some
way, a situation in which the household at hand has no dog food in stock, and, based
on plausible pragmatic inferences, in which the dog(s) living there might thus get very
hungry—owners who love their dogs will usually not be indifferent to that situation.
Even the situation delineated by “The number 7 is also a prime number” can be exciting,
or boring, depending on one’s inclinations. The possibilities are infinite: whatever we
can talk about, reality or fiction, verbally or nonverbally, might and will often be stuff we
care about, too.
Inferring the Speaker’s Stance. A second ingredient of interpreting a communica-

tive move is to infer or detect the speaker’s stance, his or her orientation to a partic-
ular state of affairs or “stance object” under discussion (Du Bois, 2007; Kiesling, 2011;
Kockelman, 2004). Stance has an epistemic and an affective side. Epistemic stance is
about aspects of the speaker’s knowledge state, such as when speaker X expresses, “The
number 7 is also a prime number” in a way, signaled by tone of voice, facial expression,
body posture, and so on, that conveys certainty and confidence, or uncertainty instead.
Depending on circumstances, this can sometimes be a trigger for emotions. However,
the speaker’s affective or evaluative stance (Hunston & Thompson, 2000), his or her
emotional orientation toward some stance object, will as a rule trigger emotion in the
addressee. The reason is that we are simply immediately sensitive to such emotional
displays of our conspecifics, via various evolutionarily sensible routes. These include
several aspects involving empathy (Decety & Cowell, 2014)—simple emotional sharing
(“resonance,” “mirroring,” “emotional contagion”), empathic concern (“caring for”),
and affective perspective-taking (i.e., more deliberately imagining somebody else’s
feelings)—as well as various other rapid interpersonal interlockings of social emotions
(Fischer & Manstead, 2016), such as when rage instills fear, admiration instills pride,
and contempt instills shame, at least initially. Returning to our examples, if the math
teacher utters, “The number 7 is also a prime number” with clear signs of annoyance
and contempt, the addressee might feel ashamed, while signs of sympathy, patience,
and encouragement will typically generate more positive emotions. The stance signals
that might accompany “Even John thinks euthanasia is acceptable in this case”—signals
that, for example, reveal deep sorrow, incredulous disbelief, rage, or contempt—will
also easily trigger strong or weak emotion in the addressee. The same holds for stance
signals accompanying “We’ve run out of dog food,” such as those that betray unpleasant
surprise, concern, or reproach.
While stance itself is usually detected relatively easily, what the stance is about often
requires some additional computation. Speaker X’s uncertainty or annoyance, for ex-
ample, might be about what is being referred to, but also about addressee Y, about
the communicative situation, or about the expected effect of the utterance. Also, the
stance signals emitted by speaker X need not all have been communicated deliberately.
Furthermore, in line with the fact that much of cognition and emotion is unconscious,
addressee Y may be affected by these signals without being aware of it at all. Either way,
the speaker’s stance will have an impact on the addressee, via its contribution to the
inferred social intention, but, unavoidably, also by itself.
In the example at hand, the verbal ingredients of the utterance single out a situation
that X wishes to draw Y’s attention to, and nonverbal ingredients mostly signal X’s stance.
But, as indicated by crossing arrows in the center of Figure 29.1, things can be otherwise.
Referents can be signaled verbally but also entirely nonverbally, by such means as eye
movements, manual pointing, or an iconic gesture (Tomasello, 2008). Also, epistemic or
affective stance can be expressed through such nonverbal signs as tone of voice, but also
by one’s choice of words and constructions, in a wide range of subtle and less subtle ways
(e.g., using “I guess that . . .” to express uncertainty, “just” to express non-commitment,
or swear words to express strong negative stance). The division of labor between how
verbal and nonverbal parts of the composite sign signal referents and stance can change
with every utterance. In fact—and important to keep in mind—the comprehension pro-
cess depicted in Figure 29.1 can also work without language (Levinson, 2006; Tomasello,
2008), as when we communicate something with a well-timed silence, a raised eyebrow,
an emoji, or a sigh.
Inferring the Speaker’s Social Intention. Addressee Y’s mental representations of speaker

X’s referential intention and (deliberately or accidentally conveyed) stance jointly
provide the basis for the third ingredient of interpreting a communicative move, the
inferring of X’s social intention. What is it that speaker X presumably wants to achieve
by making this specific move, here and now? The options are unlimited. However, ac-
cording to Tomasello (2008), speakers have three major types of social motivations
for communicating, often mixed in the same move, but conceptually distinct: (1)
requesting (or manipulating): I want you to do or know or feel something that will help
me; (2) informing: I want you to know something because I think it will help or interest
you; and (3) sharing: I want you to feel something so that we can share feelings together.
Obvious verbal examples are “Please close the door”; “Hey, you dropped your wallet”;
and “Isn’t that a great view!” In the right context, similar intentions can be expressed by
pointing to a specific open door, wallet, or view in a certain manner. Whatever the case
might be, addressee Y needs to figure out what speaker X wants him or her to do, know,
or feel.
The representations that we construct for an interlocutor’s social intention on the
basis of his or her referential intention and stance, as well as our own expectations, are
usually emotionally competent, and sometimes very strongly so—after all, it is at this
level that we deal with each other. In the prime number example, addressee Y might
infer that X just wants to help, wants to make the addressee feel small, or wants to share
amazement with him or her about this mathematical fact. In the dog food example,
Y might infer that X wants him or her to go to the store and wishes to phrase this as
a polite request, and/or that X wants him or her to feel remorse for not having done
so before. And with “Even John thinks euthanasia is acceptable in this case,” the so-
cial intention might be to persuade the addressee to agree to euthanasia, to mock the
addressee for an obviously backward opinion, or to simply share amazement over the
ease with which people apparently consider euthanasia. Note that the same utterance
can realize very different social intentions, and that addressees can (and, unfortunately,
fairly often do) infer different intentions from the one the speaker had in mind. In any
case, many of the strong or subtle emotions elicited by language use will arise at this
level of interpersonal interaction, the level where we manipulate, help, or share feelings
with each other.
Communication always involves an additional “special” social project: not only has
the speaker decided to use language and/or nonverbal signs to realize his or her primary
social intention(s), but he or she must somehow get the other person to (implicitly or ex-
plicitly) agree to and collaborate on the joint communicative project for a certain amount
of time. The implication is that whenever speaker X is drawing Y’s attention to his or
her wish to communicate (e.g., by presenting words and other obviously communica-
tive signs, possibly accompanied by special for-you signals such as eye gaze), addressee
Y already knows at least one social intention, namely that speaker X is trying to realize
whatever other social intention he or she might have via a communicative project.
Importantly, the addressee may feel good about this, or not. If you are engaged in mental
arithmetic and afraid to lose track, you may not want to be disturbed by communicated
math trivia right now. If you are busy pondering your own view on euthanasia, you may
not want somebody to tell you about other people’s opinions. And if you are fed up with
working on an exam or a paper, any remark from anybody might be a welcome distrac-
tion, even if it is about household supplies being low.
Inferring Bonus Meaning. Working out speaker X’s referential intention, stance, and so-
cial intention (and recognizing his or her communicative intention as a special case of
the latter) completes the process of inferring or understanding speaker meaning, that
which the speaker aims to convey or bring about. Some would argue that language pro-
cessing stops there (e.g., Clark, 1996). But regardless of such discipline-based demar-
cation lines, processing doesn’t of course stop there—addressee Y will consciously or
unconsciously always infer (via associative memory retrieval or more sophisticated

computation) at least some additional “bonus” meaning, things that X did not mean to
convey at all, about speaker X (e.g., “X is a really kind teacher”; “X is getting rather for-
getful”; “X is always bringing John up”), the relationship between X and Y (e.g., “X really
thinks I can do better”; “X is always nudging me”; “X never listens to me”) and the rest of
life (e.g., “I may really have a talent for math”; “Dogs are a lot of work”; “How can people
be so insensitive?”). Although not part of speaker meaning proper, such bonus meaning
will usually strongly contribute to whatever Y will think, feel, do, or say next.
The Addressee’s Current Emotional State Can Affect Processing

Finally, the addressee’s current emotional state can also affect processing, in part fully in-
dependently from the speaker’s communicative move and the active representations that
reflect its analysis. First, a preceding event may have led to a strong emotion with atten-
tional and other cognitive effects that impact further processing; such short-lived emo-
tional state changes occur rapidly enough such that the beginning of an utterance can
affect the processing of its continuation. Second, mood can impact cognitive processing
in ways that are independent of whatever information happens to flow through the pro-
cessing system. This also holds for language processing, where mood has been shown to
affect, among other things, syntactic parsing (e.g., Vissers, Virgillito, Fitzgerald, Speckens,
Tendolkar, Van Oostrom, & Chwilla, 2010), referential anticipation (e.g., van Berkum
et al., 2013) and the response to unexpected concepts in discourse (e.g., Federmeier,
Kirson, Moreno, & Kutas, 2001). Furthermore, the current emotional state can interact
with the valence of information flowing through the system (cf. mood incongruency
effects; e.g., Egidi & Caramazza, 2014; Pratt & Kelly, 2008). The ALC model allows one to
think about the impact of mood and shorter-lived emotional states in a more precise way,
by localizing that impact in one or several specific component processes.
Additional Complexity
The structured nature of representations generated by linguistic communication allows
for more complexity than discussed so far. First, because active representations of a
given type can be nested in representations of the same type, ECSs can also be embedded
in other ECSs. Such embedding was already exemplified at the situation model level
(“Even John thinks euthanasia is acceptable in this case”), but interesting variants also
occur at the level of social intentions. Consider “You are really ugly!” spoken by a friend
in a benign teasing way. The social intention ultimately construed by the addressee
should be one of playful teasing. However, the teasing part is achieved via a pretended
insult (i.e., another social move). This embedding reveals the recursive creativity of
human interaction: just like in art, people can always take an established communica-
tive pattern and start “playing” with it. However, this also opens up the possibility that
although the “outermost” social move is a positive ECS, the embedded social move can
still serve as a negative ECS.
A second level of complexity arises in narrative, the stories people tell each other,
such as when they gossip, write a novel, or report on events in the news. Such stories are
usually about other people, characters, engaging with each other in a series of more or
less fortunate events. Not only are these characters themselves affective creatures, caring
about those events in ways that make sense from their own value systems, but we as
readers or listeners affectively orient ourselves toward all that as well—this is precisely
the fun of reading a novel, or gossiping about others. From a modeling perspective,
things get very complex here. To the degree that we get transported into the story world
(e.g., Slater, Johnson, Cohen, Comello, & Ewoldsen, 2014) and identify with particular
characters, for example, we may momentarily take on somebody else’s value system (i.e.,
not just see the world through their eyes, but feel it through their emotions). The result
of this may well be something akin to bi-stable perception, with stimuli that can be, say,
a positive ECS for the character you momentarily identify with in the story world, but
a negative ECS for you in the real world (see also the following section). Furthermore,
in narrative, the really exciting events are often communicative moves, requiring you
to unpack the referential intention, stance, and social intention of the communicating
character, just as you would with a real interlocutor. And then on top of all that,
somebody—an author, a narrator—is telling you this story, with an affective stance of his
or her own.
This is not the place to unpack this additional complexity, nor to suggest that with
the current ALC model in hand, things will always remain tractable. At the same time,
it should be obvious that with a less articulate model—one that does not at least sep-
arate signs from the speaker’s referential intentions, stance, and social intentions plus
some bonus, or one that merely characterizes the comprehender as a TCP/IP-decoding
computer—we do not stand a chance at all.
Using the ALC Model to Interpret Neurolinguistics

Research Findings
By combining established ideas from the psycholinguistics of word and sentence pro-
cessing, the pragmatics of interpretation, and the nature of emotion, the ALC model
makes explicit that emotion can in principle pervade every step of the language-
comprehension process, and that mood and other aspects of one’s affective state can in
principle impact on all components of the comprehension process. Of course, this does
not mean that every potential interface between emotion and language is always highly
relevant to every bit of actual language use. What the model is supposed to do is list the
options, and help researchers think about what the operative interfaces might be in the
situations they wish to study, or have already studied. To illustrate this, I will briefly ex-
amine the results of a few neurolinguistics studies that I was involved in.
EEG Research on the Processing of Insults with Swear Words

In a recent EEG study, Struiksma, De Mulder, and Van Berkum (2018) examined the
short-term impact of verbal insults. Participants read verbal insults that contained
relatively coarse swear words (e.g., “<X> is a bitch”), insults without such swear words
(e.g., “<X> is a liar”), and compliments (e.g., “<X> is a darling”), where <X> would be
replaced by the participant’s own name or that of somebody else. To examine the robust-
ness of any differential insult effects, insults were repeated in homogeneous blocks (e.g.,
30 insults targeting you) that occurred three times over the course of the experiment.
Relative to compliments, insults with coarse swear words elicited an early P2 effect at
150–250 ms after presentation of the critical word, regardless of who was targeted by
the insult. On the assumption that being referred to in a strongly negative way is more
evocative for the person him-or herself than for somebody else, the insensitivity of this
effect to who was being insulted suggests that the ECS at the root of the P2 effect is not
the specific situation referred to, nor the (imaginary) speaker’s social intention. What
is more likely is that the swear word elicits this response at the level of the atomic sign
(see van Berkum, 2018, for a swear-word-oriented ALC analysis of word valence), and/
or at the level of the inferred stance of the speaker. The early timing of the ERP effect,
and the fact that it does not diminish with rather massive repetition, speaks in favor of a
sign-level  ECS.
In the same study, insults with coarse swear words also elicited an LPP (late positive
potential) effect around 350–500 ms, again regardless of who the target was. As with
the P2 effect, such independence would not be expected if the ECS emerges at the level
of the inferred referential or social intention. The ALC model suggests several other
options. One is that the LPP effect reflects some downstream consequence of the same
sign-level swear-word-conditioned ECS that also elicited the P2 effect, such as, for ex-
ample, increased conscious processing of salient signs. Another option is that the LPP
effect is independently triggered by the inferred stance of the speaker, or by some bonus
inference associated with that. The Struiksma et al. (2018) data do not allow us to decide
the issue. What should be clear, though, is that the ALC model can help in delineating
what the various sources of the response to verbal insults might be.
fMRI Research on the Processing of Face-Saving Indirect Replies

Bašnáková, Van Berkum, Weber, and Hagoort (2015) used fMRI to investigate the
neural correlate of comprehending face-saving indirect replies. In a scripted job inter-
view situation, participants queried several candidates over the intercom, and, at critical
moments, received either a direct reply (e.g., “I am planning to take a language course
this summer” to the question “What are your plans after graduation?”) or an indirect
face-saving reply (e.g., “I am planning to take a language course this summer” to the
question “Are you fluent in any foreign languages?”). In a different fMRI session, the
same participants also overheard somebody else do the interview with the candidates.
In both situations (i.e., as addressee or overhearer), the fMRI participants needed to
fully process the answers to come to a candidate-selection decision.
Relative to direct replies, indirect face-saving replies engaged core nodes of the
metalizing network (bilateral temporoparietal junction (TPJ), medial prefrontal
cortex, and the precuneus), as well as structures associated with other non-emotional
aspects of discourse complexity (bilateral BA45, BA47, anterior temporal lobe), and
did so equally when fMRI participants were the addressees of these replies and when
they were merely overhearers. This is compatible with the ALC model, in that cogni-
tive perspective-taking, as well as other aspects of discourse-level comprehension, is
a necessary part of inferring the speaker’s referential and social intention regardless of
whether the listener is being addressed or overhearing. However, whether participants
were the addressees of the face-saving replies or merely overhearing them did matter
to whether indirectness additionally engaged emotion-related areas: face-saving in-
directness increased activation in the left and right insula and the ACC only when
fMRI participants were addressed themselves, not when they overheard the replies
being given to somebody else. Note that in this study, face-saving replies are such that
they “cover up” potential shortcomings of the job candidate, and can thus be seen to
mislead or otherwise “socially navigate” the addressee. This may well explain why
those addressed are uniquely, and affectively, sensitive to such replies. The ALC model
provides two clear options as to where the addressee-specific ECS(s) might be located.
One is the inferred social intention, which might involve emotionally evocative things
like “he’s deliberately avoiding a straight answer to cover up his shortcomings” or “he’s
playing me,” and may as such elicit irritation or other relatively arousing emotions.
The other plausible location for one or more ECSs is the associated bonus meaning
(e.g., stereotypical ideas about the type of person who would do such a thing). As
presented in Bašnáková et al. (2015) in detail, the ALC model allows us to systemati-
cally think about which cognitive processes are taxed equally by indirectness, as well
as which of the resulting representations might specifically be emotionally evocative
for addressees.
Facial EMG Research on the Processing of Morally Loaded Stories

In two recent studies, ‘t Hart et al. (2018a, 2018b) explored the processing of utterances
such as “Mark was furious when . . .” or “Mark was happy when . . . ,” embedded in a nar-
rative fiction context where the protagonist had just exhibited morally sound or mor-
ally bad behavior. Electromyographic recordings of corrugator supercilli (“frowning
muscle”) activity suggested that the emotional response of readers in these experiments
involved a blend of two processes: simulating what was being asserted, and evaluating
what was being asserted. Evidence for the latter came from the observation that while
readers frowned more when reading “Mark was furious” as compared to “Mark was
happy” if the protagonist at hand had just been portrayed as a morally good person,
they frowned equally to “Mark was furious” and “Mark was happy” if the protagonist at
hand had just been portrayed as a morally bad person. This suggests that, as might be ex-
pected (Greene, 2014), readers have different emotions about something bad happening
to bad people (e.g., Schadenfreude) as compared to something bad happening to good
people (e.g., compassion). However, reading about furious versus happy protagonists
also made an independent additional contribution to the recorded degree of frowning,
indicating that our readers also had emotions as part of embodied language processing,
in line with earlier work on this topic (e.g., Foroni & Semin, 2009; Havas, Glenberg,
Gutowski, Lucarelli, & Davidson, 2010).
The ALC model allows us to more precisely delineate these various sources of
reader emotion. As for simulation, the increased frowning recorded when people read
sentences such as “Mark is furious” can reflect the retrieval of the meaning of the lex-
ical signs (in this case, of “furious”), and/or the construction of a situation model (i.e.,
imagining a furious specific protagonist). As for evaluation, the most likely source of
emotion here is how the entire situation referred to relates to the reader’s own norms
and values. In the fictional narratives at hand, the author’s stance or social intention is
not very likely to be an ECS. However, it is easy to imagine narratives where the author’s
or speaker’s stance and social intention do matter (e.g., blogs, gossip) and will thus have
the potential to trigger additional emotion. In all, the ALC model helps in making ex-
plicit where the various weak and strong emotions that we have when we are reading or
listening to stories may actually come from: all the usual options discussed earlier, plus
the embodied situation-model simulation of somebody else’s real or fictional emotions.
EEG Research on How Mood Affects Language Processing

Language processing research with so-called implicit causality verbs has shown that
when people read “David praised Linda because . . . ,” the verb and the surrounding con-
struction lead them to anticipate more information about Linda, not David; if a subse-
quent pronoun is inconsistent with that expectation, as in “David praised Linda because
he . . . ,” readers slow down and also display immediate processing costs that show up in
the EEG, right at the critical pronoun (see van Berkum, Koornneef, Otten, & Nieuwland,
2007, for data and review). Of relevance here, a follow-up EEG study (van Berkum et al.,
2013) indicated that the anticipatory bias varies with the participant’s current mood: while
readers in a good mood do show EEG traces of verb-based anticipation at an expectation-
disconfirming subsequent pronoun, readers in a bad mood no longer seem to anticipate
who’s going to be talked about next. In terms of the ALC model, at “David praised Linda
because . . . ,” a bad mood seems to down-regulate the rapid, real-time anticipation of
the author’s referential intention, plus possibly of the plausible signs associated with that
anticipated referential intention (in this case, the word “she”). More generally, Figure 29.1
can be said to make the hypothesis space for mood effects on language processing ex-
plicit, with mood potentially affecting all processes depicted on the left, and potentially
biasing processing toward particular representations on the right. In the experiment at
hand, mood had an impact on the degree to which readers anticipated aspects of the ref-
erential intention. At the same time, the absence of a mood-modulated ERP effect to syn-
tactic number agreement violations (van Berkum et al., 2013) indicated that, in this study,
the comprehender’s affective state did not affect aspects of syntactic processing.
Implications
So, is human emotion just a topic, a cause, or a consequent of particular instances of lan-
guage use, cleanly separated from the machinery that does the language processing, and
thus of little relevance to psycholinguistics? The central claim of the ALC model is: usu-
ally not. Every representation retrieved or computed as part of language comprehension
can in principle be an emotionally competent stimulus, with access to the brain’s affec-
tive systems via fresh appraisal or associative memory traces of past appraisal and emo-
tion; that is, for every communicative move, the individual signs used by the speaker can
be ECSs, the situation the speaker is believed to refer to may contain one or more ECSs,
the speaker’s stance is usually an ECS, the inferred speaker’s social intention is usually an
ECS (and there may be several such intentions packed in the same move), the commu-
nicative project may itself also be an ECS, and some part of the bonus meaning will often
contain one or several ECSs. In addition, the resulting or prior background emotional
state can tune and bias elements of subsequent language processing, in ways that reflect
how mood, emotions, and evaluations tune other forms of cognition and action. In all,
emotion does not just come into play after some “thermo-insulated” cold comprehen-
sion module has done its thing. The process of language comprehension is infused with
emotion right from the start, and all the way through.
Although the examples discussed have often foregrounded spoken conversation, the
ALC model is also about written language comprehension, such as when reading a text
message on your phone, a blog on the web, a textbook in class, a tax letter on your door-
step, or a novel in bed. Also, with its equal foregrounding of verbal and nonverbal signs,
the ALC model can easily be applied to multi-modal instances of communication, such
as when words and emojis are mixed together during texting. In fact, we can take all of
language out and use the ALC model to analyze the impact of completely nonverbal
communicative moves, such as an isolated emoji in WhatsApp, a raised eyebrow in face-
to-face conversation, or a communicatively intended touch. The ALC model is really
about the processing of communicative moves, whatever their form.
Who Is the Model for?
Apart from helping to make sense of past neurolinguistics research, the ALC model
makes several interesting predictions that can be tested with neurolinguistics
methods. First, signs that have been reliably coupled with particular affectively loaded
representations (e.g., of the speaker’s stance or intentions, or of the typical perlocu-
tionary effects the sign has on others) should inject their affective payload extremely
rapidly in the processing stream, a prediction that can be tested with EEG and/or MEG
(for relevant evidence, see, e.g., Citron, 2012; Schacht et al., 2012; Struiksma et al., 2018;
see also, in this volume, Leckey & Federmeier, Chapter 3, and Salmelin, Kujala, &
Liljeström, Chapter 6). Furthermore, the ALC model predicts that at least five different
levels of representation computed as part of language comprehension—signs, referential
intention, stance, social intention, and bonus meaning—should each have some way of
access to these emotion-relevant neural structures, a prediction that can be tested with
functional and structural connectivity analysis. And peripheral measures such as skin
conductance and facial electromyography (EMG) can help test the model’s prediction
that the different levels of representation disentangled by the model can all contribute
to a reader’s or listener’s affective response, and that the acquired affective meaning of a
linguistic sign can be related to each of the various potential sources of affect higher up
in the model, as language is being used in particular contexts again and again.
Several other research communities might also profit from the ALC model. For
linguists, psycholinguists, and communication researchers who are asking questions
about language and emotion, the model can serve as a tool for thinking about existing
findings and new research, and, inevitably, as a stepping stone toward a more adequate
model. Furthermore, for those in different fields that use linguistic materials (“vignettes”),
the ALC model can serve as a reminder of the complexity and multileveled nature of the
stimulus comprehension processes involved. The idea that words can affect people in sev-
eral ways that go beyond the obvious (what they refer to) is relevant not only to researchers
in basic psychological and cognitive neuroscience research on emotion, morality, and
social interaction, but also to researchers who explore institutional and interactional
processes in the political, judicial, educational, medical, financial, or business domains.
Finally, as an explicit model of language processing that also minds emotion, the ALC
model can perhaps do other work as well. The biases discussed earlier in the chapter
have led to an approach to language processing that has been fruitful: we now know a lot
more than before about how the brain cracks the language code. At the same time, the
biases have drawn attention away from what we do with language because we care about
“stuff.” I frequently come across professionals who have a general interest in language
because of the social, verbal nature of their profession (e.g., coaching, teaching, adver-
tisement, politics), but who feel that the language sciences currently have little to offer.
Models like the once proposed here can perhaps help bridge the gap.
What if I Just Do Not Care?

What about the many researchers in language and communication who are not inter-
ested in emotion in their work? Can they just ignore the current analysis, or similar cases
made by others (e.g., Besnier, 1990; Foolen, 2012; Jensen, 2014; Majid, 2012)? I would
argue that even language scientists whose work is well removed from the interfaces with
emotion should have some basic knowledge of what and where those interfaces are. One
reason is that emotion is a powerful source of variance in language processing—a source
one should be aware of and if possible control for, much like experimentalists routinely
control for word frequency. More fundamentally, every language and communica-
tion researcher should know about the interfaces with emotion for the same reason for
which those who work on, say, syntactic parsing should know a bit about phonology, se-
mantics, and pragmatics, and why those working on text comprehension should know a
bit about word recognition. We are looking at a structured yet integrated system, a bit of
nature that, although it has joints to carve it at, and subcomponents to focus on, is not a
collection of disconnected bits that can all be studied in isolation. If anywhere, that case
can be made quite easily for emotion. People use language to refer to things they care
about, and they use it to relate to each other, in ways that are almost never neutral. In the
words of Nico Besnier (1990, p. 433):
Affect permeates all utterances across all contexts because the voices of social beings,
and hence their affect, can never be extinguished from the discourse.
If you combine Besnier’s fundamental observation with basic cognitive neuroscience

knowledge about the role of emotion in cognition and action, and about emotional
learning in the brain, it is actually quite difficult to see how the study of language pro-
cessing can be complete if emotion is not included in the picture as well.
Acknowledgments
Supported by NWO Vici grant #277-89-001 to JvB. Thanks to Suzanne Dikker, Björn ‘t Hart,
Hans Hoeken, Anne van Leeuwen, Hannah De Mulder, Hugo Quené, Niels Schiller, Marijn
Struiksma, Greig De Zubicaray, and students in various courses for their help.
References
Adolphs, R. (2017). How should neuroscience study emotions? By distinguishing emotion
states, concepts, and experiences. Social, Cognitive, and Affective Neuroscience, 12(1),  24–31.
Adolphs, R., Denburg, N. L., & Tranel, D. (2001). The amygdala’s role in long-term declarative
memory for gist and detail. Behavioral Neuroscience, 115(5), 983.
Austin, J. L. (1962). How to do things with words. Cambridge, MA: Harvard University Press.
Barrett, L. F. (2014). The conceptual act theory: A précis. Emotion Review, 6, 292–297.
Barrett, L. F., & Bar, M. (2009). See it with feeling: Affective predictions during object per-
ception. Philosophical Transactions of the Royal Society B: Biological Sciences, 364(1521),
1325–1334.
Barrett, L. F., Lewis, M., & Haviland-Jones, J. M. (Eds.). (2016). Handbook of emotions.
New York: Guilford Press.
Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59, 617–645.
Bašnáková, J., Van Berkum, J. J. A., Weber, K., & Hagoort, P. (2015). A job interview in the
MRI scanner: How does indirectness affect addressees and overhearers? Neuropsychologia,
76,  79–91.
Bechara, A. (2009). The somatic marker hypothesis and its neural basis: Using past experiences
to forecast the future in decision making. In M. Bar (Ed.), Predictions in the brain (pp. 122–
133). Oxford: Oxford University Press.
Besnier, N. (1990). Language and affect. Annual Review of Anthropology, 19, 419–451.
Bless, H., Clore, G. L., Schwarz, N., Golisano, V., Rabe, C., & Wölk, M. (1996). Mood and the
use of scripts: Does a happy mood really lead to mindlessness? Journal of Personality and
Social Psychology, 71(4), 665.
Citron, F. M. (2012). Neural correlates of written emotion word processing: A review of recent
electrophysiological and hemodynamic neuroimaging studies. Brain and Language, 122(3),
211–226.
Clark, H. H. (1996). Using language. Cambridge: Cambridge University Press.

Clore, G. L., & Huntsinger, J. R. (2007). How emotions inform judgment and regulate thought.
Corver, N. (2014). Recursing in Dutch. Natural Language & Linguistic Theory, 32(2), 423–457.
Craig, A. D. (2009). How do you feel—now? The anterior insula and human awareness. Nature
Reviews Neuroscience, 10,  59–70.
Damasio, A. R. (1994). Descartes’ error: Emotion, rationality and the human brain. New York:
Putnam.
Damasio, A. (2010). Self comes to mind: Constructing the conscious mind.
New York: Pantheon.
Davidson, R. J. (2012). The emotional life of your brain. London: Penguin.
Decety, J., & Cowell, J. M. (2014). The complex relation between morality and empathy. Trends
De Houwer, J., Thomas, S., & Baeyens, F. (2001). Association learning of likes and dislikes: A
review of 25 years of research on human evaluative conditioning. Psychological Bulletin,
127(6), 853.
Dolcos, F., & Denkova, E. (2014). Current emotion research in cognitive neuroscience: Linking
enhancing and impairing effects of emotion on cognition. Emotion Review, 6(4), 362–375.
Dolcos, F., Iordan, A. D., & Dolcos, S. (2011). Neural correlates of emotion–cognition
interactions: A review of evidence from brain imaging investigations. Journal of Cognitive
Psychology, 23(6), 669–694.
Du Bois, J. W. (2007). The stance triangle. Stancetaking in Discourse: Subjectivity, Evaluation,
Interaction, 164, 139–182.
Egidi, G., & Caramazza, A. (2014). Mood-dependent integration in discourse comprehen-
sion: Happy and sad moods affect consistency processing via different brain networks.
NeuroImage, 103,  20–32.
Enfield, N. J. (2013). Relationship thinking: Agency, enchrony, and human sociality. New York:
Oxford University Press.
Federmeier, K. D., Kirson, D. A., Moreno, E. M., & Kutas, M. (2001). Effects of transient, mild
mood states on semantic memory organization and use: An event-related potential investi-
gation in humans. Neuroscience Letters, 305, 149–152.
Fillmore, C., Kay, P., & O’Connor, C. (1988). Regularity and idiomaticity in grammatical
constructions: The case of let alone. Language, 64, 501–538.
Fischer, A. H., & Manstead, A. S. (2016). Social functions of emotion and emotion regulation.
In L. F. Barrett, M. Lewis, & J. M. Haviland-Jones (Eds.). Handbook of emotions (pp. 424–
439). New York: Guilford Press.
Fodor, J. A. (1983). The modularity of mind: An essay on faculty psychology. Cambridge,
MA: MIT Press.
Foolen, A. (2012). The relevance of emotion for language and linguistics. In A. Foolen, U.
M. Lüdtke, T. P. Racine, & J. Zlatev (Eds.), Moving ourselves, moving others: Motion and
emotion in intersubjectivity, consciousness and language (pp. 349–368). Amsterdam: John
Benjamins.
Forgas, J. P. (1995). Mood and judgment: The affect infusion model (AIM). Psychological
Bulletin, 117(1),  39–66.
Foroni, F., & Semin, G. R. (2009). Language that puts you in touch with your bodily feelings: The
multimodal responsiveness of affective expressions. Psychological Science, 20(8), 974–980.
Frijda, N. H. (2008). The psychologists’ point of view. In M. Lewis, J. M. Haviland-Jones, & L. F.
Barrett, (Eds.), Handbook of emotions (pp. 68–87). New York: Guilford Press.
Frijda, N. H. (2013). Emotion regulation: Two souls in one breast? In D. Hermans, B. Rimé, & B.
Mesquita (Eds.), Changing emotions (pp. 137–143). London: Psychology Press.
Fritsch, N., & Kuchinke, L. (2013). Acquired affective associations induce emotion effects in
word recognition: An ERP study. Brain and Language, 124(1),  75–83.
Gantman, A. P., & Van Bavel, J. J. (2015). Moral perception. Trends in Cognitive Sciences, 19(11),
631–633.
Gigerenzer, G. (2007). Gut feelings: The intelligence of the unconscious. London: Penguin.
Goodwin, M., Cekaite, A., & Goodwin, C. (2012). Emotion as stance. In M. L. Sorjonen & A.
Perakyla (Eds.), Emotion in interaction (pp. 16–41). Oxford: Oxford University Press.
Greene, J. (2014). Moral tribes: Emotion, reason and the gap between us and them.
London: Atlantic Books.
Haidt, J. (2012). The righteous mind: Why good people are divided by politics and religion.
London: Allen Lane.
Hamann, S. (2012). Mapping discrete and dimensional emotions onto the brain: Controversies
and consensus. Trends in Cognitive Sciences, 16(9), 458–466.
Harmon-Jones, E., Gable, P. A., & Price, T. F. (2012). The influence of affective states varying in
motivational intensity on cognitive scope. Frontiers in Integrative Neuroscience, 6(73), 1–5.
doi: 10.3389/fnint.2012.00073
Havas, D. A., Glenberg, A. M., Gutowski, K. A., Lucarelli, M. J., & Davidson, R. J. (2010).
Cosmetic use of botulinum toxin-A affects processing of emotional language. Psychological
Science, 21(7), 895–900.
Hoffmann, M., Mothes‐Lasch, M., Miltner, W. H., & Straube, T. (2015). Brain activation to
briefly presented emotional words: Effects of stimulus awareness. Human Brain Mapping,
36(2), 655–665.
Hofmann, W., De Houwer, J., Perugini, M., Baeyens, F., & Crombez, G. (2010). Evaluative con-
ditioning in humans: A meta-analysis. Psychological Bulletin, 136(3), 390–421.
Hunston, S., & Thompson, G. (Eds.). (2000). Evaluation in text: Authorial stance and the con-
struction of discourse. Oxford: Oxford University Press.
Jaanus, H., Defares, P. B., & Zwaan, E. J. (1990). Verbal classical conditioning of evaluative
responses. Advances in Behaviour Research and Therapy, 12(3), 123–151.
Jackendoff, R. (2007). A parallel architecture perspective on language processing. Brain
Research, 1146,  2–22.
Janak, P. H., & Tye, K. M. (2015). From circuits to behaviour in the amygdala. Nature, 517(7534),
284–292.
Jay, T. (2009). The utility and ubiquity of taboo words. Perspectives on Psychological Science,
4(2), 153–161.
Jensen, T. W. (2014). Emotion in languaging: Languaging as affective, adaptive and flexible be-
havior in social interaction. Frontiers in Psychology, 5, 720.
Johnson-Laird, P. N. (1983). Mental models: Toward a cognitive science of language, inference,
and consciousness. Cambridge, MA: Harvard University Press.
Kahneman, D. (2011). Thinking, fast and slow. New York: Farrar, Straus & Giroux.
Keuper, K., Zwanzger, P., Nordt, M., Eden, A., Laeger, I., Zwitserlood, P., Kissler, J., Junghöfer,
M., & Dobel, C. (2014). How “love” and “hate” differ from “sleep”: Using combined electro/
magnetoencephalographic data to reveal the sources of early cortical responses to emotional
words. Human Brain Mapping, 35(3), 875–888.
Kiesling, S. F. (2011, April). Stance in context: Affect, alignment and investment in the analysis
of stancetaking. In iMean conference (Vol. 15). http://www.academia.edu/1037087/Stance_
in_context_Affect_alignment_and_investment_in_the_analysis_of_stancetaking
Kintsch, W. (1998). Comprehension: A paradigm for cognition. Cambridge: Cambridge

University Press.
Kockelman, P. (2004). Stance and subjectivity. Journal of Linguistic Anthropology, 14(2),
127–150.
Koelsch, S., Jacobs, A. M., Menninghaus, W., Liebal, K., Klann-Delius, G., von Scheve,
C., & Gebauer, G. (2015). The quartet theory of human emotions: An integrative and
neurofunctional model. Physics of Life Reviews, 13,  1–27.
Kragel, P. A., & LaBar, K. S. (2016). Decoding the nature of emotion in the brain. Trends in
Kuchinke, L., Fritsch, N., & Müller, C. J. (2015). Evaluative conditioning of positive and nega-
tive valence affects P1 and N1 in verbal processing. Brain Research, 1624, 405–413.
Künecke, J., Sommer, W., Schacht, A., & Palazova, M. (2015). Embodied simulation of emo-
tional valence: Facial muscle responses to abstract and concrete words. Psychophysiology,
52(12), 1590–1598.
Lakoff, G. (1987). Women, fire, and dangerous things: What categories reveal about the mind.
Chicago: University of Chicago Press.
Lazarus, R. S. (1991). Emotion and adaptation. Oxford: Oxford University Press.
Lebrecht, S., Bar, M., Barrett, L. F., & Tarr, M. J. (2012). Micro-valences: Perceiving affective va-
lence in everyday objects. Frontiers in Psychology, 17(3), 107.
LeDoux, J. (1996). The emotional brain: The mysterious underpinnings of emotional life.
New York: Simon & Schuster.
Leuthold, H., Kunkel, A., Mackenzie, I. G., & Filik, R. (2015). Online processing of moral
transgressions: ERP evidence for spontaneous evaluation. Social, Cognitive, and Affective
Neuroscience, 10(8), 1021–1029.
Levinson, S. C. (2006). On the human “interaction engine.” In N. J. Enfield & S. C. Levinson
(Eds.), Roots of human sociality: Culture, cognition and interaction (pp. 39–69). Oxford: Berg.
Li, W., Moallem, I., Paller, K. A., & Gottfried, J. A. (2007). Subliminal smells can guide social
preferences. Psychological Science, 18(12), 1044–1049.
Majid, A. (2012). Current emotion research in the language sciences. Emotion Review, 4(4),
432–443.
Marr, D. (1982). Vision: A computational investigation into the human representation and pro-
cessing of visual information. Cambridge, MA: MIT Press.
Nussbaum, M. C. (2003). Upheavals of thought: The intelligence of emotions. Cambridge:
Ortigue, S., Michel, C. M., Murray, M. M., Mohr, C., Carbonnel, S., & Landis, T. (2004).
Electrical neuroimaging reveals early generator modulation to emotional words.
NeuroImage, 21(4), 1242–1251.
Panksepp, J., & Biven, L. (2012). The archaeology of mind: Neuroevolutionary origins of human
emotions. New York: W. W. Norton.
Park, J., & Banaji, M. R. (2000). Mood and heuristics: The influence of happy and sad states on
sensitivity and bias in stereotyping. Journal of Personality and Social Psychology, 78(6), 1005.
Pell, M. D. (1999). Fundamental frequency encoding of linguistic and emotional prosody by
right hemisphere-damaged speakers. Brain and Language, 69(2), 161–192.
Peräkylä, A., & Sorjonen, M. L. (2012). Emotion in interaction. Oxford: Oxford University Press.
Pessoa, L. (2008). On the relationship between emotion and cognition. Nature Reviews
Pessoa, L. (2010). Emergent processes in cognitive-emotional interactions. Dialogues in

Clinical Neuroscience, 12(4), 433.
Pessoa, L. (2017). A network model of the emotional brain. Trends in Cognitive Sciences, 21(5),
357–371.
Phelps, E. A. (2006). Emotion and cognition: Insights from studies of the human amygdala.
Annual Review of Psychology, 57,  27–53.
Phelps, E. A., Lempert, K. M., & Sokol- Hessner, P. (2014). Emotion and decision
making: Multiple modulatory neural circuits. Annual Review of Neuroscience, 37, 263–287.
Ponz, A., Montant, M., Liegeois-Chauvel, C., Silva, C., Braun, M., Jacobs, A. M., & Ziegler, J.
C. (2014). Emotion processing in words: A test of the neural re-use hypothesis using surface
and intracranial EEG. Social, Cognitive and Affective Neuroscience, 9, 619–627.
Pratt, N. L., & Kelly, S. D. (2008). Emotional states influence the neural processing of affective
language. Social Neuroscience, 3(3–4), 434–442.
Prinz, J. J. (2004). Gut reactions: A perceptual theory of emotion. Oxford: Oxford
University Press.
Pulvermüller, F. (2012). Meaning and the brain: The neurosemantics of referential, interactive,
and combinatorial knowledge. Journal of Neurolinguistics, 25(5), 423–459.
Rowe, G., Hirsh, J. B., & Anderson, A. K. (2007). Positive affect increases the breadth of atten-
tional selection. Proceedings of the National Academy of Sciences, 104(1), 383–388.
Sander, D., & Scherer, K. (Eds.). (2009). Oxford companion to emotion and the affective sciences.
Satpute, A. B., Wager, T. D., Cohen-Adad, J., Bianciardi, M., Choi, J. K., Buhle, J. T., Wald, L.
L., & Barrett, L. F. (2013). Identification of discrete functional subregions of the human
periaqueductal gray. Proceedings of the National Academy of Sciences, 110(42), 17101–17106.
Schacht, A., Adler, N., Chen, P., Guo, T., & Sommer, W. (2012). Association with positive out-
come induces early effects in event-related brain potentials. Biological Psychology, 89,
130–136.
Scherer, K. R. (2005). What are emotions? And how can they be measured? Social Science
Information, 44(4), 695–729.
Scott-Phillips, T. (2015). Speaking Our Minds: Why human communication is different, and how
language evolved to make it special. New York: Palgrave MacMillan.
Silva, C., Montant, M., Ponz, A., & Ziegler, J. C. (2012). Emotions in reading: Disgust, empathy
and the contextual learning hypothesis. Cognition, 125(2), 333–338.
Slater, M. D., Johnson, B. K., Cohen, J., Comello, M. L. G., & Ewoldsen, D. R. (2014).
Temporarily expanding the boundaries of the self: Motivations for entering the story world
and implications for narrative effects. Journal of Communication, 64(3), 439–455.
Sperber, D., & Wilson. D. (1995). Relevance: Communication and cognition. Oxford: Blackwell.
Struiksma, M. E., De Mulder, H. N. M., & van Berkum, J. J. A. (2018). The impact of verbal
insults: The effects of person insulted, repetition, and taboo words. Manuscript submitted for
publication.
Tamietto, M., Castelli, L., Vighetti, S., Perozzo, P., Geminiani, G., Weiskrantz, L., & de Gelder,
B. (2009). Unseen facial and bodily expressions trigger fast emotional reactions. Proceedings
of the National Academy of Sciences, 106(42), 17661–17666.
‘t Hart, B., Struiksma, M. E., van Boxtel, A., & van Berkum, J. J. A. (2018a). Emotion in
Stories: Facial EMG Evidence for Both Mental Simulation and Moral Evaluation. Frontiers
in Psychology, 9, 613. doi: 10.3389/fpsyg.2018.00613.
‘t Hart, B., Struiksma, M., Van Boxtel, T., & van Berkum, J. J. A. (2018b). Online affective lan-
guage comprehension: Simulating and evaluating character affect in morally loaded narratives.
Manuscript submitted for publication.
Tomasello, M. (2008). Origins of human communication. Cambridge, MA: MIT Press.
Trueswell, J. C., & Tanenhaus, M. K. (Eds.). (2005). Approaches to studying world-situated lan-
guage use: Bridging the language-as-product and language-as-action traditions. Cambridge,
MA: MIT Press.
van Berkum, J. J. A. (2010). The brain is a prediction machine that cares about good and bad—
any implications for neuropragmatics? Italian Journal of Linguistics, 22(1), 181–208.
van Berkum, J. J. A. (2018). Language comprehension, emotion, and sociality: Aren’t we
missing something? In S. A. Rueschemeyer & G. Gaskell (Eds.). Oxford Handbook of
Psycholinguistics (pp. 644–669). Oxford: Oxford University Press.
van Berkum, J. J. A., De Goede, D., Van Alphen, P. M., Mulder, E. R., & Kerstholt, J. H. (2013).
How robust is the language architecture? The case of mood. Frontiers in Psychology, 4, 505.
van Berkum, J. J. A., Holleman, B., Nieuwland, M., Otten, M., & Murre, J. (2009). Right or
wrong? The brain’s fast response to morally objectionable statements. Psychological Science,
20(9), 1092–1099.
van Berkum, J. J. A., Koornneef, A. W., Otten, M., & Nieuwland, M. S. (2007). Establishing
reference in language comprehension: An electrophysiological perspective. Brain Research,
1146, 158–171.
Vissers, C. Th. W. M., Virgillito, D., Fitzgerald, D. A., Speckens, A. E., Tendolkar, I., van
Oostrom, I., & Chwilla, D. J. (2010). The influence of mood on the processing of syntactic
anomalies: Evidence from P600. Neuropsychologia, 48(12), 3521–3531.
Vuilleumier, P., & Huang, Y. M. (2009). Emotional attention uncovering the mechanisms of af-
fective biases in perception. Current Directions in Psychological Science, 18(3), 148–152.
Wetherell, M. (2012). Affect and emotion: A new social science understanding. Los Angeles,
CA: Sage.
Willems, R. M. (2011). Re-appreciating the why of cognition: 35 years after Marr and Poggio.
Zadra, J. R., & Clore, G. L. (2011). Emotion and perception: The role of affective information.
Wiley Interdisciplinary Reviews: Cognitive Science, 2(6), 676–685.
Zajonc, R. B. (1980). Feeling and thinking: Preferences need no inferences. American
Psychologist, 35(2), 151.
Zwaan, R. A. (1999). Situation models: The mental leap into imagined worlds. Current
Directions in Psychological Science, 8(1),  15–18.
Pa rt   V
GRAMMAR AND
C O G N I T ION
Chapter 30
Gr am matical C at e g ori e s
David Kemmerer
Introduction
Grammatical categories— also known as syntactic categories, lexical categories,

word classes, and parts of speech—are fundamental to human languages because they
strongly influence the ways in which conceptual structures are packaged in lexical
items and, even more important, the ways in which lexical items are combined to form
complex expressions. Across the roughly 7,000 languages of the world, the most basic
and widely attested categories are nouns, verbs, adjectives, and adverbs. They typically
make up large open classes whose memberships constantly fluctuate as new items are
added and old ones fade away. In contrast, other categories tend to be both small and
closed, with memberships that do not change very quickly over time. Some familiar
examples from English are prepositions, conjunctions, and articles, and some lesser
known examples from other languages are postpositions, classifiers, and focus markers.
Although a great deal has been learned about the nature of grammatical categories
during the past few decades, several competing accounts have been proposed by gener-
ative, cognitive, functional, and typological theorists, and many issues remain contro-
versial (for a survey of approaches, see Rauh, 2010; see also Haspelmath, 2012, and the
debate between Haspelmath, 2007, and Newmeyer, 2007).
Recent neurolinguistic research on grammatical categories has addressed a variety of
phenomena, but the vast majority of studies have been devoted to a single topic of cen-
tral importance, namely the noun-verb distinction. In fact, the neural underpinnings
of this distinction have been explored in countless investigations employing all of the
major brain mapping methods, including deficit-lesion correlations in brain-damaged
patients (see Wilson, Chapter 2 in this volume), positron emission tomography (PET),
functional magnetic resonance imaging (fMRI; see Heim & Specht, Chapter 4 in this
volume), transcranial magnetic stimulation (TMS; see Schuhmann, Chapter 5 in this
volume), extracranial and intracranial electrophysiology (see Duffau, Chapter 8 in
this volume, and magnetoencephalography (MEG; see Salmelin, Kujala, & Liljeström,
770 David Kemmerer
Chapter 6 in this volume) (for reviews and meta-analyses see Berlingeri et al., 2008;
Black & Chiat, 2003; Cappa & Perani, 2003; Crepaldi, Berlingeri, Paulesu, & Luzzatti,
2011, 2013; Druks, 2002; Kemmerer, 2014; Luzzatti, Aggujaro, & Crepaldi, 2006;
Mätzig, Druks, Masterson, & Vigliocco, 2009; Pillon & d’Honincthun, 2010; Shapiro &
Caramazza, 2003b, 2004; Vigliocco, Vinson, Druks, Barber, & Cappa, 2011). As a con-
sequence, the literature on this topic is now quite large and complex. But even though
it contains a wealth of empirically solid, theoretically interesting, and clinically rele-
vant material, there are still many unanswered questions. With the aim of providing a
concise overview of the field, this chapter concentrates on some of the most prominent
neurolinguistic issues regarding the noun-verb distinction. A few other topics, how-
ever, are also briefly discussed toward the end of the chapter.
Nouns and Verbs
Starting with the Universal Semantic Prototypes

According to several linguistic frameworks, the most basic grammatical categories can
be characterized partly in terms of universal prototypes that are grounded in meaning
(Anderson, 1997; Croft, 1991, 2000, 2007; Langacker, 1987; O’Grady, 1997; Wierzbicka,
2000). For example, Croft (1991, 2000, 2007) argues that prototypical nouns and verbs
reflect certain default combinations of pragmatic function and semantic class, with
nouns involving reference to objects and verbs involving predication of actions (for pre-
sent purposes, “reference” is what the speaker is talking about, and “predication” is what
the speaker asserts about the referent(s) in a particular utterance). Support for this pro-
posal comes from corpus analyses that have consistently yielded the following results
across diverse languages worldwide: referential constructions contain predominantly
object words, and object words appear predominantly in referential constructions; sim-
ilarly, predicative constructions contain predominantly action words, and action words
appear predominantly in predicative constructions.
In light of these considerations, it is not surprising that the lion’s share of neuro-
linguistic research on basic grammatical categories has focused on object nouns and
action verbs. Some of the most influential studies have shown that each of these pro-
totypical categories can be either selectively or disproportionately impaired by brain
damage. Although relative verb deficits are more common than relative noun deficits,
numerous double dissociations have been documented, especially using oral picture-
naming tasks, and often the accuracy differences between the two word classes are quite
substantial (see Mätzig et al., 2009, for a review of 240 previously reported patients and
9 new ones). For example, Table 30.1 presents data from 20 representative cases—10
with worse oral production of nouns than verbs, and 10 with the opposite performance
profile—all of whom displayed an accuracy difference of at least 30%.
Grammatical Categories 771
Table 30.1 Examples of Large (30%+) Dissociations

Between Prototypical Nouns and Verbs in Oral
Picture-Naming  Tasks.
Patients with Worse Noun Patients with Worse Verb Than
Than Verb Production Noun Production
Case Nouns Verbs Case Nouns Verbs
Mario 7 88 BW 98 60
H.Y. 35 85 FDP 96 50
H.F. 49 83 LK 93 63
E.A. 42 82 LR 92 40
Z.B.L. 41 78 EM 90 59
S.K. 47 77 TB 88 37
M.L. 38 75 UB 87 48
S.F. 27 73 FC 87 30
E.B.A. 12 72 RE 83 35
R.G. 29 64 MB 83 35
Cells indicate percent correct.

Data from Mätzig et al. (2009).
For some patients with category-related deficits like these, the locus of the cognitive
impairment appears to be at the level of conceptual knowledge, as indicated by the fol-
lowing forms of convergent evidence. First, the majority of errors are either omissions
(i.e., “don’t know” responses) or semantic substitutions (e.g., calling a penguin a duck).
Second, the relevant word class is disrupted not only during production tasks like pic-
ture naming, but also during comprehension tasks like word-picture matching. Third,
the lesions affect brain regions that are likely to be involved in representing the relevant
kinds of concepts. This last point is elaborated in the following.
Patients who exhibit significantly greater noun than verb deficits usually have damage
in the left temporal lobe, due to either a stroke (Mätzig et al., 2009) or a degenerative
disorder such as Alzheimer’s disease (Cappa et al., 1998) or herpes simplex encepha-
litis (Damasio & Tranel, 1993). These findings fit well with other literature regarding the
neural substrates of the kinds of object concepts that tend to be encoded by prototypical
nouns. When people process such concepts, they often retrieve various types of modality-
specific object properties, including shape, color, sound, smell, taste, and manipulability
(Binder & Desai, 2011). Among all of these features, however, the one that carries the
most semantic weight is shape (Gainotti, Spinelli, Scaricamazza, & Marra, 2013). It is well
established that the posterior and middle portions of the ventral temporal cortex are es-
sential for recognizing objects based on perceived shape (Damasio, Tranel, Grabowski,
Adolphs, & Damasio, 2004; Kriegeskorte et al., 2008). Additionally, a great deal of data
772 David Kemmerer
suggests that some ventral temporal regions—partly overlapping, but mostly lying next
to those involved in perception—also represent shape information when words for
objects are processed (Capitani et al., 2009; Martin, 2007). Moreover, compared to the
comprehension of action words, the comprehension of object words engages the ven-
tral temporal cortex quite rapidly, within about 200 milliseconds (ms) (Chan et al., 2011;
Pulvermüller, Lutzenberger, & Preissl, 1999). Taken together, these discoveries support
the hypothesis that the ventral temporal cortex implements the most important repre-
sentational component, namely shape, of the meanings of prototypical nouns.
As for patients who exhibit significantly greater verb than noun deficits, their lesions
are rather disparate, but usually one or more of the following left hemisphere areas is
affected: the inferior frontal gyrus (IFG), often together with nearby motor areas; the
inferior parietal lobule, especially the supramarginal gyrus (SMG); and the posterior
middle temporal gyrus (pMTG). These findings come from studies of patients who have
suffered either a stroke (Kemmerer, Rudrauf, Manzel, & Tranel, 2012) or a degenerative
disorder such as motor neuron disease (Bak & Hodges, 2004), progressive nonfluent
aphasia (Hillis et al., 2006), or corticobasal degeneration (Silveri & Ciccarelli, 2007).
Such deficit-lesion correlations dovetail with other data suggesting that the implicated
regions underlie the kinds of action concepts that tend to be encoded by prototypical
verbs. Although the precise ways in which these regions contribute to action concepts
are only beginning to be elucidated, some of the leading ideas are as follows. In the
frontal lobe, the IFG—in particular, Brodmann area 44—may subserve the sequen-
tial and hierarchical organization of action concepts (Clerget, Winderickx, Fadiga, &
Olivier, 2009; Fazio et al., 2010), and precentral motor areas may subserve the body-
part-specific motor features of action concepts (Kemmerer, 2015a; Pulvermüller, 2013).
In the parietal lobe, the SMG may subserve the spatial-relational and kinematic aspects
of action concepts (Goldenberg, 2009; Kalénine, Buxbaum, & Coslett, 2010). Moreover,
in the temporal lobe, the pMTG may subserve the visual motion patterns as well as the
participant roles (e.g., chaser vs. chasee) specified by action concepts (Watson, Cardillo,
Ianni, & Chatterjee, 2013; Wu, Waller, & Chatterjee, 2007). It is noteworthy that all of
these brain regions either overlap with or lie next to regions that are recruited during
the execution and observation of actions (Molenberghs, Cunnington, & Mattingley,
2012). Furthermore, compared to the comprehension of object words, the comprehen-
sion of action words engages motor-related frontal regions very quickly, within about
200 ms (Pulvermüller, Lutzenberger, & Preissl, 1999; Shtyrov, Butorina, Nikolaeva, &
Stroganova, 2014). Overall, these findings support the hypothesis that the frontal, pa-
rietal, and temporal regions described in the preceding implement the core representa-
tional parameters of the meanings of prototypical verbs.
Category-Related Disorders Involving

the Representation/Retrieval of Lexical Forms
So far we have focused on semantic accounts of the sorts of noun-verb dissociations that
are frequently displayed by brain-damaged patients when they perform picture-naming
tasks. It is important to realize, however, that such dissociations can also arise from post-
semantic impairments. This is revealed in a particularly striking way by patients who
exhibit word-production deficits that selectively or disproportionately affect not just
one grammatical category—either nouns or verbs—but also just one output channel—
either speaking or writing (Caramazza & Hillis, 1991; Hillis, Tuffiash, & Caramazza,
2002; Hillis, Wityk, Barker, & Caramazza, 2003; Rapp & Caramazza, 1998; see also Rapp
& Goldrick, 2006).
For example, as shown in Figure 30.1, case M.M.L., who suffered from progressive
nonfluent aphasia, manifested steadily worsening spoken production of prototypical
verbs over a 2.5-year period, while her written production of prototypical verbs, as well
as her spoken and written production of prototypical nouns, remained fairly accurate
(Hillis et al., 2002). M.M.L.’s poor spoken production of prototypical verbs could not be
due to impaired knowledge or processing of action concepts because her written pro-
duction of the very same verbs was good. Instead, her remarkably specific deficit must
reflect some kind of post-semantic problem.
Cases like this raise many intriguing questions about the relationships between
meaning, grammar, and the modality-specific (i.e., phonological and orthographic)
lexicons (for detailed theoretical discussion, see Caramazza & Miozzo, 1998; Rapp &
Caramazza, 2002). Here, however, we will restrict our attention to the issue of whether
the word forms in each lexicon are segregated according to grammatical category. On
the one hand, the neuropsychological dissociations exhibited by patients like M.M.L. are
clearly consistent with the hypothesis that the noun-verb distinction is respected at the
level of lexical forms. For example, M.M.L.’s own performance profile could be explained
in terms of increasing damage to just one set of words—specifically, the verb compart-
ment of the phonological lexicon (Figure 30.2 A). On the other hand, the available data
100
80
% Correct
60
40
20
0
8 8.5 9 9.5 10
No. of years post diagnosis
Noun/written Noun/oral
Verb/written Verb/oral
Figure 30.1. Longitudinal performance of the progressive nonfluent aphasic patient

M.M.L. on tasks involving the spoken and written naming of objects and actions with nouns and
verbs, from ~8 to 10.5 years following diagnosis. Spoken production of verbs deteriorated steadily
(red plot), but written production of verbs, as well both spoken and written production of nouns,
remained unaffected.
Source: Reproduced with permission from Shapiro & Caramazza (2003); original data from Hillis et al. (2002).
774 David Kemmerer
(a)
Semantic System
Phonological Output Lexicon Orthographic Output Lexicon
Nouns Verbs Nouns Verbs

“apple” “bite” “apple” “bite”
(b)
Semantic System
Object Concepts Action Concepts

“APPLE” “BITE”
Phonological Output Lexicon Orthographic Output Lexicon
Figure 30.2. Two alternative accounts of the performance profile of patient M.M.L. depicted

in Figure 30.1. (A) An account that assumes segregation of nouns and verbs within the modality-
specific output lexicons. (B) An account that assumes segregation of word meanings according to
conceptual category within the semantic system. In each architecture, the red cross indicates the
site of progressive dysfunction.
can also be accommodated by an alternative hypothesis that does not require nouns and
verbs to be differentiated at the lexical level, but instead assumes that the connectional
pathways between the meanings and forms of particular word classes can be independ-
ently disrupted. Applied to M.M.L., such an account could be formulated as follows:
(1) the meanings of prototypical nouns and verbs are represented in separate semantic
subsystems, as discussed in the previous section; (2) each semantic subsystem projects
to both of the modality-specific lexicons; and (3) M.M.L.’s disorder involves increasing
damage to just one route—specifically, the pathway that projects from action concepts
to the corresponding verb forms in the phonological lexicon (Figure 30.2 B).
Thus, even though patients with combined grammatical category–specific and output
channel–specific deficits support the view that the lexical forms of nouns and verbs are
segregated in the brain, they certainly do not force such a conclusion. It is noteworthy,
however, that the question of whether grammatical category distinctions are captured
at the lexical level continues to be controversial, and it has been addressed from several
other perspectives, as shown in the next section.
Efforts to Overcome Confounds Between Conceptual

and Grammatical Factors
From a methodological point of view, there are both merits and shortcomings to de-
liberately designing some neurolinguistic studies so that they focus specifically on the
most prototypical grammatical categories—that is, object nouns and action verbs. On
the positive side, such a strategy can shed light on how the brain implements the uni-
versal prototypes identified by typologically oriented linguists like Croft (1991, 2000,
2007). On the negative side, however, conflating the conceptual distinction between
objects and actions with the grammatical distinction between nouns and verbs makes
it difficult, if not impossible, to discover the neural correlates of a much wider range
of phenomena involving language-particular word classes. In an effort to overcome
this limitation, an increasing number of researchers have begun to conduct studies in
which conceptual-grammatical confounds are minimized. Two different approaches to
doing this are summarized in the following. As we will see, however, both of them have
generated mixed results, since some studies suggest that, qua lexical items, nouns and
verbs are not systematically differentiated in the brain, whereas other studies suggest
that they rely on at least partly separate neural networks.
Strategy 1: Closely Match the Meanings and Processing Demands

of Nouns and Verbs
According to some studies, when nouns and verbs are well matched for semantic
dimensions as well as processing requirements, they recruit essentially the same brain
regions to the same degrees. For instance, in a PET study carried out in Italian, Vigliocco
et al. (2006) asked subjects to simply listen to four sets of words. Crucially, all of the
word lists were carefully created so as to allow the investigators to independently de-
termine which patterns of brain activity were due to conceptual factors and which were
due to grammatical factors:
• sensory nouns (SNs), i.e., nouns with rich sensory features (e.g., lampi, “lightning”)
• motor nouns (MNs), i.e., nouns with rich motor features (e.g., giravolta, “twirl”)
• sensory verbs (SVs), i.e., verbs with rich sensory features (e.g., luccicano, “shine”)
• motor verbs (MVs), i.e., verbs with rich motor features (e.g., scuote, “shake”).
Two contrasts were performed to identify the neural correlates of sensory words and
motor words, irrespective of grammatical category, and both of these contrasts revealed
activity in some of the expected areas. Specifically, with regard to sensory words, the
contrast [(SNs + SVs)—(MNs + MVs)] revealed activity in the left ventral temporal
776 David Kemmerer
cortex, and with regard to motor words, the opposite contrast [(MNs + MVs)—(SNs
+ SVs)] revealed activity in the left lateral motor cortex. In addition, and more im-
portant, two contrasts were performed to identify the neural correlates of nouns and
verbs, irrespective of conceptual considerations; however, neither of these contrasts
yielded any significant effects whatsoever, even when the investigators switched from
whole-brain analyses to region-of-interest analyses. More precisely, no activity was
revealed by the contrast [(SNs + MNs)—(SVs + MVs)], which was meant to identify
regions unique to nouns, and likewise, no activity was revealed by the opposite con-
trast [(SVs + MVs)—(SNs + MNs)], which was meant to identify regions unique to
verbs. What these findings suggest is that when conceptual-grammatical confounds
are reduced, and when computational loads are also controlled, nouns and verbs are
not segregated in the brain, but instead depend on shared neural resources (see also
Barber, Kousta, Otten, & Vigliocco, 2010; Siri et al., 2008; Vigliocco et al., 2011; as well
as Collina, Marangolo, & Tabossi, 2001; Pulvermüller, Mohr, & Schleichert, 1999; and
Tabossi, Collina, Pizzioli, & Basso, 2010).
On the other hand, there is some recent evidence for the view that nouns and verbs
do recruit at least partly separate neural networks, even when their meanings and pro-
cessing demands are fairly similar. Focusing once again on Italian, Tsigka, Papadelis,
Braun, and Miceli (2014) conducted an MEG study in which participants silently
read noun-verb homonyms in two syntactically minimal contexts that each had two
variants (for a similar study in Spanish, only using event-related potentials, see Yudes,
Domínguez, Cuetos, & de Vega, 2016):
• article + noun
• singular (e.g., il ballo, “the dance”)
• plural (e.g., i balli, “the dances”)
• pronoun + verb
• first-person singular (e.g., io ballo, “I dance”)
• second-person singular (e.g., tu balli, “you dance”).
The results of source analyses using minimum norm estimates (MNE) are depicted in
Figure 30.3. Looking first at the time windows for function word processing, it can be
seen that the articles and pronouns activated essentially the same posterior bilateral
areas; however, relative to the articles, the pronouns evoked greater activity at right pa-
rietal sites during the 88–108 ms window and at left prefrontal sites during the 232–251
ms window. Turning now to the more interesting time windows for content word pro-
cessing, what stands out most clearly is that the verbs elicited more extensive and robust
activity than the nouns over the entire course of measurement—initially in right parietal
clusters during the 115–136 and 195–212 ms windows, then in bilateral central and mid-
line regions during the 280–300 ms window, next in the left IFG during the 297–319 ms
window, then in a large left parietal cluster during the 380–397 ms window, and finally
again in the left IFG during the 393–409 ms window. Because the noun-verb homonyms
were very close in meaning, the neurotopographic differences that emerged between
Function Word Content Word

Phrases
Noun
Phrases
Verb
Statistics
Figure 30.3. MEG-based minimum norm estimates (MNEs) for the time windows during
which significant differences between visually presented noun-phrases (NPs) and verb-phrases
(VPs) were revealed by cluster analysis. The top and middle rows display the averaged MNEs of
NP-and VP-related activity across the consecutive time slices for which cluster analysis revealed
significant differences between NPs and VPs. The bottom row shows the extension of clusters
that significantly differed between NPs and VPs. Significant clusters were defined as spatially and
temporally contiguous brain regions (>10 vertices; >12 samples) whose spatial extent, however,
could vary over time. To indicate the stability of the cluster, the percentage of samples with re-
spect to overall temporal persistence of the cluster is coded by color intensity. A value of 100%
indicates that the corresponding vertex belonged to the cluster throughout the cluster’s lifetime.
Green and pink backgrounds indicate time windows corresponding to the presentation of func-
tion word and content word, respectively.
Source: Reproduced with permission from Tsigka et al. (2014, p. 92).
them are unlikely to reflect conceptual factors. Hence, according to Tsigka et al. (2014,
p. 95), the findings “invite the conclusion that at least partly separable neural substrates
are involved in processing grammatical class information and the corresponding mor-
phosyntactic operations.” However, given that the homonymous nouns and verbs en-
gaged many of the same regions, only with verbs activating them more strongly, the
authors acknowledge that the data do not completely rule out the possibility that the two
categories have some cortical underpinnings in common at the lexical level.
Strategy 2: Investigate Both Concrete and Abstract Nouns and Verbs

Pursuing another approach to disentangling the conceptual and grammatical aspects
of word classes, Moseley and Pulvermüller (2014) carried out an fMRI study in which
participants silently read the following four types of stimuli:
• concrete nouns (e.g., mouse)

• concrete verbs (e.g., chomp)
• abstract nouns (e.g., truce)
• abstract verbs (e.g., glean)
Analyses of the imaging data revealed a significant interaction between conceptual (i.e.,
concrete vs. abstract) and grammatical (i.e., noun vs. verb) factors, especially in left
778 David Kemmerer
frontocentral regions. Further exploration, however, indicated that although the con-
crete nouns and verbs triggered distinct patterns of cortical engagement, the abstract
nouns and verbs did not. Based on these results, the authors concluded—in a manner
similar to Vigliocco et al. (2006)—that when neurotopographic differences are observed
between nouns and verbs in single-word processing tasks, they reflect semantic factors
rather than lexical segregation.
On the other hand, an earlier study by Berndt, Haendiges, Burton, and Mitchum
(2002) used neuropsychological methods to support the hypothesis that nouns and
verbs have at least partly distinct neural substrates, regardless of whether their meanings
are concrete or abstract. The investigators administered two kinds of tasks to seven
brain-damaged patients. The first task employed a standard oral picture-naming par-
adigm to assess the patients’ retrieval of prototypical object nouns and action verbs.
The second task, however, employed an auditory sentence-completion paradigm to as-
sess the patients’ retrieval of not only concrete nouns and verbs, all of which had high
imageability, but also abstract nouns and verbs, all of which had low imageability. Some
examples of the four conditions are as follows:
• concrete noun (e.g., scale): Bonny had been following a strict diet. To find out if it was
working, she weighed herself on the _____.
• concrete verb (e.g., rob): The bandits were planning their next holdup. They needed to
decide which bank would be easiest to _____.
• abstract noun (e.g., fault): Jennifer was upset that she might be blamed for the acci-
dent. She hoped everyone knew that it wasn't her _____.
• abstract verb (e.g., stay): Andy found it difficult to visit his mother in the hospital. He
wanted to get away, but she wanted him to _____.
The seven patients exhibited several different kinds of performance profiles across
the tasks, but for present purposes the most interesting case was R.E. As shown in
Figure 30.4, she manifested significantly worse production of verbs than nouns
across the board, with no effect of imageability in the sentence-completion task (for
similar data from other patients see Crepaldi et al., 2006). This demonstrates that,
irrespective of conceptual considerations, verbs can be disproportionately impaired
relative to nouns. And this in turn provides some leverage for the view that the neural
substrates of word classes are determined to some extent by genuinely grammatical
factors.
Focusing on Inflection
Another way in which researchers have transcended the limitations of prototypical
nouns and verbs is by investigating category-specific morphosyntactic operations that
are not constrained by the conceptual distinction between objects and actions. So far,
1
0.9
0.8
Proportion Correct
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
Picture Naming High Image Low Image
Sentence Completion
Noun Verb
Figure 30.4. Proportion of correct responses by patient R.E. for nouns and verbs in a task
involving object/action picture naming (left panel) and in a task requiring sentence completion
with target words rated as having high or low imageability (right panel). Verb production was sig-
nificantly worse than noun production in both tasks, and there was no effect of imageability in the
sentence completion task.
Source: Reproduced with permission from Berndt et al. (2002, p. 361).
most of this work has attempted to elucidate the neural networks that underlie number
inflection for count nouns (e.g., I have two thoughts about that) and tense/agreement
inflection for main verbs (e.g., Yesterday I developed a new plan). Overall, the litera-
ture suggests that during production tasks, these two types of category-specific mor-
phosyntactic operations depend on neural networks that are partly shared and partly
segregated (for reviews, see Kemmerer, 2015b, Chapter 13; Shapiro & Caramazza,
2009). Broca’s area (i.e., BAs 44 and 45 in the left IFG) seems to implement a relatively
late stage of processing that occurs just before morphophonological encoding and
that is essentially the same for the two classes of words (Cappelletti, Fregni, Shapiro,
Pascual-Leone, & Caramazza, 2008; Sahin, Pinker, Cash, & Halgren, 2009; Sahin,
Pinker, & Halgren, 2006). However, the computation of appropriate morphosyntactic
features—that is, number features for count nouns and tense/agreement features for
main verbs—seems to take place during an earlier, higher-order stage of processing,
and at this stage the operations for the two classes of words appear to have separate
neural underpinnings.
Some of the evidence for this view comes from neuropsychological studies that have
revealed double dissociations between number inflection for count nouns and tense/
agreement inflection for main verbs. For example, Shapiro, Shelton, and Caramazza
(2000) and Shapiro and Caramazza (2003a) reported two stroke patients, J.R. and
R.C., who were given two tasks that required them to complete sentence frames with
the correctly inflected forms of words. In the first task, the key words were real noun-
verb homonyms that had equivalent forms and closely related meanings (e.g., a guide;
to guide). As shown in the following, two sentence frames called for singular and plural
780 David Kemmerer
forms of count nouns, and two sentence frames called for third-person plural and third-
person singular forms of main verbs:
• Inflection of real nouns
• These are guides; this is a _____(guide-ø)
• This is a guide; these are _____(guides)
• Inflection of real verbs
• This person guides; these people _____(guide-ø)
• These people guide; this person _____(guides)
In the second task, the key words were pseudo-noun-verb homonyms (e.g., a fleeve; to
fleeve), which gave the researchers even more assurance that if the patients responded
differently to the two categories, it could not be due to either phonological or concep-
tual factors. As shown in the following, the sentence frames were identical to those em-
ployed in the first task:
• Inflection of pseudo-nouns
• These are fleeves; this is a _____(fleeve-ø)
• This is a fleeve; these are _____(fleeves)
• Inflection of pseudo-verbs
• This person fleeves; these people _____(fleeve-ø)
• These people fleeve; this person _____(fleeves)
The central finding was that the two patients manifested opposite performance profiles
for the two types of inflection, and their dissociations were present for both real words
and pseudo-words. In particular, J.R. was significantly worse at inflecting count nouns
for number than main verbs for tense/agreement, whereas R.C. was significantly worse
at inflecting main verbs for tense/agreement than count nouns for number (for sim-
ilar dissociations displayed by other patients, see Benetello et al., 2016; Laiacona &
Caramazza, 2004; Tsapkini, Jarema, & Kehayia, 2002). In their discussion of the com-
plementary deficits exhibited by J.R. and R.C., Shapiro and Caramazza (2003a, p. 1194)
emphasize that in both cases the disrupted capacities are not only category-specific, but
also “seem to be grammatical, and not directly involved in retrieving stored information
about word form or meaning; otherwise it is not clear how we might account for the
observed deficits with pseudo words, which presumably have no memorized features.”
The discovery that certain morphosyntactic operations for count nouns and main
verbs can be differentially impaired raises the question of which brain regions subserve
those operations. Unfortunately, very little is known about the regions that underlie the
computation of number features for count nouns, but there are some hints that this pro-
cess may depend on cooperative interactions between the left mid/anterior fusiform
gyrus, the left inferior parietal lobule, and the superior portion of Broca’s area (Sahin
et al., 2006; Shapiro & Caramazza, 2009; Shapiro, Moo, & Caramazza, 2006, 2012; see
also Domahs et al., 2012). On the other hand, considerable progress has been made in
identifying the regions that underlie the computation of tense/agreement features for
main verbs, with most of the data pointing to a network involving the left posterior
middle temporal gyrus (pMTG) and the left middle frontal gyrus (MFG). Here we will
focus on the latter region and address the specific issue of whether its anterior or poste-
rior sector is most critical.
Support for the importance of the posterior portion of the left MFG comes from case
R.C., since he was disproportionately impaired for verb inflection, and his lesion, but
not J.R.’s, extended into this territory. In addition, several fMRI studies suggest that, rel-
ative to noun inflection, verb inflection recruits the left posterior MFG immediately su-
perior to BA 44. For instance, Shapiro et al. (2006) obtained results along these lines in
a study that used real and pseudo-word tasks like those described in the preceding (see
also Shapiro et al., 2012). When the verb conditions were contrasted against the noun
conditions, one of the only activated areas was the left posterior MFG. The same re-
gion was also engaged significantly more by verb than noun inflection in an fMRI study
by Willms et al. (2011), but these researchers went two steps further than Shapiro et al.
(2006): first, they employed tasks that required English-Spanish bilingual speakers to
inflect words in both languages; and second, they used multi-voxel pattern analysis to
show that within the left posterior MFG the specific activation patterns elicited by the
verb conditions, relative to the noun conditions, were virtually identical across the two
languages. Finally, Kielar, Milman, Bonakdarpour, and Thompson (2011) recently found
that both the overt and covert production of tense/agreement morphology for English
verbs recruited the left posterior MFG, together with the caudally adjacent precentral
gyrus. Overall, then, there appears to be a fair amount of neuropsychological and fMRI
data implicating the posterior sector of the left MFG in verb-specific morphosyntactic
processing.
On the other hand, a recent fMRI study by Finocchiaro, Basso, Giovenzana, and
Caramazza (2010) generated results which suggest that the anterior portion of the left
MFG also contributes to verb-specific morphosyntactic processing. The participants
in this experiment performed tasks that required them to inflect Italian target words
in ways that conformed to certain phrasal contexts. Some of the expressions involved
count nouns (e.g., uno starnuto, “a sneeze”; molti starnuti, “many sneezes”), and others
involved main verbs (e.g., io taglio, “I cut”; tu tagli, “you cut”). When the verb conditions
were contrasted against the noun conditions, one of the only activated areas was the left
anterior MFG, at the intersection of BAs 10, 46, and 47, immediately anterior to BA 45.
Surprisingly, the posterior portion of the left MFG was not engaged.
Given these inconsistencies, it makes sense to ask whether other brain mapping
methods have been used to address the same issues. In fact, a few TMS studies have
yielded two relevant findings. First, Cappelletti et al. (2008) showed that, relative
to sham stimulation, the application of repetitive TMS to the posterior portion of the
left MFG did not interfere significantly with either verb or noun inflection. Second,
both Cappelletti et al. (2008) and Shapiro, Pascual-Leone, Mottaghi, Gangitano, and
Caramazza (2001) showed that, relative to sham stimulation, the application of repetitive
TMS to the anterior portion of the left MFG did interfere significantly more with verb
782 David Kemmerer
inflection than noun inflection (see also Finocchiaro et al., 2008). Taken together, these
findings are at odds with the fMRI studies by Shapiro et al. (2006), Willms et al. (2011),
and Kielar et al. (2011), and also with the neuropsychological data for patient R.C.; how-
ever, they are congruent with the fMRI study by Finocchiaro et al. (2010). Thus, there are
still some uncertainties regarding the precise localization of the neural mechanisms that
underlie tense/agreement inflection for main verbs, but these mechanisms most likely
depend on some portion of the left MFG.
It should be clear from this brief review that neurolinguistic research on category-
specific inflectional processes has been making significant headway. At the same
time, however, it is important to realize that this line of work does not address the
neural correlates of nouns and verbs in any sort of global (i.e., cross-constructional or
construction-independent) sense; instead, it only addresses the neural correlates of the
particular word classes that happen to occur in the inflectional constructions under in-
vestigation. This is why the classes discussed earlier are referred to rather narrowly as
count nouns and main verbs. After all, other kinds of nouns, such as mass nouns, cannot
easily be pluralized (e.g., compare books with *muds), and other kinds of verbs, such
as modal verbs, are prohibited from taking tense markers (e.g., compare walked with
*coulded). Furthermore, it bears mentioning that some languages, like Vietnamese, lack
all inflection, and others, like Sirionó, have inflection but employ it in peculiar ways,
such as by applying tense to words that encode objects and serve as syntactic arguments
of main verbs (Nordlinger & Sadler, 2004). Hence, caution is always warranted when
using generic terms like “noun” and “verb” to describe the lexical categories that occur
in certain constructions of certain languages (Kemmerer, 2014).
Dissociations Between Subcategories

Continuing with the notion that the major parts of speech can be broken down into mul-
tiple subcategories, this last section deals with a few strands of neurolinguistic research
that have begun to explore how such subcategories are implemented in the brain. Some
of the most salient findings in this small but growing literature are briefly summarized
in the following (see also Fieder, Nickels, Biedermann, & Best, 2014a; Fieder, Nickels, &
Biedermann, 2014b).
Proper versus Common Nouns

In the domain of nominal constructions, one of the most fundamental distinctions is
between proper and common nouns. These two classes of words differ both conceptu-
ally and grammatically. Proper nouns (e.g., Barack Obama) designate unique entities,
and in accord with this function they typically do not allow either articles or adjectives
(see Van Valin & LaPolla, 2007, pp. 59–60, for a brief discussion of cross-linguistic diver-
sity and the special interpretive adjustments required by unusual expressions like A tired
Barack Obama returned to the White House). In contrast, common nouns (e.g., dog) des-
ignate entire categories of entities, and for this reason they often co-occur with articles
and adjectives.
Numerous neuropsychological studies have reported patients with significantly
worse retrieval of proper nouns than common nouns, and a handful of studies also
have documented the opposite dissociation (for reviews, see Rapp & Goldrick, 2006;
Semenza, 2006, 2009). In at least one case, a selective deficit in both producing and
comprehending the proper names of people has been traced to a highly circumscribed
impairment at the level of concepts for unique individuals (Miceli et al., 2000; see also
Gainotti, Spinelli, Scaricamazza, & Marra, 2008). More often, however, patients with
greater difficulties retrieving proper than common nouns have fairly well-preserved
knowledge of the referents of the intended words. Interestingly, for some of these
patients the selective or disproportionate deficit in generating proper nouns relative to
common nouns is restricted to just one modality of output, either speaking or writing
(Cipolotti, MacNeil, & Warrington, 1993; Kemmerer, Tranel, & Manzel, 2005). And in
keeping with the earlier discussion of patient M.M.L. (see Figures 30.1 & 30.2 and the
associated text), such combined grammatical category-specific and output channel-
specific disorders raise the intriguing possibility that proper nouns may be segregated
from common nouns at the level of phonological and orthographic lexical forms.
Further research is needed, however, to evaluate this hypothesis, since it is also possible
that the two types of nouns are not systematically differentiated within each lexicon, but
instead are accessed from distinct semantic subsystems via isolable projection pathways.
Turning to the brain, the neural basis of proper noun retrieval in oral picture-naming
tasks has received a considerable amount of attention, and a sizable body of data from le-
sion studies, functional neuroimaging studies, and electrophysiological studies suggests
that the left temporal pole plays an important role (for a review, see Tranel, 2009;
see also Abel et al., 2015; Ross, McCoy, Coslett, Olson, & Wolk, 2011). In fact, there is
growing evidence that this region is essential not only for naming famous faces, but also
for naming famous voices (Waldron, Manzel, & Tranel, 2014), famous melodies (Belfi &
Tranel, 2014), and famous landmarks (Gorno-Tempini & Price, 2001; Grabowski et al.,
2001; Tranel, 2006; see Figure 30.5). The left uncinate fasciculus, which interconnects
the temporal pole with the orbitofrontal cortex, has also been implicated in proper noun
retrieval (Papagno et al., 2011, 2016), and so have several other regions, including the
temporo-occipital cortex and the inferior frontal cortex (for a review, see Semenza,
2011; for fMRI data on the comprehension of proper versus common nouns, see Wang,
Peelen, Han, Caramazza, & Bi, 2016).
Transitive versus Intransitive Verbs

Turning to the realm of verbal constructions, perhaps the most general distinction is
between transitive and intransitive verbs (Dixon, 2010, Chapter 13). Transitive verbs
designate events with two core participants, most often an “actor” and an “undergoer,”
with the actor being syntactically realized as a subject noun-phrase and the undergoer
being syntactically realized as an object noun-phrase, at least in standard active-voice
784 David Kemmerer
Figure 30.5. Lesion overlap map of patients with left temporal polar damage who have sig-
nificantly impaired retrieval of proper nouns in tasks that require naming famous faces, voices,
lankdmarks, and melodies. Images depict the overlap (from left to right) from the lateral per-
spective, ventral perspective, and mesial sagittal perspective. The “hotter” colors (orange to red)
depict a higher number of lesion overlaps.
Source: Reproduced with permission from Waldron et al. (2014, p. 53). See also Belfi & Tranel (2014).
clauses (e.g., The housewife kissed the mailman). In contrast, intransitive verbs designate
events with just one core participant, syntactically realized as a subject noun-phrase.
Usually this participant is a volitional actor (e.g., The mailman bolted), but sometimes
it is a passive undergoer (e.g., The mailman blushed). It all depends on the nature of the
verb. Transitivity itself also depends, ultimately, on the lexical specifications of partic-
ular verbs, not the real-world properties of the designated events. For example, the ge-
neral notion of “eating” is associated with the verbs eat, devour, and dine, but these three
verbs exhibit different types of transitivity. The first verb, eat, is, technically speaking,
ambitransitive, since it can be either transitive, as in Bill ate the lasagna, or intransi-
tive, as in Bill ate. The second verb, devour, is strictly transitive, since one can say Bill
devoured the lasagna but not *Bill devoured. And the third verb, dine, is strictly intransi-
tive, since one can say Bill dined but not *Bill dined the lasagna. Despite these and other
complications, there is still a strong overall tendency for two-participant verbs to be
syntactically transitive and one-participant verbs to be syntactically intransitive.
Several neuropsychological studies have documented double dissociations between
these two types of verbs, especially in production tasks. Worse retrieval of transitive than
intransitive verbs appears to be the most common pattern (Cho-Reyes & Thompson,
2012; De Bleser & Kauschke, 2003; Kemmerer & Tranel, 2000; Kim & Thompson,
2000, 2004; Luzzatti et al., 2002; Thompson, Lange, Schneider, & Shapiro, 1997), but a
few patients with the opposite performance profile have also been reported (Jonkers &
Bastiaanse, 1998; Kemmerer & Tranel, 2000). As yet, very little is known about the lesion
correlates of these kinds of deficits, but a few fMRI studies have begun to investigate the
neural substrates of transitive and intransitive verbs in healthy subjects using a variety
of both production and comprehension tasks (den Ouden, Fix, Parrish, & Thompson,
2009; Hernandez, Fairhall, Lenci, Baroni, & Caramazza, 2014; Thompson et al., 2007;
see also Assadollahi, Meinzer, Flaisch, T., Obleser, & Rockstroh, 2009; Meltzer-Asscher,
Schuchard, den Ouden, & Thompson, 2013). The findings obtained so far, however,
are quite mixed. Some of the results suggest that transitive verbs depend more than in-
transitive verbs on the left temporoparietal cortex—in particular, the region extending
from the pMTG/pSTS into the angular gyrus—as well as on Broca’s area. Other results,
though, paint a more complicated picture, with each verb class evoking rather heter-
ogeneous patterns of activation. These inconsistencies may be related to the fact that
the overarching classes of transitive and intransitive verbs are by no means monolithic,
since each of them can be broken down into many fine-grained microclasses based on
correspondences between semantic content and grammatical behavior (Levin, 1993).
Hence it may be fruitful for future neurolinguistic research to take into account not only
the general transitive-intransitive distinction, but also these more specific contrasts
(Kemmerer, 2000a, 2003, 2014; Kemmerer & Wright, 2002; Kemmerer et al., 2008).
Beyond Nouns and Verbs
The multifarious morphological and syntactic constructions that comprise the gram-
matical systems of individual languages distinguish not only between (subclasses of)
nouns and verbs, but also between numerous other grammatical categories (Croft,
2000, forthcoming; Culicover, 1999; Haspelmath, 2007; Pullum & Scholz, 2007; Taylor,
2012). None of these categories has received as much attention in neurolinguistics as
nouns and verbs, but many of them have nevertheless been investigated in various ways.
In addition to nouns and verbs, most languages have two other open categories—
namely, adjectives and adverbs (although the former category is sometimes closed, and
the latter category is often ill-defined; see Dixon, 2010, Chapter 12). Adverbs have not yet
been carefully studied in neurolinguistics, but some aspects of adjectives have begun to
be explored. For example, when multiple adjectives are strung together before a noun,
their linear order must conform to certain constraints that are primarily semantic in
nature (Bache, 1978; Feist, 2011; Frawley, 1992; Quirk, Greenbaum, Leech, & Svartvik,
1985; Scontras, Degen, & Goodman, 2017). This is why it is fine to say the other small
inconspicuous carved jade idols but not *the carved other inconspicuous jade small idols.
Interestingly, there is some evidence that brain damage—especially, but not exclusively,
in Broca’s area and/or the left inferior parietal lobule—can impair a person’s knowledge
of the positional restrictions on prenominal adjectives while leaving intact their know-
ledge of the idiosyncratic meanings of such adjectives (Kemmerer, 2000b; Kemmerer,
Tranel, & Zdansczyk, 2009; see also Kemmerer, Weber-Fox, Price, Zdansczyk, & Way,
2007; Vandekerckhove, Sandra, & Daelemans, 2013). It is also noteworthy that in a re-
cent neuropsychological study by Miozzo, Rawlins, and Rapp (2014), two patients with
lesion overlap in the left IFG manifested thematic role confusions for comparative ad-
jectival constructions (e.g., The glove is darker than the hat) and spatial prepositional
786 David Kemmerer
constructions (e.g., The box is in the bag), but not for simple transitive constructions
(e.g., The woman helps the man). These patterns of association and dissociation suggest
that nonverbal word classes, particularly adjectives and prepositions, may recruit dif-
ferent brain mechanisms for thematic role assignment than verbal word classes, partic-
ularly transitive verbs.
Whereas open-class categories provide most of the conceptual content of utterances,
closed-class categories have relatively austere meanings and contribute more to the
structural organization of utterances (Talmy, 1988). Some English examples include
articles like a and the; demonstratives like this and that; auxiliary and modal verbs like
do, have, can, could, may, might, must, ought, should, would, and will; prepositions like
in, on, over, under, across, through, for, of, until, during, and since; and conjunctions
like and, or, but, because, therefore, moreover, and however. During receptive lan-
guage processing, closed-class words elicit different electrophysiological waveforms
than open-class words, even when the two types of lexical items are matched for fre-
quency and length (Pulvermüller, Lutzenberger, & Birbaumer, 1995). Moreover, within
the sphere of closed-class elements, different categories have distinct electrophysio-
logical signatures (Weber-Fox, Hart, & Spruill, 2006). As described in the following,
dissociations have also been found during language production tasks, not only be-
tween the two large-scale domains of open-and closed-class words, but also between
different types of closed-class words.
Most of the relevant data come from the syndrome of agrammatism (Menn,
O’Connor, Obler, & Holland, 1995; see Thompson & Mack, Chapter 31 in this volume).
When attempting to formulate sentences, patients with this disorder can produce a wide
range of open-class words, but they tend to have significant trouble generating closed-
class words, with the majority of errors involving omissions. In addition, comparisons
of the performance profiles of multiple patients have revealed a great deal of variability
regarding both the kinds of closed-class words that are impaired and the degree to
which they are affected. To take a rather striking example, consider preposition-particle
homonyms that differ as to whether they are semantically or syntactically determined.
In the sentence She ran up the stairs, the preposition up denotes a specific direction of
motion, but in the sentence She called up her friend, the particle up is just an obligatory
ingredient of the verb-particle expression call up. Not surprisingly, these two instances
of up have different grammatical behaviors. Although it would be fine to say Up the stairs
she ran, it would be very odd to say *Up her friend she called, and conversely, one could
never get away with *She ran the stairs up, but there is nothing wrong with She called her
friend up. Now, given such contrasts, it is especially interesting to note that semantically
determined prepositions and syntactically determined particles can be impaired inde-
pendently of each other by brain injury (Friederici, 1982; Friederici, Schönle, & Garrett,
1982; Kohen, Milsark, & Martin, 2011). This remarkable double dissociation suggests
that the two types of closed-class words have at least partially separate neural substrates.
Agrammatism is most often manifested by patients with stroke-induced Broca’s
aphasia, but the lesions in such patients are quite diverse, variably affecting all of the
major sectors of the large left perisylvian zone, with Broca’s area sometimes being
damaged and sometimes being spared (Vanier & Caplan, 1990). For these reasons, it
has been challenging to use deficit-lesion correlations in agrammatic Broca’s aphasics
as a source of data about the neural underpinnings of closed-class items. On the other
hand, agrammatism is also displayed by patients with progressive nonfluent aphasia,
and the fact that their atrophy is typically centered in the left ventrolateral prefrontal
territory provides some support for the view that Broca’s area and the surrounding re-
gions may actually be critically involved in the processing of closed-class items (Wilson
et al., 2010). Further evidence for this view comes from a few PET and fMRI studies
that have implicated Broca’s area in some of the grammatical operations that take place
during normal sentence production, including the generation of closed-class items
(Haller, Radue, Erb, Grodd, & Kircher, 2005; Indefrey et al., 2001; Indefrey, Hellwig,
Herzog, Seitz, & Hagoort, 2004). Much more research is needed, however, to elucidate
the many similarities and differences in how various categories of closed-class words are
implemented in the brain.
References
Abel, T. J., Rhone, A. E., Nourski, K. V., Kawasaki, H., Oya, H., Griffiths, T. D., Howard, M. A.,
& Tranel, D. (2015). Direct physiologic evidence of a heteromodal convergence region for
proper naming in human left anterior temporal lobe. Journal of Neuroscience, 35, 1513–1520.
Anderson, J. M. (1997). A notional theory of syntactic categories. Cambridge: Cambridge
University Press.
Assadollahi, R., Meinzer, M., Flaisch, T., Obleser, J., & Rockstroh, B. S. (2009). The representa-
tion of the verb’s argument structure as disclosed by fMRI. BMC Neuroscience, 10, 3.
Bache, C. (1978). The order of premodifying adjectives in present-day English. Odense: Odense
University Press.
Bak, T. H., & Hodges, J. R. (2004). The effects of motor neurone disease on language: Further
evidence. Brain and Language, 89, 354–361.
Barber, H. A., Kousta, S. T., Otten, L. J., & Vigliocco, G. (2010). Event-related potentials to
event-related words: Grammatical class and semantic attributes in the representation of
knowledge. Brain Research, 1332,  65–74.
Belfi, A. M., & Tranel, D. (2014). Impaired naming of famous musical melodies is associated
with left temporal polar damage. Neuropsychology, 28, 429–435.
Benetello, A., Finocchiaro, C., Capasso, R., Capitani, E., Laiacona, M., Magon, S., & Miceli, G.
(2016). The dissociability of lexical retrieval and morphosyntactic processes for nouns and
verbs: A functional and anatomoclinical study. Brain and Language, 159,  11–22.
Berlingeri, M., Crepaldi, D., Roberti, R., Scialfa, G., Luzzatti, C., & Paulesu, E. (2008). Nouns
and verbs in the brain: Grammatical class and task specific effects as revealed by fMRI.
Cognitive Neuropsychology, 25, 528–558.
Berndt, R. S., Haendiges, A. N., Burton, M. W., & Mitchum, C. C. (2002). Grammatical class
and imageability in aphasic word production: Their effects are independent. Journal of
Neurolinguistics, 15, 353–371.
Binder, J. R., & Desai, R. H. (2011). The neurobiology of semantic memory. Trends in Cognitive
Sciences, 15, 527–536.
788 David Kemmerer
Black, M., & Chiat, S. (2003). Noun-verb dissociations: A multifaceted phenomenon. Journal
of Neurolinguistics, 16, 231–250.
Capitani, E., Laiacona, M., Pagani, R., Capasso, R., Zampetti, P., & Miceli, G. (2009). Posterior
cerebral artery infarcts and semantic category dissociations: A study of 28 patients. Brain,
132, 965–981.
Cappa, S. F., Binetti, G., Pezzini, A., Padovani, A., Rozzini, L., & Trabucchi, M. (1998). Object
and action naming in Alzheimer’s disease and frontotemporal dementia. Neurology, 50,
351–355.
Cappa, S. F., & Perani, D. (2003). The neural correlates of noun and verb processing. Journal of
Cappelletti, M., Fregni, F., Shapiro, K., Pascual-Leone, A., & Caramazza, A. (2008). Processing
nouns and verbs in the left frontal cortex: A transcranial magnetic stimulation study. Journal
Caramazza., A., & Hillis, A. E. (1991). Lexical organization of nouns and verbs in the brain.
Nature, 349, 788–790.
Caramazza, A., & Miozzo, M. (1998). More is not always better: A response to Roelofs, Meyer,
and Levelt. Cognition, 69, 231–241.
Chan, A. M., Baker, J. M., Eskandar, E., Schomer, D., Ulbert, I., Marinkovic, K., Cash, S. S.,
& Halgren, E. (2011). First-pass selectivity for semantic categories in human anteroventral
temporal lobe. Journal of Neuroscience, 31, 18119–18129.
Cho-Reyes, S., & Thompson, C. K. (2012). Verb and sentence production and comprehension
in aphasia: Northwestern Assessment of Verbs and Sentences (NAVS). Aphasiology, 26,
1250–1277.
Cipolotti, L., MacNeil, J., & Warrington, E. K. (1993). Spared written naming of proper nouns: A
case report. Memory, 1, 289–311.
Clerget, E., Winderickx, A., Fadiga, L., & Olivier, E. (2009). Role of Broca’s area in encoding se-
quential human actions: A virtual lesion study. NeuroReport, 20, 1496–1499.
Collina, S., Marangolo, P., & Tabossi, P. (2001). The role of argument structure in the produc-
tion of nouns and verbs. Neuropsychologia, 39, 1125–1137.
Crepaldi, D., Aggujaro, S., Arduino, L. S., Zonca, G., Ghirardi, G., Inzaghi, M. G., Colombo,
M., Chierchia, G., & Luzzatti, C. (2006). Noun-verb dissociation in aphasia: The role of
imageability and functional locus of the lesion. Neuropsychologia, 44,  73–89.
Crepaldi, D., Berlingeri, M., Cattinelli, I., Borghese, N. A., Luzzatti, C., & Paulesu, E. (2013).
Clustering the lexicon in the brain: A meta-analysis of the neurofunctional evidence on
noun and verb processing. Frontiers in Human Neuroscience, 7, Article 303.
Crepaldi, D., Berlingeri, M., Paulesu, E., & Luzzatti, C. (2011). A place for nouns and a place
for verbs? A critical review of neurocognitive data on grammatical class effects. Brain and
Language, 116,  33–49.
Croft, W. (1991). Syntactic categories and grammatical relations: The cognitive organization of
information. Chicago: University of Chicago Press.
Croft, W. (2000). Parts of speech as typological universals and as language particular
categories. In P. Vogel & B. Comrie (Eds.), Approaches to the typology of word classes (pp.
65–102). Berlin: Mouton de Gruyter.
Croft, W. (2007). The origins of grammar in the verbalization of experience. Cognitive
Linguistics, 18, 339–382.
Croft, W. (forthcoming). Morphosyntax: Constructions of the world’s languages. Cambridge:
Culicover, P. W. (1999). Syntactic nuts: Hard cases, syntactic theory, and language acquisition.
Damasio, A. R., & Tranel, D. (1993). Nouns and verbs are retrieved with differently distributed
neural system. Proceedings of the National Academy of Sciences, 90, 4957–4960.
Damasio, H., Tranel, D., Grabowski, T. J., Adolphs, R., & Damasio, A. R. (2004). Neural sys-
tems behind word and concept retrieval. Cognition, 92, 179–229.
De Bleser, R., & Kauschke, C. (2003). Acquisition and loss of nouns and verbs: Parallel or diver-
gent patterns? Journal of Neurolinguistics, 16, 213–229.
Den Ouden, D. B., Fix, S., Parrish, T. B., & Thompson, C. K. (2009). Argument structure effects
in action verb naming in static and dynamic conditions. Journal of Neurolinguistics, 22,
196–215.
Dixon, R. M. W. (2010). Basic linguistic theory, Vol. 2: Grammatical topics. Oxford: Oxford
University Press.
Domahs, F., Nagels, A., Domahs, U., Whitney, C., Wiese, R., & Kircher, T. (2012). Where the
mass counts: Common cortical activation for different kinds of nonsingularity. Journal of
Druks, J. (2002). Verbs and nouns: A review of the literature. Journal of Neurolinguistics, 15,
289–315.
Fazio, P., Cantagallo, A., Craighero, L., D’Ausillo, A., Roy, A. C., Pozzo, T., Calzolari, F., Granieri,
E., & Fadiga, L. (2010). Encoding of human action in Broca’s area. Brain, 132, 1980–1988.
Feist, J. (2011). Premodifiers in English: Their structure and significance. Cambridge: Cambridge
University Press.
Fieder, N., Nickels, L., Biedermann, B., & Best, W. (2014a). From “some butter” to “a butter”: An
investigation of mass and count representation and processing. Cognitive Neuropsychology,
31, 313–349.
Fieder, N., Nickels, L., & Biedermann, B. (2014b). Representation of mass and count nouns: A
review. Frontiers in Psychology, 5, Article 589.
Finocchiaro, C., Basso, G., Giovenzana, A., & Caramazza, A. (2010). Morphological com-
plexity reveals verb- specific prefontal engagement. Journal of Neurolinguistics, 23,
553–563.
Finocchiaro, C., Fierro, B., Brighina, F., Giglia, G., Francolini, M., & Caramazza, A. (2008).
When nominal features are marked on verbs: A transcranial magnetic stimulation study.
Frawley, W. (1992). Linguistic semantics. Hillsdale, NJ: Lawrence Erlbaum.
Friederici, A. D. (1982). Syntactic and semantic processes in aphasic deficits: The availability of
prepositions. Brain and Language, 15, 249–258.
Friederici, A. D., Schönle, P., & Garrett, M. (1982). Syntactically and semantically based
computations: Processing of prepositions in agrammatism. Cortex, 19, 133–166.
Gainotti, G., Ferraccioli, M., Quaranta, D., & Marra, C. (2008). Cross-modal recognition
disorders for persons and other unique entities in a patient with right fronto-temporal de-
generation. Cortex, 44, 238–248.
Gainotti, G., Spinelli, P., Scaricamazza, E., & Marra, C. (2013). The evaluation of sources of
knowledge underlying different conceptual categories. Frontiers in Human Neuroscience, 7,
Article 40.
Goldenberg, G. (2009). Apraxia and the parietal lobes. Neuropsychologia, 47, 1449–14459.
Gorno-Tempini, M. L., & Price, C. J. (2001). Identification of famous faces and buildings: A
functional neuroimaging study of semantically unique items. Brain, 124, 2087–2097.
790 David Kemmerer
Grabowski, T. J., Damasio, H., Tranel, D., Boles-Ponto, L. L., Hichwa, R. D., & Damasio, A. R.
(2001). A role for the left temporal pole in the retrieval of words for unique entities. Human
Haller, S., Radue, E. W., Erb, M., Grodd, W., & Kircher, T. (2005). Overt sentence production in
event-related fMRI. Neuropsychologia, 43, 807–814.
Haspelmath, M. (2007). Pre-established categories don’t exist: Consequences for language de-
scription and typology. Linguistic Typology, 11, 119–132.
Haspelmath, M. (2012). How to compare major word-classes across the world’s languages. In T.
Graf, D. Paperno, A. Szabolcsi, & J. Tellings (Eds.), Theories of everything: In honor of Edward
Keenan (pp. 109–130). UCLA Working Papers in Linguistics, 17. Los Angeles: UCLA.
Hernandez, M., Fairhall, S.L., Lenci, A., Baroni, M., & Caramazza, A. (2014). Predication drives
verb cortical signatures. Journal of Cognitive Neuroscience, 26, 1829–1839.
Hillis, A. E., Heidler-Gray, J., Newhart, M., Chang, S., Ken, L., & Bak, T. H. (2006). Naming and
comprehension in primary progressive aphasia: The influence of grammatical word class.
Aphasiology, 20, 246–256.
Hillis, A. E., Tuffiash, E., & Caramazza, A. (2002). Modality-specific deterioration in naming verbs
in nonfluent primary progressive aphasia. Journal of Cognitive Neuroscience, 14, 1099–1108.
Hillis, A. E., Wityk, R. J., Barker, P. B., & Caramazza, A. (2003). Neural regions essential for
writing verbs. Nature Neuroscience, 6,  19–20.
Indefrey, P., Brown, C. M., Hellwig, F., Amunts, K., Herzog, H., Seitz, R. J., & Hagoort, P. (2001).
A neural correlate of syntactic encoding during speech production. Proceedings of the
Indefrey, P., Hellwig, F., Herzog, H., Seitz, R. J., & Hagoort, P. (2004). Neural responses to the
production and comprehension of syntax in identical utterances. Brain and Language, 89,
312–319.
Jonkers, R., & Bastiaanse, R. (1998). How selective are selective word class deficits? Two case
studies of action and object naming. Aphasiology, 12, 245–256.
Kalénine, S., Buxbaum, L. J., & Coslett, H. B. (2010). Critical brain regions for action recogni-
tion: Lesion symptom mapping in left hemisphere stroke. Brain, 133, 3269–3280.
Kemmerer, D. (2000a). Grammatically relevant and grammatically irrelevant features of verb
meaning can be independently impaired. Aphasiology, 14, 997–1020.
Kemmerer, D. (2000b). Selective impairment of knowledge underlying prenominal adjective
order: Evidence for the autonomy of grammatical semantics. Journal of Neurolinguistics,
13,  57–82.
Kemmerer, D. (2003). Why can you hit someone on the arm but not break someone on the arm?
A neuropsychological investigation of the English body-part possessor ascension construc-
tion. Journal of Neurolinguistics, 16,  13–36.
Kemmerer, D. (2014). Word classes in the brain: Implications of linguistic typology for cogni-
tive neuroscience. Cortex, 58,  27–51.
Kemmerer, D. (2015a). Are the motor features of verb meanings represented in the precentral
motor cortices? Yes, but within the context of a flexible, multilevel architecture for concep-
tual knowledge. Psychonomic Bulletin and Review, 22, 1068–1075.
Kemmerer, D. (2015b). The cognitive neuroscience of language: An introduction.
Kemmerer, D., Gonzalez Castillo, J., Talavage, T., Patterson, S., & Wiley, C. (2008).
Neuroanatomical distribution of five semantic components of verbs: Evidence from fMRI.
Brain and Language, 107,  16–43.
Kemmerer, D., Rudrauf, D., Manzel, K., & Tranel, D. (2012). Behavioral patterns and lesion
sites associated with impaired processing of lexical and conceptual knowledge of actions.
Cortex, 48, 826–848.
Kemmerer, D., & Tranel, D. (2000). Verb retrieval in brain-damaged subjects: 1. Analysis of
stimulus, lexical, and conceptual factors. Brain and Language, 73, 347–392.
Kemmerer, D., Tranel, D., & Manzel, K. (2005). An exaggerated effect for proper nouns
in a case of superior written over spoken word production. Cognitive Neuropsychology,
22,  3–27.
Kemmerer, D., Tranel, D., & Zdansczyk, C. (2009). Knowledge of the semantic constraints on
adjective order can be selectively impaired. Journal of Neurolinguistics, 22, 91–108.
Kemmerer, D., Weber-Fox, C., Price, K., Zdansczyk, C., & Way, H. (2007). Big brown dog or
brown big dog? An electrophysiological study of semantic constraints on prenominal adjec-
tive order. Brain and Language, 100, 238–256.
Kemmerer, D., & Wright, S. K. (2002). Selective impairment of knowledge underlying un-
prefixation: Further evidence for the autonomy of grammatical semantics. Journal of
Kielar, A., Milman, L., Bonakdarpour, B., & Thompson, C. K. (2011). Neural correlates of covert
and overt production of tense and agreement morphology: Evidence from fMRI. Journal of
Kim, M., & Thompson, C. K. (2000). Patterns of comprehension and production of nouns and
verbs in agrammatism: Implications for lexical organization. Brain and Language, 74,  1–25.
Kim, M., & Thompson, C. K. (2004). Verb deficits in Alzheimer’s disease and
agrammatism: Implications for lexical organization. Brain and Language, 88,  1–20.
Kohen, F., Milsark, G., & Martin, N. (2011). Effects of syntactic and semantic argument struc-
ture on sentence repetition in agrammatism: Things we can learn from particles and
prepositions. Aphasiology, 25, 736–747.
Kriegeskorte, N., Mur, M., Ruff, D. A., Kiani, R., Bodurka, J., Esteky, H., Tanaka, K., &
Bandettini, P. A. (2008). Matching categorical object representations in inferior temporal
cortex of man and monkey. Neuron, 60, 1126–1141.
Laiacona, M., & Caramazza, A. (2004). The noun/verb dissociation in language produc-
tion: Varieties of causes. Cognitive Neuropsychology, 21, 103–123.
Langacker, R. W. (1987). Nouns and verbs. Language, 63,  53–94.
Levin, B. (1993). English verb classes and alternations. Chicago: University of Chicago Press.
Luzzatti, C., Aggujaro, S., & Crepaldi, D. (2006). Verb- noun double dissociation in
aphasia: Theoretical and neuroanatomical foundations. Cortex, 42, 872–883.
Luzzatti, C., Raggi, R., Zonca, G., Pistarini, C., Contardi, A., & Pinna, G. D. (2002). Verb-
noun double dissociation in aphasic lexical impairments: The role of word frequency and
imageability. Brain and Language, 81, 432–444.
Mätzig, S., Druks, J., Masterson, J., & Vigliocco, G. (2009). Noun and verb differences in pic-
ture naming: Past studies and new evidence. Cortex, 45, 738–758.
Meltzer-Asscher, A., Schuchard, J., den Ouden, D. B., & Thompson, C. K. (2013). The neural
substrates of complex argument structure representations: Processing “alternating transi-
tivity” verbs. Language and Cognitive Processes, 28, 1154–1168.
Menn, L., O’Connor, M., Obler, L. K., & Holland, A. (1995). Nonfluent aphasia in a multilingual
world. Amsterdam: John Benjamins.
792 David Kemmerer
Miceli, G., Capasso, R., Daniele, A., Esposito, T., Magarelli, M., & Tomaiuolo, F. (2000).
Selective deficit for people’s names following left temporal damage: An impairment of
domain-specific conceptual knowledge. Cognitive Neuropsychology, 17, 489–516.
Miozzo, M., Rawlins, K., & Rapp, B. (2014). How verbs and non-verbal categories navigate
the syntax/semantics interface: Insights from cognitive neuropsychology. Cognition, 133,
621–640.
Molenberghs, P., Cunnington, R., & Mattingley, J. B. (2012). Brain regions with mirror
properties: A meta-analysis of 125 human fmri studies. Neuroscience and Biobehavioral
Reviews, 36, 341–349.
Moseley, R. L., & Pulvermüller, F. (2014). Nouns, verbs, objects, actions, and abstractions: Local
fMRI activity indexes semantics, not lexical categories. Brain and Language, 132,  28–42.
Newmeyer, F. J. (2007). Linguistic typology requires crosslinguistic formal categories.
Linguistic Typology, 11, 133–157.
Nordlinger, R., & Sadler, L. (2004). Nominal tense marking in crosslinguistic perspective.
Language, 80, 776–806.
O’Grady, W. (1997). Syntactic development. Chicago: University of Chicago Press.
Papagno, C., Casarotti, A., Comi, A., Pisoni, A., Lucchelli, F., Bizzi, A., Riva, M., & Bello, L.
(2016). Long-term proper name anomia after removal of the uncinate fasciculus. Brain
Papagno, C., Miracapillo, C., Casarotti, A., Romero Lauro, L. J., Castellano, A., Falini, A.,
Casaceli, G., Fava, E., & Bello, L. (2011). What is the role of the uncinate fasciculus? Surgical
removal and proper name retrieval. Brain, 134, 405–414.
Pillon, A., & d’Honincthun, P. (2010). The organization of the conceptual system: The case of
the “object versus action” dimension. Cognitive Neuropsychology, 27, 587–613.
Pullum, G. K., & Scholz, B. C. (2007). Systematicity and natural language syntax. Croatian
Journal of Philosophy, VII(21), 375–402.
Pulvermüller, F. (2013). How neurons make meaning: Brain mechanisms for embodied and
abstract-symbolic semantics. Trends in Cognitive Sciences, 17, 458–470.
Pulvermüller, F., Lutzenberger, W., & Birbaumer, N. (1995). Electrocortical distinction of vo-
cabulary types. Electroencephalography and Clinical Neurophysiology, 94, 357–370.
Pulvermüller, F., Lutzenberger, W., & Preissl, H. (1999a). Nouns and verbs in the intact
brain: Evidence from event- related potentials and high- frequency cortical responses.
Pulvermüller, F., Mohr, B., & Schleichert, H. (1999b). Semantic or lexico-syntactic factors: What
determines word-class-specific activity in the human brain? Neuroscience Letters, 275,  81–84.
Quirk, R., Greenbaum, S., Leech, G., & Svartvik, J. (1985). A comprehensive grammar of the
English language. London: Longman.
Rapp, B., & Caramazza, A. (1998). A case of selective difficulty in writing verbs. Neurocase, 4,
127–140.
Rapp, B., & Caramazza, A. (2002). Selective difficulties with spoken nouns and written verbs: A
single case study. Journal of Neurolinguistics, 15, 373–402.
Rapp, B., & Goldrick, M. (2006). Speaking words: Contributions of cognitive neuropsycholog-
ical research. Cognitive Neuropsychology, 23,  39–73.
Rauh, G. (2010). Syntactic categories. Oxford: Oxford University Press.
Ross, L. A., McCoy, D., Coslett, H. B., Olson, I. R., & Wolk, D. A. (2011). Improved proper name
recall in aging after electrical stimulation of the anterior temporal lobes. Frontiers in Aging
Neuroscience, 3, Article 16.
Sahin, N. T., Pinker, S., Cash, S. S., & Halgren, E. (2009). Sequential processing of lexical, gram-
matical, and phonological information within Broca’s area. Science, 326, 445–449.
Sahin, N. T., Pinker, S., & Halgren, E. (2006). Abstract grammatical processing of nouns and
verbs in Broca’s area: Evidence from fMRI. Cortex, 42, 540–562.
Scontras, G., Degen, J., & Goodman, N. D. (2017). Subjectivity predicts adjective ordering
preferences. Open Mind: Discoveries in Cognitive Science, 1,  53–65.
Semenza, C. (2006). Retrieval pathways for common and proper names. Cortex, 42, 884–891.
Semenza, C. (2009). The neuropsychology of proper names. Mind and Language, 24, 347–369.
Semenza, C. (2011). Naming with proper names: The left temporal pole theory. Behavioural
Neurology, 24, 277–284.
Shapiro, K. A., & Caramazza, A. (2003a). Grammatical processing of nouns and verbs in left
frontal cortex? Neuropsychologia, 41, 1189–1198.
Shapiro, K. A., & Caramazza, A. (2003b). The representation of grammatical categories in the
brain. Trends in Cognitive Sciences, 7, 201–206.
Shapiro, K. A., & Caramazza, A. (2004). The organization of lexical knowledge in the brain: The
grammatical dimension. In M. S. Gazzaniga (Ed.), The cognitive neurosciences (3rd ed., pp.
803–814). Cambridge, MA: MIT Press.
Shapiro, K. A., & Caramazza, A. (2009). Morphological processes in language production.
In M. S. Gazzaniga (Ed.), The cognitive neurosciences (4th ed., pp. 777–788). Cambridge,
MA: MIT Press.
Shapiro, K. A., Moo, L. R., & Caramazza, A. (2006). Cortical signatures of noun and verb pro-
duction. Proceedings of the National Academy of Sciences, 103, 1644–1649.
Shapiro, K. A., Moo, L. R., & Caramazza, A. (2012). Neural specificity for grammatical op-
erations is revealed by content-independent fMR adaptation. Frontiers in Psychology, 3,
Article 26.
Shapiro, K. A., Pascual-Leone, A., Mottaghi, F. M., Gangitano, M., & Caramazza, A. (2001).
Grammatical distinctions in the left frontal cortex. Journal of Cognitive Neuroscience, 13,
713–720.
Shapiro, K. A., Shelton, J., & Caramazza, A. (2000). Grammatical class in lexical produc-
tion and morphological processing: Evidence from a case of fluent aphasia. Cognitive
Shtyrov, Y., Butorina, A., Nikolaeva, A., & Stroganova, T. (2014). Automatic ultrarapid activa-
tion and inhibition of cortical motor systems in spoken word comprehension. Proceedings of
the National Academy of Sciences, 111, E1918–1923.
Silveri, M. C., & Ciccarelli, N. (2007). The deficit for the word-class “verb” in corticobasal
degeneration: Linguistic expression of the movement disorder? Neuropsychologia, 45,
2570–2579.
Siri, S., Tettamanti, M., Cappa, S. F., Della Rosa, P., Saccuman, C., Scifo, P., & Vigliocco, G.
(2008). The neural substrate of naming events: Effects of processing demands but not of
grammatical class. Cerebral Cortex, 18, 171–177.
Talmy, L. (1988). The relation of grammar to cognition. In B. Rudzka-Ostyn (Ed.), Topics in cog-
nitive linguistics (pp. 165–206). Amsterdam: John Benjamins.
Taylor, J. R. (2012). The mental corpus: How language is represented in the mind. Oxford: Oxford
University Press.
Thompson, C. K., Bonakdarpour, B., Fix, S. C., Blumenfeld, H. K., Parrish, T. B., Gitelman,
D. R., & Mesulam, M. M. (2007). Neural correlates of verb argument structure processing.
794 David Kemmerer
Thompson, C. K., Lange, K., Schneider, S., & Shapiro, L. (1997). Agrammatic and non-brain-
damaged subjects’ verb and verb argument structure production. Aphasiology, 11, 473–490.
Tobassi, P., Collina, S., Pizzioli, F., & Basso, A. (2010). Speaking of events: The case of C.M.
Cognitive Neuropsychology, 27, 152–180.
Tranel, D. (2006). Impaired naming of unique landmarks is associated with left temporal polar
damage. Neuropsychology, 20,  1–10.
Tranel, D. (2009). The left temporal pole is important for retrieving words for unique concrete
entities. Aphasiology, 23, 867–884.
Tsapkini, K., Jarema, G., & Kehayia, E. (2002). A morphological processing deficit in verbs
but not nouns: A case study in a highly inflected language. Journal of Neurolinguistics, 15,
265–288.
Tsigka, S., Papadelis, C., Braun, C., & Miceli, G. (2014). Distinguishable neural correlates of
verbs and nouns: A MEG study. Neuropsychologia, 54,  87–97.
Vandekerckhove, B., Sandra, D., & Daelemans, W. (2013). Selective impairment of adjective
order constraints as overeager abstraction: An elaboration on Kemmerer et al. (2009).
Journal of Neurolinguistics, 26,  46–72.
Vanier, M., & Caplan, D. (1990). CT-scan correlates of agrammatism. In L. Menn & L. K. Obler
(Eds.), Agrammatic aphasia: A cross-language narrative sourcebook (Vol. 1, pp. 37–115).
Amsterdam: John Benjamins.
Van Valin, R. D., & LaPolla, R. J. (1997). Syntax. Cambridge: Cambridge University Press.
Vigliocco, G., Vinson, D. P., Druks, J., Barber, H., & Cappa, S. F. (2011). Nouns and verbs in
the brain: A review of behavioural, electrophysiological, neuropsychological, and imaging
studies. Neuroscience and Biobehavioral Reviews, 35, 407–426.
Vigliocco, G., Warren, J., Siri, S., Arciuli, J., Scott, S., & Wise, R. (2006). The role of semantics
and grammatical class in the neural representation of words. Cerebral Cortex, 16, 1790–1796.
Waldron, E. J., Manzel, K., & Tranel, D. (2014). The left temporal pole is a heteromodal hub for
retrieving proper names. Frontiers in Bioscience, S6,  50–57.
Wang, X., Peelen, M. V., Han, Z., Caramazza, A., & Bi, Y. (2016). The role of vision in the neural
representation of unique entities. Neuropsychologia, 87, 144–156.
Watson, C. E., Cardillo, E. R., Ianni, G. R., & Chatterjee, A. (2013). Action concepts in the
brain: An activation-likelihood estimation meta-analysis. Journal of Cognitive Neuroscience,
25, 1191–1205.
Weber-Fox, C., Hart, L. J., & Spruill, J. E. (2006). Effects of grammatical categories on children’s
visual language processing: Evidence from event- related brain potentials. Brain and
Language, 98,  26–39.
Wierzbicka, A. (2000). Lexical prototypes as a universal basis for crosslinguistic identification
of “parts of speech.” In P. M. Vogel & B. Comrie (Eds.), Approaches to the typology of word
classes (pp. 285–317). Berlin: Mouton de Gruyter.
Willms, J. L., Shapiro, K. A., Peelen, M. V., Pajtas, P. E., Costa, A., Moo, L. R., & Caramazza,
A. (2011). Language- invariant verb processing regions in Spanish- English bilinguals.
Wilson, S. M., Henry, M. L., Besbris, M., Ogarm, J. M., Dronkers, N. F., Jarrold, W., Miller, B. L.,
& Gorno-Tempini, M. L. (2010). Connected speech production in three variants of primary
progressive aphasia. Brain, 133, 2069–2088.
Wu, D. H., Waller, S., & Chatterjee, A. (2007). The functional neuroanatomy of thematic and
locative relational knowledge. Journal of Cognitive Neuroscience, 19, 1542–1555.
Yudes, C., Domínguez, A., Cuetos, F., & de Vega, M. (2016). The time-course of processing
grammatical class and semantic attributes of words: Dissociation by means of ERP.
Psicológica, 37, 105–126.
Chapter 31
Neuro c o g ni t i v e
Mechani sms of
Agram mat i sm
Cynthia K. Thompson and Jennifer E. Mack
Introduction
Agrammatism is a language disorder resulting from damage to the (usually) left hem-
isphere language network. The primary deficit seen in individuals with agrammatism
is difficulty producing grammatical sentences. Individuals with agrammatism produce
short, grammatically impoverished sentences or sentence fragments, consisting of pri-
marily open class, content words with errors in grammatical morphology, for example,
omission and/or substitution of verb inflection (Faroqi-Shah & Thompson, 2003;
Saffran, Schwartz, & Marin, 1980; Thompson, Cho, et al., 2012). Consider the following
narrative language sample from a 35-year-old male (L.B.S.) with chronic stroke-induced
agrammatic aphasia, telling the story of Cinderella:
She’s nice. Dusting (3 sec). Wash (3 sec). The old man (4 sec). Wicked uh sisters uh
two three five uh. The horse (5 sec) dog uh bird (3 sec) mouse uh uh is (3 sec) cut (11
sec) dress (2 sec). Be (4 sec) alright. Pumpkin. Pull over uh uh. The uh prince (2 sec).
The love uh. We dance (2 sec). Oh no uh noon. How uh get out uh uh? Shoe uh glass
shoe (5 sec) found its uh uh. Pulling (3 sec) shoe uh. No ties. Yes. The end.
Linguistic analysis of L.B.S.’s speech using a method developed by Thompson and

colleagues (see Thompson, Cho, et al., 2012; Thompson, Shapiro, Li, and Schendel,
1995) revealed a greatly reduced speech rate (i.e., 14 words per minute; mean for
healthy age-matched controls = 132.22; standard error of mean [SEM] = 5.22; data from
Thompson, Cho, et al., 2012) and utterance length (mean = 2.37 words; mean for healthy
controls = 11.11 [SEM = 0.56]). Only 10% of the patient’s sentences were syntactically
Neurocognitive Mechanisms of Agrammatism 797
correct (healthy control mean = 93% [SEM = 1%]), with a complex-to-simple sen-
tence ratio of .10. He also produced more open-class compared to closed-class words
(open:closed-class ratio = 2.41; healthy control mean = 0.95 [SEM = 0.03]), with omis-
sion of matrix verbs and marked difficulty producing verb inflections (0% correct) (with
noun inflection at 100% correct).
These patterns have been borne out in many experimental studies of language produc-
tion in agrammatic aphasia caused by stroke. In constrained tasks, agrammatic speakers
often show ability to produce simple active, canonical sentence structures; however, they
show particular difficulty producing sentences with noncanonical word order (Caplan
& Hanna, 1998; Cho-Reyes & Thompson, 2012; Faroqi-Shah & Thompson, 2003). Using
the Northwestern Assessment of Verbs and Sentences (NAVS; Thompson, 2011; also see
Cho-Reyes and Thompson, 2012), which tests production of both canonical forms (i.e.,
actives, subject-wh questions, and subject-relative structures) and noncanonical forms
(passives, object-wh questions, and object-relative constructions) using a sentence pro-
duction priming task, we (Cho-Reyes & Thompson, 2012) documented this pattern in
35 patients with agrammatic aphasia with mild to moderately severe deficits. Mean pro-
duction of canonical forms was 72% accurate, compared to a mean of 39% accuracy for
noncanonical forms. Studies explicitly examining verb production in agrammatism
also show deficits in both verb-naming and sentence-production tasks across lan-
guages, including English (Kim & Thompson, 2000; Thompson, Lange, Schneider,
& Shapiro, 1997; Thompson, Lukic, King, Mesulam, & Weintraub, 2012; Zingeser &
Berndt, 1990), German (De Bleser & Kauschke, 2003), Dutch (Bastiaanse, Hugen, Kos,
& van Zonneveld, 2002), Hungarian (Kiss, 2000), Italian (Luzzatti et al., 2002), Russian
(Dragoy & Bastiaanse, 2010), Korean (Sung, 2016), and Chinese (Bates, Chen, Tzeng, Li,
& Opie, 1991).
Other work has shown that verb inflection is particularly vulnerable in agrammatism.
Verb inflections such as subject-verb agreement (e.g., third-person singular present
tense -s in English) and tense markings (e.g., English past tense -ed) are missing or in-
correctly substituted in agrammatic speech (Arabatzi & Edwards, 2002; Bastiaanse et al.,
2011; Burchert, Swoboda-Moll, & Bleser, 2005; Dickey, Milman, & Thompson, 2008;
Druks, 2006; Faroqi-Shah & Thompson, 2007; Friedmann & Grodzinsky, 1997; J. Lee,
Milman, & Thompson, 2008; Wenzlaff & Clahsen, 2004). In a study testing production
of grammatical morphology (using the Northwestern Assessment of Verb Inflection;
J. Lee & Thompson, 2017), participants with agrammatism produced fewer correct fi-
nite (tensed) verbs (i.e., 53% correct) compared to a group of participants with anomic
aphasia (75% correct; Thompson, Meltzer-Asscher, et al., 2013). Similarly, function words
such as auxiliaries (e.g., be, have, do) and complementizers (e.g., if, that, whether) are
commonly absent (Arabatzi & Edwards, 2002; Friedmann & Grodzinsky, 1997; Milman,
Dickey, & Thompson, 2008). Further, nominal functional morphemes such as posses-
sive -s and the definite determiner the can be impaired (Wang, Yoshida, & Thompson,
2014). These patterns have been observed in English, as well as in typologically diverse
languages such as German, Hebrew, and Japanese, and across tasks, including sponta-
neous speech, sentence completion, grammaticality judgment, repetition, and naming
798 Cynthia K. Thompson and Jennifer E. Mack
(Burchert et al., 2005; Dickey et al., 2008; Friedmann, 2002; Friedmann & Grodzinsky,
1997; Hagiwara, 1995; Milman et al., 2008; Thompson, Meltzer-Asscher, et al., 2013;
Wenzlaff & Clahsen, 2004).
Individuals with stroke- induced agrammatic aphasia also evince difficulty
comprehending sentences, with particular difficulty understanding complex,
noncanonical structures and semantically reversible forms (Caplan, Hildebrandt, &
Makris, 1996; Caramazza & Zurif, 1976; Cho-Reyes & Thompson, 2012; Grodzinsky,
2000; Schwartz, Saffran, & Marin, 1980; and many others), where interpretation of “who
did what to whom” cannot be derived from semantic or real-world knowledge alone.
In one of the first studies of sentence comprehension in aphasia, Caramazza and Zurif
(1976) showed that individuals with Broca’s aphasia (and agrammatism) had problems
understanding noncanonical, center-embedded sentences such as the girl that the boy is
pushing is blond.
The most common cause of agrammatic aphasia is stroke within the distribution of
the middle cerebral artery. Associated with Broca’s aphasia, damage to anterior brain
tissue (i.e., the left inferior frontal gyrus [IFG], Brodmann’s areas 44 and 45, and ad-
jacent tissue) is common, although research shows that lesions often extend well be-
yond this region and affect both gray matter and white matter (Cappa, 2012; Catani &
Mesulam, 2008; Kertesz, Lesk, & McCabe, 1977; Lukic, Bonakdarpour, den Ouden,
Price, & Thompson, 2014; Vanier & Caplan, 1990). Kertesz (1977), using radionucleide
to examine lesions in 14 patients presenting with “low fluency associated with rela-
tively well-preserved comprehension” (p. 594), showed that although the common area
of infarction involved the IFG (in all patients except one), lesions extended to adjacent
frontal as well as temporoparietal tissue. Data from our lab support this pattern. For ex-
ample, in a recent study including 14 individuals with stroke-induced agrammatism, all
participants had lesions in left hemisphere perisylvian regions, including inferior frontal
and temporoparietal regions (see Figure 31.1). In addition, recent lesion-symptom map-
ping studies also implicate both anterior and posterior brain tissue in sentence com-
prehension and production impairments. For example, in a lesion-symptom mapping
study that included 34 patients with stroke-induced aphasia, we (Lukic et al., 2014) found
that lesions in anterior (IFG, insula) and posterior perisylvian regions (superior tem-
poral gyrus [STG], angular gyrus [AG], and supramarginal gyrus [SMG]), as well as
the arcuate fasciculus (a dorsal white matter tract), significantly predicted agrammatic
production deficit patterns (i.e., impaired verb naming, verb-argument structure pro-
duction, and production of complex sentences).
Agrammatism is also caused by neurodegenerative disease (i.e., primary progres-
sive aphasia [PPA]; see Wilson, Chapter 2 in this volume). One variant of PPA is as-
sociated with grammatical deficits (i.e., the agrammatic variant [PPA-G] or nonfluent/
agrammatic variant [naPPA]) (Gorno- Tempini et al., 2011; Mesulam, Wieneke,
Thompson, Rogalski, & Weintraub, 2012; Thompson & Mack, 2014; Wilson, Galantucci,
Tartaglia, & Gorno-Tempini, 2012). We (Mesulam, Thompson, & colleagues; e.g.,
Mesulam et al., 2009; Mesulam et al., 2012; Thompson, Cho, et al., 2012; Thompson,
Meltzer-Asscher, et al., 2013) use the term PPA-G to refer to patients with grammatical
Figure 31.1. Lesion locations in 14 individuals with agrammatism enrolled in an ongoing study

of the neural correlates of language recovery (Thompson lab, in progress). The color bar indicates
the number of participants whose lesions overlap at each voxel. Peak areas of overlap (red) in-
clude left hemisphere anterior and posterior perisylvian regions, including the inferior frontal
gyrus, superior temporal gyrus, and inferior parietal lobule.
impairments, rather than naPPA, since the latter requires one of two core features, either
of which is sufficient for classification: (1) agrammatism in language production, and/or
(2) motor speech deficits. Importantly, however, not all PPA patients with grammatical
impairments present with motor speech difficulty, and patients with “pure” progressive
motor speech deficits have been reported who do not evince grammatical impairments
(Josephs et al., 2006; Mesulam et al., 2012; Rohrer, Rossor, & Warren, 2010; Wicklund
et al., 2014; see Thompson & Mack, 2014, for review). Further, although nonfluent
speech production is seen in patients with PPA-G, other variants of PPA also may show
nonfluent speech (i.e., logopenic PPA (PPA-L), characterized by impaired word-finding
and repetition with intact grammatical abilities), and some evince dissociations be-
tween fluency and grammatical ability (Thompson, Cho, et al., 2012). Furthermore,
evidence is mixed regarding the degree of overlap between the neural substrates of flu-
ency and grammatical production in PPA (Catani et al., 2013; Gunawardena et al., 2010;
Mandelli et al., 2014; Rogalski et al., 2011; Wilson, Henry, et al., 2010).
Like stroke-induced agrammatism, patients with PPA-G show relatively spared
single-word comprehension in the face of impaired grammatical ability in produc-
tion and complex sentence comprehension impairments (Thompson, Cho, et al., 2012;
Thompson & Mack, 2014; Thompson, Meltzer-Asscher, et al., 2013; Wilson et al., 2012).
Quantitative analyses of speech samples have typically found more grammatical errors
in PPA-G than in healthy age-matched controls or other variants of PPA (e.g., PPA-L,
semantic PPA [PPA-S]) (Knibb, Woollams, Hodges, & Patterson, 2009; Thompson,
Cho, et al., 2012; Thompson, Meltzer-Asscher, et al., 2013; Wilson, Henry, et al., 2010).
Speakers with PPA-G produce fewer syntactically complex utterances (Ash et al., 2009;
Gunawardena et al., 2010; Knibb et al., 2009; Wilson, Henry, et al., 2010) than do cog-
nitively healthy controls. Some studies also show a trend toward higher open:closed
(O:C) class ratios, with production of fewer verbs compared to nouns (Ash et al., 2009;
Thompson, Ballard, Tait, Weintraub, & Mesulam, 1997; Thompson, Cho, et al., 2012;
Thompson, Meltzer-Asscher, et al., 2013; Wilson, Henry, et al., 2010) in PPA-G compared
to patients with other PPA subtypes. Finally, verb inflection errors (e.g., tense and agree-
ment) in spontaneous language production are prevalent in PPA-G (Thompson, Cho,
et al., 2012; Thompson, Meltzer-Asscher, et al., 2013; Wilson, Henry, et al., 2010; also see
Graham, Patterson, & Hodges, 2004).
In constrained production tasks, speakers with PPA-G also exhibit grammatical pro-
duction impairments, particularly for syntactically complex, noncanonical sentences,
whereas individuals with PPA-L and PPA-S show relatively unimpaired performance
(Cupit et al., 2016; DeLeon et al., 2012; Thompson, Meltzer-Asscher, et al., 2013). We
(Thompson, Meltzer-Asscher, et al., 2013) found that while speakers with PPA-G and
PPA-L produce canonical sentences (e.g., active sentences, subject wh-questions, and
subject-relative clauses) with comparable accuracy, accuracy is significantly lower
in PPA-G than in PPA-L for noncanonical forms (e.g., passive sentences, object wh-
questions, and object-relative clauses). In addition, verb-production deficits have been
quantified in picture-naming tasks. Individuals with PPA-G exhibit more significant
impairment in verb (action) naming than in noun (object) naming (Hillis et al., 2006;
Hillis, Oh, & Ken, 2004; Hillis, Tuffiash, & Caramazza, 2002; Thompson, Lukic, et al.,
2012). However, verb comprehension, like noun comprehension, is largely preserved
in PPA-G (Hillis et al., 2006; Thompson, Lukic, et al., 2012). We (Thompson, Meltzer-
Asscher, et al., 2013) also found, in a study testing verb inflection using the Northwestern
Assessment of Verb Inflection (J. Lee & Thompson, 2017), that PPA-G speakers show
greater difficulty producing finite compared to nonfinite verb inflection, suggesting a
morphosyntactic locus of verb inflection deficits in this population. Impaired morpho-
phonological processing also has been argued to contribute to verb inflection deficits
in PPA-G (Wilson, Brandt, et al., 2014). Further, like agrammatism caused by stroke,
PPA-G is associated with deficits in comprehension of syntactically complex sentences
(Cooke et al., 2003; Gorno-Tempini et al., 2011; Thompson, Meltzer-Asscher, et al., 2013;
Wilson, Dronkers, et al., 2010).
Studies measuring cortical thickness in people with PPA-G (compared to cogni-
tively healthy people) show peak atrophy in the left hemisphere IFG, although atrophy
is also observed in left hemisphere premotor, dorsolateral prefrontal, and posterior
temporoparietal regions in both early and later stages of disease (Mesulam et al., 2009;
Mesulam et al., 2012) (see Figure 31.2). Atrophy in the left IFG has been associated with
a range of features of agrammatism, including impaired complex sentence comprehen-
sion (Amici et al., 2007; Peelle et al., 2008; Wilson et al., 2011) and production (DeLeon
et al., 2012; Gunawardena et al., 2010; Rogalski et al., 2011; Wilson et al., 2011; Wilson,
Henry, et al., 2010). In addition, agrammatism in PPA is associated with damage to
Figure 31.2. Sites of cortical atrophy in agrammatic PPA (PPA-G). Orange and yellow areas
showed significant atrophy as compared to age-matched controls. Areas of peak atrophy (yellow)
in the left hemisphere include the inferior frontal gyrus (IFG), the temporoparietal junction, and
premotor and dorsolateral prefrontal regions.
Source: Mesulam et al. (2009).
white matter tracts in the left hemisphere (e.g., anterior, posterior and long segments of
the arcuate) (Catani et al., 2017; Wilson et al., 2011), as in agrammatism caused by stroke.
This chapter is focused on sentence production and comprehension deficits asso-
ciated with agrammatism, with emphasis on the neurocognitive mechanisms under-
lying these impairments. Grammatical sentence processing engages lexical as well as
morphosyntactic processes. Within the lexical domain, verb processing is particularly
germane, since verbs encode syntactic information (i.e., subcategorization frames) as
well as verb-argument or event structure (i.e., participant roles entailed within the
verb’s representation; see Kemmerer, Chapter 30 in this volume). Thus, verb-argument
structure is considered an interface between semantics (e.g., who did what to whom)
and syntax (e.g., word order) and is essential for production and comprehension of
simple monoclausal sentences, as well as complex syntactic structures, which con-
tain embedded clauses and/or noncanonical argument order (e.g., passive sentences
such as The girl was kissed by the boy, in which the Theme argument precedes the
Agent). Following many theoretical linguistic models, we assume a “lexicalist” ap-
proach in which verb-argument structure guides phrase structure building, which,
in turn, mediates sentence processing (Caramazza, 1997; Chomsky, 1981; Horvath
& Siloni, 2011; Jackendoff, 1972; Levelt, Roelofs, & Meyer, 1999; Levin & Rappaport,
1986; Reinhart, 2002; Williams, 1981; but see Goldberg, 1995, 2003, for discussion of
the “constructivist” approach, which argues that verb-argument structure is strongly
tied to semantics). The next section of this chapter discusses verb and verb-argument
structure impairments seen in agrammatic aphasia as they relate to sentence compre-
hension and production abilities. This is followed by discussion of studies examining
complex sentence processing in agrammatism. Then, we address neurocognitive
mechanisms associated with verb and sentence impairments. Throughout, based on
the extant literature, we discuss theories of agrammatism with a focus on impaired
thematic integration (i.e., the processes that support the licit combination of a verb’s
thematic roles and its constituent arguments to yield the intended meaning of a
sentence).
Verb and Verb-Argument Structure

Deficit Patterns
Verbs are used to describe different types of events, with different numbers and types
of core participants (arguments). For example, some verbs are predominantly used to
describe events with one core participant (e.g., swim), whereas others describe events
with two (lift) or three (give) core participants. Further, the meanings of verbs select for
different sets of thematic roles, encoding the roles that participants play in the event.
For example, events described by lift typically encode an Agent and Theme (e.g., John
lifted the box), whereas events described by amuse typically encode a Theme and an
Experiencer (e.g., The show amused John). On many current accounts, these thematic
role labels are a convenient stand-in for a more elaborated lexical-conceptual structure
that represents the relationships between event participants (e.g., Levin & Rappaport
Hovav, 2005). Verbs also differ with respect to the constraints they place on the syntactic
realization of their arguments (subcategorization frames). For example, discover allows
both nominal and sentential complements (They discovered the new drug; They discov-
ered that the new drug was effective) whereas produce only allows nominal complements
(They produced the new drug). We refer to the semantic and syntactic requirements of
verbs as verb-argument structure.
Many studies suggest that the representation of verb meaning and verb-argument
structure is intact in agrammatism. The evidence for this view has come from a range
of experimental paradigms, including single-word comprehension, cross-modal lexical
decision, grammaticality judgments, and structural priming. Agrammatic individuals’
single-verb comprehension is often preserved (but see Miceli, Silveri, Nocentini, &
Caramazza, 1988), with no deficits in comprehending verbs with complex argument
structures (Kim & Thompson, 2000, 2004; M. Lee & Thompson, 2004; Piñango, 2000;
Thompson, Lukic, et al., 2012). Agrammatic speakers also show normal access to sub-
categorization frames during online sentence processing. As in non-brain-damaged
volunteers, reaction times (RTs) are longer for verbs with multiple subcategorization
options versus those with only one such option, reflecting exhaustive online access to
argument structure representations (Shapiro, Gordon, Hack, & Killackey, 1993; Shapiro
& Levine, 1990). Access to information about the relative frequencies of different sub-
categorization frames also appears to be intact (DeDe, 2013a, 2013b). Individuals with
agrammatism also evince relatively preserved ability to distinguish sentences with cor-
rect argument structure (e.g., The boy is carrying the girl; The dog is barking) from argu-
ment structure violations, such as argument omission (e.g., *The boy is carrying) and
argument insertion (e.g., * The dog is barking the girl) (Grodzinsky & Finkel, 1998; Kim
& Thompson, 2000, 2004; though see Kielar, Meltzer-Asscher, & Thompson, 2012),
indicating intact access to sentence-level argument-structure representations. They also
show intact priming of the order of thematic roles in sentence production, supporting
the view that argument structure representations are intact (Cho-Reyes, Mack, &
Thompson, 2016).
However, despite apparently intact argument structure representations, some aspects
of the online processing of verbs and verb-argument structure are impaired. Individuals
with agrammatism caused by stroke have demonstrated impaired online responses
to argument-structure violations (e.g., *John sneezed the doctor), including abnormal
event-related potential (ERP; see Leckey & Federmeier, Chapter 3 in this volume) sig-
nals (Kielar et al., 2012), and reaction times (Myers & Blumstein, 2005). In contrast
with healthy adults, they also show deficits in using the verb’s selectional restrictions
(e.g., that eat requires an edible direct object, but move does not) to predict and inte-
grate arguments (Mack, Ji, & Thompson, 2013; cf. Dickey & Warren, 2015, who report
impaired online verb-argument integration in a group of four individuals with stroke-
induced aphasia, two of whom have Broca’s aphasia). In PPA-G, two studies have re-
vealed impaired online processing of verb-argument structure violations, one using
ERP (Barbieri et al., 2016) and one using reaction-time measures (Peelle, Cooke, Moore,
Vesely, & Grossman, 2007); however, one study found intact online processing of these
violations (Price & Grossman, 2005). Further, our preliminary research shows impaired
verb-argument prediction in listeners with PPA-G (Mack, Mesulam, & Thompson,
2017). Overall, the picture that emerges is that agrammatic individuals’ representation
of verb meaning and verb-argument structure seems to be spared, but thematic (i.e.,
verb-argument) integration is impaired, as shown by abnormal online responses to
violations and impaired prediction.
Verb retrieval and verb-argument structure processing is, however, impaired in
production. Speakers with agrammatism caused by stroke and PPA produce fre-
quent verb-argument structure errors in structured tasks and in spontaneous lan-
guage production (Thompson, Ballard, et al., 1997; Thompson, Cho, et al., 2012;
Thompson, Lange, et al., 1997; Thompson et al., 1995). Several studies have shown
that argument structure complexity affects verb retrieval and sentence-production
accuracy. Notably, the literature on argument-structure complexity in agrammatism
encompasses typologically diverse languages including English (Thompson, 2003),
Dutch (Bastiaanse & van Zonneveld, 2005), German (De Bleser & Kauschke, 2003),
Greek (Stavrakaki, Alexiadou, Kambanaros, Bostantjopolou, & Katsarou, 2011),
Hungarian (Kiss, 2000), Italian (Luzzatti et al., 2002), Russian (Dragoy & Bastiaanse,
2010), and Korean (Sung, 2016).
Argument number is one relevant factor that shapes verb production in
agrammatism; for example, two-and three-argument verbs (e.g., lift, give) are named
less accurately than one-argument verbs (e.g., swim) (Kim & Thompson, 2000, 2004;
Kiss, 2000; Luzzatti et al., 2002; Sung, 2016; Thompson, Lange, et al., 1997; Thompson,
Lukic, et al., 2012). These effects are shown in both stroke-induced agrammatism and
PPA-G (Thompson, Lukic, et al., 2012). The number of arguments selected by the verb
also affects both structured sentence production tasks and narrative production: two-
and three-argument sentences are produced less frequently and/or accurately than
in unimpaired speakers (Bastiaanse & Jonkers, 1998; De Bleser & Kauschke, 2003;
Dragoy & Bastiaanse, 2010; Thompson, Lange, et al., 1997). Related to this, a recent
study examined the verb’s number of subcategorization frames (i.e., complement types)
as a potential factor affecting aphasic sentence production in narratives (Malyutina,
Richardson, & den Ouden, 2016). In this study, speakers with Broca’s aphasia produced
verbs with fewer arguments and less diverse subcategorization frames, consistent with
the idea that thematic integration is impaired in production.
In addition, agrammatism is associated with deficits in producing unaccusative
verbs, that is, one-argument verbs in which the grammatical subject is assigned the
Theme role (e.g., The man fell; first described by Perlmutter, 1978), in contrast with
unergative verbs, which have an Agent subject (e.g., The man swam). These deficits
have been observed in verb naming (Luzzatti et al., 2002; Sung, 2016; Thompson,
2003) and in sentence production (Bastiaanse & van Zonneveld, 2005; Dragoy &
Bastiaanse, 2010; Kegl, 1995; M. Lee & Thompson, 2004; Stavrakaki et al., 2011), and
have been argued to reflect the syntactic and thematic complexity of unaccusatives.
Syntactically, unaccusatives (on some models of grammar, such as Government and
Binding; Chomsky, 1981) involve a movement operation, in which the Theme ar-
gument moves from an underlying object position to the subject position (The mani
fell __i) (Burzio, 1986; Levin & Rappaport Hovav, 1995). Unaccusatives are also com-
plex (and noncanonical) in terms of the mapping from thematic roles to syntax, given
that the subject is assigned the Theme role. Similarly, object-experiencer verbs (e.g.,
The woman shocked the man) also contain a noncanonical argument order (sub-
ject = Theme; object = Experiencer) and have been argued to involve syntactic move-
ment (Belletti & Rizzi, 1988). These have been shown to be impaired in agrammatic
speakers (Thompson & Lee, 2009).
A distinct line of research has examined the effect of “semantic weight” on verb pro-
duction in aphasia. This research contrasts production of semantically “heavy” verbs
(e.g., bake), which have a clear and specific meaning, to semantically “light” verbs (e.g.,
get), which have a vaguer meaning and can appear in a variety of contexts. This research
has demonstrated that agrammatism is associated with differential impairment of light
verbs (Gordon & Dell, 2003; Thorne & Faroqi-Shah, 2016). According to the Division of
Labor Hypothesis (Gordon & Dell, 2003), this is due to the complex argument structures
and subcategorization frames associated with light verbs (e.g., get can appear in a wide
range of syntactic contexts, e.g., get married, get to the party, get upset, get a haircut). In
certain contexts, light verbs also require online composition of argument structure. For
example, in a sentence such as John gave Bill an order to leave, the argument structures of
give and order must be combined to yield the sentence’s interpretation, a process that is
costly in typical language processing (Piñango, Mack, & Jackendoff, 2006; Wittenberg,
Paczynski, Wiese, Jackendoff, & Kuperberg, 2014; Wittenberg & Piñango, 2011). Thus,
impaired production of light verbs in agrammatism may also relate to difficulty with
complex argument structures.
Sentence-P rocessing Deficits

in Agrammatism
As mentioned in the introduction to this chapter, agrammatism is also associated

with impairments in comprehending and producing complex syntactic structures,
including sentences with embedding and/or a noncanonical order of arguments (see
Bornkessel-Schlesewsky & Schlesewsky, Chapter 27 in this volume). This has been
found in agrammatism caused by stroke (Bastiaanse & van Zonneveld, 2005; Burchert,
Meissner, & De Bleser, 2008; Cho-Reyes & Thompson, 2012; Dragoy & Bastiaanse, 2010;
Llinas-Grau & Martinez-Ferreiro, 2014; Thompson, Meltzer-Asscher, et al., 2013) and
in PPA-G (Cupit et al., 2016; DeLeon et al., 2012; Thompson, Meltzer-Asscher, et al.,
2013; Wilson, Dronkers, et al., 2010; Wilson, Henry, et al., 2010). Several theoretical ac-
counts have been proposed of these deficits, which vary with respect to whether deficits
are attributed to damaged linguistic representations, deficits in specific aspects of lin-
guistic processing, or a general reduction in processing capacity (see recent reviews in
Bastiaanse & Jonkers, 2012; Caplan, 2012; Druks, 2017; and Patil, Hanne, Burchert, De
Bleser, & Vasishth, 2016).
One major focus of research has been to explain impaired comprehension of
noncanonical sentences with long-distance dependencies (e.g., It was the girli whoi the
boy kissed ___i that day at school). On some linguistic theories, these structures involve
movement of a syntactic constituent (e.g., the relative pronoun who) from its original
post-verbal position, leaving behind a gap or trace (___i) (Chomsky, 1981). In psycholin-
guistic terms, the filler (e.g., who, co-indexed with the girl) must be associated with the
gap site (___i ), or with its subcategorizing verb (e.g., kissed), in order to be assigned the
correct thematic role (e.g., Theme) (Frazier, Clifton, & Randall, 1983; Pickering & Barry,
1991). Association of the filler with the gap site (or verb) involves reactivating aspects of
the representation of the filler that have decayed since its original presentation (e.g., se-
mantic representation; see review in Wagers & Phillips, 2014).
Some accounts have attributed impaired comprehension of these structures in
agrammatism to impaired representation of syntactic movement (Grodzinsky, 1986,
2000). If correct, this hypothesis would predict the absence of reactivation of the
meaning of the filler at the gap site. Studies of online processing have primarily tested
this hypothesis using two methods: cross-modal priming, where reactivation is in-
dicated by priming effects for the filler at the gap site, and visual-world eye-tracking,
where reactivation is indicated through an increase in eye movements to a picture of the
filler at the gap site. Although some early cross-modal priming studies found impaired
filler-gap processing in agrammatism (Swinney, Zurif, Prather, & Love, 1995, 1996;
Zurif, Swinney, Prather, Solomon, & Bushell, 1993), the majority of more recent priming
and eye-tracking studies found intact filler-gap reactivation effects, albeit at a delay in
some studies (Blumstein et al., 1998; Burkhardt, Piñango, & Wong, 2003; Dickey, Choy,
& Thompson, 2007; Dickey & Thompson, 2009; Love, Swinney, Walenski, & Zurif,
2008; Thompson & Choy, 2009). Further, eye-tracking experiments conducted in our
lab indicated intact processing in other types of long-distance dependencies, such as
pronouns and reflexives (Choy & Thompson, 2010; Hsu & Thompson, 2014; Thompson
& Choy, 2009), with no evidence of disrupted syntactic representation (contra, e.g.,
Grodzinsky, Wexler, Chien, Marakovitz, & Solomon, 1993).
Processing accounts of sentence comprehension deficits in agrammatism assume in-
tact syntactic representations, but impaired processing of linguistic information. The
Derived Order Problem Hypothesis (DOPH; Bastiaanse & van Zonneveld, 2006) agrees
with researchers such as Grodzinsky (1986, 2000) that syntactic movement is the
source of agrammatic comprehension deficits, but suggests that these deficits are pro-
cessing based rather than representational. In contrast, other accounts suggest that
processing slowdowns lead to comprehension deficits; specifically, slowed lexical acti-
vation (Love et al., 2008) or syntactic processing (Burkhardt et al., 2003) leads to the
overuse of default sentence-interpretation strategies (e.g., subject = Agent). A third type
of processing account suggests that a general reduction of processing resources leads
to slowdowns and “intermittent deficiencies” (i.e., inconsistent syntactic operations;
Caplan, Waters, DeDe, Michaud, & Reddy, 2007; Hanne, Sekerina, Vasishth, Burchert,
& De Bleser, 2011).
We have argued that sentence comprehension deficits relate to impaired thematic in-
tegration (Mack et al., 2013; Mack & Thompson, 2017; Meyer, Mack, & Thompson, 2012;
Thompson & Choy, 2009). This work follows the Mapping Hypothesis (Linebarger,
Schwartz, & Saffran, 1983; Schwartz, Linebarger, Saffran, & Pate, 1987) in proposing a
deficit in mapping between thematic and syntactic structure. However, our work also
attempts to identify the specific processes that break down online during thematic inte-
gration. The relevant processes include thematic prediction and thematic role assignment.
As sentences unfold in real time, unimpaired listeners predict upcoming words and
structures, supporting sentence comprehension by reducing time and resources needed
for lexical and/or structural processing (Federmeier, 2007; Kamide, 2008; Kutas,
DeLong, & Smith, 2011; Van Petten & Luka, 2012). In the domain of thematic processing,
unimpaired listeners predictively assign the Agent role to sentence-initial animate
noun phrases, in the absence of disambiguating grammatical (e.g., case) information
(Hanne, Burchert, De Bleser, & Vasishth, 2015; Kamide, Scheepers, & Altmann, 2003;
Knoeferle, Crocker, Scheepers, & Pickering, 2005; Meyer et al., 2012). For example,
upon hearing a sentence starting with The woman, unimpaired listeners predict that
the woman will be assigned the Agent role—a correct prediction if the sentence turns
out to be active (e.g., The woman lifted the man), but incorrect if it is passive (e.g., The
woman was lifted by the man). Notably, recent eye-tracking work in our lab and others’
shows that Agent-first predictions are absent in agrammatic listeners (Hanne et al.,
2015; Mack & Thompson, 2017; Mack, Wei, Gutierrez, & Thompson, 2016; Meyer et al.,
2012). Furthermore, restoring thematic prediction seems to be important for recovery
of sentence comprehension in agrammatism. In a recent study, we used eye-tracking
to examine changes in online sentence processing resulting from a course of language
treatment in 10 individuals with agrammatism (Mack & Thompson, 2017). The language
treatment protocol (Treatment of Underlying Forms [TUF]; Thompson & Shapiro,

2005) trained the production and comprehension of passive sentences. Although the-
matic prediction was not explicitly trained, participants nevertheless made more pre-
dictive eye movements following treatment. Further, the participants who showed
greater increases in thematic prediction also showed greater improvement in sentence-
comprehension accuracy.
In addition, thematic role assignment at the verb appears to be impaired in
noncanonical sentences. For example, consider an object-cleft such as It was the
girli whoi the boy kissed __i that day at school. Correct interpretation of this sentence
requires assigning the boy the Agent role and the girl the Theme role (through its co-
indexation with the gap and the relative pronoun who). As mentioned earlier, the ma-
jority of studies conducted to date suggest that agrammatic listeners can successfully
compute the syntactic dependency, reactivating the filler (who = the girl) at the gap site.
However, interpretation of the sentence is often incorrect, suggesting a subsequent
failure to assign thematic roles correctly. Eye-tracking evidence is consistent with this
hypothesis. For example, Dickey et al. (2007) found that listeners with agrammatism
evinced eye movements indicative of successful syntactic dependency formation in
object-wh questions, in sentences that were comprehended correctly as well as those
comprehended incorrectly (cf. Dickey & Thompson, 2009; Thompson & Choy, 2009).
In other words, syntactic dependency formation was generally successful, but thematic
role assignment sometimes failed. Choy and Thompson (2010) reported a similar pat-
tern of results in the processing of long-distance dependencies involving pronouns
and reflexives: intact reactivation of the antecedent online, combined with impaired
comprehension accuracy. Subsequent eye-tracking studies used a sentence-picture
matching paradigm to probe incremental thematic role assignment during sentence
comprehension. In sentences that were comprehended incorrectly, eye movements in-
dicated aberrant thematic role processing throughout the sentence (Hanne et al., 2015;
Hanne et al., 2011; Mack & Thompson, 2017; Mack et al., 2016; Meyer et al., 2012). These
findings suggest that impaired thematic prediction and thematic role assignment play
a key role in agrammatic comprehension deficits, leading to the Thematic Integration
Hypothesis.
Moving to sentence production, some accounts propose that damage to syntactic
representations underlies impairments in producing complex structures (Friedmann
& Grodzinsky, 1997). However, as in sentence comprehension, there is evidence from
sentence production that syntactic representations are intact. For example, agrammatic
speakers show intact structural priming of complex structures; that is, they often use
grammatical structures to which they have been recently exposed. This has been found
for complex structures such as datives (Cho-Reyes et al., 2016; Hartsuiker & Kolk, 1998),
passives (Cho & Thompson, 2010; Hartsuiker & Kolk, 1998), and cliticization in Italian
(Rossi, 2015). Intact structural priming effects for complex structures also have been
found in mixed groups of aphasic patients including agrammatic speakers (Saffran &
Martin, 1997; Verreyt et al., 2013). Further, structural priming boosts production of com-
plex structures in agrammatism. In one study (Cho-Reyes et al., 2016), 13 agrammatic
speakers showed impaired production of dative sentences in isolation; however, in the

context of a structural priming task, 7 of 13 speakers showed dative production accuracy
within the normal range. These findings indicate that the representations of complex
structures are intact.
Processing accounts vary with respect to the source of complex sentence production
deficits. The Derived Order Problem Hypothesis, described earlier in the context of sen-
tence comprehension, has also been applied to agrammatic production (Bastiaanse &
van Zonneveld, 2005). This account claims that deficits stem from difficulty producing
sentences with moved constituents. Other accounts propose more general processing
impairments, such as a reduced sentence-planning window that poses difficulty for
the production of complex sentences (Kolk, 1995). In contrast, we hypothesize that
agrammatic speakers have difficulty with thematic integration during sentence produc-
tion, which affects levels of processing including lexical selection, grammatical function
assignment, and morphosyntactic encoding (see the model of Bock & Levelt, 1994) as well
as the time course of sentence-production planning.
Lexical selection involves accessing and selecting lexical items (lemmas, containing
semantic and syntactic but not phonological information) that encode an event’s
meaning, including the action (typically a verb) and the event participants (typically
nouns). Accessing a verb lemma activates its argument structure, including its thematic
roles (e.g., Agent, Theme). As described in the previous section, research in our lab
and others’ indicates that access to verbs with complex argument structures is partic-
ularly difficult for agrammatic speakers (e.g., Thompson, 2003). Grammatical function
assignment concerns the mapping of selected lemmas to grammatical functions (sub-
ject, object). This process is guided by the argument structure information contained
within verb lemmas: for example, some psychological verbs map the Experiencer to the
subject role (e.g., John enjoyed the show) and others to the object role (e.g., The show
amused John). Agrammatic speakers have deficits in using thematic information during
grammatical function assignment, as reflected by difficulty in producing canonical ac-
tive sentences with complex argument structures, such as two-and three-argument
sentences (see previous section; e.g., Cho-Reyes & Thompson, 2012), as well as sentences
with noncanonical argument order (discussed further in the following).
Morphosyntactic encoding refers to the generation of a hierarchically organized syn-
tactic structure with morphologically specified words. The interplay between lexical se-
lection, grammatical function assignment, and morphosyntactic encoding is especially
important for complex sentence production. For example, to produce a passive sentence
such as The man was lifted by the woman, the speaker must access the verb lift and its
argument structure, map the Theme to the grammatical subject role and the Agent to
an adjunct role, and select the correct morphosyntactic form of the verb (i.e., auxiliary
+ past participle). Establishing the relationship between grammatical function assign-
ment and morphosyntactic encoding seems to be difficult for agrammatic speakers.
When they are primed with the morphosyntactic structure of passives, they often pro-
duce role-reversal errors (i.e., incorrect grammatical function assignment; e.g., The
woman was lifted by the man; Cho & Thompson, 2010) and when provided with a cue to
map the Theme as the grammatical subject, they often produce morphosyntactic errors
(e.g., The man was lifting by the woman; Faroqi-Shah & Thompson, 2003).
Further, agrammatic speakers often show abnormal sentence-planning processes,
relating to impaired thematic integration. Two sentence-production planning modes
have been observed in healthy speakers: verb-based structural planning, in which the
speaker initially accesses the verb and uses it to build a structural frame that guides
grammatical function assignment, and word-by-word incremental planning, in which
the speaker first retrieves the most accessible noun and links it to the grammatical
subject function. Healthy speakers use both modes of planning, depending on task
demands and linguistic factors (Hwang & Kaiser, 2014; Kempen & Huijbers, 1983; J. Lee
& Thompson, 2011a, 2011b; Schriefers, Teruel, & Meinshausen, 1998). Our prelimi-
nary work suggests that, unlike typical speakers, agrammatic speakers consistently re-
trieve verb information before speech onset, indicating verb-based structural planning
(J. Lee & Thompson, 2011a, 2011b; J. Lee, Yoshida, & Thompson, 2015). For example,
in two eye-tracking studies, speakers with agrammatism showed effects of argument
structure complexity (e.g., encoding unaccusative vs. unergative verbs) before speech
onset, whereas unimpaired speakers did so after speech onset (J. Lee & Thompson,
2011a, 2011b). This suggests a reliance on early retrieval of verb-argument structure in-
formation in agrammatic speakers, which is used to guide sentence planning (cf. J. Lee
et al., 2015).
The use of structural planning may help agrammatic speakers manage the demands
of sentence production, by temporally separating the costly processes of structural
planning and articulation. However, faulty structural planning—that is, deficits in
verb retrieval and mapping from thematic roles to syntactic structures—contribute
to sentence-production deficits. In a recent eye-tracking study with nine agrammatic
speakers (Mack, Nerantzini, & Thompson, 2017), we found evidence of abnormal struc-
tural planning during passive sentence production. However, after a course of Treatment
of Underlying Forms (Thompson & Shapiro, 2005), which successfully trained passive
sentence production, the agrammatic speakers showed more normal-like online sen-
tence planning, reflecting improved thematic integration.
Neural Substrates of Verb-

and Sentence-P rocessing Impairments
in Agrammatism
Damage to left hemisphere frontal regions, posterior perisylvian regions, and dorsal
white matter tracts have been consistently associated with agrammatism. In this
section, we review the relevant findings regarding the neural substrates of verb-and
sentence-processing impairments in agrammatism, situating them in the context of
contemporary models of the neurobiology of language. We first discuss models of verb
and verb-argument structure processing and follow this with sections on the neural
mechanisms of sentence comprehension and sentence production, respectively.
Relatively few models have specifically addressed the neural substrates of verb and
verb-argument structure processing. Thompson and Meltzer-Asscher (2014) proposed
a model in which retrieval and integration of verb and verb-argument information
is supported by a network consisting of left inferior parietal, posterior temporal, and
inferior frontal regions. Early studies on stroke-induced aphasia suggested that verb-
specific production deficits were related to left inferior frontal lesions, whereas noun-
specific impairments were related to left temporal lesions (e.g., Damasio & Tranel, 1993;
see review in Cappa & Perani, 2003; see also Kemmerer, Chapter 30 in this volume).
However, more recent studies have shown that the neural substrates of verb-specific
production deficits are more variable across individuals, and include lesions in left
frontal, posterior temporal, and inferior parietal cortical regions, as well as damage to
the insula, frontal operculum, dorsal white matter tracts, and/or subcortical structures
such as the basal ganglia (e.g., Aggujaro, Crepaldi, Pistarini, Taricco, & Luzzatti, 2006;
Kemmerer, Rudrauf, Manzel, & Tranel, 2012; Lukic et al., 2014; Piras & Marangolo,
2007; see reviews in Crepaldi, Berlingeri, Paulesu, & Luzzatti, 2011; Matzig, Druks,
Masterson, & Vigliocco, 2009). Using voxel-based lesion symptom mapping (VLSM;
Bates et al., 2003), we (Lukic et al., 2014) investigated the lesion locations associated with
impaired production of argument structure. Impaired verb naming, measured with
the Verb Naming Test of the Northwestern Assessment of Verbs and Sentences (NAVS;
Thompson, 2011), was associated with lesions in the left IFG, STG, and insula; these
effects were numerically stronger for transitive than for intransitive verbs, suggesting
that these regions support the retrieval of verbs with complex argument structures.
Further, we examined the neural correlates of impairments on the Argument Structure
Production Test (ASPT) of the NAVS. In this task, participants viewed action pictures
with the verb and event participants labeled (e.g., mail, man, letter) and were asked to
produce a grammatical sentence to describe the picture, which requires mapping the
arguments of the verb to the correct grammatical functions. Impaired performance on
the ASPT was associated with damage to the SMG and arcuate fasciculus (AF). In PPA,
few studies have directly examined the neural correlates of verb retrieval deficits. In re-
cent work, we found that impaired verb-argument structure production on the ASPT
is correlated with atrophy in the left inferior parietal lobule in a large group of patients
with PPA (Europa, Mack, Rogalski, Mesulam, & Thompson, in preparation). In line with
the model proposed by Thompson and Meltzer-Asscher (2014), these findings strongly
suggest that dorsal pathways (i.e., linking posterior temporal, inferior parietal, and infe-
rior frontal regions via the AF) play an important role in impaired production of verbs
and verb-argument structure.
Sentence comprehension impairments in agrammatism are predominantly observed
in semantically reversible and syntactically complex (e.g., noncanonical) sentences.
Several studies have attempted to localize the brain damage associated with compre-
hension impairments of this nature. In the literature on chronic agrammatism resulting
from stroke, this has resulted in mixed findings. One large-scale study of 79 patients
found that impaired comprehension of semantically reversible sentences was asso-

ciated with lesions in left posterior perisylvian regions, specifically the AG, SMG,
and posterior STG (Thothathiri, Kimberg, & Schwartz, 2012). Greater impairment of
noncanonical versus canonical sentences was associated with lesions in the left AG.
Another large (72-patient) study related lesion location to comprehension of sentences
that varied with respect to reversibility and complexity (Dronkers, Wilkins, Van Valin,
Redfern, & Jaeger, 2004). In that study, lesions in the anterior STG, AG, and left frontal
regions anterior to Broca’s area (BA 47 and BA 46) were associated with impaired sen-
tence comprehension. Further, Caplan and colleagues observed relationships between
syntactic complexity effects and lesions in the left posterior STG and AG, but not the
IFG (Caplan, Michaud, Hufford, & Makris, 2016). These relationships were task-and
structure-specific, suggesting that posterior perisylvian regions have distinct roles in
carrying out syntactic operations in the contexts of particular tasks. In contrast, two
previous studies by Caplan and colleagues found no significant relationships between
lesion location and syntactic complexity effects on sentence comprehension (e.g., rel-
atively impaired comprehension of passives vs. actives; Caplan et al., 1996; Caplan,
Waters, Kennedy et al., 2007).
Further, a few studies have suggested that stroke-induced damage to the left IFG
contributes to impaired predictive processing, which is important for sentence compre-
hension (Federmeier, 2007; Kamide, 2008; Kutas et al., 2011; Van Petten & Luka, 2012).
In one study, clinically non-aphasic stroke survivors with left IFG lesions showed im-
paired prediction of morphosyntactic information (Jakuszeit, Kotz, & Hasting, 2013). In
another study, aphasic individuals with left IFG lesions (but intact posterior perisylvian
regions) showed impaired prediction of lexical-semantic and morpho-phonological in-
formation, whereas those with left posterior perisylvian lesions (but intact frontal re-
gions) showed intact prediction (Nozari, Mirman, & Thompson-Schill, 2016). Similarly,
we found that nine individuals with agrammatism and lesions that included left frontal
regions showed impaired lexical-semantic prediction (Mack et al., 2013). These findings
suggest that the left IFG supports linguistic prediction across domains. In future work,
it will be important to test whether damage to this region is associated with impaired
thematic prediction, which may contribute to deficits in sentence comprehension in
agrammatism (Mack et al., 2013; Mack & Thompson, 2017; Mack et al., 2016; Meyer
et al., 2012).
Studies conducted with PPA patients largely support those derived from stroke-
induced aphasia. One large-scale study of 72 PPA patients found that impaired compre-
hension of semantically reversible, noncanonical sentences was associated with cortical
atrophy in the left IFG, dorsal premotor cortex (DPM), and inferior parietal lobule (AG
and SMG) (Mesulam, Thompson, Weintraub, & Rogalski, 2015). Generally consistent
with this, another study reported an association between syntactic complexity effects
(i.e., canonical>noncanonical difference scores) and atrophy in left frontal regions (IFG,
DPM, and precentral gyrus [PCG]) as well as posterior temporal regions (left STG,
middle temporal gyrus [MTG] and superior temporal sulcus [STS]) (Wilson, Dronkers,
et al., 2010). A third study found an association between overall sentence comprehension
impairment and atrophy in left posterior perisylvian regions, as well as a more specific
association between deficits in comprehending complex (relative-clause) sentences and
left IFG atrophy (Amici et al., 2007). A study by Peelle and colleagues on nonfluent PPA
found relationships between impaired comprehension of complex sentences and cor-
tical atrophy in left frontal regions (IFG, operculum, insula, dorsolateral frontal cortex),
as well as the anterior STG (Peelle et al., 2008).
In addition to cortical regions, most current models of sentence comprehension
also identify neural streams or pathways, that is, white matter tracts that connect cor-
tical regions, including dorsal pathways (e.g., linking posterior temporal regions to
inferior frontal regions via the parietal lobe, e.g., via the AF and superior longitudinal
fasciculus [SLF]) and ventral pathways (e.g., linking posterior temporal cortex to the
anterior temporal region via the temporal longitudinal fasciculus [TLF] and the inferior
frontal cortex via the uncinate fasciculus [UF] and inferior frontal-occipital fasciculus
[IFOF]) (Catani, Jones, & Ffytche, 2005). Both the dorsal and ventral pathways have
been argued to play a role in sentence comprehension, but the specific functions of the
two routes vary across models. Dorsal pathways, connecting posterior temporal to in-
ferior frontal regions, via the AF, have been argued to support processing of complex
syntactic structures (Friederici, 2012; Friederici & Gierhan, 2013) or syntactic pro-
cessing in general (Bornkessel-Schlesewsky & Schlesewsky, 2013; Hagoort & Indefrey,
2014). Because stroke-induced aphasia often is associated with marked disruption of
white matter tracts, researchers have turned to PPA to examine the relation between
tract integrity and language ability. In PPA, white matter tracts often are only partially
compromised, allowing for such analyses. Indeed, associations between damage to
dorsal routes (i.e., the AF and SLF) and impaired comprehension have been reported in
PPA (Wilson et al., 2011). Catani and colleagues (2017) found that, while both dorsal and
ventral routes were disrupted in a group of 30 patients with PPA, correlations between
sentence comprehension as tested by the NAVS (Thompson, 2011) and the posterior seg-
ment of the AF were noted. Functional neuroimaging (see Heim & Specht, Chapter 4
in this volume) studies of sentence comprehension have also supported these findings.
One study reported that listeners with PPA-G, in contrast with unimpaired adults, did
not show greater activation in the left IFG when comprehending noncanonical versus
canonical sentences (Wilson, Dronkers, et al., 2010; cf. Cooke et al., 2003). Another
study found that syntactic comprehension impairments were associated with abnormal
functional activation in left frontal regions (IFG, insula, ventral PCG) as well as poste-
rior temporal (MTG, STG) and inferior and superior parietal regions (i.e., dysfunction
of regions within the dorsal route) (Wilson et al., 2016).
In contrast, ventral language pathways connecting temporal to frontal regions have
been argued on several models to support semantic processing (Bornkessel-Schlesewsky
& Schlesewsky, 2013; Hagoort & Indefrey, 2014) and appear to play little role in sentence-
level deficits in PPA (Mandelli et al., 2014; Wilson, DeMarco, et al., 2014; Wilson et al.,
2011). In the Catani et al. (2017) PPA study, ventral route (i.e., the UF and the TLF, but
not the IFOF) disruptions were significantly correlated with single-word comprehen-
sion deficits. Some models, however, also propose a role for ventral pathways such as the
UF in basic syntactic comprehension and phrase-structure building (Friederici, 2012;

Friederici & Gierhan, 2013), as well as comprehension in general (Griffiths, Marslen-
Wilson, Stamatakis, & Tyler, 2013; Hickok & Poeppel, 2015; Saur et al., 2008).
In contrast with sentence comprehension, relatively few studies have examined the
neural correlates of impaired complex sentence production. In chronic agrammatism
caused by stroke, we recently found that impaired sentence production (measured
using the Sentence Production Priming Test of the NAVS) is associated with damage
to the insula and inferior parietal regions (AG, SMG) (Lukic et al., 2014). Damage to
the insula was specifically related to impairments in producing noncanonical (vs. ca-
nonical) sentences. Notably, Lukic et al. also found an association between lesions of
the AF and sentence production. Turning to PPA, Rogalski and colleagues (Rogalski
et al., 2011) used an anagram task to probe construction of complex sentences (the
Northwestern Anagram Test [NAT]; Thompson, Weintraub, & Mesulam, 2012) and
found that impaired performance was associated with atrophy in cortical regions in-
cluding the anterior and posterior IFG, the ventral PCG and post-central gyrus, and
the SMG. A study involving structured spoken sentence production found that im-
paired performance was associated with atrophy in the posterior IFG (DeLeon et al.,
2012). In a study of narrative language production, atrophy in the IFG and surrounding
frontal regions (e.g., the supplementary motor area [SMA]) was associated with reduced
production of complex sentences with embedding, as well as a lower “syntactic com-
posite” score (i.e., a principal component incorporating the rate of syntactic errors and
the proportion of words occurring outside of sentences) (Wilson, Henry, et al., 2010).
Further, dorsal language tracts have also been argued to play a particularly important
role in language production (Hickok & Poeppel, 2015; Saur et al., 2008). Two studies
found that damage to the AF/SLF was associated with impaired syntactic produc-
tion (Mandelli et al., 2014; Wilson et al., 2011). One of these studies also implicated a
second dorsal language pathway, connecting the left posterior IFG to the left SMA, in
impaired sentence production, consistent with the claim that this pathway supports
word sequencing (Mandelli et al., 2014). The role of this pathway in word sequencing
may be specific to spoken-language production. Catani et al. (2013) examined the lan-
guage functions of the left frontal aslant tract, which partially overlaps with the IFG-
SMA pathway investigated by Mandelli et al. (2014). Catani and colleagues found that
damage to the frontal aslant tract was associated with impaired fluency, but not with
grammatical impairments. However, the grammatical production task used by Catani
and colleagues was an anagram task (the Northwestern Anagram Test; Thompson,
Weintraub, & Mesulam, 2012), which does not require spoken-language production. In
contrast, the study by Mandelli and colleagues did require spoken-language production,
and did find an association between the IFG-SMA pathway and grammatical abilities.
In summary, the preponderance of evidence suggests that damage to cortical re-
gions, all within the dorsal language pathways, underlies both verb and sentence deficits
in agrammatism. These regions include the left posterior STG, inferior parietal re-
gions (AG, SMG), and inferior frontal regions (IFG). Additionally, sentence produc-
tion engages the insula, SMA, and ventral PCG. The extant literature also shows that
segments of the AF connect these regions, with the posterior segment associated with
verb and sentence comprehension and anterior regions associated with verb and sen-
tence production, whereas ventral language pathways play a relatively minor role.
Conclusion
In this chapter, we have provided an overview of research pertaining to verb-and

sentence- processing deficits in agrammatism and their neural correlates. In fu-
ture work, we expect that the study of agrammatism will become increasingly dy-
namic, with an emphasis on understanding change over time in language function
and its neural substrates. To this end, a growing body of research has examined the
neural correlates of aphasia recovery in the acute and subacute stages (e.g., Saur et al.,
2006) as well as treatment-related recovery of sentence comprehension and produc-
tion in chronic agrammatism (Thompson, den Ouden, Bonakdarpour, Garibaldi, &
Parrish, 2010; Thompson, Riley, den Ouden, Meltzer-Asscher, & Lukic, 2013; Wierenga
et al., 2006). Future work on recovery in stroke-induced agrammatism will likely
also incorporate insights from theories of language (re)-learning in this popula-
tion (Christiansen, Louise Kelly, Shillcock, & Greenfield, 2010; Goschke, Friederici,
Kotz, & van Kampen, 2001; Schuchard, Nerantzini, & Thompson, 2017; Schuchard &
Thompson, 2014, 2017; Zimmerer, Cowell, & Varley, 2014). A dynamic perspective is
also necessary for understanding the mechanisms of language decline in agrammatic
PPA, and how agrammatism can be effectively treated in the context of neurodegener-
ative disease (e.g., Hameister, Nickels, Abel, & Croot, 2017; Schneider, Thompson, &
Luring, 1996).
In addition, developing a dynamic understanding of agrammatism will require in-
tegrating insights from studies using a wide range of methods (e.g., offline behavioral
methods, eye-tracking, ERP, and neuroimaging measures of brain structure and func-
tion). In addition to the neuroimaging methods discussed here (i.e., structural meas-
ures of gray-and white-matter integrity, functional MRI), a fully specified model of
agrammatism will also need to incorporate information about perfusion (cerebral
blood flow; Thompson et al., 2010; Thompson et al., 2017) and functional and effective
connectivity (e.g., den Ouden et al., 2012; Xiang, Fonteijn, Norris, & Hagoort, 2010).
Thus, while much has already been accomplished in understanding the neurocognitive
mechanisms of agrammatism, there is still much work to be done.
Acknowledgments
This research was funded by NIH R01-DC001948 (Thompson), P50-DC012283 (Thompson)
and R01-DC008552 (Mesulam). We would like to thank our research participants, their
families and caregivers, and our colleagues in the Aphasia and Neurolinguistics Research
Laboratory, Center for the Neurobiology of Language Recovery, and the Cognitive Neurology
and Alzheimer’s Disease Center at Northwestern University for their contributions to
this work.
References
Aggujaro, S., Crepaldi, D., Pistarini, C., Taricco, M., & Luzzatti, C. (2006). Neuro-anatomical
correlates of impaired retrieval of verbs and nouns: Interaction of grammatical class,
imageability and actionality. Journal of Neurolinguistics, 19, 175–194.
Amici, S., Brambati, S. M., Wilkins, D. P., Ogar, J., Dronkers, N. L., Miller, B. L., & Gorno-
Tempini, M. L. (2007). Anatomical correlates of sentence comprehension and verbal
working memory in neurodegenerative disease. Journal of Neuroscience, 27(23),
6282–6290.
Arabatzi, M., & Edwards, S. (2002). Tense and syntactic processes in agrammatic speech. Brain
and Language, 80(3), 314–327.
Ash, S., Moore, P., Vesely, L., Gunawardena, D., McMillan, C., Anderson, C., . . . Grossman, M.
(2009). Non-fluent speech in frontotemporal lobar degeneration. Journal of Neurolinguistics,
22(4), 370–383.
Barbieri, E., Walenski, M., Hsu, C.-J., Bovbjerg, K., Dougherty, B., Mesulam, M. M., &
Thompson, C. K. (2016). Electrophysiological signatures of semantic and syntactic pro-
cessing in primary progressive aphasia: Integration and re-analysis processes during au-
ditory sentence comprehension. 54th Academy of Aphasia Conference, 171. Lausanne,
Switzerland: Frontiers Media SA.
Bastiaanse, R., Bamyaci, E., Hsu, C. J., Lee, J., Yarbay Duman, T., & Thompson, C. K. (2011).
Time reference in agrammatic aphasia: A cross-linguistic study. Journal of Neurolinguistics,
24, 652–673.
Bastiaanse, R., Hugen, J., Kos, M., & van Zonneveld, R. (2002). Lexical, morphological, and
syntactic aspects of verb production in agrammatic aphasics. Brain and Language, 80(2),
142–159.
Bastiaanse, R., & Jonkers, R. (1998). Verb retrieval in action naming and spontaneous speech in
agrammatic and anomic aphasia. Aphasiology, 21(11), 951–959.
Bastiaanse, R., & Jonkers, R. (2012). Linguistic accounts of agrammatic aphasia. In
R. Bastiaanse & C. K. Thompson (Eds.), Perspectives on agrammatism (pp. 17– 33).
Bastiaanse, R., & van Zonneveld, R. (2005). Sentence production with verbs of alternating
transitivity in agrammatic Broca’s aphasia. Journal of Neurolinguistics, 18,  57–66.
Bastiaanse, R., & van Zonneveld, R. (2006). Comprehension of passives in Broca’s aphasia.
Brain and Language, 96(2), 135–142; discussion 157–170.
Bates, E., Chen, S., Tzeng, O., Li, P., & Opie, M. (1991). The noun-verb problem in Chinese
aphasia. Brain and Language, 41(2), 203–233.
(2003). Voxel-based lesion-symptom mapping. Nature Neuroscience, 6(5), 448–450.
Belletti, A., & Rizzi, L. (1988). Psych-verbs and theta-theory. Natural Language and Linguistic
Theory, 6(3), 291–352.
Blumstein, S. E., Byma, G., Kurowski, K., Hourihan, J., Brown, T., & Hutchinson, A. (1998).
On-line processing of filler-gap construction in aphasia. Brain and Language, 61(2), 149–168.
Bock, K., & Levelt, W. J. M. (1994). Language production: Grammatical encoding. In
M. A. Gernsbacher (Ed.), Handbook of psycholinguistics (pp. 945– 984). San Diego,
CA: Academic Press.
Bornkessel-Schlesewsky, I., & Schlesewsky, M. (2013). Reconciling time, space and func-
tion: A new dorsal-ventral stream model of sentence comprehension. Brain and Language,
125(1),  60–76.
Burchert, F., Meissner, N., & De Bleser, R. (2008). Production of non-canonical sentences
in agrammatic aphasia: Limits in representation or rule application? Brain and Language,
104(2), 170–179.
Burchert, F., Swoboda-Moll, M., & Bleser, R. D. (2005). Tense and agreement dissociations in
German agrammatic speakers: Underspecification vs. hierarchy. Brain and Language, 94(2),
188–199.
Burkhardt, P., Piñango, M. M., & Wong, K. (2003). The role of the anterior left hemisphere in
real-time sentence comprehension: Evidence from split intransitivity. Brain and Language,
86(1),  9–22.
Burzio, L. (1986). Italian syntax: A government-binding approach. Dordrecht: Reidel.
Caplan, D. (2012). Resource reduction accounts of syntactically based comprehension
disorders. In R. Bastiaanse & C. K. Thompson (Eds.), Perspectives on agrammatism (pp. 34–
48). New York: Psychology Press.
Caplan, D., & Hanna, J. E. (1998). Sentence production by aphasic patients in a constrained
task. Brain and Language, 63(2), 184–218.
Caplan, D., Hildebrandt, N., & Makris, N. (1996). Location of lesions in stroke patients with
deficits in syntactic processing in sentence comprehension. Brain, 119(Pt 3), 933–949.
Caplan, D., Michaud, J., Hufford, R., & Makris, N. (2016). Deficit-lesion correlations in syn-
tactic comprehension in aphasia. Brain and Language, 152,  14–27.
Caplan, D., Waters, G., DeDe, G., Michaud, J., & Reddy, A. (2007). A study of syntactic pro-
cessing in aphasia. I: Behavioral (psycholinguistic) aspects. Brain and Language, 101(2),
103–150.
Caplan, D., Waters, G., Kennedy, D., Alpert, N., Makris, N., DeDe, G., . . . Reddy, A. (2007). A
study of syntactic processing in aphasia II: Neurological aspects. Brain and Language, 101(2),
151–177.
Cappa, S. F. (2012). Neurological accounts of agrammatism. In R. Bastiaanse & C. K. Thompson
(Eds.), Perspectives on agrammatism (pp. 49–59). New York: Psychology Press.
Cappa, S. F., & Perani, D. (2003). The neural correlates of noun and verb processing. Journal of
Caramazza, A. (1997). How many levels of processing are there in lexical access? Cognitive
Psychology, 14(1), 177–208.
Caramazza, A., & Zurif, E. B. (1976). Dissociation of algorithmic and heuristic processes in lan-
guage comprehension: Evidence from aphasia. Brain and Language, 3(4), 572–582.
Catani, M., Drossinos Sancho, N., Witteveen, S., Forkel, S., D’Anna, L., Dell’Acqua,
F., . . . Mesulam, M. (2017). Dorsal and ventral pathways for words and sentence pro-
cessing. Paper presented at the Organization for Human Brain Mapping, Vancouver,
Canada.
Catani, M., Jones, D. K., & Ffytche, D. H. (2005). Perisylvian language networks of the human
brain. Annals of Neurology, 57(1),  8–16.
Catani, M., & Mesulam, M. (2008). The arcuate fasciculus and the disconnection theme in lan-
guage and aphasia: History and current state. Cortex, 44(8), 953–961.
Catani, M., Mesulam, M. M., Jakobsen, E., Malik, F., Martersteck, A., Wieneke, C., . . . Rogalski,
E. (2013). A novel frontal pathway underlies verbal fluency in primary progressive aphasia.
Brain, 136(Pt 8), 2619–2628.
Cho-Reyes, S., Mack, J. E., & Thompson, C. K. (2016). Grammatical encoding and learning in
agrammatic aphasia: Evidence from structural priming. Journal of Memory and Language,
91, 202–218.
Cho-Reyes, S., & Thompson, C. K. (2012). Verb and sentence production and comprehen-
sion in aphasia: Northwestern Assessment of Verbs and Sentences. Aphasiology, 26(10),
1250–1277.
Cho, S., & Thompson, C. K. (2010). What goes wrong during passive sentence production in
agrammatic aphasia: An eyetracking study. Aphasiology, 24(12), 1576–1592.
Chomsky, N. (1981). Lectures on government and binding. Dordrecht: Foris.
Choy, J. J., & Thompson, C. K. (2010). Binding in agrammatic aphasia: Processing to compre-
hension. Aphasiology, 24(5), 551–579.
Christiansen, M. H., Louise Kelly, M., Shillcock, R. C., & Greenfield, K. (2010). Impaired artifi-
cial grammar learning in agrammatism. Cognition, 116(3), 382–393.
Cooke, A., DeVita, C., Gee, J., Alsop, D., Detre, J., Chen, W., & Grossman, M. (2003). Neural
basis for sentence comprehension deficits in frontotemporal dementia. Brain and Language,
85(2), 211–221.
Crepaldi, D., Berlingeri, M., Paulesu, E., & Luzzatti, C. (2011). A place for nouns and a place
for verbs? A critical review of neurocognitive data on grammatical-class effects. Brain and
Language, 116(1),  33–49.
Cupit, J., Graham, N. L., Leonard, C., Tang-Wai, D., Black, S. E., & Rochon, E. (2016).
Wh-questions and passive sentences in non-fluent variant PPA and semantic variant
PPA: Longitudinal findings of an anagram production task. Cognitive Neuropsychology,
33(5–6), 329–342.
Damasio, A. R., & Tranel, D. (1993). Nouns and verbs are retrieved with differently distributed
neural systems. Proceedings of the National Academy of Sciences, 90(11), 4957–4960.
De Bleser, R., & Kauschke, C. (2003). Acquisition and loss of nouns and verbs: Parallel or diver-
gent patterns? Journal of Neurolinguistics, 16(2–3), 213–229.
DeDe, G. (2013a). Effects of verb bias and syntactic ambiguity on reading in people with
aphasia. Aphasiology, 27(10–12), 1408–1425.
DeDe, G. (2013b). Verb transitivity bias affects on-line sentence reading in people with aphasia.
Aphasiology, 27(3), 326–343.
DeLeon, J., Gesierich, B., Besbris, M., Ogar, J., Henry, M. L., Miller, B. L., . . . Wilson, S. M.
(2012). Elicitation of specific syntactic structures in primary progressive aphasia. Brain and
Language, 123(3), 183–190.
den Ouden, D. B., Saur, D., Mader, W., Schelter, B., Lukic, S., Wali, E., . . . Thompson, C. K.
(2012). Network modulation during complex syntactic processing. Neuroimage, 59(1),
815–823.
Dickey, M. W., Choy, J. J., & Thompson, C. K. (2007). Real-time comprehension of wh-
movement in aphasia: Evidence from eyetracking while listening. Brain and Language,
100(1),  1–22.
Dickey, M. W., Milman, L. H., & Thompson, C. K. (2008). Judgment of functional morphology
in agrammatic aphasia. Journal of Neurolinguistics, 21(1),  35–65.
Dickey, M. W., & Thompson, C. K. (2009). Automatic processing of wh-and NP-movement in
agrammatic aphasia: Evidence from eyetracking. Journal of Neurolinguistics, 22(6), 563–583.
Dickey, M. W., & Warren, T. (2015). The influence of event-related knowledge on verb-
argument processing in aphasia. Neuropsychologia, 67,  63–81.
Dragoy, O., & Bastiaanse, R. (2010). Verb production and word order in Russian agrammatic
speakers. Aphasiology, 24,  28–55.
Dronkers, N. F., Wilkins, D. P., Van Valin, R. D., Jr., Redfern, B. B., & Jaeger, J. J. (2004).
Lesion analysis of the brain areas involved in language comprehension. Cognition,
92(1–2), 145–177.
Druks, J. (2006). Morpho-syntactic and morpho-phonological deficits in the production of
regularly and irregularly inflected verbs. Aphasiology, 20(9), 993–1017.
Druks, J. (2017). Contemporary and emergent theories of agrammatism. New York: Routledge.
Europa, E., Mack, J. E., Rogalski, E. J., Mesulam, M. M., & Thompson, C. K. (in preparation).
Neural correlates of grammatical impairments in PPA.
Faroqi-Shah, Y., & Thompson, C. K. (2003). Effect of lexical cues on the production of active and
passive sentences in Broca’s and Wernicke’s aphasia. Brain and Language, 85(3), 409–426.
Faroqi-Shah, Y., & Thompson, C. K. (2007). Verb inflections in agrammatic aphasia: Encoding
of tense features. Journal of Memory and Language, 56(1), 129–151.
prehension. Psychophysiology, 44(4), 491–505.
Frazier, L., Clifton, C., & Randall, J. (1983). Filling gaps: Decision principles and structure in
sentence comprehension. Cognition, 13(2), 187–222.
comprehension. Trends in Cognitive Sciences, 16(5), 262–268.
Friederici, A. D., & Gierhan, S. M. (2013). The language network. Current Opinion in
Friedmann, N. (2002). Question production in agrammatism: The tree pruning hypothesis.
Friedmann, N., & Grodzinsky, Y. (1997). Tense and agreement in agrammatic produc-
tion: Pruning the syntactic tree. Brain and Language, 56(3), 397–425.
Goldberg, A. E. (1995). Constructions: A construction grammar approach to argument structure.
Chicago: University of Chicago Press.
Goldberg, A. E. (2003). Constructions: A new theoretical approach to language. Trends in
Gordon, J. K., & Dell, G. S. (2003). Learning to divide the labor: An account of deficits in light
and heavy verb production. Cognitive Science, 27(1),  1–40.
Neurology, 76(11), 1006–1014.
Goschke, T., Friederici, A. D., Kotz, S. A., & van Kampen, A. (2001). Procedural learning in
Broca’s aphasia: Dissociation between the implicit acquisition of spatio-motor and phoneme
sequences. Journal of Cognitive Neuroscience, 13(3), 370–388.
Graham, N. L., Patterson, K., & Hodges, J. R. (2004). When more yields less: Speaking and
writing deficits in nonfluent progressive aphasia. Neurocase, 10(2), 141–155.
Griffiths, J. D., Marslen-Wilson, W. D., Stamatakis, E. A., & Tyler, L. K. (2013). Functional orga-
nization of the neural language system: Dorsal and ventral pathways are critical for syntax.
Grodzinsky, Y. (1986). Language deficits and the theory of syntax. Brain and Language, 27(1),
135–159.
Grodzinsky, Y. (2000). The neurology of syntax: Language use without Broca’s area. Behavioral
and Brain Sciences, 23(1), 1–21; discussion 21–7 1.
Grodzinsky, Y., & Finkel, L. (1998). The neurology of empty categories: Aphasics’ failure to de-
tect ungrammaticality. Journal of Cognitive Neuroscience, 10(2), 281–292.
Grodzinsky, Y., Wexler, K., Chien, Y. C., Marakovitz, S., & Solomon, J. (1993). The breakdown
of binding relations. Brain and Language, 45(3), 396–422.
Gunawardena, D., Ash, S., McMillan, C., Avants, B., Gee, J., & Grossman, M. (2010). Why are
patients with progressive nonfluent aphasia nonfluent? Neurology, 75(7), 588–594.
Hagiwara, H. (1995). The breakdown of functional categories and the economy of derivation.
Hagoort, P., & Indefrey, P. (2014). The neurobiology of language beyond single words. Annual
Hameister, I., Nickels, L., Abel, S., & Croot, K. (2017). “Do you have mowing the lawn?”:
Improvements in word retrieval and grammar following constraint-induced language
therapy in primary progressive aphasia. Aphasiology, 31(3), 308–331.
Hanne, S., Burchert, F., De Bleser, R., & Vasishth, S. (2015). Sentence comprehension and mor-
phological cues in aphasia: What eye-tracking reveals about integration and prediction.
Journal of Neurolinguistics, 34, 83–111.
Hanne, S., Sekerina, I. A., Vasishth, S., Burchert, F., & De Bleser, R. (2011). Chance in
agrammatic sentence comprehension: What does it really mean? Evidence from eye
movements of German agrammatic aphasic patients. Aphasiology, 25(2), 221–244.
Hartsuiker, R. J., & Kolk, H. H. (1998). Syntactic facilitation in agrammatic sentence produc-
tion. Brain and Language, 62(2), 221–254.
Hickok, G., & Poeppel, D. (2015). Neural basis of speech perception. Handbook of Clinical
Neurology, 129, 149–160.
Hillis, A. E., Heidler-Gary, J., Newhart, M., Chang, S., Ken, L., & Bak, T. H. (2006). Naming and
comprehension in primary progressive aphasia: The influence of grammatical word class.
Aphasiology, 20(2–4), 246–256.
Hillis, A. E., Oh, S., & Ken, L. (2004). Deterioration of naming nouns versus verbs in primary
progressive aphasia. Annals of Neurology, 55(2), 268–275.
Hillis, A. E., Tuffiash, E., & Caramazza, A. (2002). Modality-specific deterioration in naming verbs
in nonfluent primary progressive aphasia. Journal of Cognitive Neuroscience, 14(7), 1099–1108.
Horvath, J., & Siloni, T. (2011). Causatives across components. Natural Language and Linguistic
Theory, 29, 657–704.
Hsu, C.-J., & Thompson, C. K. (2014). Cataphora processing in agrammatic aphasia: Eye move-
ment evidence for integration deficits. Frontiers in Psychology Conference Abstract: Academy
of Aphasia 52nd Annual Meeting. doi: 10.3389/conf.fpsyg.2014.64.00035
Hwang, H., & Kaiser, E. (2014). The role of the verb in grammatical function assignment in
English and Korean. Journal of Experimental Psychology: Learning, Memory, and Cognition,
40(5), 1363–1376.
Jackendoff, R. (1972). Semantic interpretation in generative grammar. Cambridge, MA:
MIT Press.
Jakuszeit, M., Kotz, S. A., & Hasting, A. S. (2013). Generating predictions: Lesion evidence on
the role of left inferior frontal cortex in rapid syntactic analysis. Cortex, 49(10), 2861–2874.
Josephs, K. A., Duffy, J. R., Strand, E. A., Whitwell, J. L., Layton, K. F., Parisi, J. E., . . . Petersen,
R. C. (2006). Clinicopathological and imaging correlates of progressive aphasia and apraxia
of speech. Brain, 129(Pt 6), 1385–1398.
Kamide, Y. (2008). Anticipatory processes in sentence processing. Language and Linguistics

Compass, 2(4), 647–670.
Kamide, Y., Scheepers, C., & Altmann, G. T. (2003). Integration of syntactic and semantic in-
formation in predictive processing: Cross-linguistic evidence from German and English.
Journal of Psycholinguistic Research, 32(1),  37–55.
Kegl, J. (1995). Levels of representation and units of access relevant to agrammatism. Brain and
Language, 50(2), 151–200.
Kemmerer, D., Rudrauf, D., Manzel, K., & Tranel, D. (2012). Behavioral patterns and lesion
sites associated with impaired processing of lexical and conceptual knowledge of actions.
Cortex, 48(7), 826–848.
Kempen, G., & Huijbers, P. (1983). The lexicalization process in sentence production and
naming: Indirect election of words. Cognition, 14, 185–209.
Kertesz, A., Lesk, D., & McCabe, P. (1977). Isotope localization of infarcts in aphasia. Archives of
Neurology, 34(10), 590–601.
Kielar, A., Meltzer-Asscher, A., & Thompson, C. K. (2012). Electrophysiological responses to
argument structure violations in healthy adults and individuals with agrammatic aphasia.
Kim, M., & Thompson, C. K. (2000). Patterns of comprehension and production of nouns
and verbs in agrammatism: Implications for lexical organization. Brain and Language,
74(1),  1–25.
Kim, M., & Thompson, C. K. (2004). Verb deficits in Alzheimer’s disease and agrammatism:
Implications for lexical organization. Brain and Language, 88(1),  1–20.
Kiss, K. (2000). Effects of verb complexity on agrammatic aphasics’ sentence production. In
R. Bastiaanse & Y. Grodzinsky (Eds.), Grammatical disorders in aphasia. London: Whurr.
Knibb, J. A., Woollams, A. M., Hodges, J. R., & Patterson, K. (2009). Making sense of progres-
sive non-fluent aphasia: An analysis of conversational speech. Brain, 132(Pt 10), 2734–2746.
Knoeferle, P., Crocker, M. W., Scheepers, C., & Pickering, M. J. (2005). The influence of the
immediate visual context on incremental thematic role-assignment: Evidence from eye-
movements in depicted events. Cognition, 95(1), 95–127.
Kolk, H. (1995). A time-based approach to agrammatic production. Brain and Language, 50(3),
282–303.
Kutas, M., DeLong, K. A., & Smith, N. J. (2011). A look around at what lies ahead: Prediction
and predictability in language processing. In M. Bar (Ed.), Predictions in the brain: Using our
past to generate a future (pp. 190–207). Oxford: Oxford University Press.
Lee, J., Milman, L. H., & Thompson, C. K. (2008). Functional category production in English
agrammatism. Aphasiology, 22(7–8), 893–905.
Lee, J., & Thompson, C. K. (2011a). Real-time production of arguments and adjuncts in normal
and agrammatic speakers. Language and Cognitive Processes, 26(8), 985–1021.
Lee, J., & Thompson, C. K. (2011b). Real-time production of unergative and unaccusative
sentences in normal and agrammatic speakers: An eyetracking study. Aphasiology, 25(6–7),
813–825.
Lee, J., & Thompson, C. K. (2017). Northwestern Assessment of Verb Inflection (NAVI). Evanston,
IL: Northwestern University.
Lee, J., Yoshida, M., & Thompson, C. K. (2015). Grammatical planning units during real-
time sentence production in agrammatic aphasia and healthy speakers. Journal of Speech,
Language, and Hearing Research, 58, 1182–1194.
Lee, M., & Thompson, C. K. (2004). Agrammatic aphasic production and comprehension of
unaccusative verbs in sentence contexts. Journal of Neurolinguistics, 17(4), 315–330.
Levelt, W. J., Roelofs, A., & Meyer, A. S. (1999). A theory of lexical access in speech production.
Behavioral and Brain Sciences, 22(1), 1–38; discussion 38–75.
Levin, B., & Rappaport, M. (1986). The formation of adjectival passives. Linguistic Inquiry,
17(4), 623–661.
Levin, B., & Rappaport Hovav, M. (1995). Unaccusativity: At the syntax-lexical semantics inter-
face. Cambridge, MA: MIT Press.
Levin, B., & Rappaport Hovav, M. (2005). Argument realization. Cambridge: Cambridge
University Press.
Linebarger, M. C., Schwartz, M. F., & Saffran, E. M. (1983). Sensitivity to grammatical structure
in so-called agrammatic aphasics. Cognition, 13(3), 361–392.
Llinas-Grau, M., & Martinez-Ferreiro, S. (2014). On the presence and absence of that in
aphasia. Aphasiology, 28(1),  62–81.
Love, T., Swinney, D., Walenski, M., & Zurif, E. (2008). How left inferior frontal cortex
participates in syntactic processing: Evidence from aphasia. Brain and Language, 107(3),
203–219.
Lukic, S., Bonakdarpour, B., den Ouden, D. B., Price, C., & Thompson, C.K. (2014). Neural
mechanisms of verb and sentence production: A lesion-deficit study. Academy of Aphasia
Conference, Lucerne, Switzerland.
Luzzatti, C., Raggi, R., Zonca, G., Pistarini, C., Contardi, A., & Pinna, G. D. (2002). Verb-
noun double dissociation in aphasic lexical impairments: The role of word frequency and
imageability. Brain and Language, 81(1–3), 432–444.
Mack, J. E., Ji, W., & Thompson, C. K. (2013). Effects of verb meaning on lexical integration
in agrammatic aphasia: Evidence from eyetracking. Journal of Neurolinguistics, 26(6),
619–636.
Mack, J. E., Mesulam, M.-M., & Thompson, C. K. (2017). Understanding and combining
words during sentence comprehension in primary progressive aphasia. Frontiers in
Psychology Conference Abstract: Academy of Aphasia 55th Annual Meeting. doi: 10.3389/conf.
fnhum.2017.223.00011
Mack, J. E., Nerantzini, M., & Thompson, C. K. (2017). Recovery of sentence production
processes following language treatment in aphasia: Evidence from eyetracking. Frontiers in
Human Neuroscience, 11, article 101, 1–20.
Mack, J. E., & Thompson, C. K. (2017). Recovery of online sentence processing in aphasia: Eye
movement changes resulting from treatment of underlying forms. Journal of Speech,
Language, and Hearing Research, 60, 1299–1315.
Mack, J. E., Wei, A. Z., Gutierrez, S., & Thompson, C. K. (2016). Tracking sentence compre-
hension: Test-retest reliability in people with aphasia and unimpaired adults. Journal of
Malyutina, S., Richardson, J. D., & den Ouden, D. B. (2016). Verb argument structure in narra-
tive speech: Mining AphasiaBank. Seminars in Speech and Language, 37(1),  34–47.
Mandelli, M. L., Caverzasi, E., Binney, R. J., Henry, M. L., Lobach, I., Block, N., . . . Gorno-
Tempini, M. L. (2014). Frontal white matter tracts sustaining speech production in primary
progressive aphasia. Journal of Neuroscience, 34(29), 9754–9767.
Matzig, S., Druks, J., Masterson, J., & Vigliocco, G. (2009). Noun and verb differences in pic-
ture naming: Past studies and new evidence. Cortex, 45(6), 738–758.
Mesulam, M. M., Thompson, C. K., Weintraub, S., & Rogalski, E. J. (2015). The Wernicke co-
nundrum and the anatomy of language comprehension in primary progressive aphasia.
Brain, 138(Pt 8), 2423–2437.
Mesulam, M. M., Wieneke, C., Rogalski, E., Cobia, D., Thompson, C., & Weintraub, S. (2009).
Quantitative template for subtyping primary progressive aphasia. Archives of Neurology,
66(12), 1545–1551.
Mesulam, M. M., Wieneke, C., Thompson, C., Rogalski, E., & Weintraub, S. (2012). Quantitative
classification of primary progressive aphasia at early and mild impairment stages. Brain,
135(Pt 5), 1537–1553.
Meyer, A. M., Mack, J. E., & Thompson, C. K. (2012). Tracking passive sentence comprehension
in agrammatic aphasia. Journal of Neurolinguistics, 25(1),  31–43.
Miceli, G., Silveri, M. C., Nocentini, U., & Caramazza, A. (1988). Patterns of dissociation in
comprehension and production of nouns and verbs. Aphasiology, 2(3–4), 351–358.
Milman, L. H., Dickey, M. W., & Thompson, C. K. (2008). A psychometric analysis of func-
tional category production in English agrammatic narratives. Brain and Language,
105(1),  18–31.
Myers, E. B., & Blumstein, S. E. (2005). Selectional restriction and semantic priming effects in
normals and Broca’s aphasics. Journal of Neurolinguistics, 18, 277–296.
Nozari, N., Mirman, D., & Thompson- Schill, S. L. (2016). The ventrolateral prefrontal
cortex facilitates processing of sentential context to locate referents. Brain and Language,
157–158,  1–13.
Patil, U., Hanne, S., Burchert, F., De Bleser, R., & Vasishth, S. (2016). A computational evalua-
tion of sentence processing deficits in aphasia. Cognitive Science, 40(1),  5–50.
Peelle, J. E., Cooke, A., Moore, P., Vesely, L., & Grossman, M. (2007). Syntactic and thematic
components of sentence processing in progressive nonfluent aphasia and nonaphasic
frontotemporal dementia. Journal of Neurolinguistics, 20(6), 482–494.
Peelle, J. E., Troiani, V., Gee, J., Moore, P., McMillan, C., Vesely, L., & Grossman, M. (2008).
Sentence comprehension and voxel-based morphometry in progressive nonfluent aphasia,
semantic dementia, and nonaphasic frontotemporal dementia. Journal of Neurolinguistics,
21(5), 418–432.
Perlmutter, D. (1978). Impersonal passives and the unaccusative hypothesis. Proceedings of the
4th Annual Meeting of the Berkeley Linguistics Society, 157–190. https://escholarship.org/uc/
bling_proceedings/4/4
Pickering, M., & Barry, G. (1991). Sentence processing without empty categories. Language and
Cognitive Processes, 6(3), 229–259.
Piñango, M. M. (2000). Syntactic displacement in Broca’s agrammatic aphasia. In R. Bastiaanse
& Y. Grodzinsky (Eds.), Grammatical disorders in aphasia: A neurolinguistic perspective (pp.
75–87). London: Whurr.
Piñango, M. M., Mack, J. E., & Jackendoff, R. (2006). Semantic combinatorial processes in argu-
ment structure: Evidence from light-verbs. Paper presented at the 32nd Annual Meeting of the
Berkeley Linguistics Society, Berkeley, CA.
Piras, F., & Marangolo, P. (2007). Noun-verb naming in aphasia: A voxel-based lesion-
symptom mapping study. Neuroreport, 18(14), 1455–1458.
Price, C. C., & Grossman, M. (2005). Verb agreements during on-line sentence processing in
Alzheimer’s disease and frontotemporal dementia. Brain and Language, 94(2), 217–232.
Reinhart, T. (2002). The theta system: An overview. Theoretical Linguistics, 28, 229–290.
Rogalski, E., Cobia, D., Harrison, T. M., Wieneke, C., Thompson, C. K., Weintraub, S., &
Mesulam, M. M. (2011). Anatomy of language impairments in primary progressive aphasia.
Rohrer, J. D., Rossor, M. N., & Warren, J. D. (2010). Syndromes of nonfluent primary progres-
sive aphasia: A clinical and neurolinguistic analysis. Neurology, 75(7), 603–610.
Rossi, E. (2015). Modulating the sensitivity to syntactic factors in production: Evidence from
syntactic priming in agrammatism. Applied Psycholinguistics, 36, 639–669.
Saffran, E. M., & Martin, N. (1997). Effects of structural priming on sentence production in
aphasics. Language and Cognitive Processes, 12(5–6), 877–882.
Saffran, E. M., Schwartz, M. F., & Marin, O. S. (1980). The word order problem in agrammatism.
II: Production. Brain and Language, 10(2), 263–280.
Saur, D., Kreher, B. W., Schnell, S., Kummerer, D., Kellmeyer, P., Vry, M. S., . . . Weiller, C.
(2008). Ventral and dorsal pathways for language. Proceedings of the National Academy of
Sciences, 105(46), 18035–18040.
Saur, D., Lange, R., Baumgaertner, A., Schraknepper, V., Willmes, K., Rijntjes, M., & Weiller, C.
(2006). Dynamics of language reorganization after stroke. Brain, 129(Pt 6), 1371–1384.
Schneider, S. L., Thompson, C. K., & Luring, B. (1996). Effects of verbal plus gestural matrix
training on sentence production in a patient with primary progressive aphasia. Aphasiology,
10(3), 297–317.
Schriefers, H., Teruel, E., & Meinshausen, R. M. (1998). Producing simple sentences: Results
from picture-word interference experiments. Journal of Memory and Language, 39, 609–632.
Schuchard, J., Nerantzini, M., & Thompson, C. K. (2017). Implicit learning and implicit treat-
ment outcomes in individuals with aphasia. Aphasiology, 31(1),  25–48.
Schuchard, J., & Thompson, C. K. (2014). Implicit and explicit learning in individuals with
agrammatic aphasia. Journal of Psycholinguistic Research, 43(3), 209–224.
Schuchard, J., & Thompson, C. K. (2017). Sequential learning in individuals with agrammatic
aphasia: Evidence from artificial grammar learning. Journal of Cognitive Psychology, 29(5),
521–534.
Schwartz, M. F., Linebarger, M. C., Saffran, E. M., & Pate, D. (1987). Syntactic transparency and
sentence interpretation in aphasia. Language and Cognitive Processes, 2, 85–113.
Schwartz, M. F., Saffran, E., & Marin, O. (1980). The word order problem in agrammatism.
I: Comprehension. Brain and Language, 10, 249–262.
Shapiro, L. P., Gordon, B., Hack, N., & Killackey, J. (1993). Verb-argument structure processing
in complex sentences in Broca’s and Wernicke’s aphasia. Brain and Language, 45(3), 423–447.
Shapiro, L. P., & Levine, B. A. (1990). Verb processing during sentence comprehension in
aphasia. Brain and Language, 38(1),  21–47.
Stavrakaki, S., Alexiadou, A., Kambanaros, M., Bostantjopolou, S., & Katsarou, Z. (2011). The
production and comprehension of verbs with alternating transitivity by patients with non-
fluent aphasia. Aphasiology, 25(5), 642–668.
Sung, J. E. (2016). The effects of verb argument complexity on verb production in persons with
aphasia: Evidence from a subject-object-verb language. Journal of Psycholinguistic Research,
45(2), 287–305.
Swinney, D., Zurif, E., Prather, P., & Love, T. (1995). Syntactic processing in aphasia. Brain and
Cognition, 28(2), 189–189.
Swinney, D., Zurif, E., Prather, P., & Love, T. (1996). Neurological distribution of processing re-
sources in language comprehension. Journal of Cognitive Neuroscience, 8, 174–184.
Thompson, C. K. (2003). Unaccusative verb production in agrammatic aphasia: The argument

structure complexity hypothesis. Journal of Neurolinguistics, 16(2–3), 151–167.
Thompson, C. K. (2011). Northwestern Assessment of Verbs and Sentences (NAVS). Evanston,
IL: Northwestern University.
Thompson, C. K., Ballard, K. J., Tait, M. E., Weintraub, S., & Mesulam, M. (1997). Patterns of
language decline in non-fluent primary progressive aphasia. Aphasiology, 11(4/5), 297–321.
Thompson, C. K., Cho, S., Hsu, C. J., Wieneke, C., Rademaker, A., Weitner, B. B., . . . Weintraub,
S. (2012). Dissociations between fluency and agrammatism in primary progressive aphasia.
Aphasiology, 26(1),  20–43.
Thompson, C. K., & Choy, J. J. (2009). Pronominal resolution and gap filling in agrammatic
aphasia: Evidence from eye movements. Journal of Psycholinguistic Research, 38(3), 255–283.
Thompson, C. K., den Ouden, D. B., Bonakdarpour, B., Garibaldi, K., & Parrish, T. B. (2010).
Neural plasticity and treatment-induced recovery of sentence processing in agrammatism.
Thompson, C. K., Lange, K. L., Schneider, S. L., & Shapiro, L. P. (1997). Agrammatic and non-
brain-damaged subjects’ verb and verb argument structure production. Aphasiology, 11(4–
5), 473–490.
Thompson, C. K., & Lee, M. (2009). Psych verb production and comprehension in agrammatic
Broca’s aphasia. Journal of Neurolinguistics, 22(4), 354–369.
Thompson, C. K., Lukic, S., King, M. C., Mesulam, M., & Weintraub, S. (2012). Verb and noun
deficits in stroke-induced and primary progressive aphasia: The Northwestern Naming
Battery. Aphasiology, 26(5), 632–655.
Thompson, C. K., & Mack, J. E. (2014). Grammatical impairments in PPA. Aphasiology, 28(8–
9), 1018–1037.
Thompson, C. K., & Meltzer-Asscher, A. (2014). Neurocognitive mechanisms of verb argu-
ment structure processing. In A. Bachrach, I. Roy, & L. Stockall (Eds.), Structuring the argu-
ment: Multidisciplinary research on verb argument structure (pp. 141–168). Amsterdam: John
Benjamins.
Thompson, C. K., Meltzer-Asscher, A., Cho, S., Lee, J., Wieneke, C., Weintraub, S., & Mesulam,
M. M. (2013). Syntactic and morphosyntactic processing in stroke-induced and primary
progressive aphasia. Behavioural Neurology, 26(1–2),  35–54.
Thompson, C. K., Riley, E. A., den Ouden, D. B., Meltzer-Asscher, A., & Lukic, S. (2013).
Training verb argument structure production in agrammatic aphasia: Behavioral and neural
recovery patterns. Cortex, 49(9), 2358–2376.
Thompson, C. K., & Shapiro, L. (2005). Treating agrammatic aphasia within a linguistic frame-
work: Treatment of underlying forms. Aphasiology, 19(10–11), 1021–1036.
Thompson, C. K., Shapiro, L. P., Li, L., & Schendel, L. (1995). Analysis of verbs and verb-
argument structure: A method for quantification of aphasic language production. Clinical
Aphasiology, 23, 121–140.
Thompson, C. K., Walenski, M., Chen, Y. F., Caplan, D., Kiran, S., Rapp, B., . . . Parrish, T. B.
(2017). Intrahemispheric perfusion in chronic stroke-induced aphasia. Neural Plasticity,
Volume 2017, Article ID 2361691, 1–15.
Thompson, C. K., Weintraub, S., & Mesulam, M. (2012). Northwestern Anagram Test (NAT).
Evanston, IL: Northwestern University.
Thorne, J., & Faroqi-Shah, Y. (2016). Verb production in aphasia: Testing the division of labor
between syntax and semantics. Seminars in Speech and Language, 37(1),  23–33.
Thothathiri, M., Kimberg, D. Y., & Schwartz, M. F. (2012). The neural basis of reversible sen-
tence comprehension: Evidence from voxel-based lesion symptom mapping in aphasia.
Van Petten, C., & Luka, B. J. (2012). Prediction during language comprehension: Benefits, costs,
and ERP components. International Journal of Psychophysiology, 83, 176–190.
Vanier, M., & Caplan, D. (1990). CT correlates of agrammatism. In L. O. L. Menn & H.
Goodglass (Eds.), A cross-language study of agrammatism. New York: Benjamins.
Verreyt, N., Bogaerts, L., Cop, U., Bernolet, S., De Letter, M., Hemelsoet, D., . . . Duyck, W.
(2013). Syntactic priming in bilingual patients with parallel and differential aphasia.
Aphasiology, 27(7), 867–887.
Wagers, M. W., & Phillips, C. (2014). Going the distance: Memory and control processes in
active dependency construction. Quarterly Journal of Experimental Psychology, 67(7),
1274–1304.
Wang, H., Yoshida, M., & Thompson, C. K. (2014). Parallel functional category deficits in
clauses and nominal phrases: The case of English agrammatism. Journal of Neurolinguistics,
27(1), 75–102.
Wenzlaff, M., & Clahsen, H. (2004). Tense and agreement in German agrammatism. Brain and
Language, 89(1),  57–68.
Wicklund, M. R., Duffy, J. R., Strand, E. A., Machulda, M. M., Whitwell, J. L., & Josephs, K.
A. (2014). Quantitative application of the primary progressive aphasia consensus criteria.
Neurology, 82(13), 1119–1126.
Wierenga, C. E., Maher, L. M., Moore, A. B., White, K. D., McGregor, K., Soltysik, D.
A., . . . Crosson, B. (2006). Neural substrates of syntactic mapping treatment: An fMRI study
of two cases. Journal of the International Neuropsychological Society, 12(1), 132–146.
Williams, E. (1981). Argument structure and morphology. The Linguistic Review, 1, 81–114.
Wilson, S. M., Brandt, T. H., Henry, M. L., Babiak, M., Ogar, J. M., Salli, C., . . . Gorno-Tempini,
M. L. (2014). Inflectional morphology in primary progressive aphasia: An elicited produc-
tion study. Brain and Language, 136,  58–68.
Wilson, S. M., DeMarco, A. T., Henry, M. L., Gesierich, B., Babiak, M., Mandelli, M.
L., . . . Gorno-Tempini, M. L. (2014). What role does the anterior temporal lobe play in
sentence-level processing? Neural correlates of syntactic processing in semantic variant pri-
mary progressive aphasia. Journal of Cognitive Neuroscience, 26(5), 970–985.
Wilson, S. M., DeMarco, A. T., Henry, M. L., Gesierich, B., Babiak, M., Miller, B. L., & Gorno-
Tempini, M. L. (2016). Variable disruption of a syntactic processing network in primary pro-
gressive aphasia. Brain, 139, 2994–3006.
Wilson, S. M., Dronkers, N. F., Ogar, J. M., Jang, J., Growdon, M. E., Agosta, F., . . . Gorno-
Tempini, M. L. (2010). Neural correlates of syntactic processing in the nonfluent variant of
primary progressive aphasia. Journal of Neuroscience, 30(50), 16845–16854.
Wilson, S. M., Galantucci, S., Tartaglia, M. C., & Gorno-Tempini, M. L. (2012). The neural basis
of syntactic deficits in primary progressive aphasia. Brain and Language, 122(3), 190–198.
Neuron, 72(2), 397–403.
Wilson, S. M., Henry, M. L., Besbris, M., Ogar, J. M., Dronkers, N. F., Jarrold, W., . . . Gorno-
Tempini, M. L. (2010). Connected speech production in three variants of primary progres-
sive aphasia. Brain, 133(Pt 7), 2069–2088.
Wittenberg, E., Paczynski, M., Wiese, H., Jackendoff, R., & Kuperberg, G. (2014). The differ-
ence between “giving a rose” and “giving a kiss”: Sustained neural activity to the light verb
construction. Journal of Memory and Language, 73,  31–42.
Wittenberg, E., & Piñango, M. M. (2011). Processing light verb constructions. The Mental
Lexicon, 6(3), 319–413.
Xiang, H. D., Fonteijn, H. M., Norris, D. G., & Hagoort, P. (2010). Topographical functional
connectivity pattern in the perisylvian language networks. Cerebral Cortex, 20(3), 549–560.
Zimmerer, V. C., Cowell, P. E., & Varley, R. A. (2014). Artificial grammar learning in individuals
with severe aphasia. Neuropsychologia, 53,  25–38.
Zingeser, L. B., & Berndt, R. S. (1990). Retrieval of nouns and verbs in agrammatism and
anomia. Brain and Language, 39(1),  14–32.
Zurif, E., Swinney, D., Prather, P., Solomon, J., & Bushell, C. (1993). An online analysis of syn-
tactic processing in Broca’s and Wernicke’s aphasia. Brain and Language, 45(3), 448–464.
Chapter 32
Verbal Worki ng   Me mory
Bradley R. Buchsbaum
Introduction
In human verbal communication, there is often a period of time that intervenes be-
tween the sensory perception of a speech message and an appropriate response to that
message. Consider the plight of the Starbucks barista: a fastidious customer comes to
the front of the line and orders a complex expresso concoction. The cashier rings up
the order and repeats it aloud: “grande double decaf almond praline soy latte with skim
milk”—a verbal message containing all of 10 words and 15 syllables. The barista, mean-
while busy preparing a tall lightly foamed caramel machiatto for the previous customer,
must mentally record and retain in memory the content of the new request. Although
the spoken message describing an elaborate coffee drink has only a brief existence as a
stream of acoustic vibrations in the physical world, it must nevertheless be transmitted
and stably represented in the minds of three successive people. The customer produces
the message, the cashier repeats the message, and the barista must maintain the message
“in mind” until he or she has completed the order. When the drink arrives at the counter
with a cheerful: “sir, your grande double decaf almond praline soy latte with skim milk is
ready!” a bystander, uninitiated to this routine modern coffee-ordering spectacle, might
be forgiven for considering the fulfillment of the customer’s request to be something
of a miracle. From the standpoint of the study of human cognition, the success of this
commercial exchange depends critically not only on language competence—that is, the
ability to perceive and produce speech—but also the ability to consciously hold on to a
sequence of verbal information over a short period of time. In the terminology of cog-
nitive psychology and neuroscience, we refer to this ability to retain information in an
accessible state over brief periods of time as “working memory,” a cognitive system that
enables one to temporarily store and manipulate important pieces of information that
are no longer readily available in the sensory environment.
In this chapter, I will first provide a historical overview of how working memory
emerged as a concept in cognitive psychology. Next, I will describe the preeminent
828 Bradley R. Buchsbaum
cognitive model of verbal working memory, the phonological loop, and how it explains
critical laboratory phenomena associated with memory for verbal material. I will then
discuss the attempts and associated difficulties in mapping and otherwise situating the
phonological loop in the brain. Finally, I will cover the recent movement in the cogni-
tive neuroscience of language to view verbal working memory as emerging from the
language circuitry that underpins core language functions such as the perception and
production on speech.
The Development of the Concept

of Short-Term Memory
Viewing memory as comprising two main compartments, one for the current contents
of consciousness, and another for a more permanent record of experience, has gone
in and out of fashion in the last century. William James (1890) coined the terms “pri-
mary memory” and “secondary memory” to refer to these two basic concepts, setting
off a long-standing debate in the psychological sciences as to whether memory is a uni-
tary faculty or whether it can be divided into different sub-components. In the mid-
twentieth century most theorists viewed memory as a unitary system governed by a
single set of principles that were largely invariant over time (Melton, 1963; Underwood,
1957). However, in the 1960s, cognitive psychology offered evidence for the existence
of two memory systems, one for very recent events (short-term memory) and one for
events that occurred in the more distant past (long-term memory).
A key piece of evidence supporting the “dual-store” view of memory came from
studies of free recall. These studies showed that when subjects are presented with a list
of words and must recall them in any order (free recall), performance is best for the first
few items (the primacy effect) and for the last few items (the recency effect). When accu-
racy is plotted as a function of input order, it reveals a characteristic U-shaped pattern
(Davelaar, Goshen-Gottstein, Ashkenazi, Haarmann, & Usher, 2005; Glanzer & Cunitz,
1966; Waugh & Norman, 1965), which is referred to as the serial position curve. However,
if a short delay (e.g., 10 s) is placed between stimulus presentation and recall, during
which subjects are required to engage in some distracting activity, the shape of the serial
positive curve changed. Performance on early items (primacy) is relatively unaffected,
but the recency effect is abolished (Glanzer & Cunitz, 1966; Postman & Phillips, 1965).
Recency effects are attributed to a readout of the last few items in a list from short-term
memory, and primacy effects reflect a long-term memory advantage for the first few
items in a list due to the greater rehearsal devoted to those items. Moreover, recall from
the long-term store requires a more effortful and slow probabilistic form of retrieval that
depends more on associative, semantic, and contextual retrieval cues than does retrieval
from the short-term store. However, the preceding interpretation of patterns of recency
effects in immediate and delayed recall, as reflected in the operation of two stores, has
Verbal Working Memory 829
long been disputed, and is complicated by the demonstration of recency effects that can
span across minutes or even days (Bjork & Whitten, 1974; Crowder, 1982). One way to ex-
plain the existence of both short-and long-term recency effects is that they emerge from
a common cause. Howard and Kahana (2002), for example, have argued that recency
effects can be explained in terms of “temporal context,” which is an internal process
that continually evolves and drifts over time. During memory retrieval, this temporal
context is reinstated, leading to recall of items that were temporally co-occurring. Both
long-and short-term recency effects emerge because the temporal context at the offset
of a memory list is most distinctive, and therefore most likely to be accessed. This leads
to clustered recall of the most recent items of a list, and this phenomenon occurs over
short and long scales and has nothing to do with the existence of short-and long-term
memory stores. Although a more detailed discussion of this topic is beyond the scope of
this chapter, it is important to keep in mind that whereas the bulk of cognitive neurosci-
ence research assumes the existence of short-and long-term memory systems or stores,
there are alternative and, indeed compelling, views that deny them any such existence
(see also Davelaar et al. 2005; Sederberg et al. 2008; Lewandowsky et al., 2009).
In summary, short-term memory is a kind of limit of the online capacity of the brain’s
ability to hold and process information. Thus, short-term memory can be viewed as a
cup into which sensory information flows. The capacity of the cup is fixed, and is prone
to overflowing. The precise capacity of the cup varies across individuals (Unsworth &
Engle, 2007), although as George Miller (1956) memorably pointed out, it tends to hover
around a “magical number” of 7 plus or minus 2 (but see also Cowan, 2001). When in-
coming information exceeds the capacity of the cup, the spillover may yet still be re-
corded in a secondary container, that is, long-term memory.
Neurological Dissociation
of Short-and Long-Term Memory
One of the critical pieces of evidence supporting the existence of separable systems for
short-and long-term memory was the discovery of patients with brain damage who
appeared to have selective deficits affecting only long-term or only short-term memory.
The most famous example of such a dissociation was the case of the patient H.M., whose
medial temporal lobes were famously removed as a treatment of intractable epilepsy.
The surgery resulted nearly complete loss in the ability to form new long-term declar-
ative memories (Corkin, 2002; Scoville & Milner, 1957). H.M. and other patients with
bilateral medial temporal lobe lesions that have subsequently been described (Squire,
Stark, & Clark, 2004) live in a kind of “permanent present tense” (Corkin, 2013), un-
able to consciously recall events that occurred even a few minutes ago. Notwithstanding
this severe impairment in the ability to form new long-term declarative memories,
such patients appear to have little or no deficit on tests of short-term memory, such as
repeating back short strings of digits (Baddeley & Warrington, 1970; Wickelgren, 1968),
although deficits in short-term memory have been sometimes been observed in these
patients in tests using novel visual objects (see Ranganath & Blumenfeld, 2005) or in
tests of short-term associative memory (Olson, Page, Moore, Chatterjee, & Verfaellie,
2006; Ryan & Cohen, 2004; Yee, Hannula, Tranel, & Cohen, 2014).
The neurobiological distinction between short-term and long-term memory was
strongly supported by the discovery of patients with severely impaired short-term
memory for numbers and words, together with a preserved ability to learn supra-span
(e.g., greater than 10 items) word lists with repeated study (Baddeley, Papagno, & Vallar,
1988; Basso, Spinnler, Vallar, & Zanobio, 1982; Shallice & Warrington, 1970; Warrington
& Shallice, 1969). Thus, in contrast to patient H.M., these patients were shown to be
able to form new long-term (verbal) memories, and yet had little or no verbal short-
term memory. Whereas H.M.’s memory deficit was a general memory impairment that
applied to all forms of declarative information (e.g., verbal, visual, spatial), these par-
ticular “short-term memory patients,” as they were to be called, had deficits confined
to the auditory-verbal modality. The short-term memory deficits by these patients,
in the purest cases (Shallice & Butterworth, 1977; Shallice & Vallar, 1990; Shallice &
Warrington, 1977; Takayama, Kinomoto, & Nakamura, 2004; Vallar & Baddeley, 1984;
Vallar, Di Betta, & Silveri, 1997), are not accompanied by severe deficits in language
comprehension and production. Thus, patient J.B. (Shallice & Butterworth, 1977) was
able to carry on conversations normally and to speak fluently without abnormal pauses,
errors, or other aphasic symptoms. What this seemed to show was that verbal storage
is not “built in” to the language-processing system, but is an independent entity in its
own right, a memory buffer that exists not to support language per se, but rather as a
passive “holding place” where recently encountered linguistic information can be tem-
porarily stored. Thus, the discovery of “short-term memory patients” (Vallar, 2006) es-
tablished a double dissociation both in brain localization (LTM: medial temporal lobe;
verbal STM: temporoparietal cortex) and patterns of performance, between short-and
long-term memory systems. In addition, the short-term memory disorder could be
distinguished from severe disorders of language production and comprehension such
as Broca’s aphasia, Wernicke’s aphasia, and other of the classic neurological language
impairments.
Working Memory
The Working Memory model of Baddeley and colleagues (Baddeley, 1986, 2000, 2003;
Baddeley & Hitch, 1974) was developed with an aim of explaining the relevant behav-
ioral findings in the memory literature while also taking into account important neuro-
psychological case study reports, such as those reviewed in the preceding. In addition,
whereas prior models of short-term memory tended to emphasize storage buffers as the
receptacles for information arriving from the senses, Baddeley and Hitch (1974) focused
on rehearsal processes, that is, strategic mechanisms for the maintenance of items in
memory. Baddeley and Hitch (1974) attempted to account for a system that could si-
multaneously manipulate the current contents of memory and update information
in working memory in the service of task goals. Such a system is especially important
when one needs to maintain information over short periods in many complex cognitive
activities, such as reading, mental calculation, spatial reasoning, and so forth.
Research has shown that in tests of serial recall, when subjects are prevented from
engaging in subvocal rehearsal during a delay period that is inserted between stim-
ulus presentation and recall, overall performance suffers (e.g., Baddeley, Thomson, &
Buchanan, 1975). In the case of verbal material, this suggested that the ability to keep
verbal sequences in working memory depends on covert articulatory processes. This
insight was central to the development of the verbal component of working memory,
the “phonological loop” (see later discussion), and led to a broader conceptualization
of short-term memory that sought to explain not only how and why information enters
and exits awareness, but also how resources are deployed in a strategic effort to capture
and maintain the objects of memory in conscious awareness.
The major principles of the Working Memory model are that it is (1) a limited ca-
pacity system (i.e., there is only a finite amount of information directly available for pro-
cessing in memory); (2) that specialized subsystems devoted to the representation of
information of a particular type, for instance, verbal or visuospatial, are structurally in-
dependent of one another (i.e., the information represented in one domain is protected
from the interfering effects of information that may be arriving to another domain);
and, finally, (3) that storage of information in memory is distinct from the processes that
underlie sensory perception; instead, there is a two-stage process whereby sensory in-
formation is first analyzed by perceptual modules and then transferred into specialized
storage buffers that have no other role but to temporarily “hold” pre-processed units of
information. Moreover, the pieces of information that reside in these specialized buffers
are subject to passive, time-based decay as well as inter-item interference (e.g., similar-
sounding words like “man, mad, map, cap, mad” can lead to interference within a spe-
cialized phonological storage structure); finally, the storage buffers have no built-in or
internal mechanism for maintaining or otherwise refreshing their contents—rather,
this must occur from without, through the process of rehearsal, which might be a motor
or top-down control mechanism that can sequentially access and refresh the contents
that remain active within the store.
The Working Memory model, first proposed by Baddeley and Hitch (1974) and later
refined (Baddeley, 1986, 2000, 2003; Salame & Baddeley, 1982), argued for the exist-
ence of three functional components of working memory. The “central executive” was
envisioned as a control system of limited attentional capacity, responsible for coordi-
nating and controlling two subsidiary slave systems, a phonological loop and a visuospa-
tial sketchpad. The phonological loop was responsible for the storage and maintenance
of information in a verbal form, and the visuospatial sketchpad was dedicated to the
storage and maintenance of visuospatial information. In the last decade, a fourth com-
ponent, the “episodic buffer,” has been added to the model in order to capture a number
of phenomena related to interactions between short-and long-term memory that could

not be readily explained within the original framework. In the following section, we will
describe the phonological loop in more detail, as it is the central component underlying
working memory for verbal material.
The Phonological Loop
As we have noted, the Working Memory model entails a separation of domain-specific

mechanisms of memory maintenance and domain-general mechanisms of execu-
tive control (Figure 32.1). Thus, the verbal component of working memory, or the
phonological loop, is regarded as a “slave” system that is under the supervisory con-
trol of the central executive component. Within the phonological loop, two interacting
components—the phonological store and the articulatory rehearsal process—enable
verbal representations to be maintained in an active state.
The phonological store is a passive buffer in which verbal information can be stored
for brief (approximately 2-s) periods. The articulatory control process serves to re-
fresh the contents of the store, thereby allowing the system to maintain sequences of
verbal items in memory over some interval of time. This division of labor between two
Auditory input Visual input
Phonological analysis Visual analysis & STS
Orthographic to
phonological
recoding
Phonological Rehearsal process Phonological

STS output buffer
Long-term
Spoken output
Verbal Memory
Figure 32.1. An anatomo-functional model of phonological short-term memory.

Abbreviation: STS = superior temporal sulcus.
Source: Based on Vallar et al. (1997), and Baddeley, Gathercole, and Papagno (1998).
interlocking components, one an active process and the other a passive store, is crucial
to the model’s explanatory power. For instance, when the articulatory control process
is interfered with through the method of articulatory suppression (e.g., by requiring
subjects to say “hiya” over and over again), items in the store rapidly decay, and re-
call performance suffers greatly. The phonological store, then, lacks a mechanism of
reactivating its own contents but possesses memory capacity, whereas the articulatory
rehearsal process lacks an intrinsic memory capacity of its own, but can exert its effect
indirectly by refreshing the contents of the store.
The phonological loop model of verbal working memory has stood the test of time
largely because it explains many of the behavioral phenomena associated with verbal
memory performance in a simple and intuitive way. It is important to briefly note
what these core behavioral phenomena are, and how the phonological loop model ac-
counts for them. The appeal of the model comes partly from its parsimony—with only
a very minimal set of functional specifications, it is able to account for a large number
of behavioral findings. It therefore provides a benchmark that any competing model,
whether neural or purely cognitive, must be able to match. In the paragraphs that follow,
I will give an overview of how the phonological loop explains certain well-established
behavioral phenomena associated with verbal working memory, namely, the phonolog-
ical similarity effect, the word-length effect, the effect of articulatory suppression, and the
irrelevant sound effect (Repovs & Baddeley, 2006). The phonological similarity effect
refers to the finding that similar sounding sets of words are more difficult to retain in
memory than sets of phonologically dissimilar words (Conrad & Hull, 1964). The locus
of this effect is the phonological store, and it results from the increased amount of in-
terference that occurs between memory traces that share overlapping representational
(e.g., phonemic) features, relative to those that do not.
The word-length effect simply refers to the fact that lists of words that take more time
to articulate—longer words—are more poorly remembered than words that take less
time to articulate (Baddeley et al., 1975; Mueller, Seymour, Kieras, & Meyer, 2003). This
occurs not only between sets of words that have different numbers of syllables, but also
for sets of words that are equated for number of syllables but are, nevertheless, unequal
in absolute articulatory duration. The effect is explained by assuming that items in the
phonological store suffer time-based decay that can only be reversed by way of articu-
lation. Thus, as the articulatory loop cycles through a set of long words, the overall time
elapsed between successive iterations will be greater, and, therefore, the probability
that one of the several items in the store may have (irretrievably) decayed will be con-
sequently increased. This effect, then, is jointly determined by the properties of the re-
hearsal process (rate of articulation) and that of the phonological store (rate of decay).
The negative effect of articulatory suppression on recall performance is observed
when subjects are prevented from using inner speech either during presentation or
during a delay inserted before recall. Thus, as articulatory suppression interferes with
the articulatory rehearsal process, the mechanism that is ordinarily used to refresh the
items in the phonological store, the system is therefore unable to counteract trace decay,
thus leading to a decline in recall performance.
The irrelevant sound effect occurs when the to-be-remembered verbal stimuli are
accompanied by a stream of unattended auditory information (Macken, Mosdell, &
Jones, 1999; Salame & Baddeley, 1982; Tremblay, Nicholls, Alford, & Jones, 2000). These
“irrelevant sounds” need not be in the speaker’s native language or even phonemic to be
disruptive, provided there is some degree of variation in the sound stream. For instance,
a single tone or even white noise does not have an effect, although a changing sequence
of tones does cause impairment (Jones, Madden, & Miles, 1992). The locus of the irrele-
vant sound effect is in the phonological store, where the incoming acoustic information
interferes with the to-be-remembered items in the store. Because the presentation of ir-
relevant visual-verbal information does not have an effect on recall, it is assumed that
auditory information has obligatory access to the store, whereas visual-verbal informa-
tion does not. How, then, does visual-verbal information enter the phonological store?
The answer, supported by several lines of evidence (Baddeley, Lewis, & Vallar, 1984; Levy,
1971), is that textual information must first be recoded phonologically before it can enter
the store. This recoding process, moreover, requires the involvement of the articulatory
rehearsal process, as subvocalization is necessary to reroute visually derived verbal in-
formation into the phonological store. In support of this contention is the finding that
articulatory suppression abolishes the phonological similarity effect for visual, but not
auditory, presentation. Because auditory information has obligatory access to the store,
articulatory suppression has no effect on its deposition within the store. For visual pre-
sentation, however, articulatory suppression ties up the rehearsal system, preventing
phonological recoding of visual-verbal material and, consequently, blocking subvocally
mediated access to the store. In the preceding sections, we briefly outlined the main
components of the phonological loop, as well as the manner in which its architecture and
functional characteristics account for certain reliable effects observed in studies of verbal
short-term memory. We should make clear that it is not universally accepted that every
detail of the phonological loop is perfectly supported by available evidence. For instance,
there is a great deal of debate about whether the word-length effect is actually caused by
an increase in the absolute spoken duration of the items, or whether it is better explained
by, for instance, the phonological complexity of the items (Caplan, Rochon, & Waters,
1992; Mueller et al., 2003).
In summary, the Working Memory memory model of Baddeley and colleagues
describes a system for the maintenance and manipulation of information that is stored
in domain-specific memory buffers. Separate cognitive components are dedicated to
the functions of storage, rehearsal, and executive control. Informational encapsulation
and domain segregation dictates that auditory-verbal information and visual informa-
tion are kept in separate storage subsystems—the phonological loop and the visuospa-
tial sketchpad, respectively. These storage subsystems themselves comprise specialized
components for the passive storage of memory traces, which are subject to time and
interference-based decay, and for the reactivation of these memory traces by way of
simulation, or rehearsal. Thus, storage components represent memory traces, but have
no internal means of refreshing them, whereas rehearsal processes (e.g., articulatory)

have no mnemonic capacity of their own, but can reactivate the decaying traces held in
temporary stores. In the next sections, we will examine how neuroscience has built on
the cognitive foundation of the Working Memory model of Baddeley and colleagues
to refine our understanding of how information is maintained and manipulated in the
brain. We will see that in some cases, neuroscientific evidence has bolstered and rein-
forced aspects of the Working Memory model, whereas in other cases neuroscience has
compelled a departure from certain core principles of the Baddeleyan concept.
The Neuroscientific Basis of Verbal

Working Memory
Research on the neural basis of verbal working memory has some unique challenges
and is in several ways more difficult to study than, for example, visual or spatial
working memory. For example, whereas in visual working memory many of the
most influential ideas and concepts have derived from work in nonhuman primates
and other animals, verbal working memory is a uniquely human phenomenon, and
has therefore benefited from animal research only in terms of broad neuroscience
principles.
Even research on the primary modality relevant to verbal working memory, that
of audition, is surprisingly scarce in the monkey literature, owing to the difficulty in
training nonhuman primates to perform delayed response tasks with auditory stimuli,
which can take upward of 15,000 learning trials (Fritz, Mishkin, & Saunders, 2005).
Because of the lack of animal work on verbal working memory, neuroscience has
often looked to cognitive psychological models, such as the Working Memory model
of Baddeley and colleagues, for a sensible framework for interpreting neuroscientific
evidence about verbal working memory. Of course, many of the classic neurolog-
ical studies of language were carried out before “working memory” existed as a con-
cept, let alone as an object of avid neuroscience inquiry. Nevertheless, the core idea
of Baddeley and colleagues’ phonological loop—namely, that verbal information can
be reciprocally transferred between auditory and motor components—has a clear cog-
nate in the Wernicke-Lichtehim-Geschwind model of language organization, the core
of which originated in Carl Wernicke’s 1874 monograph on the aphasias (Geschwind,
1965b; Wernicke, 1874; see, in this volume, Blumstein, Chapter 1, and Wilson,
Chapter 2). A fundamental challenge to the cognitive neuroscience of verbal working
memory has been to integrate both empirical data and theoretical constructs that have
emerged from different subfields—cognitive, neurological, neuropsychological—of
inquiry in the brain and behavioral sciences.
Neurological Studies of Language

and Verbal Short-Term Memory
Early neurological investigations of patients with language disturbances, or aphasia, re-

vealed that lesions to specific parts of the cerebral cortex could cause selective deficits
in language abilities (Goodglass, 1993). Thus, lesions to the inferior frontal gyrus and
surrounding cortex are associated with Broca’s aphasia, a disorder that causes severe
impairments in speech production. Broca’s aphasia is not, however, a disorder of periph-
eral motor coordination, such as the ability to move and control the tongue and mouth,
but rather is a disorder of the ability to plan, program, and access the motor codes re-
quired for the production of speech (Goodglass, 1993; Mohr et al., 1978). Lesions to the
posterior superior temporal gyrus (STG) and surrounding cortex, on the other hand, are
associated with Wernicke’s aphasia, a complex syndrome that is characterized by fluent
but error-filled production, and poor comprehension and perception of speech. A third,
less common syndrome, called conduction aphasia, typically caused by lesions in the au-
ditory cortex and posterior Sylvian region (generally less extensive and relatively superior
to lesions causing Wernicke’s aphasia), is associated with relatively preserved speech per-
ception and comprehension, occasional errors in otherwise fluent spontaneous speech
(e.g., phoneme substitutions), and severe difficulties with verbatim repetition of words
and sentences (Axer, 2001; Baldo & Dronkers, 2006; H. Damasio & Damasio, 1980).
From the standpoint of verbal short-term memory, there are a number of impor-
tant points to be drawn from these three classic aphasic syndromes. First, the neural
structures that underlie the perception and production of speech are partly disso-
ciable (see, in this volume, Tremblay, Deschamps, & Dick, Chapter 15, and Hickok,
Chapter 20). While it is tempting to postulate that posterior temporal lesions prima-
rily affect receptive language functions and anterior lesions affect productive language
functions, this is not quite true: both Wernicke’s aphasia and conduction aphasia are
caused by posterior lesions, yet only the former is associated with a major receptive
language disturbance, while both syndromes involve a deficit in speech production.
Moreover, lesions in and around the inferior frontal gyrus (Broca’s area), although
normally associated with deficits in speech production (labored, nonfluent speech),
can also lead to deficits in the comprehension of grammatically complex sentences
and subtle deficits in speech perception (Berndt, Mitchum, & Wayland, 1997; Caplan,
Baker, & Dehaut, 1985; see Thompson & Mack, Chapter 31 in this volume). Second,
the previously mentioned disorders affect basic aspects of language processing, such
as the comprehension, production, and perception of speech. Third, the classical
Wernicke-Lichteim-Geschwind (Geschwind, 1965a) model of language explains each
of these three syndromes as disruptions to components of a neuroanatomical net-
work of areas, in the inferior frontal and superior temporal cortices, that subserve
language function. Finally, it should not be surprising that aphasic syndromes associ-
ated with poor language performance have also been shown to affect verbal working
memory (Burgio & Basso, 1997; De Renzi & Nichelli, 1975). This cannot be taken to
necessarily imply that verbal working memory is a function of the core language net-
work. For example, as we have noted earlier, input to the storage component of the
phonological loop theoretically depends on a functioning language system, while not
itself participating in core language processes such as speech perception and speech
production. Thus, the association between language disturbances and verbal working
memory impairment does not prove that the two systems are functionally equivalent.
This is especially true in light of evidence from neuropsychology, discussed earlier,
showing that verbal short-term memory impairment need not accompany a basic im-
pairment to the language faculty. An interpretation consistent with the phonological
loop model of verbal working memory is that a selective deficit to verbal short-term
memory is caused by lesions that have damaged the phonological store while leaving
language perception and production centers intact. Moreover, these patients offer cre-
dence to the conceptualization of memory, exemplified by the phonological loop, as a
distinct entity in its own right, whose functional purpose is to store and maintain infor-
mation in mind, rather than to analyze and process incoming sensory input. Although
the evidence for such a functional dissociation on the basis of these “short-term memory
patients” is intriguing and compelling, the anatomical localization of these lesions, as we
shall see, presents cognitive neuroscience with a difficult puzzle.
With respect to the “short-term memory patient” discussed earlier, one must ask how
a lesion in the middle of the perisylvian speech center encompassing the temporoparietal
area could produce such a pure short-term memory deficit without any collateral im-
pairment in basic online language functioning. One possibility is that the precise loca-
tion of the brain injury is critical, so that a particularly focal and well-placed lesion in
temporoparietal cortex might spare cortex critical for speech perception and speech pro-
duction, while damaging a region dedicated to the temporary storage of auditory-verbal
information. Indeed, this is exactly what would be predicted if the “phonological store”
component of Baddeley’s Working Memory model were to be specifically damaged.
The number of patients that have been described with a selective impairment to
auditory-verbal short-term memory is small, however, and the lesion locations that
have been reported are not clearly distinguishable from those that might, in another pa-
tient, have led to conduction or Wernicke’s aphasia (Baldo & Dronkers, 2006; Damasio,
1992; Goodglass, 1993). One way that has been used to address this puzzle is to use func-
tional neuroimaging to investigate the extent to which the brain regions supporting ge-
neral language processing (perception and production of speech) are dissociable from
regions that are recruited during verbal short-term memory.
Functional Neuroimaging Studies

of Verbal Working Memory
The first study that attempted to localize the components of phonological loop in the
brain was that of Paulesu and colleagues (1993). In one task, English letters were visually
presented on a monitor and subjects were asked to remember them. In a second task,
letters were presented and rhyming judgments were made about them (press a button if
letter rhymes with “B”). In a baseline condition, Korean letters were visually presented
and subjects were asked to remember them using a visual code. According to the authors’
logic, the first task would require the contribution of all the components of the pho-
nological loop—subvocal rehearsal, phonological storage, and executive processes—
while the second (rhyming) task would only require subvocal rehearsal and executive
processes. This reasoning was based on previous research showing that when letters are
presented visually (Vallar & Baddeley, 1984), rhyming decisions engage the subvocal
rehearsal system, but not the phonological store. Thus, a subtraction of the rhyming
condition from the letter-rehearsal condition should isolate the neural locus of the pho-
nological store. First, results were presented for the two tasks requiring phonological
processing with the baseline tasks (viewing Korean letters) that did not. Several areas
were shown to be significantly more active in the “phonological” tasks, including (in all
cases, bilaterally): Broca’s area (BA 44/45), the supplementary motor cortex (SMA), the
insula, the cerebellum, Brodmann area 22/42, and Brodmann area 40. Subtracting the
rhyming condition from the phonological short-term memory condition left a single
brain area: Brodmann area 40—the neural correlate of the phonological store.
The articulatory rehearsal process recruited a distributed neural circuit that in-
cluded the inferior frontal gyrus, cerebellum, supplementarty motor area, and premotor
cortex. Activation of multiple brain regions during articulatory rehearsal is not sur-
prising, given the complexity of the process and the variety of lesion sites associated
with a speech production deficit. On the other hand, the localization of the phonological
store in a single brain region, BA 40 (or the supramarginal gyrus of the parietal lobe),
fits well with the idea of a “receptacle” where phonological information is temporarily
stored. A number of follow-up positron emission tomography (PET) studies, using var-
ious tasks and design logic, generally replicated the basic finding of the Paulesu et al.
(1993) study, namely, a fronto-insular-cerebellar network associated with rehearsal
processes, and a parietal locus for the phonological store (Awh et al., 1996; Jonides et al.,
1998; Salmon et al., 1996; Schumacher et al., 1996; Smith & Jonides, 1999). In short, early
PET studies of verbal working memory were interpreted as broadly supporting the ar-
chitecture of the phonological loop, with storage being subserved by a left parietal lobe
structure outside the core language network, and articulatory rehearsal associated with
regions known to be involved in motor speech planning.
Becker et al. (1999), however, questioned whether the localization of the phonological
store in the left parietal cortex could be reconciled with the logical architecture of the
phonological loop. For instance, as reviewed earlier, a key element of the phonological
loop model is that auditory information (whether it be speech, tones, music, or white
noise), but not visual information, has obligatory access the phonological store. The
reason for this difference is to account for dissociations in memory performance that
depend on the modality in which information is presented. For instance, the presenta-
tion of distracting auditory information while subjects attempt to retain a list of verbal
items in memory impairs performance on tests of recall. In contrast, the presentation
of distracting visual information during verbal memory retention has no impact on

verbal recall. This phenomenon—the irrelevant sound effect—is explained by assuming
that auditory information always enters the phonological store, but that visual-verbal
information only enters the store when it is explicitly subvocalized. Becker et al., how-
ever, argued that if indeed auditory information has automatic access to the phonolog-
ical store, its “neural correlate” should be active even during passive auditory perception.
Functional neuroimaging studies of passive auditory listening (e.g., with no memory
component), however, do not show activity in the parietal region that had been asso-
ciated in previous studies with phonological storage, but rather show activation that is
largely confined to the superior temporal lobe (e.g., Binder et al., 2000).
A second difficulty with a parietal locus of the phonological store is that efforts to
show verbal mnemonic specificity to maintenance-related activity in the parietal lobe
have not been successful (Chein, Ravizza, & Fiez, 2003). Instead, it has been shown that
working memory for words, visual objects, and spatial locations all activate the area
(Badre, Poldrack, Pare-Blagoev, Insler, & Wagner, 2005; Niendam et al., 2012; Nystrom
et al., 2000; Zurowski et al., 2002). Thus, if there were a perfect “neural correlate” of the
phonological store, it must reside within the confines of the auditory cortical zone of the
superior temporal cortex.
Event-R elated fMRI Studies of Verbal

and Auditory Working Memory
Studies using event-related fMRI (see Heim & Specht, Chapter 4 in this volume), with its
ability to isolate delay-period activity during working memory, have greatly improved
our understanding of the neural circuitry associated with verbal working memory main-
tenance. Postle et al. (1999) showed, with visual-verbal presentation of letter stimuli,
that delay-period activity in single subjects was often localized in the posterior superior
temporal cortex, rather than the parietal lobe that was typically identified in the early
PET studies reviewed earlier. Buchsbaum et al. (2001) also used an event-related fMRI
paradigm, in which, on each trial, subjects were presented with acoustic speech infor-
mation that they then rehearsed subvocally for 27 seconds, followed by a rest period.
Analysis focused on identifying regions that were responsive both during the percep-
tual phase and the rehearsal phase of the trial. Activation occurred in two regions in the
posterior superior temporal cortex, one in the posterior superior temporal sulcus (STS)
bilaterally, and one along the dorsal surface of the left posterior planum temporale, that
is, in the Sylvian fissure at the parietal-temporal boundary (area Spt). Notably, while the
parietal lobe did show delay-period activity, it was unresponsive during auditory stim-
ulus presentation. In a follow-up study, Hickok et al. (2003) showed that the same supe-
rior temporal regions (posterior STS and Spt) were active both during the perception
and delay-period maintenance of short (5 s) musical melodies, suggesting that these
posterior temporal storage sites are not restricted to speech-based, or “phonological,”

information (Figure 32.2).
Several subsequent studies have confirmed the role of Spt in internal rehearsal of
musical and speech sequences (Hashimoto, Lee, Preus, McCarley, & Wible, 2010;
Hickok, Okada, & Serences, 2009; Koelsch et al., 2009; see Ogg & Slevc, Chapter 35
in this volume). Acheson et al. (2011) used fMRI to identify posterior temporal re-
gions activated during verbal working memory maintenance, and then used repeti-
tive transcranial magnetic stimulation (TMS) to these sites while subjects performed
a rapid-paced reading task that involved language production but no memory load.
TMS (see Schuhmann, Chapter 5 in this volume) applied to the posterior temporal area
(a) 2
1.5
1
listen Music
0.5
Z score
listen Speech
0 repeat Music
repeat Speech
–0.5
–1
–1.5 release rest
2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36
Time (seconds)
(b)
maintain music maintain speech
Figure 32.2. Main results from Hickok et al. (2003) study of verbal and musical working
memory maintenance. (A) Averaged time course of activation over the course of a trial in area
Spt for speech and music conditions. Timeline at bottom shows structure of each trial; black
bars indicate auditory stimulus presentation. Red traces indicate activation during rehearsal
trials, black traces indicate activity during listen-only trials in which subjects did not rehearse
stimuli at all. (B) Activation maps of in the left hemisphere (sagittal slices) showing three re-
sponse patterns for both music rehearsal (left) and speech rehearsal trials (right): auditory-only
responses shown in green; delay-period responses shown in blue; and auditory + rehearsal
responses shown in red. Arrows indicate the location of area Spt.
Source: Reproduced from Hickok et al. (2003).
significantly interfered with paced reading, arguing for common neural substrate for
language production and verbal working memory. Fegen et al. (2015) has shown that
area Spt, the dorsal premotor cortex, and the inferior frontal gyrus are modulated by
both the rate of covert rehearsal (number of items rehearsed per second) and the overall
memory load (total number of items to retain in memory). However, regions associated
with “domain-general” cognitive control processing, the dorsolateral prefrontal cortex
(PFC) and posterior parietal cortex, are only modulated by memory load. This finding
reinforces the notion that verbal working memory maintenance is a distributed opera-
tion that depends on the cooperation of multiple networks to store and maintain pho-
nological information over time.
Stevens (2004) and Rama et al. (2004) have shown that memory for voice identity, in-
dependent of phonological content (i.e., matching speaker identity as opposed to word
identity), selectively activates the mid-STS and the anterior STG of the superior tem-
poral region, but not the more posterior and dorsally situated Spt region. Buchsbaum
et al. (2005) have further shown that the mid-STS is more active when subjects recall
verbal information that is acoustically presented than when the information is visually
presented, whereas area Spt shows equally strong delay-period activity for both auditory
and visual forms of input (see also Hashimoto et al., 2010). This finding is supported by
regional analyses of structural MRI in large groups of patients with brain lesions that
have showed that damage to the STG is most predictive of auditory short-term memory
impairment (Koenigs et al., 2011; Leff et al., 2009). Leung et al. (2011) have also shown
a dissociation between auditory object and spatial working memory, with the former
activating more ventral stream auditory areas and the latter activating the dorsal pari-
etal lobe. Thus, it appears that different regions in the temporoparietal area are attuned
to different qualities or features of a verbal stimulus, such as voice information, input
modality, phonological content, spatial location, and lexical status (e.g., Martin &
Freedman, 2001)—and all of these codes may play a role in the short-term maintenance
of verbal information.
Additional support for a feature-based topography of auditory association cortex
comes from neuroanatomical tract-tracing studies in both monkeys and humans
that have revealed separate temporo-prefrontal pathways arising along the anterior-
posterior axis of the superior temporal region (Bavelier et al., 1998; Rauschecker, 2011;
Romanski, 2004; Romanski et al., 1999; Saur, Kreher, & Schnell, 2008). The posterior
part of the STG projects to dorsolateral PFC (BA 46, 8), whereas neurons in the an-
terior STG are more strongly connected to the ventral PFC, including BA 12 and 47.
Several authors have suggested, similar to the visual system, a dichotomy between
ventral-going auditory-object and dorsal-going auditory-spatial processing streams
(Arnott, Binns, Grady, & Alain, 2004; Rauschecker & Tian, 2000; Tian, Reser, Durham,
Kustov, & Rauschecker, 2001). Thus, studies have shown that the neurons in the rostral
STG show more selective responses to classes of complex sounds, such as vocalizations,
whereas more caudally located regions show more spatial selectivity (Chevillet,
Riesenhuber, & Rauschecker, 2011; Rauschecker & Tian, 2000; Tian et al., 2001).
Hickok and Poeppel (2007) have proposed that human speech processing also pro-
ceeds along diverging auditory dorsal and ventral streams, although they emphasize the
distinction between perception for action, or auditory-motor integration, in the dorsal
stream and perception for comprehension in the ventral stream (see Poeppel, Cogan,
Davidesco, & Flinker, Chapter 26 in this volume). Buchsbaum et al. (2005) have shown
with fMRI time series data that, consistent with the monkey connectivity patterns, the
most posterior and dorsal part of the superior temporal cortex, area Spt, shows the
strongest functional connectivity with dorsolateral and posterior (premotor) parts of
the PFC, while the mid-portion of the STS is most tightly coupled with BA 12 and 47 of
the ventrolateral PFC (see Figure 32.3). Moreover, gross distinctions between anterior
(BA 47) and posterior (BA 44/6) parts of the PFC have been associated with conceptual-
semantic and phonological-articulatory aspects of verbal processing (Poldrack et al.,
1999; Wagner, Pare-Blagoev, Clark, & Poldrack, 2001). fMRI studies have also shown
that maintenance of verbal-semantic information relies to a greater extent on the
anteroventral aspects of the temproal lobe than does mainteance of phonological in-
formation (e.g., nonword sequences) (Fiebach, Rissman, & D’Esposito, 2006; Shivde &
Thompson-Schill,  2004).
Taken together, findings from functional neuroimaging have shown that the main-
tenance of verbal information in working memory relies on a distributed network of
primarily frontal, temporal, and parietal brain regions. The particular topography of ac-
tivation depends on the content matter of the to-be-remembered stimuli and/or current
task goals. Moreover, activation patterns during memory for musical or tonal sequences
overlaps considerably with that for phonological sequences. There does not appear to be
Figure 32.3. Map of functional connectivity delay period maintenance of verbal stimuli from
Buchsbaum et al. (2005). Seed regions for correlation analysis are denoted by stars located in area
Spt and the middle part of the STS. Warm colors show areas more strongly correlated with Spt
than with STS; Cold colors show areas more strongly correlated with STS than Spt. Inset shows
temporal-prefrontal connectivity in the monkey.
Abbreviation: STS = superior temporal sulcus.
Source: Inset reproduced from Romanski et al. (2001).
a single brain region where verbal information is passively stored, as would be predicted
by the phonological loop model. Rather, it seems that short-term mnemonic storage
is embedded in the very neural structures that support the auditory-verbal perception
and production, and that these areas comprise a distributed fronto-temporo-parietal
network.
Reconciling Neuropsychological
and Functional Neuroimaging Data
Earlier we posed the question as to how a lesion in posterior Sylvian cortex, an area of
known importance for online language processing, could occasionally produce an im-
pairment restricted to phonological short-term memory. One solution to this puzzle
is that subjects with selective verbal short-term memory deficits from temporoparietal
lesions retain their perceptual and comprehension abilities due to the sparing of the
ventral stream pathways in the lateral temporal cortex, whereas the preservation of
speech production is due to an unusual capacity in these subjects for right-hemisphere
control of speech (Buchsbaum & D’Esposito, 2008; Hickok & Poeppel, 2004; Nadeau,
2001). The short-term memory deficit arises, then, from a selective deficit in auditory-
motor integration—or the ability to translate between acoustic and articulatory speech
codes—a function that is especially taxed during tests of repetition and short-term
memory (Buchsbaum & D’Esposito, 2008). Conduction aphasia, the aphasic syn-
drome most often associated with a deficit in auditory repetition and verbal short-term
memory in the absence of any difficulty with speech perception, may reflect a disorder
of auditory-motor integration (see Hickok, Chapter 20 in this volume). Indeed, it has
recently been shown that the lesion site most often implicated in conduction aphasia
circumscribes area Spt in the posteriormost portion of the superior temporal lobe, a
link between a disorder of verbal repetition and a region in the brain often implicated
in tasks of verbal working memory (Buchsbaum, Baldo, Okada, & Berman, 2011; see
Figure 32.4). Thus, impairment in the ability to temporarily store verbal information,
as occurs in conduction aphasia, may result from damage to a system, area Spt, which is
critical for the interfacing of auditory and motor representations of sound.
Summary and Conclusions
Elucidation of the cognitive and neural architectures underlying verbal working

memory has been an important focus of neuroscience research for much of the past two
decades. The emergence of the concept of working memory, with its emphasis on the
utilization of the information stored in memory in the service of behavioral goals, has
Conduction Aphasia fMRI Activation Map Maximal Overlap

Lesion Overlap (Encoding & Rehearsal) Lesion & fMRI
8% 85% 0% 50%
Figure 32.4. A comparison of conduction aphasia, phonological working memory in fMRI,

and their overlap. The leftmost panel surface shows the regional distribution lesion overlap in
patients with conduction aphasia (max is 12/14, or 85% overlap). Middle panel shows the per-
centage of subjects showing maintenance-related activity in a phonological working memory
task. The right panel shows the area of maximal overlap between the lesion and fMRI surfaces
(lesion > 85% overlap and significant fMRI activity for conjunction of encoding and rehearsal).
Source: Reproduced from Buchsbaum et al. (2011).
enlarged our understanding and broadened the scope of neuroscience research of short-
term memory. Data from numerous studies have been reviewed and have demonstrated
that a network of brain regions, principally in the temporal and frontal lobes, is crit-
ical for the active maintenance of internal verbal representations. It is clear from these
investigations that verbal memory cannot be localized to a single brain region, but is
rather an emergent property of the functional interactions between the frontal and pos-
terior neocortical regions. Numerous questions remain about the neural basis of this
complex cognitive system, but studies such as those reviewed in this chapter should
continue to provide converging evidence that may provide answers to the many residual
questions.
References
Acheson, D. J., Hamidi, M., Binder, J. R., & Postle, B. R. (2011). A common neural substrate for
language production and verbal working memory. Journal of Cognitive Neuroscience, 23(6),
1358–1367.
Arnott, S. R., Binns, M. A., Grady, C. L., & Alain, C. (2004). Assessing the auditory dual-
pathway model in humans. NeuroImage, 22(1), 401–408.
Awh, E., Jonides, J., Smith, E. E., Schumacher, E. H., Koeppe, R. A., & Katz, S. (1996).
Dissociation of storage and rehearsal in working memory: PET evidence. Psychological
Science, 7,  25–31.
Axer, H. (2001). Supra-and infrasylvian conduction aphasia. Brain and Language, 76(3),
317–331.
Baddeley, A. (2000). The episodic buffer: A new component of working memory? Trends in
Cognitive Sciences, 4(11), 417–423. doi: 10.1016/S1364-6613(00)01538-2
Baddeley, A. (2012). Working memory: Theories, models, and controversies. Annual Review of

Psychology, 63(1), 1–29. doi: 10.1146/annurev-psych-120710-100422
Baddeley, A., Lewis, V., & Vallar, G. (1984). Exploring the articulatory loop. The Quarterly
Journal of Experimental Psychology, 36(2), 233–252.
Baddeley, A., Papagno, C., & Vallar, G. (1988). When long-term learning depends on short-
term storage. Journal of Memory and Language, 27(5), 586–595.
Baddeley, A. D. (1986). Working memory. New York: Oxford University Press.
Baddeley, A. D., & Hitch, G. J. (1974). Working Memory. In G. Bower (Ed.), The psychology of
learning and motivation (Vol. 7, pp. 47–90). New York: Academic Press.
Baddeley, A. D., Thomson, N., & Buchanan, M. (1975). Word length and the structure of short-
term memory. Journal of Verbal Learning and Verbal Behavior, 14(6), 575–589.
Baddeley, A. D., & Warrington, E. K. (1970). Amnesia and the distinction between long-and
short-term memory. Journal of Verbal Learning and Verbal Behavior, 9(2), 176–189.
Badre, D., Poldrack, R. A., Pare-Blagoev, E. J., Insler, R. Z., & Wagner, A. D. (2005). Dissociable
controlled retrieval and generalized selection mechanisms in ventrolateral prefrontal
cortex. Neuron, 47(6), 907–918.
Baldo, J. V., & Dronkers, N. F. (2006). The role of inferior parietal and inferior frontal cortex
in working memory. Neuropsychology, 20(5), 529–538. doi: 2006-10978-003 [pii]10.1037/
0894-4105.20.5.529
Basso, A., Spinnler, H., Vallar, G., & Zanobio, M. (1982). Left hemisphere damage and selective
impairment of auditory verbal short-term memory: A case study. Neuropsychologia, 20(3),
263–274.
Bavelier, D., Corina, D., Jezzard, P., Clark, V., Karni, A., Lalwani, A., . . . Neville, H. J. (1998).
Hemispheric specialization for English and ASL: Left invariance– right variability.
Neuroreport, 9(7), 1537–1542.
Becker, J. T., MacAndrew, D. K., & Fiez, J. A. (1999). A comment on the functional localiza-
tion of the phonological storage subsystem of working memory. Brain and Cognition,
41(1),  27–38.
Berndt, R. S., Mitchum, C. C., & Wayland, S. (1997). Patterns of sentence comprehension in
aphasia: A consideration of three hypotheses. Brain and Language, 60(2), 197–221.
Cereb Cortex, 10(5), 512–528.
Bjork, R. A., & Whitten, W. B. (1974). Recency-sensitive retrieval processes in long-term free
recall. Cognitive Psychology, 6(2), 173–189.
Buchsbaum, B., Baldo, J., Okada, K., & Berman, K. (2011). Conduction aphasia, sensory-motor
integration, and phonological short-term memory: An aggregate analysis of lesion and
fMRI data. Brain and Language, 119(3), 119–128.
Buchsbaum, B., Hickok, G., & Humphries, C. (2001). Role of left posterior superior temporal
gyrus in phonological processing for speech. Cognitive Science, 25(5), 663–678.
Buchsbaum, B., Olsen, R., & Koch, P. (2005). Human dorsal and ventral auditory streams subserve
rehearsal-based and echoic processes during verbal working memory. Neuron, 48(4), 687–697.
Buchsbaum, B. R., & D’Esposito, M. (2008). The search for the phonological store: From loop to
convolution. Journal of Cognitive Neuroscience, 20(5), 762–778. doi: 10.1162/jocn.2008.20501
Burgio, F., & Basso, A. (1997). Memory and aphasia. Neuropsychologia, 35(6), 759–766.
Caplan, D., Baker, C., & Dehaut, F. (1985). Syntactic determinants of sentence comprehension
in aphasia. Cognition, 21(2), 117–175.
Caplan, D., Rochon, E., & Waters, G. S. (1992). Articulatory and phonological determinants
of word length effects in span tasks. Quarterly Journal of Experimental Psychology A, 45(2),
177–192.
Chein, J. M., Ravizza, S. M., & Fiez, J. A. (2003). Using neuroimaging to evaluate models of
working memory and their implications for language processing. Journal of Neurolinguistics,
16, 315–339.
Chevillet, M., Riesenhuber, M., & Rauschecker, J. P. (2011). Functional correlates of the antero-
lateral processing hierarchy in human auditory cortex. Journal of Neuroscience, 31(25), 9345–
9352. doi: 31/25/9345 [pii]10.1523/JNEUROSCI.1448-11.2011
Conrad, R., & Hull, A. J. (1964). Information, acoustic confusion and memory span. British
Journal of Psychology, 55, 429–432.
Corkin, S. (2002). What’s new with the amnesic patient H.M.? Nature Reviews Neuroscience,
3(2), 153–160. doi: 10.1038/nrn726nrn726 [pii]
Corkin, S. (2013). Permanent present tense: The unforgettable life of the amnesic patient.
New York: Basic Books.
Cowan, N. (2001). The magical number 4 in short-term memory: A reconsideration of mental
storage capacity. Behavioral and Brain Sciences, 24(1), 87–114; discussion 114–185.
Crowder, R. G. (1982). The demise of short-term memory. Acta Psychologica (Amsterdam),
50(3), 291–323.
Damasio, A. R. (1992). Aphasia. New England Journal of Medicine, 326(8), 531–539.
Damasio, H., & Damasio, A. R. (1980). The anatomical basis of conduction aphasia. Brain,
103(2), 337–350.
Davelaar, E. J., Goshen-Gottstein, Y., Ashkenazi, A., Haarmann, H. J., & Usher, M. (2005). The de-
mise of short-term memory revisited: Empirical and computational investigations of recency
effects. Psychological Review, 112(1), 3–42. doi: 2004-22409-001 [pii]10.1037/0033-295X.112.1.3
De Renzi, E., & Nichelli, P. (1975). Verbal and non-verbal short-term memory impairment fol-
lowing hemispheric damage. Cortex, 11(4), 341–354.
Fiebach, C. J., Rissman, J., & D’Esposito, M. (2006). Modulation of inferotemporal cortex ac-
tivation during verbal working memory maintenance. Neuron, 51(2), 251–261. doi: S0896-
6273(06)00460-0 [pii]10.1016/j.neuron.2006.06.007
Fritz, J., Mishkin, M., & Saunders, R. C. (2005). In search of an auditory engram. Proceedings of
Geschwind, N. (1965a). Disconnexion syndromes in animals and man. Brain, 88, 237–294.
Geschwind, N. (1965b). Disconnexion syndromes in animals and man. Brain, 88, 585–644.
Glanzer, M., & Cunitz, A.-R. (1966). Two storage mechanisms in free recall. Journal of Verbal
Learning and Verbal Behavior, 5, 351–360.
Hashimoto, R., Lee, K., Preus, A., McCarley, R. W., & Wible, C. G. (2010). An fMRI study of
functional abnormalities in the verbal working memory system and the relationship to
clinical symptoms in chronic schizophrenia. Cerebral Cortex, 20(1), 46–60. doi: bhp079
[pii]10.1093/cercor/bhp079
Hickok, G., Buchsbaum, B., & Humphries, C. (2003). Auditory-motor interaction revealed by
fMRI: Speech, music, and working memory in area Spt. Journal of Cognitive Neuroscience,
15(5), 673–682.
Hickok, G., Okada, K., & Serences, J. (2009). Area Spt in the human planum temporale
supports sensory-motor integration for speech processing. Journal of Neurophysiology,
101(5), 2725–2732.
Jones, D., Madden, C., & Miles, C. (1992). Privileged access by irrelevant speech to short-term
memory: The role of changing state. Quarterly Journal of Experimental Psychology A, 44(4),
645–669.
Jonides, J., Schumacher, E. H., Smith, E. E., Koeppe, R. A., Awh, E., Reuter-Lorenz, P.
A., . . . Willis, C. R. (1998). The role of parietal cortex in verbal working memory. Journal of
Neuroscience, 18(13), 5026–5034.
Koelsch, S., Schulze, K., Sammler, D., Fritz, T., Muller, K., & Gruber, O. (2009). Functional ar-
chitecture of verbal and tonal working memory: An FMRI study. Human Brain Mapping,
30(3), 859–873. doi: 10.1002/hbm.20550
Koenigs, M., Acheson, D. J., Barbey, A. K., Solomon, J., Postle, B. R., & Grafman, J. (2011). Areas
of left perisylvian cortex mediate auditory–verbal short-term memory. Neuropsychologia,
49(13), 3612–3619.
Leff, A. P., Schofield, T. M., Crinion, J. T., Seghier, M. L., Grogan, A., Green, D. W., & Price,
C. J. (2009). The left superior temporal gyrus is a shared substrate for auditory short-term
memory and speech comprehension: Evidence from 210 patients with stroke. Brain, 132(Pt
12), 3401–3410. doi: awp273 [pii]10.1093/brain/awp273
Leung, A. W. S., & Alain, C. (2011). Working memory load modulates the auditory “what” and
“where” neural networks. Neuroimage, 55(3), 1260–1269.
Levy, B. A. (1971). Role of articulation in auditory and visual short-term memory. Journal of
Verbal Learning and Verbal Behavior, 10(2), 123–132.
Lewandowsky, S., & Oberauer, K. (2009). No evidence for temporal decay in working memory.
Journal of Experimental Psychology: Learning, Memory, and Cognition, 35(6), 1545.
Macken, W. J., Mosdell, N., & Jones, D. M. (1999). Explaining the irrelevant- sound
effect: Temporal distinctiveness or changing state? Journal of Experimental
Psychology: Learning, Memory, and Cognition, 25(3), 810.
Martin, R. C., & Freedman, M. L. (2001). Short- term retention of lexical-
semantic
representations: Implications for speech production. Memory, 9(4), 261–280.
Melton, A. W. (1963). Memory. Science, 140(356),  82–86.
Miller, G. A. (1956). The magical number seven plus or minus two: Some limits on our capacity
for processing information. Psychological Review, 63(2),  81–97.
(1978). Broca aphasia Pathologic and clinical. Neurology, 28(4), 311–311.
Mueller, S. T., Seymour, T. L., Kieras, D. E., & Meyer, D. E. (2003). Theoretical implications
of articulatory duration, phonological similarity, and phonological complexity in verbal
working memory. Journal of Experimental Psychology: Learning, Memory and Cognition,
29(6), 1353–1380.
Nadeau, S. E. (2001). Phonology: A review and proposals from a connectionist perspective.
Niendam, T. A., Laird, A. R., Ray, K. L., Dean, Y. M., Glahn, D. C., & Carter, C. S. (2012). Meta-
analytic evidence for a superordinate cognitive control network subserving diverse execu-
tive functions. Cognitive, Affective, & Behavioral Neuroscience, 12(2), 241–268.
Nystrom, L. E., Braver, T. S., Sabb, F. W., Delgado, M. R., Noll, D. C., & Cohen, J. D.
(2000). Working memory for letters, shapes, and locations: fMRI evidence against
stimulus-based regional organization in human prefrontal cortex. NeuroImage, 11(5 Pt

1), 424–4 46.
Olson, I. R., Page, K., Moore, K. S., Chatterjee, A., & Verfaellie, M. (2006). Working memory
for conjunctions relies on the medial temporal lobe. Journal of Neuroscience, 26(17),
4596–4601.
Paulesu, E., Frith, C. D., & Frackowiak, R. S. (1993). The neural correlates of the verbal compo-
nent of working memory. Nature, 362(6418), 342–345.
Poldrack, R. A., Wagner, A. D., Prull, M. W., Desmond, J. E., Glover, G. H., & Gabrieli, J. D.
(1999). Functional specialization for semantic and phonological processing in the left infe-
rior prefrontal cortex. NeuroImage, 10(1),  15–35.
Postle, B., & Berger, J. (1999). Functional neuroanatomical double dissociation of mnemonic
and executive control processes contributing to working memory performance. Proceedings
Postman, L., & Phillips, L.-W. (1965). Short-term temporal changes in free recall. Quarterly
Journal of Experiemental Psychology, 17, 132–138.
Rama, P., & Courtney, S. M. (2005). Functional topography of working memory for face or
voice identity. NeuroImage, 24(1), 224–234.
Rama, P., Poremba, A., Sala, J. B., Yee, L., Malloy, M., Mishkin, M., & Courtney, S. M. (2004).
Dissociable functional cortical topographies for working memory maintenance of voice
identity and location. Cerebral Cortex, 14(7), 768–780.
Ranganath, C., & Blumenfeld, R. S. (2005). Doubts about double dissociations between short-
and long-term memory. Trends in Cognitive Sciences, 9(8), 374–380.
control and integration. Hearing Research, 271(1),  16–25.
Rauschecker, J. P., & Tian, B. (2000). Mechanisms and streams for processing of “what” and
“where” in auditory cortex. Proceedings of the National Academy of Sciences USA, 97(22),
11800–11806.
Repovs, G., & Baddeley, A. (2006). The multi-component model of working memory:
Explorations in experimental cognitive psychology. Neuroscience, 139(1),  5–21.
Romanski, L. M. (2004). Domain specificity in the primate prefrontal cortex. Cognitive,
Affective, & Behavioral Neuroscience, 4(4), 421–429.
cortex. Nature Neuroscience, 2(12), 1131–1136.
Ryan, J. D., & Cohen, N. J. (2004). The nature of change detection and online representations of
scenes. Journal of Experimental Psychology: Human Perception and Performance, 30(5), 988.
Salame, P., & Baddeley, A. D. (1982). Disruption of short-term memory by unattended
speech: Implications for the structure of working memory. Jorunal of Verbal Learning and
Verbal Behavior, 21, 150–164.
Salmon, E., Van der Linden, M., Collette, F., Delfiore, G., Maquet, P., Degueldre, C., . . . Franck,
G. (1996). Regional brain activity during working memory tasks. Brain, 119(Pt 5), 1617–1625.
Saur, D., Kreher, B., & Schnell, S. (2008). Ventral and dorsal pathways for language. Proceedings
of the National Academy of Science, 105(46), 18035–18040.
Schumacher, E. H., Lauber, E., Awh, E., Jonides, J., Smith, E. E., & Koeppe, R. A. (1996). PET ev-
idence for an amodal verbal working memory system. NeuroImage, 3(2),  79–88.
Scoville, W. B., & Milner, B. (1957). Loss of recent memory after bilateral hippocampal lesions.
Journal of Neurology, Neurosurgery and Psychiatry, 20(1),  11–21.
Sederberg, P. B., Howard, M. W., & Kahana, M. J. (2008). A context-based theory of recency
and contiguity in free recall. Psychological review, 115(4), 893.
Shallice, T., & Butterworth, B. (1977). Short-term-memory impairment and spontaneous
speech. Neuropsychologia, 15(6), 729–735.
Shallice, T., & Vallar, G. (1990). The impairment of auditory-verbal short-term storage. In G.
Vallar & T. Shallice (Eds.), Neuropsychological impairments of short-term memory (pp. 11–
53). Cambridge: Cambridge University Press.
Shallice, T., & Warrington, E. K. (1970). Independent functioning of verbal memory stores: A
neuropsychological study. Quarterly Journal of Experimental Psychology, 22(2), 261–273.
Shallice, T., & Warrington, E. K. (1977). Auditory-verbal short-term-memory impairment and
conduction aphasia. Brain and Language, 4(4), 479–491.
Shivde, G., & Thompson-Schill, S. L. (2004). Dissociating semantic and phonological mainte-
nance using fMRI. Cognitive, Affective, & Behavioral Neuroscience, 4(1),  10–19.
Smith, E. E., & Jonides, J. (1999). Storage and executive processes in the frontal lobes. Science,
283(5408), 1657–1661.
Squire, L. R., Stark, C. E., & Clark, R. E. (2004). The medial temporal lobe. Annual Review of
Neuroscience, 27, 279–306. doi: 10.1146/annurev.neuro.27.070203.144130
Stevens, A. A. (2004). Dissociating the cortical basis of memory for voices, words and tones.
Cognitive Brain Research, 18(2), 162–171.
Takayama, Y., Kinomoto, K., & Nakamura, K. (2004). Selective impairment of the auditory-
verbal short-term memory due to a lesion of the superior temporal gyrus. European
Neurology, 51(2), 115–117.
Tian, B., Reser, D., Durham, A., Kustov, A., & Rauschecker, J. P. (2001). Functional specializa-
tion in rhesus monkey auditory cortex. Science, 292(5515), 290–293.
Tremblay, S., Nicholls, A. P., Alford, D., & Jones, D. M. (2000). The irrelevant sound ef-
fect: Does speech play a special role? Journal of Experimental Psychology: Learning, Memory,
and Cognition, 26(6), 1750.
Underwood, B. J. (1957). Interference and forgetting. Psychological Review, 64(1),  49–60.
Unsworth, N., & Engle, R. W. (2007). The nature of individual differences in working memory
capacity: Active maintenance in primary memory and controlled search from secondary
memory. Psychological Review, 114(1), 104–132. doi: 10.1037/0033-295X.114.1.104
Vallar, G. (2006). Mind, brain, and functional neuroimaging. Cortex, 42(3), 402–405; discus-
sion 422–407.
Vallar, G., & Baddeley, A. (1984). Fractionation of working memory: Neuropsychological evi-
dence for a phonological short-term store. Journal of Verbal Learning and Verbal Behavior, 23,
151–161.
Vallar, G., Di Betta, A. M., & Silveri, M. C. (1997). The phonological short-term store-rehearsal
system: Patterns of impairment and neural correlates. Neuropsychologia, 35(6), 795–812.
Wagner, A. D., Pare-Blagoev, E. J., Clark, J., & Poldrack, R. A. (2001). Recovering meaning: Left
prefrontal cortex guides controlled semantic retrieval. Neuron, 31(2), 329–338.
doi: S0896-6273(01)00359-2  [pii]
Warrington, E., & Shallice, T. (1969). The selective impairment of auditory verbal short-term
memory. Brain, 92(4), 885–896.
Waugh, N. C., & Norman, D. A. (1965). Primary memory. Psychological Review, 72, 89–104.
Wernicke, C. (1874/1969). The symptom complex of aphasia: A psychological study on an an-
atomical basis. In R. S. Cohen & M. W. Wartofsky (Eds.), Boston studies in the philosophy of
science. Dordecht: D. Reidel.
Wickelgren, W. A. (1968). Sparing of short-term memory in an amnesic patient: Implications

for strength theory of memory. Neuropsychologia, 6(3), 235–244.
Yee, L. T. S., Hannula, D. E., Tranel, D., & Cohen, N. J. (2014). Short-term retention of relational
memory in amnesia revisited: Accurate performance depends on hippocampal integrity.
Frontiers in Human Neuroscience, 8, 16.
Zurowski, B., Gostomzyk, J., Gron, G., Weller, R., Schirrmeister, H., Neumeier, B., . . . Walter, H.
(2002). Dissociating a common working memory network from different neural substrates
of phonological and spatial stimulus processing. Neuroimage, 15(1),  45–57.
Chapter 33
Su b c ort i c a l
C ontribu ti ons to
L anguag e
David A. Copland and Anthony J. Angwin
Historical Perspective
Kinnier Wilson’s (1925) depiction of the basal ganglia (BG) as the dark basements of
the brain is still somewhat true when considering our understanding of the relation-
ship between the subcortex and human language function. A range of possible func-
tional attributes of these structures have been proposed based on studies of individuals
with various subcortical pathologies, surgical lesions and electrical stimulation during
neurosurgical procedures, and more recently through functional neuroimaging studies.
This effort has elucidated motor and possible higher level/cognitive functions that the
BG and thalamus may contribute to, yet the question of whether these deep structures
play a critical role in language function remains a point of contention.
The first avenue for investigating this question empirically has been through clinico-
anatomical studies of language function in individuals with vascular lesions involving
the dominant BG (see also Wilson, Chapter 2 in this volume). Yet since the observa-
tion by Marie (1906) and Moutier (1908) (as cited in Wallesch & Papagno, 1988) early
last century that subcortical lesions may be associated with language deficits, and a sub-
sequent wealth of subcortical investigations in the past 50 years, the understanding of
language function in this population has remained unclear, and advances in the theoret-
ical conception of BG language function have been largely piecemeal. Although several
theories of subcortical language function were postulated (e.g., Crosson, 1992; Wallesch
& Papagno, 1988) and revised (Crosson, 1999), this field of inquiry did not progress for
many years, unlike the revision of classical aphasia subtypes and impairments, which
was further specified in terms of underlying processing components (Blumstein, 1997;
see also Blumstein, Chapter 1 in this volume) and more recently has been incorporated
852 David A. Copland and Anthony J. Angwin
into dominant models such as the dual-stream model (see Hickok, Chapter 20 in this
volume).
One plausible explanation for this disparity, which has gained increased currency, is
the notion that the BG play no direct role in language function, but that any language
deficits which are associated with vascular BG lesions arise from cortical dysfunction
(Nadeau & Crosson, 1997). Nadeau and Crosson (1997) provide comprehensive evi-
dence for the lack of a coherent pattern or syndrome of language impairment following
BG lesions, although it should be noted that this observation is based on traditional
language measures sensitive to classical aphasia symptoms, allowing for the possibility
that other functions may still be consistently impaired following subcortical lesions.
Furthermore, observations of classical syndromes in aphasia following cortical lesions
have also been criticized on the basis of significant heterogeneity within supposed
syndromes in terms of both language presentation and lesion location. Regardless,
the argument that aphasia subsequent to BG lesions was due in particular to cortical
hypoperfusion was subsequently supported in a compelling fashion by the findings of
Hillis et al. (2002), who observed a strong relationship between the presence of aphasia
and the presence of cortical hypoperfusion after subcortical strokes, such that aphasic
symptoms were only observed in the presence of cortical hypoperfusion. Critically, such
hypoperfusion is not detected with structural imaging. This does not preclude the pos-
sibility that the BG may still contribute to language functions, but lends credence to the
notion that any role is likely to be indirect or supportive. It should also be noted that the
preceding observation that language impairments following BG strokes were predom-
inantly due to cortical hypoperfusion does not appear to extend to thalamic strokes,
where Sebastian et al. (2014) recently demonstrated the presence of aphasia following
thalamic lesions without evidence of cortical hypoperfusion. Nadeau and Crosson
(1997) also provide clear evidence of lexical-semantic impairments following thalamic
strokes, which will be described further in the following.
Recent Advances in Understanding

Relevant Subcortical Circuitry
When examining possible subcortical contributions to language, it is important to con-

sider the important position of subcortical components in relation to various cortical
regions that have been implicated in language or language-relevant processing. Indeed,
the potential role of subcortical structures in language has been largely conceptualized
in terms of subcortical-cortical circuitry (Crosson, 1992; Wallesch & Papagno, 1988).
The striatum was originally considered a funnel that integrates information received
from diverse cortical regions; however, this notion was largely superseded by the
common view of the striatum as a “multi-laned throughway” that forms part of mul-
tiple segregated circuits connecting the cortex, the BG, and the thalamus (Alexander,
Subcortical Contributions to Language 853
Delong, & Strick, 1986; Middleton & Strick, 2000). Five anatomically segregated and
possibly functionally distinct circuits are well established (see Middleton & Strick,
2000, for a review) (see Figure 33.1). These subcortical-cortical systems follow the basic
route of cortex-striatum-globus pallidus/substantia nigra-thalamus-cortex in a unidi-
rectional fashion. Each frontal-subcortical circuit contains a direct pathway between
the striatum and the globus pallidus interna/substantia nigra complex, and an indirect
pathway from the striatum to the globus pallidus interna/substantia nigra via the globus
pallidus externa and subthalamic nucleus (STN). In addition, the STN is involved in a
hyperdirect pathway receiving inputs from the frontal cortex and directly projecting to
the globus pallidum. Each circuit includes an open and closed component.
Of these five circuits, Figure 33.1 illustrates the circuits with cortical components rel-
evant to language (dorsolateral prefrontal cortex, anterior cingulate, orbitofrontal), but
it should be noted that these regions are associated with domain-general cognitive pro-
cessing (including executive function, cognitive control), rather than language-specific
functions per se (see Tremblay, Deschamps, & Dick, Chapter 15 in this volume). Crosson
et al. (2003) provided functional magnetic resonance imaging (fMRI) (see Heim &
Specht, Chapter 4 in this volume) evidence suggesting the possibility of a modified
Dorsolateral Anterior Lateral

Prefrontal Cingulate Orbitofrontal
Cortex DLC PPC, APA ACA HC, EC, STG, ITG LOF STG, ITG, ACA
Striatum Caudate VS Caudate

(DL) (VM)
Globus LDM-GP RL-GP MDM-GP

Pallidus
Thalamus Thalamus Thalamus Thalamus

(VA, MD) (MD) (VA, MD)
Figure 33.1. Proposed frontal-subcortical circuits implicated in cognition and emotion The
indirect circuits and connections of the substantia nigra and subthalamic nucleus are not shown.
Abbreviations: ACA = anterior cingulate area; APA = arcuate premotor area; EC = entorhinal cortex; DLC = dorsolateral
prefrontal cortex; GP = globus pallidus; HC = hippocampal cortex; ITG = inferior temporal gyrus; LDM = lateral
dorsomedial LOF = lateral orbitofrontal cortex; MD = medial dorsal; MDM = medial dorsomedial; PPC = posterior
parietal cortex; RL = rostrolateral; STG = superior temporal gyrus; VA = ventral anterior; VS = ventral striatum.
Source: Modified from Alexander et al. (1986).
circuit including the pre-Supplementary Motor Area (SMA), dorsal caudate–ventral

thalamus involved in lexical retrieval and selection (discussed further in the following).
There is also growing evidence of a circuit connecting the BG and Broca’s area, and it has
been hypothesized that such a circuit may influence grammar processing (e.g., Ullman,
2006). Lehericy et al. (2004) provide evidence for such a circuit involving the caudate
and left BA 45, and Teichmann et al. (2015) found support for the notion that this circuit
may be involved in syntax. Ford et al. (2013) also identified structural connections be-
tween the striatum, thalamus, and Broca’s area; however, it was the putamen rather than
caudate that showed structural connectivity with the left inferior frontal gyrus (IFG).
Studies of functional connectivity also provide insights into the possible role of
subcortical-cortical circuitry in language. Snijders, Petersson, & Hagoort (2010)
identified increased effective connectivity between the striatum and left IFG and
middle temporal gyrus (MTG) during combination of lexical-syntactic representations.
A large-scale (970 participants) resting-state functional connectivity study of language
networks also demonstrated connectivity between Broca’s and Wernicke’s areas and bi-
lateral caudate and left putamen/globus pallidus, ventral thalamus, and the STN (Tomasi
& Volkow, 2012). There is also emerging evidence that different circuits may be engaged
depending on the particular language function examined. Specifically, Jeon et al. (2014)
recently observed different cortical-subcortical circuitry for second-language learning,
depending on the form of processing involved (contextual, episodic, branching), with
higher-level cognitive operations engaging the ventro-anterior prefrontal cortex, head
of the caudate nucleus, and ventral anterior thalamus, while lower-level operations in-
volved the posterior prefrontal cortex, body of the caudate, and medial dorsal nucleus
of the thalamus (see Green & Kroll, Chapter 11 in this volume). Importantly, structural
connectivity between components of these different circuits was confirmed with diffu-
sion MRI, and similar patterns of activity were observed in parallel non-linguistic tasks
employing a similar cognitive hierarchy.
Within the striatum, the caudate and putamen have traditionally been associated
with cognitive and motor functions, respectively (and associative and motor circuits),
suggesting a more likely influence of the caudate on language functions; however, this
assumption needs revising. In particular, there is imaging evidence of putamen activity
during cognitive tasks, co-activation of the putamen and prefrontal cortex, and anatom-
ical projections connecting the putamen and cortical regions associated with cognition
(Selemon & Goldman-Rakic, 1985) and language, including Broca’s area (Ford et al.,
2013; Provost, Hanganu, & Monchi, 2015). Intriguingly, examination of co-activation
data from 5,809 human imaging studies recently identified the anterior putamen as
being associated with social and language-related functions (Pauli, O’Reilly, Yarkoni, &
Wager, 2016), although some identified terms such as “vocal” may reflect motor rather
than language functions, consistent with evidence for the role of the putamen in speech
production observed in both healthy and stroke populations (e.g. Seghier, Bagdasaryan,
Jung, & Price, 2014).
Taken together, there is growing evidence of functional and structural connectivity
between the striatum, thalamus, and cortical regions involved in language processing,
beyond the established cortical-subcortical loops (Alexander, DeLong, & Strick, 1986).
However, it is also important to be cautious regarding neuroanatomical assumptions
for cortical-subcortical circuitry involving segregated parallel circuitry. Nambu (2008)
argues that these loops may be considered on a continuum, rather than as segregated
parallel circuits with distinct components, given evidence of convergence particularly in
cortical-striatal inputs (e.g., Haber, Fudge, & McFarland, 2000). Based on a comprehen-
sive review of these data, Bell and Shine (2016) propose that we need to consider both
segregated and integrated BG-cortical circuitry. It has also been proposed that the di-
rect/indirect pathway model is oversimplified and that functions of the direct, indirect,
and hyperdirect components are not well established (Nambu, 2008), although there
have been proposals regarding potential functions relevant to language function that
may be carried out by these different components, which require further examination
(Chenery, Angwin, & Copland, 2008; Crosson, Benjamin, & Levy, 2007). We now turn
to evidence for a subcortical contribution to different aspects of language processing,
taking into account neurological patient studies and functional neuroimaging evidence.
The Thalamus and Semantic

Engagement
While there remains considerable controversy regarding the role of the BG in lan-
guage, converging lesion, stimulation, and healthy neuroimaging evidence for tha-
lamic contributions to lexical-semantic function is considerable. Yet the traditional
method of clinical-anatomical correlations has also proven difficult with regard to tha-
lamic aphasia. While thalamic aphasia has been described predominantly in terms of
word-finding difficulties and semantic paraphasia (Crosson, 1992) and these features
are often observed following thalamic hemorrhage, it has been challenging to isolate
these features to particular regions of the thalamus based on evidence from thalamic
infarcts (Crosson, 2013). Nadeau and Crosson (1997) proposed that lexical-semantic
disturbances arise when (a) lesions involve the pulvinar-lateral posterior complex
(which has connections with temporo-parietal and frontal regions relevant to language),
or (b) there is a disruption to the frontal lobe–nucleus reticularis thalami (NR) system.
There is considerable evidence supporting the role of the pulvinar and lateral-posterior
complex in lexical-semantics, and neuroanatomically these regions have connections
with cortical regions involved in language processing. Evidence from ventral anterior
and ventrolateral nuclei suggests a more minor role (Nadeau & Crosson, 1997) while
other more recent proposals suggest that the ventral anterior nuclei play a role within
BG-thalamic-cortical circuits relevant to word retrieval (Crosson, 2013).
There has also been a more recent revision of certain anatomical assumptions of this
selective engagement model (Crosson, 2013), suggesting involvement of the pulvinar
as critical, while the roles of the centromedian and parafasciculus nuclei are less clear.
The proposed role of the thalamus in language is conceived as one of “selective engage-
ment.” More specifically, it involves the temporary engagement of language-relevant
cortical networks, which is conceptualized as intentionally guided attention and also
resembles working memory processes in some respects (Nadeau & Crosson, 1997).
This selective engagement has been primarily demonstrated with respect to lexical-
semantic operations and word finding. It may be applied to the process of confronta-
tion naming, where the detailed patient study of Raymer, Moberg, Crosson, Nadeau,
& Gonzalez-Rothi (1997) supports the view that while semantic and lexical systems
may be intact following left thalamic lesions, the selective engagement of mappings
from semantics to lexical representations may be compromised. Evidence of category-
specific naming deficits following a hemorrhagic lesion involving the pulvinar further
supports the view that discrete thalamic-cortical connections may result in such spe-
cific naming deficits (Nadeau & Crosson, 1997). Later revisions of this model also em-
phasize the language-relevant involvement of the thalamus (and pulvinar in particular)
in transferring or relaying information between distinct cortical regions (including
via cortico-thalamo cortical circuits) (Crosson, 2013). Crosson (2013) provides a com-
prehensive integrated review and proposal, including the suggestion that cortico-
thalamic feedback from layer 6 cells to thalamic nuclei, which serves to sharpen the
focus of representations, may also play a role in language function. Electrical stimula-
tion studies also support the view that the cortical-pulvinar-cortical projection system
supports cortical synchronization involved in word retrieval, with stimulation leading
to anomia (Hebb & Ojemann, 2013). This role may not be limited to output, however,
with evidence from simultaneous thalamic and scalp electrode recordings showing ac-
tivity relating to the processing of semantic anomalies in sentences (Wahl et al., 2008).
While the semantic engagement model of Nadeau and Crosson (1997) proposed a role
for the thalamus in language that was independent of the BG, later revisions also pro-
pose a role for the thalamus in BG-thalamic-cortical circuits involved in lexical selec-
tion (Crosson, 2013).
The role of the thalamus and more specifically the pulvinar in semantic processing
was further emphasized by Hart and colleagues (Kraut et al., 2002; Hart et al., 2013), who
have proposed a “neural hybrid model of semantic memory.” This model is based on pa-
tient studies (including electrode recordings) and neuroimaging in healthy individuals
and is generally consistent with the proposed role of the thalamus in semantic engage-
ment (Nadeau & Crosson, 1997). According to this proposal, thalamic activity is crit-
ical to semantic feature binding, being linked to similar activity in a number of cortical
regions when combining multiple semantic features to retrieve an object representa-
tion. Subsequent development of this model suggested a pre-SMA–thalamic–caudate
circuit involved in semantic object retrieval, which is supported by activity observed in
these regions (in addition to the ventral temporal-occipital lobes associated with visual
memory) during an fMRI study of semantic object retrieval (e.g., Kraut et al., 2002).
During correct object retrieval, there is a high beta-band power increase which is con-
sistent with synchronization and communication between the pre-SMA and thalamus.
This circuit is assumed to be engaged during controlled semantic retrieval and search,
and the role of the caudate within this model is to select the correct semantic object
and suppress incorrect representations, which broadly agrees with other proposals that
the striatum is involved in selection and suppression during word production (Crosson
et al., 2003) and during lexical ambiguity resolution (Copland, 2003). The combined
role of the striatum and thalamus may also extend to new word learning, where it has
been proposed that they form part of a network that selects the appropriate meaning
from competing representations and associates this meaning with the new word form
(Ye, Mestres-Misse, Rodriguez-Fornells, & Munte, 2011).
The Basal Ganglia, Phonology,

and Lexical-Semantic Processing
The involvement of the BG in language operations will be considered in part on the

basis of studies in Parkinson’s disease (PD), given its association with impaired striatal
output due to degeneration of the nigrostriatal system (Gerfen, 1992). Similarly, studies
of Huntington’s disease (HD) have been used to examine the role of the caudate in pho-
nology and lexical-semantics. However, it is important to be cautious when making
inferences regarding BG function based only on observations in these populations
without direct measures of BG function, given that pathology is not limited to these
structures (e.g., Rüb et al., 2016).
While the majority of work on possible BG contributions has focused on lexical-
semantics and sentence-level functions, there is some direct evidence for the role of the
striatum in phonological processing. Tettamanti et al. (2005) utilized positron emission
tomography (PET) to investigate dopamine binding to D2 receptors in healthy adults
during the detection of phonological and syntactic anomalies within pseudo-word
sentences. Phonological processing accuracy correlated with tracer binding potential
in the left caudate, while reaction times correlated with tracer binding potential in the
left putamen. These findings indicated that more accurate and faster phonological pro-
cessing was associated with reduced dopamine requirements in the left BG.
Research in PD further implicates a role for BG circuitry in phonological processing,
with evidence of phonological processing deficits in this population (Elorriaga-Santiago,
Silva-Pereyra, Rodriguez-Camacho, & Carrasco-Vargos, 2013). Moreover, recordings
of neuronal activity from depth electrodes in PD patients have indicated that activity
in the right caudate may be linked to aspects of phonological processing (Abdullaev &
Melnichuk, 1997), while recordings of local field potentials from the STN in PD have re-
vealed auditory evoked potentials in response to early stage auditory perception, most
likely related to phonological processing (De Letter et al., 2014). Teichmann, Darcy,
Bachoud-Levi, & Dupoux (2009) also demonstrated that although phoneme perception
for isolated words was intact in patients with HD, phoneme discrimination deficits were
evident within phrasal contexts. The researchers suggested that these results may reflect
a role for the striatum in phonological short-term memory, which is required for word
perception in phrasal contexts.
There is considerable evidence that various aspects of lexical-semantic processing are
altered in PD. In particular, the use of semantic priming tasks has shed light on a wide
array of lexical-semantic disturbances in PD, which may be linked to dysfunction within
BG circuitry. Semantic priming refers to the well-recognized phenomenon whereby a
target word is recognized faster (e.g., during a lexical decision) when preceded by a re-
lated prime word (e.g., winter-snow) relative to an unrelated word (e.g., window-snow).
Semantic priming effects can be a result of automatic spreading activation (Collins &
Loftus, 1975; Neely, 1977), such that activation of a prime word leads to a spread of acti-
vation throughout the semantic network, which partially activates other semantically
or associatively related concepts. This spreading of activation subsequently results in
faster recognition of target words related to the prime relative to targets that are unre-
lated. Accordingly, the time course of automatic semantic activation can be charted by
manipulating the amount of time that elapses between the prime and target, known as
the stimulus onset asynchrony (SOA) or inter-stimulus interval (ISI). In addition to au-
tomatic semantic activation, however, it is possible for conscious processes such as pre-
lexical expectancies or post-lexical semantic matching strategies (see Neely, 1991, for a
review) to induce semantic priming effects.
Numerous semantic priming studies have provided evidence for a delayed time
course of semantic activation for some patients with PD, as evidenced by the emer-
gence of semantic priming effects only at longer SOAs (Angwin, Chenery, Copland,
Murdoch, & Silburn, 2007; Angwin et al., 2009; Arnott, Chenery, Murdoch, & Silburn,
2001; Grossman, Lee, Morris, Stern, Hurtig, 2002). As suggested by Grossman et al.
(2002), such delays to lexical access in PD may be due to dysfunction within dopamine-
dependent frontal-striatal circuitry that supports information-processing speed. Such
claims are further supported by an absence of semantic priming in PD patients tested
when off their levodopa medication (Angwin et al., 2007, 2009), PET research showing
that cognitive slowing in PD is linked to dopaminergic dysfunction in fronto-striatal
circuitry (Jokinen et al., 2013), and suggestions of a faster time course of semantic acti-
vation in healthy adults who have been administered levodopa relative to those on pla-
cebo (Angwin et al., 2004). Hence, the extent to which automatic semantic activation
is disturbed for an individual with PD may be modulated by the magnitude of BG dys-
function for that person, although this proposal relies on indirect evidence. This notion
would explain why automatic semantic priming appears intact in some studies of PD
(Copland, 2003; Filoteo et al., 2003), as the extent of BG dysfunction for some patients
may not be sufficient at the time of testing to induce changes to the temporal course of
lexical access.
Besides influencing the speed of lexical access, the BG may also contribute to semantic
inhibitory processes and the selection of lexical items from competing alternatives.
Research on the processing of lexically ambiguous words (e.g., bank) has provided
insights into the role of the BG in such aspects of language processing. Copland (2003)
examined semantic priming for ambiguous words in people with PD, people with
non-thalamic vascular subcortical lesions, and healthy controls. Target words were re-
lated to either the dominant (bank-money) or subordinate (bank-river) meaning of the
ambiguous prime word, or were unrelated. While all groups demonstrated priming of
both meanings at a short ISI, a different pattern between groups emerged at the long
ISI. Specifically, at a long ISI it is expected that attentional resources will be directed
toward the dominant meaning of the ambiguity, such that priming should only be
observed for the dominant meaning and not the subordinate meaning. While this pat-
tern of priming at the long ISI was indeed observed in controls, the PD and subcortical
stroke participants continued to demonstrate priming for both conditions. Similarly,
other research has illustrated difficulties with the selective activation of the appropriate
meaning of ambiguous words when presented within repeated, lexical, sentential, or
discourse-based contexts for patients with PD and/or nonthalamic subcortical lesions
(Copland, Chenery, & Murdoch, 2000a, 2000b; Copland, Chenery, & Murdoch, 2001;
Copland, Sefe, Ashley, Hudson, & Chenery, 2009).
Chenery et al. (2008) sought to clarify the nature and involvement of the BG circuitry
in the enhancement and suppression of ambiguous word meanings when presented
within a linguistic context. Specifically, when processing ambiguous word pairs with a
long ISI, Chenery et al. proposed that the inferior frontal BG circuitry is critical, with the
direct pathway enhancing the processing of both dominant and subordinate meanings
and the indirect pathway subsequently suppressing whichever meaning is contextually
inappropriate. As mentioned earlier, there is mounting evidence of BG–inferior frontal
circuitry (Ford et al., 2013), although the components of the striatum involved are not
consistent with an exclusive role of the caudate in such language functions. Providing
some support for this proposal, Ketteler et al. (2014), utilized fMRI to study ambiguity
resolution in PD. Participants performed a semantic judgment task in which they were
asked to decide whether the meanings of two words presented at the top of the screen
both matched an ambiguous target word presented below. In the “yes” trials (referred
to as the homonym condition), where both words were related to the target, one word
was related to the dominant meaning and one word to the subordinate meaning of
the target. There were three “no” conditions, consisting of either a dominant-related
or subordinate-related word together with an unrelated distractor, or two unrelated
distractors. In controls, bilateral caudate activation was evident for the homonym con-
dition, and left caudate activity was evident in the dominant-distractor condition. PD
patients demonstrated lower accuracy for the dominant-distractor and subordinate-
distractor conditions relative to controls, and left caudate activity correlated positively
with task accuracy for these two conditions in the PD group. However, these correlations
were not observed in controls. Based on these findings, Ketteler et al. suggested that left
caudate dysfunction may contribute to semantic meaning-integration deficits in PD.
Importantly, there was no significant relationship between these findings and a measure
of executive function, suggesting that this may not simply reflect broader cognitive
deficits.
The proposed involvement of the BG in lexical ambiguity resolution most likely
reflects a broader function in controlled processing, which includes semantics, a view
supported by observations of increased caudate activity during semantic judgments and

lexical decisions for non-ambiguous words (e.g., Abdullaev, Bechtereva, & Melnichuk,
1998; Binder et al., 2003; Binder et al., 1997; Kotz, Cappa, von Cramon, & Friederici,
2002). In production, Crosson et al. (2003) provided evidence for caudate activity in
healthy subjects during selection for word production as part of a caudate–thalamic–
pre-SMA circuit, while caudate activity has also been observed during word produc-
tion that involves suppressing irrelevant words (Ali, Green, Kherif, Devlin, & Price,
2010), resolving semantic interference (Canini et al., 2016), and selection processes
in sentence generation (Argyropoulos, Tremblay, & Small, 2013). Robles, Gatignol,
Capelle, Mitchell, & Duffau (2005) found that stimulation of the head of the caudate
caused perseveration during naming, consistent with a role in controlling the selec-
tion and inhibition of items during word production. In sum, these findings are con-
sistent with the proposal that the striatum (and caudate in particular) is engaged during
lexical-semantic tasks when controlled, as opposed to automatic, processing is required
(Friederici, 2006). More specifically, different portions of the caudate may be engaged
for different aspects of cognitive control (see Mestres-Misse, Turner, & Friederici, 2012).
The proposed involvement of the BG in selecting and suppressing representations is
consistent with a broader role in selecting motor representations (Mink, 1996) and in
dopaminergic working memory operations (Bäckman & Nyberg, 2013).
The Basal Ganglia Contribution

to Bilingualism
and Language Learning
There is now considerable evidence that the proposed role of the BG in language control
and in particular lexical selection (as discussed earlier) extends to bilingual language
processing (see Green & Kroll, Chapter 11 in this volume). In bilingual speakers, it has
been proposed that the BG govern controlled aspects of language processing, including
language and lexical selection and switching (Abutalebi & Green, 2007). Crinion et al.
(2006) provided compelling evidence that the left caudate monitors and controls
switching of languages and meaning, as demonstrated in a cross-language semantic
priming paradigm (see also Abutalebi & Green, 2007; Rüschemeyer, Fiebach, Kempe, &
Friederici, 2005). This finding also converges with impaired language control observed
in studies of bilingual patients with BG lesions (e.g., Abutalebi & Green, 2007; Calabria,
Marne, Romero-Pinel, Juncadella, & Costa, 2014).
In addition to controlling bilingual language processing, the BG may contribute
to aspects of language learning. There is already strong evidence for the role of the
striatum in various forms of non-linguistic learning, including various forms of pro-
cedural, reward-based, probabilistic, and category learning (Packard & Knowlton,
2002). The proposed role of the BG in procedural learning system and grammar will
be discussed later with respect to Ullman’s (2001b) procedural/declarative model;

however, the BG may also contribute to other forms of learning relevant to language
acquisition. While contemporary models of language learning often focus predom-
inantly on hippocampal-neocortical interactions driving language learning (Davis
& Gaskell, 2009), Rodriguez-Fornells, Cunillera, Mestres-Misse, and de Diego-
Balaguer, 2009 observe that the BG are connected to proposed networks responsible
for various aspects of language learning, including (a) phonological storage and re-
hearsal, (b) meaning extraction, (c) an episodic-lexical interface dependent on the
medial-temporal lobe, and (d) cognitive control and motivation, suggesting a possible
integrative role in combining information from different sources. This role might re-
late to the gating and selection of appropriate representations, as proposed elsewhere
in a non-learning context (Crosson et al, 2003). Consistent with this proposal, Ye et al.
(2011) recently observed the involvement of the caudate in a network including the
left IFG, middle frontal gyrus (MFG), and thalamus that was engaged when selecting
appropriate meanings to associate with a new word representation, while a separate
network (not including the caudate) was involved in integrating the word into a sen-
tential context. Acquisition studies that do not involve semantic learning typically
do not show BG activity (see Davis & Gaskell, 2009), further supporting the notion
that the BG contribute to developing associations between semantic and new lexical
representations.
With regard to the specific components of the BG involved in language learning, this
most likely varies, depending on the aspect of learning engaged, as demonstrated by
Jeon et al. (2014). Intriguingly, the act of learning new word meanings in context also
appears to engage the ventral striatum, suggesting a role for reward systems in this pro-
cess (Ripolle et al., 2014). While not explicitly manipulating reward, Ripolle et al. (2014)
convincingly demonstrated that successful learning of new words in a sentence con-
text was dependent on the ventral striatum, with a non-linguistic reward task showing
activation in the same region, leading the authors to suggest that word learning under
these conditions is intrinsically rewarding and dependent on these subcortical sys-
tems. A subsequent study lent further credence to this proposal, demonstrating that
successful learning of words and activation of the ventral striatum (as part of a mid-
brain–ventral striatum–hippocampus circuit) was associated with increased pleasant-
ness ratings and physiological responses consistent with increased intrinsic reward
(Ripolle et al., 2016). Ventral striatum activity also appears important to other aspects
of semantic processing that are intrinsically rewarding, such as the appreciation of
humor (Bekinschtein, Davis, Rodd, & Owen, 2011). There is also an increasing appre-
ciation of the role of the BG in language learning, based on evidence that children with
language learning difficulties show structural subcortical abnormalities (particularly in
the caudate) and impairments in more domain general–procedural aspects of learning
assumed to involve the striatum (see Krishnan, Watkins, & Bishop, 2016). It is important
to note that some of the strongest evidence for subcortical contributions to language
learning have also demonstrated that this role reflects a broader role in similar forms of
learning (Jeon et al., 2014; Ripolle et al., 2014).
The Basal Ganglia and Verb Processing
There is now a body of work suggesting that BG dysfunction (particularly as observed

in PD) is associated with a selective impairment of action/verb processing, although the
selectivity of this proposal is at odds with evidence of general lexical-semantic deficits
associated with PD and BG lesions (discussed earlier). This view has been supported
by evidence of deficits in verb generation, action naming, and action verbal fluency, to-
gether with observations of increased deficits in PD patients off levodopa medication
(see Cardona et al., 2013, for a review). Importantly, recent evidence also suggests that
impairments to action naming and action semantics in PD occur independently of ex-
ecutive function deficits (Bocanegra et al., 2015). While claims have been made that
such deficits relate to motor impairments in PD, it should be noted that the relation-
ship between verb processing and motor circuitry is not well established. Some novel
insights were provided by Peran et al. (2013), who observed increased activation within
the premotor cortex for PD patients on versus off levodopa during action word gen-
eration, which was interpreted as consistent with levodopa enhancing activity within
the motor putaminal cortical striatal loop during action semantic processing. The func-
tional significance of this brain activity is also difficult to interpret, given that there were
no behavioral changes on versus off levodopa for action word generation. More recently,
Herrera, Bermudez-Margaretto, Ribacoba, and Cuetos. (2015) tested action verbal flu-
ency in PD patients on and off medication and examined the semantic-motor meaning
of the movement verbs that patients generated during task performance. The movement
verbs were classified as those with a higher motor specificity (i.e., those that denoted an
action performed with small parts of the body, e.g., sew, knit) versus those with a lower
motor specificity (i.e., those denoting actions that included the whole body, e.g., swim,
run). Herrera et al. found that both controls and PD patients on dopamine medication
produced more verbs with a high motor specificity compared to when the patients were
tested off medication.
However, it should be noted that discrepancies remain on the particular question of
verb processing in PD. For instance, while Fernandino et al. (2013) observed impaired
processing of action but not abstract verbs, compared to controls, Kemmerer et al. (2013)
found no significant difference between controls and PD patients (both on and off do-
paminergic medication) when processing action and abstract verbs, besides an overall
slowing in RTs regardless of verb type (see Kemmerer, Miller, MacPherson, Huber, &
Tranel, 2013, for a critique of the Fernandino et al., 2013, study).
Studies investigating a phenomenon known as the action compatibility effect (ACE)
have also suggested the potential role of BG circuitry in action processing. During
an ACE task, participants are required to listen to sentences and press a button upon
comprehension of each sentence. Sentences either describe an action performed with
an open hand, describe an action performed with a closed hand, or do not describe a
hand action (neutral sentences). Critical to the ACE task is the nature of the button press
response. In some blocks of trials, participants must press the button with an open hand,
and in other blocks it must be pressed with a closed hand. Accordingly, this creates com-
patible trials (i.e., an open-hand sentence/response or closed-hand sentence/response),
incompatible trials (i.e., an open-hand sentence and closed-hand response or vice versa)
and neutral trials (i.e., a neutral sentence paired with either type of response).
In healthy adults, the ACE effect refers to longer reaction times for incompatible rel-
ative to compatible trials; however, Ibanez et al. (2013) observed an absence of the ACE
effect in early PD patients. This deficit appeared inconsistent with a general motor or
language impairment in PD, however, a post hoc item analysis found that increased
errors on a test of verb-action comprehension for trials related to hand actions was asso-
ciated with a reduced ACE effect in PD. This correlation was not evident for errors made
on trials unrelated to hand actions. It should be noted that this post hoc item-based anal-
ysis did not allow for matching of these categories on other important variables that may
also influence performance. Research also indicates the ACE effect is absent in people
with HD as well as their first-degree relatives (Kargieman et al., 2014).
Cardona et al. (2013) have proposed that BG-thalamo-cortical circuitry, with dis-
tinct motor versus semantic circuits, may modulate the motor-language interaction
evident during action/verb processing. Specifically, a frontal circuit would be involved
in the processing of motor simulation and action patterns associated with the meaning
of verbs. A temporal circuit would be involved in abstract conceptual knowledge and
could directly impact semantic processing in areas including the anterior temporal lobe
and superior temporal sulcus. Further evidence provided in support of this proposal
comes from neuroimaging and deep brain stimulation (DBS). Measurement of event-
related potentials (ERPs; see Leckey & Federmeier, Chapter 3 in this volume) has shown
aberrant motor potentials in PD relative to controls during performance of the ACE
task (Melloni et al., 2015). In both PD and controls, this motor potential effect was asso-
ciated with gray matter volume in bilateral BG structures, including the putamen, cau-
date nucleus, and globus pallidus. Moreover, increased BG atrophy in PD was correlated
with increased impairments to the motor potential. These findings supported Cardona
et al.’s proposed role of the BG in motor-language coupling through its involvement in
frontotemporal circuitry. The behavioral significance of these findings remains unclear,
however, given that the relationship between the behavioral ACE effect and the MRI and
ERP results was not tested in this study.
In a study of action and object naming in PD patients with DBS of the STN, Silveri
et al. (2012) found that while the accuracy and speed of action naming in PD patients on
stimulation did not differ to controls, the PD patients off stimulation were slower and
less accurate than controls. PD patients off DBS were also slower to name actions rela-
tive to objects, while no difference was evident for controls or patients on stimulation.
Moreover, Silveri et al. also observed qualitative differences in naming performance
between PD patients on versus off stimulation, with fewer semantic errors evident for
patients on stimulation. Silveri et al. suggested that stimulation may restore activity
within the cortico-striatal circuits necessary for retrieval and selection of lexical-
semantic information; however, the implications for proposals regarding verb-specific
processing and the BG are unclear, as direct comparisons for on versus off stimulation
showed faster and more accurate naming regardless of action versus objects.
Subcortical Contributions
to Sentence
and Grammatical Processing
Relative to healthy adults, difficulties processing complex sentences are more pro-
nounced for some patients with PD (Grossman et al., 2000; Kemmerer, 1999; Lee,
Grossman, Morris, Stern, & Hurtig, 2003; Natsopoulos et al., 1991). While some
researchers have postulated that sentence comprehension impairments in PD stem
from grammatical processing deficits (Lieberman et al., 1992; Natsopoulos et al., 1991;
Natsopoulos et al., 1993), others have attributed the deficits to limitations in the avail-
ability of cognitive resources, including working memory (Geyer & Grossman, 1994;
Grossman, Lee, Morris, Stern, & Hurtig, 2002; Kemmerer, 1999; McNamara, Krueger,
O’Quin, Clark, & Durso, 1996), attention (Grossman, Carvell, Stern, Gollomp, & Hurtig,
1992; Lee et al., 2003), and/or information-processing speed (Grossman, Zurif, et al.,
2002; Lee et al., 2003).
A related account based on healthy neuroimaging proposes that the striatum
contributes to a cortical syntactic unification network (involving the left IFG and left
MTG) by influencing the selection of lexical-syntactic representations (Snijders et al.,
2010; Hagoort, 2005). It is argued that the striatum provides a gating mechanism for
working memory, supporting the extraction of these representations from the left MTG
for the unification operations carried out by the left IFG. Accordingly, presentation of
ambiguous sentences is associated with increased co-activation of the striatum, MTG,
and left IFG. Snijders et al. (2010) hypothesize that ambiguous sentences represent sa-
lient events that cause dopamine release, with this signal allowing the striatum to mod-
ulate cortical transfer of lexical-syntactic information. Although the role of dopamine
in this process remains to be verified, there is supporting evidence for the broader role
of the striatum in updating working memory and selecting/suppressing competing
representations (Cools, 2011), including lexical items for production (Crosson
et al., 2007).
The declarative/procedural model proposed by Ullman and colleagues (Ullman,
2001a, 2001b; Ullman et al., 1997) is another model that may be applicable to the
sentence-processing deficits in PD and favors the notion that the subcortex contributes
to grammatical processing. According to this model, the declarative memory system
underlies the mental lexicon, storing knowledge of the sounds and meanings of words,
and is subserved by a medial temporal circuit. The procedural memory system, on the
other hand, is subserved by frontal/BG structures and may underlie the learning and
expression of grammatical rules. Thus, the model predicts that irregular verbs (e.g.,
bring-brought), which constitute a fixed list and do not involve overt manipulation of
morphological rules, will be stored in the declarative system. In contrast, the procedural
system may be important for rule-based building of grammatical structures such as reg-
ular verbs (e.g., walk-walked), since the learning and computation of these structures is
dependent on morphological sequencing (e.g., walked = walk + ed). Ullman proposed
that given the subcortical degeneration in PD, impairment to the procedural system
would be expected, while the declarative system should be unaffected.
In support of this theory, Ullman et al. (1997) found that PD patients experienced
greater difficulties producing regular past-tense verbs compared to irregular past-
tense verbs. Accordingly, it may be argued that sentence-processing deficits in PD are
related to a general inability to apply such procedural rules of grammar. A number of
other researchers have also attributed sentence-processing difficulties in PD to specific
grammatical parsing deficits. Lieberman et al. (1992) found that PD patients’ sentence
comprehension was impaired on a sentence-picture-matching task, and that the mag-
nitude of this impairment was a function of both the stage of PD and the complexity
of the sentence. It was argued that these deficits might reflect impairment in the ability
to apply the rules of syntax appropriately. Similarly, Natsopoulos et al. (1991, 1993) also
observed that PD patients experienced difficulties comprehending complex sentences
as measured by a sentence-picture-matching task. More important, the researchers il-
lustrated that the PD patients often assigned the noun phrase (NP) closest to the rel-
ative clause verb as the subject of that relative clause, suggesting that PD patients may
rely on some form of minimal distance principle to aid in comprehension. In terms of
language production, Phillips et al. (2012) observed a decline in accuracy and response
time for the production of regular past-tense verbs when the patients were on relative to
off DBS. This finding suggests that STN stimulation has a negative impact on grammat-
ical aspects of language processing and implicates BG circuitry.
However, Ullman’s proposed role for the BG in grammatical processing was chal-
lenged in a convincing fashion by Longworth, Keenan, Barker, Marslen-Wilson, and
Tyler (2005), who found that the ability to inflect regular verbs was not significantly
more impaired than the ability to inflect irregular verbs for patients with PD, HD, and
vascular lesions of the striatum, suggesting that BG disorders are not associated with a
selective deficit for regular inflectional morphology. Penke, Janssen, Indefrey, & Reitz
(2005) also observed no evidence supporting a deficit in processing grammatical rules in
German PD patients. Certainly, this raises doubts over whether the sentence-processing
deficits in PD are related to dysfunctional application of grammatical rules. Longworth
et al. (2005) instead found that subcortical dysfunction was associated with an impair-
ment in later language processes involving the selection and inhibition of competing
alternatives, consistent with a broader role for the BG in selection and suppression in
both language and non-language domains, as described elsewhere (Copland, 2003;
Crosson et al., 2003).
Deficits to lexical access as a result of BG dysfunction in PD may have subsequent
downstream consequences for sentence comprehension. Numerous studies have il-
lustrated longer delays to lexical-semantic activation in PD patients, with increased
difficulties comprehending complex sentences (Angwin, Chenery, Copland, Murdoch,

& Silburn, 2005; Angwin et al., 2007; Grossman et al., 2002). Other research has
found that sentence comprehension impairments in PD are not specific to more com-
plex constructions, but rather that PD is associated with a more generalized sentence
comprehension impairment that is likely dependent on executive function (Colman
et al., 2011).
In an fMRI study, Grossman et al. (2003) observed reduced striatal recruitment in
PD patients during the processing of complex sentences. More recently, Ye et al. (2012)
observed increased caudate activation in healthy controls during the processing of more
complex sentences, consistent with the increased processing demands associated with
these sentences. In contrast, the PD group demonstrated the opposite pattern, with
increased caudate activation for the less demanding sentence type.
Plausibility judgments for idiomatic ambiguous sentences have been shown to be
reduced in PD patients tested off medication, which may be consistent with reduced
availability of cognitive resources to support interpretation when off medication
(Papagno, Mattavelli, Cattaneo, Romito, & Albanese, 2013). Other research, however,
indicates that the contribution of the striatum to sentence processing is not constrained
purely to the modulation of lexical access or the availability of general cognitive re-
sources such as working memory. Sambin et al. (2012) demonstrated that patients with
HD experience specific difficulties applying infrequently used syntactic operations to
appropriate sentence contexts. Specifically, while HD patients established a co-reference
between a name and a pronoun in an ambiguous sentence such as “Paul smiled when he
entered,” where “Paul” and “he” can potentially refer to the same person, the HD patients
failed to appropriately block such a co-referential interpretation when presented with a
sentence such as “He smiled when Paul entered.” These deficits were observed despite
the normal performance of the HD patients during processing of right branching and
center-embedded relative sentences, suggesting that working memory deficits cannot
account for the observed syntactic processing difficulties. Such findings led Sambin
et al. to propose that the HD patients may have a deficit in the ability to inhibit or switch
from frequently used to infrequent syntactic operations. This account is consistent with
previously noted proposals that the striatum serves to suppress competing alternatives
in lexical-semantic processing (e.g., Copland, 2003). Impaired language-related rule ap-
plication and lexical processing in HD have also been shown to correlate with disso-
ciable components of the BG (Teichmann et al., 2008). Specifically, aspects of lexical
processing correlated with dorsal portions of the caudate and more ventral portions
of the caudate and putamen for aspects of rule application. More recently, Teichmann
observed that impairments with complex syntax comprehension following surgical
frontal-striatal lesions were associated with damage to a tract connecting the left cau-
date head with left IFG (BA 45).
Utilizing the measurement of event-related potentials during sentence processing,
Friederici, Rüschemeyer, Hahne, and Fiebach (2003) found that early automatic syn-
tactic processes, as indexed by the early negativity, were intact in PD. In contrast, how-
ever, the P600 component was reduced in PD relative to healthy controls in response to
processing of phrase structure violations, suggesting that the BG support late processes
of syntactic integration during sentence comprehension. Providing further support for
this interpretation, ERP research has also documented that verb-argument structure
violations elicit a P600 in patients with lesions that exclude the BG, whereas this com-
ponent is absent in patients with BG lesions (Kotz, Frisch, Von Cramon, & Friederici,
2003). Subsequent work suggested that this finding was not due to a more generalized
deficit in controlled or attentional processing (see Kotz, Schwartze, & Schmidt-Kassow,
2009). Instead, an intriguing alternative proposal suggested that these syntax deficits
are secondary to dysfunction in a medial pre-SMA–BG circuit that processes temporal
variations or patterns in the auditory speech signal, and making predictions in speech
comprehension (Kotz et al., 2009), reflecting a broader role in sequencing and temporal
processing.
Conclusion
There have been considerable advances in our understanding of functional and struc-
tural cortical-subcortical connectivity that further implicates subcortical structures
in language-relevant processes, and there is a growing appreciation that traditional
notions of a series of functionally and anatomically segregated subcortical-cortical
loops need to be revised. The current review did not consider in-depth evidence from
the burgeoning area of DBS in PD, which has recently provided numerous studies in
how striato-cortical circuit modulation may influence language processing, as this effort
has yielded a highly variable and at times conflicting set of observations. This includes
improvements (Castner et al., 2007; Zanini et al., 2002), impairments (Castner et al.,
2008; Parsons, Rogers, Braaten, Woods, & Troster, 2006), and no influence (Batens
et al., 2014; Castner et al., 2007; Tremblay et al., 2015) on various aspects of language,
suggesting the need to better understand and identify the actions of DBS on specific
cortical and subcortical components and circuits and the need to account for factors
such as laterality, disease duration, and levodopa, before we can meaningfully interpret
behavioral effects.
Based on patient and healthy neuroimaging studies, there is mounting evidence for
the contribution of the thalamus and striatum to various aspects of language func-
tion beyond initial proposals based on clinical-anatomical correlations. However, ev-
idence for a unique and specific role for the BG in language remains uncertain, with
mixed findings, for instance, regarding a proposed selective role in action-verb pro-
cessing. Instead, there is considerable evidence from patient studies and neuroimaging
in healthy individuals consistent with domain-general role of the thalamus in atten-
tional engagement and working memory (seen in semantic processing) and for striatal-
thalamic-cortical circuits in cognitive control, as demonstrated in the selection and
suppression of representations, which impacts on word selection (in production and
comprehension), bilingual language processing, and sentence comprehension. There
are emerging theories suggesting that other general subcortical functions relating to
learning, temporal processing, and sequencing may also influence language processing.
Combined neuroimaging and patient studies in addition to research investigating do-
paminergic and genetic contributions to the language-relevant subcortical mechanisms
are also likely to shed further light on how these regions modulate language function
across a range of populations.
References
Abdullaev, Y. G., Bechtereva, N. P., & Melnichuk, K. V. (1998). Neuronal activity of human cau-
date nucleus and prefrontal cortex in cognitive tasks. Behavioral Brain Research, 97, 159–177.
Abdullaev, Y. G., & Melnichuk, K. V. (1997). Cognitive operations in the human caudate nu-
cleus. Neuroscience Letters, 234, 151–155.
Abutalebi, J., Annoni, J. M., Zimine, I., Pegna, A. J., Seghier, M. L., Lee-Jahnke, H., et al. (2008).
Language control and lexical competition in bilinguals: An event-related FMRI study.
Abutalebi, J., Brambati, S. M., Annoni, J. M., Moro, A., Cappa, S. F., & Perani, D. (2007).
The neural cost of the auditory perception of language switches: An event-related func-
tional magnetic resonance imaging study in bilinguals. Journal of Neuroscience, 27(50),
13762–13769.
Abutalebi, J., & Green, D. (2007). Bilingual language production: The neurocognition of lan-
guage representation and control. Journal of Neurolinguistics, 20(3), 242–275.
Argyropoulos, G., Tremblay, P., & Small, S. (2013). The neostriatum and response selection in
overt sentence production: An fMRI study. Neuroimage, 82,  53–60.
Alexander, G. E., DeLong, G. E., & Strick, P. L. (1986). Parallel organization of functionally
segregated circuits linking basal ganglia and cortex. Annual Review of Neuroscience, 9,
357–381.
Ali, N., Green, D. W., Kherif, F., Devlin, J. T., & Price, C. J. (2010). The role of the left head of cau-
date in suppressing irrelevant words. Journal of Cognitive Neuroscience, 22(10), 2369–2386.
Angwin, A. J., Arnott, W. L., Copland, D. A., Haire, M. P. L., Murdoch, B. E., Silburn, P. A., &
Chenery, H. J. (2009). Semantic activation in Parkinson’s disease patients on and off levo-
dopa. Cortex, 45(8), 950–959.
Angwin, A. J., Chenery, H. J., Copland, D. A., Arnott, W. L., Murdoch, B. E. & Silburn, P. A.
(2004). Dopamine and semantic activation: An investigation of masked direct and indirect
priming. Journal of the International Neuropsychological Society, 10(1), 15–25.
Angwin, A. J., Chenery, H. J., Copland, D. A., Murdoch, B. E., & Silburn, P. A. (2005).
Summation of semantic priming and complex sentence comprehension in Parkinson’s di-
sease. Cognitive Brain Research, 25(1),  78–89.
Angwin, A. J., Chenery, H. J., Copland, D. A., Murdoch, B. E., & Silburn, P. A. (2006). Self-paced
reading and sentence comprehension in Parkinson’s disease. Journal of Neurolinguistics, 19,
239–252.
Angwin, A. J., Chenery, H. J., Copland, D. A., Murdoch, B. E., & Silburn, P. A. (2007). The speed
of lexical activation is altered in Parkinson’s disease. Journal of Clinical and Experimental
Neuropsychology, 29,  73–85.
Angwin, A. J., Copland, D. A., Chenery, H. J., Murdoch, B. E., & Silburn, P. A. (2006). The
influence of dopamine on semantic activation in Parkinson’s disease: Evidence from a
multipriming task. Neuropsychology, 20(3), 299–306.
Arnott, W. L., Chenery, H. J., Murdoch, B. E., & Silburn, P. A. (2001). Semantic priming in
Parkinson’s disease: Evidence for delayed spreading activation. Journal of Clinical and
Arnott, W. L., Chenery, H. J., Angwin, A. J., Murdoch, B. E., Silburn, P. A., & Copland, D. A.
(2010). Decreased semantic competitive inhibition in Parkinson’s disease: Evidence from
an investigation of word search performance. International Journal of Speech-Language
Pathology, 12(5), 437–445.
Bäckman, L., & Nyberg, L. (2013). Dopamine and training-related working-memory improve-
ment. Neuroscience & Biobehavioral Reviews, 37(9), 2209–2219.
Batens, K., De Letter, M., Raedt, R., Duyck, W., Vanhoutte, S., Van Roost, D., et al. (2014). The
effects of subthalamic nucleus stimulation on semantic and syntactic performance in spon-
taneous language production in people with Parkinson’s disease. Journal of Neurolinguistics,
32,  31–41.
Bekinschtein, T. A., Davis, M. H., Rodd, J. M., & Owen, A. M. (2011). Why clowns taste
funny: The relationship between humor and semantic ambiguity. Journal of Neuroscience,
31(26), 9665–9671.
Bell, P. T., & Shine, J. M. (2016). Subcortical contributions to large-scale network communica-
tion. Neuroscience & Biobehavioral Reviews, 71, 313–322.
Bilenko, N. Y., Grindrod, C. M., Myers, E. B., & Blumstein, S. E. (2009). Neural correlates
of semantic competition during processing of ambiguous words. Journal of Cognitive
Binder, J. R., Frost, J. A., Hammeke, T. A., Cox, R. W., Rao, S. M., & Prieto, T. (1997). Human
brain language areas identified by functional magnetic resonance imaging. Journal of
Binder, J. R., McKiernan, K. A., Parsons, M. E., Westbury, C. F., Possing, E. T., Kaufman, J. N.,
& Buchanan, L. (2003). Neural correlates of lexical access during visual word recognition.
Blumstein, S. E. (1997). A perspective on the neurobiology of language. Brain and Language,
60, 335–346.
Bocanegra, Y., Garcia, A. M., Pineda, D., Buritica, O., Villegas, A., Lopera, F., . . . Ibanez, A.
(2015). Syntax, action verbs, action semantics, and object semantics in Parkinson’s di-
sease: Dissociability, progression, and executive influences. Cortex, 69, 237–254.
Calabria, M., Marne, P., Romero-Pinel, L., Juncadella, M., & Costa, A. (2014). Losing control of
your languages: A case study. Cognitive Neuropsychology, 31, 266–286.
Canini, M., Della Rosa, P. A., Catricalà, E., Strijkers, K., Branzi, F. M., Costa, A., & Abutalebi, J.
(2016). Semantic interference and its control: A functional neuroimaging and connectivity
study. Human Brain Mapping, 37(11), 4179–4196.
Cardona, J. F., Gershanik, O., Gelormini-Lezama, C., Houck, A. L., Cardona, S., Kargieman,
L., . . . Ibanez, A. (2013). Action-verb processing in Parkinson’s disease: New pathways for
motor-language coupling. Brain Structure and Function, 218, 1355–1373.
Castner, J. E., Chenery, H. J., Copland, D. A., Coyne, T. J., Sinclair, F., & Silburn, P. A. (2007).
Semantic and affective priming as a function of stimulation of the subthalamic nucleus in
Parkinson’s disease. Brain, 130, 1395–1407.
Castner, J. E., Copland, D. A., Silburn, P. A., Coyne, T. J., Sinclair, F., & Chenery, H. J. (2007).
Lexical-semantic inhibitory mechanisms in Parkinson’s disease as a function of subthalamic
stimulation. Neuropsychologia, 45(14), 3167–3177.
Castner, J. E., Copland, D. A., Silburn, P. A., Coyne, T. J., Sinclair, F., & Chenery, H. J. (2008).
Subthalamic stimulation affects homophone meaning generation in Parkinson’s disease.
Journal of the International Neuropsychological Society, 14, 890–894.
Chan, S. H., Ryan, L., & Bever, T. G. (2013). Role of the striatum in language: Syntactic and con-
ceptual sequencing. Brain and Language, 125(3), 283–294.
Chenery, H. J., Angwin, A. J., & Copland, D. A. (2008). The basal ganglia circuits, dopa-
mine, and ambiguous word processing: A neurobiological account of priming studies in
Parkinson’s disease. Journal of the International Neuropsychological Society, 14(3), 351–364.
Colman, K. S. F., Koerts, J., Stowe, L. A., Leenders, K. L., & Bastiaanse, R. (2011). Sentence com-
prehension and its association with executive functions in patients with Parkinson’s disease.
Parkinson’s Disease, 2011, 213983.
Cools, R. (2011). Dopaminergic control of the striatum for high-level cognition. Current
Opinion in Neurobiology, 21(3), 402–407.
Copland, D. (2003). The basal ganglia and semantic engagement: Potential insights from se-
mantic priming in individuals with subcortical vascular lesions, Parkinson’s disease, and
cortical lesions. Journal of the International Neuropsychological Society, 9(7), 1041–1052.
Copland, D. A., Chenery, H. J., & Murdoch, B. E. (2000a). Processing lexical ambiguities in
word triplets: Evidence of lexical-semantic deficits following dominant nonthalamic sub-
cortical lesions. Neuropsychology, 14(3), 379–390.
Copland, D. A., Chenery, H. J., Murdoch, B.E. (2000b). Understanding ambiguous words in
biased sentences: Evidence of transient contextual effects in individuals with nonthalamic
subcortical lesions and Parkinson’s disease. Cortex, 36(5), 601–622.
Copland, D. A., Chenery, H. J., & Murdoch, B. E. (2001). Discourse priming of homophones in
individuals with dominant nonthalamic subcortical lesions, cortical lesions and Parkinson’s
disease. Journal of Clinical and Experimental Neuropsychology, 23, 538–556.
Copland, D. A., Sefe, G., Ashley, J., Hudson, C., & Chenery, H.J. (2009). Impaired semantic in-
hibition during lexical ambiguity repetition in Parkinson’s disease. Cortex, 45(8), 943–949.
Crinion, J., Turner, R., Grogan, A., Hanakawa, T., Noppeney, U., Devlin, J. T., . . . Price, C. J.
(2006). Language control in the bilingual brain. Science, 312(5779), 1537–1540.
Crosson, B. (1992). Subcortical functions in language and memory. New York: Guilford Press.
Crosson, B. (1999). Subcortical mechanisms in language: Lexical-semantic mechanisms and
the thalamus. Brain and Cognition, 40, 414–438.
Crosson, B. (2013). Thalamic mechanisms in language: A reconsideration based on recent
findings and concepts. Brain and Language, 126,  73–88.
Crosson, B., Benefield, H., Cato, M. A., Sadek, J. R., Moore, A. B., Wierenga, C. E., . . . Briggs,
R.W. (2003). Left and right basal ganglia and frontal activity during language gener-
ation: Contributions to lexical, semantic, and phonological processes. Journal of the
International Neuropsychological Society, 9(7), 1061–1077.
Crosson, B., Benjamin, M., & Levy, I. (2007). Role of the basal ganglia in language and seman-
tics: Supporting cast. In J. Hart, Jr., & M. Kraut (Eds.), Neural basis of semantic memory (pp.
219–243). New York: Cambridge University Press.
Davis, M. H., & Gaskell, M. G. (2009). A complementary systems account of word
learning: Neural and behavioural evidence. Philosophical Transactions of the Royal Society
B-Biological Sciences, 364(1536), 3773–3800.
De Letter, M., Aerts, A., Van Borsel, J., Vanhoutte, S., De Taeye, L., Raedt, R., et al. (2014).
Electrophysiological registration of phonological perception in the subthalamic nucleus of
patients with Parkinson’s disease. Brain and Language, 138,  19–26.
Démonet, J.-F., Puel, M., Celsis, P., & Cardebat, D. (1991). “Subcortical” aphasia: Some proposed
pathophysiological mechanisms and their rCBF correlates revealed by SPECT. Journal of
Neurolinguistics, 6(3), 319–344.
Durstewitz, D., & Seamans, J. K. (2008). The dual-state theory of prefrontal cortex dopamine
function with relevance to catechol-o-methyltransferase genotypes and schizophrenia.
Biological Psychiatry, 64(9), 739–749.
Ehlen, F., Krugel, L. K., Vonberg, I., Schoenecker, T., Kuhn, A. A., & Klostermann, F. (2013).
Intact lexicon running slowly: Prolonged response latencies in patients with subthalamic
DBS and verbal fluency deficits. PLoS One, 8, e79247.
Elorriaga-Santiago, S., Silva-Pereyra, J., Rodriguez-Camacho, M., & Carrasco-Vargas, H.
(2013). Phonological processing in Parkinson’s disease: A neuropsychological assessment.
Fernandino, L., Conant, L. L., Binder, J. R., Blindauer, K., Hiner, B., Spangler, K., et al. (2013).
Parkinson’s disease disrupts both automatic and controlled processing of action verbs. Brain
Filoteo, J. V., Friedrich, F. J., Rilling, L. M., Davis, J. D., Stricker, J. L., & Prenovitz, M. (2003).
Semantic and cross-case identity priming in patients with Parkinson’s disease. Journal of
Clinical and Experimental Neuropsychology, 25, 441–456.
Filoteo, J. V., Rilling, L. M., & Strayer, D. L. (2002). Negative priming in patients with
Parkinson’s disease: Evidence for a role of the striatum in inhibitory attentional processes.
Ford, A. A., Triplett, W., Sudhyadhom, A., Gullett, J., McGregor, K., FitzGerald, D. B., Mareci,
T., White, K., Crosson, B. (2013). Broca’s area and its striatal and thalamic connections: A
diffusion-MRI tractography study. Frontiers in Neuroanatomy, 7, 8.
Forkstam, C., Hagoort, P., Fernandez, G., Ingvar, M., & Petersson, K. M. (2006). Neural
correlates of artificial syntactic structure classification. NeuroImage, 32(2), 956–967.
Freire, L., Roche, A., & Mangin, J. F. (2002). What is the best similarity measure for motion cor-
rection in fMRI time series? IEEE Transactions on Medical Imaging, 21(5), 470–484.
Friederici, A. D. (2006). What’s in control of language? Nature Neuroscience, 9(8), 991–992.
comprehension. Trends in Cognitive Science, 16(5), 262–268.
Friederici, A. D., & Kotz, S. A. (2003). The brain basis of syntactic processes: Functional im-
aging and lesion studies. NeuroImage, 20(Suppl 1), S8–S17.
Friederici, A. D., Kotz, S. A., Werheid, K., Hein, G., & Von Cramon, D. Y. (2003). Syntactic
comprehension in Parkinson’s disease: Investigating early automatic and late integrational
processes using event-related potentials. Neuropsychology, 17, 133–142.
Friederici, A. D., Rüschemeyer, S. A., Hahne, A., & Fiebach, C. J. (2003). The role of left inferior
frontal and superior temporal cortex in sentence comprehension: Localizing syntactic and
semantic processes. Cerebral Cortex, 13(2), 170–177.
Gernsbacher, M. A., & Faust, M. E. (1991). The mechanism of suppression: A component of
general comprehension skill. Journal of Experimental Psychology: Learning, Memory &
Cognition, 17(2), 245–262.
Geyer, H. L., & Grossman, M. (1994). Investigating the basis for the sentence comprehension
deficits in Parkinson’s disease. Journal of Neurolinguistics, 8, 191–205.
Neurology, Neurosurgery, and Psychiatry, 76(7), 940–946.
Grossman, M., Carvell, S., Stern, M. B., Gollomp, S., & Hurtig, H. I. (1992). Sentence compre-
hension in Parkinson's disease: The role of attention and memory. Brain and Language, 42,
347–384.
Grossman, M., Cooke, A., DeVita, C., Lee, C., Alsop, D., Detre, J., et al. (2003). Grammatical
and resource components of sentence processing in Parkinson’s disease: An fMRI study.
Neurology, 60, 775–781.
Grossman, M., Lee, C., Morris, J., Stern, M. B., & Hurtig, H. I. (2002). Assessing resource
demands during sentence processing in Parkinson’s disease. Brain and Language, 80(3),
603–616.
Grossman, M., Kalmanson, J., Bernhardt, N., Morris, J., Stern, M. B., & Hurtig, H. I. (2000).
Cognitive resource limitations during sentence comprehension in Parkinson’s disease. Brain
and Language, 73(1), 1–16.
Grossman, M., Zurif, E., Lee, C., Prather, P., Kalmanson, J., Stern, M. B., et al. (2002).
Information processing speed and sentence comprehension in Parkinson's disease.
Haber, S. N., Fudge, J. L., & McFarland, N. R. (2000). Striatonigrostriatal pathways in primates
form an ascending spiral from the shell to the dorsolateral striatum. Journal of Neuroscience,
20(6), 2369–2382.
Hart, J. Jr, Maguire, M. J., Motes, M., Mudar, R. A., Chiang, H., Womack, K. B., & Kraut, M. A.
(2013). Semantic memory retrieval circuit: Role of pre-SMA, caudate, and thalamus. Brain
Herrera, E., Bermudez-Margaretto, B., Ribacoba, R., & Cuetos, F. (2015). The motor-semantic
meanings of verbs generated by Parkinson’s disease patients on/off dopamine medication in
a verbal fluency task. Journal of Neurolinguistics, 36,  72–78.
Hillis, A. E., Wityk, R. J., Braker, P. B., Beauchamp, N. J., Gailloud, P., Murphy, K., et al. (2002).
Subcortical aphasia and neglect in acute stroke: The role of cortical hypoperfusion. Brain,
125, 1094–1104.
Ibanez, A., Cardona, J. F., Dos Santos, Y., Blenkmann, A., Aravena, P., Roca, M., . . . Bekinschtein,
T. (2013). Motor-language coupling: Direct evidence from early Parkinson’s disease and in-
tracranial recordings. Cortex, 49, 968–984.
Jeon, H. A., Anwander, A., & Friederici, A. D. (2014). Functional network mirrored in the pre-
frontal cortex, caudate nucleus, and thalamus: High-resolution functional imaging and
structural connectivity. Journal of Neuroscience, 34, 9202–9212.
Jokinen, P., Karrasch, M., Bruck, A., Johansson, J., Bergman, J., & Rinne, J. O. (2013). Cognitive
Slowing in Parkinson’s disease is related to frontostriatal dopaminergic dysfunction. Journal
of the Neurological Sciences, 329,  23–28.
Kargieman, L., Herrera, E., Baez, S., Garcia, A.M., Dottori, M., Gelormini, C., et al. (2014). Motor-
language coupling in Huntington’s disease families. Frontiers in Aging Neuroscience, 6, 122.
Kemmerer, D. (1999). Impaired comprehension of raising- to-subject constructions in
Parkinson’s disease. Brain and Language, 66, 311–328.
Kemmerer, D., Miller, L., MacPherson, M. K., Huber, J., & Tranel, D. (2013). An investiga-
tion of semantic similarity judgments about action and non-action verbs in Parkinson’s
disease: Implications for the Embodied Cognition Framework. Frontiers in Human
Ketteler, D., Kastrau, F., Vohn, R., Huber, W. (2008). The subcortical role of language pro-
cessing. High level linguistic features such as ambiguity-resolution and the human brain; an
fMRI study. NeuroImage, 39(4), 2002–2009.
Ketteler, S., Ketteler, D., Vohn, R., Kastrau, F., Schulz, J. B., Reetz, K., & Huber, W. (2014). The
processing of lexical ambiguity in healthy ageing and Parkinsons disease: Role of cortico-
subcortical networks. Brain Research, 1581,  51–63.
Kischka, U., Kammer, T., Maier, S., Weisbrod, M., Thimm, M., & Spitzer, M. (1996). Dopaminergic
modulation of semantic network activation. Neuropsychologia, 34(11), 1107–1113.
Kish, S. J., Shannak, K., & Hornykiewicz, O. (1988). Uneven pattern of dopamine loss in the
striatum of patients with idiopathic Parkinson’s disease. New England Journal of Medicine,
318(14), 876–880.
Kotz, S. A., Cappa, S. F., von Cramon, D. Y., & Friederici, A. D. (2002). Modulation of the
lexical-semantic network by auditory semantic priming: An event-related functional MRI
study. NeuroImage, 17, 1761–1772.
Kotz, S. A., Frisch, S., Von Cramon, D. Y., & Friederici, A. D. (2003). Syntactic language
processing: ERP lesion data on the role of the basal ganglia. Journal of the International
Neuropsychological Society, 9, 1053–1060.
Kotz, S. A., Schwartze, M., & Schmidt- Kassow, M. (2009). Non- motor basal ganglia
functions: A review and proposal for a model of sensory predictability in auditory language
perception. Cortex, 45(8), 982–990.
Kraut, M. A., Kremen, S., Moo, L. R., Segal, J. B., Calhoun, V., & Hart, J., Jr. (2002). Object
activation in semantic memory from visual multimodal feature input. Journal of Cognitive
Neuroscience, 14(1),  37–47.
Krishnan, S., Watkins, K. E., & Bishop, D. V. (2016). Neurobiological basis of language learning
difficulties. Trends in Cognitive Sciences, 20(9), 701–7 14.
Lavigne, F., & Darmon, N. (2008). Dopaminergic neuromodulation of semantic priming in a
cortical network model. Neuropsychologia, 46(13), 3074–3087.
Lee, C., Grossman, M., Morris, J., Stern, M. B., & Hurtig, H. I. (2003). Attentional resource and
processing speed limitations during sentence processing in Parkinson’s disease. Brain and
Language, 85, 347–356.
Lehericy S., Ducros M., Van de Moortele, P. F., Francois, C., Thivard, L., Poupon, C., et al.
(2004). Diffusion tensor fiber tracking shows distinct corticostriatal circuits in humans.
Annals of Neurolology, 55, 522–529.
Lieberman, P., Kako, E., Friedman, J., Tajchman, G., Feldman, L. S., & Jiminez, E. B. (1992).
Speech production, syntax comprehension, and cognitive deficits in Parkinson’s disease.
Longworth, C. E., Keenan, S. E., Barker, R. A., Marslen-Wilson, W. D., & Tyler, L. K. (2005).
The basal ganglia and rule-governed language use: Evidence from vascular and degenerative
conditions. Brain, 128(3), 584–596.
McNamara, P., Krueger, M., O'Quin, K., Clark, J., & Durso, R. (1996). Grammaticality
judgements and sentence comprehension in Parkinson's disease: A comparison with broca's
aphasia. International Journal of Neuroscience, 86, 151–166.
Melloni, M., Sedeno, L., Hesse, E., Garcia-Cordero, I., Mikulan, E., Plastino, A., . . . Ibanez,
A. (2015). Cortical dynamics and subcortical signatures of motor-language coupling in
Parkinson’s disease. Scientific Reports, 5, 11899.
Mestres-Misse, A., Turner, R., & Friederici, A. D. (2012). An anterior–posterior gradient of
cognitive control within the dorsomedial striatum. Neuroimage, 62,  41–47.
Middleton, F. A., & Strick, P. L. (2000). Basal ganglia and cerebellar loops: Motor and cognitive
circuits. Brain Research Reviews, 31, 236–250.
Mink, J. W. (1996). The basal ganglia: Focused selection and inhibition of competing motor
programs. Progress in Neurobiology, 50, 381–425.
Nadeau, S. E., & Crosson, B. (1997). Subcortical aphasia. Brain and Language, 58, 355–402.
Nambu, A. (2008). Seven problems on the basal ganglia. Current Opinion in Neurobiology,
18(6), 595–604.
Natsopoulos, D., Grouios, G., Bostantzopoulou, S., Mentenopoulos, G., Katsarou, Z., &
Logothetis, J. (1993). Algorithmic and heuristic strategies in comprehension of complement
clauses by patients with Parkinsons-disease. Neuropsychologia, 31, 951–964.
Natsopoulos, D., Katsarou, Z., Bostantzopoulou, S., Grouios, G., Mentenopoulos, G., &
Logothetis, J. (1991). Strategies in comprehension of relative clauses by Parkinsonian
patients. Cortex, 27, 255–268.
Packard, M. G., & Knowlton, B. J. (2002). Learning and memory functions of the basal ganglia.
Annual Reviews in Neuroscience, 25, 563–593.
Papagno, C., Mattavelli, G., Cattaneo, Z., Romito, L., & Albanese, A. (2013). Ambiguous idiom
processing in Parkinson’s disease patients. Cognitive Neuropsychology, 30, 495–506.
Parsons, T. D., Rogers, S. A., Braaten, A. J., Woods, S. P., & Troster, A. I. (2006). Cognitive
sequelae of subthalamic nucleus deep brain stimulation in Parkinson’s disease: A meta-
analysis. Lancet Neurology, 5, 578–588.
Pauli, W. M., O’Reilly, R. C., Yarkoni, T., & Wager, T. D. (2016). Regional specialization within
the human striatum for diverse psychological functions. Proceedings of the National
Academy of Sciences of the United States of America, 113(7), 1907–1912.
Penke, M., Janssen, U., Indefrey, P., & Seitz, R. (2005). No evidence for a rule/procedural deficit
in German patients with Parkinson’s disease. Brain and Language, 95, 139–140.
Peran, P., Nemmi, F., Meligne, D., Cardebat, D., Peppe, A., Rascol, O., et al. (2013). Effect of le-
vodopa on both verbal and motor representations of action in Parkinson’s disease: A fMRI
study. Brain and Language, 125, 324–329.
Phillips, L., Litcofsky, K. A., Pelster, M., Gelfand, M., Ullman, M. T., & Charles, P. D. (2012).
Subthalamic nucleus deep brain stimulation impacts language in early Parkinson’s disease.
PLoS One, 7, e42829.
Provost, J.-S., Hanganu, A., & Monchi, O. (2015). Neuroimaging studies of the striatum in cog-
nition Part I: Healthy individuals. Frontiers in Systems Neuroscience, 9, 140.
Raymer, A. M., Moberg, P. J., Crosson, B., Nadeau, S. E., & Gonzalez-Rothi, L. J. (1997). Lexical–
semantic deficits in two cases of thalamic lesion. Neuropsychologia, 35, 211–219.
Ripolles, P., Marco-Pallares, J., Hielscher, U., Mestres-Misse, A., Tempelmann, C., Heinze, H.,
et al. (2014). The role of reward in word learning and its implications for language acquisi-
tion. Current Biology, 24, 2606–2611.
Robles, S. G., Gatignol, P., Capelle, L., Mitchell, M., & Duffau, H. (2005). The role of domi-
nant striatum in language: A study using intraoperative electrical stimulations. Journal of
Neurology, Neurosurgery, & Psychiatry, 76, 940–946.
Rodriguez-Fornells, A., Cunillera, T., Mestres-Misse, A., & de Diego-Balaguer, R. (2009).
Neurophysiological mechanisms involved in language learning in adults. Philosophical
Transactions of the Royal Society B-Biological Sciences, 364, 3711–3735.
Rüb, U., Seidel, K., Heinsen, H., Vonsattel, J. P., den Dunnen, W. F., & Korf, H. W. (2016).
Huntington's disease (HD): The neuropathology of a multisystem neurodegenerative dis-
order of the human brain. Brain Pathology, 26(6), 726–740.
Rueschemeyer, S. A., Fiebach, C. J., Kempe, V., & Friederici, A. D. (2005). Processing lexical se-
mantic and syntactic information in first and second language: fMRI evidence from German
and Russian. Human Brain Mapping, 25, 266–286.
Sambin, S., Teichmann, M., de Diego Balaguer, R., Giavazzi, M., Sportiche, D., Schlenker, P.,
& Bachoud-Levi, A. (2012). The role of the striatum in sentence processing: Disentangling
syntax from working memory in Huntington’s disease. Neuropsychologia, 50, 2625–2635.
Sebastian, R., Schein, M. G., Davis, C., Gomez, Y., Newhart, M., Oishi, K., & Hillis, A. E. (2014).
Aphasia or Neglect after Thalamic Stroke: The various ways they may be Related to Cortical
Hypoperfusion. Frontiers in Neurology, 5, 231.
Seghier, M. L., Bagdasaryan, J., Jung, D. E., & Price, C. J. (2014). The Importance of Premotor
Cortex for Supporting Speech Production after Left Capsular-Putaminal Damage. The
Selemon, L. D., & Goldman-Rakic, P. S. (1985). Longitudinal topography and interdigitation of
corticostriatal projections in the rhesus monkey. Journal of Neuroscience, 5(3), 776–794.
Silveri, M. C., Ciccarelli, N., Baldonero, E., Piano, C., Zinno, M., Soleti, F., . . . Daniele, A. (2012).
Effects of stimulation of the subthalamic nucleus on naming and reading nouns and verbs in
Parkinson’s disease. Neuropsychologia, 50, 1980–1989.
Snijders, T. M., Petersson, K. M., & Hagoort, P. (2010). Effective connectivity of cortical and
subcortical regions during unification of sentence structure. NeuroImage, 52, 1633–1644.
Stowe, L. A., Paans, A. M., Wijers, A. A., & Zwarts, F. (2004). Activations of “motor” and other non-
language structures during sentence comprehension. Brain and Language, 89(2), 290–299.
Surmeier, D. J., Ding, J., Day, M., Wang, Z., & Shen, W. (2007). D1 and D2 dopamine-receptor
modulation of striatal glutamatergic signaling in striatal medium spiny neurons. Trends in
Teichmann, M., Darcy, I., Bachoud-Levi, A., & Dupoux, E. (2009). The role of the striatum
in phonological processing: Evidence from early stages of Huntington’s disease. Cortex, 45,
839–849.
Teichmann, M., Gaura, V., Demonet, J., Supiot, F., Delliaux, M., Verny, C., et al. (2008).
Language processing within the striatum: Evidence from a PET correlation study in
Huntington’s disease. Brain, 131, 1046–1056.
Teichmann, M., Dupoux, E., Kouider, S., & Bachoud-Levi, A. C. (2006). The role of the stri-
atum in processing language rules: Evidence from word perception in Huntington’s disease.
Journal of Cognitive Neuroscience, 18(9). 1555–1569.
Teichmann, M., Dupoux, E., Kouider, S., Brugieres, P., Boisse, M. F., Baudic, S., et al. (2005).
The role of the striatum in rule application: The model of Huntington’s disease at early stage.
Brain, 128(5), 1155–1167.
Teichmann, M., Rosso, C., Martini, J., Bloch, I., Brugueres, P., Duffau, H., et al. (2015). A
cortical-subcortical syntax pathway linking broca’s area and the striatum. Human Brain
Mapping, 36, 2270–2283.
Tettamanti, M., Moro, A., Messa, C., Moresco, R. M., Rizzo, G., Carpinelli, A., . . . Perani,
D. (2005). Basal ganglia and language: Phonology modulates dopaminergic release.
Neuroreport, 16(4), 397–401.
Tomasi, D., & Volkow, N. D. (2012). Resting functional connectivity of language
networks: Characterization and reproducibility. Molecular Psychiatry, 17, 841–854.
Tremblay, C., Macoir, J., Langlois, M., Cantin, L., Prud’homme, M., & Monetta, L. (2015). The
effects of subthalmic deep brain stimulation on metaphor comprehension and language
abilities in Parkinson’s disease. Brain and Language, 141, 103–109.
Ullman, M. T. (2001a). The declarative/procedural model of lexicon and grammar. Journal of

Psycholinguistic Research, 30,  37–69.
Ullman, M. T. (2001b). A neurocognitive perspective on language: The declarative/procedural
model. Nature Reviews Neuroscience, 2, 717–726.
Ullman, M. T. (2006). Is Broca’s area a part of a basal ganglia thalamocortical circuit? Cortex,
42, 480–485.
Ullman, M. T., Corkin, S., Coppola, M., Hickok, G., Growdon, J. H., Koroshetz, W. J., et al.
(1997). A neural dissociation within language: Evidence that the mental dictionary is part
of declarative memory, and that grammatical rules are processed by the procedural system.
Van Heuven, W. J. B., Schriefers, H., Dijkstra, T., & Hagoort, P. (2008). Language conflict in the
bilingual brain. Cerebral Cortex, 18(11), 2706–2716.
Vannest, J., Polk, T. A., & Lewis, R. L. (2005). Dual-route processing of complex words: New
fMRI evidence from derivational suffixation. Cognitive Affective & Behavioral Neuroscience,
5(1),  67–76.
Vijayraghavan, S., Wang, M., Birnbaum, S. G., Williams, G. V., & Arnsten, A. F. (2007).
Inverted-U dopamine D1 receptor actions on prefrontal neurons engaged in working
memory. Nature Neuroscience, 10(3), 376–384.
Wahl, M., Marzinzik, F., Friederici, A. D., Hahne, A., Kupsch, A., Schneider, G., et al. (2008).
The human thalamus processes syntactic and semantic language violations. Neuron, 59,
695–707.
Wallesch, C. W., & Papagno, C. (1988). Subcortical aphasia. In F. C. Rose, R. Whurr, & M. A.
Wyke (Eds.). Aphasia. London: Whurr Publishers.
Ye, Z., Mestres-Misse, A., Rodriguez-Fornells, A., & Munte, T. F. (2011). Two distinct neural
networks support the mapping of meaning to a novel word. Human Brain Mapping, 32(7),
1081–90.
Zanini, S., Melatini, A., Capus, L., Gioulis, M., Vassallo, A., & Bava, A. (2002). Language re-
covery following subthalamic nucleus stimulation in Parkinson’s disease. Neuroreport, 14,
511–516.
Chapter 34
L ateraliz at i on
of L ang uag e
Lise Van der Haegen and Qing Cai
Introduction
Thousands of researchers are accumulating knowledge concerning how language is

processed in the human brain. It is such a fascinating topic because it is considered to
be one of the traits that makes humans unique. Moreover, the specialization to one ce-
rebral hemisphere is extraordinary: about 95% of right-handers and 75% of left-handers
dominantly activate their left hemisphere (LH) during speech production, whereas the
remainder of humans shows an atypical right hemispheric (RH) dominance or bilateral
organization (Knecht et al., 2000). However, it is not the ability to communicate that
differentiates humans from other animals, nor can language dominance be selectively
attributed to humans. Vocal expressions and other communicative gestures are equally
left lateralized in birds, mice, and nonhuman primates (Rogers & Andrews, 2002). It is
the intriguing joint dominance of right-handedness and LH speech lateralization that
has led to the abundance of language lateralization research in the human brain.
Let us introduce this chapter with the dominating question: Are handedness and lan-
guage lateralization linked? This is an ongoing debate, but recent evidence points to an
indirect relationship between these phenotypes. A common underlying origin was first
suggested by the higher incidence of aphasia in right-handers than left-handers after
unilateral LH brain damage (Benson & Geschwind, 1985; Hécaen & Sauguet, 1971).
Evidence for an influence of handedness on language laterality was further provided
by long-believed single-gene theories. For example, the right-shift gene theory posits
that an RS + allele leads to right-handedness and LH language dominance, whereas
an RS –allele randomly assigns handedness and language dominance. The combi-
nation of the two alleles most likely results in typical LH language dominant right-
handers (Annett, 1998). Single-gene theories, however, have been refuted because
evidence from most recent studies clearly shows that both handedness and language
878 Lise Van der Haegen and Qing Cai
dominance are determined by multiple genes, with so far no candidate genes common
to both functions (e.g., LRRTM1 [Francks et al., 2007] and PCSK6 [Brandler et al., 2013]
for handedness, and FOXP2 for language dominance [Ocklenburg et al., 2013; Pinel
et al., 2012]). Still, the prevalence of LH language lateralization in right-handers
compared to left-handers appears to be too high to be incidental. Differences in re-
search approaches may explain why this puzzling coherence has not been unraveled yet.
For example, evolutionary research suggests that the nature of manual actions must be
taken into account. Language dominance may be linked to the lateralization of com-
municative gestures, but not to noncommunicative manual actions, such as grasping
an object (Cochet, 2015). In language laterality research, however, handedness scores
are often derived from the Edinburgh Handedness Inventory, asking manual preference
for daily noncommunicative actions (Oldfield, 1971). Another possibility is that alter-
native measures of handedness, such as familial sinistrality, are more closely related to
language lateralization. For example, having a left-handed first-degree relative reduced
the surface area of the planum temporale in the LH up to 10%, independently of the
individual’s own handedness, in Tzourio-Mazoyer, Simon, et al. (2010). Finally, the de-
gree, apart from direction, of handedness and language lateralization may also play a
role in the link between both phenotypes (Mazoyer et al., 2014).
Despite the unclear role of handedness in language lateralization, left-handers are
considered an interesting sample when studying cerebral asymmetries. It is wrong to
take them as a homogeneous atypical group based on their handedness because the ma-
jority of them still show typical LH language dominance (see the following section), but
some are RH language dominant. Their heterogeneity provides a unique perspective on
how cognitive functions are related in healthy participants. If, for example, speech pro-
duction shifts to the RH, related functions are expected to lateralize to the same atyp-
ical side in order to optimize information exchange, whereas unrelated functions reside
in the contralateral hemisphere (Willems, Van der Haegen, Fisher, & Francks, 2014).
Language laterality can be an ideal starting point to reveal interactions between cog-
nitive functions such as cognitive control, memory, and so on, because it is the most
well-documented lateralized function until now (Cai & Van der Haegen, 2015). Studies
including left-handers with or without known atypical lateralization will therefore be
extensively reviewed in this chapter, even though one should bear in mind that handed-
ness correlates with language lateralization without revealing direct causes.
Lateralization research is, of course, far from limited to a comparison between right-
and left-handers. A long history of research made clear that language lateralization is
influenced by anatomical, evolutionary, genetic, developmental, and experiential
factors, and impairment; some of these factors were already introduced earlier. We will
briefly describe evolutionary and anatomical influences on language lateralization in
general in this introduction. More detailed studies investigating the influence of these
factors on language sub-processes will be discussed in further sections. Evolutionary,
hemispheric specialization might have arisen when the cortex expanded and unilateral
specialized regions became more advantageous than redundant processing of the same
function in homologue areas in the LH and RH. In an increased brain volume, intra-
hemispheric processing facilitates information exchange between connected brain
Lateralization of Language 879
regions. Moreover, lateralization increases brain capacity by creating cortical space

available for other cognitive functions. In particular, the expansion of the frontal cortex
in primates, housing Broca’s area (involved in speech production), may have contributed
to language asymmetry (Toga & Thompson, 2003; see also Corballis, 2009; Hopkins &
Cantalupo, 2008). Anatomically, three remarkable asymmetries have consistently been
observed. First, the so-called right frontal and left occipital petalias refer to the anterior
extension of the RH beyond the LH and the occipital protrusion of the LH beyond the
RH. In addition, RH frontal and LH occipital areas are wider. Second, the Sylvian fissure
runs more anteriorly and steeper in the RH than LH. Third and presumably most im-
portant for language lateralization, the planum temporale is about 35% larger on the left
side (Toga & Thompson, 2003; see also Hugdahl, 2011).
An important note is that so far, we have collapsed lateralization findings across lan-
guage functions. A large part of the literature has indeed long treated language as one
unitary function, mostly taking the asymmetry of speech production as an equivalent of
language lateralization. It became clear, however, that language processing is much more
complex: The focus of neurocognitive research has shifted from identifying individual
neural nodes to structural and functional networks, and not all language sub-processes
are lateralized to the same degree or even in the same direction. We will therefore review
lateralization in more detail by discussing the sub-processes. We will first give an over-
view of the most important recent studies on the lateralization of speech production,
auditory speech processing, and reading, as these are considered to be the three core
sub-processes of language. We limit this chapter to higher-order processes, as the low-
level stages of language in the primary auditory and visual cortex are not lateralized—
with the exception of articulation, which is controlled by unilateral motor cortices. We
also refer to other chapters for a more detailed overview of non-lateralized brain re-
gions involved in these functions. For all three core sub-processes, we first describe the
most important findings with respect to the direction and degree of lateralization, and
thereafter summarize speculations on what might drive this lateralization based on the
seven possibly influencing factors mentioned earlier. We then discuss the role of the RH,
which has a pivotal role in the processing of prosody and metaphors, but also gains im-
portance when language is impaired (e.g., in the case of dyslexia) or extended (e.g., in
the case of bilingualism). Finally, the relationship between language lateralization and
other asymmetric functions has become a hot topic in research, in line with the shift
from individual neural nodes to highly interactive brain networks within and across
domains. It will become clear that language lateralization can be a unique gateway to
gain knowledge about how the human brain is organized.
Speech Production
In the nineteenth century, Marc Dax attributed speech production to the LH for the
first time. Paul Broca published his seminal paper in 1865 stating that we speak with
our left hemisphere, after observing a lesion to the third gyrus of the inferior frontal
cortex in two severely speech-impaired patients (Broca, 1865; see Wilson, Chapter 2 in
this volume). The lesioned region corresponded to what we still call Broca’s area today,
including the pars opercularis (approximately Brodmann area 44) and pars triangularis
(approximately Brodmann area 45). Despite widely accepted insights that language is
much more complex than speaking with Broca’s area, activity in this region has long
been equalized to language, in particular in the domain of laterality. Clinicians, for ex-
ample, localize language most often in the LH because many patients are prevented
from speaking when the LH is temporarily anesthetized by sodium amytal in the
Wada test (Wada & Rasmussen, 1960). The preponderance of the LH in language re-
ceived even stronger confirmation from research with split-brain patients, also in the
1960s (Gazzaniga, Bogen, & Sperry, 1962). These patients have their corpus callosum
and anterior commissure severed in order to isolate intractable epileptic seizures in one
hemisphere. They could not name an object held in their left hand, connected to the
contralateral RH. This verified that the LH houses speech production, as it could not
receive the necessary information via interhemispheric transfer (see Gazzaniga, 1975,
2005, for reviews). Remarkably, these findings were generalized as evidence for overall
LH language dominance for decades. This is reflected in laterality assessments of more
recent research: Many behavioral and neuroimaging paradigms are compared against
outcomes of the Wada test (e.g., Binder et al., 1996; Hirata et al., 2010), or start from pro-
duction when studying language lateralization, albeit with the notion that their results
reflect speech production lateralization and not language lateralization as a whole (e.g.,
Abbott, Waites, Lillywhite, & Jackson, 2010).
Behaviorally, visual half-field tasks reveal an LH speech dominance when pictures
presented in the right visual field are named faster than pictures in the left visual field
because the partial crossing of optic fibers at the optic chiasm sends visual informa-
tion to the contralateral hemisphere. Van der Haegen, Cai, Seurinck, and Brysbaert
(2011) presented this behavioral approach as a screening method to identify (a)typi-
cally speech-lateralized participants. Their visual half-field picture-naming latencies
correlated significantly (r = 0.65) with a functional magnetic resonance imaging (fMRI;
see Heim & Specht, Chapter 4 in this volume) word-generation task in 50 left-handers
with variable directions and degrees of lateralization. Neuroimaging techniques—
and in particular fMRI because of its good spatial resolution—are today widely used
to map brain regions involved in speech production and their lateralization. The verb-
generation task (i.e., saying a verb associated with an object), word-generation task (i.e.,
generating a word starting with a target letter), and picture naming are most popular for
speech-production lateralization. The results are often expressed in indices reflecting
the direction and degree of lateralization, in line with the view that lateralization is
graded and not absolute (Behrmann & Plaut, 2015), a notion that is not taken into ac-
count by the Wada test. The widely used toolbox by Wilke and Lidzba (2007), for ex-
ample, compares the difference in neural activity between LH and RH homologue areas,
thereby giving more weight to voxels activated at a higher activity level to avoid basing
the indices on arbitrarily chosen statistical thresholds (see Seghier, 2008, for methodo-
logical issues in calculating lateralization indices). Early large-scale studies using word
generation in fMRI (Pujol, Deus, Losilla, & Capdevila, 1999) or functional transcranial
cerebral Doppler sonography (comparing blood flow velocity in the left and right
middle cerebral arteries; Knecht et al., 2000) estimated LH speech dominance to be pre-
sent in about 95% of right-handers but only 75% of left-handers, boosting the belief that
handedness affects language lateralization. Mazoyer et al. (2014) scanned 297 healthy
participants balanced for handedness during a sentence production task (i.e., silent sen-
tence generation versus repetition of months of the year as overlearned sequence rep-
etition). Lateralization indices based on individual contrast-activity maps (with most
activity in the LH inferior frontal gyrus and lower part of the precentral gyrus, and RH
activity at the junction of the middle and inferior temporal gyri in the occipital lobe) di-
vided the participants into typical LH dominants (88% right-handers, 78% left-handers),
atypical RH dominants (the remaining 7% left-handers), and participants with bilateral
patterns (12% right-handers, 15% left-handers). Overall, these studies agree that the ma-
jority of participants produce speech dominantly with their LH, but left-handers have a
higher incidence of atypical bilateral or RH dominance.
These laterality studies focused on Broca’s area during speech production. Most
lateralization indices were based on combined activity in the pars opercularis and
triangularis. It should be noted that Broca is a large region that can be divided into
several sub-regions, for example six regions based on neurotransmitter receptor type
(Amunts et al., 2010). It is not surprising, then, that it also turns out to be a heteroge-
neous area at the functional level, linked to semantic processing, syntactic processing,
motor functions, music perception and execution, and so on (e.g., Fadiga, Craighero, &
D’Ausilio, 2009) apart from phonological processing. It is thus important for researchers
to specify on which regions their lateralization estimates were based. Speech produc-
tion also activates a mosaic of cerebral regions outside Broca’s area. Again, even though
researchers are aware of this, individual indices should be reported separately for other
speech production-related areas if we want to reach a complete image of lateralization.
Indeed, the lack of reporting lateralization indices of sub-processes reflects the general
duality between psycholinguistics focusing on higher-order level processes (e.g., the
pars opercularis and triangularis for word retrieval) and motor control neuroscientists
investigating lower-level articulatory processes (e.g., pre-and post-central motor re-
gions associated with mouth movements; Indefrey, 2011; Price, 2012), as highlighted
by Poeppel, Emmorey, Hickok, and Pylkkänen (2012). Broca’s area may indeed re-
main the most important region for speech production, and if all regions co-lateralize,
a limitation to the classical pars opercularis and triangularis may not be problematic,
but differences in the degree of lateralization may help explain relationships with other
brain regions and possible causes of lateralization, which we will now discuss.
Anatomically, Broca’s area has been found to be larger in the LH than RH, but this
may mainly apply to Brodmann area 44 (pars opercularis). Amunts et al. (1999) di-
vided Broca’s area into 10 different regions based on cytoarchitectonic laminar patterns.
Despite large between-participant differences, only area 44 showed a clear left asym-
metry in all brains. The same authors later reported a significant lateralization of cho-
linergic M2 receptors in this area (Amunts et al., 2010). The relationship between
functional and anatomical asymmetries remains puzzling, however. Keller et al. (2011)
compared 15 healthy participants with a functional LH dominance during word gen-
eration with 10 RH dominants. Volume asymmetry of the insula could predict 87%
LH and 90% RH functional dominance, and the termination of the right posterior
Sylvian fissure lay more vertical in LH dominants, but planum temporale volume
asymmetry had no predictive value. Even more surprisingly, gray matter asymmetry
of Broca’s area, considered as the core region activated during word generation, did
not correlate with functional asymmetry. Likewise, Greve et al. (2013) found volume
asymmetry differences in the insula in 34 LH speech-dominant and 21 RH speech-
dominant left-handers identified by Van der Haegen et al. (2011), but not in the pars
opercularis/triangularis (again, participants were strikingly divided into an LH-and an
RH-dominant group based on their functional asymmetry in these regions), nor in the
planum temporale or Heschl’s gyrus. Exploratory surface-based analysis did find small
differences in the posterior temporal gyrus (overlapping with planum temporale) and
the ventral occipito-temporal region involved in reading.
Evolution is a second factor that has been related to speech lateralization. It has been
theorized that language evolved from manual gestures. The main argument is found
in the similar location of Broca’s area in humans and the F5-area in monkeys, housing
the mirror-neuron system. Mirror-neurons respond to both the execution of manual
actions and the perception of the same movements, such as reaching and grasping.
This created the possibility to imitate, and from there, more complex movements such
as speech articulation could evolve, leading to communication via speech (Rizzolatti
& Arbib, 1998). Bipedalism and more complex hand movements in tool use further
contributed to the evolution of complex human communication with, for example,
syntax (Corballis, 2003; see Vingerhoets et al., 2013, for a correlated lateralization pat-
tern between tool use pantomiming and speech production). It is the vocal mechanism
in Broca’s area that would have led to unilaterality, as articulation does not require bilat-
eral control, even though this view has not fully been accepted yet because asymmetric
speech perception may have evolved first; the vocal tract used for articulation developed
in Homo sapiens 170,000 years ago, whereas left lateralization of Broca’s area was already
reported for Homo habilis nearly two million years ago (Corballis, 2003). In this light, it
would be useful to dissociate sub-regions of Broca involved in manual gestures related
to communicative from non-communicative actions.
With respect to development, studies examining the lateralization of speech in young
infants are sparse but seem to agree that strong LH frontal lateralization during expres-
sive language (e.g., verb generation, verbal fluency) is already present in children be-
tween the ages of 3 and 18 years (e.g., Holland et al., 2007; Paquette et al., 2015; Sowman,
Crain, Harrison & Johnson, 2014; see Minagawa & Cristia, Chapter 7 in this volume).
Paquette et al. (2015) tested a sample with an age range between 3 and 30 years old with
a near-infrared spectroscopy verbal fluency task in which participants had to sum up
as many semantic category items as possible. Neural activity bilaterally increased with
age in fronto-temporal language regions, but the degree of LH lateralization remained
stable from infancy until young adulthood.
Investigating changes in laterality due to increased experience with speech is diffi-

cult, given the early use of expressive language in life. Inferior frontal activity has been
observed during sign language by deaf participants, with minimal differences compared
to normal hearing participants (e.g., Emmorey, Mehta, & Grabowski, 2007, using a posi-
tron emission tomography (PET) picture-naming task with face orientation as a control
condition), pointing to an amodal expressive language by Broca’s area and in line with
previously described commonly developed neural networks for speech and gestures.
Sign language does evoke more LH parietal activity, such as in the supramarginal gyrus
linked to hand configuration and superior parietal lobule associated with proprioceptive
monitoring of gestures (Emmorey et al., 2007). Allen, Emmorey, Bruss, and Damasio
(2013) observed a larger volume of the pars triangularis in deaf signers compared to
normal hearing signers and non-signers, along with increased gray matter volume in
the visual calcarine sulcus, but no differences in lateralization. These results suggest that
deaf signers have increased language areas bilaterally because of their highly demanding
language system.
Finally, speech laterality explained by impairment brings us back to where we started
this section, namely the observation of Paul Broca that the so-called motor aphasia
in his right-handed patients was caused by an LH lesion in the inferior frontal gyrus
(Broca, 1865). Later reports estimated aphasia due to unilateral LH lesions to occur in
60% of right-handers and 32% of left-handers, whereas right-hemispheric lesions led
to aphasia in only 2% of right-handers but 24% of left-handers (Benson & Geschwind,
1985). Interestingly, the temporal cortices that are considered to contain the second most
important areas for language were discovered in a similar way, namely by the observa-
tion of so-called sensory aphasia after an LH lesion in Wernicke’s area, now known to be
important for semantic auditory language processing.
Auditory Speech Processing
About one decade after Paul Broca published his seminal paper describing patients
with speech problems as having motor aphasia, the German neurologist Carl Wernicke
introduced the so-called sensory aphasia to refer to the syndrome of losing speech
comprehension due to a lesion in the LH posterior temporal gyrus (Wernicke, 1874).
In laterality research, auditory speech comprehension also seems to be considered as
the second most important language sub-process, with speech being more often taken
as an equivalent to general language lateralization (see Poeppel, Cogan, Davidesco, &
Flinker, Chapter 26 in this volume). From an ontogenetic point of view, this is counter-
intuitive, as speech comprehension develops before speech, already in utero (Partanen
et al., 2013). Some researchers do use auditory speech perception as a language laterality
measurement, such as in the widely used behavioral dichotic listening task (Kimura,
1961). In this paradigm, participants are presented with auditory stimuli in both ears,
often consonant-vowel syllables. The participant is asked to indicate which of two
different input stimuli (s)he heard best. Reporting most signals from the right ear is seen
as a marker of LH auditory speech dominance because of the preponderance of audi-
tory pathways running from the ear to the contralateral auditory cortex—but note that
in the auditory modality, ipsilateral connections are also highly present (Kimura, 1961).
Hugdahl et al. (1997) found a right-ear advantage in 92% of right-handers who were LH
(speech) dominant in the Wada-test. Tzourio-Mazoyer, Petit, et al. (2010) assessed lan-
guage dominance, operationalized as story listening in the participants’ mother tongue,
versus an unknown foreign language in an fMRI study with 94 right-handers. LH asym-
metry was found in the posterior superior temporal sulcus, the inferior frontal gyrus,
and precentral gyrus, with a reduced lateralization degree in the presence of familial
sinistrality, a smaller head size, and absence of strong manual preference.
Auditory speech recognition thus seems to lateralize to the LH. However, studies
manipulating different components of speech comprehension point at two pathways.
Bozic, Tyler, Ives, Randall, and Marslen-Wilson (2010) found bilateral fronto-temporal
activity for the processing of general perceptual complexity and sound-to-meaning
conversion during speech comprehension, whereas an LH inferior frontal asymmetry
was only found when linguistic morpho-syntactic complexity of the auditory stimuli
was increased. Hickok and Poeppel (2007; see also Hickok, Chapter 20 in this volume)
described the auditory dual network in more detail. According to their theory, a ven-
tral stream, including middle and posterior temporal cortices, connects acoustic sig-
nals to lexical meaning and is hence targeted by speech recognition tasks. In contrast, a
dorsal stream running through the superior temporal sulcus bilaterally, a left part of the
Sylvian fissure at the border of the parietal and temporal lobe, and left posterior frontal
regions translate acoustic signals into articulatory codes and are activated during sub-
lexical tasks such as syllable discrimination. Importantly, the ventral stream is bilaterally
represented, which explains why speech comprehension is not necessarily devastated
after unilateral damage, but it is the dorsal stream accounting for speech perception that
is LH lateralized (Hickok & Poeppel, 2007). In addition to a ventral and dorsal stream,
auditory speech processing has been thought to show time-related interhemispheric
differences. The LH would be specialized in fast information processing, such as in the
case of identifying rapidly changing sounds in speech, and the RH for slower spectral
information that is required for prosodic analysis of a speech stream (Zatorre, Belin,
& Penhune, 2002). Hickok and Poeppel (2007), in contrast, propose that the ventral
stream integrates slow speech information in the RH, but fast information bilater-
ally (but see Scott & McGettigan, 2013). A review by Specht (2013), however, suggests
that a more pronounced LH asymmetry is observed for lexico-semantic compared to
auditory-phonetic processing and from posterior to anterior regions involved in speech
processing showing that it is still under debate which aspects of speech processing are
lateralized.
Comparing the degree of lateralization for speech production and comprehension
might provide further insights, given their close functional relationship, especially in
the dorsal stream. Tzourio-Mazoyer, Josse, Crivello, and Mazoyer (2004) found stronger
functional asymmetries for production (verb generation) than comprehension (story
listening, both against rest) in a PET study. The weaker asymmetry during speech com-
prehension was mainly driven by the superior temporal gyri and Heschl’s gyri. On the
other hand, Häberling, Steinemann, and Corballis (2016) recently found no significant
difference between three lateralization indices based on activity in the inferior frontal
gyrus during verbal fluency word generation (against fixation) and the inferior frontal
gyrus and middle/superior temporal gyrus during a synonym judgment comprehen-
sion task (against similarity of letter strings). It may be that the importance of lexico-
semantic information in the visual synonym task boosted left lateralization, which again
highlights that a variety of paradigms and especially control conditions can lead to dif-
ferent outcomes. Häberling et al. conclude that it makes sense for the fronto-temporal
regions to tune their lateralization in order to optimize information exchange, and that
the left lateralization of the dorsal stream introduced by Hickok and Poeppel (2007)
uniquely adapted to vocalization in humans.
In sum, auditory speech processing is at least partly LH lateralized, but future re-
search needs to clarify exactly which aspects are specialized to one hemisphere and
whether this specialization is less extremely pronounced than speech production. We
will now discuss studies providing information on the possible factors affecting later-
ality. First, Häberling et al. (2016) included an overrepresentation of left-handers to en-
sure sufficient variability in the lateralization indices and indeed found less asymmetry
in this group. Van der Haegen, Westerhausen, Hugdahl, and Brysbaert (2013) pursued
the left-handed approach more extremely and compared their atypically RH speech
production lateralized left-handers (based on an fMRI verbal fluency word-generation
task) with LH dominant left-and right-handers in a behavioral dichotic listening task.
At the group level, RH speech dominants indeed turned out to have a left-ear advan-
tage, whereas LH speech dominants had the expected right-ear advantage. At the indi-
vidual level, however, the dichotic listening task showed variability in all groups. This
can be due to more variability in a behavioral task, or an indication that speech percep-
tion in the case of syllable discrimination is more symmetric than speech production
(Hickok & Poeppel, 2007). No effects of handedness were found when left-handers were
considered as a homogenous group, which suggests the importance of their individual
differences in hemispheric speech lateralization.
Evolutionary speculations on the development of auditory perception and compre-
hension go along with anatomical asymmetries. The planum temporale around the su-
perior temporal gyrus, involved in imagining and hearing sounds (Price, 2012), is larger
in the LH in humans (Geschwind & Levitsky, 1968) as well as chimpanzees (Hopkins,
Marino, Rilling, & MacGregor, 1998), but not in rhesus monkeys and baboons (Wada,
Clarke, & Hamm, 1975). These results point to the importance of leftward asymmetries
in the planum temporale, which might be linked with some communicative functions
(Corballis, 2009; Hopkins & Nir, 2010). Familial sinistrality affected the planum
temporale in Tzourio-Mazoyer, Simon, et al. (2010): Having a left-handed relative
decreased the surface size of the area by 10% and led to a larger gray matter volume,
accompanied by a smaller leftward asymmetry. The size and gyrification pattern of
Heschl’s gyrus (belonging to the bilateral superior temporal gyri and involved in speech
and non-speech sound processing; Price, 2012) also seems to play a role in establishing
language lateralization. Marie et al. (2015) observed less duplications of Heschl’s gyrus in
the RH and a decreased LH asymmetry of the anterior gyrus of Heschl in left-handers.
Tzourio-Mazoyer et al. (2014) linked the duplication and decrease in anatomical sur-
face area of the anterior gyrus to a decrease in functional asymmetry in Heschl’s gyrus
during word-list listening. Leroy et al. (2015) reported a superior temporal asymmetrical
pit that may be unique to humans, that is, more LH sulcal interruptions in a superior
temporal sulcus region ventral to Heschl’s gyrus, leading to a deeper structure in 95%
of humans. This may be a precursor of human language lateralization as the asymmetry
is present in healthy adults as well as infants, (a)typically lateralized speech dominants,
left-and right-handers, and participants with situs inversus, autism spectrum disorder,
Turner syndrome, or corpus callosum agenesis, but not in chimpanzees. A final ana-
tomical asymmetry is found in the long-distance tracts of the arcuate fasciculus, con-
necting frontal and temporoparietal language areas such as Broca’s area and Wernicke’s,
respectively. Its direct pathway (apart from the indirect pathway via the inferior pari-
etal cortex) is LH lateralized in more than 80% of healthy participants (Catani et al.,
2007). The structural asymmetry, however, seems to be independent from handed-
ness and functional language lateralization as measured with a verbal fluency verb-
generation task against tone listening (Vernooij et al., 2007). The recent diffusion tensor
imaging study by Allendorfer et al. (2016) similarly reported no asymmetry differences
for the pathway between left-and right-handers, even though left-handers as a group
were less extremely LH lateralized in the functional verb-generation task. In contrast,
Ocklenburg, Schlaffke, Hugdahl, and Westerhausen (2014) did find positive correlations
between tract volume and fractional anisotropy of the LH arcuate fasciculus with the
degree of right-ear advantage in a dichotic listening task.
Apart from evolutionary and anatomical insights, genetic studies are slowly adding
information. With respect to speech perception, Ocklenburg et al. (2013) linked two
polymorphisms of the FOXP2 gene, rs2396753 and rs12533005, to the right-ear advan-
tage in a dichotic listening task presenting consonant-vowel syllables to about 450
healthy participants. Variations in the FOXP2 gene, however, also have been associated
with language functions other than speech perception (e.g., reading ability; see discus-
sion later in the chapter), and dichotic listening activates many brain regions such as the
superior and middle temporal gyrus, pre-and post-central gyrus, supplementary motor
area, and middle and superior frontal gyrus (Van den Noort, Specht, Rimol, Ersland, &
Hughdahl, 2008). As mentioned by the authors themselves, it remains unclear how ex-
actly molecular changes affect speech processing laterality.
Further, changes in auditory speech lateralization have been observed during devel-
opment. Perani et al. (2011) found that perisylvian language areas in the inferior frontal
and superior temporal cortices are already activated by 2-day-old babies listening to
speech, but the auditory cortex showed RH dominance. LH speech perception was pre-
sent in 3-month-old infants (Dehaene-Lambertz, Dehaene, & Hertz-Pannier, 2002),
accompanied by a structural LH asymmetry of the arcuate fasciculus (Dubois et al.,
2009). Another important difference compared to the adult language network pointed
out by Perani et al. (2011) was the functional connectivity: Whereas most adults’ fronto-
temporal connections are strongest within the LH, 2-day-old babies mainly activated
interhemispheric connections between LH and RH temporal regions. This preponder-
ance of interhemispheric connectivity is still present by the age of 6 years (Friederici,
Brauer, & Lohmann, 2011).
In deaf people (see also, in this volume, Newman, Chapter 14, and Corina & Lawyer,
Chapter 16), the superior temporal gyrus reacts on visual motion stimuli and is more
strongly connected to the calcarine fissure. This functional connectivity is positively
correlated with the duration of wearing a hearing aid in hearing-impaired participants
(Shiell, Shampoux, & Zatorre, 2014). With respect to the lateralization of a reorganized
auditory cortex after unilateral deafness, Van der Haegen et al. (2016) found no differ-
ence in LH dominance for right-sided deaf participants who had profound hearing loss
from birth during a semantic speech-listening task (i.e., judging whether or not a heard
sentence refers to an animal against noise).
Reading
Reading (see Paz-Alonso, Oliver, Quiñones, & Carreiras, Chapter 24 in this volume),
the third main subprocess of language, developed much later than speech production
and comprehension in evolution. It originated about 6,000 years ago, even though being
able to read was still rare only a century ago. Reading also develops last at the individual
level, after extensive learning via instruction.
A region in the LH ventral occipito-temporal (vOT) sulcus has been identified as
a crucial area and was consequently called the “visual word form area” (Cohen et al.,
2000). Dehaene and colleagues argue that the area became specialized in a way of neu-
ronal “recycling” (i.e., neurons were tuned to recognize visual words, but this evolved
in an existing framework that was not genetically manipulated by learning to read).
Plasticity could only occur within the existing constraints of the brain, and reading
evokes most neural activity in the ventral visual cortex because of the close relation-
ship to recognition of line junctions that form objects (see, e.g., Dehaene & Cohen, 2011,
for a review). This area has, however, been shown to be activated by nonvisual stimuli
as well, and interacts with top-down information coming from frontal phonological
areas (Price & Devlin, 2011). The latter is especially interesting in light of lateralization.
Seghier and Price (2011) found that the LH dominance of the vOT relative to pictures
varied across three sub-regions of the area, mainly driven by decreased activity in the
RH. Laterality indices of the posterior vOT were mostly influenced by RH reduction
for letters and words relative to objects and nonobjects, indicating that this sub-region
is related to visual attributes. The anterior vOT, on the other hand, was more activated
by familiar (words and objects) than unfamiliar (Greek letters and nonobjects) stimuli
and by a semantic task rather than reading aloud, suggesting that the lateralization of
the anterior vOT is influenced by connections with frontal phonological and semantic
regions. The middle vOT was affected by a mixture of visual and nonvisual factors.
Inferior frontal regions already play a role in very early stages of visual word recogni-
tion. Cornelissen et al. (2009) did not find a significant time difference between the peak
of the inferior frontal gyrus (after 130 ms) and the visual mid-fusiform gyrus (after 140
ms) in a passive word-viewing task. Words further elicited activity in the anterior and
left posterior middle temporal gyrus, left superior temporal gyrus, and angular and
supramarginal gyri. Co-lateralization of the inferior frontal gyrus during word gener-
ation and vOT during lexical decision was also found in left-handers with typical LH
and atypical RH dominance (Cai, Lavidor, Brysbaert, Paulignan, & Nazir, 2008; Cai,
Paulignan, Brysbaert, Ibarrola, & Nazir 2010; Van der Haegen, Cai, & Brysbaert, 2012).
Pinel et al. (2014) compared reading (words versus scrambled stimuli) and auditory
speech (native language sentences versus tone listening) lateralization in a one-back
task. Participants with the strong leftward asymmetry in the middle superior tem-
poral sulcus during speech processing also had a stronger LH lateralized vOT, driven
by reduced RH activity, as in Seghier and Price (2011). These findings together reveal
an influence from earlier developed functions such as speech production and compre-
hension on the recently developed reading skill. The vOT remains the most investigated
region involved in visual word recognition, but it is clear that reading, as all cognitive
functions, functionally relies upon a broader network, including frontal and temporal
regions to support phonological and semantic processing. Richardson, Seghier, Leff,
Thomas, and Price (2011) presented three possible reading routes connecting the pos-
terior inferior occipital, posterior superior temporal gyrus, anterior superior temporal
sulcus, and vOT region, with one pathway even running through the first three regions
without involvement of the vOT. To our knowledge, no laterality differences have been
reported for these routes.
Returning to our seven possible factors influencing laterality, we will now discuss ev-
olutionary theories further than the previously mentioned neuronal recycling view, as
reading is too recent and too unique for humans to speculate on parallel development of
brain regions with nonhuman primates. Anatomically, two white-matter tracts end near
the vOT: the inferior fronto-occipito fasciculus and the inferior longitudinal fasciculus
running from the occipital lobe to anterior and medial temporal lobe. A third fasciculus,
the vertical occipital fasciculus, runs from the lateral occipito-temporal sulcus to lateral
occipital and inferior parietal lobes (Yeatman, Rauschecker, & Wandell, 2013; see Catani
& Forkel, Chapter 9 in this volume). Again, different asymmetries of these pathways
related to atypical (functional) lateralization have not yet been reported, to our know-
ledge. The only significant difference found was the more leftward/rightward gray
matter volume asymmetry for speech LH and RH dominants, respectively, in Greve
et al. (2013).
Pinel et al. (2012) scanned 94 healthy participants while they silently read sentences
and linked the two single-nucleotide polymorphisms rs6980093 and rs7799109 in the
FOXP2 gene (associated with language impairment) to their functional LH lateralization
of the frontal cortex, and rs17243157 variants in the KIAA0319/TTRAP/THEM2 gene (as-
sociated with reading disabilities) to reduced LH asymmetry in the superior temporal
sulcus. Other genetic influences on the lateralization of the vOT were found in Pinel
et al. (2014), who tested monozygotic and dizygotic twins during their one-back visual
task. Monozygotic twins showed intra-pair correlations as high as intra-participant
correlations measuring replicability for LH vOT activity. The number of voxels activated
above a t > 1 threshold was also more similar between monozygotic than dizygotic twins,
suggesting that LH vOT activity is at least partly driven by genetic influences. This may
be contradictory to the evolutionarily recent acquirement of the reading skill, but Pinel
and colleagues conclude that this can be explained by the neural constraints in which
the reading circuit developed. Genetic underpinnings of properties of the visual cortex
to be able to recognize detailed information in central vision and connections to frontal
and temporal regions may drive the heritability of the LH vOT activity.
To explore how the vOT evolves during learning to read in education, Ben-Shachar,
Dougherty, Deutsch, and Wandell (2011) scanned children aged 7–12 four times in one
year during an implicit reading task: Children were asked to indicate the color of a fix-
ation mark that was presented together with word stimuli varying on visibility at four
levels. Sensitivity to the word visibility was linked with better performance of overt
speed reading and increased with age, especially in the LH posterior occipito-temporal
sulcus nearby the vOT. The occipital V1 and posterior parietal cortex did not show such
correlation, and remarkably neither did the RH occipito-temporal reading region. Thus,
for developing perceptual expertise of words in a noisy context, the LH seems to play a
more important role than the RH homologue area.
Dehaene, Cohen, Morais, and Kolinsky (2015) review how gaining reading experi-
ence by illiterates changes neural circuits. Not only letters, but also objects and faces, are
better discriminated when acquiring reading, as reflected in an increase of activity in
the early bilateral occipital regions (Dehaene et al., 2010). Moreover, the ventral visual
pathway, including the vOT, becomes specialized for the script that is being trained,
with a positive correlation between the amount of neural activity and reading speed
(Dehaene et al., 2010). Learning an artificial language can even change the brain after a
short training: Xue, Chen, Jing and Dong (2006) conducted two weeks of visual training
and two weeks of phonological and semantic training with normal readers. Visual
training decreased activity bilaterally in the fusiform cortex and LH inferior occipital
cortex, whereas phonological training had the opposite effect. Semantic training af-
fected the RH fusiform area more. Reading acquisition further improves the processing
of phonological representations; the vOT is activated by phonology only in literates
(Dehaene et al., 2015). Perhaps the most striking change in terms of laterality is the influ-
ence that reading experience has on the lateralization of non-reading-related face recog-
nition. The fusiform face area specialized for faces in the ventral visual cortex is reduced
in the LH but enhanced in the RH when literacy increases (Dehaene et al., 2015). We will
discuss the remarkable relationship between the laterality of word and face recognition
in more detail in the following section.
Finally, reading impairment has been associated with differences in laterality. A meta-
analysis of functional connectivity in dyslexic and control participants pointed at weaker
LH connections between regions involved in visual and visuo-phonological processes,
such as the LH inferotemporal region, fusiform gyri, inferior frontal, premotor and
supramarginal cortices. In addition, dyslexics activated a dorsal fronto-parietal network
less, including LH parietal and premotor cortices, associated with altered motor and
visuospatial attention (Paulesu, Danelli, & Berlingeri, 2014). Diffusion tensor imaging
(see Catani & Forkel, Chapter 9 in this volume) studies generally found lower frac-
tional anisotropy in white-matter tracts running through temporoparietal and frontal
regions, especially in the LH arcuate fasciculus and corona radiata. Ventral tracts re-
lated to reading, such as the inferior longitudinal fasciculus and inferior fronto-occipital
fasciculus, were less affected (see Vandermosten, Boets, Wouters & Ghesquière, 2012,
for a review).
The Role of the Right Hemisphere

in Language Processing
In contrast to the widely studied core sub-processes discussed earlier, the role of the
RH receives much less attention in neuroscientific research. It is well-known, though,
that the RH is active in several ways: (1) It plays a dominant role in some sub-processes
such as prosody and metaphor comprehension (see, in this volume, Rapp, Chapter 28,
and van Berkum, Chapter 29). (2) Neuroimaging has made clear that language lateral-
ization in all functions is not absolute, processes are distributed across the hemisphere
with a dominance in the LH for most language processes, and the RH can even be the
dominant hemisphere in atypical left-handers. Moreover, individual differences are
often explained by variations in RH activation. (3) Language impairments and mental
disorders with a language component often increase the importance of the RH, as well
as an extension of the language system, such as in the case of bilingualism. We will now
discuss these three domains in more detail.
Prosody refers to either the variations in pitch and rhythm that form the emotional
expression of the speaker, called emotional prosody, or the stress within sentences and
words, called linguistic prosody. A meta-analysis by Belyk and Brown (2014) makes clear
that the two types of prosody share many common brain regions, but are nevertheless
distinct and differ in their lateralization. Pitch modulation is a common factor of both
prosody types, leading to joint activation in the posterior superior temporal gyrus, with
an RH dominance for emotional prosody and a more bilateral distribution for linguistic
prosody. Common activity was also found in the RH supplementary motor area linked to
speech production. A differentiation was found in the inferior frontal gyrus: evaluating
affective cues in speech activates the pars orbitalis bilaterally (Brodmann area 47), a re-
gion connected to areas involved in experiencing and perceiving emotions, such as the
amygdala, whereas linguistic prosody is more related with the bilateral pars opercularis
(Brodmann area 44), which can be explained by its role in lexico-syntactic processing
(Belyk & Brown, 2014). These meta-analytic results thus suggest RH involvement of
auditory temporal regions and bilateral involvement of frontal regions irrespective of

prosody type, in contrast to inconsistencies regarding lateralization in individual neu-
roimaging studies. For example, Witteman et al. (2014) found a right-ear advantage for
both emotional and linguistic evaluation of dichotic listening stimuli associated with an
early negativity event-related potential latency (see Leckey & Federmeier, Chapter 3 in
this volume) in the contralateral hemisphere. In contrast, Wildgruber et al. (2005) did
find functional asymmetries in an fMRI experiment in which participants were asked
to either judge the emotions (basic emotions: anger, happiness, sadness, fear, and dis-
gust) or linguistic content (i.e., reporting the first vowel following the first heard /a/) of
sentences. Apart from common widespread bilateral activation, emotional judgment
tapped into an RH network, including the posterior superior temporal sulcus, and dorso-
lateral and orbitobasal frontal cortices. The linguistic task made use of LH speech areas,
such as the dorsolateral frontal cortex. These studies show a less consistent pattern than
suggested in Belyk and Brown (2014), which could in part be explained by the different
paradigms used.
A similar controversy of RH involvement probably depending on the type of task
being used can be found in the literature on non-literal language comprehension, such
as metaphor processing. Here again, the fronto-temporal language network is impor-
tant with LH involvement in inferior, middle, and superior temporal gyri and the in-
ferior frontal gyrus. Yang, Edens, Simpson, and Krawczyk (2009) reported that the
amount of RH involvement mainly depends on the difficulty level of the sentences
used, and not whether participants have to judge novel or known metaphors. Semantic
judgments rely more on frontal areas, whereas imageability affected parietal regions
such as the precuneus, but the laterality of these regions depended on the difficulty level
of the stimuli.
This leads to the assumption that RH variations can generally explain individual
differences in language laterality. Vigneau et al. (2011) evaluated the role of the RH in
language tasks in a meta-analysis showing that LH activity occurred much more often
in a unilateral fashion, whereas most RH peaks were observed in the case of bilateral
peaks (i.e., in combination with LH activity in a homologue region). Phonological
tasks activated RH auditory and motor areas (with the exception of only LH mouth-
related activity and verbal working-memory tasks). Lexico-semantic tasks made use
of RH frontal regions to support attentional and working-memory processes. During
high-level sentence and text processing, the RH even had an exclusive role in temporal
regions. Seghier, Kherif, Josse, and Price (2011) in addition highlighted that laterality
differs across regions, confirming our general claim that language laterality should not
be expressed in one unitary lateralization index. In Seghier et al. (2011), high LH angular
gyrus activity was connected with low LH ventral precentral laterality. Even more inter-
esting, interregional and inter-participant variability across 50 activated brain regions
and 82 healthy participants performing a semantic word-matching task was mainly
driven by differences in RH rather than LH activation. A dynamic causal modeling anal-
ysis of data from the same task revealed that the degree of LH laterality during word
matching increased when the coupling from LH to RH dorsal frontal cortices decreased
(Seghier, Josse, Leff, & Price, 2011).
If RH activity determines individual laterality patterns, can it also predict differences
in performance? Van Ettinger-Veenstra et al. (2010) ran a dichotic listening task (taken
as a general language laterality measure) and fMRI sentence completion task (i.e., com-
plete the last word of a sentence versus the same sentences with asterisks replacing
letters) to correlate their lateralization indices with performance on a wide variety of
language tasks (e.g., sentence, text and metaphor comprehension, picture naming, and
verbal fluency). Verbal fluency correlated positively with right-ear reports during di-
chotic listening, but overall, a decreased right-ear advantage led to better performance
in almost all language assessment tasks. Rightward fMRI indices were also associated
with better reading ability in the posterior temporal lobe and better comprehension
scores in the inferior frontal cortex. On the other hand, Prat, Mason, and Just (2010)
found increased RH frontal, temporal, and inferior occipital activity in participants
with a smaller vocabulary size when they had to read more complex sentence pairs. This
suggests that the RH is activated more if the participant experiences difficulties in lan-
guage comprehension. Dehaene et al. (2010) reported increasing LH activity in the vOT
with increasing literacy. Pinel et al. (2014) localized the vOT more anteriorly (associated
with larger orthographic units) in participants who showed a smaller difference in reac-
tion time when reading pseudo-words compared to words, but did not find differences
in the amount of activity. Related to reading impairment, dyslexia was associated with
less optimal connections between language areas in the LH only (Paulesu et al., 2014;
Vandermosten et al., 2012). The different paradigms again impede straightforward
conclusions about the relationship between the degree and direction of laterality, on
the one hand, and language performance, on the other hand. Moreover, the functions
tested in behavioral performance assessments are often different from those used to cal-
culate the functional neural lateralization indices (see also Boles & Barth, 2011). One
could speculate that, for example, RH activity increases when phonological, semantic,
or reading processes become too difficult for the LH alone, and that language becomes
more LH lateralized when performance increases because the RH is no longer needed.
A similar idea was formulated by Prat et al. (2010) as the RH spillover hypothesis: that is,
the RH contains coarser language abilities than the LH, but can take over if needed, for
example, after a lesion or when the LH exceeds its capability during difficult tasks.
The dynamic role of the RH can be studied when existing knowledge is extended
by learning a new language (see Green & Kroll, Chapter 11 in this volume). Xiang et al.
(2015) collected longitudinal diffusion tension imaging data from native German
speakers following an intensive six-week Dutch course. They found that tracts between
Brodmann area 6 and temporal regions, overlapping with the arcuate fasciculus, be-
came more RH lateralized in the beginning stages of the course but again turned into
an LH dominance with better proficiency in their second language. It thus seems that
structural connectivity is indeed reorganized to the RH when bilingualism challenges
the language network, but then again becomes typically LH dominant when proficiency
is high enough. In line with this point of view, Kepinska et al. (2017) also pointed out that
fronto-parietal structural connectivity (right anterior segment of arcuate fasciculus) in

the right hemisphere plays an important role in superior language learning, which might
involve attentional processes and reasoning abilities. A functional increase has also been
observed for bilinguals relative to monolinguals during native language picture naming
and reading aloud in the LH dorsal precentral gyrus, pars opercularis/triangularis, supe-
rior temporal gyrus, and planum temporale (Parker Jones et al., 2012). Parker Jones and
colleagues attributed this difference to a higher demand of word retrieval, articulation,
and speech monitoring in bilinguals. A meta-analysis of behavioral lateralization tasks
by Hull and Vaid (2007) pointed to the importance of taking age of acquisition into ac-
count: Early bilinguals who learned their second language before the age of 6 activated
language areas more bilaterally, whereas later bilinguals were more LH dominant, es-
pecially when proficiency was low, English was the second language, and laterality was
measured in a dichotic listening task. Learning a new language even influences brain
areas recruited during native-language processing. Mei et al. (2014) trained Chinese
speakers who had English as their second language with an artificial language based
on Korean Hangul characters. The training focused on semantic learning, not just on
acquiring visual-phonological correspondences. The results showed reduced LH ac-
tivity in the pars opercularis/triangularis, and bilateral inferior temporal gyrus, fusiform
gyrus, and inferior occipital gyrus while reading Chinese. The semantic training even
influenced similar regions during English reading. In line with Xiang et al. (2015), the
effects were strongest in the initial stages of the training and decreased with increasing
proficiency. Finally, laterality differences in bilingualism depend on the similarity be-
tween the two acquired languages, usually tested in bilinguals with knowledge of al-
phabetical languages such as English and logographic languages such as Chinese and
Japanese Kanji. English-Chinese bilinguals activate the RH posterior fusiform cortex
more than English monolinguals, in a similar way as Chinese monolinguals show a
more rightward lateralization during reading than English monolinguals (Mei et al.,
2015). Koyama et al. (2014) drew similar conclusions from late bilinguals who were ei-
ther native speakers of Japanese syllabic Kana and logographic Kanji languages and
learned English, or native English speakers who mastered Japanese. A visual one-back
reading task revealed less LH lateralization in the posterior lateral occipital region for
logographic Kanji compared to syllabic Kana and alphabetic English, irrespective of the
participant’s first language. If the first and second language were non-logographic, no
lateralization differences were found.
Finally, more RH activity has been associated with language-related areas in mental
disorders. FOXP2 polymorphisms, previously linked to reduced reading ability and
dyslexia, were also associated with schizophrenia and autism spectrum disorder
(Gong et al., 2004; Li et al., 2013). Schizophrenia patients in Bleich-Cohen et al. (2012)
showed a reduced LH asymmetry and interhemispheric connectivity in the inferior
frontal gyrus, an effect that was more pronounced with increasing severity of negative
symptoms. Kleinhans, Müller, Cohen, and Courchesne (2008) found increased activity
in RH frontal and superior temporal regions during verbal fluency word generation in
participants with autism spectrum disorder, characterized by impairments in speech,
syntax, and pragmatic knowledge compared to controls (see also Lindell & Hudry, 2013,
for a review concluding that asymmetries become more atypical with increasing lan-
guage impairments).
Language Laterality in Relation

to Other Functions
It is clear from what we have discussed so far that all sub-processes of language can vary
in their lateralization across individuals. This can be a fruitful starting point to gain
knowledge about how the human brain is organized, both with respect to language and
non-language functions (Cai & Van der Haegen, 2015). There are two dominant views
on how functions are distributed across the RH or LH. The statistical independence
view states that functions develop irrespective of how already established functions are
organized. Alternatively, according to the competitive complementary hypothesis, cor-
tical space is limited, and thus brain functions compete with each other for resources.
The latter view makes clear that apart from the seven possible influences on laterality we
reviewed per language sub-process (handedness, anatomy, evolution, genetics, develop-
ment, experience, and impairment) there is an eighth possible factor: cooperative and
competitive non-language functions (Behrmann & Plaut, 2013). Recent neuroimaging
studies most extensively explored the relationship between reading and face recogni-
tion, and between speech production and visuospatial attention.
Behrmann and Plaut (2013) introduce three views on brain functioning: (1) the one-
to-one vision in which one structure can be coupled with one cognitive function and
vice versa; (2) the one-to-many vision in which a cognitive function triggers a net-
work of structures and vice versa; (3) the many-to-many vision that integrates the
first two perspectives and argues that structures and functions form distributed but
interacting networks. Evidence for the many-to-many view can be found in the ventral
visual cortex, where nodes are optimized for a specific visual category such as the vOT
for words (Cohen et al., 2000) and the fusiform face area for faces (FFA; Kanwisher,
McDermott, & Chun, 1997). At first sight, words and faces have more differences than
commonalities. They both contain detailed information that has to be processed in cen-
tral vision, but face recognition developed much earlier in evolution than reading and
consists of coarser elements than the fine-grained line junctions in letters (Behrmann
& Plaut, 2013). Statistical independence could argue that spatial frequencies pushed
reading into the LH and faces into the RH, with the LH/RH being inherently more sensi-
tive to high/low spatial frequencies, respectively (see Woodhead, Wise, Sereno, & Leech,
2011, who reported interhemispheric sensitivity to spatial frequencies; but see Ossowski
& Behrmann, 2015, for evidence that LH high spatial frequency sensitivity was not found
in pre-literate children, suggesting that the sensitivity is a consequence rather than cause
of reading). Yet, interactions between reading and face recognition have been reported
in several recent studies supporting the competitive hypothesis (Dehaene et al., 2010;
Dundas, Plaut, & Behrmann, 2013, 2015; Cantlon, Pinel, Dehaene, & Pelphrey, 2011; Li
et al., 2013). For example, Dundas et al. (2013) found a right/left visual field advantage
for words and faces, respectively, but only in adults and not in children or adolescents.
The younger participants showed only a right field advantage for words, with no visual
field preference for faces. Face lateralization additionally correlated with reading per-
formance. This could be explained from a developmental point of view that face recog-
nition is bilaterally processed at first, but is then pushed into an RH dominance because
words occupy overlapping neurons in the LH in order to optimize the connections with
earlier LH lateralized fronto-temporal language regions (Dundas et al., 2013). Cantlon
et al. (2011) found an LH occipito-temporal sensitivity to letters and digits, and an RH
mid-fusiform dominance for faces in 4-year-old children in an fMRI study showing
pictures of these visual categories and shoes. The early RH face dominance somewhat
contradicts Dundas et al. (2013), who did not find a face lateralization before the mean
adult age of 21, but more interestingly, Cantlon et al. (2011) add evidence for a competing
view on word and face processing because children’s behavioral symbol (i.e., letters and
numbers together) matching scores correlated negatively with face-selective activity in
the LH occipito-temporal region, whereas there was not even a correlation found with
symbol neural activity itself in the same region. Similarly, behavioral face-matching
accuracy was not associated with increased neural sensitivity for faces in the RH face-
selective fusiform area, but with decreased activity for shoes, with no association with
neural activity for symbols. Thus, visual areas generally seem to specialize by means of
pruning away non-preferred categories, rather than favoring preferred categories. With
respect to word and face lateralization in particular, an anti-correlation was found in
the LH but not in the RH, even though the authors remark that this could be different in
other stages of development. Looking at what atypical handedness could tell about this
issue, no study to date has compared word and face lateralization in left-handers whose
reading lateralization was clearly RH dominant, but Willems, Peelen, and Hagoort
(2009) and Bukowski, Dricot, Hanseeuw, and Rossion (2013) found overall reduced RH
face lateralization in the FFA of left-handers compared to right-handers. Dundas et al.
(2015) confirmed this finding, together with LH word discrimination superiority for
both right-and left-handers. In their study, more negative LH N170 event-related poten-
tial components (previously shown to distinguish orthographic from non-orthographic
stimuli in the LH and sensitive to faces in the RH) during word presentations predicted
a stronger amplitude for the N170 for faces and more RH behavioral asymmetries for
faces. In the earlier discussed study by Pinel et al. (2014) using a one-back visual task
with words and faces, among other visual categories and a speech-processing paradigm,
different contributing factors to reading and face lateralization came forward: vOT ac-
tivity showed significant intra-pair correlations for monozygotic twins but the FFA did
not, suggesting that the vOT partially develops under genetic influences, whereas the
FFA is more sensitive to environmental factors. RH FFA lateralization did not directly
correlate with LH vOT lateralization, but was related to LH lateralization of the supe-
rior temporal sulcus during speech listening and reading skill, as operationalized by the
additional time cost in naming pseudo-words compared to words. Finally, Badzakova-

Trajkov, Häberling, Roberts, and Corballis (2010) confirmed the correlation between
RH face recognition and LH word generation, but were among the first to relate these
two lateralized functions to another widely studied RH-dominant cognitive func-
tion: visuo-spatial attention.
According to Kosslyn (1987), visual attention to stimuli in the environment, like
speech production, is unilaterally processed because rapid and precise actions are best
coordinated by one control system. In addition, the control systems are best located
in opposite hemispheres so that they do not interfere. This is in line with the compet-
itive view in which functions can be crowded out, and in contrast with the statistical
independence view in which lateralization of functions develop independently from
each other. The 155 participants from Badzakova-Trajkov et al. (2010) did show a small
but significant negative correlation between the verbal fluency word-generation task
in frontal areas and the bisection task measuring visuospatial attention (i.e., judging
whether a vertical line bisects a horizontal line versus indicating whether a vertical line
is present in the control condition) in parietal areas. The bisection task was, however,
the only task that did not elicit more atypical patterns in left-handers, and its lateral-
ization did not correlate with RH face lateralization. We have argued before that an
effect of handedness is not necessary to observe (a)typical lateralization patterns, and
visuospatial attention has theoretically not been linked to face processing, but the rela-
tionship between speech production and visuospatial attention is indeed more mixed
in the literature than the complementary lateralization patterns of reading and face
recognition.
The first doubts about causal complementarity came from patient observations with a
unilateral brain lesion. If speech and visuospatial attention indeed crowd each other out,
then LH-damaged patients should have aphasia with spatial ability problems, and vice
versa for RH-lesioned patients (Bryden, Hécaen, & DeAgostini, 1983). This, however,
was not always the case: LH lesions led to aphasia, but also to spatial ability deficits or
both, with a comparable picture for RH lesions. Zago et al. (2016) also found indepen-
dence between lateralization indices of speech production (measured by contrasting
sentence production based on a line drawing versus recalling the months of the year)
and visuospatial attention (measured by line bisection versus fixation). At the group
level, the 293 participants balanced for handedness mostly activated LH fronto-temporal
language regions during speech production and RH frontal and posterior occipito-
parietal-temporal regions during visuospatial attention, but a negative correlation was
only found for left-handers with a strong manual preference. A clear interhemispheric
distinction was reported by Cai, Van der Haegen, and Brysbaert (2013): All but one of
16 left-handers who were clearly LH lateralized for speech production (i.e., verbal flu-
ency word generation versus nonword repetition) were RH lateralized for visuospatial
attention (i.e., line bisection versus indicating whether a vertical line touched a hori-
zontal line), and all 13 left-handers with clear RH speech dominance were LH dominant
for visuospatial attention that elicited activation in the dorsal fronto-parietal attention
pathway and inferior frontal regions.
Why is there agreement that reading and face recognition negatively correlate in line
with a causal complementary view on brain functions, whereas speech production and
visuospatial attention only seem to lateralize to the opposite hemisphere in extreme
left-handers but support statistical independence in the remainder of the population? It
could be that visuospatial attention tasks evoke less stable neural patterns than, for ex-
ample, the verbal fluency task. Pernet et al. (2016) evaluated the between-and within-
subject variance of a speech-production (verb generation triggering Broca’s area)
and landmark task (indicating which of two lines are smaller or larger, activating the
intraparietal cortex) in order to estimate their reliability as clinical presurgical tools.
A reliable paradigm should have higher consistency over two sessions tested within
the same participant than at the group level, which was the case for word repetition but
not for the landmark task. Another possible explanation for a stronger causal comple-
mentarity between reading and faces than between speech production and visuospatial
attention is that the interplay between the function pairs is based on different under-
lying mechanisms. In the first pair, faces are presumably crowded out of the LH once
the visual cortex needs to adapt to reading, which is preferably dominated by the LH to
co-lateralize with earlier developed fronto-temporal language functions (Behrmann &
Plaut, 2013). This is a competition for cortical space in homologue areas between an al-
ready established and newly developing cognitive function. Language and visuospatial
attention, on the other hand, are more widespread throughout the cortex and are pre-
sumably separated because they each need a unilateral control system (Kosslyn, 1983).
Petit et al. (2015) suggested another evolutionary explanation: In the sample studied by
Zago et al. (2016), the dorso-parietal attention network was especially RH lateralized in
strong left-handers, and even more so if they were right-eye sighted. Petit et al. argue
that processing visuospatial control of the environment in the same hemisphere as
their dominant hand may give these left-handers an advantage in, for example, inter-
active sports. It may thus be that visuospatial attention evolutionarily first lateralized
to the RH because of functional advantages, and that language was then crowded out
to the LH without having large overlapping neural populations that have to move for
a competing function. Whatever reasons future studies will point out with respect
to competitive or statistical independent relationships between functions, this over-
view makes clear that lateralization research, and in particular studies including atyp-
ically lateralized left-handers, can reveal interesting insights into brain mechanisms
in general.
Conclusion
We can conclude from this chapter that language laterality gives unique information
on how language and non-language functions are being processed in the human brain.
It can be seen (前面刚用了 this overview made clear) that most language functions,
such as speech production, auditory speech processing, and reading are dominantly
processed in the LH in most humans, even though the degree of laterality can differ
across functions and across individuals. Left-handers introduce more variability in
the samples studied, leading to insights into mutual interactions when one factor such
as speech production is atypically lateralized. Possible causes of an LH dominance
have been proposed for each sub-process from studies reporting anatomical, evolu-
tionary, genetic, developmental, experiential factors, and consequences of impaired
functions and lesion studies in patients. We further discussed that the role of the RH
within language remains underestimated, but prosody and metaphors are well-known
to be dominated by the RH in most participants, and its importance to explain indi-
vidual variability in healthy participants, dyslexia, and mental disorders such as au-
tism spectrum disorder and schizophrenia is widely acknowledged. Recent laterality
literature is dominated by neuroimaging studies mapping the direction and degree
of lateralized functions, often (but far from always) reporting results of behavioral
correlates that make it still unclear to what extent the lateralization of functions affects
cognitive performance. It will thus be important to continue to combine different
methodologies, while trying to optimize paradigms in order to facilitate the compa-
rability across studies. Finally, language lateralization can be an ideal starting point to
explore general brain mechanisms. We have discussed competitive complementarity
between reading and face recognition, and, at least in strong or atypically lateralized
left-handers, between speech production and visuospatial attention. From a wider
perspective, future laterality studies can also reveal how the human brain works,
beyond focusing on one or two cognitive functions. For example, Liu, Stufflebeam,
Sepulcre, Hedden, and Buckner (2009) identified four asymmetric networks during
resting state activity that could roughly be related to regions involved in vision, in-
ternal thought, attention, and language. If these networks are indeed independent,
with their own degree of inter-individual variability, then neuroscientists should not
try to find one explanation (competitive complementarity or statistical independence)
for the whole brain, but instead should investigate how and why separate clusters are
grouped into networks underlying different functions, and how they are related to
each other. One view on how the numerous and complex functions our brain houses
can function so well together was given by Fedorenko and Thompson-Schill (2014).
They argue that apart from a domain-general attention-related multiple demand net-
work, the language network consists of functionally specialized core regions (i.e.,
regions that are consistently activated during language tasks and co-activate) and
nonspecialized periphery regions (i.e., regions that are activated during language
tasks, but can also co-activate with regions belonging to another specialized network
depending on the current task). In this chapter, we have mainly focused on the core
regions of language because we know most about the laterality of these regions, but
future lateralization studies could test the view of Fedorenko and Thompson-Schill
(2014), among other views on brain mechanisms, to further chart functional and ana-
tomical asymmetries, starting from variations in language lateralization.
References
Abbott, D. F., Waites, A. B., Lillywhite, L. M., & Jackson, G. D. (2010). fMRI assessment of lan-
guage lateralization: An objective approach. NeuroImage, 50(4), 1446–1455.
Allen, J. S., Emmorey, K., Bruss, J., & Damasio, H. (2013). Neuroanatomical differences in
visual, motor, and language cortices between congenitally deaf signers, hearing signers, and
hearing non-signers. Frontiers in Neuroanatomy, 7, 26.
Allendorfer, J. B., Hernando, K. A., Hossain, S., Nenert, R., Holland, S. K., & Szaflarski, J. P.
(2016). Arcuate fasciculus asymmetry has a hand in language function but not handedness.
Amunts, K., Lenzen, M., Friederici, A. D., Schleicher, A., Morosan, P., Palomero-Gallagher,
N., & Zilles, K. (2010). Broca’s region: Novel organizational principles and multiple receptor
mapping. PLoS Biology, 8(9), e1000489.
Amunts, K., Schleicher, A., Burgel, U., Mohlberg, H., Uylings, H. B. M., & Zilles, K. (1999).
Broca’s region revisited: Cytoarchitecture and intersubject variability. Journal of Comparative
Neurology, 412(2), 319–341.
Annett, M. (1998). Handedness and cerebral dominance: The right shift theory. Journal of
Neuropsychiatry and Clinical Neurosciences, 10(4), 459–469.
Badzakova-Trajkov, G., Haberling, I. S., Roberts, R. P., & Corballis, M. C. (2010). Cerebral
asymmetries: Complementary and independent processes. Plos One, 5(3), e9682.
Behrmann, M., & Plaut, D. C. (2013). Distributed circuits, not circumscribed centers, mediate
visual recognition. Trends in Cognitive Sciences, 17(5), 210–219.
Behrmann, M., & Plaut, D. C. (2015). A vision of graded hemispheric specialization. Annals of
the New York Academy of Sciences, 1359,  30–46.
Belyk, M., & Brown, S. (2014). Perception of affective and linguistic prosody: An ALE meta-
analysis of neuroimaging studies. Social Cognitive and Affective Neuroscience, 9(9),
1395–1403.
Ben-Shachar, M., Dougherty, R. F., Deutsch, G. K., & Wandell, B. A. (2011). The development of
cortical sensitivity to visual word forms. Journal of Cognitive Neuroscience, 23(9), 2387–2399.
Benson, D. F., & Geschwind, N. (1985). Aphasia and related disorders: A clinical approach. In
M.-M. Mesulam (Ed.), Principles of behavioral neurology. (pp. 193–238). Philadelphia: Davis.
Binder, J. R., Swanson, S. J., Hammeke, T. A., Morris, G. L., Mueller, W. M., Fischer,
M., . . . Haughton, V. M. (1996). Determination of language dominance using functional
MRI: A comparison with the Wada test. Neurology, 46(4), 978–984.
Bleich-Cohen, M., Sharon, H., Weizman, R., Poyurovsky, M., Faragian, S., & Hendler, T.
(2012). Diminished language lateralization in schizophrenia corresponds to impaired inter-
hemispheric functional connectivity. Schizophrenia Research, 134(2–3), 131–136.
Boles, D. B., & Barth, J. M. (2011). Does degree of asymmetry relate to performance? A critical
review. Brain and Cognition, 76(1),  1–4.
Bozic, M., Tyler, L. K., Ives, D. T., Randall, B., & Marslen-Wilson, W. D. (2010). Bihemispheric
foundations for human speech comprehension. Proceedings of the National Academy of
Sciences USA, 107(40), 17439–17444.
Brandler, W. M., Morris, A. P., Evans, D. M., Scerri, T. S., Kemp, J. P., Timpson, N.
J., . . . Paracchini, S. (2013). Common variants in left/right asymmetry genes and pathways
are associated with relative hand skill. Plos Genetics, 9(9), e1003751.
Broca, P. (1865). On the location of the faculty of articulate language in the left hemisphere of
the brain. Bulletin of the Society of Anthropology, 6, 377–393.
Bryden, M. P., Hecaen, H., & DeAgostini, M. (1983). Patterns of cerebral organization. Brain
and Language, 20(2), 249–262.
Bukowski, H., Dricot, L., Hanseeuw, B., & Rossion, B. (2013). Cerebral lateralization of face-
sensitive areas in left-handers: Only the FFA does not get it right. Cortex, 49(9), 2583–2589.
Cai, Q., Lavidor, M., Brysbaert, M., Paulignan, Y., & Nazir, T. A. (2008). Cerebral lateralization
of frontal lobe language processes and lateralization of the posterior visual word processing
system. Journal of Cognitive Neuroscience, 20(4), 672–681.
Cai, Q., Paulignan, Y., Brysbaert, M., Ibarrola, D., & Nazir, T. A. (2010). The left ventral
occipito-temporal response to words depends on language lateralization but not on visual
familiarity. Cerebral Cortex, 20(5), 1153–1163.
Cai, Q., & Van der Haegen, L. (2015). What can atypical language hemispheric specialization
tell us about cognitive functions? Neuroscience Bulletin, 31(2), 220–226.
Cai, Q., Van der Haegen, L., & Brysbaert, M. (2013). Complementary hemispheric speciali-
zation for language production and visuospatial attention. Proceedings of the National
Academy of Sciences USA, 110(4), E322–E330.
Cantlon, J. F., Pinel, P., Dehaene, S., & Pelphrey, K. A. (2011). Cortical representations of symbols,
objects, and faces are pruned back during early childhood. Cerebral Cortex, 21(1), 191–199.
Catani, M., Allin, M. P. G., Husain, M., Pugliese, L., Mesulam, M. M., Murray, R. M., & Jones,
D. K. (2007). Symmetries in human brain language pathways correlate with verbal recall.
Cochet, H. (2015). Manual asymmetries and hemispheric specialization: Insight from develop-
mental studies. Neuropsychologia, 93, 335–341.
Cohen, L., Dehaene, S., Naccache, L., Lehericy, S., Dehaene-Lambertz, G., Henaff, M. A., &
Michel, F. (2000). The visual word form area: Spatial and temporal characterization of an ini-
tial stage of reading in normal subjects and posterior split-brain patients. Brain, 123, 291–307.
Corballis, M. C. (2003). From mouth to hand: Gesture, speech, and the evolution of right-
handedness. Behavioral and Brain Sciences, 26(2), 199–208.
Corballis, M. C. (2009). The evolution and genetics of cerebral asymmetry. Philosophical
Transactions of the Royal Society of London, Series B, Biological Sciences, 364(1519), 867–879.
Cornelissen, P. L., Kringelbach, M. L., Ellis, A. W., Whitney, C., Holliday, I. E., & Hansen, P. C.
(2009). Activation of the left inferior frontal gyrus in the first 200 ms of reading: Evidence
from magnetoencephalography (MEG). Plos One, 4(4), e5359.
Dehaene-Lambertz, G., Dehaene, S., & Hertz-Pannier, L. (2002). Functional neuroimaging of
speech perception in infants. Science, 298(5600), 2013–2015.
Dehaene, S., Cohen, L., Morais, J., & Kolinsky, R. (2015). Illiterate to literate: Behavioural
and cerebral changes induced by reading acquisition. Nature Reviews Neuroscience, 16(4),
234–244.
Dehaene, S., Pegado, F., Braga, L. W., Ventura, P., Nunes Filho, G., Jobert, A., . . . Cohen, L.
(2010). How learning to read changes the cortical networks for vision and language. Science,
330(6009), 1359–1364.
Dubois, J., Hertz-Pannier, L., Cachia, A., Mangin, J. F., Le Bihan, D., & Dehaene-Lambertz,
G. (2009). Structural asymmetries in the infant language and sensori-motor networks.
Dundas, E. M., Plaut, D. C., & Behrmann, M. (2013). The joint development of hemispheric
lateralization for words and faces. Journal of Experimental Psychology: General, 142(2),
348–358.
Dundas, E. M., Plaut, D. C., & Behrmann, M. (2015). Variable left-hemisphere language and or-
thographic lateralization reduces right-hemisphere face lateralization. Journal of Cognitive
Emmorey, K., Mehta, S., & Grabowski, T. J. (2007). The neural correlates of sign versus word
production. NeuroImage, 36(1), 202–208.
Fadiga, L., Craighero, L., & D’Ausilio, A. (2009). Broca’s area in language, action, and music.
Annals of the New York Academy of Sciences, 1169, 448–458.
Francks, C., Maegawa, S., Lauren, J., Abrahams, B. S., Velayos- Baeza, A., Medland, S.
E., . . . Monaco, A. P. (2007). LRRTM1 on chromosome 2p12 is a maternally suppressed gene
that is associated paternally with handedness and schizophrenia. Molecular Psychiatry,
12(12), 1129–1139.
Friederici, A. D., Brauer, J., & Lohmann, G. (2011). Maturation of the language network: From
inter-to intrahemispheric connectivities. Plos One, 6(6), e20726.
Gazzaniga, M.S. (1975). Review of the split brain. Journal of Neurology, 209(2),  75–79.
Gazzaniga, M. S. (2005). Forty-five years of split-brain research and still going strong. Nature
Gazzaniga, M. S., Bogen, J. E., & Sperry, R. W. (1962). Some functional effects of sectioning the
cerebral commissures in man. Proceedings of the National Academy of Sciences USA, 48(10),
1765–1769.
Geschwind, N., & Levitsky, W. (1968). Human brain: Left-right asymmetries in temporal
speech region. Science, 161(3837), 186–187.
Gong, X., Jia, M., Ruan, Y., Shuang, M., Liu, J., Wu, S., . . . Zhang, D. (2004). Association be-
tween the FOXP2 gene and autistic disorder in Chinese population. American Journal of
Medical Genetics B: Neuropsychiatric Genetics, 127b(1), 113–116.
Greve, D. N., Van der Haegen, L., Cai, Q., Stufflebeam, S., Sabuncu, M. R., Fischl, B., &
Brysbaert, M. (2013). A surface-based analysis of language lateralization and cortical asym-
metry. Journal of Cognitive Neuroscience, 25(9), 1477–1492.
Häberling, I. S., Steinemann, A., & Corballis, M. C. (2016). Cerebral asymmetry for lan-
guage: Comparing production with comprehension. Neuropsychologia, 80,  17–23.
Hécaen, H., & Sauguet, J. (1971). Cerebral dominance in left-handed subjects. Cortex,
7,  19–48.
Hirata, M., Goto, T., Barnes, G., Umekawa, Y., Yanagisawa, T., Kato, A., . . . Yoshimine,
T. (2010). Language dominance and mapping based on neuromagnetic oscilla-
tory changes: Comparison with invasive procedures. Journal of Neurosurgery, 112(3),
528–538.
Holland, S. K., Vannest, J., Mecoli, M., Jacola, L. M., Tillema, J.-M., Karunanayaka, P.
R., . . . Byars, A. W. (2007). Functional MRI of language lateralization during development in
children. International Journal of Audiology, 46(9), 533–551.
Hopkins, W. D., & Cantalupo, C. (2008). Theoretical speculations on the evolutionary origins
of hemispheric specialization. Current Directions in Psychological Science, 17(3), 233–237.
Hopkins, W. D., Marino, L., Rilling, J. K., & MacGregor, L. A. (1998). Planum temporale
asymmetries in great apes as revealed by magnetic resonance imaging (MRI). Neuroreport,
9(12), 2913–2918.
Hopkins, W. D., & Nir, T. M. (2010). Planum temporale surface area and grey matter
asymmetries in chimpanzees (Pan troglodytes): The effect of handedness and comparison
with findings in humans. Behavioral Brain Research, 208(2), 436–443.
Hugdahl, K. (2011). Hemispheric asymmetry: Contributions from brain imaging. Wiley
Interdisciplinary Reviews: Cognitive Science, 2(5), 461–478.
Hugdahl, K., Carlsson, G., Uvebrant, P., & Lundervold, A. J. (1997). Dichotic-listening perfor-
mance and intracarotid injections of amobarbital in children and adolescents: Preoperative
and postoperative comparisons. Archives of Neurology, 54(12), 1494–1500.
Hull, R., & Vaid, J. (2007). Bilingual language lateralization: A meta-analytic tale of two
hemispheres. Neuropsychologia, 45(9), 1987–2008.
Indefrey, P. (2011). The spatial and temporal signatures of word production components: A
critical update. Frontiers in Psychology, 2, 255.
Kanwisher, N., McDermott, J., & Chun, M. M. (1997). The fusiform face area: A module in
human extrastriate cortex specialized for face perception. Journal of Neuroscience, 17(11),
4302–4311.
Keller, S. S., Roberts, N., Garcia-Finana, M., Mohammadi, S., Ringelstein, E. B., Knecht, S., &
Deppe, M. (2011). Can the language-dominant hemisphere be predicted by brain anatomy?
Kepinska, O., Lakke, E. A. J. F, Dutton, E. M., Caspers, J., & Schiller, N. O. (2017). The perisylvian
language network and language analytical abilities. Neurobiology of Learning and Memory,
144, 96–101.
Kimura, D. (1961). Cerebral-dominance and the perception of verbal stimuli. Canadian Journal
of Psychology, 15(3), 166–171.
Kleinhans, N. M., Muller, R. A., Cohen, D. N., & Courchesne, E. (2008). Atypical functional
lateralization of language in autism spectrum disorders. Brain Research, 1221, 115–125.
Knecht, S., Drager, B., Deppe, M., Bobe, L., Lohmann, H., Floel, A., . . . Henningsen, H.
(2000). Handedness and hemispheric language dominance in healthy humans. Brain, 123,
2512–2518.
Kosslyn, S. M. (1987). Seeing and imagining in the cerebral hemispheres: A computational ap-
proach. Psychological Review, 94(2), 148–175.
Koyama, M. S., Stein, J. F., Stoodley, C. J., & Hansen, P. C. (2014). A cross-linguistic evaluation
of script-specific effects on fMRI lateralization in late second language readers. Frontiers in
Leroy, F., Cai, Q., Bogart, S. L., Dubois, J., Coulon, O., Monzalvo, K., . . . Dehaene-Lambertz,
G. (2015). New human-specific brain landmark: The depth asymmetry of superior temporal
sulcus. Proceedings of the National Academy of Sciences USA, 112(4), 1208–1213.
Li, S., Lee, K., Zhao, J., Yang, Z., He, S., & Weng, X. (2013). Neural competition as a develop-
mental process: Early hemispheric specialization for word processing delays specialization
for face processing. Neuropsychologia, 51(5), 950–959.
Li, T., Zeng, Z., Zhao, Q., Wang, T., Huang, K., Li, J., . . . Shi, Y. (2013). FoxP2 is significantly as-
sociated with schizophrenia and major depression in the Chinese Han population. World
Journal of Biological Psychiatry, 14(2), 146–150.
Lindell, A. K., & Hudry, K. (2013). Atypicalities in cortical structure, handedness, and functional
lateralization for language in autism spectrum disorders. Neuropsychology Review, 23(3),
257–270.
Liu, H., Stufflebeam, S. M., Sepulcre, J., Hedden, T., & Buckner, R. L. (2009). Evidence from
intrinsic activity that asymmetry of the human brain is controlled by multiple factors.
Marie, D., Jobard, G., Crivello, F., Perchey, G., Petit, L., Mellet, E., . . . Tzourio-Mazoyer, N.
(2015). Descriptive anatomy of Heschl’s gyri in 430 healthy volunteers, including 198
left-handers. Brain Structure & Function, 220(2), 729–743.
Mazoyer, B., Zago, L., Jobard, G., Crivello, F., Joliot, M., Perchey, G., . . . Tzourio-Mazoyer, N.
(2014). Gaussian mixture modeling of hemispheric lateralization for language in a large
sample of healthy individuals balanced for handedness. Plos One, 9(6), e101165.
Mei, L., Xue, G., Lu, Z. L., Chen, C., Wei, M., He, Q., & Dong, Q. (2015). Long-term experi-
ence with Chinese language shapes the fusiform asymmetry of English reading. NeuroImage,
110,  3–10.
Mei, L., Xue, G., Lu, Z.-L., Chen, C., Zhang, M., He, Q., . . . Dong, Q. (2014). Learning to
read words in a new language shapes the neural organization of the prior languages.
Ocklenburg, S., Arning, L., Gerding, W. M., Epplen, J. T., Guentuerkuen, O., & Beste, C. (2013).
FOXP2 variation modulates functional hemispheric asymmetries for speech perception.
Ocklenburg, S., Schlaffke, L., Hugdahl, K., & Westerhausen, R. (2014). From structure to func-
tion in the lateralized brain: How structural properties of the arcuate and uncinate fasciculus
are associated with dichotic listening performance. Neuroscience Letters, 580,  32–36.
Oldfield, R. C. (1971). The assessment and analysis of handedness: The Edinburgh Inventory.
Ossowski, A., & Behrmann, M. (2015). Left hemisphere specialization for word reading poten-
tially causes, rather than results from, a left lateralized bias for high spatial frequency visual
information. Cortex, 72,  27–39.
Paquette, N., Lassonde, M., Vannasing, P., Tremblay, J., Gonzalez-Frankenberger, B., Florea,
O., . . . Gallagher, A. (2015). Developmental patterns of expressive language hemispheric lat-
eralization in children, adolescents and adults using functional near-infrared spectroscopy.
Parker Jones, O., Green, D. W., Grogan, A., Pliatsikas, C., Filippopolitis, K., Ali, N., . . . Price,
during picture naming and reading aloud. Cerebral Cortex, 22(4), 892–902.
Partanen, E., Kujala, T., Naatanen, R., Liitola, A., Sambeth, A., & Huotilainen, M. (2013).
Learning-induced neural plasticity of speech processing before birth. Proceedings of the
National Academy of Sciences USA, 110(37), 15145–15150.
Paulesu, E., Danelli, L., & Berlingeri, M. (2014). Reading the dyslexic brain: Multiple dysfunc-
tional routes revealed by a new meta-analysis of PET and fMRI activation studies. Frontiers
in Human Neuroscience, 8, 830.
Perani, D., Saccuman, M. C., Scifo, P., Anwander, A., Spada, D., Baldoli, C., . . . Friederici, A. D.
(2011). Neural language networks at birth. Proceedings of the National Academy of Sciences
USA, 108(38), 16056–16061.
Pernet, C. R., Gorgolewski, K. J., Job, D., Rodriguez, D., Storkey, A., Whittle, I., & Wardlaw, J.
(2016). Evaluation of a pre-surgical functional MRI workflow: From data acquisition to re-
porting. International Journal of Medical Informatics, 86,  37–42.
Petit, L., Zago, L., Mellet, E., Jobard, G., Crivello, F., Joliot, M., . . . Tzourio-Mazoyer, N. (2015).
Strong rightward lateralization of the dorsal attentional network in left-handers with right
sighting-eye: An evolutionary advantage. Human Brain Mapping, 36(3), 1151–1164.
Pinel, P., Fauchereau, F., Moreno, A., Barbot, A., Lathrop, M., Zelenika, D., . . . Dehaene, S.
(2012). Genetic variants of FOXP2 and KIAA0319/TTRAP/THEM2 locus are associated
with altered brain activation in distinct language-related regions. Journal of Neuroscience,
32(3), 817–825.
Pinel, P., Lalanne, C., Bourgeron, T., Fauchereau, F., Poupon, C., Artiges, E., . . . Dehaene, S.
(2014). Genetic and environmental influences on the visual word form and fusiform face
areas. Cerebral Cortex, 25(9), 2478–2493.
Poeppel, D., Emmorey, K., Hickok, G., & Pylkkänen, L. (2012). Towards a new neurobiology of
language. Journal of Neuroscience, 32(41), 14125–14131.
Prat, C. S., Mason, R. A., & Just, M. (2011). Individual differences in the neural basis of causal
inferencing. Brain and Language, 116(1),  1–13.
contributions to reading. Trends in Cognitive Sciences, 15(6), 246–253.
Pujol, J., Deus, J., Losilla, J. M., & Capdevila, A. (1999). Cerebral lateralization of language in
normal left-handed people studied by functional MRI. Neurology, 52(5), 1038–1043.
Richardson, F. M., Seghier, M. L., Leff, A. P., Thomas, M. S. C., & Price, C. J. (2011). Multiple
routes from occipital to temporal cortices during reading. Journal of Neuroscience, 31(22),
8239–8247.
Rizzolatti, G., & Arbib, M. A. (1998). Language within our grasp. Trends in Neurosciences, 21(5),
188–194.
Rogers, L. J., & Andrew, J. R. (Eds.). (2002). Comparative vertebrate lateralization. Cambridge:
Scott, S. K., & McGettigan, C. (2013). Do temporal processes underlie left hemisphere domi-
nance in speech perception? Brain and Language, 127(1),  36–45.
Seghier, M. L. (2008). Laterality index in functional MRI: Methodological issues. Magnetic
Resonance Imaging, 26(5), 594–601.
Seghier, M. L., Josse, G., Leff, A. P., & Price, C. J. (2011). Lateralization is predicted by reduced
coupling from the left to right prefrontal cortex during semantic decisions on written words.
Seghier, M. L., Kherif, F., Josse, G., & Price, C. J. (2011). Regional and hemispheric determinants
of language laterality: Implications for preoperative fMRI. Human Brain Mapping, 32(10),
1602–1614.
Seghier, M. L., & Price, C. J. (2011). Explaining left lateralization for words in the ventral
occipitotemporal cortex. Journal of Neuroscience, 31(41), 14745–14753.
Shiell, M. M., Champoux, F., & Zatorre, R. J. (2015). Reorganization of auditory cortex in early-
deaf people: Functional connectivity and relationship to hearing aid use. Journal of Cognitive
Sowman, P. F., Crain, S., Harrison, E., & Johnson, B. (2014). Lateralization of brain activa ion
in fluent and non-fluent preschool children: A magnetoencephalographic study of picture-
naming. Frontiers in Human Neuroscience, 8, 354.
Specht, K. (2013). Mapping a lateralization gradient within the ventral stream for auditory
speech perception. Frontiers in Human Neuroscience, 7, 629.
Toga, A. W., & Thompson, P. M. (2003). Mapping brain asymmetry. Nature Reviews
Tzourio-Mazoyer, N., Josse, G., Crivello, F., & Mazoyer, B. (2004). Interindividual variability in
the hemispheric organization for speech. NeuroImage, 21(1), 422–435.
Tzourio-Mazoyer, N., Marie, D., Zago, L., Jobard, G., Perchey, G., Leroux, G., . . . Mazoyer,
B. (2014). Heschl’s gyrification pattern is related to speech-listening hemispheric laterali-
zation: fMRI investigation in 281 healthy volunteers. Brain Structure and Function, 220(3),
1585–1599.
Tzourio- Mazoyer, N., Petit, L., Razafimandimby, A., Crivello, F., Zago, L., Jobard,
G., . . . Mazoyer, B. (2010). Left hemisphere lateralization for language in right-handers is
controlled in part by familial sinistrality, manual preference strength, and head size. Journal
of Neuroscience, 30(40), 13314–13318.
Tzourio-Mazoyer, N., Simon, G., Crivello, F., Jobard, G., Zago, L., Perchey, G., . . . Mazoyer,
B. (2010). Effect of familial sinistrality on planum temporale surface and brain tissue
asymmetries. Cerebral Cortex, 20(6), 1476–1485.
van den Noort, M., Specht, K., Rimol, L. M., Ersland, L., & Hugdahl, K. (2008). A new
verbal reports fMRI dichotic listening paradigm for studies of hemispheric asymmetry.
NeuroImage, 40(2), 902–911.
Van der Haegen, L., Acke, F., Vingerhoets, G., Dhooge, I., De Leenheer, E., Cai, Q., & Brysbaert,
M. (2016). Laterality and unilateral deafness: Patients with congenital right ear deafness do
not develop atypical language dominance. Neuropsychologia, 93(Pt B), 482–492.
Van der Haegen, L., Cai, Q., & Brysbaert, M. (2012). Colateralization of Broca’s area and the
visual word form area in left-handers: fMRI evidence. Brain and Language, 122(3), 171–178.
Van der Haegen, L., Cai, Q., Seurinck, R., & Brysbaert, M. (2011). Further fMRI validation of
the visual half field technique as an indicator of language laterality: A large-group analysis.
Van der Haegen, L., Westerhausen, R., Hugdahl, K., & Brysbaert, M. (2013). Speech dominance
is a better predictor of functional brain asymmetry than handedness: A combined fMRI
word generation and behavioral dichotic listening study. Neuropsychologia, 51(1),  91–97.
Van Ettinger-Veenstra, H. M., Ragnehed, M., Hallgren, M., Karlsson, T., Landtblom, A. M.,
Lundberg, P., & Engstrom, M. (2010). Right-hemispheric brain activation correlates to lan-
guage performance. NeuroImage, 49(4), 3481–3488.
Vandermosten, M., Boets, B., Wouters, J., & Ghesquiere, P. (2012). A qualitative and quanti-
tative review of diffusion tensor imaging studies in reading and dyslexia. Neuroscience and
Biobehavioral Reviews, 36(6), 1532–1552.
Vernooij, M. W., Smits, M., Wielopolski, P. A., Houston, G. C., Krestin, G. P., & Van der Lugt,
A. (2007). Fiber density asymmetry of the arcuate fasciculus in relation to functional hemi-
spheric language lateralization in both right-and left-handed healthy subjects: A combined
fMRI and DTI study. NeuroImage, 35(3), 1064–1076.
Vigneau, M., Beaucousin, V., Hervé, P.-Y., Jobard, G., Petit, L., Crivello, F., . . . Tzourio-Mazoyer,
N. (2011). What is right-hemisphere contribution to phonological, lexico-semantic, and sen-
tence processing? Insights from a meta-analysis. NeuroImage, 54(1), 577–593.
Vingerhoets, G., Alderweireldt, A.-S., Vandemaele, P., Cai, Q., Van der Haegen, L., Brysbaert,
M., & Achten, E. (2013). Praxis and language are linked: Evidence from co-lateralization in
individuals, with atypical language dominance. Cortex, 49(1), 172–183.
Wada, J., & Rasmussen, T. (1960). Intracarotid injection of sodium amytal for the lateral-
ization of cerebral speech dominance: Experimental and clinical observations. Journal of
Neurosurgery, 17(2), 266–282.
Wada, J. A., Clarke, R., & Hamm, A. (1975). Cerebral hemispheric asymmetry in
humans: Cortical speech zones in 100 adult and 100 infant brains. Archives of Neurology,
32(4), 239–246.
Wernicke, C. (1874). De aphasische Symptomencomplex. Eine psychologische Studie auf
anatomische Basis. Breslau: Cohn & Weigert.
Wildgruber, D., Riecker, A., Hertrich, I., Erb, M., Grodd, W., Ethofer, T., & Ackermann, H.
(2005). Identification of emotional intonation evaluated by fMRI. NeuroImage, 24(4),
1233–1241.
Wilke, M., & Lidzba, K. (2007). LI-tool: A new toolbox to assess lateralization in functional
MR-data. Journal of Neuroscience Methods, 163(1), 128–136.
Willems, R. M., Peelen, M. V., & Hagoort, P. (2010). Cerebral lateralization of face-selective and
body-selective visual areas depends on handedness. Cerebral Cortex, 20(7), 1719–1725.
Willems, R. M., Van der Haegen, L., Fisher, S. E., & Francks, C. (2014). On the other
hand: Including left-handers in cognitive neuroscience and neurogenetics. Nature Reviews
Witteman, J., Goerlich-Dobre, K. S., Martens, S., Aleman, A., Van Heuven, V. J., & Schiller, N.
O. (2014). The nature of hemispheric specialization for prosody perception. Cognitive and
Affective Behavioral Neuroscience, 14(3), 1104–1114.
Woodhead, Z. V. J., Wise, R. J. S., Sereno, M., & Leech, R. (2011). Dissociation of sensitivity to
spatial frequency in word and face preferential areas of the fusiform gyrus. Cerebral Cortex,
21(10), 2307–2312.
Xiang, H., van Leeuwen, T. M., Dediu, D., Roberts, L., Norris, D. G., & Hagoort, P. (2015). L2-
proficiency-dependent laterality shift in structural connectivity of brain language pathways.
Brain Connectivity, 5(6), 349–361.
Xue, G., Chen, C., Jin, Z., & Dong, Q. (2006). Language experience shapes fusiform activa-
tion when processing a logographic artificial language: An fMRI training study. Neuroimage,
31(3), 1315–1326.
Yang, F. G., Edens, J., Simpson, C., & Krawczyk, D. C. (2009). Differences in task demands
influence the hemispheric lateralization and neural correlates of metaphor. Brain and
Language, 111(2), 114–124.
Yeatman, J. D., Rauschecker, A. M., & Wandell, B. A. (2013). Anatomy of the visual word
form area: Adjacent cortical circuits and long-range white matter connections. Brain and
Language, 125(2), 146–155.
Zago, L., Petit, L., Mellet, E., Jobard, G., Crivello, F., Joliot, M., . . . Tzourio-Mazoyer, N. (2016).
The association between hemispheric specialization for language production and for spatial
attention depends on left-hand preference strength. Neuropsychologia, 93(Pt B), 394–406.
Chapter 35
N eural Mec ha ni sms

of M u sic and L a ng uag e
Mattson Ogg and L. Robert Slevc
Introduction
Music and language share the distinction of being among the most uniquely human
propensities: both are universally present in cultures throughout the world (Berwick,
Friederici, Chomsky, & Bolhuis, 2013; Brown & Jordania, 2011; Hauser, Chomsky &
Fitch, 2002; Savage, Brown, Sakai, & Currie, 2015), and are acquired over stereotyped
trajectories in development (Hannon & Trainor, 2007; Kuhl, 2004; Trehub, 2003). These
parallels, as well as their similar functions as channels of acoustic communication, nat-
urally lead to the supposition that music and language share deeper neural-processing
substrates (Brandt, Gebrian, & Slevc, 2012; Heffner & Slevc, 2015; Koelsch, 2005; Patel,
2003; 2008; Schulze & Koelsch, 2012; Slevc & Okada, 2015). At the very least, they tread
similar ground as they ascend the auditory pathway.
These similarities aside, why should the study of neurolinguistics be concerned with
the processing of music? First, music and language rely on many of the same acoustic
computations. In other words, when processing signals in either domain, the brain
must solve similar problems. This includes the perception of pitch, spectrotemporal
modulation cues, temporal regularities, sound source identification, and “streaming”
or tracking an acoustic target through an auditory scene (Bregman, 1990; McAdams &
Bregman, 1979; see Poeppel, Cogan, Davidesco, & Flinker, Chapter 26 in this volume).
Some of these phenomena are integral and highly developed in music processing.
A better understanding of how the brain solves these problems for music can in turn il-
luminate how they are carried out for language.
This leads directly to the second way in which neurolinguistics can benefit from
the study of music: musical traditions rely on conceptual and hierarchical cognitive
representations similar to language. These representations have been especially well
characterized in Western music (Krumhansl & Kessler, 1982) and, if studied alongside
908 Mattson Ogg and L. Robert Slevc
speech and language, perhaps provide a tractable and testable model of how high-level
representations are maintained and interfaced with during communication. These
representations even lend themselves to hierarchical syntactic (Lerdahl & Jackendoff,
1983; Longuet-Higgins, 1976; Rohrmeier, 2011) and prosodic (Heffner & Slevc, 2015;
Lerdahl & Jackendoff, 1983) linguistic analyses.
A third manner in which neurolinguistics could profit from an improved under-
standing of music processing is that music can help us explore and better understand
how the listener builds meaning in auditory sequences over time. Exploring the tem-
poral dynamics of how information is coherently integrated as it unfolds requires an
understanding of the higher and lower limits of the various cognitive and percep-
tual processes involved. Music can be a powerful tool in exploring these processes.
Examples of this have already been demonstrated in a set of studies examining
neural oscillators and their close relation to hierarchical structure building
(Doelling & Poeppel, 2015; Ding, Melloni, Zhang, Tian, & Poeppel 2015) and in au-
ditory stream segregation (Bregman, 1990; Bregman & Campbell, 1971; McAdams &
Bregman, 1979).
Examining music and language together can also shed light on more general domains
of cognition and perception. For example, human listeners have a remarkable ability to
focus on a single target sound, such as a voice or instrumental line, despite interference
from other voices in a crowded room or symphony. Better understanding this “cock-
tail party problem” (Cherry, 1953) or auditory “streaming” (Bregman, 1990; McAdams
& Bregman, 1979) can further our understanding of attention, object representations,
and feature binding beyond audition. Similarly, understanding musical patterns
and traditions in terms of their statistical availability in speech signals (Ross, Choi, &
Purves, 2007; Schwartz, Howe, & Purves, 2003) can be used to more broadly under-
stand probabilistic models of neuronal connections and their relation to higher-level
perceptual tendencies. An excellent example of this comes from research on under-
standing musical consonance (Bowling & Purves, 2015). Furthermore, examining the
mechanisms by which learners develop an understanding for statistical patterns in tem-
poral domains such as music (Loui, Wu, Wessel, & Knight, 2009) and speech (Saffran,
Newport, & Aslin, 1996; Saffran, Newport, Aslin, Tunick, & Barrueco, 1997) can inform
our understanding of statistical learning more broadly (Zhao, Ngo, McKendrick, &
Turk-Browne,  2011).
Finally, the combined study of music and language can inform our understanding of
the evolution of interpersonal communication. The topic of the evolutionary interplay
between music and language has a long, rich history dating back (at least) to Darwin
(1871). A discussion of these issues would warrant its own chapter (Cross, 2007) or book
(Mithen, 2005), and will not be addressed in detail here, except to point out how useful
the framing of a comparative perspective has proven to be to our understanding of the
neural processing of acoustic communicative sounds (Hauser et al., 2002; McDermott
& Hauser, 2005). Accordingly, findings from animal work will be described where ap-
propriate to illuminate common neural mechanisms involved in music and language
processing that might otherwise be difficult to assess in humans.
Neural Mechanisms of Music and Language 909
The structure of this chapter will follow the neural processing of music and speech
as acoustic signals ascending the auditory pathway. Figure 35.1 outlines this processing
stream and provides a visual indication of what neural structures are involved. We will
describe findings regarding how music and speech are relayed, coded, or processed
starting first in the brainstem, then in early auditory processing areas in cortex, followed
finally by higher-level cortical regions involved in more abstract cognitive opera-
tions. At each step in the processing pathway, we will describe how the auditory system
carries out operations common to both types of signals, as well as how the different
demands imposed by music or speech are managed by the brain. In doing so, it will
often be useful to describe music and speech in terms of both their common and unique
acoustic features or characteristics, a perspective that may be helpful for research
in this area generally (Giordano, Pernet, Charest, Belizaire, Zatorre, & Belin, 2013;
Giordano, McAdams, Zatorre, Kriegeskorte, & Belin, 2012; Samson, Zeffiro, Toussaint
& Belin, 2011).
Attention to the unique, acoustic-level processing demands of music and speech will
also illuminate how experience in one domain (e.g., musical training, multilingualism)
can influence neural processes involved in the other. Plasticity in the auditory pathway
resulting from long-term, top-down demands will be described in view of the OPERA
hypothesis (which stands for Overlap, Precision, Emotion, Repetition, and Attention,
Patel, 2011, 2012, 2014), which speculates that increased precision, attention, repetition,
and motivation from one domain can influence overlapping neural processes that sub-
serve both domains. Highlighting instances of cross-domain plasticity provides strong
evidence that the neural structures involved (and pressed upon by a particular domain,
like musical training, for example) are common to processing in both domains.
Brainstem Auditory Cortex Cortical Processing
Tonotopy and Periodotopy3 Object Processing4,5,6

Phase-locked stimulus
tracking1,2 Semantics5,6
Pitch3
Refinement following
long-term ecperience1,2 Attentional Modulation3 Syntax/Structure5,6,7
Localization1 Low-Level Feature Extraction3 Rhythm8,9,10,11
Figure 35.1. An overview of the processes involved in the perception of music and language
and the neural structures that support these functions. Numbers approximately correspond to
the auditory and cortical processing hierarchy: (1) olivary complex, (2) inferior colliculus, (3) au-
ditory cortex: Heschl’s gyrus/superior temproal cortex, (4) planum temporale, (5) anterior supe-
rior temporal cortex, (6) temporal pole, (7) Broca’s area (left hemisphere), (8) pre-motor cortex,
(9) supplementary motor area, (10) cerebellum, and (11) basal ganglia.
Brainstem
Speech and music enter the ear as acoustic signals and create vibrations in the eardrum,
which are transmitted over the ossicles to the oval window and create disturbances in
the fluid that fills the cochlea. These disturbances are transduced into electrical signals
by hair cells in the basilar membrane and are then transmitted to the brain via the audi-
tory nerves. Incoming signals then pass through nuclei in the brainstem on their way to
auditory cortex (see Figure 35.1). Activity within these brainstem nuclei exhibits excep-
tional temporal resolution. This allows for the critical sound localization computations
of inter-aural time and level differences to be performed (Carr & Konishi, 1988). These
occur within the medial superior olive and lateral superior olive, respectively (Grothe,
2003). Localization cues rely on sub-millisecond differences in timing (Grothe, 2003;
Schnupp & Carr, 2009), and are crucial for auditory streaming (Cherry, 1953).
The robust temporal resolution of the brainstem is realized by temporally phase-
locked patterns of neural firing that match the acoustic patterns of incoming stimuli
(Smith, Marsh, & Brown, 1975). This phase-locked activity, after a brief transient period
following sound onset, can track acoustic patterns up to 1,000 Hz, which covers a va-
riety of key auditory features such as pitch, harmonic information, and some formants.
Such cues are crucial to the perception of speech and music, making the brainstem
and subcortical responses an especially interesting area of study. Neural activity in the
brainstem (specifically, the auditory brainstem response) can be recorded externally by
electrodes placed on the scalp (see Leckey & Federmeier, Chapter 3 in this volume), and
provides a powerful clinical and research tool (Chandrasekaran, Hornickel, Skoe, Nicol,
& Kraus, 2009; Chandrasekaran & Kraus, 2010; Kraus & Chandrasekaran, 2010). The
exact neural source of the response recorded at the scalp is not entirely clear, but findings
based on timing and temperature ablation indicate that the inferior colliculus (Smith
et al., 1975) or superior olive (Hoormann, Falkenstein, Hohnsbein, & Blanke, 1992) may
be the principal generators of this signal. However, cortical generation sites have also
been suggested by findings in magnetoencephalography (MEG) (Coffey, Herholz,
Chepesiuk, Baillet, & Zatorre, 2016; see Salmelin, Kujala, & Liljeström, Chapter 6 in
this volume). Nonetheless, electrical correlates of brainstem activity recorded at the
scalp provide an amazingly detailed representation of what is being heard—so much
so that the amplified playback of (averaged) electroencephalograph (EEG) recordings
of responses to speech are intelligible to new listeners (Galbraith, Arbagey, Branski,
Comerci, & Rector, 1995).
In recent years, it has become apparent that these brainstem responses are plastic
and sensitive to both musical and linguistic long-term experience. For example, the
brainstem recordings of speakers of tone languages where pitch differences in speech
are an important phonemic cue (in this case, Mandarin) show more robust encoding of
pitch information as well as the second harmonic when listening to tonal syllables than
speakers of non-tone languages (Krishnan, Xu, Gandour, & Cariani, 2005). Musicians
who speak non-tone languages show similarly enhanced brainstem encoding of pitch
(Musacchia, Sams, Skoe, & Kraus, 2007) not only when listening to musical tones, but
also to syllables from tone languages (Wong, Skoe, Russo, Dees, & Kraus, 2007). The
enhanced brainstem responses of musicians are especially strong when listening to the
timbre of their primary instrument (Strait, Chan, Ashley, & Kraus, 2012) and to emo-
tive vocal expressions (Strait, Kraus, Skoe, & Ashley, 2009). In general, tone-language
speakers and musicians appear to have similar advantages in brainstem response fi-
delity, compared to non-musician, non-tone-language speakers (Bidelman, Gandour,
& Krishnan, 2010). However, there is evidence for some domain-dependent differences
between tone-language and musician groups. For example, musicians show enhanced
brainstem pitch tracking compared to tone-language speakers when tones in the stimuli
correspond to Western musical pitches (see also Bidelman, Gandour, & Krishnan, 2011).
This enhanced brainstem response fidelity is related to behavioral and perceptual
benefits as well. Analyses of the spectral properties of musicians’ brainstem responses
to triads (Bidelman & Krishnan, 2011) and musical intervals (Bidelman & Krishnan,
2009) reveals stronger pitch information than dissonant intervals, which correlates
very highly with behavioral judgments of consonance and dissonance. In other words,
judgments of consonance follow the standard hierarchy of Western musical theory,
with small interval pitch relations being judged as more consonant. These consonant
stimuli are also represented more robustly than dissonant ones in brainstem responses.
Better speech in noise perception has also been linked to more accurate timing of
brainstem responses, with poorer speech in noise perception being linked to longer
delays in neural responses, particularly around formant transitions (Anderson, Skoe,
Chandrasekaran, & Kraus, 2010). Similarly, more consistently isochronous tapping is
associated with better phase locking in auditory brainstem responses (Tierney & Kraus,
2013), and, among preschoolers, is also associated with better early language abilities
(Woodruff Carr, White-Schwoch, Tierney, Strait, & Kraus, 2014). Given that musical
training can increase the fidelity of brainstem responses, it should be no surprise that
the brainstem fidelity benefits of musical training correlate with improved speech in
noise perception (Parbery-Clark, Skoe, & Kraus, 2009; Parbery-Clark, Tierney, Strait, &
Kraus, 2012) and hearing abilities in other challenging settings, such as in highly rever-
berant conditions (Bidelman & Krishnan, 2010).
Perhaps less obviously, brainstem response fidelity and speech in noise abilities are
also associated with reading ability in children, although potentially in different ways.
For example, one study found that models of different brainstem response measures
accounted for a large amount of the variability in either reading or speech in noise per-
formance among school-age children. However, the models of brainstem response
measures that predicted performance on one outcome measure (reading or speech)
did not generalize well when fit to the other outcome measure. This suggests that non-
overlapping components of brainstem responses might be associated with reading
and speech in noise performance, respectively (Hornickel, Chandrasekaran, Zecker,
& Kraus, 2011). Hornickel and colleagues (Hornickel, Anderson, Skoe, Yi, & Kraus,
2012) also suggest that the increased accuracy of brainstem responses and hearing ability
aid in children’s mapping of acoustic input to orthographic features when learning to

read. Encouragingly, when we move beyond correlational findings, musical training has
also been shown to improve brainstem response fidelity and speech in noise perception
in a randomized trial among school-age children (Kraus, Slater, Thompson, Hornickel,
Strait, Nicol, & White-Schwoch 2014) and among adolescents who self-selected musical
training compared with a control group who also received extra-curricular training that
was not musical (Tierney, Krizman, & Kraus, 2015).
On the other end of the age spectrum, the neural coding of acoustic signals at the
brainstem declines as listeners grow older (Anderson, Parbery-Clark, White-Schwoch,
& Kraus 2012), and this decline is associated with poorer perception of speech in noise
(Anderson, White-Schwoch, Parbery-Clark, & Kraus, 2013). However, musical training
can offset these declines in neural acuity and predicts performance on certain per-
ceptual tasks such as categorical vowel perception (Bidelman & Alain, 2015). Taken
together, these findings indicate that brainstem responses are very sensitive to both mu-
sical and linguistic experience, and align with explanations offered by the OPERA hypo-
thesis (Patel, 2011, 2012, 2014) in that experience in either domain might affect common
circuits through repetition of tasks requiring higher auditory acuity. Enhanced brain-
stem responses to acoustic signals also may be associated with many positive learning
and quality-of-life outcomes. Thus, musical training may be a useful tool for accessing
these advantages. These benefits are thought to result from top-down cortical modu-
lation that interacts with activity in inferior colliculus via corticofugal pathways while
performing these tasks (Chandrasekaran & Kraus, 2010; Tzounopoulos & Kraus,
2009). Notably, these cross-domain benefits would not be possible if music and speech
did not share overlapping neural circuitry and were amenable to plasticity (Patel, 2011,
2012, 2014).
Early Cortical
Processing: Auditory Cortex
Early auditory cortical areas receive input from the brainstem (see Figure 35.1) and con-
tinue to maintain a detailed representation of the input stimulus (Mesgarani & Chang,
2012; Pasley et al., 2012), however, subsequent processing follows a hierarchical struc-
ture, with downstream functions exhibiting more complexity and selectivity (DeWitt
& Rauschecker, 2012; Okada et al., 2010; Peelle, Johnsrude, & Davis, 2010). Overall, this
approximates a dual-stream, ventral and dorsal, “what” and “where”/“how” organiza-
tion (Rauschecker & Scott, 2009; Rauschecker & Tian, 2000), similar to vision (Mishkin,
Ungerleider, & Macko, 1983). Even though music and language are expressive systems
unique to humans, they both rely on basic processes carried out in auditory cortex. These
processes are fundamental to extracting features from sounds and thus contribute to
later, more complex cognitive functions. Additionally, many auditory cortical functions,
particularly in more primary areas, are preserved among mammals (Theunissen & Elie,
2014; Tsunada & Cohen, 2014). Thus, the crucial insights gained from the measure-
ment of neuronal populations in various species, including primates (Hackett, Preuss, &
Kaas, 2001; Kaas & Hackett, 2000), ferrets (Chi, Ru, & Shamma, 2005), and guinea pigs
(Occelli, Suied, Pressnitzer, Edeline, & Gourevitch, 2015), provide a unique window into
the computations being performed at the neural level that subserve the more intricate
operations that humans complete when listening to speech and music.
Mapping Primary and Nonprimary Auditory

Cortex: Tonotopy and Periodotopy
The first cortical destination for speech, music, and all acoustic signals in humans is
primary auditory cortex, which is generally thought to be located in Heschl’s gyrus, on
the superior plane of the temporal cortex, tucked behind where the temporal and pa-
rietal cortices meet (Zatorre, Belin, & Penhune, 2002). Tonotopic organization places
neurons that respond to similar frequencies close to each other in cortical geography
and thus represents the harmonic and formant structure of speech and musical sounds
in a straightforward manner. This maximizes spatial organization and increases the ef-
ficiency of downstream processing. The sensitivity of the different frequency responses
along this tonotopic axis is especially influenced by the acoustic characteristics of
speech (Moerel, De Martino, & Formisano, 2012). Primary auditory cortex is gener-
ally defined by its tonotopic organization in response to simple tones and reversals of
this tonotopic gradient (low, high, low, etc.). However, considerable debate exists re-
garding the exact location and boundaries of primary and nonprimary auditory areas
in humans (Da Costa et al., 2011). Da Costa and colleagues (2011) used high-resolution
functional magnetic resonance imaging (fMRI) (see Heim & Specht, Chapter 4 in this
volume) to record functional activity while participants heard a series of low to high
tones. The resulting patterns of activation outlined high-to-low-to-high mirror sym-
metric tonotopic gradients that ran perpendicular to Heschl’s gyrus in every par-
ticipant, with the low portion of the response being at the apex of the gyrus and the
higher frequency sensitive areas receding into the sulci on each side. Similar work in
fMRI has also implicated Heschl’s gyrus, although some specifics, such as the direction
of the tonotopic gradients, vary between studies (Formisano et al., 2003; Humphries,
Liebenthal, & Binder, 2010). While the certainty of tonotopic organization becomes less
clear at the neural level (Bandyopadhyay, Shamma, & Kanold, 2010; Kanold, Nelken,
& Polley, 2014), it has nonetheless been a widely used heuristic for mapping human
auditory cortex (Da Costa et al., 2011; Formisano et al., 2003; Humphries et al., 2010),
localizing basic auditory functions in human fMRI studies (Lewis et al., 2009; Norman-
Haignere, Kanwisher, & McDermott, 2013, 2015; Talkington, Rapuano, Hitt, Frum, &
Lewis, 2012; Wessinger et al., 2001), and identifying different auditory fields in primates
(Romanski et al., 1999; Rauschecker, Tian, & Hauser, 1995).
Additional gradient patterns as well as other weaker tonotopic areas that have been
observed near Heschl’s gyrus (Formisano et al., 2003; Humphries et al., 2010; Moerel
et al., 2012; Wessinger et al., 2001) are very similar to secondary auditory areas meas-
ured in primates (Hackett, Preuss, & Kaas, 2001; Kaas & Hackett, 2000) and have been
identified as sites whose projections might initiate subsequent processing along the
“what” and “where” pathways (Formisano et al., 2003; Kass & Hackett, 1999; Moerel
et al., 2012; Romanski et al., 1999; Wessinger et al., 2001). Neighboring “belt” areas in
primates respond to tones, but are more responsive to complex stimuli and attributes
such as band-passed noise (Romanski et al., 1999) as well as conspecific vocalizations
anteriorly (Rauschecker et al., 1995) and location more posteriorly (Tian et al., 1997;
Rauschecker & Scott, 2009). An investigation of this dual-stream “what” and “where”
processing structure in humans used MRI-guided transcranial magnetic stimulation
(TMS; see Schuhmann, Chapter 5 in this volume) to selectively inhibit anterior or poste-
rior auditory cortex (Ahveninen et al., 2013). A double dissociation was found whereby
TMS pulses disrupting posterior auditory cortex impaired sound localization perfor-
mance, while anterior TMS disruption impaired identification, suggesting that this
dual-stream specialization is maintained in humans.
Other aspects of sound also appear to be represented by a highly structured orga-
nization throughout primary and secondary auditory cortex. These include a mapping
of temporal modulation rates (or periodotopy) perpendicular to frequency modula-
tion gradients (Barton, Venezia, Saberi, Hickok, & Brewer 2012). Although the orga-
nization of auditory cortex and these temporal modulation gradients may be driven by
the organization of tonotopic frequency gradients (Leaver & Rauschecker, 2016), the
overlapping organization of frequency and temporal modulation bands together sup-
port findings that delineate auditory cortical function in terms of joint spectrotemporal
dynamics (Hullett, Hamilton, Mesgarani, Schreiner, & Chang, 2016; Santoro et al., 2014;
Santoro et al., 2017; Schönwiesner & Zatorre, 2009). This, in turn, might support more
complex operations relevant to the perception of speech (Elliot & Theunissen, 2009;
Holdgraf et al., 2016) and music (Elliot, Hamilton, & Theunissen, 2013; Theunissen &
Elie, 2014).
Adaptive Neural Function: Attending to Goals

and Targets
In addition to responding to tonotopic, periodotopic, and spectrotemporal informa-
tion, early auditory areas modulate their activity based on attentive processing and be-
havioral goals (Fritz, Elhilali, David, & Shamma, 2007). These attentive modulations
support the listener’s ability to focus on speech or music in busy auditory scenes, such as
in conversations at a cocktail party. Fritz and colleagues found that stimulus-dependent
responses to rewarding targets in an operant task modulated receptive fields of neurons
in primary auditory cortex toward the features of the rewarding stimuli, and suppressed
activity for nearby, non-target acoustic bands (Fritz, Elhilali, & Shamma, 2005; Fritz,
Shamma, Elhilali, & Klein, 2003). This pattern of enhancement and suppression of re-
sponsiveness in these early areas as a function of task-related goals is thought to sharpen
attentiveness to desired stimulus features. In a later study that measured neurons in
both primary auditory cortex and frontal cortex, modulations in auditory cortex were
found to be driven by top-down activity from frontal neurons, which rapidly gated their
responses depending on the demands of the task (Fritz, David, Radtke-Schuller, Yin,
& Shamma, 2010). For example, these frontal neurons showed no preference for a tone
stimulus in passive listening before the task, but during task performance these neurons
modified their activity in response to the same tone when it was now a target and rele-
vant to behavioral goals.
Moving up the processing hierarchy from the simpler receptive fields of primary au-
ditory cortex, attention-modulated stimulus tracking is also observed in nonprimary
auditory areas. Tracking a target speaker in the presence of competitor speech is a dif-
ficult problem since the signals are intertwined when they reach the ear. Nonetheless,
human listeners are able to attend to a given speaker quite easily (Cherry, 1953). A typ-
ical paradigm for investigating this phenomenon involves presenting a participant with
two streams of speech, and asking him or her to attend or respond to one target, while
ignoring the other. Analogous operations are thought to occur when a listener tracks a
musical part in a multi-instrumental composition or auditory scene (Bizley & Cohen,
2013; Bregman 1990). Neural correlates of an attended speaker, even in the presence of
the competitor, can be reconstructed from neuronal signals in nonprimary auditory
areas. Ding and Simon (2012) demonstrated that the temporal envelope of the speech
of an attended speaker can be reconstructed from an MEG signal when two overlapping
speech signals are delivered to the listener. This representation also demonstrated in-
variance, as it was not affected by intensity changes to the competitor, a critical feature
of auditory objects (Griffiths & Warren, 2004). Furthermore, the fidelity of this repre-
sentation was found to be a significant predictor of perceptual performance (Ding &
Simon, 2013).
Similar attentionally mediated reconstructions have also been generated from
electrocorticography (ECoG) recordings in humans undergoing treatment for epilepsy,
using neural activity within the high-gamma range (75–150 Hz). One study found that
the neuronal reconstructions maintained features of an attended, but not an unattended,
speaker, and that these reconstructions were related to performance on a comprehen-
sion task (Mesgarani & Chang, 2012). Later work showed that attentional modulation
of neural responses is present in earlier auditory areas, showing increased responses to
targets relative to distractor speakers, while later stages of processing (mostly beyond
auditory cortex) contained only information for the attended speaker, with no trace of
the distractor speaker (Zion Golumbic et al., 2013). These reconstruction techniques
provide a powerful demonstration of how attention can modulate sensory input, and
can adjust the neural representations of complex stimuli in difficult situations at various
levels of processing.
Related ECoG findings during music listening suggest that similar mechanisms may
be at play. Potes, Gunduz, Brunner, and Schalk (2012) demonstrated that high-gamma
activity in posterior STG electrodes tracked the amplitude envelope of musical stimuli,
and that other acoustic and timbral features were characterized by electrodes in tem-
poral cortex in a manner similar to speech (Strum, Blankertz, Potes, Schalk, & Curio,
2014). However, these results are based on a single musical work, and the designs did not
include attentional modulations or stimulus reconstruction. Thus, it remains to be seen
how these patterns of activity during music listening might be modulated by additional
cognitive demands. Lyrical portions of the musical work did create the largest response
during music listening, suggesting that the lead melody might have primacy in these
cortical patterns (Strum et al., 2014). However, without an additional melodic control,
this pattern could also be due to a possible primacy of speech processing in the left hem-
isphere or increased signal density from the addition of vocal parts. It will be exciting to
see how this line of investigation develops, and taken with the attentionally modulated
findings in speech processing, these timbral and acoustically driven findings during
multi-part musical compositions may present an ideal arena for further examining the
effects of attention modulated cortical activity in MEG, EEG, or ECoG.
Pitch
In addition to object tracking and streaming, music and speech both rely heavily on
the perception of pitch. In music, pitch is central to establishing tonal contexts and
composing melodic and harmonic relations. In speech, pitch is vital for intonation and
prosodic cues, and is critical for differentiating phonemes in tonal languages. Pitch and
spectrotemporal coherence are also fundamental cues used in stream segregation, which
is important in both speech and music (Pressnitzer, Suied, & Shamma, 2011; Shamma,
Elhilali, & Micheyl, 2011). However, the perception of pitch does not align precisely
with fundamental frequency (or a tone’s lowest harmonic), and this can complicate
investigations of pitch processing: Does a neural response reflect tonotopic activation,
or the perception of pitch (Norman-Haignere et al., 2013)? Thus, the investigation of the
neural substrates of pitch requires creative experimental designs and stimuli.
One powerful tool has been the use of the missing fundamental phenomenon,
wherein the fundamental frequency of a harmonically complex tone can be removed
completely, while leaving the rest of the tone’s harmonic structure intact. Despite the
loss of the fundamental frequency, the perception of the tone’s pitch is not impaired due
to the regularity of the tone’s harmonics, which correspond to integer multiples of the
tone’s fundamental (Von Békésy, 1972). This is quite a common phenomenon in tele-
phone communication (as the telephone network is band-limited, only transmitting
frequencies between ~300 Hz and 3,400 Hz) or when listening to music through small
speakers. The output of such devices does not extend to the lowest frequencies of some
voices or instruments where the fundamental resides. The ability to perceive pitch de-
spite the absence of the fundamental has also been observed in cats (Heffner & Whitfield,
1976), primates (Tomlinson, & Schwarz, 1988), and infants (Clark & Clifton, 1985).
Using this to their advantage, Bendor and Wang (2005) recorded neuronal responses
from primates while they played tones with and without a fundamental frequency. A set
of neurons on the anterior edge of primary auditory cortex responded to both stimuli,
meaning they were responsive to pitch over and above tonotopic responsiveness.
The findings in primates align closely with fMRI evidence in humans. Studies that
manipulate pitch chroma (the notes of a piano within an octave) independently of
pitch height (the location of the octave on a piano; Warren, Uppenkamp, Patterson, &
Griffiths, 2003) find pitch-responsive activation in anterolateral Heschl’s gyrus. A sim-
ilar pattern of activity is found in studies that are able to manipulate pitch salience via
resolved and unresolved harmonics (Norman-Haignere et al., 2013; Penagos, Melcher,
& Oxenham, 2004), and studies that employ varying levels of harmonic structure in
pitches created by regular-interval noise complexes (Patterson, Uppenkamp, Johnsrude,
& Griffiths, 2002). Similar regular-interval noise stimuli have been used in a study of
direct neural recordings in human epilepsy patients. This study found that neuronal
populations tracked stimulus regularity in the noise throughout Heschl’s gyrus, but
high-gamma oscillatory activity, a hallmark of neural activity related to attention, only
emerged when the regularity of the stimuli was within the range that pitches can be
perceived in humans (above approximately 30 Hz; Griffiths et al., 2010). However, this
study found high-gamma activity to be maximal in an area of Heschl’s gyrus more me-
dial to the lateral areas identified in fMRI, indicating that more research is needed to
bridge these bodies of work.
A special case in human pitch perception exists that may shed additional light on
these neural processes and differences in the processing requirements of speech and
music. Deutsch, Henthorn, and Lapidis (2011) reported a phenomenon wherein regu-
larly spoken sentences appeared to transform into more melodic sequences, similar to
music, with repeated listening. This has been called the speech to song illusion (Deutsch
et al., 2011). While the acoustic and psychological mechanisms that allow a speech ut-
terance to transform into a melodic utterance are still under debate (Falk, Rathcke, &
Dalla Bella, 2014; Vanden Bosch der Nederlanden, Hannon, & Snyder, 2015a), this il-
lusion provides a fascinating opportunity to examine parallels between music and
speech processing (Vanden Bosch der Nederlanden, Hannon, & Snyder, 2015b). One
fMRI study (Tierney, Dick, Deutsch, & Sereno, 2012) measured functional changes
in participants while listening to stimuli that did and did not induce this illusion
(transforming from speech to song) and found that when the sentence had “converted”
from speech to music, various neural areas involved in music and melodic processing
were activated relative to when it was perceived as speech. Notably, this included the
same lateral area of Heschl’s gyrus described in previous fMRI studies that responded
specifically during the perception of pitch. That this area was activated more when the
perception of pitch and melodic information was emphasized, despite the consistency
of the incoming acoustic information, is powerful evidence that activation in this area
is linked to the psychological experience of pitch, rather than other acoustic stimulus
features.
The Influence of Experience on the Structure of Early

Auditory Processing Areas
Could producing or paying attention to different kinds of stimuli, like speech or music,
over long periods of time be associated with particular structural changes in auditory
cortices? Such a result would align with the view of the OPERA hypothesis (Patel, 2011,
2012, 2014), and a variety of findings suggest that there are associations between brain
structure and listening abilities. Golestani and colleagues (2007) found that participants
who learned the speech sounds of a new language the fastest also had greater white-
matter density in left Heschl’s gyrus and were five times more likely to have an anatom-
ical split or duplication of this area. This pattern was echoed by Wong and colleagues
(2007), who found a similar relationship between left Heschl’s gyrus volume (but in gray
rather than white matter) and learning linguistic cues that incorporated pitch. This was
further explored in a later study that examined expert phoneticians (Golestani, Price,
& Scott, 2011): expert phoneticians, compared with controls, had larger transverse gyri
and were more likely to have split gyri. However, these patterns in auditory cortices were
not associated with years of transcription training, leading the authors to hypothesize
that these neuroanatomical differences created an aptitude for phonetic processing in
these individuals that preceded their phonetic training.
Anatomical differences in Heschl’s gyrus have also been found in musicians relative
to non-musicians (Gaser & Schlaug, 2003). However, Bermudez and Zatorre (2005)
found anatomical differences in musicians relative to controls in areas posterior and an-
terior to Heschl’s gyrus. Nevertheless, structural differences in Heschl’s gyrus translate
into superior music perception abilities and stronger neural responses to tones early in
processing (Schneider et al., 2002). These findings were replicated in a later study, which
also observed that morphological differences associated with musical experience were
not associated with an inclination to resolve pitch differences based on fundamental
frequency or spectral cues to pitch (Schneider et al., 2005). While not related to mu-
sical training (but see Seither-Preisler et al., 2007), a participant’s inclination to resolve
pitch based on fundamental frequency or spectral cues was found to be associated with
greater volumes of left and right Heschl’s gyrus, respectively (Schneider et al., 2005).
Warrier and colleagues (2009) went a step further by examining how individual varia-
tion in the structure of Heschl’s gyrus related to individual differences in the functional
processing of temporal or spectral complexity. They found that individual differences in
Heschl’s gyrus anatomy were highly correlated with individual differences in the geo-
graphical extent of functional neural responses to temporal (left hemisphere) or spectral
(right hemisphere) processing.
Structural and functional changes related to musical training or language experience
can also be observed beyond primary areas. Moreover, plastic changes following mu-
sical training or exposure may also be induced very early in development. Zhao and
Kuhl (2016) found that 9-month-old infants exposed to triple-meter rhythmic patterns
(via random assignment) had greater functional response sensitivity in frontal and
auditory areas to rhythmic deviations in both speech and music compared to controls.
Motor areas, in particular, seem to be related to proclivities in music and language pro-
duction. Perhaps the most salient example is the relationship between handedness
and language dominance of the contralateral hemisphere (see Van der Haegen & Cai,
Chapter 34 in this volume). Knecht and colleagues (2000) demonstrated a parametric
relationship between the strength of an individual’s handedness and dominance of the
contralateral hemisphere in a word-production task among left-handed and ambi-
dextrous participants. Musical training also appears to increase the size of functional
responses in motor cortex specific to a musician’s main instrument (Elbert, Pantev,
Wienbruch, Rockstroh, & Taub, 1995) and also results in stronger responses in tonotopic
areas to harmonically rich tones compared to controls (Pantev, Oostenveld, Engelien,
Ross, Roberts, & Hoke, 1998). Structural changes are also found in motor areas and the
corpus callosum following as little as 15 months of training (Gaser & Schlaug, 2003;
Hyde et al., 2009). These changes, particularly white-matter changes in corpus callosum,
may also be subject to a critical period prior to 7 years of age (Steele, Bailey, Zatorre, &
Penhune, 2013).
It should be noted that in all of the previously described findings it is difficult to disen-
tangle effects of experience, such as musical training or linguistic exposure, from those
of predispositions to a certain vocation or activity that are enabled by abilities derived
from neural anatomy. Both likely play a role; for example, Golestani and colleagues
(2011) hypothesized that left Heschl’s anatomy preceded and influenced phoneticians’
abilities, rather than stemming from experiential effects. On the other hand, Golestani
and colleagues (2011) also found that the surface area of the left pars opercularis (the an-
terior part of Broca’s area) in phoneticians was related to their amount of phonetic tran-
scription training, fitting with other work suggesting that anatomical differences reflect
experience-dependent plasticity (e.g., Herholz & Zatorre, 2012; Münte, Altenmüller,
& Jäncke, 2002). This could be due in part to evidence for positive changes in auditory
cortices and abilities following even a relatively short (15-week) training regimen (Hyde
et al., 2009). It may be that “nature” and “nurture” are both at play in speech and language
development, as well as anatomical plasticity, but that experience affects plasticity differ-
ently for music and language processing. In any case, changes observed within Heschl’s
gyrus and other parts of auditory cortex tend to be associated with benefits to lower-level
discriminative abilities and do not necessarily translate into the higher-level cognitive
benefits that musical experience may (e.g., Slevc, Davey, Buschkuehl, & Jaeggi, 2016) or
may not (e.g., Bigand & Poulin-Charronnat, 2006; Schellenberg, 2015) engender.
Acoustic Feature Extraction: Instruments, Speakers,

and Phonemes
Many of the in-depth fMRI studies that seek to identify the neural processing substrates
of music and language in humans find overlapping activation in early auditory cortices.
These activation patterns are typically thought to be involved in low-level acoustic fea-
ture extraction, and the focus in these studies turns instead to higher cortical areas
specialized for processing one domain specifically (Leaver & Rauschecker, 2010;
Norman-Haignere et al., 2015). However, we know from animal work and some fMRI
work that a wide variety of interesting acoustic computations are going on at these early
stages, which might serve to sort or direct incoming signals for later analyses by more
specialized higher-level areas (Bizley & Cohen, 2013; Theunissen & Elie, 2014; Tsunada
& Cohen, 2014). Indeed, these acoustic components are bound to early perceptual units
such as phonemes or notes. We have already touched on some of these components,
such as location information, pitch, and how the perception and processing of these
features may be modulated by attention. But, at this early stage, the brain is also trying to
decode the source or identity of a given sound, and similarly, to sort incoming informa-
tion into more manageable subcomponents like auditory objects.
Sound sources in music are referred to as timbres. Timbre encompasses the acoustic
properties or features that distinguish the sounds of different instruments from one an-
other while pitch and loudness are kept equal. Timbre typically refers to a combination
of spectrotemporal attributes or dimensions that distinguish a set of instruments (Elliot,
Hamilton, & Theunissen, 2013; McAdams & Giordano, 2010; McAdams, Winsberg,
Donnadieu, De Soete, & Krimphoff, 1995). Timbre analogs in the speech domain are
twofold. First, when viewed in terms of unique spectrotemporal cues, timbre is often
operationally defined by different vowels (Town, Atilgan, Wood, & Bizley, 2013), whose
first and second formant frequencies create characteristic peaks in the spectral pattern
of a sound token (Hillenbrand, Getty, Clark, & Wheeler, 1995). Second, timbre in speech
could refer to the characteristic, identifiable features of an individual speaker’s voice.
Understanding how individuals are identified by voice presents an empirical problem
that, like musical timbre, is not completely understood but is defined by a multidimen-
sional set of relevant acoustic cues (Creel & Bregman, 2011; Elliot & Theunissen, 2009;
Schweinberger et al., 2014). Much of the complex processing required to extract and in-
terpret these acoustic cues begins in auditory cortex. Indeed, this information must al-
ready be accessible at early auditory stages to tune attention to certain speaker targets in
the studies on attentive processing and streaming mentioned previously (Mesgarani &
Chang, 2012; Pasley et al., 2012; Zion Golumbic et al., 2013). Also, timbre and pitch both
rely on similar spectral cues and can interact with one another (Krumhansl & Iverson,
1992; Melara & Marks, 1990). Thus, where does timbral perception fit into early auditory
decoding?
A good deal of information pertinent to the source of a sound can be decoded from
primary auditory cortex, including areas shared with other mammals. Representations
of instrumental timbres based on a computational model of ferret primary cortex
can provide enough information for a machine-learning algorithm to identify an ex-
tremely large set of instruments with over 98% accuracy, and where this algorithm
fails, it confuses stimuli in a manner similar to humans (Patil, Pressnitzer, Shamma,
& Elhilali, 2012). Similarly, using in vivo recordings from ferret auditory cortex while
listening to a speech corpus, Mesgarani and colleagues (2008) found that responses
from subpopulations of neurons matched the spectrotemporal features of vowels and

consonants. These populations also contained mappings of dimensions related to for-
mant frequencies in vowels or place and manner of articulation in consonants (which,
like vowels, are differentiated by unique spectrotemporal features). Again, a pattern
classifier trained on the neural data performed similarly to human listeners on identi-
fication judgments and made a similar pattern of confusions on the task. The temporal
dynamics of this coding regime were recently investigated by comparing human per-
formance on identifying vowels at varied stimulus presentation durations (2 to 128 ms)
with neural responses in guinea pigs to the same temporally constrained tokens (Occelli,
Suied, Pressnitzer, Edeline, & Gourévitch, 2015). Many vowels could be identified in
the neural data based on the unique characteristics of certain vowels and matching
spectrotemporal tuning patterns of individual neurons. These neuronal responses
could be used to reliably identify vowel stimuli, even at the extremely short durations.
In contrast, human performance was near chance at the shortest durations, although
it improved rapidly, plateauing to near ceiling performance around 16 milliseconds.
However, while the neural response accuracy did improve some with longer stimuli,
it never approached ceiling. This disconnect between behavioral responses and the
information carried in neural responses of auditory cortex indicates that behavior is
subject to many intervening processing steps, but that much of the necessary infor-
mation needed for accurate identification can, in principle, be gleaned from primary
auditory areas.
A collection of studies by Bizley, Walker, and colleagues (2009, 2011) sought to
identify how processes involved in pitch, location, and timbre extraction were col-
lectively organized in primary auditory cortex by presenting ferrets with different
combinations of all three cues. Four locations, vowels (timbres), and fundamental
frequencies were all crossed, resulting in 64 stimuli. Results of the first study
(Bizley et al., 2009) revealed that these processes were highly overlapping, and
intermingled in auditory cortex. Neurons often responded to different levels of mul-
tiple dimensions. Timbre and pitch, in particular, tended to covary and were some-
what more differentiated from location. A later study (Walker et al., 2011) found that
within individual neurons, sensitivity to certain dimensions and information rates
tended to cluster into certain anatomical areas. Additionally, many neurons relayed
information for multiple acoustic dimensions throughout the time course of their
responses. Building on this work, Allen, Burton, Olman, and Oxenham (2017) showed
that acoustic variation among the dimensions of pitch and timbre (operationalized
as spectral centroid, or brightness) produced largely overlapping patterns of activa-
tion in the fMRI responses of humans. However, similar to the findings in ferret au-
ditory cortex (Bizley et al., 2009; Walker et al., 2011), the pitch and timbre changes
that they presented to the participants could be decoded from patterns of activity
within this area of common activation. Together, these studies emphasize not only
how intricately woven together timbre, location, and pitch sensitivity are in auditory
cortex, but also that mechanisms are already in place at these early areas to parse and
transmit the multidimensional world of sound. As Walker and colleagues point out,
these multiplexed response patterns may aid in feature binding for object perception
in downstream cortical and cognitive operations.
Before moving on from auditory cortex to later cortical structures that support lan-
guage and music processing, let us review how the processes just described fit with the
OPERA hypothesis (Patel, 2011, 2012, 2014). Functional and structural changes can be
observed following musical (and potentially linguistic) training at these early stages of
cortical processing (albeit less consistently demonstrated in cortical auditory areas than
at the brainstem). In particular, there is evidence for the functional adaptability of audi-
tory cortical receptive fields via top-down attentional biases. This attentional component
and its early acoustical processing targets represent two prerequisites that the OPERA
hypothesis suggests are required for music to tune processes related to speech: atten-
tion and overlap. Given the salience of pitch in speech and music-perception tasks, the
neural mechanisms for pitch could also make for an ideal target for attentional modula-
tion. Indeed, in most instances where plasticity has been demonstrated (following long-
term experience with tone languages and music), there is a strong bias toward features
related to pitch, suggesting that it is a useful target for attentional focus all the way down
to subcortical processes, as pointed out by Patel (2012).
Cortex
Human neocortex is responsible for many of the characteristics that set humans apart
from the rest of the animal kingdom. Given how unique speech and music are among
our species, it is no surprise that these faculties rely heavily on these evolutionarily
newer cortical areas. Indeed, language is the tool with which we are able to consider,
describe, and wrestle with abstract notions. Hierarchical models of auditory pro-
cessing and many strong findings suggest that increasingly complex representations and
concepts are built up from the acoustic to the more abstract as processing advances from
auditory cortices and progresses along the superior portions of the temporal lobes in the
ventral pathway (Bizley & Cohen, 2013; DeWitt & Rauschecker, 2012; Hickok & Poeppel,
2007; Rauschecker & Scott, 2009; Tsunada & Cohen, 2014). The following sections will
outline how music and language rely on both unique and domain-general cortical oper-
ations beyond typical auditory areas to successfully execute the operations that result in
our everyday experience of music and language (see Figure 35.1).
A number of studies have taken a high-level, comparative approach to examining how
music and language may or may not be preferentially processed throughout cortex using
fMRI. These studies typically involve presenting participants with stimuli from a wide
variety of sound categories, and having the participant respond to noncritical targets to
simply maintain attention. Leaver and Rauschecker (2010) presented participants with
a variety of 300 millisecond tokens from a variety of sound categories such as musical
instruments, speech, and other animal vocalizations. They found overlapping, non-
category-specific activity in areas near primary auditory cortex, but found more distal
regions that responded preferentially to speech in middle superior temporal gyrus and
sulcus, and regions that responded preferentially to music on the more medial surface
of right anterior superior temporal cortex, even after statistically controlling for a set
of acoustic features of the stimuli. Rogalsky, Rong, Saberi, and Hickok (2011) focused
on hierarchical structure processing in language, presenting sentences, scrambled
sentences, and novel melodies to participants, and found patterns of activation similar
to Leaver and Rauschecker (2010) for melodies (medial and anterior temporal cortex)
and for sentences (lateral temporal cortex) when amplitude envelope modulation was
controlled for. Angulo-Perkins and colleagues (2014) played longer, 1.5-second-long
sentences and melodies, in addition to animal vocalizations and environmental object
sounds, and found a strong preference for musical sounds in right anterior superior
temporal cortex, while speech demonstrated extensive activation in more lateral areas
of superior temporal cortex. Another study by this group (Armony, Aubé, Angulo-
Perkins, Peretz, & Concha, 2015) found that the same area of cortex only expressed ad-
aptation to music, not to other auditory stimuli.
Norman-Haignere, Kanwisher, and McDermott (2015) played an extensive and
varied set of 2-second musical, vocal, nonvocal, mechanical, and environmental sounds
to participants and performed a voxel decomposition analysis to examine the under-
lying neural response profiles. They found six components that covered different parts
of cortex: the first two followed tonotopic gradients, and the second two responded
to spectrotemporal modulation patterns in the stimuli. These first four components
accounted for all non-speech and non-musical sounds in the stimulus set and were
located most proximally to primary auditory areas. The last two components, how-
ever, corresponded only to speech (extending laterally from primary auditory areas),
and music (extending posteriorly and anteriorly from primary auditory areas), but no
strong hemispheric differences were apparent. Taken together, these findings indicate
that areas of cortex dedicated to processing speech sounds exist extending along su-
perior temporal sulcus (see also Overath, McDermott, Zarate, & Poeppel, 2015), while
areas specific to processing musical sounds exist more anteriorly and medially to these
speech areas.
It should be noted, however, that while some studies include factors accounting for
acoustic properties as covariates, these studies largely maintain relatively high-level
views of music and speech. Indeed, the varied stimulus sets described beg the ques-
tion of what preferential activation for speech and music at a broad, categorical level
really indicates about the underlying neural computations or functions involved. It
is similarly difficult to interpret activation from regions that demonstrate overlap be-
tween domains. This issue requires a deeper examination of the constituent parts of
these signals, both conceptually (phonemes, musical consonance) and acoustically
(temporal structure, fine spectral processing). As one example, a study by Giordano
and colleagues (2014) found category-specific activations to music, speech, and en-
vironmental sounds similar to previous studies in both active and passive listening
scenarios, but showed that much of this activation could be explained by a theoreti-
cally driven set of acoustic covariates. Other work examining environmental sounds
and objects similarly emphasizes the importance of acoustic features in the neural
responses to sound (Giordano, McAdams, Zatorre, Kriegeskorte, & Belin, 2013; Lewis,
Brefczynski, Phinney, Janik, & DeYoe, 2005; Lewis et al., 2009; Lewis, Talkington,
Tallaksen, & Frum, 2012; Samson et al., 2011).
Categorical Perception and Processing in the Ventral

Auditory Pathway
As acoustic processing proceeds from primary auditory areas, computations become
more object-oriented and abstract (Lewis et al., 2012; Rauschecker & Scott, 2009).
This is beneficial in that it allows us to perceive the identity of a phoneme, for ex-
ample, consistently in different contexts and across speakers, despite the acoustic
changes that may be introduced by acoustic interference or individual speakers. In
primates, superior temporal gyrus is responsible for decoding conspecific calls be-
fore they are projected to ventral prefrontal cortex (Russ, Ackelson, Baker, & Cohen,
2008; Tsunada & Cohen, 2014), where more abstract features are coded (Cohen,
Hauser, & Russ, 2006), integrated with spatial information (Cohen, Russ, Gifford,
Kiringoda, & MacLean, 2004), and used in behavioral decisions (Cohen et al., 2009).
A similar organization is found for speech along the superior temporal gyrus, par-
ticularly in the left hemisphere. In a meta-analysis of neuroimaging findings from
studies of language, DeWitt and Rauschecker (2012) observed a progression in sen-
sitivity from phonemic processing in middle superior temporal gyrus, to words in
anterior superior temporal gyrus, and finally to phrases in anterior superior temporal
sulcus. A similar hierarchical representation can be found for musical stimuli from
primary auditory areas to more anterior locations along the superior temporal lobe,
perhaps more strongly localized to the right hemisphere (Zatorre et al., 2002). The
robust representation of phonemic information in superior temporal gyrus was also
demonstrated via direct neural recordings (Mesgarani, Cheung, Johnson, & Chang,
2014). ECoG recordings while participants listened to a corpus of English sentences
allowed researchers to capture all of the salient phonemic units of English, which
allowed them to then decode the response profiles of neurons and groups of neurons
to each phoneme. Subpopulations of neurons could, together, represent the salient
acoustic features of individual consonants and vowels and even encoded salient artic-
ulatory cues like voice onset time and formant frequencies.
A particularly remarkable aspect of human phoneme perception is its robust cate-
gorical nature; that is, sounds that vary along a continuous underlying dimension
are perceived as belonging to discrete groups, rather than changing gradually with
the changed dimension. This is marked by insensitivity to differences among group
members and high sensitivity between exemplars belonging to different groups, de-
spite the same absolute difference in the underlying dimension. It is these perceptual
categories that help a listener encode the wide range of phonemes that are essential to
language, even when the specific acoustics of a given phoneme vary as a function of
noise, distortion, or dynamics of the speaker. This perceptual phenomenon is partic-
ularly salient among speech phonemes (Liberman, Harris, Hoffman, & Griffith, 1957),
and has even been argued to be unique to speech (Eimas, 1963; Mattingly, Liberman,
Syrdal, & Halwes, 1971). However, categories of speech phonemes can be learned by
animals that do not possess language, including the chinchilla (Kuhl & Miller, 1975),
Japanese quail (Kluender, Diehl, & Killeen, 1987), and budgerigars (Dooling & Brown,
1990). Categorical perception has also been demonstrated for sequential (Siegel & Siegel,
1977a, 1977b; Burns & Ward, 1978; Burns & Campbell, 1994) and simultaneous musical
intervals (Zatorre & Halpern, 1979) among musicians, and in some cases for single tones
among musicians with absolute pitch (Siegel & Siegel, 1977a; Burns & Campbell, 1994;
Miyazaki, 1988). Notably, the findings in animals and among musicians emphasize the
importance of experience, since naïve animals and non-musicians do not perceive con-
tinuous changes between stimuli in this manner.
fMRI evidence in humans implicates the superior temporal sulcus in the support
of these highly categorical percepts. In speech, activation from learned phonemic
categories activates regions of left middle superior temporal gyrus more than (non-
categorical) complex acoustic stimuli (Libenthal, Binder, Spitzer, Possing, & Medler,
2005). However, other studies that examine specific patterns of voxel activation
within the fMRI signal implicate Broca’s area (Lee, Turkeltaub, Granger, & Raizada,
2012). This activation is in some cases in addition to activation in left superior tem-
poral gyrus, as well as parietal and motor areas. Left superior temporal gyrus may
be most responsive in low noise situations, whereas activation in Broca’s and motor
areas may be more reliably observed in noisier situations (Du, Buchsbaum, Grady, &
Alain, 2014).
Chang and colleagues (2010) provided a particularly strong demonstration of the
categorical nature of representations of speech phonemes in superior temporal gyrus
at the neural level with ECoG. Using direct recordings of activity in superior temporal
gyrus, they found that phonemic categories could be decoded from distributed re-
cording sites. A classifier trained on the neural responses performed best at the peak
in neural activity 110 milliseconds after phoneme onset, and confused phonemes
in a manner similar to the behavioral performance of the participants. Later results
suggest that the neural encoding of phonemes in superior temporal gyrus (along
with predictions from frontal cortex) might support the phoneme restoration ef-
fect wherein part of a word is perceived even though it has been acoustically masked
(Leonard, Baud, Sjerps, & Chang, 2016). Interestingly, fMRI studies of categorical
perception for musical intervals implicate similar superior temporal structures in
the right hemisphere based on adaptation and discrimination paradigms (Klein &
Zatorre, 2011), and through the use of multi-voxel pattern analysis (Klein & Zatorre,
2015). This agrees with processing asymmetries that have been proposed for left and
right hemispheres that best suit speech and language operations, respectively (Zatorre
et al., 2002).
Sound Sources: Timbres, Speakers, and Questions

of Hemispheric Specialization
In addition to phonemic information, speech carries identifiable information
pertaining to the identity of the speaker. Indeed, this is especially pertinent to the field
of forensic phonetics, which assists in legal or investigative settings where accurate
speaker identification might be required (Jessen, 2008; Köster, Hess, Schiller, & Künzel,
1998; Nolan, 1991). A complete accounting of the acoustic landmarks that are impor-
tant to speaker identity are still somewhat unclear, but speech signals contain cues
from features of the vocal folds and vocal tract that can be used to indicate the speaker’s
size and gender (Creel & Bregman, 2011; Elliot & Theunissen, 2009; Schweinberger
et al., 2014). The correspondence between specific physical cues in speech signals and
the identity of individuals is similar to that of face perception in the visual modality
(Latinus & Belin, 2011; Von Kriegstein, Kleinschmidt, Sterzer, & Giraud, 2005). Faces
are known to be processed by specialized areas in cortex (Kanwisher, McDermott,
& Chun, 1997), leading some investigators to wonder if the same is true of voices.
Belin and colleagues (2000) played participants a wide variety of synthetic and nat-
ural stimuli and found a portion of anterior superior temporal cortex that was espe-
cially sensitive to the human voice, which they termed the temporal voice area (Belin,
Fecteau, & Bedard, 2004). This was echoed by another study that employed an fMRI
adaptation paradigm while subjects listened to blocks of the same utterance made by 12
different speakers, or blocks of the same speaker making 12 different utterances (Belin
& Zatorre, 2003). The difference in activation between these two blocks demonstrated
that an area in the right anterior superior temporal cortex was highly active throughout
the multi-speaker blocks and was engaged to a greater degree than the multi-syllable
blocks, indicating that this area of cortex is sensitive to differences between speaker’s
voices. Follow-up work has demonstrated that this area can be modulated by atten-
tion to the speaker rather than the linguistic content of an utterance (Bonte, Hausfeld,
Scharke, Valente, & Formisano 2014; Von Kriegstein, Eger, Kleinschmidt, & Giraud,
2003), or by the recognition of familiar speakers (Von Kriegstein, & Giraud, 2004),
and can dynamically interact with the fusiform face area (Von Kriegstein et al., 2005).
Multi-voxel pattern analysis techniques have also been used to decode the identity
of individual speakers from right anterior superior temporal cortex, as well as from
superior temporal cortex bilaterally (Bonte, Hausfeld, Valente, & Formisano 2014;
Formisano, Martino, Bonte, & Goebel, 2008).
Other evidence for the importance of these right-hemisphere areas in voice-identity
processing comes from cases of phonagnosia. Patients with phonagnosia cannot iden-
tify individuals by voice, often as a consequence of a lesion to their right temporal lobe
(Hailstone, Crutch, Vestergaard, Patterson, & Warren, 2010; Van Lancker, Cummings,
Kreiman, & Dobkin, 1988; Van Lancker, Kreiman, & Cummings, 1989; review: Slevc &
Shell, 2015). This cortical specialization may have arisen in our ancestors, as it is pre-
sent in other mammals that rely on voice identification, such as dogs (Andics, Gácsi,
Faragó, Kis, & Miklósi, 2014) and macaques (Perrodin, Kayser, Logothetis, & Petkov,
2011; Petkov et al., 2008).
A parallel line of research also implicates this area in timbre processing. Indeed, many
of the processes involved in identifying speakers and timbres are similar: multidimen-
sional representations (McAdams & Giordano, 2009; Theunissen & Elie, 2014), the
need to identify a target by matching it to a representation based on its salient acoustic
features (Bizley & Cohen, 2013), and using these features to separate targets from the
background of an acoustic scene (Pressnitzer et al., 2011). Findings from patients with
right anterior hemispheric lesions, which overlap with the voice-processing areas
discussed previously, indicate that the resolution of spectral and temporal cues involved
in timbral judgments may also rely on right anterior temporal lobe structures (Samson,
2003). In these studies, patients with left and right anterior temporal lobe lesions were
asked to discriminate (Samson & Zatorre, 1994) or rate the similarity of synthetic tones
that varied in their onset times (temporal manipulation) or number of harmonics (spec-
tral manipulation; Samson, Zatorre, & Ramsay, 2002). Patients with right-hemispheric
lesions were particularly impaired at discriminating temporal and spectral cues, and
their perceptual spaces based on multidimensional scaling of similarity ratings were
distorted relative to left-hemisphere patients and controls (Samson & Zatorre, 1994;
Samson, Zatorre, & Ramsay, 2002). Interestingly, some cases of phonagnosia associated
with anterior temporal lobe dysfunction are also comorbid with a reduced ability to rec-
ognize and discriminate different musical instruments (Hailstone et al., 2010).
Taken together, these results suggest that the ability to identify individual speakers
and the ability to perceive differences in instrumental timbres may both rely on the
right anterior temporal lobe. It seems that in both cases, the brain is solving a similar
problem, potentially by utilizing similar cues to identify the source of a complex sound.
Timbre itself is an area of music perception that has not received a good deal of direct
focus (but see Allen et al., 2017, Menon et al., 2002; Warren, Jennings, & Griffiths, 2005),
but it is implied and inherent in many of the other studies of music and language pro-
cessing that use a varied set of musical stimuli (Angulo-Perkins et al., 2014; Armony
et al., 2015; Leaver & Rauschecker, 2010; Norman-Haignere et al., 2015). Notably, these
studies all implicate the right anterior temporal lobe (or planum polare) and neces-
sarily incorporate different timbral stimuli in their manipulations. These findings using
more naturalistic stimuli also appear to engage more anterior areas in temporal cortex
than studies examining timbre with more controlled (synthesized) stimuli (Allen et al.,
2017; Menon et al., 2002; Warren et al., 2005). In a direct comparison of various sound
sources that were controlled for pitch and spectral modulation, utilizing multi-voxel
pattern analysis methods previously employed for speaker identification, Staeren and
colleagues (2009) found large distributed activation patterns that were involved in the
decoding of sound sources, whether it was a human or animal vocalization or an instru-
ment. Notably, the areas that aided in the decoding of Staeren and colleagues’ sounds
extended further along superior temporal gyrus in both hemispheres than the voxels in-
volved in the decoding of the pitch of the tones, which were clustered around more pri-
mary areas. Thus, the small body of work comparing speaker and timbre discrimination
suggests a set of related neural processes; however, there is a need for further work di-
rectly comparing tasks involved in speaker identification and tasks involved in instru-
mental timbre discrimination.
One explanation for potentially overlapping processes in speaker and timbre iden-
tification is that they both rely on large time-scale spectral cues (Warren et al., 2005;
Warren, Scott, Price, & Griffiths, 2006). A number of studies (Menon et al., 2002;
Warren et al., 2005; Warren et al., 2006) have suggested that an area in the posterior tem-
poral lobes, known as planum temporale, is involved in analyzing these spectral cues
(Griffiths & Warren, 2002). Others have also suggested that such analyses are prefer-
entially processed in the right hemisphere (McGettigan, & Scott, 2012; Poeppel, 2003;
Zatorre et al., 2002). The sensitivity of the right hemisphere to spectral information has
been well established (Schönwiesner, Rübsamen, & Von Cramon, 2005; Zatorre & Belin,
2001). Moreover, right-hemisphere structures are especially responsive to slower tem-
poral modulation rates (Belin et al., 1998; Boemio, Fromm, Braun, & Poeppel, 2005),
which allows for the processing of spectral complexity given inherent trade-offs in time
and frequency resolution (Zatorre et al., 2002). McGettigan and Scott (2012) also dis-
cuss a right-hemisphere advantage for spectral processing, and emphasize that this
right-hemisphere sensitivity is more consistent across studies than findings of greater
sensitivity to faster temporal modulations in the left hemisphere.
Syntactic and Structural Operations

Syntactic rules allow for long-distance dependencies and hierarchical or referential rela-
tions among concepts in sentences. Like speech and language, music is also constrained
by a set of rules and hierarchies. Indeed, Western music has an exceptionally well-
characterized representational structure that even lends itself to hierarchical analysis
using tools developed in linguistics (Granroth-Wilding & Steedman, 2014; Heffner &
Slevc, 2016; Katz & Pesetsky, 2011; Krumhansl & Kessler, 1982; Lerdahl & Jackendoff,
1983). The hierarchical structure of music comes from the importance of certain notes
within a musical scale and influences the tonal relationships in the natural develop-
ment of a musical piece. Such a harmonic progression typically begins by establishing
a tonal context or structure, then subtly deviating from that context to create tension
before finally resolving that tension back at the original tonal center. The context of a
musical sequence is established by harmonic relationships between notes or chords
early in a sequence, and is actively updated as new harmonic information arrives, as in
the case of a key modulation (Krumhansl & Kessler, 1982). A harmonic structure then
facilitates expectancies and processing for new information that fits within it (Bigand,
Poulin, Tillmann, Madurell, & D’Adamo, 2003). The creation of tension or ambiguity
in a harmonic structure is a valuable tool for composers and performers and is thought
to be responsible for much of the emotional power of music (Huron, 2006; Krumhansl,
2002; Meyer, 1956). The relations between pitches, keys, and harmonic components
can be mapped onto the surface of a toroid, or inner-tube-like shape (Krumhansl &
Kessler, 1982). These relations and to some degree even this representational shape itself
are maintained in cortex and are updated as a musical sequence moves through tonal
space (Janata et al., 2002). If the proper parallels are drawn, music could be a powerful
tool for understanding how abstract representations are engaged and updated in the
brain during online tasks, providing an excellent test of many linguistic principles and
questions.
A cardinal feature of both music and language is how they unfold over time. A par-
ticularly vibrant subject of debate concerns how music and speech may or may not
share neural resources to temporally and sequentially integrate acoustic information
into coherent wholes (Ding et al., 2015; Doelling & Poeppel, 2015, Patel, 2003; Peretz
& Coltheart, 2003; Peretz, Vuvan, Lagrois, & Armony, 2015). Current findings on the
topic of shared neural resources for the structural and syntactic processing of speech
and music are split. Lesion studies indicate that agrammatism and atonalia can occur
independently from one another and stem from double-dissociable patterns of neural
damage (Peretz, 1993; Peretz & Coltheart, 2003; Slevc, Faroqi-Shah, Saxena, & Okada,
2016), implying separate neural substrates for structural processing in music and lan-
guage. Meanwhile, fMRI (Janata, Tillmann, & Bharucha, 2002; Koelsch, Gunter,
Zysset, Lohmann, & Friederici, 2002; LaCroix et al., 2015; Oechslin, Van De Ville,
Lazeyras, Hauert, & James, 2013; Peretz et al., 2015) and MEG (Maess, Koelsch, Gunter,
& Friederici, 2001) studies tend to indicate that processing musical structure activates
areas of the brain also implicated in language and working memory, particularly left
inferior frontal areas. Additionally, event-related potential (ERP) studies (Minati et al.,
2008; Patel, Gibson, Ratner, Besson, & Holcomb, 1998) have shown parallels in electro-
physiological signatures associated with musical and linguistic structural irregularities.
This divide has led to the suggestion by Patel (2003) that music and language have in-
dependent long-term storage substrates (thus accounting for lesion findings) but share
resources involved in online syntactic processing (accounting for overlapping activa-
tion in imaging data and behavioral interactions), a hypothesis known as the shared syn-
tactic integration resource hypothesis, or SSIRH. Support for this idea has been borne out
in behavioral and electrophysiologial paradigms (Fedorenko et al., 2009; Hoch, Poulin-
Charronnat, & Tillmann, 2011; Koelsch, Gunter, Wittfoth, & Sammler, 2005; Koelsch &
Siebel, 2005; Koelsch & Friederici, 2003; Koelsch, Gunter, Friederici, & Schröger, 2000;
Koelsch, Rohrmeier, Torrecuso, & Jentschke, 2013; Slevc, Rosenberg, & Patel, 2009; re-
view: Kunert & Slevc, 2015). An early right-anterior negative ERP component (ERAN)
occurring around 150–250 milliseconds after chord onset has been shown to be a hall-
mark of these musical syntactic violations (Koelsch & Friederici, 2003; Koelsch, 2005;
Koelsch et al., 2000; Koelsch et al., 2013). This ERP component is similar to (Koelsch
& Friederici, 2003; Koelsch, 2005) and even interacts with (Koelsch et al., 2005) ante-
rior negative ERP components elicited by linguistic syntactic manipulations. Note, that
these early negativities are characterized by hemispheric dominance, but not exclu-
sivity. A similar pattern of results was echoed in an intracranial EEG study conducted by
Sammler and colleagues (2012). Syntactic violations in music and language exhibited sig-
nificant (but not complete) overlap among frontal and temporal electrodes, bilaterally.
Musical violations tended to generate more activity in frontal electrodes, while lin-
guistic violations produced more activation in temporal lobes, which did not respond
to musical syntactic violations. Additionally, language-related responses peaked earlier
among left-hemisphere sites, while music-related responses peaked earlier in the right
hemisphere.
However, some of these interactions may be highly task-dependent (LaCroix et al.,
2015). In conflict with the SSIRH, fMRI work by Fedorenko and colleagues (Fedorenko
et al., 2011; Fedorenko et al., 2012) indicates that when controlling for low-level pro-
cessing aspects of language (via sentences vs. strings of pronounceable nonwords to
control phonological processes) and music (via recordings of songs vs. rhythm-and
pitch-scrambled versions to control rhythm-and pitch-extraction processes), indi-
vidually defined regions of interest (ROIs) for either language or music showed little
activity in response to changes in the other domain when processing higher-level
features, like syntax. Finally, Farbood, Heeger, Marcus, Hasson, and Lerner (2015) di-
rectly examined fMRI activations and inter-subject response correlations between
larger-scale hierarchical scrambling manipulations of a musical piece at the level of
measures, phrases, and sections. This activity was compared with a verbally delivered
narrative, also hierarchically scrambled at the word, sentence, and paragraph level
(Lerner, Honey, Silvert, & Hasson, 2011). They found that more primary, acoustically
sensitive auditory areas (the bottom of the hierarchy) were sensitive to all levels of
scrambling in both domains, but higher-level structure in music engaged regions
extending across middle superior temporal gyrus into frontal cortex. This pattern
was distinct from the areas sensitive to higher-level structure in linguistic scrambling,
which progressed more posteriorly, into the temporoparietal junction and more ante-
rior frontal areas.
Another fMRI study by Rogalsky and colleagues (2011) compared music with
speech and scrambled speech and demonstrated that areas sensitive to the structural
manipulations in the linguistic stimuli were dissociable from voxels responsive to
musical stimuli. These authors, along with Slevc and Okada (2015), suggest that ev-
idence of overlap between music and speech processing may in fact be due to a mu-
tual reliance on domain-general cognitive resources such as working memory or
cognitive control. Support for this hypothesis comes from evidence for overlapping
regions of cortex in left inferior frontal gyrus that are activated both by cognitive
control tasks and linguistic syntax processing (Hsu, Novick, & Jaeggi, 2017; January,
Trueswell, & Thompson-Schill, 2008). Complementary support for this idea comes
from findings that harmonic violations in music and syntactic violations in language
interact and drive left inferior frontal gyrus activation (Kunert, Willems, Casasanto,
Patel, & Hagoort, 2015), suggesting that cognitive control resources may be engaged
in the resolution of both violations. Although the relationship between cognitive con-
trol and syntactic violations requires further study, it is worth noting that prefrontal
areas implicated in cognitive control are also implicated in other fMRI data that find
parallels in the processing of language and music (Janata et al., 2002; Koelsch et al.,
2002; Oechslin et al., 2013).
Semantics
Semantic processing is integral to all linguistic operations. It allows us to identify and
name referents or ideas when communicating. However, an analog is difficult to place in
the musical domain, given that communication through music does not typically rely on
conveying referential information (Slevc & Patel, 2011). In language, semantic information
partially involves object, sensory, and referential knowledge about a given word, which
can be decoded from distributed activation patterns in the brain (Carlson, Simmons,
Kriegeskorte, & Slevc, 2014; Correia et al., 2014; Mitchell et al., 2008; see, in this volume,
Bauer & Just, Chapter 21, and Musz & Thompson-Schill, Chapter 22). The left anterior tem-
poral pole may function as a hub that connects and activates individual representations
(Correia et al., 2014). This hypothesis is strongly supported by work on semantic de-
mentia, which involves the progressive loss of the ability to name objects in the environ-
ment and is associated with degeneration of the anterior temporal lobe (Patterson, Nestor,
& Rogers, 2007). Importantly, semantic knowledge also includes more abstract, concep-
tual components that cannot map directly onto objects or bodily experiences, which is
supported by a larger network that includes angular gyrus, fusiform gyrus, inferior
frontal gyrus, middle temporal gyrus, posterior cingulate gyrus, superior frontal gyrus,
supramarginal gyrus, and ventromedial prefrontal cortex (Binder & Desai, 2011). Huth,
de Heer, Griffiths, Theunissen, and Gallant (2016) conducted a comprehensive examina-
tion of long passages of natural speech using voxel-wise model estimation that predicts
activity of individual voxels following a given word by its co-occurrence with all of the
words in a large corpus of English. The general semantic areas implicated in previous work
were again identified using this method. Principal component and clustering analyses also
uncovered 12 main semantic categories in the neural activation patterns. These could be
mapped back onto the surface of the brain, creating an atlas of where activation for each
word resided. The results showed that category-related words in the stimulus set clustered
together, and that these representations were consistent across individuals.
Impressive decoding efforts like those just described would be difficult to instantiate
in musical stimuli; however, there is some indication that musical knowledge can access
this same general semantic network. Koelsch and others (2004) took advantage of the
well-known N400 ERP response that is a hallmark of semantic integration difficulty,
exhibiting higher amplitudes when words or concepts are out of place in a given refer-
ential context (Kutas & Federmeier, 2011; Lau, Phillips, & Poeppel, 2008). They utilized
a semantic priming paradigm, wherein a short sentence describes a situation and is
followed by a word, which may or may not fit the context of the sentence. When the
context did not fit, EEG activity 400 milliseconds after word onset tended to be stronger
than for words that were more appropriate. The key manipulation in this study, how-
ever, was that some target words were preceded by musical sequences that resembled
objects or established a connotative context, such as for “wideness,” “stairs,” or “bird.”
Target words that did not match the context set up by the preceding musical sequence
exhibited an N400 effect similar to the linguistic trials, demonstrating that a semantic
context was recalled and established by the musical sequence.
This clever design establishes that semantic knowledge can interface with musical
stimuli, but this interaction likely does not possess the richness of meaning that a sen-
tence might. More importantly, the constituent parts in the musical sequences (notes
and chords) themselves have little to no specific semantic meaning for most listeners in
the same way that each word in a sentence does. Thus, the connotative meaning of the
sequence requires more time to be built up, and would be difficult to manipulate on a
finer level. Musical processing also appears to be preserved in some cases of semantic
dementia (Weinstein et al., 2011), suggesting that musical concepts are dissociable from
the processes with which the anterior temporal lobes are involved.
Perhaps a more interesting musical parallel to linguistic semantics is absolute pitch.
Absolute pitch is the rare ability to name individual notes without a tonal reference
(Siegel & Siegel, 1977a; Miyazaki, 1988). This is thought to be due to a proficiency in
labeling abilities, as absolute pitch participants easily provide the nearest categorical
note name to a tone that might vary continuously throughout the frequency spectrum
(Levitin & Rogers, 2004). Notably, however, there are indications they do not perceive the
notes categorically in the same way that variability around anchors in speech categories
is ignored (Levitin & Rogers, 2004); that is, they provide poorer goodness of fit ratings
as frequencies move from tonal centers, and have the same frequency-difference detec-
tion thresholds as other non-absolute pitch possessors. Thus, these individuals have the
ability to map the sound of the note to its semantic label readily, despite small deviations
in frequency, similar to how listeners provide the name of a word following an appro-
priate acoustic pattern. Given that this ability is particular to the 12-pitch chroma that
make up Western music, this could provide an interesting avenue to test lexical or se-
mantic access with a reduced dimensionality compared to that of a lexical or semantic
knowledge database.
Rhythm
Music and language unfold over time as highly structured sequences. The temporal na-
ture of both domains requires the orchestration of a wide variety of neural and cogni-
tive processes in both the producer of the sequence (be it a melody or sentence) and
the listener (Janata & Grafton, 2003). Regularity in the timing of these sequences helps
align the systems of both the transmitting and receiving ends of the communication by
directing attention to certain points in time to help create expectations and predictions.
These anticipatory processes, in turn, help to guide attention, and to coordinate actions
and movements in response. How quickly does such activity need to occur? A review of
a large corpus of speech and music by Ding and colleagues (2017) suggests that there are
regularities in the rates at which music and speech unfold in time. Musical information
typically unfolds at a rate of about 2 Hz, while speech unfolds somewhat more rapidly
at 5 Hz.
The regularities observed across such a wide array of stimuli may be a function of
the perceptual or neural constraints on the part of the interlocutors. Indeed, MEG
findings suggest that low-frequency neural oscillations (< 8 Hz) in the delta (1–3 Hz)
and theta (4–8 Hz) bands are involved in entrainment to musical (Doelling & Poeppel,
2015) and linguistic information (Ding et al., 2015). This aligns closely with the temporal
regularities indicated in the stimuli just described. An influential hypothesis (Giraud &
Poeppel, 2012) suggests that neural oscillations at these rates are responsible for pack-
aging or chunking incoming information, which in turn allows for an alignment be-
tween the neural system and the stimulus as a given communication unfolds in time,
perhaps by driving or realigning gamma (25–35 Hz) band activity, which is thought to
be related to attention (Griffiths et al., 2010; Mesgarani & Chang, 2012; Zion Golumbic
et al., 2013).
There are also important differences between music and speech in the role that
rhythm plays. Rhythmic information in speech is used for prosodic cues, to convey
metrical stress as well as for differentiating phonological cues, such as voice onset time
(Patel, 2012), which may operate at the level of syllables (Ghitza, 2012). However, these
rarely evoke a regular pulse in the same way that musical rhythm does (Patel, 2006).
Beta band activity (around 20 Hz, notably distinct from the ranges just mentioned) has
been found to track the rhythmic regularity of an incoming stimulus in a predictive
manner (Fujioka, Trainor, Large, & Ross, 2012), while gamma activity appears to follow
the metrical regularity as it is internally represented (Fujioka, Trainor, Large, & Ross,
2009; Large & Snyder, 2009). This was demonstrated in an MEG study where a small
number of tones in a sequence that demarcated an isochronous rhythm were removed.
Regular beta activity tracked with inter-beat intervals and was disrupted by the missing
tones, while gamma activity tracked the regularity of the sequence, regardless of the
missing tones (Fujioka et al., 2009).
The relationship between beta band activity and music, as well as between beta range
activity and movement more generally (Hari & Salmelin, 1997), underscores another
major difference between speech and musical processing: musical rhythm is often as-
sociated with (sometimes spontaneous) motor activity such as foot tapping, head nod-
ding, and dancing (Janata & Grafton, 2003; Patel, 2006); that is, the regularity found
in musical rhythm supports and encourages synchronous movements to a beat.
Accordingly, it has been found that the processing of musical rhythms engages sensori-
motor structures in the brain such as the supplementary motor area, premotor cortex,
cerebellum (Chen, Penhune, Zatorre, 2008a, 2008b; Janata & Grafton, 2003), and basal
ganglia (Grahn, 2009; Grahn & Brett, 2007; Grahn & Rowe, 2009). Notably, many of the
studies demonstrating the engagement of motor areas during rhythm perception typi-
cally involve passive listening rather than overt movements (Chen et al., 2008a; Grahn &
Rowe, 2009) and are similar to the areas involved in perceiving speech rhythms (Geiser,
Zaehle, Jancke, & Meyer, 2008). Moreover, these areas often display a large degree of
functional connectivity during rhythm perception, which can be modulated by beat
strength (Grahn & Rowe, 2009)
Finally, the strong proclivity for humans to move rhythmically to music is something
of an anomaly in the animal kingdom, as most animals, including our close primate
relatives, do not entrain to external rhythms (Patel, 2006). Why are humans capable
and inclined to move synchronously to rhythms? One hypothesis, known as the vocal
learning and rhythmic synchronization hypothesis (Patel, 2006), suggests that the ability
to entrain and move to a beat is related to the need to learn the vocalizations of one’s
species. Specifically, the control required to direct and refine vocalizations establishes
connections between auditory and motor areas in the brain that support movements
based on acoustic input. Evidence in favor of this hypothesis comes from examples of
animals such as the cockatoo (Patel, Iverson, Bregman, & Schultz, 2009a; Schachner,
Brady, Pepperberg, & Hauser, 2009), which can spontaneously produce accurate, sus-
tained movements in time with music (Patel, Iverson, Bregman, & Schultz, 2009b).
Note, however, that there is emerging evidence that a non-vocal-learning species can
be trained to follow a beat (Takeya, Kameda, Patel, & Tanaka, 2017; Wilson & Cook,
2016), so more work is needed to understand the processes underlying rhythmic
entrainment.
Conclusions and Future Directions
It is perhaps no surprise that, as acoustic signals, music and language tread much the
same ground as they ascend the auditory pathway from the brainstem to primary au-
ditory cortex and beyond. This is emphasized by the less obvious and quite interesting
evidence that long-term experience can, in some cases, induce plastic changes in neural
function and even in cortical structure in parts of auditory cortex, motor cortex, and
corpus callosum. As to how experience might induce these changes, the OPERA hy-
pothesis suggests that sustained top-down attention, repetition, and motivation in a
precise task, required by either music or language experience, could play a prominent
role in modifying neural circuitry shared by both domains (Patel, 2011, 2012, 2014).
However, in higher areas of cortex, certain processes appear to become more specialized
for each domain and are influenced by other cognitive functions. Overlap in linguistic
and musical neural processing is less clear at these stages, with quite disparate neural
resources engaged by each domain in many cases. Picking apart where operations be-
tween domains do and do not overlap is an active area of research and debate. Given
the evidence of positive, cross-domain behavioral outcomes associated with neural
plasticity induced by long-term experience in the brainstem, identifying higher neural
regions and cognitive operations that could be improved by musical training is an ex-
citing possibility. However, a limiting factor beyond auditory cortex appears to be the
often inconsistent evidence for overlap in the neural structures involved in processing
music and language. Based on evidence reviewed here, more work is needed to clarify
more precisely which higher linguistic and cognitive functions might be candidates
for improvement via cross-domain experience and plasticity. As others have pointed
out (Patel, 2006, 2012), one promising possibility for future research will be to inves-
tigate the possibility of overlapping rhythmic processes between music and language
(Doelling & Poeppel, 2015).
This search may be hampered by the level of resolution that is currently available
among the methods in our toolbox. EEG, MEG, and fMRI have proven invaluable, but
as Peretz and colleagues (2015) have pointed out, it remains possible that “overlap” in
these studies may reflect domain-specific networks that overlap geographically (at least
within the spatial resolution of a given method) but do not interact functionally. For
example, activation of a given voxel or region when listening to speech could be the re-
sult of speech-related neural computations by a few thousand of the many thousands
of neurons represented by that voxel. A similar level of activation may also be found for
that voxel when performing a music-related task; however, this could be the result of ac-
tivity within the same few thousand neurons, or a completely different set of neighboring
neurons. Resolving issues like this in fMRI is quite difficult, although new data-driven
methods may provide some solutions (Norman-Haignere et al., 2015), as will future,
cleverly designed studies perhaps aided by interference paradigms (Kunert & Slevc,
2015). ECoG might be another useful tool for resolving these issues. While this method
has been very profitably employed for phonological (Chang et al., 2010; Mesgarani et al.,
2014), syntactic (Ding et al., 2016), and streaming investigations (Mesgarani & Chang,
2012; Zion Golumbic et al., 2013) in speech, there are only a handful of studies employing
this technique to examine musical sequences (Potes et al., 2012, Potes, Brunner, Gunduz,
Knight, & Schalk, 2014; Sammler et al., 2012; Sturm et al., 2014). It will be very interesting
to see what is revealed about the processing of music, speech, and the auditory system as
musical ECoG work develops. Studies of attention-modulated cortical activity in ECoG,
MEG, or EEG may particularly benefit from examining an acoustically rich, multi-part
stimulus like music. Additionally, employing stimulus reconstruction techniques in
these studies could be particularly illuminating for the cognitive aspects of streaming
operations and attentional processes that are engaged during music listening, especially
given the use of timbre and timbral fusion as compositional tools (McAdams, 1982;
McAdams & Bregman, 1979).
Another useful approach may be to focus on the more primitive cognitive or acous-
tical operations at play in both domains, similar to how linguists have re-examined
concepts such as syntax (Ding et al., 2016; Fedorenko, Duncan, & Kanwisher, 2012),
semantics (Binder & Desai, 2011; Lau et al., 2008), and phonology (Du et al., 2014;
Scharinger, Idsardi, & Poe, 2011) through different sets of constituent cognitive opera-
tions. Music research in particular might benefit from this type of refinement and, in-
deed, work to catch up to language in this respect is underway (Parbery-Clark, Strait,
Anderson, Hittner, & Kraus, 2011; Schulze & Koelsch, 2012; Slevc et al., 2016; Slevc &
Okada, 2015). A better understanding of the acoustical analyses the brain engages in
when perceiving timbre, which is perhaps the least understood aspect of music per-
ception from a neural perspective, could shed better light on the acoustical operations
common to both domains. Additionally, examining how music and language tasks
might mutually or differently engage more primary executive functions such as con-
flict management, switching, and working-memory updating (Diamond, 2013; Miyake
& Friedman, 2012) might indicate what operations are taxed by both domains and could
better clarify previous findings (Kraus, Strait, & Parbery-Clark, 2012; LaCroix et al.,
2015; Schulze & Koelsch, 2012; Slevc & Okada, 2015). Finally, our understanding of lan-
guage and music will surely benefit from a better understanding of the neural processes
involved in the production of language and music. So far, there has been relatively little
work directly comparing speech and music production or improvisation (e.g., Brown,
Martinez, & Parsons, 2003; Callan et al., 2006; Zarate, 2013); however, insights from pro-
duction are likely to be informative for both domains.
While the relationship between music and language is still a somewhat controver-
sial topic (e.g., Heffner & Slevc, 2015; Leaver & Rauschecker, 2010; Norman-Haignere
et al., 2015; Patel, 2012; Peretz et al., 2015; Schellenberg, 2015; Slevc & Okada, 2015), it is
clear that fundamental similarities in early stages of processing provide a foundation for
cross-domain modulation by long-term musical and linguistic experience. The overlap
observed at earlier stages of processing appears to give way to greater neural special-
ization at higher levels of cortex, which might preclude any cross-modal experience-
dependent plasticity (Patel, 2012). Thus, whether similar interactions resulting from the
plastic effects of experience in one domain or the other are indeed present at higher
levels remains to be seen. To resolve these issues, a deeper understanding of the neural
mechanisms involved in perceiving and processing music is needed, which, in turn, will
likely illuminate and motivate new hypotheses on aspects of speech processing, auditory
processing, and the cognitive operations that support these two fundamental faculties
that color and enrich the human experience.
Acknowledgments
We would like to thank Annirudh Patel and Christopher Heffner for their helpful feedback
during the writing of this manuscript. We would also like to give special thanks to Andrew
Borrell for his assistance with the figure.
References
Ahveninen, J., Huang, S., Nummenmaa, A., Belliveau, J. W., Hung, A. Y., Jääskeläinen, I.
P., . . . Raij, T. (2013). Evidence for distinct human auditory cortex regions for sound location
versus identity processing. Nature Communications, 4, 2585.
Allen, E. J., Burton, P. C., Olman, C. A., & Oxenham, A. J. (2017). Representations of pitch and
timbre variation in human auditory cortex. Journal of Neuroscience, 37(5), 1284–1293.
Anderson, S., Parbery-Clark, A., White-Schwoch, T., & Kraus, N. (2012). Aging affects neural
precision of speech encoding. Journal of Neuroscience, 32(41), 14156–14164.
Anderson, S., Skoe, E., Chandrasekaran, B., & Kraus, N. (2010). Neural timing is linked to
speech perception in noise. Journal of Neuroscience, 30(14), 4922–4926.
Anderson, S., White-Schwoch, T., Parbery-Clark, A., & Kraus, N. (2013). A dynamic auditory-
cognitive system supports speech-in-noise perception in older adults. Hearing Research,
300,  18–32.
Andics, A., Gácsi, M., Faragó, T., Kis, A., & Miklósi, Á. (2014). Voice-sensitive regions in the
dog and human brain are revealed by comparative fMRI. Current Biology, 24(5), 574–578.
Angulo-Perkins, A., Aubé, W., Peretz, I., Barrios, F. A., Armony, J. L., & Concha, L. (2014).
Music listening engages specific cortical regions within the temporal lobes: Differences be-
tween musicians and non-musicians. Cortex, 59, 126–137.
Armony, J. L., Aubé, W., Angulo-Perkins, A., Peretz, I., & Concha, L. (2015). The specificity of
neural responses to music and their relation to voice processing: An fMRI-adaptation study.
Neuroscience Letters, 593,  35–39.
Bandyopadhyay, S., Shamma, S. A., & Kanold, P. O. (2010). Dichotomy of functional organiza-
tion in the mouse auditory cortex. Nature Neuroscience, 13(3), 361–368.
Barton, B., Venezia, J. H., Saberi, K., Hickok, G., & Brewer, A. A. (2012). Orthogonal acoustic
dimensions define auditory field maps in human cortex. Proceedings of the National
Belin, P., & Zatorre, R. J. (2003). Adaptation to speaker’s voice in right anterior temporal lobe.
Neuroreport, 14(16), 2105–2109.
Belin, P., Fecteau, S., & Bedard, C. (2004). Thinking the voice: Neural correlates of voice per-
ception. Trends in Cognitive Sciences, 8(3), 129–135.
Belin, P., Zatorre, R. J., Lafaille, P., Ahad, P., & Pike, B. (2000). Voice-selective areas in human
auditory cortex. Nature, 403(6767), 309–312.
Belin, P., Zilbovicius, M., Crozier, S., Thivard, L., Fontaine, A., Masure, M. C., & Samson, Y.
(1998). Lateralization of speech and auditory temporal processing. Journal of Cognitive
Bendor, D., & Wang, X. (2005). The neuronal representation of pitch in primate auditory
cortex. Nature, 436(7054), 1161–1165.
Bermudez, P., & Zatorre, R. J. (2005). Differences in gray matter between musicians and
nonmusicians. Annals of the New York Academy of Sciences, 1060, 395–399.
Berwick, R. C., Friederici, A. D., Chomsky, N., & Bolhuis, J. J. (2013). Evolution, brain, and the
nature of language. Trends in Cognitive Sciences, 17(2),  89–98.
Bidelman, G. M., & Alain, C. (2015). Musical training orchestrates coordinated neuroplasticity
in auditory brainstem and cortex to counteract age-related declines in categorical vowel per-
ception. Journal of Neuroscience, 35(3), 1240–1249.
Bidelman, G. M., Gandour, J. T., & Krishnan, A. (2010). Cross-domain effects of music and lan-
guage experience on the representation of pitch in the human auditory brainstem. Journal of
Bidelman, G. M., Gandour, J. T., & Krishnan, A. (2011). Musicians demonstrate experience-
dependent brainstem enhancement of musical scale features within continuously gliding
pitch. Neuroscience Letters, 503(3), 203–207.
Bidelman, G. M., & Krishnan, A. (2009). Neural correlates of consonance, dissonance, and
the hierarchy of musical pitch in the human brainstem. Journal of Neuroscience, 29(42),
13165–13171.
Bidelman, G. M., & Krishnan, A. (2010). Effects of reverberation on brainstem representation
of speech in musicians and non-musicians. Brain Research, 1355, 112–125.
Bidelman, G. M., & Krishnan, A. (2011). Brainstem correlates of behavioral and compositional
preferences of musical harmony. Neuroreport, 22(5), 212–216.
Bigand, E., & Poulin-Charronnat, B. (2006). Are we “experienced listeners”? A review of the
musical capacities that do not depend on formal musical training. Cognition, 100, 100–130.
Bigand, E., Poulin, B., Tillmann, B., Madurell, F., & D’Adamo, D. A. (2003). Sensory versus
cognitive components in harmonic priming. Journal of Experimental Psychology: Human
Perception and Performance, 29(1), 159–171.
Binder, J. R., & Desai, R. H. (2011). The neurobiology of semantic memory. Trends in Cognitive
Sciences, 15(11), 527–536.
Bizley, J. K., & Cohen, Y. E. (2013). The what, where and how of auditory-object perception.
Nature Reviews Neuroscience, 14(10), 693–707.
Bizley, J. K., Walker, K. M., Silverman, B. W., King, A. J., & Schnupp, J. W. (2009).
Interdependent encoding of pitch, timbre, and spatial location in auditory cortex. Journal of
Neuroscience, 29(7), 2064–2075.
Boemio, A., Fromm, S., Braun, A., & Poeppel, D. (2005). Hierarchical and asymmetric tem-
poral sensitivity in human auditory cortices. Nature Neuroscience, 8(3), 389–395.
Bonte, M., Hausfeld, L., Scharke, W., Valente, G., & Formisano, E. (2014). Task-dependent
decoding of speaker and vowel identity from auditory cortical response patterns. Journal of
Neuroscience, 34(13), 4548–4557.
Bowling, D. L., & Purves, D. (2015). A biological rationale for musical consonance. Proceedings
Brandt, A. K., Slevc, L. R., & Gebrian, M. (2012). Music and early language acquisition. Frontiers
in Psychology, 3, 327.
Bregman, A. S. (1990). Auditory scene analysis: The perceptual organization of sound.
Cambridge, MA: MIT Press.
Bregman, A. S., & Campbell, J. (1971). Primary auditory stream segregation and perception of
order in rapid sequences of tones. Journal of Experimental Psychology, 89(2), 244–249.
Brown, S., & Jordania, J. (2011). Universals in the world’s musics. Psychology of Music, 41(2),
229–248.
Brown, S., Martinez, M. J., & Parsons, L. M. (2006). Music and language side by side in the
brain: A PET study of the generation of melodies and sentences. The European Journal of
Neuroscience, 23(10), 2791–2803.
Burns, E. M., & Campbell, S. L. (1994). Frequency and frequency-ratio resolution by possessors
of absolute and relative pitch: Examples of categorical perception? The Journal of the
Acoustical Society of America, 96(5), 2704–2719.
Burns, E. M., & Ward, W. D. (1978). Categorical perception—phenomenon or epiphenom-
enon: Evidence from experiments in the perception of melodic musical intervals. The
Journal of the Acoustical Society of America, 63(2), 456–468.
Callan, D. E., Tsytsarev, V., Hanakawa, T., Callan, A. M., Katsuhara, M., Fukuyama, H., &
Turner, R. (2006). Song and speech: Brain regions involved with perception and covert pro-
duction. NeuroImage, 31(3), 1327–1342.
Carlson, T. A., Simmons, R. A., Kriegeskorte, N., & Slevc, L. R. (2014). The emergence of se-
mantic meaning in the ventral temporal pathway. Journal of Cognitive Neuroscience, 26(1),
120–131.
Carr, C. E., & Konishi, M. (1990). A circuit for detection of interaural time differences in the
brain stem of the barn owl. Journal of Neuroscience, 10(10), 3227–3246.
Chandrasekaran, B., Hornickel, J., Skoe, E., Nicol, T., & Kraus, N. (2009). Context-dependent
encoding in the human auditory brainstem relates to hearing speech in noise: Implications
for developmental dyslexia. Neuron, 64(3), 311–319.
Chandrasekaran, B., & Kraus, N. (2010). The scalp- recorded brainstem response to
speech: Neural origins and plasticity. Psychophysiology, 47(2), 236–246.
13(11), 1428–1432.
Chen, J. L., Penhune, V. B., & Zatorre, R. J. (2008a). Listening to musical rhythms recruits
motor regions of the brain. Cerebral Cortex, 18(12), 2844–2854.
Chen, J. L., Penhune, V. B., & Zatorre, R. J. (2008b). Moving on time: Brain network for
auditory-motor synchronization is modulated by rhythm complexity and musical training.
Cherry, E. C. (1953). Some experiments on the recognition of speech, with one and with two
ears. The Journal of the Acoustical Society of America, 25(5), 975–979.
Chi, T., Ru, P., & Shamma, S. A. (2005). Multiresolution spectrotemporal analysis of complex
sounds. The Journal of the Acoustical Society of America, 118(2), 887–906.
Clarkson, M. G., & Clifton, R. K. (1985). Infant pitch perception: Evidence for responding
to pitch categories and the missing fundamental. The Journal of the Acoustical Society of
America, 77(4), 1521–1528.
Coffey, E. B., Herholz, S. C., Chepesiuk, A. M., Baillet, S., & Zatorre, R. J. (2016). Cortical
contributions to the auditory frequency-following response revealed by MEG. Nature
Communications, 7, 11070.
Cohen, Y. E., Hauser, M. D., & Russ, B. E. (2006). Spontaneous processing of abstract catego-
rical information in the ventrolateral prefrontal cortex. Biology Letters, 2(2), 261–265.
Cohen, Y. E., Russ, B. E., Gifford, G. W., Kiringoda, R., & MacLean, K. A. (2004). Selectivity
for the spatial and nonspatial attributes of auditory stimuli in the ventrolateral prefrontal
cortex. Journal of Neuroscience, 24(50), 11307–11316.
translation: FMRI decoding of spoken words in bilinguals reveals language-independent se-
mantic representations in anterior temporal lobe. Journal of Neuroscience, 34(1), 332–338.
Creel, S. C., & Bregman, M. R. (2011). How talker identity relates to language processing.
Language and Linguistics Compass, 5(5), 190–204.
Cross, I. (2007). Music and cognitive evolution. In R. I. M. Dunbar & L. Barrett (Eds.), Oxford
handbook of evolutionary psychology (pp. 649–667). Oxford: Oxford University Press.
Da Costa, S., van der Zwaag, W., Marques, J. P., Frackowiak, R. S., Clarke, S., & Saenz, M. (2011).
Human primary auditory cortex follows the shape of Heschl’s gyrus. Journal of Neuroscience,
31(40), 14067–14075.
Darwin, C. (1871). The descent of man, and selection in relation to sex. London: John Murray.
Deutsch, D., Henthorn, T., & Lapidis, R. (2011). Illusory transformation from speech to song.
The Journal of the Acoustical Society of America, 129(4), 2245–2252.
DeWitt, I., & Rauschecker, J. P. (2012). Phoneme and word recognition in the auditory ventral
stream. Proceedings of the National Academy of Sciences, 109(8), E505–514.
Diamond, A. (2013). Executive functions. Annual Review of Psychology, 64, 135.
Ding, N., Melloni, L., Zhang, H., Tian, X., & Poeppel, D. (2016). Cortical tracking of hierar-
chical linguistic structures in connected speech. Nature Neuroscience, 19(1), 158–164.
Ding, N., Patel, A., Chen, L., Butler, H., Luo, C., & Poeppel, D. (2017). Temporal modulations in
speech and music. Neuroscience & Biobehavioral Reviews 81, 181–187.
Ding, N., & Simon, J. Z. (2012). Emergence of neural encoding of auditory objects while lis-
tening to competing speakers. Proceedings of the National Academy of Sciences, 109(29),
11854–11859.
Ding, N., & Simon, J. Z. (2013). Adaptive temporal encoding leads to a background-insensitive
cortical representation of speech. Journal of Neuroscience, 33(13), 5728–5735.
Doelling, K. B., & Poeppel, D. (2015). Cortical entrainment to music and its modulation by
expertise. Proceedings of the National Academy of Sciences, 112(45), E6233-E6242.
Dooling, R. J., & Brown, S. D. (1990). Speech perception by budgerigars (Melopsittacus

undulatus): Spoken vowels. Perception & Psychophysics, 47(6), 568–574.
Du, Y., Buchsbaum, B. R., Grady, C. L., & Alain, C. (2014). Noise differentially impacts pho-
neme representations in the auditory and speech motor systems. Proceedings of the National
Academy of Sciences, 111(19), 7126–7 131.
Eimas, P. D. (1963). The relation between identification and discrimination along speech and
non-speech continua. Language and Speech, 6(4), 206–217.
Elbert, T., Pantev, C., Wienbruch, C., Rockstroh, B., & Taub, E. (1995). Increased cortical repre-
sentation of the fingers of the left hand in string players. Science, 270(5234), 305–307.
Elliott, T. M., Hamilton, L. S., & Theunissen, F. E. (2013). Acoustic structure of the five per-
ceptual dimensions of timbre in orchestral instrument tones. The Journal of the Acoustical
Society of America, 133(1), 389–404.
Elliott, T. M., & Theunissen, F. E. (2009). The modulation transfer function for speech intelligi-
bility. PLoS Computational Biology, 5(3), e1000302.
Falk, S., Rathcke, T., & Dalla Bella, S. (2014). When speech sounds like music. Journal of
Experimental Psychology. Human Perception and Performance, 40(4), 1491–1506.
Farbood, M. M., Heeger, D. J., Marcus, G., Hasson, U., & Lerner, Y. (2015). The neural pro-
cessing of hierarchical structure in music and speech at different timescales. Frontiers in
Fedorenko, E., Behr, M. K., & Kanwisher, N. (2011). Functional specificity for high-level lin-
guistic processing in the human brain. Proceedings of the National Academy of Sciences,
108(39), 16428–16433.
Fedorenko, E., Duncan, J., & Kanwisher, N. (2012). Language-selective and domain-general re-
gions lie side by side within Broca’s area. Current Biology, 22(21), 2059–2062.
Fedorenko, E., McDermott, J. H., Norman-Haignere, S., & Kanwisher, N. (2012). Sensitivity to
musical structure in the human brain. Journal of Neurophysiology, 108(12), 3289–3300.
Fedorenko, E., Patel, A., Casasanto, D., Winawer, J., & Gibson, E. (2009). Structural integra-
tion in language and music: Evidence for a shared system. Memory & Cognition, 37(1),  1–9.
Fishman, Y. I., Arezzo, J. C., & Steinschneider, M. (2004). Auditory stream segregation in
monkey auditory cortex: Effects of frequency separation, presentation rate, and tone dura-
tion. The Journal of the Acoustical Society of America, 116(3), 1656–1670.
Formisano, E., De Martino, F., Bonte, M., & Goebel, R. (2008). “Who” is saying “what”? Brain-
based decoding of human voice and speech. Science, 322(5903), 970–973.
Formisano, E., Kim, D. S., Di Salle, F., van de Moortele, P. F., Ugurbil, K., & Goebel, R. (2003).
Mirror-symmetric tonotopic maps in human primary auditory cortex. Neuron, 40(4),
859–869.
Fritz, J. B., David, S. V., Radtke-Schuller, S., Yin, P., & Shamma, S. A. (2010). Adaptive, behav-
iorally gated, persistent encoding of task-relevant auditory information in ferret frontal
Fritz, J. B., Elhilali, M., David, S. V., & Shamma, S. A. (2007). Auditory attention: Focusing the
searchlight on sound. Current Opinion in Neurobiology, 17(4), 437–455.
Fritz, J. B., Elhilali, M., & Shamma, S. A. (2005). Differential dynamic plasticity of A1 receptive
fields during multiple spectral tasks. Journal of Neuroscience, 25(33), 7623–7635.
Fritz, J., Shamma, S., Elhilali, M., & Klein, D. (2003). Rapid task- related plasticity of
spectrotemporal receptive fields in primary auditory cortex. Nature Neuroscience, 6(11),
1216–1223.
Fujioka, T., Trainor, L. J., Large, E. W., & Ross, B. (2009). Beta and gamma rhythms in human
auditory cortex during musical beat processing. Annals of the New York Academy of Sciences,
1169(1),  89–92.
Fujioka, T., Trainor, L. J., Large, E. W., & Ross, B. (2012). Internalized timing of isochronous
sounds is represented in neuromagnetic beta oscillations. Journal of Neuroscience, 32(5),
1791–1802.
Galbraith, G. C., Arbagey, P. W., Branski, R., Comerci, N., & Rector, P. M. (1995). Intelligible
speech encoded in the human brain stem frequency-following response. Neuroreport, 6(17),
2363–2367.
Gaser, C., & Schlaug, G. (2003). Brain structures differ between musicians and non-musicians.
Geiser, E., Zaehle, T., Jancke, L., & Meyer, M. (2008). The neural correlate of speech rhythm as
evidenced by metrical speech processing. Journal of Cognitive Neuroscience, 20(3), 541–552.
Ghitza, O. (2012). On the role of theta-driven syllabic parsing in decoding speech: Intelligibility
of speech with a manipulated modulation spectrum. Frontiers in Psychology, 3, 238.
Giordano, B. L., McAdams, S., Zatorre, R. J., Kriegeskorte, N., & Belin, P. (2012). Abstract
encoding of auditory objects in cortical activity patterns. Cerebral Cortex, 23(9), 2025–2037.
Giordano, B. L., Pernet, C., Charest, I., Belizaire, G., Zatorre, R. J., & Belin, P. (2014). Automatic
domain-general processing of sound source identity in the left posterior middle frontal
gyrus. Cortex, 58, 170–185.
Giraud, A. L., & Poeppel, D. (2012). Cortical oscillations and speech processing: Emerging
computational principles and operations. Nature Neuroscience, 15(4), 511–517.
Golestani, N., Molko, N., Dehaene, S., LeBihan, D., & Pallier, C. (2007). Brain structure predicts
the learning of foreign speech sounds. Cerebral Cortex, 17(3), 575–582.
Golestani, N., Price, C. J., & Scott, S. K. (2011). Born with an ear for dialects? Structural plas-
ticity in the expert phonetician brain. Journal of Neuroscience, 31(11), 4213–4220.
Grahn, J. A. (2009). The role of the basal ganglia in beat perception. Annals of the New York
Academy of Sciences, 1169(1),  35–45.
Grahn, J. A., & Brett, M. (2007). Rhythm and beat perception in motor areas of the brain.
Grahn, J. A., & Rowe, J. B. (2009). Feeling the beat: Premotor and striatal interactions in musicians
and nonmusicians during beat perception. The Journal of Neuroscience, 29(23), 7540–7548.
Granroth-Wilding, M., & Steedman, M. (2014). A robust parser-interpreter for jazz chord
sequences. Journal of New Music Research, 43(4), 355–374.
Griffiths, T. D., Kumar, S., Sedley, W., Nourski, K. V., Kawasaki, H., . . . Howard, M. A. (2010).
Direct recordings of pitch responses from human auditory cortex. Current Biology, 20(12),
1128–1132.
Griffiths, T. D., & Warren, J. D. (2002). The planum temporale as a computational hub. Trends
in Neurosciences, 25(7), 348–353.
Griffiths, T. D., & Warren, J. D. (2004). What is an auditory object? Nature Reviews
Grothe, B. (2003). New roles for synaptic inhibition in sound localization. Nature Reviews
Hackett, T. A., Preuss, T. M., & Kaas, J. H. (2001). Architectonic identification of the core re-
gion in auditory cortex of macaques, chimpanzees, and humans. The Journal of Comparative
Neurology, 441(3), 197–222.
Hailstone, J. C., Crutch, S. J., Vestergaard, M. D., Patterson, R. D., & Warren, J. D. (2010).
Progressive associative phonagnosia: A neuropsychological analysis. Neuropsychologia,
48(4), 1104–1114.
Halpern, A. R., & Zatorre, R. J. (1979). Identification, discrimination, and selective adaptation
of simultaneous musical intervals. The Journal of the Acoustical Society of America, 65(S1),
384–395.
Hannon, E. E., & Trainor, L. J. (2007). Music acquisition: Effects of enculturation and formal
training on development. Trends in Cognitive Sciences, 11(11), 466–472.
Hari, R., & Salmelin, R. (1997). Human cortical oscillations: A neuromagnetic view through
the skull. Trends in Neurosciences, 20(1),  44–49.
Hauser, M. D., Chomsky, N., & Fitch, W. T. (2002). The faculty of language: What is it, who has
it, and how did it evolve? Science, 298(5598), 1569–1579.
Heffner, C. C., & Slevc, L. R. (2015). Prosodic structure as a parallel to musical structure.
Heffner, H., & Whitfield, I. C. (1976). Perception of the missing fundamental by cats. The
Journal of the Acoustical Society of America, 59(4), 915–919.
Herholz, S. C., & Zatorre, R. J. (2012). Musical training as a framework for brain plas-
ticity: Behavior, function, and structure. Neuron, 76(3), 486–502.
Hillenbrand, J., Getty, L. A., Clark, M. J., & Wheeler, K. (1995). Acoustic characteris-
tics of American English vowels. The Journal of the Acoustical Society of America, 97(5),
3099–3111.
Hoch, L., Poulin-Charronnat, B., & Tillmann, B. (2011). The influence of task-irrelevant
music on language processing: Syntactic and semantic structures. Frontiers in Psychology,
2, 112.
Holdgraf, C. R., De Heer, W., Pasley, B., Rieger, J., Crone, N., Lin, J. J., . . . Theunissen, F. E.
(2016). Rapid tuning shifts in human auditory cortex enhance speech intelligibility. Nature
Communications, 7, 13654.
Hoormann, J., Falkenstein, M., Hohnsbein, J., & Blanke, L. (1992). The human frequency-
following response (FFR): Normal variability and relation to the click-evoked brainstem re-
sponse. Hearing Research, 59(2), 179–188.
Hornickel, J., Anderson, S., Skoe, E., Yi, H. G., & Kraus, N. (2012). Subcortical representation of
speech fine structure relates to reading ability. Neuroreport, 23(1),  6–9.
Hornickel, J., Chandrasekaran, B., Zecker, S., & Kraus, N. (2011). Auditory brainstem measures
predict reading and speech-in-noise perception in school-aged children. Behavioural Brain
Research, 216(2), 597–605.
Hsu, N. S., Jaeggi, S. M., & Novick, J. M. (2017). A common neural hub resolves syntactic
and non-syntactic conflict through cooperation with task-specific networks. Brain and
Language, 166,  63–77.
Hullett, P. W., Hamilton, L. S., Mesgarani, N., Schreiner, C. E., & Chang, E. F. (2016). Human
superior temporal gyrus organization of spectrotemporal modulation tuning derived from
speech stimuli. Journal of Neuroscience, 36(6), 2014–2026.
Humphries, C., Liebenthal, E., & Binder, J. R. (2010). Tonotopic organization of human audi-
tory cortex. NeuroImage, 50(3), 1202–1211.
Huron, D. (2006). Sweet anticipation: Music and the psychology of expectation. Cambridge:
MIT Press .
Huth, A. G., de Heer, W. A., Griffiths, T. L., Theunissen, F. E., & Gallant, J. L. (2016). Natural
speech reveals the semantic maps that tile human cerebral cortex. Nature, 532(7600),
453–458.
Hyde, K. L., Lerch, J., Norton, A., Forgeard, M., Winner, E., Evans, A. C., & Schlaug, G. (2009).
Musical training shapes structural brain development. Journal of Neuroscience, 29(10),
3019–3025.
Janata, P., Birk, J. L., Van Horn, J. D., Leman, M., Tillmann, B., & Bharucha, J. J. (2002). The
cortical topography of tonal structures underlying Western music. Science, 298(5601),
2167–2170.
Janata, P., & Grafton, S. T. (2003). Swinging in the brain: Shared neural substrates for behaviors
related to sequencing and music. Nature Neuroscience, 6(7), 682–687.
Janata, P., Tillmann, B., & Bharucha, J. J. (2002). Listening to polyphonic music recruits
domain-general attention and working memory circuits. Cognitive, Affective, & Behavioral
January, D., Trueswell, J. C., & Thompson-Schill, S. L. (2009). Co-localization of Stroop and
syntactic ambiguity resolution in Broca’s area: Implications for the neural basis of sentence
processing. Journal of Cognitive Neuroscience, 21(12), 2434–2444.
Jessen, M. (2008). Forensic phonetics. Language and Linguistics Compass, 2(4), 671–7 11.
Kaas, J. H., & Hackett, T. A. (2000). Subdivisions of auditory cortex and processing streams in
primates. Proceedings of the National Academy of Sciences, 97(22), 11793–11799.
Kanold, P. O., Nelken, I., & Polley, D. B. (2014). Local versus global scales of organization in au-
ditory cortex. Trends in Neurosciences, 37(9), 502–510.
Kanwisher, N., McDermott, J., & Chun, M. M. (1997). The fusiform face area: A module in
human extrastriate cortex specialized for face perception. Journal of Neuroscience, 17(11),
4302–4311.
Klein, M. E., & Zatorre, R. J. (2011). A role for the right superior temporal sulcus in categorical
perception of musical chords. Neuropsychologia, 49(5), 878–887.
Klein, M. E., & Zatorre, R. J. (2015). Representations of invariant musical categories are de-
codable by pattern analysis of locally distributed BOLD responses in superior temporal and
intraparietal sulci. Cerebral Cortex, 25(7), 1947–1957.
Kluender, K. R., Diehl, R. L., & Killeen, P. R. (1987). Japanese quail can learn phonetic
categories. Science, 237(4819), 1195–1197.
Knecht, S., Dräger, B., Deppe, M., Bobe, L., Lohmann, H., Flöel, A., . . . Henningsen, H. (2000).
Handedness and hemispheric language dominance in healthy humans. Brain, 123(12),
2512–2518.
Koelsch, S. (2005). Neural substrates of processing syntax and semantics in music. Current
Opinion in Neurobiology, 15(2), 207–212.
Koelsch, S., & Friederici, A. D. (2003). Toward the neural basis of processing structure in
music. Annals of the New York Academy of Sciences, 999(1),  15–28.
Koelsch, S., Gunter, T. C., Cramon, D. Y. V., Zysset, S., Lohmann, G., & Friederici, A. D. (2002).
Bach speaks: A cortical “language-network” serves the processing of music. NeuroImage,
17(2), 956–966.
Koelsch, S., Gunter, T. C., Friederici, A. D., & Schröger, E. (2000). Brain indices of music pro-
cessing: “Nonmusicians” are musical. Journal of Cognitive Neuroscience, 12(3), 520–541.
Koelsch, S., Gunter, T. C., Wittfoth, M., & Sammler, D. (2005). Interaction between syntax pro-
cessing in language and in music: An ERP study. Journal of Cognitive Neuroscience, 17(10),
1565–1577.
Koelsch, S., Kasper, E., Sammler, D., Schulze, K., Gunter, T., & Friederici, A. D. (2004). Music,
language and meaning: Brain signatures of semantic processing. Nature Neuroscience, 7(3),
302–307.
Koelsch, S., Rohrmeier, M., Torrecuso, R., & Jentschke, S. (2013). Processing of hierarchical
syntactic structure in music. Proceedings of the National Academy of Sciences, 110(38),
15443–15448.
Koelsch, S., & Siebel, W. A. (2005). Towards a neural basis of music perception. Trends in
Köster, O., Hess, M. M., Schiller, N. O., & Künzel, H. J. (1998). The correlation between audi-
tory speech sensitivity and speaker recognition ability. Forensic Linguistics: The International
Journal of Speech, Language and the Law, 5,  22–32.
Kraus, N., & Chandrasekaran, B. (2010). Music training for the development of auditory skills.
Nature Reviews Neuroscience, 11(8), 599–605.
Kraus, N., Slater, J., Thompson, E. C., Hornickel, J., Strait, D. L., Nicol, T., & White-Schwoch, T.
(2014). Music enrichment programs improve the neural encoding of speech in at-risk chil-
dren. Journal of Neuroscience, 34(36), 11913–11918.
Kraus, N., Strait, D. L., & Parbery-Clark, A. (2012). Cognitive factors shape brain networks for
auditory skills: Spotlight on auditory working memory. Annals of the New York Academy of
Sciences, 1252, 100–107.
Krishnan, A., Xu, Y., Gandour, J., & Cariani, P. (2005). Encoding of pitch in the human brain-
stem is sensitive to language experience. Cognitive Brain Research, 25(1), 161–168.
Krumhansl, C. L. (2002). Music: A link between cognition and emotion. Current Directions in
Psychological Science, 11(2),  45–50.
Krumhansl, C. L., & Iverson, P. (1992). Perceptual interactions between musical pitch and
timbre. Journal of Experimental Psychology: Human Perception and Performance, 18(3),
739–751.
Krumhansl, C. L., & Kessler, E. J. (1982). Tracing the dynamic changes in perceived tonal or-
ganization in a spatial representation of musical keys. Psychological Review, 89(4), 334–368.
Kuhl, P. K., & Miller, J. D. (1975). Speech perception by the chinchilla: Voiced-voiceless distinc-
tion in alveolar plosive consonants. Science, 190(4209),  69–72.
Kunert, R., & Slevc, L. R. (2015). A commentary on: “Neural overlap in processing music and
speech.” Frontiers in Human Neuroscience, 9, 330.
Kunert, R., Willems, R. M., Casasanto, D., Patel, A. D., & Hagoort, P. (2015). Music and lan-
guage syntax interact in Broca’s area: An fMRI study. PloS One, 10(11), e0141069.
Kutas, M., & Federmeier, K. D. (2011). Thirty years and counting: Finding meaning in the N400
component of the event related brain potential (ERP). Annual Review of Psychology, 62, 621.
LaCroix, A. N., Diaz, A. F., & Rogalsky, C. (2015). The relationship between the neural
computations for speech and music perception is context-dependent: An activation likeli-
hood estimate study. Frontiers in Psychology, 6, 1138.
Large, E. W., & Snyder, J. S. (2009). Pulse and meter as neural resonance. Annals of the New York
Academy of Sciences, 1169(1),  46–57.
Latinus, M., & Belin, P. (2011). Human voice perception. Current Biology, 21(4), R143–R145.
Lau, E. F., Phillips, C., & Poeppel, D. (2008). A cortical network for semantics: (De)
constructing the N400. Nature Reviews Neuroscience, 9(12), 920–933.
Leaver, A. M., & Rauschecker, J. P. (2010). Cortical representation of natural complex

sounds: Effects of acoustic features and auditory object category. Journal of Neuroscience,
30(22), 7604–7612.
Leaver, A. M., & Rauschecker, J. P. (2016). Functional topography of human auditory cortex.
Lee, Y. S., Turkeltaub, P., Granger, R., & Raizada, R. D. (2012). Categorical speech processing
in Broca’s area: An fMRI study using multivariate pattern-based analysis. Journal of
Neuroscience, 32(11), 3942–3948.
Leonard, M. K., Baud, M. O., Sjerps, M. J., & Chang, E. F. (2016). Perceptual restoration of
masked speech in human cortex. Nature Communications, 7, 13619.
Lerdahl, F., & Jackendoff, R. (1983). An overview of hierarchical structure in music. Music
Perception: An Interdisciplinary Journal, 1(2), 229–252.
Lerner, Y., Honey, C. J., Silbert, L. J., & Hasson, U. (2011). Topographic mapping of a hier-
archy of temporal receptive windows using a narrated story. Journal of Neuroscience, 31(8),
2906–2915.
Levitin, D. J., & Rogers, S. E. (2005). Absolute pitch: Perception, coding, and controversies.
Trends in Cognitive Sciences, 9(1),  26–33.
Lewis, J. W., Brefczynski, J. A., Phinney, R. E., Janik, J. J., & DeYoe, E. A. (2005). Distinct cor-
tical pathways for processing tool versus animal sounds. Journal of Neuroscience, 25(21),
5148–5158.
Lewis, J. W., Talkington, W. J., Walker, N. A., Spirou, G. A., Jajosky, A., Frum, C., & Brefczynski-
Lewis, J. A. (2009). Human cortical organization for processing vocalizations indicates
representation of harmonic structure as a signal attribute. Journal of Neuroscience, 29(7),
2283–2296.
Liberman, A. M., Harris, K. S., Hoffman, H. S., & Griffith, B. C. (1957). The discrimination of
speech sounds within and across phoneme boundaries. Journal of Experimental Psychology,
54(5), 358–368.
Liebenthal, E., Binder, J. R., Spitzer, S. M., Possing, E. T., & Medler, D. A. (2005). Neural
substrates of phonemic perception. Cerebral Cortex, 15(10), 1621–1631.
Longuet-Higgins, H. C. (1976). Perception of melodies. Nature, 263, 646–653.
Loui, P., Wu, E. H., Wessel, D. L., & Knight, R. T. (2009). A generalized mechanism for percep-
tion of pitch patterns. Journal of Neuroscience, 29(2), 454–459.
Maess, B., Koelsch, S., Gunter, T. C., & Friederici, A. D. (2001). Musical syntax is processed in
Broca’s area: An MEG study. Nature Neuroscience, 4(5), 540–545.
Mattingly, I. G., Liberman, A. M., Syrdal, A. K., & Halwes, T. (1971). Discrimination in speech
and nonspeech modes. Cognitive Psychology, 2(2), 131–157.
McAdams, S. (1982). Spectral fusion and the creation of auditory images. In M. Clynes (Ed.),
Music, mind, and brain (pp. 279–298). New York: Plenum.
McAdams, S., & Bregman, A. S. (1979). Hearing musical streams. Computer Music Journal,
3(4),  26–43.
McAdams, S., & Giordano, B. L. (2009). The perception of musical timbre. In S. Hallam,
I. Cross, & M. Thaut (Eds.), The Oxford handbook of music psychology (pp. 72–80).
McAdams, S., Winsberg, S., Donnadieu, S., De Soete, G., & Krimphoff, J. (1995). Perceptual
scaling of synthesized musical timbres: Common dimensions, specificities, and latent sub-
ject classes. Psychological Research, 58(3), 177–192.
McDermott, J., & Hauser, M. (2005). The origins of music: Innateness, uniqueness, and evolu-
tion. Music Perception: An Interdisciplinary Journal, 23(1),  29–59.
McGettigan, C., Evans, S., Rosen, S., Agnew, Z. K., Shah, P., & Scott, S. K. (2012). An application
of univariate and multivariate approaches in fMRI to quantifying the hemispheric laterali-
zation of acoustic and linguistic processes. Journal of Cognitive Neuroscience, 24(3), 636–652.
McGettigan, C., & Scott, S. K. (2012). Cortical asymmetries in speech perception: What’s
wrong, what’s right and what’s left?. Trends in Cognitive Sciences, 16(5), 269–276.
Melara, R. D., & Marks, L. E. (1990). Interaction among auditory dimensions: Timbre, pitch,
and loudness. Perception & Psychophysics, 48(2), 169–178.
Menon, V., Levitin, D. J., Smith, B. K., Lembke, A., Krasnow, B. D., Glazer, D., . . . McAdams,
S. (2002). Neural correlates of timbre change in harmonic sounds. NeuroImage, 17(4),
1742–1754.
Mesgarani, N., & Chang, E. F. (2012). Selective cortical representation of attended speaker in
multi-talker speech perception. Nature, 485(7397), 233–236.
Mesgarani, N., Cheung, C., Johnson, K., & Chang, E. F. (2014). Phonetic feature encoding in
human superior temporal gyrus. Science, 343(6174), 1006–1010.
Mesgarani, N., David, S. V., Fritz, J. B., & Shamma, S. A. (2008). Phoneme representation and
classification in primary auditory cortex. The Journal of the Acoustical Society of America,
123(2), 899–909.
Meyer, L. B. (1956). Emotion and meaning in music. Chicago: University of Chicago Press.
Minati, L., Rosazza, C., D’Incerti, L., Pietrocini, E., Valentini, L., Scaioli, V., . . . Bruzzone, M.
G. (2008). FMRI/ERP of musical syntax: Comparison of melodies and unstructured note
sequences. Neuroreport, 19(14), 1381–1385.
Mishkin, M., Ungerleider, L. G., & Macko, K. A. (1983). Object vision and spatial vision: Two
cortical pathways. Trends in Neurosciences, 6, 414–417.
Mitchell, T. M., Shinkareva, S. V., Carlson, A., Chang, K. M., Malave, V. L., Mason, R. A., &
Just, M. A. (2008). Predicting human brain activity associated with the meanings of nouns.
Science, 320(5880), 1191–1195.
Mithen, S. (2005). The singing Neanderthals: The origins of music, language, mind and body.
London: Weidenfeld & Nicolson.
Miyake, A., & Friedman, N. P. (2012). The nature and organization of individual differences
in executive functions four general conclusions. Current Directions in Psychological Science,
21(1),  8–14.
Miyazaki, K. (1988). Musical pitch identification by absolute pitch possessors. Perception &
Psychophysics, 44(6), 501–512.
Moerel, M., De Martino, F., & Formisano, E. (2012). Processing of natural sounds in human
auditory cortex: Tonotopy, spectral tuning, and relation to voice sensitivity. Journal of
Neuroscience, 32(41), 14205–14216.
Münte, T. F., Altenmüller, E., & Jäncke, L. (2002). The musician’s brain as a model of
neuroplasticity. Nature Reviews Neuroscience, 3(6), 473–478.
Musacchia, G., Sams, M., Skoe, E., & Kraus, N. (2007). Musicians have enhanced subcor-
tical auditory and audiovisual processing of speech and music. Proceedings of the National
Nolan, F. (1991). Forensic phonetics. Journal of Linguistics, 27(2), 483–493.
Norman-Haignere, S., Kanwisher, N., & McDermott, J. H. (2013). Cortical pitch regions in
humans respond primarily to resolved harmonics and are located in specific tonotopic re-
gions of anterior auditory cortex. Journal of Neuroscience, 33(50), 19451–19469.
Norman-Haignere, S., Kanwisher, N. G., & McDermott, J. H. (2015). Distinct cortical pathways
for music and speech revealed by hypothesis-free voxel decomposition. Neuron, 88(6),
1281–1296.
Occelli, F., Suied, C., Pressnitzer, D., Edeline, J. M., & Gourévitch, B. (2016). A neural substrate
for rapid timbre recognition? Neural and behavioral discrimination of very brief acoustic
vowels. Cerebral Cortex, 26(6), 2483–2496.
Oechslin, M. S., Van De Ville, D., Lazeyras, F., Hauert, C. A., & James, C. E. (2013). Degree
of musical expertise modulates higher order brain functioning. Cerebral Cortex, 23(9),
2213–2224.
Okada, K., Rong, F., Venezia, J., Matchin, W., Hsieh, I. H., Saberi, K., . . . Hickok, G. (2010).
Hierarchical organization of human auditory cortex: Evidence from acoustic invariance in
the response to intelligible speech. Cerebral Cortex, 20(10), 2486–2495.
Overath, T., McDermott, J. H., Zarate, J. M., & Poeppel, D. (2015). The cortical analysis
of speech- specific temporal structure revealed by responses to sound quilts. Nature
Pantev, C., Oostenveld, R., Engelien, A., Ross, B., Roberts, L. E., & Hoke, M. (1998). Increased
auditory cortical representation in musicians. Nature, 392(6678), 811–814.
Parbery-Clark, A., Skoe, E., & Kraus, N. (2009). Musical experience limits the degradative
effects of background noise on the neural processing of sound. Journal of Neuroscience,
29(45), 14100–14107.
Parbery-Clark, A., Strait, D. L., Anderson, S., Hittner, E., & Kraus, N. (2011). Musical experi-
ence and the aging auditory system: Implications for cognitive abilities and hearing speech
in noise. PloS One, 6(5), e18082.
Parbery-Clark, A., Tierney, A., Strait, D. L., & Kraus, N. (2012). Musicians have fine-tuned
neural distinction of speech syllables. Neuroscience, 219, 111–119.
Pasley, B. N., David, S. V., Mesgarani, N., Flinker, A., Shamma, S. A., Crone, N. E., . . . Chang, E.
F. (2012). Reconstructing speech from human auditory cortex. PLoS Biology, 10(1), e1001251.
Patel, A. D. (2003). Language, music, syntax and the brain. Nature Neuroscience, 6(7), 674–681.
Patel, A. D. (2006). Musical rhythm, linguistic rhythm, and human evolution. Music
Perception: An Interdisciplinary Journal, 24(1), 99–104.
Patel, A. D. (2008). Music, language, and the brain. New York: Oxford University Press.
Patel, A. D. (2011). Why would musical training benefit the neural encoding of speech? The
OPERA hypothesis. Frontiers in Psychology, 2, 142.
Patel, A. D. (2012). The OPERA hypothesis: Assumptions and clarifications. Annals of the
New York Academy of Sciences, 1252, 124–128.
Patel, A. D. (2014). Can nonlinguistic musical training change the way the brain processes
speech? The expanded OPERA hypothesis. Hearing Research, 308, 98–108.
Patel, A. D., Gibson, E., Ratner, J., Besson, M., & Holcomb, P. J. (1998). Processing syntactic
relations in language and music: An event-related potential study. Journal of Cognitive
Patel, A. D., Iversen, J. R., Bregman, M. R., & Schulz, I. (2009a). Experimental evidence for
synchronization to a musical beat in a nonhuman animal. Current Biology, 19(10), 827–830.
Patel, A. D., Iversen, J. R., Bregman, M. R., & Schulz, I. (2009b). Studying synchronization to
a musical beat in nonhuman animals. Annals of the New York Academy of Sciences, 1169(1),
459–469.
Patil, K., Pressnitzer, D., Shamma, S., & Elhilali, M. (2012). Music in our ears: The biological
bases of musical timbre perception. PLoS Computational Biology, 8(11), e1002759.
Patterson, K., Nestor, P. J., & Rogers, T. T. (2007). Where do you know what you know? The rep-
resentation of semantic knowledge in the human brain. Nature Reviews Neuroscience, 8(12),
976–987.
Patterson, R. D., Uppenkamp, S., Johnsrude, I. S., & Griffiths, T. D. (2002). The processing of
temporal pitch and melody information in auditory cortex. Neuron, 36(4), 767–776.
Peelle, J. E., Johnsrude, I. S., & Davis, M. H. (2010). Hierarchical processing for speech in
human auditory cortex and beyond. Frontiers in Human Neuroscience, 4, 51.
Penagos, H., Melcher, R., & Oxenham, J. (2004). A neural representation of pitch salience in
nonprimary human auditory cortex revealed with functional magnetic resonance imaging.
Peretz, I. (1993). Auditory atonalia for melodies. Cognitive Neuropsychology, 10(1),  21–56.
Peretz, I., & Coltheart, M. (2003). Modularity of music processing. Nature Neuroscience, 6(7),
688–691.
Peretz, I., Vuvan, D., Lagrois, M. É., & Armony, J. L. (2015). Neural overlap in processing music
and speech. Philosophical Transactions of the Royal Society B, 370(1664), 20140090.
Perrodin, C., Kayser, C., Logothetis, N. K., & Petkov, C. I. (2011). Voice cells in the primate tem-
poral lobe. Current Biology, 21(16), 1408–1415.
Petkov, C. I., Kayser, C., Steudel, T., Whittingstall, K., Augath, M., & Logothetis, N. K. (2008). A
voice region in the monkey brain. Nature Neuroscience, 11(3), 367–374.
Potes, C., Gunduz, A., Brunner, P., & Schalk, G. (2012). Dynamics of electrocorticographic
(ecog) activity in human temporal and frontal cortical areas during music listening.
NeuroImage, 61(4), 841–848.
Pressnitzer, D., Suied, C., & Shamma, S. A. (2011). Auditory scene analysis: The sweet music of
ambiguity. Frontiers in Human Neuroscience, 5, 158.
primates illuminate human speech processing. Nature Neuroscience, 12(6), 718–724.
Rauschecker, J. P., & Tian, B. (2000). Mechanisms and streams for processing of what and where
in auditory cortex. Proceedings of the National Academy of Sciences, 97(22), 11800–11806.
Rauschecker, J. P., Tian, B., & Hauser, M. (1995). Processing of complex sounds in the macaque
nonprimary auditory cortex. Science, 268(5207), 111–114.
Rogalsky, C., Rong, F., Saberi, K., & Hickok, G. (2011). Functional anatomy of language and
music perception: Temporal and structural factors investigated using functional magnetic
resonance imaging. Journal of Neuroscience, 31(10), 3843–3852.
Rohrmeier, M. (2011). Towards a generative syntax of tonal harmony. Journal of Mathematics
and Music, 5(1),  35–53.
Ross, D., Choi, J., & Purves, D. (2007). Musical intervals in speech. Proceedings of the National
Russ, B. E., Ackelson, A. L., Baker, A. E., & Cohen, Y. E. (2008). Coding of auditory-stimulus
identity in the auditory nonspatial processing stream. Journal of Neurophysiology,
99(1),  87–95.
Saffran, J. R., Newport, E. L., & Aslin, R. N. (1996). Word segmentation: The role of distribu-
tional cues. Journal of Memory and Language, 35(4), 606–621.
Saffran, J. R., Newport, E. L., Aslin, R. N., Tunick, R. A., & Barrueco, S. (1997). Incidental lan-
guage learning: Listening (and learning) out of the corner of your ear. Psychological Science,
8(2), 101–105.
Sammler, D., Koelsch, S., Ball, T., Brandt, A., Grigutsch, M., Huppertz, H. J., . . . Friederici, A. D.
(2013). Co-localizing linguistic and musical syntax with intracranial EEG. NeuroImage, 64,
134–146.
Samson, F., Zeffiro, T. A., Toussaint, A., & Belin, P. (2011). Stimulus complexity and catego-
rical effects in human auditory cortex: An activation likelihood estimation meta-analysis.
Samson, S. (2003). Neuropsychological studies of musical timbre. Annals of the New York
Samson, S., & Zatorre, R. J. (1994). Contribution of the right temporal lobe to musical timbre
discrimination. Neuropsychologia, 32(2), 231–240.
Samson, S., Zatorre, R. J., & Ramsay, J. O. (2002). Deficits of musical timbre perception after
unilateral temporal-lobe lesion revealed with multidimensional scaling. Brain, 125(3),
511–523.
Santoro, R., Moerel, M., De Martino, F., Goebel, R., Ugurbil, K., Yacoub, E., & Formisano, E.
(2014). Encoding of natural sounds at multiple spectral and temporal resolutions in the
human auditory cortex. PLoS Computational Biology, 10(1), e1003412.
Santoro, R., Moerel, M., De Martino, F., Valente, G., Ugurbil, K., Yacoub, E., & Formisano, E.
(2017). Reconstructing the spectrotemporal modulations of real-life sounds from fMRI re-
sponse patterns. Proceedings of the National Academy of Sciences, 114(18), 4799–4804.
Savage, P. E., Brown, S., Sakai, E., & Currie, T. E. (2015). Statistical universals reveal the
structures and functions of human music. Proceedings of the National Academy of Sciences,
112(29), 8987–8992.
Schachner, A., Brady, T. F., Pepperberg, I. M., & Hauser, M. D. (2009). Spontaneous motor en-
trainment to music in multiple vocal mimicking species. Current Biology, 19(10), 831–836.
Scharinger, M., Idsardi, W. J., & Poe, S. (2011). A comprehensive three-dimensional cortical
map of vowel space. Journal of Cognitive Neuroscience, 23(12), 3972–3982.
Schneider, P., Scherg, M., Dosch, H. G., Specht, H. J., Gutschalk, A., & Rupp, A. (2002).
Morphology of Heschl’s gyrus reflects enhanced activation in the auditory cortex of
musicians. Nature Neuroscience, 5(7), 688–694.
Schneider, P., Sluming, V., Roberts, N., Scherg, M., Goebel, R., Specht, H. J., . . . Rupp, A. (2005).
Structural and functional asymmetry of lateral Heschl’s gyrus reflects pitch perception pref-
erence. Nature Neuroscience, 8(9), 1241–1247.
Schnupp, J. W., & Carr, C. E. (2009). On hearing with more than one ear: Lessons from evolu-
tion. Nature Neuroscience, 12(6), 692–697.
Schönwiesner, M., Rübsamen, R., & Von Cramon, D. Y. (2005). Hemispheric asymmetry for
spectral and temporal processing in the human antero-lateral auditory belt cortex. The
European Journal of Neuroscience, 22(6), 1521–1528.
Schönwiesner, M., & Zatorre, R. J. (2009). Spectro-temporal modulation transfer func-
tion of single voxels in the human auditory cortex measured with high-resolution fMRI.
Schulze, K., & Koelsch, S. (2012). Working memory for speech and music. Annals of the
New York Academy of Sciences, 1252, 229–236.
Schwartz, D. A., Howe, C. Q., & Purves, D. (2003). The statistical structure of human speech
sounds predicts musical universals. Journal of Neuroscience, 23(18), 7160–7 168.
Schweinberger, S. R., Kawahara, H., Simpson, A. P., Skuk, V. G., & Zäske, R. (2014). Speaker
perception. Wiley Interdisciplinary Reviews: Cognitive Science, 5(1),  15–25.
Seither-Preisler, A., Johnson, L., Krumbholz, K., Nobbe, A., Patterson, R., Seither, S., &
Lütkenhöner, B. (2007). Tone sequences with conflicting fundamental pitch and timbre
changes are heard differently by musicians and nonmusicians. Journal of Experimental
Psychology: Human Perception and Performance, 33(3), 743–751.
Shamma, S. A., Elhilali, M., & Micheyl, C. (2011). Temporal coherence and attention in audi-
tory scene analysis. Trends in Neurosciences, 34(3), 114–123.
Siegel, J. A., & Siegel, W. (1977a). Absolute identification of notes and intervals by musicians.
Perception & Psychophysics, 21(2), 143–152.
Siegel, J. A., & Siegel, W. (1977b). Categorical perception of tonal intervals: Musicians can’t tell
sharp from flat. Perception & Psychophysics, 21(5), 399–407.
Slevc, L. R., Davey, N. S., Buschkuehl, M., & Jaeggi, S. M. (2016). Tuning the mind: Exploring
the connections between musical ability and executive functions. Cognition, 152, 199–211.
Slevc, L. R., Faroqi-Shah, Y., Saxena, S., & Okada, B. M. (2016). Preserved processing of musical
structure in a person with agrammatic aphasia. Neurocase, 22(6), 505–511.
Slevc, L. R., & Okada, B. M. (2015). Processing structure in language and music: A case for
shared reliance on cognitive control. Psychonomic Bulletin & Review, 22(3), 637–652.
Slevc, L. R., & Patel, A. D. (2011). Meaning in music and language: Three key differences:
Comment on “Towards a neural basis of processing musical semantics” by Stefan Koelsch.
Physics of Life Reviews, 8(2), 110–111.
Slevc, L. R., Rosenberg, J. C., & Patel, A. D. (2009). Making psycholinguistics musical: Self-
paced reading time evidence for shared processing of linguistic and musical syntax.
Psychonomic Bulletin & Review, 16(2), 374–381.
Slevc, L. R., & Shell, A. R. (2014). Auditory agnosia. Handbook of Clinical Neurology, 129, 573–587.
Smith, J. C., Marsh, J. T., & Brown, W. S. (1975). Far-field recorded frequency-following
responses: Evidence for the locus of brainstem sources. Electroencephalography and Clinical
Staeren, N., Renvall, H., De Martino, F., Goebel, R., & Formisano, E. (2009). Sound categories
are represented as distributed patterns in the human auditory cortex. Current Biology, 19(6),
498–502.
Steele, C. J., Bailey, J. A., Zatorre, R. J., & Penhune, V. B. (2013). Early musical training and
white-matter plasticity in the corpus callosum: Evidence for a sensitive period. Journal of
Neuroscience, 33(3), 1282–1290.
Strait, D. L., Chan, K., Ashley, R., & Kraus, N. (2012). Specialization among the special-
ized: Auditory brainstem function is tuned in to timbre. Cortex, 48(3), 360–362.
Strait, D. L., Kraus, N., Skoe, E., & Ashley, R. (2009). Musical experience and neural effi-
ciency: Effects of training on subcortical processing of vocal expressions of emotion. The
European Journal of Neuroscience, 29(3), 661–668.
Sturm, I., Blankertz, B., Potes, C., Schalk, G., & Curio, G. (2014). ECoG high gamma activity
reveals distinct cortical representations of lyrics passages, harmonic and timbre-related
changes in a rock song. Frontiers in Human Neuroscience, 8, 798.
Takeya, R., Kameda, M., Patel, A. D., & Tanaka, M. (2017). Predictive and tempo-flexible syn-
chronization to a visual metronome in monkeys. Scientific Reports, 7, 6127.
Talkington, W. J., Rapuano, K. M., Hitt, L. A., Frum, C. A., & Lewis, J. W. (2012). Humans mim-
icking animals: A cortical hierarchy for human vocal communication sounds. Journal of
Neuroscience, 32(23), 8084–8093.
Theunissen, F. E., & Elie, J. E. (2014). Neural processing of natural sounds. Nature Reviews
Tierney, A., Dick, F., Deutsch, D., & Sereno, M. (2013). Speech versus song: Multiple pitch-
sensitive areas revealed by a naturally occurring musical illusion. Cerebral Cortex, 23(2),
249–254.
Tierney, A., & Kraus, N. (2013). The ability to move to a beat is linked to the consistency of
neural responses to sound. Journal of Neuroscience, 33(38), 14981–14988.
Tierney, A. T., Krizman, J., & Kraus, N. (2015). Music training alters the course of adolescent au-
ditory development. Proceedings of the National Academy of Sciences, 112(32), 10062–10067.
Tomlinson, R. W. W., & Schwarz, D. W. (1988). Perception of the missing fundamental in non-
human primates. The Journal of the Acoustical Society of America, 84(2), 560–565.
Town, S. M., Atilgan, H., Wood, K. C., & Bizley, J. K. (2015). The role of spectral cues in timbre
discrimination by ferrets and humans. The Journal of the Acoustical Society of America,
137(5), 2870–2883.
Trehub, S. E. (2003). The developmental origins of musicality. Nature Neuroscience, 6(7),
669–673.
Tsunada, J., & Cohen, Y. E. (2014). Neural mechanisms of auditory categorization: From across
brain areas to within local microcircuits. Frontiers in Neuroscience, 8, 161.
Tzounopoulos, T., & Kraus, N. (2009). Learning to encode timing: Mechanisms of plasticity in
the auditory brainstem. Neuron, 62(4), 463–469.
Van Lancker, D. R., Cummings, J. L., Kreiman, J., & Dobkin, B. H. (1988). Phonagnosia: A dis-
sociation between familiar and unfamiliar voices. Cortex, 24(2), 195–209.
Van Lancker, D. R., Kreiman, J., & Cummings, J. (1989). Voice perception
deficits: Neuroanatomical correlates of phonagnosia. Journal of Clinical and Experimental
Vanden Bosch der Nederlanden, C. M., Hannon, E. E., & Snyder, J. S. (2015a). Everyday mu-
sical experience is sufficient to perceive the speech-to-song illusion. Journal of Experimental
Psychology: General, 144(2), e43–e49.
Vanden Bosch der Nederlanden, C. M., Hannon, E. E., & Snyder, J. S. (2015b). Finding the
music of speech: Musical knowledge influences pitch processing in speech. Cognition, 143,
135–140.
Von Békésy, G. (1972). The missing fundamental and periodicity detection in hearing. The
Journal of the Acoustical Society of America, 51(2B), 631–637.
Von Kriegstein, K., Eger, E., Kleinschmidt, A., & Giraud, A. L. (2003). Modulation of neural
responses to speech by directing attention to voices or verbal content. Cognitive Brain
Research, 17(1),  48–55.
Von Kriegstein, K., & Giraud, A. L. (2004). Distinct functional substrates along the right supe-
rior temporal sulcus for the processing of voices. NeuroImage, 22(2), 948–955.
Von Kriegstein, K., Kleinschmidt, A., Sterzer, P., & Giraud, A. L. (2005). Interaction of face and
voice areas during speaker recognition. Journal of Cognitive Neuroscience, 17(3), 367–376.
Walker, K. M., Bizley, J. K., King, A. J., & Schnupp, J. W. (2011). Multiplexed and robust
representations of sound features in auditory cortex. The Journal of Neuroscience, 31(41),
14565–14576.
Warren, J. D., Jennings, A. R., & Griffiths, T. D. (2005). Analysis of the spectral envelope of
sounds by the human brain. NeuroImage, 24(4), 1052–1057.
Warren, J. D., Scott, S. K., Price, C. J., & Griffiths, T. D. (2006). Human brain mechanisms for
the early analysis of voices. NeuroImage, 31(3), 1389–1397.
Warren, J. D., Uppenkamp, S., Patterson, R. D., & Griffiths, T. D. (2003). Separating pitch
chroma and pitch height in the human brain. Proceedings of the National Academy of
Sciences, 100(17), 10038–10042.
Warrier, C., Wong, P., Penhune, V., Zatorre, R., Parrish, T., Abrams, D., & Kraus, N. (2009).
Relating structure to function: Heschl’s gyrus and acoustic processing. Journal of
Weinstein, J., Koenig, P., Gunawardena, D., McMillan, C., Bonner, M., & Grossman, M. (2011).
Preserved musical semantic memory in semantic dementia. Archives of Neurology, 68(2),
248–250.
Wessinger, C. M., VanMeter, J., Tian, B., Van Lare, J., Pekar, J., & Rauschecker, J. P. (2001).
Hierarchical organization of the human auditory cortex revealed by functional magnetic
resonance imaging. Journal of Cognitive Neuroscience, 13(1),  1–7.
Wilson, M., & Cook, P. F. (2016). Rhythmic entrainment: Why humans want to, fireflies can’t
help it, pet birds try, and sea lions have to be bribed. Psychonomic Bulletin & Review, 23(6),
1647–1659.
Wong, P. C., Skoe, E., Russo, N. M., Dees, T., & Kraus, N. (2007). Musical experience shapes
human brainstem encoding of linguistic pitch patterns. Nature Neuroscience, 10(4), 420–422.
Wong, P. C., Warrier, C. M., Penhune, V. B., Roy, A. K., Sadehh, A., Parrish, T. B., & Zatorre, R.
J. (2007). Volume of left Heschl’s gyrus and linguistic pitch learning. Cerebral Cortex, 18(4),
828–836.
Woodruff Carr, K., White-Schwoch, T., Tierney, A. T., Strait, D. L., & Kraus, N. (2014). Beat
synchronization predicts neural speech encoding and reading readiness in preschoolers.
Zarate, J. M. (2013). The neural control of singing. Frontiers in Human Neuroscience, 7, 237.
Zhao, J., Ngo, N., McKendrick, R., & Turk-Browne, N. B. (2011). Mutual interference be-
tween statistical summary perception and statistical learning. Psychological Science, 22(9),
1212–1219.
Zhao, T. C., & Kuhl, P. K. (2016). Musical intervention enhances infants’ neural processing of
temporal structure in music and speech. Proceedings of the National Academy of Sciences,
113(19), 5212–5217.
Zion Golumbic, E. M., Ding, N., Bickel, S., Lakatos, P., Schevon, C. A., McKhann, G.
M., . . . Poeppel, D. (2013). Mechanisms underlying selective neuronal tracking of attended
speech at a “cocktail party.” Neuron, 77(5), 980–991.
Index
absorption coefficient 155, 156, 158 American Sign Language (ASL) 271, 342,
abstract 403, 404
concepts 536–538 amodal representations 9, 440, 441, 461, 750, 883
representations 586, 677, 929 amplitude (physiological signals) 49–52, 56,
acoustics 3, 9, 61, 78, 172, 175, 270, 300–305, 58, 59, 61, 83, 84, 106, 121–123, 165, 167,
347, 348, 352, 499, 503, 506, 507, 634, 249, 252, 305, 481, 483, 485, 487, 626,
648–658, 662–665, 688, 694, 738, 827, 631, 635, 652, 656, 659, 662, 664, 665,
834, 839, 843, 884, 907–910, 912, 913, 696, 697, 895, 916, 923
915–917, 919–927, 929, 932, 934 amygdala 163, 235, 251–253, 745, 890
action potential, 43, 95, 117, 189, 190, 193 amyotrophic lateral sclerosis (ALS), 711
actions anarthria 187, 191, 196, 199, 203
naming, 862, 863 anisotropy 214–216, 222, 296, 332, 614, 633,
representations, 580, 593, 594 886, 890
semantics, 577, 862 anomia 28, 29, 191, 196, 198, 220, 241, 404, 416,
understanding, 403, 417 577, 797, 856
words, 660, 661, 770, 772, 862 anterior temporal
activation patterns, 7, 80, 84, 85, 135, 327, 329, cortex 125, 218, 220, 267, 324, 588, 923
331, 412, 452, 505, 521, 524, 526–535, lobe 8, 23, 27, 125, 218, 220, 270, 332, 416, 525,
538, 539, 562, 634, 651, 781, 842, 920, 587–589, 651, 681, 689, 756, 863, 927, 931
927, 931 aphasia 2–8, 10, 11, 19–24, 27–35, 107, 108, 141,
adolescence 233, 235, 240, 244, 249, 251–253, 202, 213, 219, 220, 232, 322, 330, 386,
330, 349, 895, 912 402–405, 407, 408, 443, 461, 472, 506,
affective 232–235, 246–248, 251, 253, 457, 508, 613, 680, 687, 711, 772, 773, 786,
568, 736–740, 743, 744, 746–752, 755, 796–798, 801, 803, 804, 810–812, 814,
758–760,  890 837, 851, 852, 855, 883, 896
aging 12, 58, 281, 282, 295–299, 301, agrammatic 28, 796, 797, 798, 801
302–308,  324 Broca’s 5, 20, 27, 31, 32, 213, 798, 803, 804,
agnosia 30, 501, 502 830, 836
agrammatism 20, 25, 27–29, 31, 786, 796–807, conduction 193, 199, 505, 510, 836, 843
809–811, 813, 814, 929 progressive (see primary progressive aphasia)
agraphia 417 transcortical motor 31, 193
alexia 187, 191, 198 Wernicke’s 2, 3, 7, 8, 31, 240, 404, 405, 658,
allele 635, 636 830, 836, 837
allographic 426, 427 Wernicke’s model 20, 195, 504
allophonic 628, 649 Wernicke-Broca-Geschwind model  217
Alzheimer’s Disease, 23, 280, 307, 416, 583, 771 Wernicke-Lichtheim model 82, 234
ambiguity (linguistic) 60, 298, 611, 684–686, Wernicke-Lichtheim-Geschwind model
710, 720, 722, 739, 857, 859, 928 835–836
954 Index
aphasiology 1, 23 pathway 504, 884, 907, 909, 924, 934

apraxia 191, 378, 440, 504, 577, 580, 581, 584, perception 354, 462, 839, 857, 885
593, 658 processing 5, 7, 329, 350, 351, 353, 354, 414,
speech 27, 32, 33, 221, 454, 456, 458–461, 465 450, 607, 631, 678, 909, 918, 922
aprosodia 221 stimuli 78, 83, 131, 161, 165, 302, 351, 630, 664,
arcuate fascicle 187, 193, 197, 199, 200, 203, 215, 835, 883, 884, 923
217–219, 222, 267, 277, 371, 384, 385, system 304, 347–349, 352, 353, 500, 503, 652,
460, 506, 633, 634, 681, 798, 810, 886, 679, 909, 935
890, 892, 893 visual 233, 607, 664, 841, 879
arterial spin labeling 76, 474 Australia 342, 685
articulators 79, 343, 380, 409, 415, 454 indigenous language 688
articulatory 2, 3, 5, 10, 104, 141, 197, 199, 200, autism 23, 154, 177, 711, 725, 886, 893, 898
266, 268, 273, 305, 307, 385, 406, 407, autopsy 19, 27, 29–32
416, 427, 453–455, 457–461, 464, 465, averaging 44, 49, 62, 120, 124, 126, 135, 165, 168–
474, 482, 499, 506, 508, 607, 657–660, 170, 221, 245, 430, 487
681, 831–835, 838, 843, 881, 884, 924 axons 43, 188, 193, 195, 202, 203, 214, 216, 217,
artifacts (neurophysiological measurements) 221, 332, 349, 353, 381, 382, 605, 634
44, 47–51, 53, 62, 74, 76, 79, 124–
126, 129, 137, 154, 167, 168, 176, 221,
472–475,  481 babbling 233, 236, 245, 249, 301, 343
ataxia 454–456, 460, 463, 464, 502, 579, 583, 593 basal ganglia 21, 163, 194, 264, 266, 267, 275,
atlas 130, 196, 202, 203, 386, 931 327, 373, 377–381, 388, 407, 412, 415, 450,
atrophy 22, 23, 29, 33, 281, 282, 680, 689, 800, 456–459, 464, 523, 583, 632, 810, 851,
801, 810–813, 863 857, 860, 862, 909, 933
attention 53, 59, 61, 62, 76, 82, 98, 160, 165, 167, Basque 616
176, 177, 201, 246, 265, 269, 296, 298, bayesian 83, 84, 653
301, 302, 307, 329, 352, 356, 371, 410, 417, beamforming 122, 124
427, 430, 431, 435, 437, 438, 443, 476, behavior 1, 2, 29, 30, 34, 53, 96, 97, 99, 155, 220,
484, 520, 530, 585, 604, 626, 629–631, 246, 251, 274, 277, 278, 298, 302, 306,
635, 649, 651, 654, 657, 661, 684, 720, 321, 355, 371, 460, 486, 489, 635, 636,
721, 738, 742, 745, 746, 752, 753, 760, 739, 740, 742, 745, 746, 749, 750, 757,
773, 783, 785, 856, 864, 890, 894, 896– 785, 921
898, 908, 909, 915–918, 920, 922, 926, behavioral 4, 7, 8, 23, 33–35, 42, 44, 47, 48, 53,
932–934 56, 57, 62, 64, 81, 97–99, 106, 108, 109,
audition 500, 501, 503, 510, 678, 679, 686, 687, 134, 140, 143, 144, 160, 167, 173, 201, 231,
835, 908 232, 234, 235, 243, 251, 263, 271, 274–278,
auditory 282, 295, 297–299, 301, 302, 304, 305,
comprehension 2–4, 34, 505 308, 339, 353, 415, 425, 429, 431–433, 435,
cortex 30, 61, 118, 130, 131, 135, 137, 269, 305, 436, 438, 443, 451, 472, 473, 476, 482,
348–355, 383, 414, 503, 527, 539, 633, 488, 502, 503, 522, 527, 534, 536, 540,
635, 652, 653, 656, 662, 665, 678, 836, 557, 559, 561, 567, 570, 576, 580, 587, 589,
884, 886, 887, 909, 910, 912–915, 917, 590, 593, 609, 611, 628, 635, 636, 649,
919–922,  934 650, 658, 659, 661, 685, 690, 691, 741,
input 30, 200, 348, 353, 357, 358, 414, 505, 631, 742, 814, 830, 833, 835, 843, 862, 863,
647, 657, 832 867, 880, 883, 885, 892, 893, 895, 898,
motor 219, 461, 498, 499, 501, 503–510, 680– 911, 914, 915, 921, 924, 925, 929, 934
682, 835, 843, 891, 934 beta (frequency) 120, 140, 856, 933
Index 955
bilingualism 64, 191, 262–275, 277–282, 307, 308, childhood 174, 219, 248, 250–253, 276, 320, 345,
358, 411–413, 458, 534, 563, 603–605, 607, 346, 412, 458, 459, 463, 613, 724
609, 611, 613–615, 617, 781, 860, 867, 879, children 61, 238, 239, 242, 244, 248, 302,
890, 892, 893 343, 344, 347, 355, 356, 628, 629, 631,
blood 895, 912
oxygen level dependent (BOLD) signal chimpanzees 885, 886
73, 74, 76–78, 83, 84, 160, 474, 482, Chinese 269–271, 275, 413, 482, 485, 611, 615,
487, 580 713, 715, 723, 797, 893
oxygenation 73, 137, 155, 157, 159, 165, 439, chromophore 155, 157, 158
442, 474, 481, 520, 577, 725 chronometric 96, 97, 103, 104, 109, 477–479
bradykinesia 415, 457 cingulate 64, 139, 193, 200, 220, 262–264, 301,
brain mapping 73, 108, 130, 186, 191, 769, 781 305, 306, 327, 373, 376, 381, 382, 387, 413,
brainstem 269, 282, 349, 373, 381, 382, 909–912, 454, 457, 458, 477, 524, 565, 683, 723, 745,
922, 934 853, 931
British sign language (BSL) 342, 402, 404 clause 47, 244, 245, 300, 303, 682, 784, 800,
brain derived neurotrophic factor (BDNF) 636 801, 865
Broca, Paul, 1, 2, 19, 20, 30, 72, 212, 232, 234, clitic 279, 280, 807
241, 614, 654, 657, 879–883 cochlea nerve 340, 341, 347, 348, 651, 910
Broca’s area 2, 5, 21, 27, 31, 32, 72, 81–85, 101, cochlear implant 162, 175, 340, 341, 345, 346,
103–105, 108, 134, 195, 196, 202, 213, 218, 348, 349, 351, 354, 356–358
220, 221, 267, 327, 328, 371, 405, 406, code-switching 263, 280, 282
413, 454, 459, 474, 477, 521, 613, 719, 779, cognate 63, 835
780, 785, 786, 811, 836, 838, 854, 879– cognitive
883, 886, 909, 919, 925 control 64, 137, 200, 274, 281, 301, 307, 411,
Brodmann, Korbinian 82 413, 431, 488, 683, 725, 841, 853, 860,
861, 867, 878, 930
decline 261, 280, 306, 308, 416
Catalan 64, 376 function 72, 98, 154, 158, 188, 191, 197, 198,
catechol-O-methyltransferase (COMT)  635 231, 261, 266, 296, 307, 308, 319, 437, 475,
category-specific 521, 535, 564, 774, 778–780, 635, 677, 851, 878, 879, 894, 896–898,
783, 922, 923 912, 934
cathode 188–190 neuropsychological 20, 24–26, 425, 427, 428,
caudate nucleus 193, 197, 200, 263, 265–268, 441, 580
273–275, 281, 308, 376, 377, 387, 413, 456, neuroscience 1, 43, 96, 106, 109, 307, 500,
487, 615, 853, 854, 857, 859–861, 863, 866 519, 520, 539, 549, 604, 617, 647, 650,
cell 43, 116, 119, 155, 348, 381, 503, 557, 661, 771, 654, 677, 680, 683, 736–738, 744, 747,
856, 910 748, 760, 828, 829, 835, 837
membrane 43, 188, 190, 214, 216, 217, coherence 122, 123, 129, 137–139, 171, 614, 633,
476, 910 664, 878, 916
cerebellum 138, 264, 266–268, 275, 280, 327, coil 94–100, 109, 117–119, 353
373, 374, 377, 378, 380, 388, 450, 454– communication 12, 43, 82, 162, 163, 213, 217, 219,
456, 459, 463, 481, 507, 721, 838, 909, 933 232, 234, 253, 296, 331, 342, 344, 358, 388,
cerebral 519, 521, 522, 524, 603, 711, 739, 740, 746,
artery 21, 22, 32, 107, 238, 798 750, 753, 754, 759, 760, 827, 856, 882,
blood flow (CBF) 814 907, 908, 916, 931–933
cortex 154, 156, 373, 454, 455, 836 compensatory mechanisms 106, 107, 297, 298,
hemispheres 21, 319, 325, 456 303, 324, 326, 355, 383, 410, 455, 632
956 Index
comprehension consonants 28, 173, 236, 371, 380, 460, 631,

in aphasia 141, 798 921, 924
deficits 8, 28, 29, 33, 246, 404, 405, 417, 655, consonant-vowel 648, 883, 886
801, 806, 807, 812 conspecifics 743, 751, 914, 924
impairment 2–4, 404, 799, 810, 812, 864, 866 conversation 219, 238, 240–242, 247, 263, 282,
metaphors 711, 717, 721, 723, 725 748, 830, 914
production 236, 262, 267, 299, 344, 346, 418, corpus callosum 222, 281, 332, 350, 880, 886,
798, 801, 814, 830 919, 934
computational models 6, 11, 12, 34, 109, 144, cranial nerve 373, 381–383, 454
386, 429, 508, 680, 920 culture 340–343, 535, 740, 750, 907
computations 9, 11, 12, 302, 455, 594, 610, 648, currents 43, 95, 116, 119, 120, 123, 127, 128, 190,
650, 651, 742, 907, 910, 913, 920, 923, 192, 193
924, 935 cytoarchitectonic 82, 406, 505, 605, 881
concept 9, 193, 307, 410, 500, 519–540, 548–552,
560–563, 567, 569, 571, 581, 613, 696, 747,
750, 827, 828, 835, 843 data driven methods 529, 553, 556, 558, 559,
concepts 267, 343, 409, 500, 519–540, 548–557, 566, 567, 572, 935
559–563, 565, 567, 569–572, 581, 593, datives 533, 807, 808
614, 647, 660, 721, 754, 771–774, 783, deafness 12, 30, 175, 271, 339–358, 402–405, 407,
828, 835, 858, 922, 928, 931, 932, 935. 409–416, 418, 883, 887
(See semantic representations) decision-making 10, 53, 318, 661, 746
abstract 536–538 decoding 563–565, 586, 611, 626, 628, 634, 663,
concrete 48, 109, 520, 521, 523, 524, 532–534, 664, 740, 920, 924, 927, 931
536–538, 695, 741, 747, 751, 777, 778 default mode 84, 85, 524, 607
conceptual processing 28, 29, 33, 63, 106, 123, degenerative disorders 30, 33, 388, 415, 455, 583,
174, 267, 270, 372, 407, 473, 475, 476, 771, 772
485, 490, 498–500, 523, 549–554, 557, delta (frequency) 633, 635, 662, 664, 665, 933
560, 562–565, 570–572, 576, 579–584, dementia 22, 23, 25–29, 261, 416, 583, 711,
588, 660, 661, 677, 680, 687, 689, 721, 931, 932
748, 769, 771, 774–778, 780, 786, 842, dendrites 43, 116, 298, 348
863, 907, 931 deoxygenated
confounds 33, 44, 48, 50, 99, 128, 172, 261, 276, blood 74, 154, 156–159
347, 357, 473, 475, 564, 571, 616, 775, 776 haemoglobin (deoxy-Hb) 74, 76, 155–157,
congenital conditions 339, 341, 348, 350, 352, 160, 161, 169, 175
353, 355, 356 depolarization 95, 188, 190, 192
connectionist 200, 202, 510, 690 depression 339, 711
connectivity 9, 10, 81, 83–85, 94, 109, 116, 123, developmental
124, 128, 129, 135, 137–139, 142, 143, 159, disabilities 162
164, 171, 175, 186–188, 191–193, 195, 197, disorder 154, 627, 711
201, 202, 219, 252, 263, 268, 272, 273, dyslexia 626
281, 299, 353, 372, 376, 377, 381, 384–388, trajectory 232, 236, 277, 278, 540
413, 427, 457, 459, 590–593, 605, 614, diadochokinesis 463, 464
615, 617, 632, 634, 677, 678, 681, 684– diaschisis 35, 456
686, 690, 719, 720, 725, 747, 759, 814, diffusion tensor imaging (DTI) 7, 35, 82, 192,
842, 854, 867, 887, 889, 892, 893, 933 212–215, 217, 219, 221, 222, 281, 296, 331,
connectomics 201, 203 385, 386, 388, 612, 682, 854, 890
consciousness 160, 201, 500, 741, 828 dipole 43, 48, 51, 117, 119, 122, 127, 128
Index 957
disconnection 193, 213, 330, 593 stimulation (intraoperative) 186, 187–189,

discourse 10, 54, 60, 219, 234, 240, 242–245, 191–193, 195, 199, 201, 203, 212, 321, 340,
307, 408, 410, 684, 688, 754, 756, 341, 381, 385–387, 476, 851, 856
757, 859 electrocorticography (ECoG) 137, 463, 648,
dissection 212, 216, 222 659, 660, 690, 915, 916, 924, 925, 935
dissociations 25, 106, 142, 194, 198, 200, 201, electrode 45, 49–51, 55, 63, 162, 188, 190, 192,
266, 418, 432, 433, 436, 440, 454, 501, 249, 322, 347, 475, 476, 690, 856, 857,
502, 508, 551, 557, 565, 582, 586, 607, 648, 910, 916, 929, 930
650, 682, 683, 692, 770–773, 782–784, electroencephalography (EEG)
786, 799, 829, 830, 837, 841, 914 intracranial 142, 475, 477, 484, 485, 929
dizygotic 889 scalp 42, 85, 98, 116, 154, 302, 349, 379, 473,
domain general processes 9, 59, 60, 62, 63, 262, 633, 648, 690, 747, 910
264, 298, 304, 430, 462, 465, 473, 477, electromagnetic 43, 94, 95, 119, 120, 123, 124,
488, 489, 655, 693, 832, 841, 853, 861, 127, 130, 135, 137, 349
867, 898, 922, 930 electromyogram (EMG) 124, 125, 126, 759
dopamine 377, 415, 857, 858, 860, 862, 864 electrophysiology 42, 43, 45, 47, 49, 51, 53, 57,
dorsal 59, 61, 63, 186, 478, 483, 486, 489, 654,
pathway 267, 450, 459–461, 610–612, 684, 693, 725, 769
686, 810, 812 embodied cognition 9, 440, 441, 523, 531, 581,
stream 198, 199, 385, 386, 461, 478, 482, 498, 660, 677, 711, 720, 721, 757, 758
499, 501–507, 579, 590, 610, 679–682, emojis 747–750, 752, 759
687, 688, 884, 885 emotions 53, 163, 167, 187, 201, 219, 231–235,
dorsolateral prefrontal cortex (DLPFC) 197, 246–248, 251–253, 347, 380, 382, 450,
198, 201, 303, 351, 379, 413, 800, 801, 451, 456, 457, 459, 465, 523, 524, 530, 531,
841, 853 569, 697, 710, 720, 725, 736–747, 750–
dual coding theory 532, 538 755, 757–760, 853, 890, 891, 909
dual route model 609, 658 encephalitis 233, 330, 331, 771
dual stream model 501, 503, 504, 677 endophenotypes 634
dysarthria 196, 386, 407, 451–458, 460, English 24, 27, 58, 134, 173, 239, 242, 268–271,
463, 464 277, 279, 331, 342, 344, 345, 402, 411–413,
dyscalculia 191, 627 416, 520, 534, 610–612, 615, 616, 687,
dysfluencies 403, 457, 460, 461, 472 688, 692, 712–7 16, 723, 769, 781, 786,
dysgraphia 427–429, 435, 440, 443 797, 803, 837, 893, 924, 931
dyslexia 25, 26, 626–636, 879, 892, 893, environmental sounds 28, 56, 131, 923
888–890 epilepsy 23, 101, 117, 128, 140, 186, 189, 190–193,
dysmetria 455, 456 195, 317–332, 452, 475, 651, 829, 880,
dysplasia 319, 324 915, 917
dyspraxia 658, 659 error monitoring 58, 264, 307
dystonias 457 error-related negativity (ERN) 64, 307, 383,
455, 696
event-related potentials (ERPs) 6, 43, 44, 47–
education 281, 282, 324, 339, 342, 345, 355, 356, 50, 52–54, 61, 62, 64, 106, 249, 263, 270,
538, 626, 889 276–279, 299, 354, 475, 481, 633, 691–
electrical 693, 776, 803, 863, 866, 891, 895, 929
activity 42, 44, 52, 115, 452 evoked responses 62, 115, 116, 120–122, 130,
currents 43, 95, 116, 119, 120, 123, 127, 128, 134–137, 143, 193, 349, 479, 484, 572,
190, 192, 193 659, 857
958 Index
evolution 138, 213, 253, 341, 342, 375, 417, 418, lobe 5, 19, 20, 126, 199, 217, 220, 328, 350, 386,
535, 538, 603, 740, 742, 744, 751, 878, 403, 413, 414, 451, 454, 504, 505, 583,
882, 885–889, 894, 908, 922 632, 650, 720, 772, 855
excitability 94, 96, 188, 190, 661, 662, 665 operculum 139, 199, 221, 235, 301, 331, 405,
external capsule 332, 384, 387, 388 433, 481, 505, 810, 812
extreme capsule 220, 267, 385, 386, 681 functional connectivity 10, 83, 159, 171, 191, 219,
eye-movements 53, 60, 162, 167, 725, 805–807, 268, 272, 273, 281, 353, 372, 413, 591–593,
809, 814 605, 617, 632, 677, 684, 685, 719, 720,
747, 842, 854, 887, 889, 933
functional magnetic resonance imaging
facial (fMRI) 6–8, 10, 22, 72–85, 99, 101, 107,
expression 163, 167, 232, 233, 235, 246–248, 109, 116, 120, 135, 137, 142–144, 155, 160–
251, 252, 382, 748, 749, 751 168, 170–172, 174, 192, 203, 219, 250, 252,
muscles 454, 455 270, 274, 275, 277, 278, 301, 303, 305, 306,
feedback 144, 194, 199, 201, 380, 383, 384, 436, 318, 325–332, 349, 374–377, 379, 383, 413,
455, 498, 507, 508, 510, 678, 690, 697, 416, 430, 434, 441–443, 473–476, 478,
698, 856 480–487, 489, 490, 504, 505, 519–521,
fibers (white matter) 536, 537, 540, 549–551, 553, 554, 568, 571,
bundles 221, 266, 387, 461, 612 586, 588, 605, 608, 611, 630, 632, 633,
pathways 32, 384, 385, 388 635, 648–651, 659, 660, 683, 684, 690,
tracts 7, 82, 332, 459, 463, 612, 615, 682, 685, 711–713, 716, 717, 719, 720, 722, 725, 747,
687, 689 756, 757, 769, 777, 781–784, 839, 840,
figurative language 660, 710, 721, 722 842, 853, 856, 859, 866, 880, 881, 884,
filtering 885, 891, 892, 895, 913, 917, 919–922, 925,
EEG 46, 47, 51 926, 929, 930, 935
NIRS 168, 169 functional near infrared spectroscopy (fNIRS)
Speech 305, 663, 665 154–156, 158, 162–164, 166, 167, 171, 349
fingerspelling 416 functional neuroanatomy 20, 372
Finnish 61, 133, 135, 649 fusiform gyrus 84, 139, 235, 251, 436, 441, 442,
fluency 2, 33, 34, 164, 220, 250, 264, 268, 271, 481, 525, 565, 567, 569, 578, 579, 586,
274, 307, 324, 327, 330–332, 345, 386, 388, 593, 719, 725, 780, 889, 890, 893–895,
416, 615, 628, 798, 799, 813, 862, 882, 926, 931
885, 886, 892, 893, 896, 897
French 19, 272, 278, 342, 712, 714, 715
frequencies gamma (frequency) 120, 137, 142, 635, 662,
auditory 304, 340, 347 665, 933
electrophysiology 46, 47, 120, 123, 137, garden-path  219
139, 159 gender 33, 72, 136, 200, 279, 280, 303, 327, 328,
formant 920, 921, 924, 932 634, 693, 926
frontal genes 144, 356, 634–636, 886, 888
aslant tract 197, 200, 218, 220, 267, 376, genetics 143, 348, 356, 402, 458, 535, 627, 633,
385–387,  813 634, 878, 886, 889, 894, 895, 898
cortex 85, 126, 131, 134, 141, 143, 158, 235, 265– genome 144, 635, 636
267, 297, 305, 306, 376, 387, 454, 459, genotypes 635
485, 551, 657, 678, 725, 783, 812, 853, 879, German 20, 42, 64, 402, 460, 635, 694, 712–7 16,
888, 891, 892, 915, 925, 930 797, 803, 865, 883, 892
Index 959
Geschwind, Norman 2, 3, 30, 72, 199, 213, hemorrhage 21, 233, 855, 856
504, 593 heritability 634, 636, 889
gestational 160, 176, 233 herpes simplex encephalitis 771
gestures 233, 240, 246, 353, 417, 418, 459, 465, hippocampus 163, 272, 319, 320, 327, 328, 329,
506, 581, 657–659, 661, 711, 748, 749, 485, 526, 636, 853
750, 752, 878, 882, 883 homologue 10, 72, 84, 85, 106, 107, 108, 192, 232,
glioma 186, 187, 192, 200, 202, 203, 324 239–241, 245, 248, 250, 251, 253, 297,
globus pallidus 373, 374, 456, 457, 853, 854, 863 298, 303, 326, 327, 417, 430, 474, 536,
glucose metabolism 351 720, 878, 880, 889, 891, 897
glutamate 636 homonyms 685, 776, 777, 779, 780, 786, 859
gradiometers 117, 118 homophone 306
grammar 5–9, 11, 136, 163, 165, 234, 276, 277, hypoglossal 382, 451
280, 342, 409, 683, 750, 773, 804, 854, hypokinesia 407, 456–458, 460
860, 865 hypoperfusion 32, 35, 456, 852
categories (class) 25, 532, 769–771, 773–775, hypophonia 457
777, 779, 781, 783, 785 hypothalamus 745
gender 200, 279
morphology 344, 796, 797
structure 240, 532, 807, 865 ideational apraxia 593
graphemes 427, 428, 611, 627, 628, 636 idioms 234, 710–7 15, 719–725, 748, 866
to-phoneme 134, 607, 610, 616 infants 61, 138, 154, 160–162, 166, 167, 171–177,
gyrification 885 232, 233, 236, 246–249, 251–253, 320,
343, 345–347, 358, 404, 458, 463, 464,
614, 633, 654, 725, 882, 886, 917, 918
handedness 72, 84, 167, 318, 320, 327, 328, 878, infarct 21, 32, 233, 453, 455, 798, 855
881, 885, 886, 894–896, 919 inferior frontal
haplotypes 144, 635, 636 cortex 85, 134, 141, 143, 265–267, 454, 459,
harmonics 688, 910, 913, 916, 917, 919, 927, 657, 678, 783, 892
928, 930 gyrus 5, 20, 30, 31, 61, 101, 139, 163, 198–200,
hearing 4, 9, 30, 83, 167, 175, 176, 274, 301–305, 202, 218, 220, 234, 250, 264, 300, 327,
339, 340, 342–354, 356–358, 383, 403, 330, 372, 373, 376, 387, 414, 430, 450,
407, 410–414, 417, 418, 503, 806, 883, 459, 477, 524, 531, 537, 538, 604, 605,
885, 887, 911 632, 650, 660, 683, 717–7 19, 723, 772,
Hebrew 27, 28, 611, 612, 713–7 16, 720, 722, 797 798, 801, 836, 838, 841, 854, 881, 883,
hemineglect 194, 196 884, 888, 890, 891, 893, 930, 931
hemiparesis 319, 404 pars opercularis 31, 197, 199, 202, 220, 267,
hemiplegia 19, 319 268, 273, 274, 385–387, 404, 405, 417,
hemispherectomy 330, 331 454, 459
hemispheric dominance pars orbitalis 202, 331, 604, 605, 610, 612,
2, 108, 171, 172, 193, 194, 195, 264, 317, 319, 615, 683, 890
320, 328, 329, 330, 404, 406, 410, 412, pars triangularis 108, 197, 202, 220, 267,
453, 460, 499, 654, 685, 878, 880–882, 268, 273, 331, 385, 404, 406, 604, 605,
884, 886–888, 890, 892, 895, 896, 898, 610–612, 614, 683, 880–883, 893
919, 929 sulcus 374, 376, 378, 682
hemodynamic 74, 75, 78, 143, 155, 159–161, 163– inferior fronto-occipital fasciculus 187, 193, 197,
165, 299, 325, 329, 374, 553, 649, 654 198, 218, 220, 221, 267, 273, 281, 332, 385,
hemoglobin 154, 156 386, 888, 890
960 Index
inferior longitudinal fasciculus 187, 198, 218, acquisition 61, 171, 174, 175, 177, 219, 232, 238,
220, 221, 385, 888, 890 239, 244, 245, 249, 329, 330, 452, 682
inferior occipital gyrus 139, 197, 893 areas 8, 20, 23, 32, 33, 82, 98, 102, 108, 166,
inferior occipitotemporal cortex 132, 133, 188, 199, 212, 317, 318, 321–323, 325–328,
138, 418 344, 351, 418, 593, 883, 886, 892, 893
inferior parietal comprehension 8, 20, 47, 53, 56, 62, 143, 267,
cortex 102, 266, 405, 593, 607, 614, 615, 632, 299, 305, 339, 344, 404, 405, 410, 411,
685, 886 417, 418, 472, 647, 651, 652, 655, 677, 684,
lobe 217, 218, 417, 461, 478, 611, 612, 632, 888 687, 696, 720, 721, 736, 737, 739, 741, 743,
lobule 139, 199, 202, 263, 265, 377, 384, 385, 745, 747–751, 753, 755, 757, 759, 891, 892
409, 482, 579, 580, 586–588, 590–593, control 263, 266, 267, 281, 282
660, 678, 681, 719, 721, 772, 780, 785, deficits 2, 3, 20, 22, 23, 27, 29, 32, 33, 415, 851
799, 810, 811 development 232, 233, 236, 238, 240, 243–245,
inferior temporal 249, 250, 331, 339–341, 343, 345, 347, 349,
cortex 197, 324, 681 351, 353, 355–358, 408
gyrus 139, 199, 251, 385, 407, 524, 536–538, dominance 319, 320, 329, 654, 878, 880,
579, 585, 853, 890, 893 884, 919
inflection 24, 25, 27, 28, 133, 134, 136, 403, 685, experience 172, 352, 411, 418, 614, 918, 934
778–782, 796, 797, 800, 865 function 2, 21, 22, 24, 25, 27, 35, 115, 130, 136,
inhibitory 137, 140, 142, 143, 154, 159, 203, 250, 253,
control 264, 274, 296 317, 322, 388, 406, 814, 836, 851, 852,
processes 64, 858 854–856,  867
insula 30, 32, 33, 139, 197, 199, 202, 221, 235, 267, impairment 8, 11, 627, 631, 636, 852, 863, 888
269, 301, 306, 307, 331, 350, 373, 378, 379, lateralization 140, 320, 326, 328, 329, 878–
404, 406, 433, 459, 499, 616, 745, 757, 881, 883, 886, 890, 898
798, 810, 812, 813, 838, 882 learning 130, 272, 276, 278, 649, 680, 854,
intelligence 167, 232, 550, 626, 711, 742 860, 861, 893
intonation 173, 221, 452, 750, 916 network 82, 104, 109, 166, 193, 217, 218, 220,
intracarotid amobarbital test. See Wada test 250, 262, 266, 267, 269, 272, 273, 281,
intraoperative electrical stimulation. See 282, 331, 332, 684, 796, 837, 838, 886,
electrical stimulation 891, 892, 898
intraparietal sulcus 412, 579, 580, 586, 589, neural basis 6, 11, 12, 23, 27, 186, 195, 201
592, 593 neurobiology 72, 443, 627, 629, 631, 633, 635,
ischemia 21, 214 676, 677, 679, 681, 683, 685, 687, 689,
Italian 27, 28, 173, 262, 271, 610, 712–7 15, 723, 691, 693, 695, 697, 809
775, 776, 781, 797, 803, 807 pathways 216, 317, 812–814
plasticity 273, 318, 319, 321–323, 325, 327,
329, 331
Japanese 173, 712–7 15, 797, 893, 925 processing 4, 7, 8, 11, 12, 21, 28, 33, 42, 52, 54,
jittering 47, 50, 77, 78, 120, 125 57, 59, 64, 83, 85, 97, 100–103, 105, 107,
109, 115–117, 119, 122, 123, 125, 127, 129,
131, 133, 135, 137, 138, 140–144, 161, 171–
kinematics 593, 772 175, 197, 198, 212, 231–233, 250, 252, 253,
268, 279, 295, 296, 298, 308, 348, 349,
402, 403, 407, 408, 413, 415, 417, 498,
landmarks 98, 165, 166, 176, 318, 653, 783, 926 533, 538, 611, 615, 654, 655, 676, 677, 680,
language 681, 683, 684, 686, 690, 696, 737, 738,
Index 961
746, 747, 753, 754, 758–760, 804, 836, ambiguity 857, 859

837, 843, 854, 855, 858, 860, 865, 867, categories 27, 533, 769, 782
879, 883, 890, 893, 919, 927 competition 5, 11, 273, 298–300, 304
production 61, 63, 99, 105, 107, 125, 162, 163, decision 4, 26, 42, 436, 608, 650, 802, 858,
234, 240, 265, 305, 307, 371, 376, 377, 860, 888
403, 405, 406, 417, 418, 425, 443, 472, orthographic 431, 432, 436, 441, 610
476, 613, 786, 797, 799, 800, 813, 830, processing 5, 102, 270, 434, 686, 697, 866
840, 841, 865, 919 representations 9, 410, 431, 538, 606, 650,
proficiency 56, 249, 265, 272, 332, 604, 614, 617 652, 653, 661, 738, 856, 861
reorganization 318, 319, 327, 329, 331 retrieval 220, 272, 305, 739, 854
representation 188, 319–322, 325–327 selection 11, 268, 473, 488, 510, 808, 856, 860
switching 200, 264, 265, 275, 412 semantic 8, 56, 131, 132, 140, 141, 250, 426,
written 31, 413, 425, 435, 443, 710, 759 476–478, 484, 487–489, 506, 577, 583,
languages 27, 28, 56, 64, 102, 173, 261–263, 267, 585, 606, 617, 650, 693, 811, 852, 855–857,
269–271, 273, 275, 278–282, 341–343, 858, 860, 862–865, 866, 884, 891
402, 407–412, 416–418, 534, 585, 610– syntactic 854, 864, 890
614, 616, 631, 649, 682, 686, 688, 712, lexicon 2, 7, 11, 26, 62, 236, 299, 307, 426, 427,
756, 769, 770, 781, 782, 785, 797, 803, 472, 508, 577, 683, 690, 748, 773, 774,
860, 893, 910, 911, 916, 922 783, 864
larynx 101, 371, 373, 381, 382, 406, 450–452, limbic 220, 235, 253, 451, 454, 457, 458, 465,
456, 457 720, 723
latency 50–52, 56, 58, 59, 103, 159, 161, 165, 252, linguistics 218, 232, 504, 626, 683, 737, 739,
349, 350, 474, 477, 478, 480–482, 487, 743, 928
692, 694–696, 880, 891 listening 78, 102, 119, 131, 161, 167, 175, 266, 268,
lateralization 45, 49, 51, 63, 84, 132, 140, 164, 301, 304, 340, 416, 505, 650, 654–657,
172, 174, 176, 177, 219, 249, 250, 297, 298, 659, 758, 839, 883–886, 888, 891–893,
318–320, 326, 328–330, 353, 412, 453, 895, 910, 911, 913, 915–918, 920, 923,
455, 647, 654, 685, 717, 720, 724, 867, 933, 935
878–898 literacy 10, 889, 892
left-handers 84, 320, 407, 878, 880–883, 885, literal language 712, 717–722, 724, 891
886, 888, 890, 895–898, 919 localization 2, 4, 5, 8, 27, 31, 72, 82, 128, 132,
lemma 33, 200, 407, 476, 480, 808 143, 154, 158, 172, 174, 202, 213, 221,
lesion 270, 317, 318, 326, 331, 353, 356, 357,
deficit 20, 29, 30, 33, 275, 441 476, 695, 712, 718, 782, 830, 837, 838,
studies 2, 4, 5, 10, 82, 320, 417, 430, 459, 480, 910, 914
648, 719, 720, 723, 783, 929 logographic 414, 893
symptom mapping 2, 7, 8, 33–35, 72, 269, logopenic 22, 799
378, 429, 434, 436, 478, 479, 483, 486, longitudinal studies 272, 276, 307, 332, 354
489, 687, 689, 798, 810 loudness 78, 371, 457, 464, 920
letters 54, 133, 134, 345, 416, 425–428, 431–435, low-pass filter 46, 51, 169, 173, 301, 305
440–442, 585, 607, 610, 628, 629, 635, lying 78, 107, 772
837, 838, 887, 889, 892, 894, 895
levodopa 858, 862, 867
lexeme 476, 510 macaque 384, 385, 388, 417, 501, 504, 536, 537,
lexical 580, 592, 605, 660, 927
access 5, 8, 33, 63, 270, 407, 585, 594, 628, 629, magnetic fields 94–96, 98, 99, 115–119, 122, 123,
647, 649, 650, 690, 697, 858, 865, 866 125–128, 142, 349, 523, 693
962 Index
magnetic resonance motor

imaging (MRI) 212, 270, 453, 710, 747 areas 139, 192–194, 197, 199, 200, 220, 373,
spectroscopy (MRS) 85, 154, 349, 636, 654, 374, 376, 377, 387, 451, 477, 616, 660, 813,
712, 882 838, 854, 890, 909, 919, 925, 933
magnetoencephalography (MEG) 6, 115, 192, control 343, 375, 383, 384, 416, 450, 451,
212, 276, 331, 344, 473, 475, 520, 553, 455, 456, 462, 464, 465, 498, 499,
648, 690, 747, 769, 910 680, 881
magnetometers 117, 118, 142 cortex (see premotor cortex; primary motor
magnocellular 630, 631 cortex)
mammals 42, 382, 524, 557, 564, 565, 653, 736, disorders 378, 450
740, 746, 913, 920, 926 execution 372, 378, 405, 407, 450, 456
maturation 160, 189, 219, 349, 350, 459, 636, 724 functions 108, 375, 454, 457, 458, 462, 464,
medial frontal cortex 126, 220, 327, 387, 490 854, 881
medial temporal cortex 319, 321, 329, 407, 523, integration 219, 405, 461, 505, 506, 681,
533, 561, 562, 829, 830, 864, 888 842, 843
meta-analysis 78, 81, 203, 372, 473, 478, 589, interface 499, 501, 503, 505, 507
605, 612, 680, 684, 710–7 12, 716, 717, learning 452, 459, 462–465
719–721, 723, 743, 890, 891, 893, 924 nuclei 381, 382, 450–452, 454, 455, 463
metabolism 35, 76, 99, 107, 212, 351 output 459, 463, 465, 592, 657, 661
metaphors 219, 234, 710–725, 879, 890–892 planning 31, 305, 306, 372, 375, 377, 380, 426,
middle cerebral artery 21, 22, 32, 107, 238, 450, 454, 459, 465, 500, 505, 506
239, 798 programs 374, 375
middle frontal gyrus 30, 105, 139, 197, 301, 373, representations 104, 452, 460, 498, 504,
376, 379, 481, 717, 718, 722, 781, 861 843, 860
middle temporal gyrus 7, 33, 105, 139, 200, 303, speech disorders 451, 453, 455, 457, 459,
305, 327, 384, 404, 477, 563, 578, 579, 461–463,  465
585–587, 592, 593, 719, 722, 723, 772, 781, multilingualism 56, 261, 307, 308, 358, 411,
811, 854, 886, 888, 931 613, 909
mirror neurons 417, 418, 660, 661, 882 multivariate 50, 79, 80, 144, 519, 522, 525–527,
mismatch negativity (MNN) 60, 61, 628, 692, 533, 537, 539, 686
694, 695 multivoxel pattern analysis (MVPA) 270, 374,
modularity 7, 9, 20, 295, 504, 658, 519, 525, 527, 549, 554, 556–558, 562, 563,
690–692,  739 565, 567–569, 571, 572, 585, 586, 656,
monkey 221, 251, 501, 503, 536, 537, 605, 679, 659, 781, 925, 926
686, 835, 842, 882, 885 music 269, 483, 555, 632, 839, 840, 842, 907–913,
monolinguals 264, 271, 279–282, 412, 413, 915, 916, 918–920, 922–925, 927–935
616, 893 musicians 463, 910, 911, 918, 925
morphemes 28, 47, 237, 342, 651, 652, 681, 685, training 269, 909, 911, 912, 918, 919, 934
687, 797 mutism 31, 101, 220, 331
morphology 24, 25, 27, 28, 245, 331, 344, 611, myelin 160, 188–190, 214, 216, 217, 219, 298, 349,
685, 686, 781, 796, 797, 865 350, 605
errors 240–242, 244
processing 25, 685
proficiency 241, 243, 244, 246 naming 2, 12, 26, 33, 35, 42, 62, 84, 103–105, 108,
syntactic 58, 59, 133–135, 242, 245, 266, 130, 132, 134, 138, 139, 141, 143, 144, 191,
268, 276, 277, 279, 343, 403, 692–694, 194–196, 199–201, 220, 262, 264, 270, 273,
778–7 81, 800, 801, 808, 809, 811, 884 306, 307, 319, 322–324, 330, 374, 386, 405,
Index 963
407, 409, 413, 416, 475, 477–490, 508, 510, 842, 843, 851, 855, 856, 863, 864, 867,
531, 560, 561, 581–583, 585, 615, 627, 629, 880, 890, 894, 898, 924
650, 770, 771, 773, 779, 783, 784, 797, 798, neuromodulation 194, 377, 380
800, 804, 810, 856, 860, 862–864, 880, neurons 6, 43, 50, 51, 95, 115, 116, 189, 193, 201,
892, 893, 896 231, 298, 377, 381, 417, 437, 453, 552, 653,
narrative 219, 233, 240, 242–244, 246, 712, 754, 660, 662, 772, 841, 887, 895, 913–915,
755, 757, 796, 804, 813, 930 917, 921, 924, 935
neglect 410, 501 neuroplasticity 22, 187, 188, 202, 203, 231, 233–235,
neocortex 319, 329, 456, 745, 922 237, 239, 241, 243, 245–247, 249, 251, 253
neonates 160, 162, 173, 175, 176, 232, 251, 654 neuropsychology 4, 20, 24–26, 29, 232, 262,
neural 275, 281, 343, 425, 427, 428, 441, 508,
activation 6, 7, 116, 119, 120, 124, 126, 128, 131, 580–582, 588, 593, 594, 606, 613, 773,
135, 138, 144, 160, 165, 407, 414, 630, 633, 778, 779, 781–785, 835, 843
634, 636, 651, 931 neuroscience 1, 43, 73, 96, 106, 109, 155, 186,
activity 6, 43, 45, 52, 97–99, 108, 120, 122, 300, 307, 402, 500, 519, 520, 539, 549,
127, 135, 136, 164, 304, 306, 308, 325, 351, 604, 617, 635, 647, 650, 654, 676, 677,
434, 476, 535, 551, 553, 554, 556–559, 564, 680, 683, 736–738, 740, 744, 747, 748,
568–570, 586, 588, 880, 882, 887, 889, 760, 827–829, 835, 837, 843
895, 910, 915, 917, 925 nonwords (pseudowords) 11, 24, 25, 120, 133, 134,
networks 56, 98–100, 104, 105, 107, 109, 232, 144, 200, 249, 266, 273, 278, 300, 375,
251, 252, 298, 377, 462, 613, 775, 776, 432–435, 438, 479, 505, 607–610, 631, 650,
779, 883 658, 659, 692, 695–697, 781, 842, 892, 896
plasticity 235, 250, 251, 253, 280, 330, 332, 452, noun phrase 410, 682, 688, 695, 777, 783, 784,
462, 463, 934 806, 865
representations 263, 519–521, 523–527, 529,
531–537, 539, 540, 563, 570, 586, 616,
650, 915 Objects
responses 44, 263, 270, 277, 437, 438, 441, 552, affordances 409, 584, 585
555–557, 566, 567, 615, 616, 664, 911, 915, concepts 519, 521, 523, 524, 528, 531, 534, 549,
918, 921, 925 550, 551, 562, 563, 593, 771, 774
similarity 528, 535, 552–554, 557–560, 565, function 577, 583, 585, 589, 590
566, 569, 570, 572 knowledge 551, 576, 583, 584, 587
systems 5, 8, 12, 331, 407, 411 manipulation 524, 576, 577, 579–581, 585–587,
neuroanatomy 20, 26, 97, 251, 372, 502, 510, 589, 590, 771
722, 725 naming 62, 63, 103, 104, 108, 136, 141, 306,
neuroimaging 4, 7–9, 29, 32, 35, 72, 73, 79, 81, 319, 322, 407, 409, 472, 479, 581, 582,
82, 85, 102, 109, 142, 155, 156, 173, 186, 771, 772, 778, 783, 800, 863, 880, 883
191, 192, 197, 202, 203, 234, 248, 249, recognition 143, 198, 306, 502, 576, 653,
251, 252, 265, 270, 303, 305, 318, 319, 678–680, 688, 697
323, 325, 340, 343, 349, 351–353, 374, representations 348, 563, 570, 908
376–378, 380, 406, 407, 411, 416–418, use 576, 580–582, 584–587, 591, 593
425, 431–435, 437, 438, 441, 443, 453, occipitotemporal cortex 132, 138, 144, 434, 525,
472–475, 479, 484, 490, 519, 521, 524, 526, 528, 568
548, 550, 551, 553, 577, 580, 585, 588, 603, optical brain imaging 155, 157, 159, 161, 163, 165,
605, 606, 610, 611, 613–617, 627, 632, 167, 169, 171, 173, 175, 177, 192
654–656, 658, 659, 679, 680, 682, 684, signal 159, 171
687, 689, 710, 783, 812, 814, 837, 839, topography 157, 158, 171
964 Index
optic ataxia 502, 579, 593 perisylvian cortex 166, 172, 173, 253, 385, 433,
optic radiations 197 434, 459, 474, 604, 811
orthographic long term memory (O-LTM) perseveration 191, 200, 860
426–431, 433, 435–438, 440, 441 phenotypes 634, 636, 878
orthography 133, 432, 585, 603–605, 610–613, phonagnosia 926, 927
616, 617, 631, 697 phonation 371, 373, 378, 381, 384
processing 413, 414, 436, 438, 608 phonemes 3, 33, 54, 61, 131, 161, 164, 187, 196,
representations 26, 416, 426, 433, 437, 439, 198, 199, 203, 267, 268, 274, 343, 345,
441, 585 354, 374, 380, 404, 405, 416, 434, 456,
working memory (O-WM) 426–431, 436, 461, 478, 505, 610, 627, 628, 659, 662,
438, 440, 441 679, 834, 910, 916, 919, 920, 923–926
oscillations 120, 121, 137, 163, 174, 635, 648, 657, phonetics 3, 8, 10, 77, 135, 199, 342, 345, 346,
661–665, 917, 933 380, 406, 416, 427, 452, 456, 459, 460,
465, 475, 476, 634, 635, 656, 658, 659,
661, 918, 919, 926
pallidum 373, 374, 387, 456, 457, 853, 854, 863 phonological
pantomimes 418, 580, 581, 586, 882 awareness 140, 345, 626, 627, 629, 631,
parahippocampus 328, 481, 482, 524, 565, 636, 633, 634
717, 719–721, 723, 725 deficit 3, 432, 435, 626, 627, 631, 634
parallel distributed processing (PDP) 6, encoding 372, 374, 375
201, 609 facilitation 480, 482, 484
paraphasias 33, 187, 191, 196, 198, 199, 320, 404, information 63, 306, 627, 628, 685, 808, 811,
405, 416, 456, 461, 855 838, 841, 842
paresis 452, 454, 455, 460 loop 628, 828, 831–835, 837, 838, 843
parietal memory 627, 634, 636
cortex 102, 105, 172, 250, 266, 330, 410, 416, neighbors304, 374, 478
482, 504, 578, 579, 588, 590, 591, 593, processing 80, 101, 102, 199, 272, 351, 434,
604–607, 610, 614, 615, 632, 838, 841, 435, 460, 478, 482, 606, 607, 610, 611,
853, 889, 913 626–631, 633, 636, 685, 800, 857, 881
lobe 217, 218, 386, 417, 430, 478, 502, 505, 591, rehearsal 267
594, 611, 612, 632, 772, 812, 838, 839, 841 representations 26, 62, 374, 412, 479, 538,
lobule 139, 199, 202, 377, 384, 385, 409, 410, 585, 626, 628, 630, 633, 634, 889
438, 482, 578–580, 586–588, 590–593, store 832–834, 837–839
660, 678, 681, 719, 721, 772, 780, 785, word form 25, 306, 374–376, 461, 476,
799, 810, 811, 883 478, 629
regions 21, 30, 32, 267, 271–273, 344, 408, working memory 199, 427, 434, 538
411–414, 417, 501, 590, 606, 607, 683, phonology 2, 7, 63, 134, 193, 265, 268, 305, 345,
812, 813, 891 432, 461, 603, 607, 610, 627, 629, 647,
Parkinson’s disease 23, 194, 377, 379, 415, 416, 684, 686, 760, 857, 889, 935
457, 458, 464, 711, 857 phrases 31, 465, 473, 534, 679, 682, 688, 689,
passive listening 78, 119, 650, 656, 659, 915, 694, 721, 777, 806, 924, 930
923, 933 picture-word
pediatric. See childhood, children interference paradigm (PWI) 479, 482, 483
perfusion 22, 35, 76, 474, 484, 485, 487, 814 matching 106, 581, 695
periaqueductal 382, 745 planum temporale 83, 84, 353, 478, 505, 506,
perinatal stroke 231, 233, 235, 237, 239, 241, 243, 839, 878, 879, 882, 885, 893, 909,
245, 247–253 927, 928
Index 965
plasticity 7, 31, 202, 203, 231, 233–235, 239, 247, primates 216, 219, 251, 385, 451, 500, 503,
250, 251, 253, 261, 273, 276, 280, 317–319, 526, 529, 557, 559, 591, 665, 678–680,
321–323, 325, 327, 329–332, 414, 415, 452, 686, 687, 835, 879, 888, 913, 914, 917,
462, 463, 616, 636, 887, 909, 912, 919, 924, 933
922, 934 priming 143, 270, 279, 306, 436, 490, 650,
plosives 653, 659 797, 802, 803, 805, 807, 808, 813,
polymorphisms 634, 635, 636, 886, 888, 893 858–860,  931
pons 373, 384, 388, 454 production. See sentence production; word
positron emission tomography (PET) 35, 73, prosody 137, 164, 174, 232, 235, 247, 250, 263, 371,
99, 161, 212, 297, 349, 409, 475, 632, 651, 631, 654, 655, 736, 737, 749, 879, 884,
712, 769, 838, 883 890, 891, 898, 908, 916, 933
post-lexical 461, 476, 477, 479–481, 510, 696, proverbs 710, 715, 721, 722
697, 858 pseudowords. See nonwords
postsynaptic potentials 43, 50, 116, 348 psychiatry 213
pragmatics 10, 58, 78, 234, 276, 277, 408, 684, psycholinguistics 1, 3, 737–740, 747, 748, 755,
747, 750, 751, 755, 760, 770, 894 759, 881
praxis 577, 582, 592 pulvinar 855, 856
precuneus 139, 481, 524, 565, 719, 721–723, putamen 264–268, 273, 280, 377, 379, 380, 387,
756, 891 407, 456–458, 616, 854, 857, 863, 866
prefrontal cortex 101, 105, 167, 198, 201, 253, 262– pyramidal cells 43, 116, 119, 381
264, 296, 300, 303, 351, 376, 378–380, 387,
388, 413, 526, 565, 615, 660, 678, 683, 687,
719, 721, 724, 745, 841, 853, 854, 924, 931 reaction times (RTs) 102–108, 239, 296, 378,
pre-lexical 133, 407, 608, 648, 649, 696 500, 548, 725, 802, 803, 857, 863, 892
premotor cortex 83, 84, 141, 187, 197, 199, 202, reading
303, 373, 374, 454, 457, 459, 460, 481, aloud 26, 101, 487, 607, 887, 893
507, 522, 578, 579, 585, 592, 593, 612, 659, comprehension 611, 636
678, 681, 721, 811, 841, 862, 933 difficulties 432, 628
preoperative assessments 101, 108, 187, 192, 195, network 604, 609, 610, 611, 614, 616
317, 323, 330, 331, 654, 897 reading disabilities/disorders 604, 632, 888
presupplementary motor area (preSMA) 200, tasks 436, 606, 608, 610, 632, 840, 889, 893
220, 263–265, 274, 373, 374, 376, 378– recall 244, 303, 304, 344, 416, 458, 665, 689, 725,
380, 385–388, 854, 860 828, 829, 831, 833, 834, 838, 839, 841
presurgical assessments. See preoperative receptor 348, 373, 591, 881
preterm infants 160, 176 recovery 7, 10, 12, 20–22, 27, 107, 108, 199, 213,
primary 219, 220, 275, 318, 330, 331, 443, 614,
auditory cortex 30, 61, 348, 353, 527, 662, 913, 799, 806, 814
914, 915, 917, 920, 921, 922, 934 referential intention 749, 751–753, 755, 758, 759
motor cortex 63, 194, 450, 451, 452, 453, 454, regions of interest (ROIs) 83, 553, 586, 609,
457, 462, 463, 659 712, 930
progressive aphasia (PPA) 22, 220, 386, 680, rehabilitation 7, 11, 12, 108, 109, 166
687, 798 rehearsal 267, 272, 274, 356, 460, 828, 831–835,
agrammatic variant (PPA-G) 798–801, 838–841,  861
803, 805, 812 reliability 76, 85, 186, 192, 200, 203, 572, 897
logopenic variant (PPA-L) 799, 800 repetition priming 436, 650
semantic variant (PPA-S) 799, 800 (see representational similarity. See multivoxel
also semantic dementia) pattern analysis
966 Index
respiration 163, 167, 169, 176, 371, 373, 382, 450, representations 25, 130, 265, 506, 550, 555,
452, 455, 457, 460 565, 606, 610, 805
resting state 76, 84, 85, 129, 142, 162, 174, retrieval 487, 563, 605, 856
188, 272, 281, 413, 443, 591, 605, 684, similarity 552, 553, 557, 559, 561, 562, 564,
854, 898 566–568,  572
rhyming 606, 658, 838 system 81, 106, 198, 534, 774
rhythms 44, 96, 120, 135–137, 143, 173, 371, sensorimotor 9, 126, 136, 138, 140, 160, 187, 191,
465, 632, 663, 664, 688, 890, 918, 930, 196, 266, 267, 371, 375, 380, 383, 384,
932, 933 450, 455–457, 460, 463, 499, 501, 502,
Russian 27, 28, 649, 797, 803 510, 523, 531, 533, 537, 538, 581, 612, 626,
677, 681, 721, 933
cortex 138, 140, 450
saccades 49, 630, 692 integration 383, 384, 450, 677
sample size 81, 326, 351 system 9, 375, 502, 721
schizophrenia 154, 710, 720, 722, 725, 893, 898 sentences
school 166, 238, 245, 248, 626, 911, 912 complex 5, 219, 241, 242, 244, 249, 267,
searchlight classification. See multivoxel 302, 355, 798–801, 808, 812, 813, 836,
pattern analysis 864–866,  892
seizures 187, 189, 195, 239, 317–319, 322, 324, 327, processing 408, 681, 695, 805, 809, 814,
328, 330, 452, 475, 880 864, 865
self-monitoring 473, 477, 478, 485 production 797, 803, 808, 809
semantics 2, 7, 25, 26, 58, 63, 81, 193, 197, 198, short term memory 141, 272, 274, 417, 428, 431,
426, 436, 566, 577, 603, 605, 610, 661, 628, 755, 828–832, 834, 836–838, 841,
684, 686, 687, 694, 760, 801, 856, 857, 843, 858
859, 862, 931, 932, 935 sign language 274, 295, 342, 344, 403–412, 413,
access 56, 932 415–418
categories 271, 487, 555, 558, 563–566, 578, signing 342, 343, 344, 345, 358, 403, 404, 405,
695, 882, 931 407, 408, 410, 411, 413, 414, 415, 416, 417
content 63, 527, 529, 530, 532–534, 536, 553, simulation 235, 531, 720, 758, 834, 863
565, 566, 785 single nucleotide polymorphisms (SNPs)
context 302, 486 634–636
dementia 22, 23, 25–29, 931, 932 social
features 144, 510, 552, 561, 562, 566, 572, 856 cognition 524, 711
impairment 25, 26, 29, 584 communication 213, 217
interference 480, 481, 483–485, intention 749, 752–759
487–490,  860 somatosensory cortex 128, 235, 507, 586
knowledge 28, 29, 106, 553, 564, 581, 583, somatotopy 452, 721
931, 932 sonography 320, 881
memory 29, 386, 520, 535, 548, 550, 551, sounds 2, 4, 28, 31, 54, 56, 61, 78, 83, 84, 131, 134,
722, 856 138, 144, 172, 175, 270, 271, 273, 305, 340,
network 8, 56, 520, 566, 858, 931 345, 348, 353, 356, 375, 377, 378, 381, 426,
priming 270, 858, 931 432, 433, 435, 527, 563, 627, 628, 634, 648,
processing 10, 26, 33, 102, 116, 131, 132, 134, 651, 655, 680, 834, 841, 864, 884, 885,
140, 141, 143, 199, 267, 268, 273, 281, 302, 908, 912, 913, 918, 920, 923, 924, 927
384, 386, 477, 484, 520, 525, 531, 534, 536, Spanish 64, 269, 279, 534, 610–612, 616, 776
538, 565, 604, 607, 611, 614, 684, 685, 812, sparse temporal sampling (fMRI) 75, 78,
856–858, 861–863, 866, 867, 881, 888, 931 79, 480
Index 967
spatial resolution 10, 85, 98, 128, 142, 154, 159, electrostimulation 187, 189, 191–193, 195, 199,
161–163, 192, 648, 880, 935 201, 203
speech lesions 275, 851, 852, 859
apraxia 191, 378 structures 43, 107, 186, 203, 237, 252, 265, 272,
articulation 32, 202, 386 407, 632, 745, 810, 852, 867
comprehension 62, 262, 304, 351, 354, 355, sublexical 25, 301, 431–433, 443, 610, 648–650,
358, 665, 867, 883–885 659, 660
disorders 451, 453, 455, 457, 459, 461–465 substantia nigra 373, 377, 415, 456, 457, 853
dominance 880, 881, 884, 896 subthalamic nucleus 194, 387, 853
errors 62, 460, 465, 472, 479, 505, 506 superior frontal gyrus 139, 376, 387, 538, 717,
impairment 220, 453, 456, 460, 465 886, 931
lateralization 882, 885, 886 superior longitudinal fasciculus 199–201, 203,
perception 6, 9, 10, 78, 84, 130–132, 137, 141, 267, 281, 332, 460, 506, 612, 681, 812
172, 301, 305, 340, 352, 354, 356, 384, 408, superior parietal lobule 139, 197, 410, 431, 438,
505, 628, 647–651, 653, 655, 657, 659–661, 578–580, 591, 883
663–665, 680, 836, 837, 843, 882–886 superior temporal
processing 61, 132, 143, 176, 177, 351, 354, 355, cortex 132, 133, 138, 143, 374, 383, 414, 459,
405, 406, 508, 632, 635, 648, 661, 678– 650, 839, 842, 909, 923, 926
681, 842, 879, 883–886, 888, 897, 916, gyrus 5, 7, 30, 102, 105, 139, 194, 199, 218, 235,
917, 930 280, 301, 303, 324, 350, 373, 374, 404,
production 3, 10, 12, 29, 30, 32, 33, 76, 79, 81, 414, 433, 450, 477, 504, 604–606, 650,
100, 126, 137, 141, 266, 267, 305–307, 346, 717, 719, 723, 799, 836, 853, 885, 887, 888,
371–381, 383–388, 456, 472–477, 479, 890, 893, 923–925, 927, 930
481, 483, 485, 487, 489, 490, 498, 499, lobe 20, 234, 414, 839, 843, 924
505, 506, 508, 510, 680, 799, 836–838, sulcus 7, 77, 84, 198, 235, 251, 327, 374, 413,
843, 878–882, 884, 885, 887, 888, 890, 507, 589, 592, 656, 678, 680, 682, 811,
894, 896–898 832, 839, 842, 863, 884, 888, 891, 895,
sounds 31, 83, 138, 273, 305, 381, 627, 628, 634, 923, 925
648, 655, 918, 923 superordinate categories 522, 524, 556, 567
spelling 58, 425–443, 626, 634 supplementary motor area (SMA) 139, 192, 193,
spinal cord 373, 381, 382, 451 200, 218, 220, 263, 273, 373, 374, 376,
split-brain  880 387, 451, 616, 813, 838, 854, 886, 890,
statistical parametric mapping 519 909, 933
striatum 200, 262, 272, 273, 373, 376–378, supramarginal gyrus 7, 106, 139, 163, 199, 202,
387, 452, 458, 465, 852–854, 857–861, 217, 265, 271, 384, 385, 405, 406, 433,
864–867 482, 524, 586–588, 591, 606, 612, 772,
stroke 21–23, 31–34, 107, 108, 128, 140, 193, 202, 798, 838, 883, 931
213, 216, 219, 220, 231–237, 239–241, 243, surgery
245, 247–253, 262, 275, 299, 322, 329, awake 97, 108, 190, 191, 195, 196, 200, 375,
386, 404, 426, 436, 443, 502, 685, 771, 379, 388, 405
772, 779, 797, 798, 800, 801, 803, 805, syllabic 305, 372, 374, 375, 460, 631, 635, 651–
810, 811, 813, 854, 859 653, 662–664, 893
stuttering 141, 200, 387 syllabification 104, 372, 375, 473, 476
subcortical syllables 306, 371, 374, 380, 452, 459, 464, 465,
circuits 201, 268, 456, 852, 854, 855 627, 633, 648, 649, 659, 827, 833, 883,
contributions 852, 853, 855, 857, 859, 861, 886, 910, 911, 933
863–865,  867 symbols 9, 418, 461, 480, 581, 603, 607, 895
968 Index
syntactic 31, 55, 60, 136, 238, 249, 267, 302, 776, temporal resolution 109, 116, 142, 161, 167, 212,
783, 784, 786, 796, 800, 804, 810 474, 648, 690, 719, 725, 910
complexity 72, 241, 242, 244, 246, 298, 300, thalamus 21, 264–266, 350, 373, 379, 381, 388,
307, 681, 682, 688, 801, 805, 811, 812, 455, 632, 722, 851–857, 861, 867
866, 884 theta (frequency) 96, 120, 137, 635, 651, 657,
movement 804–806 662–665,  933
processing 10, 58–60, 80, 132, 200, 275, 277, timbres 920, 921, 926, 927
456, 534, 583, 684–686, 691, 758, 806, tones 119, 131, 140, 173, 174, 176, 655, 834, 838,
812, 866, 881, 890, 929 911, 913, 914, 917–919, 925, 927, 933
structure 47, 59, 342, 738, 806, 808 tongue muscles 125, 134, 382
violations 276–279, 929, 930 tonotopy 913, 914, 916, 917, 919, 923
syntax 2, 5, 7, 58, 135, 193, 196, 219, 234, 238, tool processing 576–578, 592
241–244, 246, 263, 266–268, 275–280, Tourette syndrome 458
300, 302, 405, 416, 614, 681–684, 686– tractography 192, 199, 213, 215–217, 219–222,
688, 694, 737, 801, 804, 854, 865–867, 332, 388, 453
882, 894, 909, 930, 935 transcranial direct current stimulation (tDCS)
476, 480–482, 484, 487, 488
transcranial magnetic stimulation (TMS) 6,
Talairach space 589, 711, 716 94, 95, 97, 99, 101–103, 105, 107–109,
temporal cortex 125, 126, 132, 133, 135, 136, 138, 268, 376, 476, 477, 577, 586, 648, 680,
143, 158, 197, 218, 220, 250, 267, 277, 299, 769, 840, 914
301, 302, 305, 324, 374, 383, 407, 414, treatment 107–109, 128, 141, 317, 319, 340, 357,
430, 433, 434, 459, 476, 481, 551, 556, 452, 462, 463, 806, 807, 809, 829, 915
558, 588, 592, 605, 606, 608, 633, 650, trigeminal nerve 373, 382, 451, 455, 718
654–656, 664, 681, 771, 772, 812, 839, tumor 23, 108, 128, 186, 187, 190, 191, 194,
842, 843, 909, 913, 916, 923, 926, 927 203, 388
lobe 5, 8, 20, 30, 56, 187, 199, 202, 217, 218,
220, 234, 242, 270, 272, 319, 321, 332,
386, 405, 414, 430, 483–485, 488, uncinate fasciculus 197, 198, 218, 220, 267, 273,
502, 521–523, 525, 533, 562, 586–589, 281, 332, 385, 386, 506, 681, 783, 812
592–594, 612, 651, 655, 681, 683, 684, unconscious 485, 741, 743, 749, 752
689, 720, 756, 771, 772, 829, 830, 839, unification 683, 684, 864
843, 861, 863, 884, 888, 892, 924, 926, univariate 34, 35, 79, 144, 519, 521, 522, 524, 526,
927, 931 527, 537–539, 553
lobes 23, 27, 125, 218, 323, 324, 375, 414, 416, utterances 29, 62, 232, 233, 240, 248, 346, 355,
487, 561, 581, 583, 588, 635, 650, 680, 739, 748, 757, 786, 800, 926
720, 723, 829, 922, 928, 930, 932
pole 35, 139, 197, 198, 218, 221, 269, 273, 281,
328, 783, 909, 931 ventral
posterior middle temporal 26, 33, 200, 404, pathway 266, 267, 386, 610–612, 617, 683,
563, 578, 579, 585, 586, 593, 684, 723, 888 684, 812, 922
posterior superior temporal 20, 102, 194, premotor 187, 197, 199, 202, 373, 374, 454, 459,
324, 327, 373, 374, 385, 404, 414, 680– 465, 579, 585, 593
682, 836, 839, 884, 888, 890, 891 stream 197, 198, 384–386, 499, 502, 506, 559,
temporal cues 655, 665, 927 561, 562, 577–579, 584, 590, 678–681,
temporal processing 304, 626, 649, 655 687–689, 841–843,  884
Index 969
temporal 23, 29, 386, 430, 431, 433, 434, 436, Wernicke’s area 21, 30, 102, 103, 106, 187, 196,
501, 502, 551, 567, 578, 592, 771, 772, 202, 218, 234, 267, 324, 326, 351, 371, 405,
775, 856 476, 478, 720, 854, 883
verbs 24, 25, 135, 136, 144, 270, 520, 524, 532– white-matter 82, 281, 614, 615, 814, 888,
534, 721, 758, 769–786, 797, 800–804, 890, 919
808–810, 862–865 Wittgenstein 550
agent 801, 802, 804, 806–808 word
argument structure 798, 801–803, 809, classes 25, 133, 134, 532, 533, 534, 536, 695,
810, 867 769, 770, 774, 775, 777, 778, 782, 786
transitivity 136, 418, 783–786, 810 comprehension 29, 220, 221, 234, 236, 245,
unaccusative 804, 809 249, 799, 802, 812
virtual lesioning 6, 8, 188, 193, 587 finding difficulties 141, 194, 305, 404,
visual cortex 98, 103, 137, 159, 198, 301, 482, 501, 799, 855
525, 529, 556, 565, 695, 879, 887, 889, form 33, 136, 198, 271, 416, 438, 439, 476, 478,
894, 897 507, 562, 695, 696, 780, 857, 887
visual field 48, 196, 352, 353, 410, 630, 880, 895 frequency 298, 374, 428, 433, 692, 760
visual word form area (VWFA), 198, 416, 438, generation 83, 264, 862, 882, 885, 888,
439, 887 893, 896
visuomotor 462–464, 502, 504, 505, 579, 592 learning 6, 136, 219, 272, 278, 857, 861
visuospatial 82, 187, 191, 201, 402, 408–410, 430, length 48, 134, 428, 430, 438, 461
608, 630, 631, 831, 890, 894, 896–898 meaning 4, 8, 133, 140, 144, 426, 538, 577,
vocabulary 135, 136, 236–241, 262, 263, 265, 267, 750, 859, 861 (see lexical semantics)
270–272, 278, 280, 296, 342, 358, 520, 892 order 279, 342, 682, 687, 688, 693, 797, 801
vocalization 236, 240, 246–248, 381, 382, 451, perception 131, 299, 301, 302, 304
458, 679, 841, 885, 914, 922–934, 927 processing 54, 130, 144, 249, 299–301, 302,
vocal tract 74, 371, 380, 383, 450, 452, 457, 461, 776, 778
462, 464, 499, 507, 508, 882, 926 production 126, 138, 234, 236, 237, 245, 307,
vocoding 305, 664 372, 376, 407, 425, 426, 428, 473–475,
vowels 140, 173, 177, 371, 380, 460, 631, 634, 648, 477, 479, 488, 489, 510, 607, 857, 860
920, 921, 924 reading 25, 26, 53, 132, 133, 138, 143, 414, 432,
voxel based lesion symptom mapping 436, 585, 616, 627, 629, 634
(VLSM). See lesion symptom mapping recognition 134, 249, 301, 345, 607, 608, 610,
voxel based morphometry (VBM) 378 629, 648–650, 651, 689, 760, 888
repetition 200, 897
retrieval 219, 306, 606, 855, 856, 881, 893
Wada test 108, 140, 318–321, 323, 325, 654, 880, 884 wordnet 520, 521, 524
waveform 44, 48–52, 55, 481, 482, 485, 488, 489, working memory 59, 60, 102, 105, 194, 197,
651–654, 662, 786 199–201, 219, 274, 296, 298, 351, 356,
wavelengths 155–157 416, 417, 426–430, 434, 440, 443, 456,
wavelet 171 538, 628, 682, 827–835, 837–843, 856,
Wernicke, Carl, 1, 2, 20, 31, 72, 212, 232, 234, 860, 864, 866, 867, 891, 929, 930, 935
241, 267, 384, 500, 504, 510, 654, 655, writing. See language, written
657, 835, 883 system 425, 426, 603, 610, 611

The Oxford Handbook of Neurolinguist (2019)

Uploaded by

Copyright:

Available Formats

The Oxford Handbook of Neurolinguist (2019)

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

The Oxford Handbook of Neurolinguist (2019)

Uploaded by

Copyright:

Available Formats

T h e Ox f o r d H a n d b o o k o f

Published in the United States of America by Oxford University Press

© Oxford University Press 2019

All rights reserved. No part of this publication may be reproduced, stored in

You must not circulate this work in any other form

Library of Congress Cataloging-in-Publication Data

8. What Has Direct Cortical and Subcortical Electrostimulation

PA RT I I I . A RT IC U L AT ION A N D P RODU C T ION

19. Investigating the Spatial and Temporal Components of Speech

Greig I. de Zubicaray and Niels O. Schiller

Neurolinguistics is a highly interdisciplinary field, with influences from psycholin-

Ingrid Aichert, PhD, is a speech-​language pathologist. She works as a Research

Ido Davidesco is a Research Assistant Professor at the Teaching and Learning

Isabelle Deschamps is a Professor at Georgian College, Orillia, Ontario, Canada, and

Anthony Steven Dick is Associate Professor of Developmental Science and Director

Hugues Duffau (MD, PhD) is Professor and Chairman of the Neurosurgery

Kara D. Federmeier is a Professor in the Department of Psychology and the

Adeen Flinker is an Assistant Professor of Neurology at the New York University

Stephanie J. Forkel is an Honorary Lecturer at the Departments of Neuroimaging and

currently applies to identify neuroimaging predictors of language recovery after brain

Frank E. Garcea completed his PhD in Cognitive Neuroscience in the Department of

Center. Her research focuses on language function in healthy populations as well as

Neuropsychology of Language and

Experimental Approaches to the Study

• Are speech perception representations motor (articulatorily or gesturally based)

Future Directions and Challenges

Friston, K. J. (1994). Functional and effective connectivity in neuroimaging: A synthesis.

Patients with Acquired Aphasias

Second, neurodegenerative diseases tend to impact functional networks of regions,

The Cognitive Neuropsychological

“This is a guide; these are _​_​_​” (guides)

The Syndromic Approach

A second major approach to patient-​based studies of the neural basis of language is to

The Lesion-​Deficit Approach

(a) (b) (c)

(d) (e) (f)

Patients with acquired aphasias due to stroke, neurodegenerative disease, neurosur-

Levelt, W. J. M. (2013). A history of psycholinguistics: The pre-​Chomskyan era. Oxford: Oxford

Michelle Leckey and Kara D. Federmeier

Introduction and History

Event-​R elated Potentials:

of artificially creating a low-​frequency signal in the data by undersampling a high-​

Event-​R elated Potentials:

To collect interpretable ERPs, several important design considerations should be kept

susceptible to noise. However, when multiple, differentiable responses occur in rapid

In addition to amplitude differences and their timing, the latency of well-​defined

Known ERP Components Used

critical word, to avoid contamination by co-​articulatory information). However, this,

(B) Frontal Positivity

language and differences between the hemisphere’s comprehension biases (Federmeier,

A well-​established property of the N400 is its insensitivity to contextual constraint

P600 and Other Posterior Positivities

Another well-​known ERP component used in studies of language processing is

The P3b is a domain-​general response that is sensitive to stimulus probability, among

Left Anterior Negativity

NRef and Other

The mismatch negativity (MMN) is seen in response to an auditory change, with

Lateralized Readiness Potential

The lateralized readiness potential (LRP) is a domain-​general index of motor-​response

Error-​R elated Negativity

Ingrid Aichert, PhD, is a speech-language pathologist. She works as a Research

“This is a guide; these are ___” (guides)

A second major approach to patient-based studies of the neural basis of language is to

The Lesion-Deficit Approach

Levelt, W. J. M. (2013). A history of psycholinguistics: The pre-Chomskyan era. Oxford: Oxford

Event-R elated Potentials:

of artificially creating a low-frequency signal in the data by undersampling a high-

Event-R elated Potentials:

In addition to amplitude differences and their timing, the latency of well-defined

critical word, to avoid contamination by co-articulatory information). However, this,

A well-established property of the N400 is its insensitivity to contextual constraint

Another well-known ERP component used in studies of language processing is

The P3b is a domain-general response that is sensitive to stimulus probability, among

The lateralized readiness potential (LRP) is a domain-general index of motor-response

Error-R elated Negativity

Stefan Heim and Karsten Specht

The Resting Brain: Low-L evel Baseline

Buxton, R. B. (2012). Dynamic models of BOLD contrast. NeuroImage, 62, 953–961.

Chronometric versus Repetitive TMS

Pioneers in the stimulation of language-related areas were not cognitive neuroscientists,

assumption often is made that a successful TMS-induced behavioral change is indica-

1999). In these studies, rTMS to Wernicke’s area influenced picture-naming latencies,

Silvanto, J., & Pascual-Leone, A. (2008). State-dependency of transcranial magnetic stimula-

Real-Time MEG Signals Facilitate Several

MEG Facilitates Analysis of Real-Time Connectivity

Spatial Dependence of Distributed Source-Level

New Devices May Increase the Spatial Accuracy of MEG

Task-Related Connectivity Can Be Tracked as