Netherlands

Event structure describes the relationships between general semantics (Aktionsart) of the verb and its syntactic properties, separating verbs into two classes: telic verbs, which denote change of state events with an inherent end-point or boundary (catch, rescue), and atelic, which refer to homogenous activities (tease, host). As telic verbs describe events, in which the internal argument (Patient) is affected, we hypothesized that processing of telic verb template would activate syntactic position of the Patient during sentence comprehension. Event-related brain potentials (ERPs) were recorded from 20 English speakers, who read sentences with reduced Object relative clauses, in which the verb was either telic or atelic. ERPs in relative clauses diverged on the definite article preceding the Agent: the atelic condition was characterized by larger amplitude negativity at the N100. Such processing differences are explained by activation of the syntactic position for the Patient by the event structure template of telic verbs.

Motion capture studies show that American Sign Language (ASL) signers distinguish end-points in telic verb signs by means of marked hand articulator motion, which rapidly decelerates to a stop at the end of these signs, as compared to atelic signs (Malaia and Wilbur, in press). Non-signers also show sensitivity to velocity in deceleration cues for event segmentation in visual scenes (Zacks et al., 2010; Zacks et al., 2006), introducing the question of whether the neural regions used by ASL signers for sign language verb processing might be similar to those used by non-signers for event segmentation.The present study investigated the neural substrate of predicate perception and linguistic processing in ASL. Observed patterns of activation demonstrate that Deaf signers process telic verb signs as having higher phonological complexity as compared to atelic verb signs. These results, together with previous neuroimaging data on spoken and sign languages (Shetreet et al., 2010; Emmorey et al., 2009), illustrate a route for how a prominent perceptual-kinematic feature used for non-linguistic event segmentation might come to be processed as an abstract linguistic feature due to sign language exposure.► Humans segment visual scenes based on speed and acceleration of objects. ► Signers' hands move and decelerate faster when signing change-of-state ASL verbs. ► Kinematic features used for perceptual segmentation might be phonological in ASL. ► fMRI shows that change-of-state ASL verbs activate STG more than other verbs. ► Scene-segmenting kinematic features are processed as phonological features by signers.

Using sign language research as an example, we argue that both the cross-linguistic descriptive approach to data, advocated by Evans and Levinson (2009), as well as abstract (‘formal’) analyses are necessary steps towards the development of “neurolinguistic primitives” for investigating how human languages are instantiated in the brain.

Early acquisition of a natural language, signed or spoken, has been shown to fundamentally impact both one's ability to use the first language, and the ability to learn subsequent languages later in life . This review summarizes a number of recent neuroimaging studies in order to detail the neural bases of sign language acquisition. The logic of this review is to present research reports that contribute to the bigger picture showing that people who acquire a natural language, spoken or signed, in the normal way possess specialized linguistic abilities and brain functions that are missing or deficient in people whose exposure to natural language is delayed or absent. Comparing the function of each brain region with regards to the processing of spoken and sign languages, we attempt to clarify the role each region plays in language processing in general, and to outline the challenges and remaining questions in understanding language processing in the brain.

Verbs contain multifaceted information about both the semantics of an action, and potential argument structures. Linguistic theory classifies verbs according to whether the denoted action has an inherent (telic) end-point (fall, awaken), or whether it is considered homogenous, or atelic (read, worship). The aim of our study was to examine how this distinction influences on-line sentence processing, investigating the effects of verbal telicity on the ease of syntactic re-analysis of Object reduced relative clauses. Event-related brain potentials (ERPs) were recorded from 22 English speakers as they read sentences in which the main verb was either telic or atelic, e.g., “The actress awakened/worshipped by the writer left in a hurry”. ERPs elicited by telic and atelic verbs, the preposition “by” introducing the second argument (Agent), and the second argument itself, e.g., “writer”, were compared. Additionally, participants were grouped according to receptive syntactic proficiency: normal (NP) or high (HP). ERPs from the NP group first diverged at the second argument, with the atelic condition eliciting larger amplitude negativity at the N100, and continuing to the P200 interval. In contrast, ERPs from the HP group first diverged earlier in the sentence, on the word “by”. ERPs elicited by “by” in the atelic condition were also characterized by increased negativity, in this case significant at P200 and Anterior Negativity between 320 and 500 ms post stimulus onset. Our results support the postulated conceptual/semantic distinction underlying the two verb categories, and demonstrate that world-knowledge about actions designated by verbs and syntactic proficiency are reflected in on-line processing of sentence structure.

This paper considers neurological, formational and functional similarities between gestures and signed verb predicates. From analysis of verb sign movement, we offer suggestions for analyzing gestural movement (motion capture, kinematic analysis, trajectory internal structure). From analysis of verb sign distinctions, we offer suggestions for analyzing co-speech gesture functions.

This study explores the interaction between working memory systems-episodic and verbal working memory-and language processing by examining how differences in working memory capacity (WMC) modulated neural activation levels and functional connectivity during sentence comprehension. Activation of the inferior frontal gyrus and medial prefrontal cortex was correlated with WMC during the probe phase, suggesting that not all of the activation observed in the left IFG is language-specific, but is instead related to executive processes recruited during comprehension. Posterior cingulate/precuneus activation was negatively correlated with WMC scores during sentence reading. In addition, functional connectivity between the PCC and IFG was positively correlated with WMC while that between the PCC and the inferior parietal cortex was negatively correlated with WMC. Together these results suggest that high-capacity and low capacity readers may utilize different processing strategies with high capacity readers utilizing episodic memory systems during sentence reading to generate a representation of the events (event representation). Both groups use executive working memory processes during the probe phase.

This article presents an experimental investigation of kinematics of verb sign production in American Sign Language (ASL) using motion capture data. The results confirm that event structure differences in the meaning of the verbs are reflected in the kinematic formation: for example, in the telic verbs (THROW, HIT), the end-point of the event is marked in the verb sign movement by significantly greater deceleration, as compared to atelic verbs (SWIM, TRAVEL). This end-point marker is highly robust regardless of position of the verb in the sentence (medial vs. final), although other prominent kinematic measures, including sign duration and peak speed of dominant hand motion within the sign, are affected by prosodic processes such as Phrase Final Lengthening. The study provides the first kinematic confirmation that event structure is expressed in movement profiles of ASL verbs, up to now only supported by apparent perceptual distinctions. The findings raise further questions about the psychology of event representation both in human languages and in the human mind.

Prior studies investigating cortical processing in Deaf signers suggest that life-long
experience with sign language and/or auditory deprivation may alter the brain’s
anatomical structure and the function of brain regions typically recruited for auditory
processing (Emmorey et al., 2010; P´enicaud et al., 2013 inter alia).We report
the first investigation of the task-negative network in Deaf signers and its functional
connectivity—the temporal correlations among spatially remote neurophysiological
events. We show that Deaf signers manifest increased functional connectivity between
posterior cingulate/precuneus and left medial temporal gyrus (MTG), but also
inferior parietal lobe and medial temporal gyrus in the right hemisphere- areas that
have been found to show functional recruitment specifically during sign language
processing. These findings suggest that the organization of the brain at the level of
inter-network connectivity is likely affected by experience with processing visual
language, although sensory deprivation could be another source of the difference.
We hypothesize that connectivity alterations in the task

This study examined the electrophysiological signatures of deductive and probabilistic
reasoning. Deduction is defined as the case in which a conclusion can be found to
be true or false due to validity of argument. In probabilistic reasoning, however, conclusions
can be considered to be likely or unlikely, but not with certainty due to the lack of validity in
the form of the argument. 16 participants were presented with both types of arguments while
response times and ERPs were carried out. Participants had to decide with the presentation of
each argument, what type of reasoningwas appropriate andwhich of four responses (certainly
yes, probably yes, probably no and certainly no) was the most appropriate. Response times
indicated faster processing of deductive arguments. N2 amplitude distinguished between
positive and negative responses in the deductive condition, but not in the probabilistic one,
suggesting partial differentiation between the cognitive processes required for the two types
of reasoning.

The interaction between language and perception networks in the brain can hold the key to the
biological bases of language evolution. In language, every sentence is built around a verb, which
describes an event. During perception, humans constantly and automatically segment reality into
individualized event (verb) units. The cognitive connection between the perceptual segmentation
of events and their grammatical expression in language is a novel direction for research into the
neurobiology of language, but it may be a key example of the emergence of linguistic structure –
grammatical category – from individualized meaning.
This article summarizes the current state of this research, and is divided into three parts. The initial
overview of cross-linguistic typology of events shows that events and their boundaries are of vital
importance to linguistic communication. We then summarize what is known about perceiving and
identifying event boundaries in the most well-studied domain, that of visual perception. Here, sign
languages provide the link between visual perception and linguistic expression, showing a complex
mapping from perceptual and cognitive event segmentation to linguistic structures at the phonology–
syntax interface. The last section reviews current evidence for neural bases of event processing, showing
that identification of an event boundary (linguistic or perceptual) is used for memory updates, and
suggesting the possible role of an event-type universal in the syntactic structure of human languages.

The impact of handedness on language processing has been studied extensively and the results indicate
that there is a relationship between the two variables; however, the nature of the relationship is not at all
clear. In the current study we explored degree of handedness (DH) opposed to direction in a group of
right-handed individuals. fMRI was used to explore the impact of DH on the sentence comprehension
network. The results revealed that during sentence comprehension activation in regions linked to
semantic memory (e.g., anterior temporal cortex) were modulated by DH. Also, unexpectedly the
precuneus/posterior cingulate gyrus which has been linked to episodic memory was also affected by
DH. These results extend those reported previously by showing that the neural architecture that supports
sentence comprehension is modulated by DH. More specifically, together the results presented here
support the hypothesis proposed by Townsend, Carrithers, and Bever (2001) that DH interacts with the
language system and impacts the strategy used during sentence comprehension.

Sign language users recruit physical properties
of visual motion to convey linguistic information. Research
on American Sign Language (ASL) indicates that signers
systematically use kinematic features (e.g., velocity,
deceleration) of dominant hand motion for distinguishing
specific semantic properties of verb classes in production
(Malaia & Wilbur, 2012a) and process these distinctions as
part of the phonological structure of these verb classes in
comprehension (Malaia, Ranaweera, Wilbur, & Talavage,
2012). These studies are driven by the event visibility
hypothesis by Wilbur (2003), who proposed that such use
of kinematic features should be universal to sign language
(SL) by the grammaticalization of physics and geometry for
linguistic purposes. In a priormotion capture study, Malaia and
Wilbur (2012a) lent support for the event visibility hypothesis
in ASL, but there has not been quantitative data from other
SLs to test the generalization to other languages.
Method: The authors investigated the kinematic parameters of
predicates in Croatian Sign Language (Hrvatskom Znakovnom
Jeziku [HZJ]).
Results: Kinematic features of verb signs were affected both
by event structure of the predicate (semantics) and phrase
position within the sentence (prosody).
Conclusion: The data demonstrate that kinematic features of
motion inHZJ verb signs are recruited to conveymorphological
and prosodic information. This is the first crosslinguistic motion
capture confirmation that specific kinematic properties of
articulator motion are grammaticalized in other SLs to express
linguistic features.

This study explores the interaction between
working memory systems and language processing by
examining how differences in working memory capacity
(WMC) modulates neural activation levels and functional
connectivity during sentence comprehension. The results
indicate that two working memory systems may be
involved in sentence comprehension, the verbal working
memory system and the episodic buffer, but during different
phases of the task. A sub-region of the left inferior
frontal gyrus (BA 45) was correlated with WMC during the
probe and not during sentence reading while the only
region to reveal a correlation with WMC during sentence
reading was the posterior cingulate/precuneus area, a
region linked to event representation. In addition, functional
connectivity analysis suggests that there were two
distinct networks affected by WMC. The first was a
semantic network that included the middle temporal cortex,
an anterior region of the inferior frontal gyrus and the
inferior parietal region. The second included the posterior
cingulate and BA 45 of the inferior frontal gyrus. We
propose here that high capacity readers may generate an
event representation of the sentence during reading that
aids in comprehension and that this event representation
involves the processing of the posterior cingulate cortex.

We had human subjects perform a one-out-of-six class action
recognition task from video stimuli while undergoing functional magnetic
resonance imaging (fMRI). Support-vector machines (SVMs) were
trained on the recovered brain scans to classify actions observed during
imaging, yielding average classification accuracy of 69.73% when tested
on scans from the same subject and of 34.80% when tested on scans
from different subjects. An apples-to-apples comparison was performed
with all publicly available software that implements state-of-the-art action
recognition on the same video corpus with the same cross-validation
regimen and same partitioning into training and test sets, yielding classification
accuracies between 31.25% and 52.34%. This indicates that
one can read people’s minds better than state-of-the-art computer-vision
methods can perform action recognition.

How does the human brain represent simple compositions of objects, actors, and
actions? We had subjects view action sequence videos during neuroimaging
(fMRI) sessions and identified lexical descriptions of those videos by decoding
(SVM) the brain representations based only on their fMRI activation patterns. As
a precursor to this result, we had demonstrated that we could reliably and with
high probability decode action labels corresponding to one of six action videos
(dig, walk, etc.), again while subjects viewed the action sequence during scanning
(fMRI). This result was replicated at two different brain imaging sites with
common protocols but different subjects, showing common brain areas, including
areas known for episodic memory (PHG, MTL, high level visual pathways, etc.,
i.e., the ‘what’ and ‘where’ systems, and TPJ, i.e., ‘theory of mind’). Given these
results, we were also able to successfully show a key aspect of language compositionality
based on simultaneous decoding of object class and actor identity.
Finally, combining these novel steps in ‘brain reading’ allowed us to accurately
estimate brain representations supporting compositional decoding of a complex
event composed of an actor, a verb, a direction, and an object.