A Dynamical Systems Approach to Language Change

: The Nature of Grammatical Stability

Whitney Tabor, Hyosun Lee

PAW Talk

University of Connecticut

March 8, 2024

PAW Talk University of Connecticut Marc

Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
1 / 34: The Nature of Grammatical Stability
Overview

WHAT? Language change — Grammaticalization

HOW? Using dynamical systems approach — Metric Grammars

→ Build a “sticky medium” of language system and find the attractor

PAW Talk University of Connecticut Marc

Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
2 / 34: The Nature of Grammatical Stability
Self-organization

Working Def. Under restricted en-

vironmental conditions, a group of au-
tonomously acting but regularly inter-
acting small entities (e.g., molecules,
neurons, animals, local flow character-
istics, etc.) give rise to organization at
the scale of the ensemble.

→ Self-organization of language system

PAW Talk University of Connecticut Marc
Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
3 / 34: The Nature of Grammatical Stability
Grammaticalization

Traditional Definition
: A historical change where content morphemes (e.g., noun, verb, adjective) in
specific contexts turn gradually (over 10s to 100s of years) into functional
morphemes (e.g., preposition, determiner, pronoun) (Meillet, 1912)

There are crucial bridge constructions during this evolution process

Language undergoes gradual change over time

Bridge construction
: ambiguous between two meanings
: speakers use the bridge construction a lot when transition starts to occur

PAW Talk University of Connecticut Marc

Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
4 / 34: The Nature of Grammatical Stability
Grammaticalization

Traditional Definition
: A historical change where content morphemes (e.g., noun, verb, adjective) in
specific contexts turn gradually (over 10s to 100s of years) into functional
morphemes (e.g., preposition, determiner, pronoun) (Meillet, 1912)

Example
be going to
(a) I am going to school. [Original: a content morpheme referencing motion]
(b) It is going to rain. [Modern: a marker of future tense]
(c) “The kings and princes are going to see the queen’s picture.” (Shakespeare)
[Bridge Construction: Ambiguous (motion/future)]

PAW Talk University of Connecticut Marc

Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
5 / 34: The Nature of Grammatical Stability
Grammaticalization

More General Definition

: In addition to a traditional definition (content morphemes to function
morphemes) it includes a case where “less grammatical” morpheme turns into “more
grammatical” morpheme

Example
fun
(a) “Don’t mind me tho’–For all my fun and jokes.” (1751, OED) [Original:
Noun]
(b) The picnic is very fun. [Modern: Adjective (first in early 1800s, OED)]
(c) This is fun. [Bridge Construction: Ambiguous (Noun/Adjective)]

PAW Talk University of Connecticut Marc

Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
6 / 34: The Nature of Grammatical Stability
Traditional Theory: Context Free Grammar (CFG) tree

NP VP

Det N’[count] V NP

The picnic was AdjP N’[mass]

great fun

PAW Talk University of Connecticut Marc

Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
7 / 34: The Nature of Grammatical Stability
Probabilistic Context Free Grammar (PCFG)
Phrasal Rules Lexical Rules

1.00 Root → S EndP 0.29 N’[count] → 0.55 bear, 0.27 picnic, 0.18 mouse

1.00 S → NP VP 0.29 N’[mass] → 0.55 water, 0.27 work, 0.18 fun

0.67 NP → Det N’[count] 0.29 AdjP → 0.55 great, 0.27 quiet, 0.18 sweet
0.33 NP → N’[mass]
0.44 AAdvP → 0.55 very, 0.27 really, 0.18 somewhat
0.33 N’[count] → AdjP N’[count]
0.29 AdvP → 0.55 soon, 0.27 quickly, 0.18 slowly
0.33 N’[mass] → AdjP N’[mass]
0.19 VP → 0.55 sleeps, 0.27 eats, 0.18 pauses
0.33 AdjP → AAdv AdjP
0.44 Vtans → 0.55 has, 0.27 lieks, 0.18 chases
0.33 AdvP → AAdv AdvP
0.55 Det → 0.55 the, 0.27 a, 0.18 some
0.67 VP → Vtrans NP
0.33 VP → BE AdjP 0.55 BE → 0.55 is, 0.27 seems, 0.18 appears
0.67 VP → BE N’[mass]
0.33 VP → Adv VP 1.00 EndP → .

PAW Talk University of Connecticut Marc

Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
8 / 34: The Nature of Grammatical Stability
Language Evolution

What trees CAN do

: Tree structures (i.e., classical theory of mental representation) can depict
structures of given sentences

What trees CANNOT do

: Tree structures cannot explain the evolution process itself
: NOT able to account for gradual change due to its discrete topology

PAW Talk University of Connecticut Marc

Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
9 / 34: The Nature of Grammatical Stability
QQCG
Quantitative-Qualitative Correlations in Grammaticalization (QQCG)
: Quantitative change is followed by qualitative change

Example sentences

(a) What fun! (Noun)

(b) It is fun. (Ambiguous)

(c) The dinner was rather fun.

(Predicate Adjective)

(d) We had a fun time. (Attributive

Adjective)
Figure: Data from De Smet, 2012
PAW Talk University of Connecticut Marc
Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
10 / 34: The Nature of Grammatical Stability
Connection to Dynamical System

QQCG suggests BIFURCATION

QQCG: Quantitative change is followed by qualitative change
Bifurcation: There is a critical value of control parameter that leads to a
phase transition (i.e., change of attractor configuration)

If we take the dynamical approach, a key question is:

What are the attractors of language system?

QQCG = Quantitative-Qualitative Correspondence in Grammaticalization

PAW Talk University of Connecticut Marc

Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
11 / 34: The Nature of Grammatical Stability
Proposal

PCFGs (Probabilistic Context Free Grammars) are the

attractors of the natural language evolution.

PAW Talk University of Connecticut Marc

Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
12 / 34: The Nature of Grammatical Stability
Connection to Another Dynamical Regime
Circle map

K
θi+1 = θi + Ω + 2π
sin(2πθi+1 )

Arnold Tongue
In the parameter space (K, Ω) of the circle map,
different parameter settings are associated with
different rhythmic patterns

→ We envision a similar diagram which can

portray the locations of various PCFG
attractors of human languages

PCFG = Probabilistic Context Free Grammar

PAW Talk University of Connecticut Marc
Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
13 / 34: The Nature of Grammatical Stability
Egg OR Chicken?

Q. Driving force of grammaticalization?

H1. inherent tendency of attractiveness captured by PCFG
H2. external (world) force which push languages AWAY from
context-freeness

Causal conundrum occurs especially in the evolutionary context

→ Instead of choosing either one of those, we can talk about the
dynamics between those two
PCFG = Probabilistic Context Free Grammar

PAW Talk University of Connecticut Marc

Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
14 / 34: The Nature of Grammatical Stability
Sticky Medium

PAW Talk University of Connecticut Marc

Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
15 / 34: The Nature of Grammatical Stability
Connection to Dynamical System
Language system as a sticky medium
Belousov Zhabotinsky reaction
https://www.youtube.com/watch?v=jRQAndvF4sM

Sticky medium
: given environment that is able to
reflect the external force not
disappearing over time

Perturbation
: external (world) force

Visual structure/ion
concentration
: structure at the moment
PAW Talk University of Connecticut Marc
Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
16 / 34: The Nature of Grammatical Stability
Goal

1 Our goal is to discover appropriate sticky medium of language

system
2 We replace discrete topology with connected metric topology
3 We map the tree information in the symbolic space to the structure
in the metric grammar space (i.e., metric grammar tree)

PAW Talk University of Connecticut Marc

Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
17 / 34: The Nature of Grammatical Stability
PCFG vs. Metric Grammars
Example Sentence: The picnic is great fun

PCFG Metric Grammar

Root

S
S EndP

“.”
NP VP
NP VP

BE AdvP
Det N’[count] BE NP
Det “seems”
“is” “soon”
“quickly”
NbarCount NbarMass AdjP

The picnic is AdjP N’[mass] “a” “this”

AAdv

“picnic” “bear” “great”

“fun” “work”

“sweet” “very”
“mouse” “water” “quiet”
great fun “really”

PCFG = Probabilistic Context Free Grammar

PAW Talk University of Connecticut Marc
Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
18 / 34: The Nature of Grammatical Stability
Metric Grammar Trees: a distinctive feature

Mothers-below and Daughters above

(In a standard PCFG tree, these are the same entity, but in
a Metric Grammar Tree (MG tree), they are separate
objects.)
The separation of mothers-below from daughters-above provides the
opening for self-organization
PCFG = Probabilistic Context Free Grammar

PAW Talk University of Connecticut Marc

Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
19 / 34: The Nature of Grammatical Stability
Metric Grammar Architecture

A metric grammar is composed of five components plus control parameters:

M G = (M, T, L, N N, Root, Control Params) (1)

M is a (connected) metric space, T is a list of treelets (see below), L is a list of
words, and N N is a neural network that maps treelet daughters to treelet
mothers. Root ∈ M is the starting point for sentence generation. Control
Params = α, boost, nlayers, lrates.

The treelets take two forms:

A mother point, a left daughter-point, and a right daughter-point, all in M
(phrasal treelets).
A mother point in M and a word. (lexical treelets).
PAW Talk University of Connecticut Marc
Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
20 / 34: The Nature of Grammatical Stability
Metric Grammar Processing: Generation

1. Start at the Root point

2. Randomly select a treelet, tj , favoring nearby treelets (pj = dij + boost−α).
3. If tj is a lexical treelet, generate its word.
4. Otherwise (if tj is a phrasal treelet), take each daughter in turn as the new
“Root” and go to Step 1.
Repeat until finished. The sentence is the generated sequence of words.

Key observation: If a cluster is concentrated on a single point, then the metric

grammar exactly emulates a Probabilistic Context Free Grammar (PCFG).

Thus we define the degree of context freeness of a metric grammar as the total
variance of its clusters.
PAW Talk University of Connecticut Marc
Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
21 / 34: The Nature of Grammatical Stability
Metric Grammar Processing: Parsing

Poor Man’s Self-Organization:

1. Consider a sequence of words of length n.
2. Generate all binary trees over n words.
3. For each binary tree, start at the bottom, use NN to find a mother for each
pair of daughters. Randomly select a daughter-above for each mother-below,
favoring nearby daughters-above.
4. Train the neural network to map the daughters below to a mother
closer to the chosen daughter above.
5. Replace old treelets with newly generated ones.
6. Repeat from Step 3 until the root of the binary tree is reached.
Key property: Processing a single sentence changes the grammar.
PAW Talk University of Connecticut Marc
Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
22 / 34: The Nature of Grammatical Stability
Autonomous language evolution dynamics

mg(t + 1) = f (mg(t)) (2)

where f = <generate a sentence, parse the sentence, update the metric grammar>
Procedure:

PCFG configures the initial metric grammar state, mg(0).

Generate. Parse. → mg(1)
Generate. Parse. → mg(2)
Generate. Parse. → mg(3)
...

PCFG = Probabilistic Context Free Grammar

PAW Talk University of Connecticut Marc

Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
23 / 34: The Nature of Grammatical Stability
Simple Model

Let pi be the point associated with the mother of the i’th treelet of the
metric grammar. Let dij be the distance between pi and pj . Consider
the following iterated map dynamical system:
X
pi (t + 1) = d−α
ij (t) pj (t) (3)
j̸=i

(α is a control parameter, here set to 2).

Intuition: each treelet updates to the average of its neighbors where

nearby neighbors weigh more heavily in the average.
PAW Talk University of Connecticut Marc
Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
24 / 34: The Nature of Grammatical Stability
Demo

Matlab Demo

Contraction phenomenon: the model contracts onto a countable

(zero-dimensional) set.

Language interpretation of contraction: certain PCFG’s are attractors.

PCFG = Probabilistic Context Free Grammar

PAW Talk University of Connecticut Marc

Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
25 / 34: The Nature of Grammatical Stability
The Simple Model stabilizes on context freeness

PAW Talk University of Connecticut Marc

Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
26 / 34: The Nature of Grammatical Stability
Case Study: fun

Examples
(a) What fun! (Noun—18th cent.)
(b) It is fun. (Ambiguous—18th cent.)
(c) The dinner was rather fun. (Predicate Adjective—19th century)
(d) We had a fun time. (Attributive Adjective)

PAW Talk University of Connecticut Marc

Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
27 / 34: The Nature of Grammatical Stability
Grammaticalization simulation: set-up
• Use a PCFG to configure a metric grammar model of 18th century fun.

• Distort
the position of some instances of the treelet, N → fun to
decrease the frequency of unambiguous noun occurrences
and increase the frequency of ambiguous cases.
• Compute the mother position
induced by the neural network for the phrase, BE + fun
• Measure the distance
between this mother location and MEAN(AdjP).

• Prediction: The distance will decrease.

PCFG = Probabilistic Context Free Grammar
PAW Talk University of Connecticut Marc
Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
28 / 34: The Nature of Grammatical Stability
Grammaticalization simulation: results
Evidence for QQCG of fun (actually sort of/kind of) in a metric grammar
model:

QQCG = Quantitative-Qualitative Correspondence in Grammaticalization

PAW Talk University of Connecticut Marc
Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
29 / 34: The Nature of Grammatical Stability
Assessing the PCFG Attractor claim
The feature of a Metric Grammar that gives it the sticky-medium property and
thus allows it to predict QQCG is the training of the neural network to bring
each mother-below closer to its daughter-above.
This property causes PCFGs to be attractors
Are PCFGs attractors?
Answer: Yes, we think so:
All linguistic theories use (P)CFGs
Linguistics generally agree that the syntactic theory needs at most a mildly
context-sensitive grammar (very close to context free)
Many basic grammar changes involve a shift to greater context freeness (e.g.,
corpora → corpuses, whom → who)
18 grammaticalization episodes carefully studied by [3] exhibit a change from
less context freeness to more context freeness.

PCFG = Probabilistic Context Free Grammar

QQCG = Quantitative-Qualitive Correspondence in Grammaticalization PAW Talk University of Connecticut Marc
Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
30 / 34: The Nature of Grammatical Stability
Conclusions
1. Focus has been on grammaticalization (e.g, Motion be going to → Future be
going to, Noun fun → Adj fun
2. Current theories, built on discrete topologies (symbolic atoms), are
challenged by QQCG.
3. Turn to dynamical systems theory: bifurcations, sticky medium,
autonomous and driven dynamics
4. Metric grammar models, built on connected (metric) topologies, are sticky
media, and exhibit QQCG.
5. PCFGs are attractors
6. Sticky medium: the world provokes a shift and the language system picks it
up and runs with it.

PCFG = Probabilistic Context Free Grammar

QQCG = Quantitative-Qualitive Correspondence in Grammaticalization
PAW Talk University of Connecticut Marc
Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
31 / 34: The Nature of Grammatical Stability
Backup Slide: Examples of Grammaticalization

English
be going to (V to Aux)
sort/kind of (N+P to AdvP)
fun (N to Adj)

German
wegen (N to P)

Korean
keun-dae (sentence-initial/medial position to sentence-final position)

PAW Talk University of Connecticut Marc

Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
32 / 34: The Nature of Grammatical Stability
Change Language Group Source
Verb [of saying] > Complementizer [embedding] Kwa (Niger-Congo) Lord 1976
Verb [of locational existence] > Prep [locative] Kwa (Niger-Congo) Lord 1973
Verb [locational exisitence] > Postposition [locative] Senufo (Niger-Congo) Carlson 1991
Verb [’meet, assemble’] > Preposition [comitative] Kwa (Niger-Congo) Lord 1973
Verb [’know’] > Auxiliary [’can’] English (IE) Lightfoot 1979
Noun [’time+Dat’] > Compl. [while] English (IE) Traugott 1993
Noun [’way’ +Dat’] > Preposition [’because of’] German (IE) Hopper and Traugott 1993
Noun [’mind’] > Adjective → Adverb suffix Romance (IE) Lausberg 1962 (H&T 93)
Noun [’step’] > Negation Marker Romance (IE) Möhrer 1943, etc.
Adjective [’full’] > Noun → Noun suffix English (IE) Marchand 1966
Article > Gender Marker (bantu) Greenberg 1978
Postposition > (inflectional) Complementizer Rama (Chibchan) Craig 1991
Preposition > Conjunction (’and’) Kwa (Niger Congo) Lord 1973
Preposition > Conjunction To’aba’ita (Austr.) Lichtenberk 1991
Noun-mission [Shi ’使(命)’] > Verb-Causative [Shi ’使(导致)’] Chinese Liu 2023

Table: Some Transition-types in Grammaticalizations

PAW Talk University of Connecticut Marc

Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
33 / 34: The Nature of Grammatical Stability
References

Hendrik De Smet.
The course of actualization.
Language, 88:601 – 633, 2012.
Antoine Meillet.
L’evolution des formes grammaticales.
In Linguistique Historique et Linguistic Generale, pages 13–48. Champion,
Paris, 1912.
Ian G. Roberts and Anna Rousseau.
Syntactic change : a minimalist approach to grammaticalization.
Cambridge University Press, Cambridge, U.K., 2003.

PAW Talk University of Connecticut Marc

Whitney Tabor, Hyosun Lee A Dynamical Systems Approach to Language Change
34 / 34: The Nature of Grammatical Stability