Temporal Measures of Fluency Automatic

Temporal measures of fluency:
automatic and manual extraction

of temporal variables
Soohwan Park
Purdue University
Fluency
 Two Senses of Fluency (Lennon 2000)

 A broad sense: corresponding roughly to all-round
oral proficiency
 A narrow sense: referring to the speed and
smoothness of oral delivery
 “The rapid, smooth, accurate, lucid, and efficient

translation of thought or communicative intention into
language under the temporal constraints of on-line
processing”
12/1/2009 2
(Continued)
 Fluency: „the speed and smoothness of oral

delivery‟
Speed Temporal Variables
Fluency
Oral Proficiency Smoothness

Accuracy
12/1/2009 3
Temporal Variables
 Pause
 Hesitation phenomena in utterance
 Types of pauses (Riggenbach 1991)
 Micro pauses (Articulation)
 Hesitation
 Unfilled pauses
 Filled pause (uh, um, uhr, …)
12/1/2009 4
(Continued)
 Silent pause
 Hesitation
 Generally over 0.25 seconds
 Syllables
 Basic unit of production
 Run: utterance between pauses of 0.25 second
and above (Kormos and Denes 2004)
12/1/2009 5
Temporal measures of fluency
(Ginther, Dimova and Yang 2009)
 Quantity of production
 Total response time
 Total time to produce speech sample including all
utterances and pauses
 Speech time
 Time spent on speaking including all semantic units
(partial words and filled pauses)
 Speech time ratio
 Speech time / Total response time
12/1/2009 6
(Continued)
 Rates of production
 Speech rate
 Number of syllables / Total response time * 60
 Articulation rate
 Number of syllables / Speech time * 60
 Mean syllables per run
 Number of syllables / Number of runs
 The frequency and length of pauses
12/1/2009 7
(Continued)
 Which one is better?

 Rates of production
 Normalized value of production unit
 Length (e.g. pausing time) is not reliable
 Speech rate: number of syllables and response time
 Total response time is fixed
 MSR: number of syllables and number of runs
 Number of runs = number of pauses
 Using more frequency information
12/1/2009 8
The Oral English Proficiency Test
(Ginther, Dimova and Yang 2009)
 The OEPT (Oral English Proficiency Test)
 A computer-administered, semi-direct test of oral
English proficiency
 Examinees: Graduate ITAs (international teaching
assistants) at Purdue University
 10 test items with 8 operational forms
 The responses are monologic, and fixed to the
items
 Rated by holistic scoring: the OEPT scale ranges
from 3 to 6
12/1/2009 9
Automatic measurement of temporal variables
(de Jong and Wempe 2009)
 Using a Praat script to extract the number of
syllables in a speech sample
 Finding syllable nuclei using intensity (dB) and
voicedness
 Also made the script that automatically detects
pauses (now incorporated in Praat)
 The script is available on the personal webpage of
the first author
12/1/2009 10
(Continued)
 Result from de Jong and Wempe (2009)

 Analyzed Dutch speech data
 Compare speech rate by hand and automatically
 Speech rate per speech file: r = 0.88
 50 speech files from 8 tasks of 258 speakers
 Speech rate calculation per spurt: r = 0.71
 441 spurts: pause longer than 0.4 seconds as spurt
boundary; spurt of 5 seconds or more
12/1/2009 11
Data
Chinese 3 Chinese 4 Chinese 5 Hindi 5 Hindi 6 Native Total

NP 25 25 25 25 25 25 150
CNC 25 25 25 25 25 25 150
 Chinese 3: the OEPT level 3 of Chinese examinees

 Hindi 5: the OEPT level 5 of Hindi examinees
 Hindi 6: the OEPT level 6 of Hindi examinees
 Native: the native speakers of English
 NP: the news item
 CNC: the compare and contrast item
12/1/2009 12
Procedure
 Manual extraction
 300 speech samples were transcribed manually
using Praat to extract temporal information
 Automatic extraction
 300 speech samples were processed by Praat
 Pauses: using the TextGrid (to silences) button in Praat
 Syllables: using the Praat script from de Jong and
Wempe (2009)
12/1/2009 13
Manual extraction
12/1/2009 14
Automatic extraction
12/1/2009 15
Results: relation between the two methods
Variable N Mean Std Dev Minimum Maximum
rt 300 95.86667 25.91488 26.00000 122.00000
pm 300 35.73667 14.01697 7.00000 82.00000
sm 300 271.88000 99.38627 57.00000 538.00000
pp 300 34.98667 18.08378 1.00000 80.00000
sp 300 275.35000 90.37577 72.00000 480.00000
 rt: response time

 pm: the number of pauses extracted manually
 sm: the number of syllables extracted manually
 pp: the number of pauses extracted by Praat
 sp: the number of syllables extracted by Praat
12/1/2009 16
(Continue)
 Correlation: the number of syllables
12/1/2009 17
(Continue)
 Correlation: the number of pauses
12/1/2009 18
Result: manual transcription
12/1/2009 19
(Continue)
12/1/2009 20
Result: automatic method
12/1/2009 21
(Continue)
12/1/2009 22
Discussion
 MSR
 Detecting pausing boundary might be sensitive to
speech samples
 Need to improve Praat algorithm
 Number of syllables and number of pauses might
be highly correlated
12/1/2009 23
(Continued)
 Speech rate
 Maybe less valid than MSR, but more cost-
effective
 Only using the information of syllables
 Number of syllables
 The script missed mostly unstressed syllables
 Filled pauses can be detected as syllables
 Problem in counting the number of syllables manually
12/1/2009 24
References
 Lennon, Paul (2000). The lexical element in spoken second language
fluency. In Heidi Riggenbach (Eds.), Perspectives on fluency. (pp. 25-42).
University of Michigan Press
 Kormos, J. and M. Denes, (2004). Exploring measures and perceptions of
fluency in the speech of second language learners. System 32, 145-164.
 Riggenbach, Heidi., (1991). Towards an understanding of fluency: a
microanalysis of nonnative speaker conversation. Discourse Processes 14,
423–441.
 Ginther, April, Slobodanka Dimova and Rui Yang (2009). Conceptual and
empirical relationship between temporal measures of fluency and Oral
English Proficiency and implication for automated scoring. Language
Testing.
 De Jong, Nivja and Ton Wempe (2009). Praat script to detect syllable nuclei
and measure speech rate automatically. Behavior research methods 41-2,
385-390.
12/1/2009 25

Temporal Measures of Fluency Automatic

Uploaded by

Copyright:

Available Formats

Temporal Measures of Fluency Automatic

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Temporal Measures of Fluency Automatic

Uploaded by

Copyright:

Available Formats

Temporal measures of fluency:

automatic and manual extraction

 Two Senses of Fluency (Lennon 2000)

 “The rapid, smooth, accurate, lucid, and efficient

 Fluency: „the speed and smoothness of oral

Oral Proficiency Smoothness

 Filled pause (uh, um, uhr, …)

 The frequency and length of pauses

 Which one is better?

 Result from de Jong and Wempe (2009)

Chinese 3 Chinese 4 Chinese 5 Hindi 5 Hindi 6 Native Total

 Chinese 3: the OEPT level 3 of Chinese examinees

 rt: response time

You might also like