David Orrego-Carmona
THE RECEPTION OF (NON)PROFESSIONAL SUBTITLING
DOCTORAL THESIS
Supervised by Professor Anthony Pym
Department of English and German Studies
Tarragona
2015
Professor Anthony Pym
URV. Av. Catalunya 35
43002 Tarragona, Spain
February 27, 2015
I hereby certify that the study The Reception of (Non)Professional Subtitling, presented by
David Orrego-Carmona for the award of the degree of Doctor, has been carried out under my
supervision at the Department of English and German Studies of the Universitat Rovira i
Virgili.
The research and the thesis fulfill all the conditions for the award of an INTERNATIONAL
DOCTORATE, in accordance with current Spanish legislation.
Anthony Pym
Intercultural Studies Group
Universitat Rovira i Virgili
Tarragona, Spain
President
European Society for
Translation Studies
Acknowledgements
It is incredible how many people was directly or indirectly involved in this project to
whom thanks I extend. Foremost, I am highly indebted to my Doktorvater, Anthony
Pym, not only for the constant support, guidance and the insightful revision of my
research all these years, but also for putting in place all the system that eventually
allowed me to start navigating the waters of academia. This system includes the
scholarship grant from the Intercultural Studies Group that has covered my tuition and
maintenance for the last four years. I will be forever grateful for the extraordinary
opportunity to be paid for studying.
Thanks to the staff at the Department of English and German Studies for creating
a welcoming and friendly environment. I shared an office with a great group of fellow
researchers at the Department: Carlos Teixeira, Esther Torres, Kasia Baran, Andrea
Bellot, Nune Ayvazyan and Esmaeil Haddadian. The place resembled a UN assembly
and it provided a sense of community and camaraderie. Alberto Fuertes, Alba Escriu
and Sara Miguel should be in this group, so thanks to the extended “office 2.20 et al.”
Esther took care of the cumbersome bureaucracy, and Carlos and Alberto helped me
revise this thesis and provided invaluable feedback.
I had the chance to learn from many translation scholars, both during my Masters
in Tarragona and at the CETRA Summer School in Leuven, Belgium. It was nice to put
faces on the names I had seen on the books, but it was even better to discover how
willing they were to share their comprehensive knowledge. A Summer School Grant
from the European Society for Translation Studies allowed me to attend CETRA, which
proved to be one of the most enriching opportunities during my PhD studies. I would
like to thank Dr. Sara Ramos Pinto for her invaluable support and advice throughout the
doctorate, starting from the moment I met her in CETRA, when she decided to share her
knowledge with a total stranger.
On more technical grounds, I would like to acknowledge the help of Marco Porta,
who kindly modified his software so I could use it; to Akshay Minocha, who saved me
from endless hours of painful manual data preparation, and to Oliver Valero Coppin, for
his advice regarding the statistical analysis. Having a space to set up the eye-tracking
lab was possible thanks to the people in charge of the Aula d’Anàlisi de la Parla at the
Universitat Rovira i Virgili.
i
The most enjoyable part of this PhD has been meeting wonderful people working
on Translation Studies or related areas. I wish to warmly thank Dr. Minako O’Hagan
and Dr. Sharon O’Brien for welcoming me at Dublin City University, Dr. Elisa Perego
and her colleagues for hosting me at the IUSLIT in Trieste, and Dr. Michael Carl and
Dr. Srinivas Bangalore for inviting me to participate of the Translation Data Analytics
project at Copenhagen Business School. The great people I met at these three places are
too numerous to mention by name, but it was a great craic to fare il caffè with you all!
Tak! Many aspects of this thesis also took shape through the discussions at various
conferences where I presented my work in progress. I am grateful for all the feedback,
but extremely happy to count some of the participants at those conferences among my
friends.
Extraordinarily, I did not only keep my friends from Colombia, but rather
multiplied the number of people I call friends, even outside academia. Some of them
have been involved in the writing of the thesis, but most of them have simply helped me
to stay as sane as possible. To my friends from the Universidad de Antioquia, especially
to the study group on audiovisual translation, with whom my passion for audiovisual
translation started. To Katherin Pérez and Matthias Apfelthaler, for reading and
criticizing pretty much every paper I have written. To David Peláez, for refusing to let
me hide in academia by occasionally popping up to take me to the professional
translation world. To those who helped me crowdsource proofreading the final version
of the thesis on a very short notice.
To all the participants who so willingly gave me their time to participate in this
project: without your disinterested acts, this research would have been impossible.
Finalmente, los mayores agradecimientos son para mi familia. Esta tesis en
realidad no empezó conmigo, sus raíces están en todo lo que ellos han hecho para que
yo puedo estar aquí. Este es el resultado de las oportunidades que me han brindado, la
seguridad que me inculcaron y la libertad que me han dado para pensar por mí mismo.
También, gracias a los que no pudieron ver esta tesis terminada: Javiera López, que me
llamó doctor desde antes de que yo siquiera pensara tomar este camino, Carlos Cruz y
mi papá.
ii
Abstract
People all over the world, especially young viewers, rely on non-professional subtitles
to access audiovisual content. This research investigates the reception of professional
and non-professional subtitles among university students with different levels of
proficiency in English. Three versions of subtitles were included: the professional DVD
subtitles distributed in Spain and two non-professional subtitle versions: one Spanish
and the other Latin-American.
An initial survey, which obtained 332 responses, was used to screen the
population and test their level of English. For the second stage, which consisted of an
eye-tracking experiment and an interview, 52 participants (26 with a low level of
English and 26 with a high level of English) had their eyes tracked while they were
watching three clips from a popular TV show. Each clip was shown with a different
type of professional or non-professional subtitling and participants answered a
comprehension questionnaire after each of them.
The findings show that the type of subtitles does not affect the distribution of
attention, but the professional subtitles were received with shorter mean fixation
durations. The participants’ performance with the professional and the Latin-American
non-professional versions were similar and gave better scores than with the Iberian nonprofessional version.
The eye-tracking measurements show that the level of proficiency in English
affects subtitle-reading behavior. The participants with low proficiency had a more
standardized behavior and exhibited a high degree of reliance on the subtitles. The
highly proficient participants’ behavior has more variation, with some participants
depending on the subtitles as much as the other group and other participants with almost
no attention to the subtitles. The level of proficiency in the source language and the
participants’ perceived effort to read the subtitles were identified to have an effect on
their enjoyment of the audiovisual content.
The results from this study show that, from the audience’s perspective, nonprofessional subtitles can provide results similar to professional subtitles. Additionally,
the study shows that the level of proficiency in the source language affects the subtitlereading behavior and that viewers might have some degree of individual control over
subtitle reading.
iii
Resumen
Actualmente, la subtitulación no profesional permite a personas en todo el mundo
acceder a material audiovisual en otros idiomas. Esta investigación estudia la recepción
de la subtitulación profesional y no profesional entre estudiantes universitarios con
diferentes niveles de inglés. El estudio incluye los subtítulos profesionales para DVD
distribuidos en España y dos versiones no profesionales, una española y una
latinoamericana.
Se usó una encuesta inicial para estudiar a la población y evaluar su nivel de
inglés. En esta etapa se recolectaron 332 respuestas. La segunda etapa combinó una
sesión de visionado con rastreo ocular y una entrevista. Los participantes, 26 con nivel
alto de inglés y 26 con nivel bajo de inglés, vieron tres segmentos de una serie de
televisión mientras se registraban los movimientos de sus ojos. Después de cada
segmento, los participantes respondieron una prueba de comprensión. Cada segmento
usaba uno de los tipos de subtítulo mencionados anteriormente.
De acuerdo con los resultados, el tipo de subtítulos no afecta la distribución de la
atención, sin embargo se registraron fijaciones promedio más cortas con los subtítulos
profesionales. En la prueba de comprensión, los participantes obtuvieron resultados más
altos con los subtítulos profesionales y la versión no profesional latinoamericana que
con la versión no profesional española.
Los participantes leyeron los subtítulos de forma diferente dependiendo de su
nivel de inglés. Aquellos con un nivel bajo leyeron casi todos los subtítulos y mostraron
un comportamiento similar. Por otro lado, aquellos con un nivel alto de inglés
mostraron mayor variación: algunos siguieron los subtítulos tanto como el otro grupo y
otros casi no los miraron. Tanto el nivel de inglés como la percepción del esfuerzo
necesario para leer los subtítulos afectan la percepción del material audiovisual.
Los resultados del estudio muestran que, desde la perspectiva de la audiencia, la
recepción de material audiovisual con subtítulos profesionales y no profesionales puede
ser comparable. Además, el nivel de conocimientos del idioma original del material
audiovisual afecta la forma como los espectadores leen los subtítulos. Finalmente, el
comportamiento de los participantes parece indicar que los espectadores tienen algún
grado de control sobre la lectura de los subtítulos.
v
Resum
Actualment, la subtitulació no professional permet a persones de tot el món accedir a
material audiovisual en altres idiomes. Aquesta recerca estudia la recepció de la
subtitulació professional i no professional entre estudiants universitaris amb diferents
nivells d’anglès. L’estudi inclou els subtítols professionals per a DVD distribuïts a
Espanya i dues versions no professionals, una espanyola i una llatinoamericana.
Es va fer servir una enquesta inicial per estudiar la població i avaluar el seu
nivell d’anglès. En aquesta etapa es van recol·lectar 332 respostes. La segona etapa va
combinar una sessió de visionat amb rastreig ocular i una entrevista. Els participants, 26
amb nivell alt d’anglès i 26 amb nivell baix d’anglès, van veure tres segments d’una
sèrie de televisió mentre es registraven els moviments dels ulls. Després de cada
segment, els participants van respondre una prova de comprensió. Cada segment feia
servir un dels tipus de subtítol esmentats anteriorment.
Segons els resultats, el tipus de subtítols no afecta la distribució de l’atenció,
però es van registrar fixacions mitjanes més curtes amb els subtítols professionals. En la
prova de comprensió, els participants van obtenir resultats més alts amb els subtítols
professionals i la versió no professional llatinoamericana que amb la versió no
professional espanyola.
Els participants van seguir els subtítols de forma diferent depenent del seu nivell
d’anglès. Aquells amb un nivell baix van llegir gairebé tots els subtítols i van mostrar un
comportament similar. D’altra banda, aquells amb un nivell alt d’anglès van mostrar
més variació: alguns van seguir els subtítols tant com l’altre grup i uns altres gairebé no
els van mirar. Tant el nivell d’anglès com la percepció de l’esforç necessari per llegir els
subtítols afecten la percepció del material audiovisual.
Els resultats de l’estudi mostren que, des de la perspectiva de l’audiència, la
recepció de material audiovisual amb subtítols professionals i no professionals pot ser
comparable. A més, el nivell de coneixements de l’idioma original del material
audiovisual afecta la forma com els espectadors llegeixen els subtítols. Finalment, el
comportament dels participants sembla indicar que els espectadors tenen algun grau de
control sobre la lectura dels subtítols.
vii
Declaration
I, David Orrego-Carmona, hereby declare that this thesis is entirely my own work,
carried out at the Universitat Rovira i Virgili for the degree of Doctor of Philosophy in
Translation and Intercultural Studies, and that it has not been submitted as an exercise
for a degree at any other university. Where other sources of information have been used,
they have been acknowledged. Some parts of this thesis have been published previously
in:
Orrego-Carmona, David. 2014. “Subtitling, video consumption and viewers: The impact
of the young audience”. Translation Spaces 3. 51-70.
Orrego-Carmona, David. 2014. “Where is the audience? Testing the audience reception
of non-professional subtitling”. In Translation Research Projects 5, Esther
Torres-Simon and David Orrego-Carmona (eds). Tarragona: Intercultural Studies
Group. 77-92.
Tarragona, March 2, 2015
David Orrego-Carmona
ix
Disclaimer
Although this dissertation analyzes non-professional subtitling communities and their
products, it does not promote, support, sanction, condone, foster or encourage any
illegal activities involving copyrighted material. The different types of activities carried
out over the Internet, such as file sharing, streaming, non-professional subtitling and all
others related to the unauthorized distribution, translation and consumption of
copyrighted products are referenced for research purposes only and should not be
construed as an encouragement to engage in said activities.
xi
Table of contents
Acknowledgements ........................................................................................................... i
Abstract ............................................................................................................................ iii
Resumen ........................................................................................................................... v
Resum ............................................................................................................................. vii
Declaration....................................................................................................................... ix
Disclaimer........................................................................................................................ xi
Table of contents ........................................................................................................... xiii
List of tables ................................................................................................................. xvii
List of figures .............................................................................................................. xxiii
List of abbreviations ..................................................................................................... xxv
Chapter 1. Introduction ................................................................................................. 1
1.1. Objectives .......................................................................................................... 5
1.2. Overview of the methodology ........................................................................... 6
1.3. Structure ............................................................................................................. 7
Chapter 2. Background .................................................................................................. 9
2.1. Empowered viewers ......................................................................................... 10
2.1.1.
The emergence of empowered users in the media context ..................... 10
2.1.1.
The impact of prosumers ........................................................................ 14
2.1.2.
The impact of US media ......................................................................... 17
2.2. Prosumers that become translators................................................................... 20
2.3. Defining the audience: prosumers and lurkers................................................. 21
Chapter 3. Literature review ....................................................................................... 25
3.1. Collaborative translation .................................................................................. 25
3.2. Studies on non-professional subtitling in Translation Studies ......................... 27
3.2.1.
Fansubbing as part of the fandom ........................................................... 28
3.2.2.
Initial descriptive studies ........................................................................ 29
3.2.3.
Non-professional and professional subtitles from a professional point of
view ......................................................................................................... 32
3.2.4.
Profiling the subtitlers ............................................................................. 37
3.2.5.
Studies on specific non-professional subtitling features ......................... 39
3.2.6.
Conceptualization of non-professional subtitling and translation .......... 43
3.3. Research on reception ...................................................................................... 46
3.3.1.
Exploring the viewers’ reception of translated audiovisual products ..... 46
3.3.2.
Empirical studies on audiovisual translation .......................................... 47
Chapter 4. Methods ...................................................................................................... 63
4.1.
Methodological framework.............................................................................. 63
xiii
4.1.1.
Aims of the study .................................................................................... 64
4.1.2.
Research questions and initial hypotheses .............................................. 64
4.1.3.
Operationalization ................................................................................... 65
4.1.4.
Sub-hypotheses ....................................................................................... 69
4.2. Instruments....................................................................................................... 70
4.2.1.
Mixed-methods approach........................................................................ 70
4.2.2.
Questionnaires......................................................................................... 73
4.2.3.
Interviews ................................................................................................ 76
4.2.4.
Eye-tracking data .................................................................................... 77
4.2.5.
Eye-tracking system ................................................................................ 77
4.2.6.
Eye-tracking data elicitation ................................................................... 88
4.2.7.
Types of eye-tracking data ...................................................................... 91
4.2.8.
Questionnaire and interview data............................................................ 98
4.3. Population and sampling .................................................................................. 99
4.3.1.
Participants ............................................................................................ 102
4.4. Materials ........................................................................................................ 105
4.4.1.
Audiovisual material ............................................................................. 106
4.4.2.
Subtitles................................................................................................. 107
4.5. Ethical considerations .................................................................................... 108
4.6. Procedure ....................................................................................................... 109
4.7. Pilot study ...................................................................................................... 111
4.7.1.
Participants ............................................................................................ 111
4.7.2.
Procedure .............................................................................................. 112
4.7.3.
Results ................................................................................................... 113
4.7.4.
Discussion ............................................................................................. 116
4.8. Lessons learned from the pilot study ............................................................. 117
4.8.1.
Revision of the instruments and the methods ....................................... 117
4.9. Statistical analysis framework ....................................................................... 119
Chapter 5. Quantitative results ................................................................................. 123
5.1. Pre-experiment questionnaire data ................................................................. 123
5.1.1.
Linguistic knowledge ............................................................................ 123
5.1.2.
Audiovisual consumption ..................................................................... 124
5.1.3.
Audiovisual translation modalities ....................................................... 125
5.1.4.
Use of the Internet ................................................................................. 126
5.2. Experiment data exploration .......................................................................... 126
5.2.1.
Percentage of fixations .......................................................................... 127
5.2.2.
Percentage of duration of fixations ....................................................... 128
5.2.3.
Mean fixation duration .......................................................................... 129
5.2.4.
Reception capacity ................................................................................ 130
5.2.5.
Self-reported comprehension ................................................................ 131
5.2.6.
Subtitle-reading effort ........................................................................... 132
5.2.7.
Audience enjoyment ............................................................................. 132
5.3. Quantitative analysis per variable .................................................................. 133
xiv
5.3.1.
Preparation of variables for statistical analysis ..................................... 135
5.3.2.
Attention allocation ............................................................................... 137
5.3.3.
Skipped subtitles ................................................................................... 148
5.3.4.
Mean fixation duration on the subtitle area .......................................... 152
5.3.5.
Mean fixation duration on the image area ............................................ 156
5.3.6.
Attention shift ratio ............................................................................... 158
5.3.7.
Subtitle-reading effort ........................................................................... 164
5.3.8.
Audience enjoyment ............................................................................. 168
5.3.9.
Self-reported comprehension ................................................................ 170
5.3.10. Reception capacity ................................................................................ 176
5.3.11. Iconic attention...................................................................................... 179
5.3.12. Narrative attention ................................................................................ 180
5.3.13. Verbal attention ..................................................................................... 182
5.3.14. Gender ................................................................................................... 184
5.4. Quantitative analysis summary ...................................................................... 184
Chapter 6. Qualitative data and discussion ............................................................. 187
6.1. Hypothesis testing .......................................................................................... 187
6.1.1.
Hypothesis 1: Type of subtitle .............................................................. 187
6.1.2.
Hypothesis 2: Level of L2/L3 ............................................................... 196
6.1.3.
Hypothesis 3: Type of non-professional subtitles ................................. 199
6.1.4.
Hypotheses based on qualitative data ................................................... 201
6.2. Findings from the qualitative data ................................................................. 203
6.2.1.
Level of L2/L3 ...................................................................................... 203
6.2.2.
Subtitling and L2/L3 ............................................................................. 208
6.2.3.
Effect of the Clip variable ..................................................................... 210
6.2.4.
Viewers’ familiarity with subtitles........................................................ 213
6.2.5.
Quality from the viewer’s perspective .................................................. 218
6.3. Audience control over the viewing experience .............................................. 222
6.3.1.
Foreign audiovisual content and distribution delays ............................ 223
6.3.2.
Making decisions about dubbing or subtitling ...................................... 224
6.4. Discussion summary ...................................................................................... 227
Chapter 7. Conclusions .............................................................................................. 229
7.1. Research questions answered ......................................................................... 230
7.2. Complementary findings ................................................................................ 234
7.3. Limitations of the research............................................................................. 235
7.4. Applicability .................................................................................................. 238
7.5. Implications and avenues for future research ................................................ 240
References.................................................................................................................... 243
Appendices .................................................................................................................. 259
Appendix 1.
Appendix 2.
Appendix 3.
Appendix 4.
Consent form ................................................................................... 259
Subtitles used for the eye-tracking session ..................................... 261
Pre-experiment questionnaire .......................................................... 270
Listening-comprehension test ......................................................... 272
xv
Appendix 5.
Appendix 6.
xvi
Experiment questionnaire ................................................................ 274
Eye-tracking data quality ................................................................ 278
List of tables
Table 1. Type of eye-tracking data and methods of exploration used (adapted from
Saldanha and O’Brien 2013) .............................................................................. 92
Table 2. Factorial table for the main experiment.......................................................... 101
Table 3. Participants included in the main experiment ................................................ 103
Table 4. Duration of clips and number of words in the original dialogue .................... 107
Table 5. Subtitle density per condition and clip ........................................................... 108
Table 6. Presentation sequence used ............................................................................ 111
Table 7. Sequence randomization for the pilot experiment .......................................... 112
Table 8. Means fixation (in milliseconds) by area of interest, group and subtitle
condition (PRO, NP1 and NP2) ........................................................................ 113
Table 9. Percentage of correct questions by type of question and condition ............... 116
Table 10. Self-reported comprehension and Subtitle-reading effort by Type of subtitle
.......................................................................................................................... 116
Table 11. Weekly consumption of audiovisual products in terms of time and means of
access (pre-experiment questionnaire) ............................................................. 124
Table 12. Weekly consumption of audiovisual products in terms of time and means of
access (main experiment) ................................................................................. 125
Table 13. Use of audiovisual translation modalities for the pre-experiment questionnaire
.......................................................................................................................... 125
Table 14. Use of audiovisual translation modalities for the main experiment ............. 125
Table 15. Use of the Internet per day for participants in the pre-experiment
questionnaire and the main experiment ............................................................ 126
Table 16. Mean percentage of fixations by area of interest, Level of L2/L3 and Type of
subtitle............................................................................................................... 127
Table 17. Mean percentage of duration of fixations by area of interest, Level of L2/L3
and Type of subtitle ........................................................................................... 128
Table 18. Mean fixation duration by area of interest, Level of L2/L3 and Type of subtitle
.......................................................................................................................... 130
Table 19. Percentage of correct answers for different types of attention and Reception
capacity ............................................................................................................. 131
Table 20. Self-reported comprehension per Clip, Level of L2/L3 and Type of subtitle 131
Table 21. Subtitle-reading effort per Clip, Level of L2/L3 and Type of subtitle........... 132
Table 22. Audience enjoyment per Clip, Level of L2/L3 and Type of subtitle .............. 132
Table 23. List of variables with codes and descriptions ............................................... 134
Table 24. Estimated marginal means of Total fixation duration on the subtitle area (ms)
with the effect of Use of subtitling ................................................................... 138
Table 25. Pairwise contrast of estimated marginal means of Total duration of fixations
on the subtitle area (ms) with the effect of Use of subtitling ........................... 139
Table 26. Estimated marginal means of Total fixation duration on the subtitle area (ms)
with the effect of Level of L2/L3....................................................................... 140
xvii
Table 27. Estimated marginal means of Total duration of fixations on the subtitle area
(ms) with the interaction effect between Level of L2/L3 and Use of subtitling. .....
.......................................................................................................................... 140
Table 28. Pairwise contrasts of estimated marginal means of Total duration of fixations
on the subtitle area (ms) with the interaction effect between Level of L2/L3 and
Use of subtitling ................................................................................................ 141
Table 29. Estimated marginal means of Percentage of fixations with the effect of Level
of L2/L3 ............................................................................................................. 142
Table 30. Estimated marginal means of Percentage of fixations on the subtitle area with
the effect of Clip ............................................................................................... 142
Table 31. Pairwise contrasts of estimated marginal means of Percentage fixations on the
subtitle area with the effect of Clip .................................................................. 143
Table 32. Estimated marginal means of Percentage of fixations on the subtitle area with
the effect of Use of subtitling............................................................................ 143
Table 33. Pairwise contrasts of estimated marginal means of Percentage of fixations on
the subtitle area with the effect of Use of subtitling ........................................ 143
Table 34. Estimated marginal means of Percentage of fixations on the subtitle area with
the interaction effect between Level of L2/L3 and Prior knowledge of the clips
.......................................................................................................................... 144
Table 35. Pairwise contrasts of estimated marginal means of Percentage of fixations on
the subtitle area with the interaction effect between Level of L2/L3 and Prior
knowledge of the clips ....................................................................................... 144
Table 36. Final model of Percentage of fixations on the subtitle area applied to
Percentage of fixations on the image area, with the same main effects and
interactions ........................................................................................................ 146
Table 37. Final model of Percentage of fixations on the subtitle area applied to
Percentage of duration of fixations on the subtitle area, with the same main
effects and interactions ..................................................................................... 147
Table 38. Estimated marginal means of Skipped subtitles with the effect of Level of
L2/L3 ................................................................................................................. 148
Table 39. Estimated marginal means of Skipped subtitles with the effect of order of
presentation ....................................................................................................... 149
Table 40. Pairwise contrasts of estimated marginal means of Skipped subtitles with the
effect of Order of presentation ......................................................................... 150
Table 41. Estimated marginal means of Skipped subtitles with the effect of Use of
subtitling ........................................................................................................... 151
Table 42. Pairwise contrasts of estimated marginal means of Skipped subtitles with the
effect of Use of subtitling ................................................................................. 151
Table 43. Estimated marginal means of Mean fixation duration on the subtitle area (ms)
with the effect of Type of subtitle ..................................................................... 152
Table 44. Pairwise contrasts of estimated marginal means of Mean fixation duration on
the subtitle area (ms) with the effect of Type of subtitle .................................. 153
Table 45. Estimated marginal means of Mean fixation duration on the subtitle area (ms)
with the effect of Use of subtitling ................................................................... 153
xviii
Table 46. Pairwise contrasts of estimated marginal means of Mean fixation duration on
the subtitle area (ms) with the effect of Use of subtitling ................................ 154
Table 47. Estimated marginal means of Mean fixation duration on the subtitle area (ms)
with the interaction effect between Clip and Type of subtitle........................... 155
Table 48. Pairwise contrasts of estimated marginal means of Mean fixation duration on
the subtitle area (ms) with the interaction effect between Clip and Type of
subtitle (Type of subtitle as reference factor) .................................................... 155
Table 49. Estimated marginal means of Mean fixation duration on the image area (ms)
with the effect of Type of subtitle ..................................................................... 157
Table 50. Pairwise contrasts of estimated marginal means of Mean fixation duration on
the image area (ms) with the effect of Type of subtitle .................................... 158
Table 51. Estimated marginal means of Mean fixation duration on the image area (ms)
with the effect of Level of L2/L3....................................................................... 158
Table 52. Estimated marginal means of Attention shift ratio with the effect of Level of
L2/L3 ................................................................................................................. 159
Table 53. Estimated marginal means of Attention shift ratio with the effect of Clip ... 159
Table 54. Pairwise contrasts of estimated marginal Attention shift ratio with the effect of
Clip ................................................................................................................... 159
Table 55. Estimated marginal means of Attention shift ratio with the effect of Use of
subtitling ........................................................................................................... 160
Table 56. Pairwise contrasts of estimated marginal Attention shift ratio with the effect of
Use of subtitling ................................................................................................ 161
Table 57. Estimated marginal means of Attention shift ratio with the effect of Type of
subtitle............................................................................................................... 161
Table 58. Pairwise contrasts of estimated marginal means of Attention shift ratio with
the effect of Type of subtitle ............................................................................. 162
Table 59. Estimated marginal means of Attention shift ratio with the interaction between
Clip and Type of subtitle (Type of subtitle as reference factor) ........................ 162
Table 60. Pairwise contrasts of estimated marginal means of Attention shift ratio with
the interaction effect between Clip and Type of subtitle (Clip as reference factor)
.......................................................................................................................... 163
Table 61. Final model with significant main effects and interactions of Subtitle-reading
effort .................................................................................................................. 164
Table 62. Estimated marginal means of Subtitle-reading effort with the effect of Prior
knowledge of the clips ....................................................................................... 164
Table 63. Estimated marginal means of Subtitle-reading effort with the effect of Use of
subtitling ........................................................................................................... 165
Table 64. Pairwise contrasts of estimated marginal means of Subtitle-reading effort with
the effect of Use of subtitling............................................................................ 165
Table 65. Estimated marginal means of Subtitle-reading effort with the interaction effect
between Clip and Type of subtitle ..................................................................... 166
Table 66. Pairwise contrasts of estimated marginal means of Subtitle-reading effort with
the interaction effect between Clip and Type of subtitle (Clip as reference factor)
.......................................................................................................................... 166
xix
Table 67. Final model with significant main effects on Audience enjoyment .............. 168
Table 68. Estimated marginal means of Audience enjoyment with the effect of Level of
L2/L3 ................................................................................................................. 168
Table 69. Estimated marginal means of Audience enjoyment with the effect of Clip.. 168
Table 70. Pairwise contrasts of estimated marginal means of Audience enjoyment with
the effect of Level of L2/L3 ............................................................................... 169
Table 71. Estimated marginal means of Audience enjoyment with the effect of Subtitlereading effort .................................................................................................... 169
Table 72. Pairwise contrasts of estimated marginal means of Audience enjoyment with
the effect of Subtitle-reading effort .................................................................. 169
Table 73. Final model with significant main effects and interactions of Self-reported
comprehension .................................................................................................. 171
Table 74. Estimated marginal means of Self-reported comprehension with the effect of
Prior knowledge of the clips ............................................................................. 171
Table 75. Estimated marginal means of Self-reported comprehension with the effect of
Use of subtitling ................................................................................................ 171
Table 76. Pairwise contrasts of estimated marginal means of Self-reported
comprehension with the effect of Use of subtitling .......................................... 172
Table 77. Estimated marginal means of Self-reported comprehension with the effect of
Audience enjoyment .......................................................................................... 172
Table 78. Pairwise contrasts of estimated marginal means of Self-reported
comprehension with the effect of Use of subtitling .......................................... 173
Table 79. Estimated marginal means of Self-reported comprehension with the effect of
Narrative attention............................................................................................ 174
Table 80. Pairwise contrasts of estimated marginal means of Self-reported
comprehension with the effect of Narrative attention ...................................... 174
Table 81. Estimated marginal means of Self-reported comprehension with the
interaction effect between Clip and Type of subtitle......................................... 175
Table 82. Pairwise contrasts of Self-reported comprehension with the interaction effect
between Clip and Type of subtitle (Clip as reference factor)............................ 175
Table 83. Final model with significant main effects of Reception capacity ................ 176
Table 84. Estimated marginal means of Reception capacity with the effect of Clip ... 177
Table 85. Pairwise contrasts of estimated marginal means of Reception capacity with
the effect of Clip ............................................................................................... 177
Table 86. Estimated marginal means of Reception capacity with the effect of Type of
subtitle............................................................................................................... 178
Table 87. Pairwise contrasts of Reception capacity with the effect of Type of subtitle 178
Table 88. Estimated marginal means of Reception capacity with the effect of Prior
knowledge of the clips ....................................................................................... 179
Table 89. Estimated marginal means of Iconic attention with the effect of Clip ......... 180
Table 90. Pairwise contrasts of estimated marginal means of Iconic attention with the
effect of Clip ..................................................................................................... 180
Table 91. Estimated marginal means of Narrative attention with the effect of Prior
knowledge of the clips ....................................................................................... 181
xx
Table 92. Estimated marginal means of Narrative attention with the effect of Subtitlereading effort .................................................................................................... 181
Table 93. Pairwise contrasts of estimated marginal means of Narrative attention with
the effect of Subtitle-reading effort .................................................................. 181
Table 94. Estimated marginal means of Verbal attention with the effect of Prior
knowledge of the clips ....................................................................................... 182
Table 95. Estimated marginal means of Verbal attention with the effect of Type of
subtitle............................................................................................................... 183
Table 96. Pairwise contrasts of Verbal attention with the effect of Type of subtitle ... 183
Table 97. Estimated marginal means of Verbal attention with the effect of Clip ........ 183
Table 98. Pairwise contrasts of Verbal attention with the effect of Clip ..................... 183
Table 99. Summary of dependent variables with main effects and interactions marked as
× ........................................................................................................................ 185
Table 100. Summary of the hypothesis testing for H1.................................................. 195
Table 101. Summary of the hypothesis testing for H2.................................................. 199
Table 102. Summary of the hypothesis testing for H3.................................................. 201
Table 103. Summary of the hypothesis testing for H14 and H15 ................................... 203
Table 104. Confirmation of sub-hypotheses for H1: Type of subtitle .......................... 230
Table 105. Confirmation of sub-hypotheses for H3: Type of non-professional subtitles
.......................................................................................................................... 232
Table 106. Confirmation of sub-hypotheses for H2: Level of L2/L3 ........................... 232
Table 107. Confirmation of Quantitative data sub-hypotheses: H14 and H15 ............... 234
xxi
List of figures
Figure 1. Layout of the recording area used for the experiment .................................... 78
Figure 2. Experiment setting .......................................................................................... 79
Figure 3. Sequence of presentation for the tests ............................................................. 80
Figure 4. Tobii Studio Track Status box......................................................................... 83
Figure 5. Fixation and saccades ...................................................................................... 89
Figure 6. Areas of Interest .............................................................................................. 90
Figure 7. Scanpath as displayed in ESE ......................................................................... 93
Figure 8. Heat map with colored areas representing the areas most fixated upon ......... 94
Figure 9. Close-up on ESE summary information for subtitle segment ......................... 95
Figure 10. Example of log file exported from Tobii Studio ........................................... 97
Figure 11. Percentage of fixations on areas of interest by Level of L2/L3 and Type of
subtitle............................................................................................................... 114
Figure 12. Percentage of duration of fixations on areas of interest by Level of L2/L3 and
Type of subtitle .................................................................................................. 115
Figure 13. Percentage of fixations by area of interest, Level of L2/L3 and Type of
subtitle............................................................................................................... 128
Figure 14. Percentage of duration of fixation by area of interest, Level of L2/L3 and
Type of subtitle .................................................................................................. 129
Figure 15. Mean fixation duration by area of interest, Level of L2/L3 and Type of
subtitle............................................................................................................... 130
Figure 16. Use of subtitling, original and transformed ................................................. 136
Figure 17. Subtitle-reading effort, original and transformed ....................................... 137
Figure 18. Use of subtitling per Level of L2/L3 ............................................................ 139
Figure 19. Estimated marginal means of Total duration of fixations on the subtitle area
(ms) with the interaction effect between Level of L2/L3 and Use of subtitling ......
.......................................................................................................................... 141
Figure 20. Estimated marginal means of Percentage of fixations on the subtitle area
with the effect of Clip ....................................................................................... 142
Figure 21. Estimated marginal means of Percentage of fixations with the effect of Use
of subtitling ....................................................................................................... 144
Figure 22. Estimated marginal means of Percentage of fixations on the subtitle area
with the interaction effect between Level of L2/L3 and Prior knowledge of the
clips ................................................................................................................... 145
Figure 23. Percentage of duration of fixations and Percentage of fixations on the
subtitle area ...................................................................................................... 146
Figure 24. Skipped subtitle (%) per Order of presentation and Level of L2/L3 ........... 149
Figure 25. Estimated marginal means of Skipped subtitles (%) with the effect of Order
of presentation .................................................................................................. 150
Figure 26. Estimated marginal means of Skipped subtitles (%) with the effect of Use of
subtitling ........................................................................................................... 151
xxiii
Figure 27. Estimated marginal means of Mean fixation duration on the subtitle area
with the effect of Type of subtitle ..................................................................... 153
Figure 28. Estimated marginal means of Mean fixation duration on the subtitle area
(ms) with the effect of Use of subtitling ........................................................... 154
Figure 29. Estimated marginal means of Mean fixation duration on the subtitle area
(ms) with the interaction effect between Clip and Type of subtitle .................. 156
Figure 30. Estimated marginal means of Mean fixation duration on the image area (ms)
with the effect of Type of subtitle ..................................................................... 157
Figure 31. Estimated marginal means of Attention shift ratio with the effect of Clip . 160
Figure 32. Estimated marginal means of Attention shift ratio with the effect of Use of
subtitling ........................................................................................................... 161
Figure 33. Estimated marginal means of Attention shift ratio with the effect of Type of
subtitle............................................................................................................... 162
Figure 34. Estimated marginal means of Attention shift ratio with the interaction
between Clip and Type of subtitle ..................................................................... 163
Figure 35. Estimated marginal means of Subtitle-reading effort with the effect of Use of
subtitling ........................................................................................................... 165
Figure 36. Estimated marginal means of Subtitle-reading effort with the interaction
effect between Clip and Type of subtitle........................................................... 167
Figure 37. Estimated marginal means of Audience enjoyment with the effect of Subtitlereading effort .................................................................................................... 170
Figure 38. Estimated marginal means of Self-reported comprehension with the effect of
Use of subtitling ................................................................................................ 172
Figure 39. Estimated marginal means of Self-reported comprehension with the effect of
Audience enjoyment .......................................................................................... 173
Figure 40. Estimated marginal means of Self-reported comprehension with the
interaction effect between Clip and Type of subtitle......................................... 175
Figure 41. Estimated marginal means of Reception capacity with the effect of Clip .. 177
Figure 42. Estimated marginal means of Reception capacity with the effect of Type of
subtitle............................................................................................................... 178
Figure 43. Example of attention shifts when subtitles overlap..................................... 194
Figure 44. Heat maps of the fixations of HLE and LLE participants in Order of
presentation 5 ................................................................................................... 206
Figure 45. Example of HLE participants’ fixations ..................................................... 208
Figure 46. Heat maps with fixations durations made by all participants in Order 1 .... 211
Figure 47. Percentage of duration of fixations per Clip and Level of L2/L3................ 212
Figure 48. Heat maps for participants who use subtitles never (left) and occasionally
(right) ................................................................................................................ 216
Figure 49. Layout of the subtitles on the screen ........................................................... 219
xxiv
List of abbreviations
AOI
Area of interest
HLE
High Level of English
LLE
Low Level of English
L2/L3
Second/Third language
ms
Milliseconds
NP1
Iberian non-professional subtitles
NP2
Latin-American non-professional subtitles
PRO
Professional subtitles
pro-am
Professional amateur
xxv
A mi mamá, mi abuela y mi tía,
por todo lo que me han dado
Chapter 1. Introduction
I am a translator. In fact, I am a professional translator. In 2009 I received my degree in
English-French-Spanish Translation from the Universidad de Antioquia in Colombia,
the only university offering an undergraduate program in translation in the country. The
conditions for training translators in Colombia are tough for everyone: trainers and
trainees alike. Given the education system and the geographical location of the country,
in most cases trainee translators start their studies at the university without any
knowledge of English or French at all. There is little exposure to foreign languages and
the possibilities of going abroad to learn them are scarce due to socioeconomic factors.
In just four years, students are expected to learn two new languages, plus how to
translate; their trainers, in turn, are expected to make that happen.
When I was at university, even trainers would say, “It is impossible to live just on
translating in Colombia”. At the time, this message painted a very dark perspective for
us as students. Even if the conditions are rough, you do not want to be told you are
never going to be able to build a career as a professional because there are no market
opportunities in your domain. Although things are changing for the better, that was our
introduction to the Colombian translation market only a couple of years ago.
The recognition of our profession at other levels of the society was even worse.
We used to joke about how unknown the role of translators was among people we met.
It was often the case that when we said we studied translation, people would assume we
were being trained as language teachers or, in the best of cases, they guessed we were
studying to become interpreters.
This poses some questions that, although no directly addressed in this research,
may indicate an underlying motivation: How can you define professionalism in a
context where the profession is hardly recognized? How can you identify yourself as a
professional when you are told you are not going to make a career on it?
In my case, part of my first self-identification as a translator stemmed from my
research work. From 2006 I participated in a study group on audiovisual translation
within the Research Group on Translation Studies at the Universidad de Antioquia. This
nicely complemented my training as a professional translator. In the translation
program, we were trained as scientific and technical translators – audiovisual translation
was not included in the curriculum.
1
The Reception of (Non)Professional Subtitling
The study group was made up of four people, all translation students, with a
shared interest in subtitling and enough free time to meet weekly to discuss
publications, criticize translations and translation choices, and occasionally subtitle
some films. In a sense, our mechanisms for grasping the essence of subtitling and
developing subtitling skills were similar to those used by non-professional translation
communities: we used the technology at our disposal; we developed our understanding
of subtitling according to the information available on the Internet and in the existing
literature; we tried to replicate what we considered good subtitling on television and
cinema; we relied on peer-revision as a way to progress; and we translated in teams,
always. Understandably, we related everything we learned to our previous knowledge
and our professional training, which not only provided us with guidelines for subtitling
but also made it easier for us to acquire a metalanguage in order to talk about it.
After a while, we paired up with a film club that was screening underground
films from all over the word. We translated Danish, Japanese, Chinese and Polish films
using the English non-professional subtitles as the pivot. That made our activity even
more similar to that of non-professional subtitlers: we were unpaid, we were using a
pivot language to translate underground films depicting underrepresented cultures from
the other sides of the world, and our subtitles were used by people like us. The viewers
were a self-selected niche audience with a special interest in a certain type of films.
During one of our meetings, one of my research fellows started commenting
about a very bad translation she had come across. She was talking about some subtitles
for the film 300 with serious mistakes, the type of translation mistakes that occur when
translators work under poor conditions: low-quality video and audio, and without a
script. She mentioned some examples and we became interested in what people would
think of such subtitles and how the subtitle quality could affect the impression people
had of the film.
In order to explain the relevance of this poor translation, I need to elaborate on
the media piracy market in Colombia. At least since the cassette era, a large share of the
Colombian media market is taken up by piracy. First, there were door-to-door cassette
salesman or hawkers that would provide people with pirated tapes with the latest hits. In
the 1990s, these hawkers later replaced their cassettes with CDs, which were easier to
reproduce and could be sold cheaply. Then, when movies started to be distributed in
DVDs, the sellers added this product to their portfolio. The sales of pirated films in CDs
and DVDs grew to become a large and very active market. DVD players were within
2
Chapter 1. Introduction
the purchasing power of more people, and DVDs were cheaper than cinema tickets.
Further, they allowed more people to enjoy the content: unlike cinema tickets, a DVD
copy could be enjoyed by a group of people at the same time, and viewers could
rewatch the film as many times as they wanted.
Most towns in Colombia have no cinema; only the mid-size or large cities, of
which there are just a handful, have cinemas and thus access to new releases. Even in
those cities, not many people from low-income families have the opportunity to go to
the cinema, in some cases out of habit, but in most cases, due to a lack of money. In my
town, for instance, the two cinemas that were once in operation closed down in the late
1980s and early 1990s. For me, it was not until 2002 when my father took me to the
cinema in Medellín that I had my first experience of the light beaming from the
projector at a cinema theater. We watched George Lucas’s Star Wars: Episode II Attack of the Clones, and I was fourteen.
In more than one way, the pirated film industry opened the doors for many
Colombians to access newly released audiovisual content for the first time and gave
them (us) the possibility to catch up with the privileged classes dwelling in the cities.
Soon, pirated DVD merchants were competing to acquire copies of the newest films and
compiling the largest collection of titles. Big Hollywood blockbusters were (and still
are) the most popular films, so when hawkers offered their clients the films that were
showing at theaters at that moment, their market was benefiting from the expectation
created by the official film advertising and merchandise.
That is how the copy with the poor translation of 300 came to be. Someone had
filmed a crooked cam-recorded version in a cinema, with very bad video quality and
poor sound – the kind of copy in which occasionally you can see the heads of the people
sitting in the front row when they move or stand up. The translator probably worked
without a script, directly from the rough audio track, causing some hilarious mistakes.
We decided to set up a small experiment to see how this would affect viewers. We
created an alternative version of the subtitles ourselves according to professional
standards and invited some friends to watch one or both versions and then fill out a
questionnaire.
As it will become evident, this initial exploration resembles the experiment I
describe in this thesis, although our design was rudimentary. We had fun doing it, even
though our main result was that it is difficult for people to recall specific subtitles when
they watch a whole film. We assumed people would remember details of the subtitles,
3
The Reception of (Non)Professional Subtitling
which was not the case. We were too focused on “good quality subtitles” back then and
too much given to follow industry guidelines, because that is your take when you are
supposed to be a professional translator.
Following the 300 experiment, we received some funding to carry out a research
project on the state of audiovisual translation in Colombia and I started working as an
in-house translator. I set reception studies aside and did not even consider fansubs as a
possible research topic. I wanted to study the translation of Colombian films into
English.
In one of my first Master’s courses in Tarragona, we had a seminar with
Michaela Wolf on Translation as Social Field. We were discussing the role of
translators in society and the sociology of translators in terms of Bourdieu’s habitus and
capitals and Latour’s actor network theory. At some point, I commented on fansubbing
activities to support my argument and Michaela asked if I was working on the topic of
non-professional subtitling. I was not, in fact, but her question and interest in the subject
made me wonder why I should not be.
After an initial documentation and some discussions with my supervisor, it
became evident that fansubbing represented a promising area of research. Little had
been done on the area and most of it followed very traditional approaches, while the
phenomenon as such was broad, had many outlets and included many different
manifestations.
The work done for my Master’s gave me the tools to become acquainted with the
topic. I decided to take a more holistic approach. Reading the scarce initial literature, I
realized that most of the work, with a few exceptions, did what I and my research
fellows at the Universidad de Antioquia had done: assess non-professional subtitles
according to professional standards. However, as some authors were already
commenting, non-professional subtitles belong to a niche market with particular
features. Using Chesterman’s concept of “functional quality” (Chesterman 2004), I set
out to explore the production and the reception conditions of the subtitles in order to see
if non-professional subtitles could still be considered low-quality translations when
assessed within a more open framework.
I adopted an ethnographic approach to detail the production conditions of a nonprofessional subtitling community that translated US television series and films into
Spanish. By chance, one of the administrators of the community was living in Barcelona
at the time and agreed to meet with me. I interviewed him and he offered to help in
4
Chapter 1. Introduction
contacting other members in the group. He was also instrumental in accessing the forum
records of the group. After describing the production conditions, I decided to move onto
the reception side.
1.1.
Objectives
Having learned that non-professional subtitling communities could be structured groups
with their own internal guidelines, training mechanisms and quality assurance
processes, I became more interested in exploring the phenomenon from another
perspective: the receivers of the translations.
The reach of non-professional subtitling communities and its international
expansion, as shown by my research as well as by other researchers, indicated that nonprofessional subtitles were serving a wide audience. In a sense, the use of subtitles
signaled a certain degree of acceptance among users: people were consciously
downloading the subtitles and the membership of the communities was always on the
rise.
Taking this into account, I embarked on the task of testing whether nonprofessional subtitles affected the reception of the audiovisual material in some way. It
was necessary to narrow down the topic, so I restricted my research to the reception of
those subtitles that try to replicate professional subtitles (professional-amateur
subtitles). The population of the study was also defined as viewers between 18 and 30
years of age, which is the most popular key demographic for US audiovisual products,
and I assumed that young viewers would be more familiar with non-professional
subtitling due to their exposure to technology. I was aiming to test the subtitles based on
the way they were received by the participants. Additionally, I wanted to learn about the
viewers’ opinions regarding non-professional subtitling, and to find out whether they
were able to tell professional from non-professional subtitles. Learning about the users’
motivations for consuming subtitles was equally interesting: most research pointed at
the lengthy release delays as being the most important reason for people to use nonprofessional subtitling, but I wanted to see if there was something else that would arise
from the viewer’s opinions. This could shed some light on the reception of nonprofessional subtitles; it should provide information on how viewers are currently
engaging with translated audiovisual products under the new media consumption
context, as part of an international rather than local audience.
5
The Reception of (Non)Professional Subtitling
During the design of the study, an additional element that gained relevance was
assessing the viewer’s reception of the subtitles based on their level of proficiency in
the original language of the audiovisual content. Initially, I entertained the possibility of
testing the reception of audiovisual material in a language that was totally foreign to the
participants. By so doing, I could have isolated any spoken verbal language interference
from the audio track, as well as assessed the participants’ comprehension on the
grounds that they would have been exclusively relying on the subtitles. Studies
involving eye-tracking methods commonly avoid using audiovisual content with audio
tracks in languages that are known to the participants.
However, I considered the current consumption patterns and aimed at a more
ecologically valid study. I was more interested in exploring the reception conditions in a
setting as comparable as possible to the real-life use of subtitles. In a world where in
most places the supply of audiovisual content is largely dominated by the United States,
and where English has become a dominant international language, people, especially
young people, are exposed to English on a regular basis. The language is not totally
foreign to them anymore. Including audiovisual material with an audio track in English
would allow for the assessment of consumption conditions that are comparable to
everyday life circumstances of viewers in Spain, where I decided to carry out my
research.
1.2.
Overview of the methodology
Starting with the aim of testing the differences in reception of different types of subtitles
according to different levels of proficiency in English, I decided to follow a
methodology that combined different data collection methods, in order to see how the
findings from them compare and contrast. Given that the research took place in
Tarragona, Spain, I included the professional subtitles distributed in the DVD version of
the popular TV series The Big Bang Theory. For the non-professional version, I used the
one produced by the Argentine community I had previously analyzed for my Master’s
research. This community produces subtitles in “neutral Spanish”, so the audience from
Spain could be considered part of their target. However, in order to have another point
of comparison to reflect the variation among non-professional subtitling communities, I
searched for an additional non-professional subtitle version, settling on one prepared by
an Iberian group producing subtitles in Iberian Spanish.
6
Chapter 1. Introduction
For the data collection, the study included a questionnaire, eye tracking and
interviews and was carried out in two complementary stages. The first stage consisted
of a questionnaire that screened the population in terms of audiovisual consumption
habits, linguistic knowledge and use of the Internet, and a language test to select the
participants for the second stage. In total, 332 valid responses to the questionnaire were
collected. For the second stage, which consisted of individual eye-tracking sessions and
interviews, 52 participants had their eyes tracked while watching three excerpts from
the TV series. Each clip was shown with a different type of professional or nonprofessional subtitles and the participants were asked to answer a content-recall and
satisfaction questionnaire after each of them. Right after the eye-tracking session, the
participants were interviewed following a semi-structured procedure. Each interview
was individual and lasted for between 15 and 20 minutes.
1.3.
Structure
Chapter 2 introduces the concept of prosumers and its implications for the media
industry as the background for the study of non-professional subtitling. Prosumerism
results from the democratization of technologies, which gives consumers the means to
exert some power over the products they consume. By analyzing how the relations
between producers and consumers are changing the conditions under which audiovisual
products are produced and consumed all around the world, I see non-professional
subtitling as part of a participatory culture resulting from the prosumers’ engagement.
My project finds its foundations in two areas of Translation Studies: studies on
non-professional subtitling and research related to the reception of translated
audiovisual content. The literature review, arranged according to these two areas, is
presented in Chapter 3. The review of works related to non-professional subtitling
purposefully ignores the studies related to other manifestations of non-professional
translation, such as crowdsourcing or the non-professional translation of nonaudiovisual texts. Along the same lines, in the reception section, this revision focuses
strictly on the studies that explore the reception of audiovisual material.
The methods for my research are presented in Chapter 4. The first section
provides an account of the methodological framework: it introduces the research
questions, the hypothesis, and the operationalization of the variables. The chapter goes
on to describe the selection and use of the data collection methods and comments on the
7
The Reception of (Non)Professional Subtitling
processing of the data. This chapter also offers information about the population under
study and the sample selection process, as well as a description of the versions of the
videos and subtitles included and the selection and preparation of the audiovisual
material. Subsequently, it describes the pilot study and the conclusions resulting from it,
followed by the revisions applied to the methodology. It concludes with an introduction
to the statistical methods applied to analyze the data.
Chapter 5 presents the quantitative results obtained from the pre-experiment
questionnaire and the eye-tracking sessions. First, the chapter describes the general
panorama presented by the data, using descriptive statistics. Then, relying on mixedeffect models, the chapter presents the analysis of the participants’ performances. A
final section sums up the results of the quantitative analyses.
The results of the data analysis are discussed in Chapter 6. The chapter first tests
the hypotheses presented in Chapter 4 and then highlights additional findings that
proved to be interesting aspects of the participants’ reception. Both quantitative and
qualitative data are offered to support the discussion. Lastly, the chapter offers an
account of the participants’ engagement with translated audiovisual content in their
everyday life based on the interview data.
The Conclusions summarize the main findings of the study and indicate the
possible contributions to Translation Studies. The final chapter also presents the study’s
limitations and suggests future avenues that could help move forward the research on
non-professional subtitling and the reception of translated audiovisual products.
8
Chapter 2. Background
This chapter gives a general overview of the current modes of interaction between
viewers, audiovisual content, technological developments and other participants
currently involved in the conception, production and consumption of audiovisual
content as well as the repercussions these interactions have nowadays in our society. By
describing some of the changes media have undergone in recent decades, this overview
positions non-professional subtitling as involving prosumers, users that participate
actively in the production of the content they consume. This overview is presented in
order to describe the context in which non-professional subtitling unfolds and the
position of the audience within that context.
Undoubtedly, technology has become intertwined with most, if not all, human
activities. As with writing and books before, the democratization of technology
prompted by the popularization of personal computers and the expansion of the Internet,
has changed people’s lives and fostered innovation (Hippel 2005). Technology has not
only had an impact on the working conditions of most people in the developed and the
developing world, but it has also modified the way people communicate and interact
with others, and how they spend their leisure time. As Pope Benedict XVI put it in his
message for the 47th World Communications Day, “[t]he digital environment is not a
parallel or purely virtual world but is part of the daily experience of many people,
especially the young” (2013:n.p.). People have learned to live in the virtual and physical
worlds at the same time, and for young people, their virtual activities are part of their
reality.
As a result, many people with technological tools enjoy spending their time on
the Internet and seem to relish the possibilities offered by global connectivity. Further,
through their interactions over the Internet, these people understand how they can
enhance their activities and promote shared enjoyment by joining their efforts and
operating as a group rather than as individuals. They start acting as structured
communities of like-minded people that share similar motivations and are able to
cooperate to promote shared goals. Some of these communities are made by consumers
that come together, guided by their shared interest in specific products. Their
interactions with other consumers change the way they engage with the products they
consume and how they think about those products. Capitalizing on this new behavior,
9
The Reception of (Non)Professional Subtitling
consumers may start to take on a more active role in the distribution of content. Some of
them, active consumers, move from the traditional, more passive role of a consumer to
become highly active and empowered users that are able to interact with the production
side of the chain.
2.1.
Empowered viewers
2.1.1. The emergence of empowered users in the media context
Whether we want to accept it or not, the current status of networks and interconnections
in the digital era places all of us in a network society (Castells 2000): we all participate
in it, voluntarily or involuntarily, and we are all agents in it, consciously or
unconsciously. Co-creational practices have evolved to such an extent that we take part
in them without even being aware of it and without making any additional effort.
One of the most widespread examples of collaborative participation where users
are not aware of their involvement is the use of reCAPTCHAs. By using distorted text
images, CAPTCHAs (Completely Automated Public Turing test to tell Computers and
Humans Apart) are used to tell real people from computers. They aim at preventing email spamming and fake blog and forum posts, as well as protecting websites from
automated software in general. Every regular Internet user is probably required to
perform this authentication process at least once a day. Being aware of this fact, the
reCAPTCHA project employs collective effort to digitalize books en masse:
reCAPTCHAs display images that belong to digitalized books produced using OCR
software (Ahn et al. 2008). About 200 million reCAPTCHAs are solved every day,
which accounts for an aggregate task that would require 150,000 hours of work per day;
and most of this is done by users who are not aware of their involvement.
Being involved in a collective activity with such a high impact without realizing
it sounds like the antithesis of empowerment. The real effect arises not only when users
are participating in large-scale activities, but when they consciously decide to
participate and concentrate their efforts on something that interests them. The
combination of work effort is what makes collaboration strong on the Internet. In the
participatory culture context, consumers groups “leverage the combined expertise of
their members” (Jenkins 2006:27). It is almost impossible for everyone to know
everything, but it is possible for everyone to know a bit and then combine it with others’
10
Chapter 2. Background
knowledge in order to have broader common knowledge. These groups of collective
knowledge are not strictly new, but they operate more effectively thanks to the new
technological tools for collaboration. This knowledge shared among networks is what is
known as cognitive surplus (Shirky 2010). The cognitive surplus that results from
participatory practices is what collectivities and companies exploit to carry out their
endeavors. The combined knowledge and skills of regular people from all around the
world make possible undertakings that were virtually unthinkable twenty years ago.
Before the existence of the Internet and affordable home technology,
communication over long distances was either expensive or slow (Surowiecki 2005);
economic, temporal and geographical factors affected the structures and functioning of
collectivities. Sharing a physical location or having a previous relationship of some sort
was mandatory if one was to know about somebody else’s opinions or standpoints.
Nowadays, the emergence of cheaper and more efficient technology for instant longdistance communication has empowered collectivities by overcoming those obstacles.
Technology provides the power to operate at a global scale and has thus amplified the
amount and impact of the work of such groups (Godin 2008:7). The ubiquity produced
by the Internet and the tools produced by information technology make possible easier
and more stable connections and enable constant communication among individuals
with similar interests, values and goals.
Deuze (2007) explains that consumers’ participation in the production process as
co-creators has increased significantly in the last ten years in fields as varied as
journalism, television and film production, advertising, design, and communication, to
name but a few. For instance, citizen journalists report news through Twitter and have
become a valuable source for TV news stations; amateur photographers have the chance
to offer royalty-free photos for sale through online photo stocks; the crowdsourcing
models of the fashion industry involve the consumers as part of the design process. The
engagement of media consumers in media production highlights the emergence of usergenerated content, not just as an addition to the industry-produced content but also as an
important and necessary part of the industry’s final product.
With these possibilities available to them, some consumers have become active
consumers – prosumers (Tapscott and Williams 2006) – since the changes have allowed
them to assume some of the powers and responsibilities that traditionally resided with
the producers. The mechanisms of binary production-consumption interaction are
11
The Reception of (Non)Professional Subtitling
thereby altered, creating a place for the combined role. This emerging prosumer role is
linked to both the traditional poles of the media industry.
Media systems are subject to constant reshaping under current conditions.
Technology-empowered consumers are conscious of their powers and are pro-active
regarding the products they are offered. It is thus inevitable that this has an impact on
the system as a whole. All consumers, those who continue with the traditional, passive
role as well as those who have adopted a more active participation, are directly affected.
That is, the actions of prosumers have repercussions on the products other consumers
buy or access, and most importantly, these actions are reflected in the way the products
are conceived, generated and distributed. This prosumer part of the audience has
become a more compact self-elected public that develops a profound affection for
certain products and has a deep interest in participating in or influencing the creation
and development of those products.
This context, in which users enact some powers that previously resided only with
producers, allows the emergence of what Jenkins (2006) calls convergence. As he
describes it, convergence involves all the changes that occur at the social, technological,
industrial and cultural levels of the media industry, prompted by the actions and
reactions of audiences. These new behaviors affect media flows and create an
environment where different types of media systems coexist. The nature of these
systems is highly flexible and fragmented, given that they depend on the constant
redefinition of the relationships between the participants involved in them.
As pointed out by Jenkins (2004), there is a convergence between media
companies and the consumers’ interests, as well as between their behaviors. Media
companies, especially those based in the United States, are aware of the impact of user
activities and “are learning how to accelerate the flow of media content across delivery
channels to expand revenue opportunities, broaden markets and reinforce viewer
commitments”, while at the same time consumers are enacting the control they have
received through media technologies and “are fighting for the right to participate more
fully in their culture, to control the flow of media in their lives and to talk back to mass
market content” (Jenkins 2004:37).
These two forces are in constant dispute. Media companies respond in
contradictory forms, ranging from encouragement to resistance, while consumers are
still unsure about how much power they can actually exert, and their actions are in
constant flux between legal and illegal activities. Nevertheless, Deuze (2007) argues
12
Chapter 2. Background
that the role of costumers as co-creators in the media industry is indeed finding
increasing acceptance in the culture industries. This does not necessarily mean that
power has shifted from one extreme to the other, since the audience’s newly acquired
capacity still depends on concessions by the companies, which can still commoditize the
audience’s tastes and actions. The audience can challenge the company’s decisions, but
the final say remains with the company. However, Jenkins suggests companies care
about what the audience think because offering what they want makes them more loyal
to their products and, in the end, this generates higher revenue (2006:64).
In general, companies have leaned towards accepting and even promoting userproducer collaboration. However, there is no standard rule and the situations are
determined on a case-by-case basis. It is worth noting that although the production of
user-generated content accounts for a significant amount of total content, Aris
comments that “it is estimated that about 1% of consumers contribute around 90% of all
user-generated content” (2011:269). I will return to this at the end of this chapter.
Participation also depends on the type of collaboration. The forms of viewer
involvement are various and very different in extent. For instance, a reduced number of
users leaves a full review of films in the Rotten Tomatoes website, but a very large
group contributes with content rating because it only takes one click. In the end, this
influences the wider audience (cf. Farrelly 2008). Thus the major changes the media
industry is undergoing might actually be caused by a minority of active consumers.
Furthermore, they might significantly influence the success of media content and its
adoption by the general public.
To some degree, these active users act as early adopters who test not only the
content but also what they can demand from the producers. In older models, when
content was directed at a mass audience, this might have been a problem. Nevertheless,
the current entertainment system is more targeted and fragmented, focused on niche
markets rather than large segments of society. To attend to these markets, Anderson
(2006) proposes that companies, instead of aiming at selling to a wide audience as soon
as the product is released, opt for an approach called less of more. In his long tail
model, Anderson suggests companies would be better off if they put more titles in
circulation and made them accessible to niche audiences. In the long run, the combined
market for low-sale products can generate more return than an initial blockbuster. It can
also reveal the diversified nature of demand by serving a large number of small niche
13
The Reception of (Non)Professional Subtitling
tastes. The success of such a model depends highly on the prosumers’ ability to
consciously decide what they want to consume and channel their resources to locate it.
2.1.1. The impact of prosumers
The impact of the audience’s involvement in company’s decisions, as has been noted,
can be identified at different levels. I will rely on several examples in the following
sections to describe how companies have capitalized on audience actions to develop
their products or to avoid potentially negative outcomes: the case of Game of Thrones
exemplifies a re-assessment of practices for HBO; Netflix used House of Cards to test
whether audiences were ready to take on new forms of content distribution; the case of
Alice of Arabia shows how companies protect themselves from taking on controversial
projects, and Amazon is asking its subscribers what content they want to watch before
the company starts producing it.
One of the strongest examples of newly acquired television consumption habits
influencing people’s viewing choices is that of Game of Thrones. The configuration of
global audiences has enabled the creation of alternative, unauthorized distribution
channels. Since audiences tend to operate as an international unit, they also expect or
want to have access to new content as soon as possible, ideally at the same time it is
first released in the United States. According to TorrentFreak, a BitTorrent blog and
measurement site, Game of Thrones has been the most pirated TV show every year
since 2012, peaking at 8 million estimated downloads in 2014, while the estimated TV
viewers in the United States for the same year, where it is available to existing HBO
television subscribers, is 7.2 million viewers.
There are two main reasons for people to access the material illegally. The first is
the delay in international airing. For example, Australia was at the top of the list at the
time with 10.1% of all illegal downloads per episode of Game of Thrones because the
TV show airs there one week after the original broadcasting in the United States, and
viewers did not want to wait for a whole week. Second, viewers are not offered a standalone online streaming option. Those viewers in the United States who are not
subscribed to HBO on top of their cable contract have no option to legally watch the
show online. This indicates how the company’s business model does not meet the new
needs of their audiences. HBO has confirmed they will try to combat piracy by reducing
the delay in international airings, a strategy that has been applied since companies
14
Chapter 2. Background
realized the impact of piracy. This has resulted in delay reduction from about six
months to just one week or even hours (as has been the case in Latin America, Italy and
Spain). However, this only partially remedies the problem. On April 7, 2015 a standalone version of HBO Go streaming service, HBO Now, went live. The launch of this
service coincided with the premiere of the fifth season of Game of Thrones on April 12.
Another highly important change has been alteration in the way series are
released. Serialized TV programs were originally designed to be consumed weekly,
following a model that goes back to the 19th-century weekly distribution of serialized
novels for publication in newspapers. Viewers would go back every week to the TV at a
set time and wait for the show to start. The Internet, on-demand TV and digital TV have
changed that. With the possibility to watch the episodes whenever they want, some
devoted fans have developed the habit of binge viewing (i.e. watching several episodes
or seasons of a serialized program all at once). Under these circumstances, companies
are trying to turn the odds in their favor. In January 2013, Netflix broadcast House of
Cards, the first drama series produced exclusively for an on-demand Internet streaming
media provider. The drama is also the first serialized program that has been designed for
binge viewing: all of the thirteen episodes that comprise the first season were made
available online at once, on the same day, in all territories where Netflix was available.
Additionally, for a month, free access to the first episode was offered to nonsubscribers. The strategy proved successful and Netflix has extended it to other new TV
series produced by them, and has also used it to bring back old TV series that were
cancelled but that are still popular among fans. Although Netflix does not release
information about its viewership numbers, its original shows create intense worldwide
expectation and are constantly reviewed and rated highly by critics, as well being
popular among regular viewers.
The power that audiences have achieved extends to the pre-production processes
as well. On March 17, 2013 ABC announced that Alice in Arabia was in pre-production
stage. The drama series was intended to depict the life of an Arabic-descendant girl
from the United States who is forced to move to Saudi Arabia and suddenly faces a
culture and a vision of the world completely different from what she knew. The heavy
criticism on social networks right after the announcement and the concerns voiced by
Islamic organizations about the possibly misleading portrayal of Muslims caused the
cable network to back away from the idea and cancel the TV show even before the pilot
15
The Reception of (Non)Professional Subtitling
was shot. By March 21, only four days after the announcements, the project was shut
down.
Yet another example of the direct power of the viewers is Amazon’s strategy to
select the new shows they will produce. On their stand-alone service Amazon Prime,
Amazon has put in place a polling system to let its viewers decide what TV series
should be produced. When participating in the survey, viewers watch a collection of
pilots produced by Amazon and then vote to support the ones they think should become
a full show. The highest-rated shows are picked up for a full season. The system proved
successful and Transparent and Mozart in the Jungle became some of the most popular
TV shows of 2014. Further, there is already expectation about the next batch of series,
which was selected in January 2015.
Amazon’s initiative was prompted mainly to counteract the growth of Netflix.
Until recently, Netflix was the clear frontrunner in video-on-demand services
worldwide. Now Amazon Prime is gaining terrain and soon HBO Go stand-alone will
enter the competition. The emergence of more video-on-demand services provided by
big companies proves that these players in the media production and distribution market
are recognizing that users want to acquire more control and, if companies do not react,
users will continue searching for workarounds on their own. These initiatives aim to
give more power to the consumers while preserving the producers’ rights and the
benefits of being the main source of audiovisual content.
These cases exemplify the convergence described by Jenkins (2004) and show
that companies have adopted a range of positive responses to the audience’s actions.
The consumers have managed to use technological tools to bring part of the production
and distribution process under their control and, in response to this shift, the production
side has seen some added value in these activities and has thereby implicitly recognized
the validity of the consumers’ actions. Companies have started to look for new
strategies that are more in line with the audiences’ new behaviors. Under the principle
of affective economics (Jenkins 2006), the industry invites the viewers to go inside the
brand, thus creating an emotional connection. As commented above, in this case there is
clearly a positive response from the company. However, we can hardly talk about a loss
of control: companies still have the means to reach a wide audience, through both legal
and illegal distribution. For the audience, on the other hand, this significantly alters the
traditional ways of accessing and engaging with media content.
16
Chapter 2. Background
Nevertheless, there are other cases where producers resist the actions of
consumers, in what Jenkins (2006) calls media concentration. In these cases, instead of
reinforcing each other, the two forces enter into a struggle, since producers assume that
convergence represents a risk for their status and, in the end, their possibilities of
controlling the flows in the more traditional fashion.
2.1.2. The impact of US media
Except for audiovisual content (i.e. features and short films, documentaries, and TV
series, among others) produced in the United States, media used to be mostly local, as in
the European approach, where most countries used to have their own national media
organizations and operate as independent profit centers (Polonska-Kimunguyi and
Kimunguyi 2011), or regional, as in the cases of Spanish-speaking countries in Central
and South America or TV production in the Arab world, mostly centered in Egypt.
Initially, the products were subject to adaptations before being distributed on an
international scale in different countries, and the rights for broadcasting and distribution
were structured on a country-by-country basis (Aris 2011). This has changed. Although
Hollywood has dominated the movie market since the first half of the last century, the
internationalization of US television is a more recent phenomenon, going back only a
couple of decades. At the same time, this has caused the emergence of international
audiences. Viewers outside the US have grown used to US content and do not entertain
the possibility of waiting longer than their peers in the US to access new content.
It could be argued that convergence and the rise of participatory culture in
general is more strictly tied to the US industry and audience. This is true for the official
channels of distribution and for more of the alternative distribution channels, but in the
latter there is also a conscious decision-making process for selecting content that
increases the variety of products. The US is indeed the largest exporter of television
series and films in the world and clearly dominates the international market with an
astonishing advantage over the other players, regardless of the fact that it is not the
biggest producer of audiovisual content. The US exports its cultural products which
project its branded image of nation. The discourse of cultural imperialism and the
Americanization of television suggest that these conditions benefit the representation
and reproduction of a reality created through the lens of the US industry, instead of
allowing for local constructions of self-identity transmitted through productions. In
17
The Reception of (Non)Professional Subtitling
short, US ideologies are channeled through audiovisual content and are imposed on
other regions as a vision of the world.
Supporting the argument of cultural imperialism through the illegal distribution
of video content is the fact that a big share of the audiovisual content distributed over
the Internet is produced in the United States. Underground films and audiovisual
productions from other countries also make it through the network created by
prosumers, however US products are always at the top of most downloaded TV series
and films. The actions of prosumers maintain the paradox observed initially with the fan
distribution of anime: the subversive distribution activities actually help in the
popularization of the products, in this case, mainstream products. As such, prosumers
help in the distribution of the US views of the world and its ideologies.
Nevertheless, the argument of cultural imperialism applies to a certain extent
only. Having decided to bet on translational products, US producers also take into
account the international audience when they are evaluating future projects (Moura
2011). The international popularity of shows influences “what kind of programming
gets produced for domestic consumption in the United States and other countries that
are active in the global market” (Bielby and Harrington 2008:54).
Further, although largely centered on the media industry in the United States, due
to the configuration of international audiences, the impact of active users and usergenerated content on the media industry can and should be analyzed from a global
perspective (Bielby and Harrington 2008). Assuming that audiovisual products are
immune to the reception conditions and are assimilated without filters by the receivers
presupposes an extreme passivity on their part and overlooks any type of engagement.
As postulated by those who oppose the view of cultural imperialism, even though the
contents are produced in external realities “audiences play an active role in selecting the
media they consume and the textual readings they make of it; those readings produce
meanings that connect with the viewers’ own social experience” (Bielby and Harrington
2008:41). For instance, when researching about soap operas around the world, Matelski
(1999) found that cultural products are given meaning by the audience through their
own local frame. Looking at the cultural effect of the incursion of live broadcasting of
translational television in Belize, Wilk (2002) recognized that the internationalization of
Belizean television did in fact took away some power from the national television and
transferred it to the global system, in line with cultural imperialism. However, instead of
affecting the creation of a national identity, this had a different effect:
18
Chapter 2. Background
Fashion and products from outside the country has lost some of their magic –
they are no longer gifts from the future that carry a message of inevitability.
Local products are seeing something of a resurgence because they are no longer
pale imitations of the real thing, mimics stuck “behind the times”. (Wilk
2002:182)
These are only examples of situations in which cultural imperialism does not
produce the expected straightforward outcome signaled by some scholars. As pointed
out by Bielby and Harrington (2008), more research is still needed to support these
arguments. The forces involved in the production, distribution and consumption of
audiovisual content are constantly shifting and the effects are hardly generalizable. In
this protean media consumption environment, it is difficult to predict the effects that the
users’ or producers’ actions may have. There is always the possibility of unfavorable
repercussions for the receiving culture, but that is not the only possible case scenario.
Television producers have helped a great deal in the creation of the global
audiences, and media in general have reinforced the feeling of an international
consumption scenario. Producers and distributors are interested in keeping in contact
with the audience and have embraced social media (Twitter profiles, Facebook pages,
websites dedicated to shows and the fans) as a way to promote their products. Engaging
the audiences through multi-platform strategies has become a useful strategy not only to
create loyal viewers, but also to attract new ones. Along the same lines, local
newspapers and magazines in different countries now report the news of the TV series
and films even before the productions reach their own countries or local markets. In
doing so, they recognize the importance the audiovisual products have in their readers’
lives. Furthermore, reporting news on a given series or a film according to the original
release schedule implicitly acknowledges that viewers have found workarounds to
overcome the barriers and access the content before the official programming.
The Internet and the technological tools make it possible for the international
audiences to stay connected at a global level and to share content. This connectivity
solves the problems imposed by different geographical locations; it creates alternative
distribution channels, parallel to the official ones. Nevertheless, the language problem
still remains: for people in many countries to access the content, it is necessary to have
it translated. To overcome this problem, communities have formed on the Internet to
provide translations of the audiovisual content.
19
The Reception of (Non)Professional Subtitling
2.2.
Prosumers that become translators
The creation of alternative channels for distributing content confronts two main
problems: geography and languages. Geographical problems have been solved mostly
thanks to the technological developments that enable interconnectivity and have helped
in the configuration of international audiences, as detailed above. Solving the language
problem is more complicated, however, since the process of making translations is timeconsuming and requires specific skills. Users have nevertheless learned to cope with
this problem and cooperate in the production of translations that harness the distribution
of content and broaden the audience for the products. By organizing themselves and
exploiting the technological tools at their disposal, tech-savvy prosumers have turned
into non-professional translators, willing to invest their time and effort in creating
subtitles for other users in their countries or who speak their language.
The emergence of non-professional collaborative translation can be traced back to
the 1980s with the start of fansubbing, before the advent of affordable and widely
available technology. Fansubbing first appeared as an audience-based critique of the
management and distribution patterns put in place by industry at the time. It developed
as a revolutionary practice by fans in the United States who wanted to have access to
more and uncensored anime (Japanese animated productions) in their own language
(Leonard 2005). These fans were bothered by the cleansed results of mainstream
translation and the small amount of content imported into the U.S.
The popularization of technological tools and easy access to the Internet made it
possible for fansubbing to later spread to different audiovisual genres and other types of
audiovisual content (e.g. comedies, dramas, action, thrillers, political activism) as well
as other formats (feature films, documentaries, short films, TV series, interviews, etc.).
The non-professional subtitling phenomenon grew thanks to the fans’ realization that
they can pool their efforts and assume not only the distribution but also the translation
tasks.
When, at the beginning of the 2000s, international audiences started to become
interested in TV series and films produced in the United States, new communities
emerged that were dedicated to producing subtitles for these new products. They
developed organization and production structures that resembled those of fansubbing
communities. Different types of groups with different goals came together all around
the world to create self-managed non-professional translation projects. The groups have
20
Chapter 2. Background
formed around different factors, such as the languages they speak, the products they like
or their geographical location. The distribution and positive reception of US-produced
TV series caused these groups to multiply across countries, regions, and languages. In
Latin America and Spain, the phenomenon prompted the development of nonprofessional subtitling groups who create translations from English into Spanish.
2.3.
Defining the audience: prosumers and lurkers
Non-professional subtitles produced on the Internet are the result of collaborative
activities performed by prosumers organized in online communities. As part of the
Internet culture, the non-professional subtitle scene is configured by fragmented
subtitling groups. These communities bring together a number of collaborators and
create systems of interaction. They are made of active consumers who are willing to
devote their time and efforts to the tasks they set as a group.
The observation of online communities has shown that, regardless of the type of
community and their shared goals, large-scale groups that rely on the production of
user-generated content are actually sustained by the major efforts of a subgroup of its
members. The degree to which the communities are regulated and the interaction among
participants varies depending on their size and their evolution over time. They start, as a
rule, as horizontally structured groups that allow free communication, but the
refinement of the systems and the unbalanced growth of power force the development
of hierarchically more complex systems (Pym et al. Forthcoming). The roles of
participants in online communities are thus unbalanced. Their power within the group
stems from the participants’ abilities and the knowledge and products they bring to the
community, and since some participants exert more power than others, different
categories emerge.
Regularly, most members of these communities do not contribute in the contentproduction process: the largest part of the content is produced by a reduced portion of
the members. Only highly-committed users participate actively in co-creational
practices, which essentially means that these people work mostly as a leading subset
that paves the way and makes it easier for others. Media scholars refer to this as the
“90-9-1 principle” or the “1% rule”. This states that 90% of the members participating
in the discussion fora of a given community do not contribute but are rather observers
and passive users of the content, they are known as lurkers; 9% of all members are
21
The Reception of (Non)Professional Subtitling
intermittent contributors, they collaborate from time to time and their participation
accounts for about 10% of postings; and finally 1% of the members are considered
heavy contributors, super-users whose participation produces approximately 90% of
postings in the forum (Nielsen 2006; Ochoa and Duval 2008).
As a type of online community, non-professional subtitling groups can be
understood in terms of this structure. Their activity coincides with this three-layered
structure (Lindgren 2013). A general radiography (Orrego-Carmona 2011; Lindgren
2013) indicates that there is a group of occasional collaborators who are not highly
engaged with the activities, a large group of lurkers who benefit from the translations,
and a more structured and hierarchically organized main subgroup that operates as the
staff, made of intermittent and heavy contributors who are highly committed to the
functioning of the community. The framework of their activities is well defined and the
objectives tend to be clear to the participants involved in the project and to general
users. As a group, they act under a strict social code to decide on the activities they will
carry out. Participation is on voluntary bases only: people are not rewarded monetarily
for their time and effort. On the contrary, it seems that people participate in these
communities mostly for self-fulfillment reasons, such as learning or improving
linguistic skills in a foreign language, helping other people to have access to the content
or simply achieving and maintaining a status within the group.
I am bringing this into the discussion because it gives a sense of how these
communities operate. User-managed communities producing subtitles for films and TV
series are set up, controlled and maintained by users in accordance with internally
developed hierarchical structures, and their products are made available for the general
public, lurkers. While it is true that lurkers do not participate directly in the creation of
content, their passive presence in the group affects the actions of active users: they visit
the community’s website and benefit from the products, and in so doing, they create a
need for these products and encourage the existence of the more active users. Without
an increase in the number of participants, the groups would hardly cater for a pool of
prospective collaborators large enough to finally take on the role of prosumers. Large
communities are needed in order to filter the active consumers and ascertain the
continuity of the group.
In this project, I address the reception of subtitles by an audience that can be
identified with the lurkers of the subtitles produced by prosumers. I thus shift the focus
from the producers to the consumers of subtitles, but I am not focusing on the
22
Chapter 2. Background
prosumers themselves. The existence of a larger mass of lurkers, however, gives
importance to the actions of those active participants, it gives them a reason to continue
producing content and augments the impact they have on the established systems.
23
Chapter 3. Literature review
The previous chapter was dedicated to place non-professional subtitling as part of the
larger prosumer manifestations in media workflows and the evolution of the media
industry. In this chapter, I position the topic of this dissertation specifically within the
field of Translation Studies. In Section 3.1 I briefly discuss the consequences of
collaborative translation activities for our understanding of translation as a global
activity. The other two sections review the research developed in two wide areas in
which this thesis subscribed. In Section 3.2 I present the current state of the research
concerning non-professional subtitling. To conclude, in Section 3.3, I offer a summary
of the investigations that have dealt with the reception of translated audiovisual
products.
3.1.
Collaborative translation
The development of non-professional subtitling has brought a wide list of issues into a
shared discussion. The impact of collaborative practices on media-related environments
has been studied comprehensively in Media Studies, as discussed in Chapter 2.
Nevertheless, according to Pérez-González (2013:158), the studies analyzing the impact
of technology on the global flow of media content have failed to account for the
influence of translation in the bigger historical picture. As is often the case, translation
plays a backstage role not only in the production processes but also in discussions about
the factors that are affecting these processes and the way people engage with the
resulting products.
Scholarly attention within Translation Studies is increasing and several studies on
non-professional subtitling have been published over the past decade. In these
publications, scholars have considered issues related to quality, politics, translation
guidelines, and social implications (see Section 3.2). Interest has grown among
researchers due to several reasons: on the one hand, scholars and professional subtitlers
are concerned about the expansion of non-professional subtitling practices, and on the
other, successful crowdsourcing initiatives have shown the possible impact of these
models on the translation market. This reinforces the influence of user-generated
content on translation flows and on the way translation is conceived as an object of
study.
25
The Reception of (Non)Professional Subtitling
According to Cronin, when the audience of the translation is the same group that
is producing the translation, a consumer-oriented model of internality applies: “It is no
longer a question of the translator, for example, projecting a target-oriented model of
translation on to an audience, but of the audience producing its own self-representation
as a target audience” (Cronin 2012:100). This new framework cannot be totally
explained by most of the previous translation theories. The debate about the orientation
of a translation, either towards the target or towards the source, does not apply when
there is no external agent producing a translation to be consumed by unknown users. In
that sense, when the one individual (or group of individuals) embodies different roles,
our models need to be revised. In this case, it is the target, the end-user of the
translation, who makes the decisions that will affect and tailor the translation. This
imposes further changes to modes of translation that we have been trying to map for the
past fifty years in Translation Studies. Further, it challenges previously defined concepts
by adding a new social layer of actions and interpretations. Already in 2005, Tymoczko
was talking about the consequences of the internationalization of translation, essentially
the challenge to Western assumptions about the nature of translation and the need to
reconsider those notions. Further, she was already signaling the important influence of
technology:
The world has begun another immense transition of this type that is changing
translation, the transition associated with current developments in information
technologies and the media, ranging from new mass media to the Internet, from
CAT systems to translation imperatives associated with globalization. A major
branch of translation research and theorizing in the next decades will respond to
the changes in the conceptualization and practice of translation associated with
this transition, and research in translation and reevaluations of the nature of
translation both as process and product in the coming decades will inevitably
focus on these changes. Research has already begun on these topics, but can only
accelerate. (Tymoczko 2005:1088)
The areas of globalization and Computer Assisted Translation have been moving
forward and the subsequent theoretical discussions have progressed, as pointed out by
Tymoczko. However, the impact of non-professional translation, which in 2005 was not
as developed as it is now, still calls for a more systematic attention with a special focus
26
Chapter 3. Literature review
on its implications to Translation Studies. The future looks promising, with more solid
and sound academic reflections that challenge previously defined orthodoxies already
making their way into Translation Studies circles. Nevertheless, user-generated
translation in general is a heterogeneous and protean phenomenon, so what applies in
one instance may not cover another. This creates confusion and sometimes even
contradictory situations. Time is needed before a comprehensive account of the whole
non-professional translation panorama can actually be achieved, and perhaps more
importantly, scholars should have an open mind, identifying the pitfalls of the practices
while at the same time recognizing the value they may bring.
3.2.
Studies on non-professional subtitling in Translation Studies
The last ten years have been especially interesting for non-professional subtitling. On
the one hand, non-professional subtitling communities have mushroomed and
diversified over the Internet. Amateur subtitling has found different outlets and
committed users have helped in the emergence and consolidation of the communities.
At the same time, the emergence of studies on the topic has helped in the recognition of
non-professional subtitling and amateur translation as a valid field of research in
Translation Studies (Fernández Costales 2013).
The progress in the area has helped to narrow down the topics and has produced
studies that are more oriented towards specific aspects. Thanks to the abundance of
material and the easy access to it, case studies have been the most popular way to
approach the subject. The sections that follow will introduce previous studies mostly
following a chronological order.
Apart from considering the findings of these studies, we should assess the
conditions under which they have been developed. The vision of non-professional
subtitling as a disruptive practice traditionally prompted new researchers to embark on
quality-assurance quests to uncover the mistakes in the translations or to be too cautious
about the implications of non-professional practices, instead of considering the
sociological or educational dimensions, among other possibilities. Although this view
has changed in academia, in professional circles the discussion mostly turns on
professional encroachment, ethics and legal questions. This is due to the fact that
professional subtitlers and companies are now very concerned about proprietary
regulations. Another relevant reason for this is the increasing relevance that the illegal
27
The Reception of (Non)Professional Subtitling
distribution of content over the Internet is gaining in the political agendas of various
countries and international organizations.
3.2.1. Fansubbing as part of the fandom
Fansubbing, as the early manifestation of non-professional subtitling, was first
approached in studies anchored in Asian Studies. Even today, fansubbing is still heavily
associated with the non-professional subtitling of anime, as commented above. Scholars
described fansubbing as a constituent of fandom. Some examples of this initial approach
might be the article by the Asian Cinema professor and translator Nornes (1999), who
labels fansubbing as abusive subtitling (in relation to the concept of abusive translation
proposed by Philip E. Lewis), since it breaks the translation norms that had been
generally agreed upon. Japanese Culture professor Napier (2001) discusses the main
topics related to anime, taking a literary-critical point of view. Her book mainly deals
with the general situation of fandom, but it also points out the level of engagement
achieved by fan translators. Cubbison (2005), who is also interested in Fan Studies,
explains the situation within a quest framework, where fans are struggling to have some
say in the market decisions and the market has not yet defined if it wants to follow the
fans’ lead or, on the contrary, is trying to modify their preferences. Leonard (2005), an
engineer and anime fan, presents a detailed review of the progress of fansubbing and the
industrialization of anime.
Nornes is the only one who directly relates his discourse to Translation Studies
and focuses specifically on the concept of subtitling in the fandom in a deeper analytical
way. He argues that fansubs manifest what he defines as abusive subtitling, alterations
of the mainstream corruptive subtitling techniques that have implications for the inertia
of its accepted conventions and follow a more foreignizing approach. He states:
There is a potential and emerging subtitling practice that accounts for the
unavoidable limits in time and space of the subtitle, a practice that does not feign
completeness, that does not hide its presence through restrictive rules. We must
reconsider our own historical moment and work toward a subtitling that engages
today’s sensibilities with a violence which is not corrupt, but abusive. (1999:28)
Nornes describes abusive subtitling as being the regular practice in nonprofessional subtitling, but also in some uncommon cases of professional subtitling. The
28
Chapter 3. Literature review
subtitlers’ lack of formal training and their foreignizing approach motivate them to carry
out the subtitling by instinct and give them the chance to use new strategies and to
freely explore different modes of subtitling when trying to solve the problems they find.
3.2.2. Initial descriptive studies
At the same time as the above papers were published, studies on non-professional
subtitling, or fansubbing, started to appear in Translation Studies journals and at
conferences. Scholars mostly in the field of Audiovisual Translation published papers
dealing with this topic from the mid-2000s. The focus of their initial take on nonprofessional subtitling was mainly the general description of the fansubbing activity and
the structure of the fansubbing process, with the exception of Kayahara, who looks at
the impact of DVD technologies and fansubbing.
Kayahara (2005) expands the reach of non-professional subtitling by pointing out
that fansubbing was not restricted to Japanese-produced material only. His article
explains that there are three main reasons that prompt the desire to produce fansubs:
[A] desire to make minor films (that go unnoticed by the major distribution
companies) more widely available to non-Japanese speakers; to have minor
films noticed, and hopefully redistributed, by the major companies; and to make
available a subtitled version where only a dubbed version exists. (2005:69)
Kayahara’s article is especially important because, after presenting these reasons,
he goes on to explain that the ease of copying DVDs might have a major impact on the
fansubbing activity. He points out how technological empowerment could lead to the
popularization of fansubbing for audiovisual material in a more general and broader
sense. Kayahara argues that thanks to DVD storage capabilities, it is possible that “other
genres will pick up on fansubbing, thus providing a more diverse field of source
material for audiovisual translation theorists to work with, and raising the visibility of
subtitling as a practice” (2005: 69).
Ferrer Simó (2005) focuses on user-generated translations of Japanese material:
fansubbing and scanlation. With reference to fansubbing, she points out that fans prefer
a product they can easily recognize as a translation, that is to say, a product reflecting
the Japanese culture embedded in the source material. The article goes into a detailed
analysis of both the technical and the linguistic characteristics and problems that are
29
The Reception of (Non)Professional Subtitling
commonly presented in the subtitles produced by fans: number of characters per lines,
lines per subtitle, synchronization problems, translation errors, spelling mistakes,
location of the subtitle on the screen, etc. The characteristics of the fansubbed products
are contrasted with the professional stand: most of them are considered transgressive
and special emphasis is placed on translation errors and calques. Fansubbing is seen as a
source-oriented type of translation that also includes glosses, is closer to the original,
tends to be wordier and does not subscribe to mainstream standards. Due to these issues,
the fansubbed versions as such are not given much credit, although they are given a
cultural value and are considered a trigger for the decision to translate more anime.
Ferrer Simó (2005) also comments on the implications fansubbing could have on
professional versions and the translation market. Since the fansubbed version normally
predates any professional version, sometimes the audience is already inclined towards
certain aspects in the translation (such as keeping names in the original language rather
than adapting them). Professional translators are then required to align with these
decisions in order to go in line with the audience habits.
Also taking a production-focused approach, Díaz Cintas and Muñoz Sánchez
(2006) describe how human resources are allocated in the fansubbing production
workflow and what the requirements to produce the subtitles are. They also describe the
production process, detail technical requirements, and the human resources involved in
the activity. The concordance between the division of tasks among fansubbing groups
concords with the professional process of subtitling. The last section of the paper
includes examples of translation errors. The authors state that “[g]iven the amateur
nature of this translational practice and the languages involved, particularly Japanese,
mistakes tend to be fairly common” (2006:47), and they present some examples of
issues that commonly arise in fansubbing. They state that “[o]ne of the most interesting
facts about fansubs is that translators know that they are addressing a rather special
audience made up of people very interested in the world of anime and, by extension, in
Japanese culture” (2006: 46). This encourages Japanese fansubbers to transgress the
rules of formal subtitling: they argue that producing a more source-oriented version
would help the viewers to approach to the source culture.
Díaz Cintas and Muñoz Sánchez (2006) comment on the different formats used
for fansubbing and how the glosses used to add information or fill cultural gaps are
rather challenging for readers, since a high amount of content is presented on the screen
at the same time. Díaz Cintas and Muñoz Sánchez (2006) also comment on the
30
Chapter 3. Literature review
possibility for mainstream subtitling and fansubbing to coexist in the same space.
Fansubbing is seen as targeted at a specific audience that is more interested in the
source culture and is willing to put up with having more information displayed on the
screen at the same time. Considering this, the authors suggest fansubbing could become
an alternative subtitle version that can be included in official DVD releases. This option
was actually taken up by some distributors. For instance, in a study that will be
reviewed later in this chapter, Caffrey (2009) explores the perception of DVD anime
that includes optional glosses to explain culture references.
Díaz Cintas and Muñoz Sánchez (2006) dedicate a section to legal aspects. They
mention some companies do not see any benefit in taking actions against fansubbing
because they consider “free distribution of fansubs can have a very positive impact in
the promotion of a given anime series in other countries” (2006:44). They also comment
on the different types of copyright law and the difficulty to launch any international
prosecution against the groups. Additionally, they argue it is possible to see the
fansubbing activity as another instance of the fair use permitted by the copyright law, as
recently commented by Casarini (2014). They conclude by reinforcing the more
orthodox view on fansubbing: “in the end, regardless of ethics, or motive, fansubs are
technically illegal” (Díaz Cintas and Muñoz Sánchez 2006:45).
Pérez-González (2006) explores various features of the fansubbing process and
analyzes them in contrast to the regular rules followed in the mainstream subtitling
practice in commercial industry. He criticizes the partial approach that previous studies
(Kayahara 2005; Ferrer Simó 2005) have taken when appraising fansubbing practices as
having an impact on translation only and argues that the potential propagation and
implication of fansubbing have been considered “exclusively by recent developments in
the audiovisual translation industry, rather than the media industry as a whole” (PérezGonzález 2006:277), leaving aside the repercussions that Web 2.0 has had on fan and
viewer interaction with the production side of the industry. Pérez-González points out
the relevance of the fan-technology relation to different fields of translation activity. He
further argues that active consumers have used technology to start changing the
previous decision-making process in the whole media industry. Pérez-González’s
approach takes the discussion into a dimension that is embedded in the Media Studies
context described in Chapter 2 above. He states that user-producer interaction in the
current situation and the new consumer initiatives indicate that “the power of media
consumers is set to grow further in the future” (Pérez-González 2006:275). Hence, to
31
The Reception of (Non)Professional Subtitling
gain real insight into the fansubbing phenomenon, it is appropriate to visit the common
grounds of media and the language industries to develop more solid basis for advanced
studies.
In a subsequent article, Pérez-González (2007) revisits the most specific
differences between non-professional subtitling and professional subtitling, and he
presents one of the first empirical studies on fansubbing to use a sample corpus. He
relies on a multimodal theoretical framework as the methodological strategy to evaluate
the state of fansubbing conventions. His analysis goes throughout the corpus describing
how the fansubbing techniques transgress, modify or enlarge the most-accepted
techniques of mainstream subtitling. The aim of such detailed analysis is to examine
how “one of the most consolidated emerging subtitling cultures is contributing to the
evolution of subtitling practices” (2007:78) by exploring new possibilities and
exploiting the range of possible options offered, using different fonts, colors and
location on the screen. The development of the technological means that are accessible
to the non-professional subtitlers makes it clear that “the rationale for the use of
audiovisual translation in the near future is likely to become more heterogeneous, less
predictable” (Pérez-González 2007:77). In this sense, Pérez González’s view coincides
with the perspectives on the influence of technology presented by Tymoczko (2005).
However, with some specific exceptions, it could be said that this renovation of
mainstream subtitling based on the characteristics of non-professional subtitling has not
really taken off, and it does not seem to be just around the corner.
3.2.3. Non-professional and professional subtitles from a professional point of view
Fansubbing has experienced an exponential growth over the last decade. Nonprofessional subtitling websites with audiovisual material other than anime have been
created all around the world and have produced subtitles for both licensed and nonlicensed films and TV shows. The spectrum of user-generated subtitling has broadened
and so has the field of study. Empirical studies analyzing quality issues have been
carried out by scholars looking at different language pairs and at the translation of nonanime products, including Bogucki (2009), La Forgia and Tonin (2009) and Feitosa
(2009).
Discussing amateur subtitling rather than fansubbing, Bogucki (2009) analyzes
the errors in the amateur translation of The Fellowship of the Ring into Polish. The
32
Chapter 3. Literature review
inferior quality offered by the non-professional subtitles is stated at the beginning of the
article since, according to the author, “the product under discussion does not qualify as
fully-fledged subtitling” (Bogucki 2009:49). Bogucki argues that quality assessment is
not merely about error-hunting, but his revision is restricted to 15 errors found in an
amateur version released six weeks prior to the official release of the film in Polish in
2001.
The error analysis presents a linguistic assessment of the non-professional
subtitles, which allows the errors to be connected to a certain production condition in
the translation process. Bogucki clearly states that non-professional subtitling is guided
more by instinct than by reliance on the subtitling standards agreed within subtitling
industry:
The problem with amateur subtitling lies not so much in squeezing the gist of
what the original characters say into 30 or so characters per line to enable the
audience to appreciate the filmic message without too much effort; the problem,
it seems, lies mostly in the quality of the source material and the competence and
expertise of the translator. (Bogucki 2009: 50)
Bogucki argues it is impossible to compare professional and amateur subtitling
since the production conditions of both differ enormously, particularly considering that
“the product of amateur subtitling tends to be marred by translational error due to the
translator’s lack of linguistic competences in the SL, incomplete source texts, or both”
(2009:56). As long as amateur subtitling continues to be produced under such
extenuating conditions, it is not a threat to professional subtitles and is not worthy of
becoming a topic for research. Home-made subtitling would need to be produced under
more professional conditions and would require to be performed by more capable
people before:
[T]he resulting target text can be subject to translation quality assessment and
compared to professional cinema subtitling. Then – and only then – can it be
studied by academics and scholars. If, however, amateur subtitling continues to
be done on the basis of incomplete information, it will necessarily also be
imperfect and not available to academic study due to its high degree of
unpredictability. (Bogucki 2009:56–57)
33
The Reception of (Non)Professional Subtitling
A very similar approach to that of Bogucki is taken by Sajna (2013). He relies on
a corpus of eight feature films to carry out an error analysis of non-professional
subtitlers from the perspective of professional subtitlers. Using a typology of translation
errors developed by Hejwowski, Sajna shows how all instances of these errors can be
found in the non-professional version. Instead of exploring the new insights brought
about by non-professional subtitling (Pérez-González 2006), this type of study tries to
fit non-professional activities into the mold of professional subtitles.
Unlike Bogucki and Sajna, I consider non-professional efforts are valuable
particularly because they show there are alternatives to professional standards. As
pointed out by Nornes (2007), mainstream subtitling is part of a wider media system
whose main interests might not coincide with those of non-professional subtitlers. I
posit that the possibilities of these practices are a ripe environment for innovation that is
not allowed within accepted orthodoxy and that could help move forward not only
audiovisual translation but also the way we conceive translation in general.
In his reflection on non-professional activities, Sajna (2013) does provide an
interesting consideration regarding the expansion of fansubbing. He argues fansubbers
should be fans of the products they translate in order to be called so, and contends this is
no longer the case for many communities. There are non-professional subtitlers who
“appear to be fans of translation and to have fun engaging in the very activity of
subtitling films” (2013:3). Thus, Sajna labels these translators funsubbers. The
popularization of non-professional subtitling and the expansion of the phenomenon
have actually made this the rule rather than the exception, as similar considerations can
be found in previous works by Dwyer and Uricaru (2009) and Orrego-Carmona (2011).
Feitosa (2009) presents a systematic study comparing professional and nonprofessional subtitles. By applying corpus studies methods, he compares Brazilian
Portuguese commercial subtitles and pirate subtitles. No differences in terms of number
of lines were found in the corpus, and the mean duration for subtitles was the same for
the professional and the non-professional versions. Non-professional subtitles turned
out to be wordier than the professional ones. Feitosa found an explanation for this in the
forums, where subtitlers commented on the fact that these subtitles are an alternative to
the commercial ones and that the translation is longer and closer to the original (in the
sense of number of words) because people want to know “tudo que os personages
falam” [everything the characters say] (2009:66). Consequently, omission, reductions
and condensations were more common in the commercial subtitles than in the non34
Chapter 3. Literature review
professional versions. Feitosa (2009) concludes it is not possible to say one version is
necessarily more adequate than the other, since the differences seem to respond to 1) the
different ways of expressing a message, and 2) the approach taken in the research and
the annotation system of the corpus.
The Italian non-professional subtitling sphere is very active and has been
explored extensively. Barra (2009) talks about the general Italianization of US series
and dedicates a section to non-professional subtitles. The expansion of the nonprofessional subtitling activities goes back to the release of the series Lost, which
caused a massive growth in the communities. Barra describes something similar to what
Pérez-González (2013) later labeled the remediation of the subtitles: non-professional
subtitles make themselves visible and identifiable to the users. Echoing Díaz Cintas and
Muñoz Sánchez (2006), Barra mentions that this is possible because non-professional
subtitles, unlike their professional counterparts, aim at a reduced audience of fans with
shared interests. This does not mean that there is no regulation: as he claims, the amount
of time non-professional communities have been operating indicates that “in recent
years they have evolved production routines, roles and tasks becoming more and more
similar to professional dubbing (Barra 2009:521).
In another case study, La Forgia and Tonin (2009) make an analysis of how intertextual references are handled in the official Italian and Spanish dubbed and
professionally subtitled versions, as well as the fansubbed versions of the TV series
Supernatural. The decision to include fansubbed versions in the study responded to the
assumption that they are intended to be more loyal to the original version. Fansubbers
are more favorably inclined towards source-oriented translations that are also more
liberal regarding the inclusion of taboo words. The results, however, were different to
what was expected:
[T]he fansub translations, in general, are of much lower quality than the official
translations. First, this is due to the lack of linguistic and pragmatic competence
we found in many cases, and second, because they tend to overlook the function
of intertextual references, hence treating them as mere linguistic and not cultural
“facts”, which leads them to translate (not always adequately) only the linguistic
form, losing the cultural component. (La Forgia and Tonin 2009: my translation)
35
The Reception of (Non)Professional Subtitling
Massidda (2012) describes the two largest non-professional subtitling
communities and compares the subtitles they produce to professional DVD subtitles. As
in other places, the non-professional activity in Italy has had a significant impact on
television. The two non-professional subtitling communities analyzed, ItaSA and
Subsfactory, have penetrated deeply into the country’s tradition and their work has
caused television channels to reconsider some of their choices in terms of translation of
popular TV shows and delays in the distribution of new content. Massidda found that
non-professional subtitles have refined over time. They follow previously established
standards in terms of length of subtitles and have even achieved good spotting, one of
the most neglected aspects in non-professional subtitles. There are occasions on which
the official DVD version resembles the original, as pointed out by Barra (2009).
Surprisingly, when analyzing one episode in her corpus, Massidda found that the
original version had more inadequate renderings than the non-professional counterparts.
Demonstrating that professionals and non-professionals are still considered to be ruled
by dissimilar directives, Massidda argues “[w]e might of course accept such erratic and
unpredictable behavior if it had been found in the translation carried out by fansubbers,
but we would certainly not expect this level of inaccuracy on the part of professional
translators” (2012:159).
Also in Italy, Casarini (2014) analyses the changes of Italian viewership through
the framework of the globalization of non-Anglophone viewership and the evolution
and adoption of audiovisual translation practices, namely dubbing and fansubbing.
Claiming that the changes in the viewership as well as in the consumption and
production habits are irreversible, Casarini suggests the audiovisual industry in Italy
should reassess its strategies to respond to the needs of a more engaged and active
audience. Casarini argues that, although non-professional subtitling communities do not
aim at invading the space gained by dubbed products, their existence creates an
alternative for the audience and alters the status quo. Some of the advantages of
fansubbing are that it permits more interaction among viewers and allows for the
emergence of viewership 2.0. Under these circumstances, the dubbing industry should,
Cassarini claims, “assimilate the greatest lesson offered by fansubbing, which is how to
cater to evolving audiences by focusing on the specific traits of each show and learning
how to navigate the paths of intertextuality” (2014:21).
With respect to this evolution of audiovisual translation, Casarini says that the
context of video consumption in Italy might benefit, in the future and given the proper
36
Chapter 3. Literature review
disposition of audiovisual translation providers, from a convergence between dubbing
and non-professional subtitling. A model that integrates the renowned features of the
Italian dubbing industry:
such as dialogue fluency, careful synching, and powerful voice-acting, […]
combined with the less domesticating approach and the deep knowledge of each
show’s specificity that are typical of fansubbers. Hiring the latter (or particularly
conversant fans) as consultants would allow adaptation teams to benefit from a
collaborative epistemic pool that is imperative in the contemporary scenario
(Casarini 2014:285).
This proposal takes the discussion forward and focuses on the now commonly
mentioned co-existence of non-professional activities and mainstream audiovisual
translation modalities.
Wilcock (2013) provides another study of non-professional subtitling that
compares DVD subtitles with non-professional subtitles. In this case the original
language is French and the target language is English, unlike in most of the previous
cases in which the source content is either in English or in Japanese and uses English as
a pivot language. In her study, as in previous cases, Wilcock found the non-professional
version to be more source-oriented, retaining more elements from the source version
and displaying a more complete rendering of the dialogues. However, she explains this
in terms of the translations being suited to different needs, as pointed out by Díaz Cintas
and Muñoz Sánchez (2006) and Barra (2009), among others. The fact that the nonprofessional translation has a self-selected and characteristic audience implies that those
subtitles are operationalized for them: “For the fansubbers, communication is tailored
towards an audience with some background knowledge, and that is willing to make
more effort in order to understand cultural references” (Wilcock 2013:108).
3.2.4. Profiling the subtitlers
One aspect that has also become attractive for researchers is the identity of the nonprofessional translators and why they are doing these translations. Although several
small-scale studies have been carried out, I am going to reference some of the largest
that have tried to profile non-professional subtitlers. In his Master’s thesis, Kreb (2011)
sets out to investigate the landscape of the Dutch non-professional subtitling
37
The Reception of (Non)Professional Subtitling
community. Most of the respondents were men with an average age of 33 years, living
in the Netherlands and Belgium. The majority of the respondents said they produce
subtitles because of personal reasons, but only half of them see subtitling as a hobby.
Unlike other non-professional communities, more than half of the participants said they
would accept or at least consider accepting a paid subtitling job. Community building
was also one of the reasons most commonly given by participants as an important
aspect of their subtitling activity: the respondents saw their fellow subtitlers as friends.
The questionnaire also asked the participants about their degree of agreement with the
subtitling guidelines included in Díaz Cintas and Remael (2007), and most of the
participants said they agreed with the guidelines. However, as no subtitles were
analyzed in the study, it is impossible to know to what extent the guidelines were
actually adopted.
Chu (2012) reports on a survey of Chinese fansubbers. In total, 80 respondents
completed the questionnaire. The findings indicate:
that young, well-educated, urban Chinese are using fansubbing as both a form of
serious leisure and a learning platform, which helps to account for the altruism
and volunteerism they display in their highly commercialized and competitive
everyday lives. (Chu 2012:274)
As suggested by smaller scale studies (Orrego-Carmona 2011), Chu found that
most non-professional subtitlers take on translation activities as a hobby and do not seek
monetary reward, not even in the future. The respondents emphasize they operate in
what we might call an interculture and they acknowledge language-learning rewards
and altruistic motivations. Among the respondents, there is also a high degree of
commitment to the community and an atmosphere of camaraderie in the group.
In a similar study, Luczaj et al. (2014) surveyed the Czech and Polish nonprofessional subtitling communities. They collected 68 questionnaires in the Czech
Republic and 40 in Poland. As in Chu’s study, most respondents were well-educated,
tech-savvy, young urban people. In both countries, most of the respondents were men.
They found variations in the methods of working in the two countries: Czech translators
were found to work mostly in groups, while the Poles seem more individualistic and did
not see themselves as a community. The respondents also claimed they enjoyed creating
the subtitles and felt they were doing something useful for other people. The subtitlers
38
Chapter 3. Literature review
in the Czech Republic were found to be more likely to meet other subtitlers in person,
while a high number of Polish non-professional subtitlers did not know any other
subtitlers.
3.2.5. Studies on specific non-professional subtitling features
There has been an assumed relationship between non-professional subtitling practices
and censorship. Overcoming government and industry control implies that subtitlers are
not constrained by external regulations. In an account of the audiovisual translation
panorama in Romania, Dwyer and Uricaru (2009) introduce the work of a nonprofessional subtitling group in the country. After commenting on the effect of
censorship and user-generated translation in Communist Romania, non-professional
subtitling is shown to be a part of the Romanian piracy context. The authors refer to a
group that produces subtitles for foreign content.
Interestingly, after describing the regular features of fansubbing communities,
Dwyer and Uricaru emphasize that the group they are describing “mirrors the activities
of anime fansubbing networks […] yet with one essential difference: they are
fansubbers pure and simple in that they are actually fans of subtitling itself rather than
of any particular genre or product” (2009:50). They make a distinction between
fansubbing being genre-specific and the other groups who more generally provide nonprofit translation for foreign media. Dwyer and Uricaru’s reasoning concords with
Sajna’s claim about the difference between fansubbing and funsubbing, which stresses
the different reasons to participate in non-professional subtitling activities.
Moving away from the actual subtitles and focusing more on non-professional
subtitling communities as systems, Hemmungs Wirtén (2012) presents an account of a
2009 strike of fansubbers in Sweden. The paper explores the interactions between
various non-professional subtitling communities with respect to international copyright,
but it does not focus on the fans’ infringement of the copyright laws in detriment of the
audiovisual content copyright holder: it focuses instead on fan-to-fan piracy.
An administrator of a non-professional subtitling website in Sweden was
appropriating the subtitles produced by the competing groups by taking them and
removing any reference to the original translators before posting the subtitles on his
own website. The original creators of the subtitles were not concerned about the
redistribution of content, which is a regular practice among non-professional
39
The Reception of (Non)Professional Subtitling
communities, but were instead disturbed by the fact that they were stripped of their
authorship and no longer recognized as producers. Unexpectedly, the wronged subtitlers
decided to do what a union would do and went on a strike, so that the distribution
website would not have new subtitles coming in for newly released shows. The
discussion about authorship is presented from various perspectives, with some groups of
non-professional subtitlers considering it important for individuals to be recognized as
authors and willing to articulate a defense of their rights, while another group argued it
was the group as such that should be identified as the author, not the individuals.
This is hardly surprising. Since non-professional subtitlers do not receive
monetary payment for their work, they develop a high degree of attachment to their
products. Being recognized as translators of their subtitles is their reward for their work.
Their motivations are mainly personal (Kreb 2011; Luczaj et al. 2014) and the visibility
they have as translators is valuable to them. Hemmungs Wirtén draws links between
this situation and the discussions that led to the Berne Convention and its revision. Once
again, translation is found at the center of the discussion, but more importantly, even in
a paraprofessional activity that is often judged as illegal, the concept of author remains
central and the debates about authorship, control and piracy are still current.
Also commenting on transformative and authorial practices, Pérez-González
(2012) analyzes the co-creational practices occurring in non-professional subtitling.
Pérez-González sees the emergence of transformative subtitling practices as a result of
the de-legitimization of industrial subtitling. Presenting industrial subtitling as a selfeffacing activity, he argues that:
By smoothing over cultural differences on the grounds of medial constraints and
hiding their presence through restrictive rules, corrupt subtitles fail to explore
those spaces of generative cultural and linguistic multiplicity that emerge in any
instance of cross-cultural transaction. (Pérez-González 2012:5)
The constraints to subtitling are harnessed by the regulations of the industry.
However, emancipated collaborative translation practice is not subjected to industry
constraints and “often involves a re-positioning of the text within the geography of the
frame, a shift from a representational to an interventionist agenda that erodes the
privileged status of the original text” (Pérez-González 2012:6). Using fansubbing as an
example, Pérez-González explores how consumers and producers of fansubbing differ
40
Chapter 3. Literature review
from other audiences in how they engage with content. Fansubbers have developed their
own standards and voluntarily position themselves outside of the mainstream flow. The
fansubber’s intervention is seen as means to remediate the product and augment the
original text.
This time focusing on the implications of non-professional subtitling for online
citizenship, Pérez-González (2013) shows how the democratization of technologies, in
this case the ones involved in subtitling, promotes radical citizenship responses. The
article explores the role of activist non-professional subtitlers in the circulation of media
content and the participatory media environment. Building on the theories of cocreational practices, Pérez-González analyzes the case of a group of activist nonprofessional subtitlers who distributed subtitled footage featuring Spanish Prime
Minister Aznar’s ultraconservatist views. Pérez-González’s account of civic
engagement through subtitling constitutes another layer of non-professional subtitling
that has not been completely covered. Non-professional subtitling gains the role of an
act of resistance not only against media corporations but also in the face of political
discourses that are strictly controlled. The type of subtitles produced by the community
are also an instrument of remediation of media content. According to Deuze (2006),
remediation is the remix of old and new media. Active users remediate when they adopt
old media strategies or products but manipulate and modify them according to their
ideas in order to create a new reality. This is often the case with fansubbing (PérezGonzález 2012). These non-professional subtitlers want to “make their presence felt”
and openly voice their own opinions about what is being said in the video. They
distance themselves purposefully from mainstream practices, not to bridge cultural gaps
but to specifically embed themselves into the resulting content.
Dwyer (2012) offers a thorough account of a start-up company, ViKi, founded
around non-professional subtitling. ViKi started translating Korean programs but has
grown to include many other genres. The company capitalizes on technology
affordability and collective intelligence to break international boundaries and alter the
media distribution system. The popularity of the website caused it to grow exponentially
and reorganize itself in the form of a full-fledged monetized company. The main
innovation of the group is its online software that allows subtitlers to collaborate in the
translations and interact in real time. In many senses, as pointed out by Dwyer, ViKi
cannot be understood in the same way as other non-professional subtitling communities.
Like previous authors, Dwyer comments on the need to move forward from the concept
41
The Reception of (Non)Professional Subtitling
of fansubbing and include translation for other genres from all over the world. ViKi
disrupts the traditional Western focus of translation and “brings to the world media
content from small language-communities, developing countries and Asian superpowers
alike – including, among others, Bangladesh, China, Egypt, Iran, Laos, Mongolia,
Nigeria and Turkey” (Dwyer 2012:231).
Interestingly, the growth of the community has also had an effect on the subtitles,
which have become more similar to mainstream subtitles. This not only concerns the
layout of the subtitles, but has also caused ViKi’s subtitlers to adopt more domesticating
techniques, which is clearly an exception within the context of non-professional
subtitling communities.
Although ViKi has adopted a more commercially oriented business model, the
profile of its members is similar to those observed in other communities: “ViKi
fansubbers belong to the same tech-savvy, cyber-elite demographic typical of other
fansub communities. Their interventionist involvement in the media marketplace is
dictated by matters of geography, language, economics, market size and legality”
(Dwyer 2012:234).
Izwaini (2014) comments on instances of non-professional subtitling in Arab
countries. Subtitling is not only used to translate content but also to create parodic
translations, similar to the cases presented by Pérez-González (2013). The content
translated is of different kinds, from cartoons to religion, and while most of the content
is translated from English, there are also other Middle Eastern languages involved. One
practice that is commonly used is relay translation: videos are subtitled using English as
a pivot language. Izwaini stresses that non-professional Arabic subtitles tend to be
written in a rather colloquial register and, since they are not censored, they “tend not to
mitigate strong language by choosing less offensive expressions” (Izwaini 2014:103).
According to him, the quality of non-professional subtitles is normally low and nonprofessional subtitles are not in competition with professional translations. However, he
recognizes there are different levels of quality, and claims this could depend on the
genre, with political and religious texts demonstrating a higher degree of quality. At the
same time, it could also depend on the translators, since individual projects tend to be of
lower quality than translations published by organized non-professional subtitling
groups that have set their own internal quality assurance process.
42
Chapter 3. Literature review
3.2.6. Conceptualization of non-professional subtitling and translation
In cases such as mine, monetary reward is taken as the decisive factor. O’Hagan (2011)
argues that some scholars prefer to focus on other aspects, such as philanthropic
motivations, as the central element to define community translation.
Due to the nature of fansubbing and its status as the oldest mode of nonprofessional subtitling, it is not uncommon for scholars to refer to all types of nonprofessional subtitling in general as fansubbing (Barra 2009; Massidda 2012). However,
the expansion and proliferation of non-professional subtitling activities makes it
imperative to adopt a broader definition to refer to the creation of subtitles by people
(Dwyer 2012). It is no longer the case that everyone who participates in these endeavors
can be labeled a fan of the product when, for instance, some people create subtitles
regardless of the content, merely for the fun of it (Orrego-Carmona 2011; Sajna 2013).
Referring to the broad area of non-professional translation, O’Hagan (2009)
resorts to the concept of user-generated content and proposes the term user-generated
translation. She describes the term as “a wide range of Translation, carried out based on
free user participation in digital media spaces where Translation is undertaken by
unspecified self-selected individuals” (O’Hagan 2009:97).
Positing that the central difference is the lack of monetary reward, Pym (2011)
suggests using volunteer translation and argues it is the natural opposite to professional
translation. The term, nevertheless, suggests an activist meaning and brings a political
agenda into discussion – volunteers are normally called to offer their service for a cause
they believe in. Although some instances of non-professional translation fit into this
description, there are other types of non-professional translation that, as explained by
O’Hagan, “are embedded in strict commercial contexts applied by for-profit
organisations that are generally devoid of such political agenda, which cannot be said of
cases of fan translation and its variants” (2011:14). Recognizing the complications of
settling on an umbrella term for the phenomenon, O’Hagan herself relies on the label
community translation to refer to the set of practices that are tied to online communities
within the Web 2.0 context. She argues that the term nevertheless requires clarification
because it can be confusing due to its multiplicity of meanings, such as the natural link
to community interpreting. In her view, the medium and the structure of the groups
producing the subtitles determine the label to use.
43
The Reception of (Non)Professional Subtitling
Fernández Costales (2012; 2013) uses both collaborative translation and
community translation as umbrella terms to refer to cases in which translation is done
by unpaid groups of people motivated to work together towards a common aim and
whose operations are mediated by technology. Using these two terms is a consequential
choice, since the more detailed characteristics that define the differences between the
practices, such as who the initiator of the translation is and whether the translation is
paid or not, emerge at a more advanced level. Both professional and non-professionals
can work collaboratively and normally partake in translation communities, so these
labels allow for a recognition of the points they have in common. Fernández Costales
(2012:121) differentiates between crowdsourcing and community translation.
Crowdsourcing initiatives are coordinated and put it place by a company, while
collaborative translation projects are initiated and completed by the translation
communities themselves. Nevertheless, looking at the mechanisms used by many
crowdsourcing projects, it is clear that these groups also tend to form communities and
work in collaborative ways in order to complete their tasks.
Gambier (2014) talks about communities but does not refer to community
translation. He offers a typology with five different categories of translation practices:
machine translation, amateur translation, collaborative translation (teamwork),
translation with open source tools and volunteer networked translation. The
classification seems somehow arbitrary and leaves room for discussion. In Gambier’s
explanation of collaborative translation, the activity is restricted to professionals only:
Collaborative translation (teamwork) that is carried out on a same, single
document by professionals places dematerialized computer resources at the
common disposal of all. This includes document research, terminology, rereading and revision. It is manifest in such sites such as Proz, Translator’s Café,
etc. “Cloud” cannot be confused with “crowd”. (Gambier 2014:4–5)
However, as touched upon above, teamwork is not only restricted to
professionals. Additionally, some non-professional platforms actually follow patterns
comparable to those of Proz or Translator’s Café. The categories overlap and respond to
different classification features: some are defined by the type of tools used while others
refer to the communication and organization of the people doing the translation. This
creates conflicts since, depending on the case, the hierarchy can be easily changed and
44
Chapter 3. Literature review
one concept could include the other or at least be at the same level. For instance,
collective translation (including crowdsourcing) is considered a branch of amateur
translation, totally independent from collaborative translation even though the
mechanisms are very similar in both.
Non-professional subtitling is a controversial topic. When it comes to the
discussion of the subtitles as such, there are scholars who question the very nature of
non-professional subtitling and do not see it as a proper concern of Translation Studies
unless it adopts more professional standards (Bogucki 2009; Cornu 2013; Sajna 2013).
Others are still cautious about quality issues (Díaz Cintas and Muñoz Sánchez 2006)
and its possible implications for the translation profession (Cronin 2012; Fernández
Costales 2012). However, the current response to the phenomenon is generally positive.
While not promoting wholesale adoption, some scholars think that acknowledging nonprofessional subtitling can actually benefit the audiovisual translation industry in
different aspects (Pérez-González 2007; O’Hagan 2008, 2011; Dwyer 2012). Nonprofessional subtitling is seen as a driver of innovation in audiovisual translation,
encouraging collaboration methods rather than individual translation enterprises. It
highlights the relevance of translators being content or genre experts who take a holistic
and rather foreignizing approach to translation, against the domesticating mainstream
preference.
O’Hagan’s claim that “Translation Studies can no longer afford to overlook the
fan translation phenomenon” (2008:178) seems to be partially vindicated. Researchers
in Translation Studies have embarked on the mission of exploring user-generated
translation on different fronts. Although still highly concentrated in the audiovisual
translation field, non-professional translation is being discussed and researched and,
judging by the increase in the number of publications in recent years, this trend is likely
to continue.
In this review I have focused almost exclusively on works studying nonprofessional subtitling. We clearly now know more about the people making the
subtitles, how they organize themselves and the impact they may have on the
distribution of content. Additionally, we know about the subtitles as cultural products
and how they compare with their professional counterpart. That is only one part of the
picture, however. The landscape of user-generated translation is much richer and more
complex. It ranges from the study of ethics (McDonough Dolmaya 2011a) and its
comparability to professional codes of ethics (Drugan 2011) to the impact of
45
The Reception of (Non)Professional Subtitling
crowdsourcing on translation practice and translators (Mesipuu 2012; McDonough
Dolmaya 2012; Fernández Costales 2012). One aspect that is becoming more interesting
for researchers is the motivations that prompt people to participate in user-generated
translation endeavors (McDonough Dolmaya 2011b; Olohan 2014). Those issues
concern production. Less research, however, has been focused on reception.
3.3.
Research on reception
3.3.1. Exploring the viewers’ reception of translated audiovisual products
The relevance of reception studies has been discussed in audiovisual translation studies
since the 1990s. Most of the works published over the past decade have echoed
Gambier’s (2003; 2008) words on the lack of studies dealing with reception:
Very few studies have dealt with the issue of reception in screen translation, and
even fewer have looked at it empirically, even though we continually make
reference to readers, viewers, consumers, users, etc. (2003:184)
Although reception was already part of the picture of audiovisual translation in
the early stages of the boom of empirical studies in audiovisual translation in the 1990s
(Kovačič 1995), the study of reception in Audiovisual Translation, specifically, and in
Translation Studies, in general, has not really taken off (Brems and Ramos Pinto 2013).
The main reasons for the lack of studies are the difficulty of defining what reception is,
the large samples required, and the difficulty of generalizing the results. In order to
define some common ground, Chesterman (2007) proposes a set of terms that will
provide a causal model for the understanding of translation effects and help define
reception. Chesterman argues that translations cause reactions, responses and
repercussions. These categories, which are not restricted to translation effects only,
were adopted by Gambier and brought into audiovisual translation studies.
On the bases of Chesterman’s discussion and Kovačič’s (1995: 376) descriptions
of the different levels of reception, Gambier (2006; 2008; 2012) presents a model that
he labels as the 3 Rs (reactions, responses and repercussions). This is a comprehensive
way to distinguish the different levels of reception of translated audiovisual products.
The framework analyzes reactions on the psyco-cognitive level, responses on the
perceptual level, and repercussions, which are understood in two distinct ways: as an
46
Chapter 3. Literature review
attitudinal issue, in terms of the viewer’s preferences and habits, and also as the “sociocultural dimension of the non-TV context which influences the receiving process”
(Gambier 2006:22).
Recent years have seen an increase in the number of studies exploring audience
reception. Most of these have focused on subtitling and accessibility, but some include
other modalities of audiovisual translation as well. Along with the broadening of the
literature, there has also been a development in the methods used to explore audiovisual
translation reception. This section will report on reception studies, highlighting the
methods researchers used, the topics they were exploring and their main findings.
3.3.2. Empirical studies on audiovisual translation
In their review of studies that have assessed the effectiveness and the effects of subtitled
material, de Linde and Kay (1999:35) propose a typology that serves to understand the
landscape of research in audiovisual translation studies dealing with reception:
-
Survey method: “eliciting viewers’ responses to questions about their experience
of subtitled television”
-
Semi-controlled experiment: “looking at viewers’ responses to different sets of
pre-categorized subtitles”
-
Controlled experiment: controlling “both medium and viewer in order to gain
precise behavioural information about how particular subtitle characteristics are
received”.
The use of survey methods has not been the preferred approach within
Translation Studies. It consists of large-scale studies aimed at defining the general
preferences of audiences. As such, this type of research has fallen to government
agencies, national TV agencies, TV channels, distributors and audiovisual material
providers. Some examples could be the reports published by Ofcom in the United
Kingdom or, more properly speaking, the studies on the dubbing and subtitling industry
(2007) and on the use of subtitling in Europe (2011) that the Media Consulting Group
has carried out for the European Commission.
Semi-controlled and controlled experiment methods have been more popular
among Translation Studies researchers, due to the more detailed information they
provide. In the former, the researchers define the specific features of the audiovisual
product or translation they are interested in and check the effects these features have on
47
The Reception of (Non)Professional Subtitling
the viewers. This method differs from the survey method in maintaining the same
variables for all participants. The controlled experiment methods, apart from controlling
the input that is used, also record the “actual motor behaviour” (de Linde and Kay
1999:37) of the users. Linde and Kay talk specifically about recording eye movement,
but there are other biometric possibilities such as heart rate, skin conductance and EEG
measurements. With these methods, researchers normally resort to questionnaires as the
means to elicit the viewers’ reactions and opinions. Questionnaires are time-efficient
and allow access to a large number of responses in a short time. The combination of
questionnaires and eye-tracking data is becoming more and more popular, but
researchers have also used direct observation, interviews, focus groups, and tasks to
collect data on reception.
3.3.2.1. Studies in Leuven
Eye-tracking methods have been part of research on reception of translated audiovisual
content for a long time. Much of the research carried out in the area since 1980s was
performed under Géry d’Ydewalle and his colleagues at the Katholieke Universiteit
Leuven (d’Ydewalle et al. 1985; d’Ydewalle et al. 1987; d’Ydewalle and van
Rensbergen 1989; d’Ydewalle et al. 1991; d’Ydewalle and Pavakanun 1995;
d’Ydewalle and Van de Poel 1999; Bruycker and d’Ydewalle 2003; d’Ydewalle and
Bruycker 2007). In an eye-tracking experiment, d’Ydewalle et al. (1985) found
participants do not read the subtitles in their entirety: “They first look at the visual
image, jump quite accurately to the keywords of the subtitle (i.e., the words conveying
the most important parts of the conversation) and then go back to the visual image.” If
there is time left, some participants return to the subtitle area and read the entire
subtitles. Subsequent studies (d’Ydewalle et al. 1987) found subtitle reading to be a
more or less automatic behavior: viewers cannot avoid reading the subtitles when they
are shown on the screen, even when they understand the source language. Further, their
research shows that shifting attention between the image and the subtitles is effortless
for viewers. These initial reports do not offer a complete description of the experiment,
and in some the number of participants or the details of the translated audiovisual
product are not included, which complicates the interpretation of the results.
Nevertheless, the findings served as the foundation for future studies. They
demonstrated that people look at the subtitles and indicate there is a degree of
automation, rather than a thought decision, in the reaction to the visual input. The
48
Chapter 3. Literature review
researchers used these studies to formulate hypotheses that were tested in other
experiments. d’Ydewalle and van Rensbergen (1989) report on an experiment with
children and show that subtitle reading depends on the type of content and is not a fully
automatic action. When watching a cartoon with a heavy verbal load, children had gaze
patterns similar to those of adult viewers, but when they were watching an actionoriented cartoon, they relied less on the subtitles.
In yet another study, d’Ydewalle et al. (1991) tested the use of intralingual
subtitles among English native speakers not used to subtitling and Dutch native
speakers who were used to subtitling. Both groups spent a considerable amount of time
in the subtitles, which seemed to indicate that participants who are used to subtitling
look at the subtitle area even if they understand the spoken language and regardless of
their familiarity with subtitles. However, on this occasion, the researchers found there
was a degree of control in subtitle reading, even if it coexists with a more common
automatic behavior. Using news broadcast as input, d’Ydewalle and Gielen (1992)
found viewers look longer at the subtitle area when the audiovisual product conveys a
lot of information in a short period of time. The participants looked at the subtitles at a
faster pace and for a longer period of time, even when the news broadcast was in their
own language. Further, the findings support the idea that reading subtitles on the screen
is effortless and an almost automatic behavior.
Koolstra et al. (1999) study the applicability of the six-second rule. By setting up
an experiment in which subtitles were shown at three different speeds, (6, 8 and 10
seconds per subtitle), they found that the longer the subtitles were shown on the screen,
the longer viewers, in this case children, would look at the subtitle area. Another
relevant result from the experiment indicates that poorer readers spend only around 30%
of the time looking at the subtitles (much less than other participants) when the subtitles
created according to the six-second rule. Probably, the researchers argue, viewers were
discouraged and decided not to put much effort into trying to follow the subtitles. On
the other hand, the same group spent more time reading the subtitles with the 10-second
duration, which indicates they are able to adapt to the duration of the subtitles.
The above studies approached subtitle reading from a global point of view and
centered mostly on attention allocation. However, d’Ydewalle and Bruycker (2007)
report on the detailed behavior of participants when reading standard and reversed
subtitles (i.e. foreign language subtitles and native language soundtrack). They analyze
the reactions of children and adult participants who watched excerpts of a movie with
49
The Reception of (Non)Professional Subtitling
standard and reversed subtitles. Additionally, they added the number of lines in the
subtitles as a variable in the experiment. The participants skipped fewer subtitles in the
standard (interlingual) subtitle condition, which also means they had slightly more
fixations and spent a longer amount of time in the subtitle area under this condition.
Under the standard subtitling condition, the participants also had a more regular reading
behavior when watching two-line subtitles. With standard subtitling, the mean fixation
duration was 178 ms for adults and 248 ms for children.
3.3.2.2. Asking users to assess subtitles
The studies reported above by the group at Katholieke Universiteit Leuven relied almost
exclusively on eye-tracking measurements to test the hypotheses. As mentioned, other
methods have also been used in reception studies of translated audiovisual material.
Gottlieb (1995) used a protest button to elicit simultaneous responses from viewers. The
participants were asked to press the button every time they notice a deviation from
subtitle standards. The 123 participants in the experiment were deaf and hard-of-hearing
people. Interestingly, Gottlieb (1995) found that participants did not react to faulty
subtitles according to professional standards, but judged regular subtitling techniques,
such as condensation, as faulty subtitles. Further, the audience reacted only to extreme
cases of heavy cutting or distortion of the information. The protest button is an
innovative technique and the study demonstrates that professional standards do not
necessarily coincide with audience expectations. Nevertheless, asking participants to
assess the subtitles while they are watching the audiovisual content affects the
ecological validity of the experiment. Apart from dividing their attention between the
image and the subtitles, the participants also need to devote attention to the button and
be alert to judge the translations, which results in a non-natural viewing experience.
3.3.2.3. The reception of humor and culture-specific items
In another study, Fuentes Luque (2003) uses direct observation to explore the reception
of translated humor from English into Spanish. The data collection consisted of three
stages: direct observation of the participants’ reactions during the screening, a
questionnaire, and a brief interview. The study includes 30 participants, ten for each
condition: original film in English, dubbed version into Spanish, and subtitled version
into Spanish. Although the sample is not very big, as commented by the author, possibly
the most problematic aspect is the wide age range of the study: going from 16 to 64-
50
Chapter 3. Literature review
years old. This is especially problematic since studies have shown there are differences
between age groups in a task like watching TV (d’Ydewalle et al. 1989). Fuentes Luque
found that the participants watching both translated versions showed lower positive
reception of humor elements when compared with participants who watched the original
version in English.
Direct observation of the participants while they are watching the input is another
method. Unlike with eye tracking, participants do not have to stay still and can behave
more naturally. There are no obtrusiveness problems, unless the researcher is in the
participant’s range of vision. This approach is, however, not unproblematic: the
codification of reactions is highly dependent on the researcher and measuring the
participant’s reactions implies that participants are supposed to react similarly to each
input, which is not necessarily the case.
The study of humor has been popular in audiovisual translation in general, as well
as in its reception studies branch. Another cluster of research in reception studies has
developed in Forlì, Italy, and is oriented towards the study of humor and culturespecific items. Delia Chiaro, Rachele Antonini, Chiara Bucaria and other colleagues
have worked on the reception of audiovisual content translated into Italian. The
continuity of the line of research undertaken by this group has allowed them to exploit
the resources they have at hand and to create a framework that accommodates and tests
different approaches (humor studies, audience reception studies). Further, it has allowed
them to provide a solid and rather complete overview of the reception of translated
products in Italy.
Chiaro (2004) used questionnaires to collect data on the perception of translated
verbally expressed humor dubbed into Italian. Antonini (2005) used an adapted humorappreciation test to test the reception of verbally expressed humor, visual humor and
satire on a sample of 32 participants. The results show participants had problems
understanding verbal humor. However, strangely, participants were able to recreate
puns that were not present in the subtitles. Antonini suggests this could be an effect of
the canned laughter in the products. Bucaria and Chiaro (2007) is an especially
interesting case because the researchers surveyed cinema and TV experts, linguists,
practitioners in the field of dubbing, and members of the general audience. The
researchers argue that the results indicate Italian audiences seem to be becoming more
familiar with foreign cultures, but this at the same time generates a higher acceptance of
“dubbese” (hybrid language used by dubbing industry) and makes it more difficult for
51
The Reception of (Non)Professional Subtitling
the audience to differentiate between “what is and is not real spoken Italian” (Bucaria
and Chiaro 2007:115). At the same time, the results indicate that issues arise when there
are gaps in the audiences’ encyclopedic knowledge that makes it impossible for them to
understand the content.
In another study about dubbese, Antonini (2008) uses an online questionnaire to
test the participants’ understanding of translated content. This study reports something
very interesting: most of the participants’ declared understanding differs greatly from
their actual understanding of the translated content. In her study, Antonini found that
more than 60% of the respondents declared they understood the cultural references in
the translation, but in 70% of the cases they did not understand them. This raises
concerns about the reliability of self-reported comprehension methods and accentuates
the need for triangulation. At the same time, it offers valuable insights into the viewer’s
thinking. Following on from this, Antonini and Chiaro (2009) studied the Italian
audience’s perception of dubbese. They found the features of dubbese language are
recognized as such by respondents; they are aware of the phenomenon. The dubbese
might not be adopted, but it is understood and accepted as an addition to the language.
By going a step further, Antonini and Chiaro (2009) shed some light on the social
repercussions of translation and point to the rapid effects that translation decisions have
on the audience.
Chiaro (2007) explores how Italian audiences perceive verbally expressed humor.
This time she compares the reaction of 34 Italian viewers and 22 British informants, to
analyze how each group responds to original and dubbed or subtitled content. This time
again, the sample included a large age group: from 17 to 76-years old. There is a small
disparity in the results, with British participants giving higher scores to the verbally
expressed humor than the Italian participants. This led the researcher to wonder if
humor should be consider culture-specific, coinciding with Fuentes Luque’s (2003)
results.
Cavaliere (2008) tested the reception of a domestic Neapolitan soap opera
subtitled in English. Using questionnaires, she measured the degree of comprehension
and enjoyment of the content among Neapolitans, a group of Milanese and a group of
Americans living in Naples (only the Americans used the subtitled version). In line with
Chiaro’s (2007) and Fuentes Luque’s (2003) conclusions, the Neapolitans were the ones
who were most able to appreciate the culture-specific elements. The absence of a
translation of these elements or the standardization of the subtitles negatively affected
52
Chapter 3. Literature review
the comprehension and enjoyment of the content. The focus on dialect in this study is
particularly interesting since up to now intralingual subtitling has been considered
almost restrictively as part of the subtitling for the deaf and hard-of-hearing people or in
research aiming at testing the advantages of subtitling for language learning, not at a
sociolinguistic level.
3.3.2.4. Eye-tracking and accessibility
Intralingual and interlingual subtitles for the deaf and hard-of-hearing people has also
been a prolific area of research. de Linde and Kay (1999) present a series of
experiments testing different subtitle features: subtitle rate, onset of speech, shot
changes, subtitle editing and visibility of the speaker. Their analysis relies mostly on
eye-movement measurements but also on the participants’ comprehension of the
content. The study included 20 participants, with an equal amount of hearing and deaf
people. In all cases the sound of the clips was off, so the participants relied only on the
subtitles for the verbal information. While this decision makes good sense to compare
the hearing participants’ under the conditions of the deaf participants, it affects the
comparison of these results with results testing hearing participants with access to the
soundtrack. Similarly to d’Ydewalle and Gielen (1992), they discovered that subtitle
pace affects reading speed: the faster the subtitles, the faster people read them. They
also discovered that slow subtitles induce re-reading, proving that both extremes, slow
or fast subtitles, have direct effects on viewers’ behavior. Further, shot changes caused
more deflections (eye movements when the viewer first looks at the subtitle, then
focuses on the image to lastly return to the subtitle area again) and the position of
subtitles affects viewers’ reading behavior, as does the visibility of the speaker on the
screen. Deaf viewers seemed to rely more on the image as a source of information,
which hints at a different type of engagement between the image and the viewer
depending on the latter’s hearing condition.
A series of studies about subtitling for the deaf and hard-of-hearing people has
developed in the United States. Jensema (1998) found that the comfortable subtitle
speed reported in a group of 578 people including deaf, hard-of-hearing and hearing
viewers was 145 words per minute, but the participants adapted well to increased speed
until they reach a 171-word per minute speed rate. Jensema et al. (2000b) found the
addition of subtitles to the screen drastically changed the behavior of the six participants
in the study and turned the viewing process into a reading activity. When watching
53
The Reception of (Non)Professional Subtitling
subtitled content, subtitle reading manifests as the primary activity, and viewing the
action on the screen becomes secondary. Jensema also found that a higher subtitle speed
makes the viewers spend more time on the subtitle area. In a larger study, Jensema et al.
(2000a) collected eye-movement data from 23 participants who were watching subtitled
television programs. The results indicate the participants spent 84% of the time looking
at the subtitles, with little variation (82%-86%) when the subtitle speed changed from
100 words per minute to 180 words per minute. Additionally, the researchers found no
influence of age and sex, contrary to other research (d’Ydewalle et al. 1987; Bruycker
and d’Ydewalle 2003). The education level of participants seemed to be a relevant
variable, but the collected data was not enough to interpret these results in practical
terms.
Szarkowska et al. (2013) report on a series of experiments conducted to test
subtitling for the deaf and hard-of-hearing and audiodescription. In one of the studies,
they tested three different types of subtitles: edited (simplified and reduced) subtitles,
standard subtitles and verbatim subtitles. The 40 participants in the study were deaf,
hard-of-hearing and hearing people. The results indicate the best condition was the
edited version of the subtitles, which allowed the participants to look more at the image.
When it comes to comprehension scores, the verbatim version produced the best results.
However, it was also in this version that the participants fixated on the subtitles the
most. In general, the authors point at contradictory results: in many cases the
participants’ preferred translation mode was not the one that produced the best
comprehension scores. The inconclusive results of this experiment motivated the
authors to carry out another study. The second study is still unpublished, but the
description included in Szarkowska et al. (2013) indicates they have collected data from
over 130 participants. The experiment includes two different subtitle speeds and five
types of audiovisual programs.
A subsample from the larger study commented on in Szarkowska et al. (2013) is
reported in Krejtz et al. (2013). In this paper, the authors focus on the effects of shot
changes on eye movements in subtitling. The results include the recordings of 71
participants (including deaf, hard-of-hearing and hearing people) who watched clips
from two genres: documentaries and news programs. The results indicate that shot
changes do not seem to prompt subtitle re-reading when they occur, but they do seem to
alter the viewer’s behavior by triggering more gaze shits from the subtitles to the image.
This coincides with the effect of deflections detected by de Linde and Kay (1999).
54
Chapter 3. Literature review
When going to the subtitle, the viewer’s gaze does not go to the beginning of the
subtitle in order to reread the whole text but is often directed to the middle of the screen.
In Szarkowska et al. (2013), deaf participants had more fixations on the subtitle area
than did hearing participants. In this second study, the participants with hearing
impairment had a longer first fixation duration and also spent more time on the subtitle
area and made more fixations. This supports the idea that people with hearing
impairment are slower readers.
3.3.2.5. Comparing the reception of subtitling and dubbing
Another interesting avenue that has opened up for empirical reception studies is the
traditional debate between dubbing and subtitling: which is better for the viewer? In a
study that used questionnaires to test the viewers’ immersion and enjoyment, Wissmath
et al. (2009) found that although the subtitles in a foreign language decrease the feelings
of spatial presence and transportation, the differences between the scores obtained with
the dubbing and subtitling conditions did not reach significance. Additionally, the
translation modality did not affect the viewer’s enjoyment. Wissmath et al. (2009) argue
that these results could be due to the Swiss audience’s regular use of subtitling and
dubbing.
Bairstow and Lavaur (2012) also report on a study comparing dubbing and
subtitling. In total, four conditions are included in their study: original version in
English, dubbed version in French, original version with French subtitles, and a
reversed condition (dubbed version with English subtitles). When it comes to the
comprehension of the dialogues, the dubbed versions score highest, while the version
with French subtitles produces comprehension that is almost as good as the subtitled
version. The original version scored the lowest results, due to the participants’ low level
of English.
In a study carried out by Perego et al. (2014) comparing dubbing and subtitling,
the findings seem to concord with Wissmath et al. (2009): subtitling does not affect the
viewers’ enjoyment and appreciation of the film when compared to dubbing. However,
subtitling proved better when it comes to support “the lexical aspects of performance”.
The study was carried out in Italy, a traditional dubbing country, which might refute the
limitation pointed out by Wissmath et al. (2009) concerning the participants’ regular use
of subtitling and dubbing.
55
The Reception of (Non)Professional Subtitling
3.3.2.6. Reception and proficiency in L2
Taylor (2003) reports on an experiment on the reception of subtitles in a non-subtitling
country (Italy). He presented two groups of university students, 50 non-English
speakers and 50 English speakers, with a cartoon excerpt subtitled in two modalities: a
minimum modality (condensed subtitles) and a maximum modality (longer subtitles
with more information). In general, both groups of participants preferred the minimum
version. Both results from interviews and questionnaires showed that “the disturbance
caused by having to concentrate on the maximum titles outweighed the benefits of the
extra information” (Taylor 2003:204). When watching the condensed subtitle version,
the participants were able to access meaning through other semiotic modalities, but this
was not the case with the extended version of the subtitles.
Some reception studies have already tackled the issue of incidental learning of
foreign language with subtitled content (d’Ydewalle and Pavakanun 1995; d’Ydewalle
and Van de Poel 1999; Van de Poel and d’Ydewalle 2001; Bisson et al. 2014). These
studies fall beyond the scope of this research, since language learning or assessment is
not among the main variables of the study. The focus of this study is on testing how
people at different levels of proficiency in L2 use subtitles. This has been precisely the
topic tackled by a series of studies conducted by Lavaur and Bairstow in France (Lavaur
and Nava 2008; Lavaur and Bairstow 2011; Bairstow 2011; Bairstow and Lavaur 2012).
Based on the findings of Lavaur and Nava (2008) that indicate that subtitles could
have a negative effect when they are not necessary for the viewers, Lavaur and Bairstow
carried out two studies using questionnaires to test to what extent knowledge of L2
makes subtitles redundant. Bairstow (2011) reports on an experiment with monolingual
and bilingual participants watching a clip in its original version with English soundtrack
or with French subtitles. The results suggest subtitles have a facilitating effect for
monolinguals but constitute a distracting element for bilinguals. Noticeably, it seems the
subtitled version helped monolinguals to understand visual information, and not only to
access the linguistic information. This coincides with Taylor’s findings for participants
who watched condensed subtitles and were able to better complement their viewing
experience with information from other semiotic channels at the same time (Taylor
2003). Bairstow and Lavaur (2012) argue this could be because the monolinguals in the
groups were perhaps not totally monolinguals, which means their engagement with the
subtitles was different due to their reduced but existing knowledge of the source
language.
56
Chapter 3. Literature review
This motivated a more careful follow-up experiment. Lavaur and Bairstow (2011)
again tested the two conditions (original version and interlingual subtitled version, in
French) and added an intralingual subtitled version (in English). This time the
participants were high-school students arranged into three groups depending on their
proficiency in English: beginners, intermediate and advanced. Their results reinforce the
two roles of subtitles, which act as a distraction from the visual information but help
viewers with low level of language proficiency to recover the linguistic information.
The three groups of participants had very different behaviors. The beginners’ visual
attention dropped with the presence of English subtitles and even more with the French
subtitles. However, the effects of the subtitles, or the lack of them, were completely the
opposite for dialogue comprehension. Intermediate participants processed dialogue
information better than visual information in all three conditions, while advanced
participants obtained best results with the original version.
There are some concerns that could be raised in terms of ecological validity
within what is normally expected in Translation Studies. On the one hand, the
participants in the studies were asked directly to watch the film attentively. On the
other, it seems the subtitles were prepared specifically for the experiment: “we
proceeded to adapt the subtitles to make them as compatible as possible with the spoken
dialogues (while following the recommendations set forth by Carroll and Ivarsson
(1998) regarding oral and written language modes)” (Bairstow and Lavaur 2012:276),
and they could thus differ from professional subtitles.
In a reception study carried out in Finland, Tuominen (2012) used focus groups to
discuss the reception of subtitling. She included three groups: one of near-experts,
participants who were majoring in English or translation, and two of non-experts.
Tuominen (2012) found that the near-expert group actually relied heavily on the
subtitles and was able to recall them during the discussion stage. This shows that the
participants with the strongest abilities in the source language in her study also decided,
probably consciously, to follow the translation. Regardless of the variation in the level
of proficiency in English of the participants, they were able to watch the film
comfortably and enjoyed it. The subtitles did not play a distracting role, as reported by
Bairstow (2011). Tuominen explains that this can be due to the fact that Finnish people
are more used to subtitling, unlike the regular situation in France.
57
The Reception of (Non)Professional Subtitling
3.3.2.7. Eye-tracking, audiovisual translation and triangulation
Some of the above studies have applied eye-tracking methods to assess the reception of
translated audiovisual products from a psycholinguistic or accessibility focus. In this
section I present only eye-tracking studies carried out within Translation Studies that
test some of the assumed rules or the characteristic features of mainstream subtitling.
In an study measuring performance and analyzing eye movements, Perego et al.
(2010) assessed the viewers’ reactions to different types of subtitle segmentation. In
total, 41 university students were included in the study, but eye-tracking data was
collected from 16 of them only. The subtitles for an excerpt of a Hungarian film were
created specifically for the study. The control condition included subtitles that followed
professional standards, while in the treatment condition, the subtitle segmentation was
contrary to those standards. The analysis of the eye-tracking data subset indicates that
the participants spent an average of 67% of the time looking at the subtitle area. The
subtitle area had a higher number of fixations, while the fixations on the image were
fewer but longer. In general, for both conditions, the participants performed well in the
recognition of subtitles and scenes, which indicates that subtitles are cognitively
effective. Subtitled segmentation was not found to have an effect on cognitive
processes, since scores were similar among participants in both conditions.
In a pilot study with four participants, Secară (2011) discusses the use of txt lingo
(the creative spelling used in chats, SMS and electronic communication) in subtitling
for specific environments such as short online videos. The participants reported no
problems reading the subtitles and relating to the viewing experience. Being shorter, the
subtitles using txt lingo allowed viewers to spend more time on the image, according to
the eye-tracking data. However, the small sample size requires that these findings be
treated with caution. Secară notes that this type of subtitling might be suitable for a
certain age group and a specific type of video.
Ghia (2012) compares the reception of subtitles created in accordance with two
translation strategies: literal and non-literal subtitles. The study included thirteen
university students with an intermediate level of English. The subtitles in both
conditions were in Italian and the soundtrack was in English. Gaze data indicate that
non-literal translation caused more deflections on the participants. The higher number of
deflections occurred when reduction was applied to the subtitles. This indicates that
intermediate viewers’ behavior can be affected by different translation strategies and
58
Chapter 3. Literature review
this might also imply the participants were following and comparing the dialogue in L1
with the subtitles in L2.
Another study exploring specific characteristics of subtitling is presented in
Moran (2012). The study includes two features: word frequency and cohesion.
Intuitively, the findings of the study indicate that the presence of high-frequency words
in the subtitles facilitates subtitle reading behavior. The group of participants who
watched the subtitles with the high-frequency words had “significantly lower fixation
durations associated with reading the subtitle, spent significantly more time viewing the
image and scored better in the post-experiment questionnaire” (2012:215). The subtitles
in the high-cohesion condition also prompted better results and allowed the participants
to spend more time on the image, which might be considered another argument against
reduction in subtitling. Both high-frequency and high-cohesion subtitles contained more
characters than their alternative conditions. Moran suggests that instead of focusing on
character count, subtitling should be looking more at reading facilitation.
Exploring a different application of subtitles, Kruger et al. (2013) assessed the
benefits of using intralingual subtitles for educational purposes. Their purpose was to
test the cognitive load of students when they were watching academic lectures with and
without intralingual subtitles. Adopting a highly innovative approach, the authors used a
wide range of data collection methods in their study: eye tracking (pupil dilation), EEG,
self-reported ratings and performance measures. The study shows that subtitles proved
to be beneficial in these circumstances: they help students by facilitating the processing
and comprehension of the lecture. Further, the students who watched the subtitled
version had a lower cognitive load than those who watched the non-subtitled version.
Kruger (2014) found that subtitles are not only cognitively effective, but also have a
positive impact on immersion and enjoyment. The study included 88 university students
with different mother tongues (English, Chinese, Japanese and Korean) who watched an
excerpt with or without intralingual subtitles.
One of the conclusions of Bucaria and Chiaro (2007) was that there are
sociocultural gaps in the encyclopedic knowledge of viewers, and these gaps obscure
comprehension. This made them wonder about the possibility “for screen translation to
be integrated with extra information to make up for possible gaps in the sociocultural
context” (Bucaria and Chiaro 2007:115). At least three independent studies on subtitling
have explored the possibility of including additional text on the screen to provide more
59
The Reception of (Non)Professional Subtitling
information to viewers (Caffrey 2009; Künzli and Ehrensberger-Dow 2011; Ramos
Pinto 2013).
Caffrey (2009) studies the cognitive effort of participants who watch anime
subtitled in English, both with and without additional pop-up glosses, using eyetracking and questionnaires. The pop-up glosses in the experiment are occasionally used
to explain culturally marked elements in the anime. The results suggest that increased
processing effort is required when a pop-up gloss is on screen, which results in less
processing time allocated to the subtitle and a greater number of skipped subtitles. The
study found that participants had a better understanding of culturally marked items
when they watched the videos with pop-up glosses. Nevertheless, the presence of popup glosses on the screen gave the participants in the treatment condition the impression
that the subtitles were too fast. In line with other studies (d’Ydewalle and Gielen 1992;
de Linde and Kay 1999), Caffrey found that when there was more information on the
screen (subtitles plus glosses), the participants read the subtitles faster.
Künzli and Ehrensberger-Dow (2011) study the audience’s response to standard
subtitles and standard subtitles combined with surtitles, subtitles that appear at the top
of the screen and offer metalinguistic information, explaining specific cultural
references in the segments. Their method also combines the use of eye tracking and
questionnaires in order to collect data on the participants’ cognitive effort and
performance. They conclude that the material using surtitles produces a higher cognitive
load, but “participants’ performance in terms of retention of various verbal and visual
elements in the movie excerpts was identical in the two conditions” (2011:197).
Additionally, since the participants’ reception capacity was not affected,2011; Künzli
and Ehrensberger-Dow (2011) argue that viewers are able to process a larger amount of
information than previously conceived without compromising their comprehension or
enjoyment. The authors point out that using surtitles for an entire film could yield
different results, such as fatigue and/or reduced reception capacity. Künzli and
Ehrensberger-Dow also stress that the acceptance of innovative subtitling might be
related to the age and literacy level of the users. However, in an experiment also using
surtitles in Portuguese to explain cultural items in a Finnish film subtitled into
Portuguese, Ramos Pinto (2013) found some of the viewers consciously decided
whether or not to follow the surtitles and thus achieved greater surtitle processing
efficiency.
60
Chapter 3. Literature review
The combination of eye-tracking data with data obtained from other collection
methods is becoming more and more popular in audiovisual translation reception
studies. Eye tracking can provide accurate gaze data information, but says little about
the repercussions of the viewer’s behavior. Interviews and questionnaires provide
abundant self-reported information, however they lack the cognitive dimension of
information offered by gaze data. Following the combination of methods used by,
Caffrey (2009), Künzli and Ehrensberger-Dow (2011) and Ramos Pinto (2013), I rely
on eye-tracking measures and questionnaires plus interviews as the methods to collect
the data required to answer my research questions.
61
Chapter 4. Methods
In this chapter I present the methodological framework of my study. First I introduce
the research questions and hypotheses that guided my research, then I offer a description
of the design of the study and the methods used to collect and analyze the data. In
Section 4.1 I revisit the aims of the study in order to define the methodological
framework, which also includes the research questions and hypotheses, as well as the
operationalization of the main variables. Section 4.2 describes the instruments I used to
carry out both the pilot study and the main experiment. The process of data collection
and preparation for the analysis is also explained in detail here. Section 4.3 is dedicated
to the description of the participants and the sampling procedure, while the subsequent
Section 4.4 introduces the materials I used. I address the ethical considerations of the
study in Section 4.5. A description of the data-collection procedure is introduced in
Section 4.6, followed by Section 4.7, which presents the pilot study carried out in order
to ascertain the suitability of the proposed methodology. I describe the conditions for
the main experiment in Section 4.8, detailing the modifications made to the research
design based on the issues identified in the pilot study. Finally, in section 4.9 I first
present a brief description of mixed-effects modelling to illustrate how the statistical
models function and how I proceed in the data analysis of the data collected in the main
experiment.
4.1.
Methodological framework
This thesis presents the results of an experiment that uses eye-tracking, questionnaires,
and interviews to collect data on participants’ reception of subtitled audiovisual material
when they engage with professional and non-professional subtitling. The Spanishspeaking participants included in the study were grouped according to their listeningcomprehension skills in English. I assume that different levels of proficiency in English
as L2/L3 will generate a difference in behavior in the ways participants engage with the
material and the manifested level of reliance on the subtitles. The fixation data from the
eye tracker is complemented with information from questionnaires and interviews
regarding the participants’ audiovisual habits, their performance in content-recall testing
and their attitude towards subtitling and other audiovisual translation modalities.
63
The Reception of (Non)Professional Subtitling
4.1.1. Aims of the study
The objective of this study is twofold. First, I want to explore the reception of
professional and non-professional subtitling. I want to know if the audience’s reception
of the subtitled audiovisual material is somehow influenced by the type of subtitles used
with the material. Second, I test whether the users’ level of proficiency in the original
language of the product affects the reception of the content. In order to grasp the
viewing habits and processes of new audiences, I analyze the ways in which users with
different levels of proficiency in the source language engage with audiovisual material
subtitled by professionals and non-professionals.
4.1.2. Research questions and initial hypotheses
In this research I explore how the audience engages with professional and nonprofessional subtitles and what their attitude towards this latter instance of volunteer
translation is. In order to do that, I analyze how the audiences’ language proficiency in
the source language of the product correlates with different types of engagement with
the audiovisual content. The main research questions are thus:
-
Does audience reception indicate any difference between the professional and
the non-professional subtitles?
-
Do better reception scores correlate with professional subtitles?
-
Does the users’ level of proficiency in L2/L3 affect the reception of
professionally and non-professionally subtitled audiovisual material?
-
Do participants notice any difference between professional and non-professional
subtitles?
Previous researchers who have compared professional and non-professional
subtitles coincide in that the quality of non-professional subtitles is held to be lower
than the real or expected quality of professional products (La Forgia and Tonin 2009;
Bogucki 2009; Sajna 2013) as presented in Section 3.1) and that non-professionally
produced subtitles “would probably not be considered acceptable in a professional
context” (Wilcock 2013:108).
To test these judgments and considerations, the main initial hypotheses for this
study are as follows:
-
H1: Participants’ comprehension scores will be higher with professional rather
than non-professional subtitles.
64
Chapter 4. Methods
-
H2: Participants with a high level of proficiency in L2/L3 will be less dependent
on subtitles than will participants with a low level of L2/L3.
-
H3: Participants’ reception scores will be higher with the non-professional
subtitles produced in Spain (their own country) than with the non-professional
subtitles produced in Latin America.
4.1.3. Operationalization
The operationalization of a variable consists in reducing it to “abstract terms in such a
way that it really can be tested in practice” (Williams and Chesterman 2002:80). In
order to carry out this research, there are two key concepts that should be clearly
understood and defined: non-professional subtitling and reception. Non-professional
translation is defined in opposition to professional translation and is positioned in the
community translation spectrum. I subscribe to the opposition professional / nonprofessional because I believe it is the one that best describes the current state of affairs:
professional translators identify themselves as such as opposed to unpaid outsiders who
do translations. This offers me the possibility of relying on monetary reward as the
characteristic to differentiate between the two groups. It is true that, as O’Hagan
(2011:13) and others have pointed out, “participants in community translation settings
are not all untrained volunteers; professional translators also respond to a particular call
which they consider worthwhile, despite a lack of remuneration.” Nevertheless, the
resulting product of their activities should still be considered a non-professional
translation due to the conditions in which the professional translators were operating at
the moment of the creation of the product.
Reception is the other key concept that should be operationalized. Using
Gambier’s model (2006; 2009), which distinguishes between three types of reception
(the three Rs), the different types of reception are operationalized in this study as
follows:
-
Reactions at the cognitive level were gathered by collecting gaze data that were
used as indicators of attention allocation.
-
Responses were elicited in terms of the verbal, iconic and narrative attention of
the participants as well as their general understanding of the audiovisual content
and their subjective judgment of difficulty and enjoyment of the videos.
Response data were collected using recall testing and questionnaires.
65
The Reception of (Non)Professional Subtitling
-
Repercussions were measured in terms of the users’ audiovisual consumption
habits and their attitude towards subtitled material; a combination of
questionnaire data and interviews helped assess the repercussions.
The variables for this study are operationalized as follows.
4.1.3.1. Level of L2/L3
The level of L2/L3 English is considered an independent variable in this study. I use the
label L2/L3 when referring to the position English occupies as a foreign language from
the participants’ perspective, since most of the participants are bilinguals who speak
Catalan and Spanish at L1 level. I tested the prospective participants’ listening
comprehension in English prior to the experiment in order to decide who could take part
in the experiment. The study proposes a comparison between the two extremes of a
continuum: thus only participants with a high or a low level of proficiency in English
were considered suitable. As will be explained in Section 4.2.2.1, I used a purposefully
designed listening-comprehension test to divide the participants into three groups (high,
middle and low listening-comprehension proficiency in English) and excluded
participants from the middle group.
4.1.3.2. Type of subtitle
The study compares one professional version of the subtitles with two non-professional
versions. It should be noted that all three types of subtitles in the experiment were
presented as they were produced, with no alteration, revision or modification performed
at any linguistic or technical level. Considering the variation in types of nonprofessional subtitles, I thought it would be more interesting and ecologically valid to
have more than one point of comparison. I included a set of subtitles produced by a nonprofessional community based in Argentina (NP1) and another community based in
Spain (NP2).
4.1.3.3. Attention allocation
Attention allocation refers to the allocation of cognitive resources to the different areas
on the screen, namely the image and the subtitles. It is an indicator of the participants’
distribution of attention. Following Künzli and Ehrensberger-Dow (2011), two relative
measurements are used for this variable: the fixation duration and the number of
fixations.
66
Chapter 4. Methods
Total fixation duration on the subtitle area. The total fixation duration on a given
area is the result of adding up all the fixations that were recorded within that area. This
reports the time that participants spent on the subtitle area while watching the clip. For
instance, the sum of the durations of all 401 fixations on the subtitle area for a
participant in the study resulted in a total duration of fixations on the subtitle area of
75288 milliseconds (ms). Since there are three clips and different subtitled versions of
each clip, it is expected that the Clip variable will have a significant effect on the
Percentage of duration of fixations variable.
Percentage of fixations. In the case of the number of fixations, attention
allocation was calculated as the percent ratio between the total number of fixations on
an area of interest and the total number of fixations in the recording. For example, if a
participant had 350 fixations on the image area and 401 fixations on the subtitle area,
then they would have allocated 53% of the attention to the subtitles and 47% of the
attention to the image. By calculating attention allocation in percentages, the differences
in the number of subtitles and the duration of the videos do not interfere directly with
the measurement.
Percentage of duration of fixations. The duration of the fixations is calculated in
milliseconds. Attention allocation in terms of fixation duration is calculated as the
percent ratio between the total time spent on a given area of interest and the total gaze
time recorded by the eye tracker. For instance, if the total fixation duration on the
subtitle area for a participant is 75288 ms and the total fixation duration on the image
area for the same participant is 119693 ms, then they would have allocated 39% of their
attention to the subtitle area and 61% of their attention to the image area.
4.1.3.4. Attention shift ratio
Attention shifts are the constant changes of focus from the image to the subtitles and
vice versa. They are used as an indicator of the participants’ response to the processing
of the content. As a way to standardize the number of attention shifts, I calculated this
value as the ratio between the number of shifts and the total number of subtitles in the
clip. Attention shifts are related to what de Linde and Kay (1999) call deflections,
although they refer only to “the number of times a viewer’s eyes deflected away from
the subtitle area to focus on the image” (de Linde and Kay 1999:61). Another label used
for attention shifts is back-and-forth shifts (d’Ydewalle and Bruycker 2007), which
includes shifts in both directions.
67
The Reception of (Non)Professional Subtitling
4.1.3.5. Skipped subtitles
The number of unread subtitles was used to calculate the degree to which participants
relied on the subtitles as a source of information. The percentage of skipped subtitles per
clip was calculated by counting the number of unfixated subtitles (subtitles without a
single fixation) and dividing this figure by the total number of subtitles in the clip.
4.1.3.6. Mean fixation duration
The mean fixation duration is computed for each of the two areas of interest: the subtitle
area and the image area. The value is obtained by adding up the duration of all fixations
on the area and dividing the result by the number of fixations on the area. For instance,
drawing on the figures in the previous example, the mean fixation duration for the
subtitle area in the case of this participant would be obtained by dividing 75288 ms by
401 fixations, which would result in a mean fixation duration of 187.75 ms on the
subtitle area.
4.1.3.7. Reception capacity
Reception capacity is measured by a content comprehension test. The number of correct
answers in the test served to assess the performance of the participants. This is
calculated from a set of free and cued-recall questions asking about three sources of
information in the video: verbal, iconic and narrative information (Künzli and
Ehrensberger-Dow 2011) plus a gist comprehension question.
4.1.3.8. Self-reported comprehension
Participants were asked to rate their comprehension of the videos, ranging from 0=no
comprehension to 5=very good, right after they had finished watching each clip. A 6point Likert scale was used in accordance with Caffrey’s (2009) argument, which favors
a Likert scale with an even number of items. Since the scale does not include a neutral
position, it allows to force a choice-decision on the respondents (Trochim 2006) to
determine if they lean more towards declaring they understood the content (values
between 3 and 5) or towards declaring they did not understand it (values between 0 and
2). In the end, this division was not necessary for the mixed-effects models.
4.1.3.9. Audience enjoyment
The exploration of the audience’s enjoyment relied on the subjective judgment of the
participants. They were asked to report their degree of enjoyment of the content. The
68
Chapter 4. Methods
questionnaire included a scaled item to collect their opinions on each video segment.
The video clips were taken from a comedy TV series. Considering that the goal of a
comedy is to entertain the audience, a 4-point scale from 1=boring to 4=great fun was
used to measure audience enjoyment.
4.1.3.10. Subtitle-reading effort
The perceived complexity of the subtitles was measured in terms of the participants’
self-reported difficulty to follow them. A 6-point Likert scale ranging from 0=very
difficult to 5=very easy was used to determine the ease with which participants read the
subtitles. In the phrasing of the question, however, I used the term translation instead of
subtitling in order to prevent the participants from focusing excessively on the subtitles.
4.1.4. Sub-hypotheses
In my main experiment, a group of 52 participants with two levels of proficiency in
English (high and low) watched three clips with three different types of subtitles (one
professional version, one non-professional version produced in Spain and one nonprofessional version produced in Latin America). Data on their reception were gathered
using gaze-fixation data, a questionnaire and interviews.
Given this research design, a set of sub-hypotheses stemming from the set of
initial hypotheses were postulated to aim at more specific goals and results. In these
sub-hypotheses I have tried to abide by the orthodox perspective as far as possible in
order to design the route map for the study and, when possible, I have included the
specific nature of the relation between the variables involved:
-
H1: Type of subtitle:
-
H1.1: Reception capacity is higher with professional subtitles.
-
H1.2: Subtitle-reading effort is lower with professional subtitles.
-
H1.3: Self-reported comprehension is greater with professional subtitles.
Additionally, an explanation for the higher comprehension scores could be
supported with eye-tracking measurements. Assuming professional subtitles are created
following professional standards that are deemed appropriate to less demanding reading
habits, I propose the following set of hypotheses:
69
The Reception of (Non)Professional Subtitling
-
H1.4: With professional subtitles, more attention is allocated to the image
area.
-
H1.5: Mean fixation duration is shorter with professional subtitles.
-
H1.6: Fewer subtitles are skipped when participants are watching
professional subtitles.
-
-
H1.7: Attention shift varies depending on type of subtitle.
H2: Level of L2/L3:
-
H2.8: Participants with a high level of L2/L3 allocate less attention to the
subtitle area.
-
H2.9: Mean fixation duration varies depending on the level of L2/L3.
-
H2.10: Audience enjoyment is higher for participants with a high level of
proficiency in L2/L3.
-
H3: Type of non-professional subtitles:
-
H3.11: Reception capacity is higher with NP2.
-
H3.12: Subtitle-reading effort is lower with NP2.
-
H3.13: Self-reported comprehension is greater for NP2.
Two additional hypotheses are postulated, to be answered based on the qualitative
analysis of the interviews:
-
H14: Participants will be more positively inclined towards the professional rather
than non-professional subtitles.
-
H15: Participants will be more positively inclined towards the Iberian rather than
Latin-American subtitles.
4.2.
Instruments
4.2.1. Mixed-methods approach
The multimodal nature of audiovisual material and the complexity of the reception
process mean that multiple research methods are necessary if one is to approach a robust
understanding of the problem tackled in this thesis. This normally involves a mixedmethods approach. By studying the events through the data provided by different tools,
mixed methods allow us to find common aspects and contradictions in the data, thus
70
Chapter 4. Methods
producing a rich and interesting interpretation of the phenomenon under study. When
using eye-tracking methods to gather information, it is recommended to combine them
with other types of data to increase the validity and reliability of the results. As Soluch
and Tarnowski (2013:91) put it, “it is said that analyses of findings concerning eye
movements as a response to a stimulus situation are inaccurate because one does not
take into account the complexity of human reactions, associations and classification of
the perceived objects”. While this is true, the data provide us with the tools to put our
hypotheses to test.
Mixed-methods research is presented by Johnson et al. (2007) as a type of
research that in itself “combines elements of qualitative and quantitative research
approaches […] for the broad purposes of breadth and depth of understanding and
corroboration” (2007:124). The mixing stage, the specific moment when the
quantitative and qualitative approaches are combined, can occur at any moment during
the research development (at the design stage, data collection stage, the data
examination stage or the interpretation stage), but both approaches should be involved
in the interpretation of the data. Implementing both quantitative and qualitative
approaches provides the necessary tools to offer a more comprehensive view of a
complex phenomenon that is approached for the first time (Johnson et al. 2007).
An approach closely associated with mixed-methods research is pragmatism
because the approach ultimately aims at finding the best answers to the research
questions. By assuming a pragmatic approach, the researcher has the possibility of
giving primary importance to the goals of the research rather than to the methods used.
There is no prior methodological or theoretical commitment, but rather an orientation
towards using the methods that better suit the purpose. A mixed-methods approach
orientates one towards “what works” best for the questions asked when deciding on the
methods to apply for the study (Creswell and Plano Clark 2011:60). Within the mixedmethods research paradigm (Johnson et al. 2007), the qualitative approach allows the
researcher to become acquainted with the experiences and behavior of the participants.
In this study, a qualitative approach was used to collect information on the
participants’ perception and enjoyment of the content. It was important to know what
participants made out of the material they were watching in the experiment and if they
enjoyed what they were watching. Further, by asking the participants about their habits
and behavior, it was possible to contextualize the information collected through
71
The Reception of (Non)Professional Subtitling
quantitative methods. The quantitative approach, on the other hand, provided me with
measurable data that can be analyzed statistically, which reinforces the external validity
of the results and enables between-group comparison. By applying both quantitative and
qualitative methods in a mixed-method research design, the study seeks to report the
subjective experiences of the users. The common points as well as the contradictions
found in the different types of data allow for a richer understanding of the users’
reception of the audiovisual content.
In the search for a more nuanced understanding of findings, triangulation has
become important in Translation Studies (O’Brien 2010; Saldanha and O’Brien 2013).
Triangulation is in line with mixed methods since it implies the combination of different
data collection methods in a project in order to make both the comparison and contrast
of the results possible. Apart from verifying the reliability of the findings, triangulation
can also help identify the cases in which results seem contradictory and might merit an
analysis of a different type. While the validation of results is a good motivation for
using triangulation, the selection of methods should also take into account the scope of
the research and the possibilities of the researchers involved. Collection methods can
produce large amounts of data that will need careful processing to provide reliable
resources. There is always a tradeoff between the aim of the study, the data collection
approach and the feasibility of the project.
The data collection process for my experiment consists of three stages,
corresponding to three data collection methods:
1. Initial survey: This stage includes two questionnaires: an initial questionnaire
aimed at collecting data on the participants’ audiovisual habits and language
knowledge, as well as a listening-comprehension test to categorize the
participants according to their listening-comprehension skills.
2. Eye-tracking sessions: This part of the experiment served to collect gazefixation data and data reporting on the participant’s performance on recall
tests, as well as subjective opinions about the materials.
3. Interviews: The interview session aimed at gaining more insight into the
participants’ audiovisual habits and their opinions about professional and
non-professional subtitling.
72
Chapter 4. Methods
4.2.2. Questionnaires
The experiment required two different sets of questionnaires: the pre-experiment
questionnaires, to gain some knowledge about the population and screen the prospective
participants, and the experiment questionnaires, which included the content recall
testing and subjective ratings.
4.2.2.1. Pre-experiment questionnaires
This stage involved a pre-experiment questionnaire on the participants’ background,
media-consumption habits and linguistic skills, followed by a listening-comprehension
test to select the participants for the next part of the study. This initial stage had a
twofold purpose: to acquire some knowledge about the university students’ audiovisual
consumption habits, and to screen the population in order to select the participants for
the second stage of the project. The data collected with this questionnaire concern
Gambier’s third R: the repercussions of the products at the attitudinal and social levels.
Both the questionnaire and the test were designed to be administered on paper
and in an environment where I could have direct access to groups of participants, such
as in class or in the library at the university. Having direct access to the participants
allowed me to control the number of times participants watched the clip that was used to
test their listening-comprehension proficiency. Furthermore, it allowed me to inform
them verbally about the research, clarify any doubts they had, and in some cases
confirm their willingness and availability to participate in the second stage of the study.
By administering the questionnaires to a group of students at the same time I could
reach a higher number of respondents.
The questionnaire contained nine questions, related to language skills,
audiovisual consumption habits and audiovisual translation preferences (see Appendix
3). The anchors for the Likert scale enquiring about the use of audiovisual translation
modalities were adapted from Cañadas Osinki and Sánchez Bruno (1998). The guiding
idea of this initial questionnaire was to diagnose how the university students engage
with audiovisual content and how they use different audiovisual translation modalities.
Since standardized tests are designed to assess a wide range of skills, I developed
a specific proficiency test to categorize the participants according to their listeningcomprehension skills alone. Considering that, from a linguistic point of view,
understanding a TV series requires mostly listening skills, I designed a listening-
73
The Reception of (Non)Professional Subtitling
comprehension test using an excerpt from the TV series used for the experiment. As
Thomas (1994: 322) explains, this type of test “has the advantages that if all participants
are tested uniformly, proficiency within the sample may at least have internal
consistency and that subgroups may be compared with respect to proficiency on some
rational basis.” Students were asked to watch a 103-second excerpt from the TV series
The Big Bang Theory, in English and without subtitles (due to the large number of
participants expected at this stage and the fact that I was not assessing them in terms of
reception capacity, contrary to what I did in the main experiment, I did not control for
the participants’ previous knowledge of the series). Right after that, they took a
multiple-choice test that included seven questions about the events in the video. Each
question had four options and a fifth “I don’t know” option (see questionnaire in
Appendix 4). Based on the number of correct answers, the students were classified into
three groups: low (0-2 correct answers), mid (3-4 correct answers) and high (5-7 correct
answers). One of the main objectives of the pilot experiment was to validate the
appropriateness of this test as a tool to categorize the participants.
I initially chose three different clips that could be used as part of the listening
comprehension test. I then discussed the nature of the test with two lecturers with
experience in teaching English as a foreign language in the Department of English and
German Studies at the Universitat Rovira i Virgili in order to select which one would
best suit the requirements of the test. After listening to their opinions and
recommendations, I created a set of ten possible listening-comprehension questions for
one of the clips. To make sure the questions were in line with what is required in
listening-comprehension tests for English learners, I asked the lecturers again to give
their opinions and help me select the most appropriate one. The opinions of the experts
and their collaboration in the design of the test helped to confirm its agreement with the
nature of listening-comprehension tests and the assessment method students are familiar
with.
4.2.2.2. Experiment questionnaires
The experiment questionnaire was presented to the participants as part of the on-screen
material. Unlike the pre-experiment questionnaire, the questions were displayed one by
one on the screen and participants answered them verbally. There was no time limit for
answering the questions. Like the subtitles, the questionnaire was written in Spanish.
The complete questionnaire is included in Appendix 5. The questionnaire mixes open-
74
Chapter 4. Methods
ended questions and multiple choice questions, as well as Likert scales. Thus, the
questionnaire combines free recall and cued testing, which aims at obtaining more
reliable results. The questionnaire was divided into four sections: one unique
introductory set of questions, and one set of content and enjoyment questions after each
clip.
After each clip, the participants answered 16 questions about their comprehension
of the content of the clip, in terms of both recall testing and assessment. The first
question asked whether they had seen the episode before (to use this information as a
control variable in the statistical analysis) and then they were asked to narrate what
happened in the video. The next six questions assessed their reception capacity in terms
of three different types of information: narrative, verbal and iconic. This typology of
information was adopted from Künzli and Ehrensberger-Dow (2011). The participants
were offered the answer “I don’t know” in the multiple-choice questions; during the
description of the experiment I told them they could also use this option for the openended questions. Three additional questions asked the participants to assess (on a 6point Likert scale, as suggested by Caffrey (2009) the content of the dialogue and their
degree of enjoyment, as well as the difficulty they had in following the subtitles
(although, as mentioned, the phrasing of the question used the term translation in order
to prevent the participants from focusing excessively on the subtitles). After indicating
each of these scores, they were also asked to give reasons for their ratings.
Different versions of the questionnaires were created and modified in the
preparation stage. I created about ten content questions for each clip and discussed them
with my supervisor as well as with Dr. Sara Ramos Pinto from the University of Leeds.
Their feedback and advice contributed to the design of the initial complete version of
the questionnaire. That version was then discussed with members of the Intercultural
Studies Group and other PhD candidates at the Universitat Rovira i Virgili. I
implemented further modifications to the questionnaires and asked two Spanish friends
who fit the profile to watch the clips and answer the content questions. This last testing
provided me with information related to issues in the phrasing of the questions and gave
me an estimate of the time required to complete the test.
75
The Reception of (Non)Professional Subtitling
4.2.3. Interviews
Interviews have also proved to be useful for reception studies dealing with audiovisual
translation (cf. Fuentes Luque 2003; Tuominen 2012; Ramos Pinto 2013).
Questionnaires are practical data collection tools because the data produced is concise
and can be processed in a fast and structured manner. Nevertheless, questionnaires fall
short when it comes to exploring individual differences and discovering aspects that
were not initially evident or relevant in the eye of the researcher. Since this study aims
at analyzing the reception of subtitled material in a holistic way, it seemed reasonable to
gain more insight into the participants’ thinking and to provide them with the
opportunity to freely voice their opinions regarding their audiovisual consumption and
the various audiovisual translation modalities.
Thus I interviewed the participants immediately after the eye-tracking session.
The semi-structured interviews included questions related to the participants’
audiovisual consumption habits (to obtain information more fine-grained than that
provided by the pre-experiment questionnaire), as well as questions regarding the
participants’ knowledge and assessment of non-professional subtitling. The interviews
lasted between 15 and 25 minutes. This duration allowed me to have a natural exchange
with them and to acquire more detailed information at the individual level.
After collecting the interview data, it was evident that I would have missed a
relevant source of information had I decided to rely on questionnaires only. For
instance, some participants told me that they had grown used to using closed captions in
the absence of regular subtitling. Although they admit there is a delay and the format
could be bothering for them, they say it helps them when the dialogues include
unknown words or when characters are speaking too fast. Closed caption was not
addressed by my research design, but this information can nevertheless complement the
information related to the emergence of new behaviors among viewers.
Another important part of the interviews was the debriefing of the participants
with respect to the whole rationale and aim of the study. At the end of the interview, I
informed them about the specific purpose of the research and offered to answer any
questions they might have. All they knew until then was that the broad purpose of the
study was to explore the reception of popular audiovisual products produced in the
United States and distributed in Spain. To finish, I showed them parts of the eye-
76
Chapter 4. Methods
tracking recordings and offered the possibility of giving them more information or the
data if they were interested.
4.2.4. Eye-tracking data
The type of eye-movement data provided by the eye tracker can be classified into three
categories: fixations, saccades and pupil size. In this study, I rely mainly on fixations as
the source of attentional and cognitive information for the project. Although
pupillometrics has been used in the field of audiovisual translation, I decided against it
mainly due to practical reasons. I considered that the number of participants I wanted to
include and the conditions for the experiment would make it very difficult to also
include pupil dilation as a variable. The pupil is highly sensitive to a broad range of
stimulants, so an additional series of elements need to be controlled in order to use this
data in the analyses. Further, the processing of data produced by the combination of
surveys, interviews and fixations measurements was already a challenging task that
would have been multiplied exponentially with the inclusion of pupillometrics.
Eye-tracking studies are based on the eye-mind hypothesis. It is assumed that
when the eye fixates on a specific point, the brain is engaged in unobservable cognitive
processes related to the fixated area (Just and Carpenter 1980). The general principle
guiding the application of eye-tracking methods to audiovisual translation assumes that
there is a division of the viewer’s attention. The viewer’s fixations on the image area
indicate processing of the visual input that is shown on the screen. On the other hand,
when the fixations are on a different area, such as the subtitle area, this means that the
visual attention is focused on the information provided there.
4.2.5. Eye-tracking system
There are different types of eye trackers available on the market nowadays. They vary
depending on the type of setup desired and the method they use to record the eye
movements. At the Intercultural Studies Group we use a remote video-based eye
tracker, Tobii X120. This type of eye tracker is an independent device (unlike the Tobii
T series models, which are integrated to a computer monitor) that can be connected to
different types of screening devices. Furthermore, the Tobii remote eye tracker does not
require the participant’s head to remain completely still or to be secured with any
additional tool such as a bite bar or a chin rest. Thus, the Tobii X120 helps to create a
77
The Reception of (Non)Professional Subtitling
more naturalistic environment. Nevertheless, the distance from the participant’s head to
the eye tracker should be constant and the participant should remember to focus on the
screen, thus preventing accuracy problems, excessive drift effect or gaze data loss.
The eye tracker was connected to a laptop that complied with Tobii’s
recommendations included in their White Paper (Tobii Technology 2010) and the laptop
was connected to a 23” LCD monitor that acted as presentation screen. The laptop and
the external monitor had a 1920×1080 resolution, which allowed me to activate the Live
Viewer function in Tobii Studio and monitor the recording of the session in real time.
The eye tracker was placed below the screen, a webcam to record the participant’s face
was placed on top of the screen and a stand-alone microphone was also placed on the
desk. The eye tracker and all the other peripheral devices used were controlled using
Tobii Studio.
Figure 1. Layout of the recording area used for the experiment
The layout of the recording area can be seen in Figure 1. Although it is not
always recommended because it could alter the participant’s behavior, I stayed in the
room with the participant during the eye-tracking session. The participant was also
answering the questionnaires during the session, so it could have been strange for them
to be left alone in the room since it might suggest to them that they were being
monitored by someone outside their visual field, which was not the case. At a more
practical level, at the Universitat Rovira i Virgili we do not have an eye-tracking
laboratory where one could leave participants alone with their tasks while the
experiment is monitored. This was a concern at the beginning of my project, so I asked
78
Chapter 4. Methods
the volunteers and participants during the pre-pilot and the pilot sessions if my presence
created any disturbance. Only one of the participants in the pilot experiment mentioned
that the situation was uncomfortable at the beginning but she quickly adapted to it.
4.2.5.1. Apparatus
The experiment took place in the Aula d’Anàlisi de la Parla at the Campus Catalunya of
the Universitat Rovira i Virgili in Tarragona. During the experiment sessions, data was
collected in the form of voice-recordings and eye-tracking data.
Figure 2. Experiment setting
Carrying out all the sessions in the same setting helps to avoid effects that occur
when a remote eye tracker needs to be moved, such as variations in the distance of the
participants to the monitor and in the distance between the monitor and the eye tracker.
Additionally, it makes it easier to control lighting conditions. The participants came to
the laboratory individually and I made sure they were seated approximately 60 cm away
from the eye tracker. I calibrated the eye tracker for each participant before the
experiment: participants focused their gaze on a dot that stopped at nine data points in a
3×3 grid displayed on the screen. Lighting was kept relatively constant by closing the
blinds and turning on the same lights for all sessions. All of the participants wore overear headphones during the eye-tracking session and the volume was maintained at the
same level. The microphone and a recorder were used to record the participants’
79
The Reception of (Non)Professional Subtitling
answers and the interview. The webcam was used to record the participants’ reactions,
although this material was not used in the end.
The eye tracker is located below the screen and a camera inside it gathers the
reflections on the cornea and calculates the position of the gaze on the screen. When
operating at 120Hz, the X120 eye tracker provides fine-grained measurements since it
records eye movement every 8.3 milliseconds. At the same time, working with this
sample rate, the eye tracker offers an average accuracy of 0.5 degrees, which translates
as up to 1 cm of inaccuracy.
4.2.5.2. Input preparation
As explained in Section 4.5, I decided to use six different orders of presentation to allow
for a randomized presentation of the subtitles and to avoid introducing bias. Given the
design, I needed to have control over the order of presentation of the videos and make
sure each order had the required amount of participants. The Tobii Studio software
includes an option for randomizing the presentation of the media, but it follows a preestablished method for the randomization feature. After consulting with the distributor
Alt64/Tobii in Barcelona, I discovered it was impossible to have this type of control
over the sequence of presentation from within Tobii Studio and I had to predefine six
different tests within the project.
Figure 3. Sequence of presentation for the tests
Tobii Studio tests consist of a sequence of media elements structured along a
timeline. The distribution and properties assigned to the elements in the timeline define
the order, timing and duration of the stimulus presentation. The order in which the
80
Chapter 4. Methods
elements are placed in the timeline coincides with the order in which the participants
will watch the stimulus. Figure 3 presents the structure of the tests I used. The tests for
this study comprise a sequence of on-screen questions and video clips. As mentioned
above, the only variation in the tests was the type of subtitle shown each time; the rest
of the questions and their position on the timeline remained the same.
The tests started with three general questions, followed by a synopsis of the TV
show and an image introducing the main characters that would appear in the videos (the
image was included as a result of the pilot test, as described in Section 4.8). Following
the introductory section, the sequence consisted of an alternation of the clips and the
questionnaires. There was no time limit for the participants to answer the questions: I
told them to indicate when they had finished answering each question so I could be sure
and change to the next item in the timeline. At some point during the preparation of the
experiment I considered the possibility of letting participants control their own timing
and use the keyboard, the mouse or a remote to move along the stimulus presentation,
but I considered this might cause other problems, such as participants shifting position
during the recording. When creating the tests in Tobii Studio, the element Movie was
used to add the videos and the element Instructions was used for the on-screen
questions and instructions for the participants. Videos were imported in AVI format and
were prepared to have a 1920×1080 resolution, given that Tobii Studio presents the
elements in actual size, without any resizing or a fit-to-the-screen option. The
appearance of the subtitles on the screen was not altered in any way.
The professional version of the subtitles was extracted directly from the DVD
distributed in Spain by Warner Bros and the non-professional subtitles were embedded
in the video using the default appearance settings of FairUse Wizard 3D R2.
Although Tobii Studio allows for a pre-visualization of the stimuli as a quality
assurance tool, I considered the tests to be too long and included too many elements for
this purpose. In order to avoid any problem going unnoticed, I decided to pilot test it.
Since finding the necessary number of students for the experiment was a very difficult
task, I could not risk losing data because of the experiment equipment. Taking this into
consideration, I tested the entire setup under laboratory conditions with the help of
colleagues and friends acting as participants. Before starting the pilot study and once the
questionnaires were prepared, I asked two members of the Intercultural Studies Group
to come to the laboratory to test the experiment. Considering this was the first time I
81
The Reception of (Non)Professional Subtitling
was using an eye tracker, this test proved very valuable because I was able to correct
problems in the design of the stimulus and become familiar with the functioning of the
apparatus. Once I had altered the experiment based on the comments made by my
colleagues and the notes taken during these tests, I invited a friend to come to the lab
and run a complete test. These testing sections were carried out before the official pilot
study. As pointed out by Holmqvist (2011:114), reiterative pilot studies are common in
eye-tracking experiments for many reasons, ranging from technical issues, such as
making sure all the devices are properly connected, to more personal reasons, such as
reassuring the researcher about the validity of the experiment and also to gain practice
in the handling of participants and the equipment.
4.2.5.3. Recording
Before starting the calibration process, I explained the functioning of the eye tracker to
the participants and offered them the possibility of asking any question they might have
about the experiment or the eye-tracking system. Then I used the Tobii Studio Track
Status tool (see Figure 4) to position the participant in front of the eye tracker. The
Track Status box displays the participant’s eyes as two white dots against a black
background. The eye tracker calculates and displays the distance between the device
and the participant’s eyes. As indicated in Section 4.2.5.1, the eye tracker should stand
at about 60 cm from the participant’s eyes. In the black live viewer box, the reflection of
the eyes of the participant should be centered to allow enough room for possible head
movements. The track status box also proved a very useful tool to show the participants
how the eye tracker identifies their eyes and the range of movement they could have
during the experiment. Since the eye-tracking session with each participant lasted
approximately 20 minutes, it was important to inform the participants that they could
reposition their head and sit comfortably during the recording.
82
Chapter 4. Methods
Figure 4. Tobii Studio Track Status box
4.2.5.4. Data quality
Once eye-tracking data has been collected and before it is analyzed as part of any study,
it is mandatory to run thorough inspection of the quality of the data and determine
whether it is safe to include it in the analyses and rely on the findings that will originate
from its exploration. Eye-tracking data are sensitive to a wide range of factors and even
when all precautions are taken, a certain degree of data loss is expected to occur
(O’Brien 2010). Saldanha and O’Brien (2013:139) suggest including 25% more than the
minimum number of participants required, just to make up for any data losses resulting
from poor-quality recordings. The data-quality process aims at ensuring that a sufficient
amount of data is actually available and in good shape to run the analysis and test the
hypotheses.
As described above, I took different types of precautions to minimize the effect of
the different factors that could affect the data. As part of the quality assurance process,
after the data collection stage was completed, I used three methods to evaluate the
quality of the data: sample rate, fixation measure and gaze time on screen. Tobii’s
sample rate is a measurement that is popular in Translation Studies, but due to the
design of the stimuli in my experiment I could not rely on it as a quality indication.
Using the fixation measure as a data-quality technique proved problematic due to the
nature of the data collected and the lack of literature reporting enough details to
compare the results. In the end, gaze time on screen proved to be the most reliable and
consistent method in my case. The following paragraphs will describe these three
methods and a table presenting the results of applying them all to the collected data per
participant and clip is presented in Appendix 6.
83
The Reception of (Non)Professional Subtitling
Sample rate
The sample rate is a rough metric provided by Tobii Studio as an indication of the
quality of the eye-tracking data in a recording. Tobii Studio assigns a validity code per
eye to each gaze sample it records. When no data are collected at any given moment
(i.e. when the eye tracker does not find one or both eyes), this data point is marked as
invalid. A 100% sample rate in Tobii Studio means both eyes were correctly identified
throughout the recording session, but this is a highly unusual scenario. Accordingly, a
75% sample rate indicates that 25% of the possible data was not recorded: this means
one eye was not found for 25% of the time or that the sum of the moments when the eye
tracker did not find either of the eyes accounts for 25% of the time.
The benefits of this method are mainly related to the immediacy with which this
value is acquired: Tobii Studio calculates the sample rate automatically when the
recording is finished, whereas the other methods to assess the quality of a recording
require the researcher to export the raw data and process it manually. However, this
calculation does not take into account the data loss due to the other experiment
conditions. If the design requires (or allows) the participant to look away from the
screen at any given point, this will be erroneously considered data loss. In my case, for
example, this value was almost irrelevant. As described in Section 4.2.5.1, all of the
elements in the tests were shown one after the other and only one session recording file
was created per participant. Thus, the recording included not just the gaze data from the
videos, but also the data collected during the instructions and the on-screen questions.
Use of a gaze rate based on the data from all these elements was not a reliable
measurement, especially considering that people tend to look away from the screen
when they are thinking or do not know the answer to a question. Since these data were
not going to be part of the analysis, I could have deactivated the eye-tracking option
provided by Tobii Studio while administering the questionnaires. Nevertheless I was
interested in following the participants’ attention during the entire experiment in case I
could use it as an additional tool to corroborate data loss or problems in the other
sections of the recordings. I also wanted to have their faces recorded during the entire
experiment in case this data could be used for further analyses. Considering the amount
of data I collected and the effort it required, it was interesting to consider other ways to
exploit the data in the future.
84
Chapter 4. Methods
Fixation measure
This method relies on mean fixation duration to assess the quality of eye-tracking data.
It is common for the researcher to decide on a minimum mean fixation duration
threshold and use it to assess the quality of the data. Any recording with a mean fixation
duration lower than the threshold is discarded as poor-quality data. Hvelplund (2014)
argues that it is possible to use other studies as a standard to define the threshold. He
refers to the average fixation means presented by Rayner (1998:373) as a point of
reference. These means range from 225 ms for silent reading to 400 ms for typing. The
method is popular in Translation Process Research, but it is not unproblematic. As
pointed out by Hvelplund (2014), it might be misleading because it does not take into
account the potential degrees of incompleteness: it is calculated based on the data that is
successfully collected by the eye tracker, even if it is fractioned and regardless of the
quality of the data collection itself. That is to say, a mean fixation duration could be
deemed appropriate even if calculated with only a minimum fraction of the possible
data.
By itself, this already offers a valid argument for not using this method as a fully
reliable measurement, but in my case I had an additional problem: defining the
thresholds. First of all, to apply this method as a standard for judging the quality of
subtitled-material reception, it is necessary to recognize that watching subtitled material
involves analyzing the visual animated input and the written text. Mean fixations are
highly variable and have proved to be task-dependent: the eye behaves differently, even
when looking at the same object, depending on the task. Based on the existing research
presented in Section 3.3.2, it is safe to assume that viewers’ reading behavior varies and
that reading subtitles and watching the video would produce different types of fixations.
In a study exploring translation for dubbing, Hvelplund (Forthcoming) found very long
fixations during film processing (285 ms) when translators are watching the clip that
will be dubbed, whereas the static source text had much sorter fixations (208 ms).
Nevertheless, these means are not as long as the mean fixation duration for scene
perception (330 ms) reported by Rayner (1998:373). Taking this into consideration, I
opted to calculate two different means (one for each area) and then assess the data based
on those results, since it seemed more appropriate.
Defining the thresholds for these two areas with the existing literature proved to
be an impossible task. Rayner’s thresholds are based on the extensive literature in eye-
85
The Reception of (Non)Professional Subtitling
tracking research. However, research on subtitling using eye-tracking is not yet
abundant and, even among the published research, it is difficult to find enough
information to actually define a grounded and solid standard. For instance, Caffrey
(2009:163) reports a mean fixation duration of 237 ms for one-line subtitles and 201.43
ms for two-line subtitles, while d’Ydewalle and Bruycker (2007) found a mean fixation
duration of 178 ms for standard subtitles among adults. The mean fixations in these two
studies differ greatly and, even more problematic, neither of them includes the value of
the mean fixation duration on the image area. Perego et al. (2010:259) included both
values. Their mean fixation duration for the subtitle area was 221 ms, while they
obtained a mean fixation duration of 422 ms on the image area. Moran (2012) does not
test standard subtitles but uses different experimental conditions in her study: the means
presented range from 507.30 ms to 877.91 ms for the subtitle area and 966.29 ms to
1298.86 ms for the image area. Secară (2011:166) reports the means of the total fixation
time on the image and on the subtitles: a mean total fixation time of 1890.33 ms for the
subtitle area and 409.04 ms for the image area. Künzli and Ehrensberger-Dow (2011)
use a different format to report the mean fixation found in the mainstream subtitle
condition of their experiment (0.36 s), and the difference in format makes it difficult to
compare this mean with other mean fixations, although evidently it can be considered a
very long mean since it should be somewhere between 360 ms and 369 ms. These are
means of the total fixation time per subtitle, so they are not comparable to the mean
fixation duration presented in other publications. As exemplified by this wide range, the
lack of standardization among the studies in the field makes it very difficult to set a
minimum fixation duration to use as a tool to assess eye-tracking data quality.
Before totally ruling out the possibility of using this method, I tried an alternative
modified version that would set the thresholds for excluding participants based on the
collected data itself. I calculated the mean fixation duration for all the participants for
the subtitle area (M=187.91 ms, SD=37.32) and the image area (M=345.45 ms,
SD=93.92). I then flagged the participants falling outside two standard deviations of the
mean. Results showed the means for some of the recordings of the participants P04,
P08, P12, P21 and P46 fall outside the two standard deviations of the mean for all the
recordings. Before deciding to exclude these participants, I noticed all of them were in
the High English Level group and spent less time on the subtitle area than on the image
area. Since this could be an indicator of a different behavior and all of them were above
86
Chapter 4. Methods
the mean plus two standard deviations instead of below the mean, I decided to keep
them for the analysis.
Gaze time on screen
Calculating the exact percentage of gaze data collected by the eye tracker during the
session required post-recording cutting of the data files. This process had to be done
manually and independently of Tobii Studio. The raw gaze data are exported to a
spreadsheet and then processed using different formulas in an Excel spreadsheet. Since
only the gaze data collected for the media elements were pertinent for my research, the
gaze time on screen was calculated for the three clips watched by each participant,
producing three different and independent scores. To obtain the gaze time on screen, the
ratio between the total gaze data duration and the total media element duration is
multiplied by 100. The scores then provide a percentage of the amount of data that was
captured by the eye tracker in contrast to all the data that could have been captured.
Hence, this method avoids the completeness issue resulting from the fixation measures
explained above. One of the problems brought up by this method is its inability to
account for the occasions when the participants are actually forced or allowed to watch
away from the screen. This poses a problem for Translation Process Research, as
indicated by Hvelplund (2014), because translators may be allowed to use print
dictionaries or notes, causing them to focus on an element other than the screen.
However, this does not pose a problem for audiovisual reception studies, since
participants are expected to engage with the content and focus on the screen during the
recording. It does bring about other issues, such as the participants’ responses to the
content. In my case, some of the excerpts amused the participants to the point that their
laughter caused them to close their eyes. Perhaps this should be taken into account for
future audiovisual translation studies involving comedy material and eye tracking.
The decision to include or exclude the participants would again rely on setting a
threshold that would flag the participants with low gaze times on screen, indicating low
quality data. I first considered using the mean for all the participants and recordings,
since it has been done previously (cf. Hvelplund 2011; 2014; Doherty 2012). The mean
for my data was very high at 93 %, with a 6% standard deviation. This was a good sign
for the quality of the data, but applying such a high value as a threshold would leave out
valuable and valid data. Considering that O’Brien (2010:257) notes that thresholds as
low as 70% of the total time of the experiment duration have been regarded as valid for
87
The Reception of (Non)Professional Subtitling
analysis, I decided to adopt an 85% threshold, as used by Caffrey (2009) and applied by
Künzli and Ehrensberger-Dow (2011). Resorting to an arbitrary threshold might seem
questionable. Nevertheless, an 85% data threshold means, for example, that data are
collected from 8 minutes and 30 seconds out of 10 minutes. It could be safe to assume
this percentage of the data reflects the participants’ behavior. Furthermore, using a
threshold that has been used by other studies in the field makes it possible to start
establishing some standards that can easily be followed and will, eventually, bring more
credibility and robustness to the methods we apply.
Once confirmed that this method was the best suited to gauge the eye-tracking
data quality, I used the threshold to see which participants would be discarded. In the
end, all the recordings of participants P18 and P22 as well as the Clip 3 recording of
P03 and the Clip 1 recording of P17 were removed from the dataset. Interestingly
enough, I did not find any overlapping between the participants flagged based on the
fixation measures and the recordings excluded based on the gaze time on screen. This
confirmed the impossibility of combining the two methods, at least until a reliable
golden standard is defined in terms of mean fixations for subtitle reading and video
exploration. One additional participant, P37, was removed from the analysis regardless
of the good quality of his recordings. During the interview session, it became evident
that P37 had found information relevant to the study, probably on my personal webpage
where a summary of the research as well as my previous conference presentations were
available. Facing possible bias due to this additional information, I decided to remove
his recordings to preserve the quality of the data.
4.2.6. Eye-tracking data elicitation
Eye-tracking data, as it has been used specifically in Translation Studies, can be
classified into two categories: attentional data and cognitive data. The eye tracker
output, as raw data, consists of a series of data points recording eye movements every
8.3 ms when the sampling data rate is 120Hz. Each data point consists of a timestamp,
an X and Y coordinate and the pupil size. This means there are 120 data points per
second, which represents an enormous and unwieldy amount of data. The process of
transforming this raw data into visualizable and manageable data requires the
application of a fixation filter. Tobii Studio includes a series of fixation filters that can
identify and filter out fixations from the raw collected data. Detection filters consist of a
88
Chapter 4. Methods
series of transformation steps that are followed to classify eye movements. Different
detection filters use different algorithms, which differ primarily with respect to the type
of eye movement that is of interest. For processing my data, I used the Tobii IV-T
fixation filter (Tobii Technology 2012) provided by Tobii Studio. The most interesting
sub-set of eye movements relevant for my research were fixations, so I used a fixation
classification algorithm rather than a saccade classification algorithm. However, Tobii
IV-T Fixation Filter also offers information about saccades. Tobii IV-T is a velocitybased algorithm that identifies fixations based on the speed of the eyes moving between
different data points. The filter assumes that low velocity among data samples indicates
a fixation and high velocity, a saccade.
Figure 5. Fixation and saccades
Among the different possible classifications of the movements the human eye can
make, the most relevant and widely reported in eye-tracking studies are of two events:
fixations and saccades (Holmqvist 2011). Fixation refers to the periods of time when
the gaze focuses on a specific point and the eye remains relatively stable. As Duchowski
puts it, “[f]ixations are eye movements that stabilize the retina over a stationary object
of interest” (2007:46). The name could be somewhat misleading, because even if the
eye is focusing on a given object, it is never still. Fixations are made of miniature eye
movements, involving microsaccades, drift and tremor. As indicated by the raw eyetracking data, every fixation is actually made of a grouping of data points reporting
89
The Reception of (Non)Professional Subtitling
those micro movements that are later combined into a fixation because the movements
among them are too slow. Saccade refers to the rapid and sudden movements that occur
when the eye is shifting focus from any area to a new one. Duchowski defines them as
“rapid eye movements used in repositioning the fovea to a new location in the visual
environment” (2007:42). It is equivalent to re-adjustment of the visual attention.
Saccades occur in-between fixations and their spans depend on the distance between the
previous and the new fixation. No visual information is acquired by the eye during the
saccadic movements. An example of the visualization of fixations and saccades can be
seen in Figure 5.
To facilitate the analysis of the captured data, Tobii Studio offers the possibility
of drawing areas of interest (AOIs) around the elements that are more relevant to the
researcher depending on the hypotheses that are being tested. In the case of eye-tracking
data from subtitled material, the standard practice is to draw an AOI around the subtitles
(Subtitle AOI) and another area of interest for the rest of the image (Image AOI). Figure
6 shows an example of the AOIs I drew for analyzing my data in Tobii Studio. AOIs
can later be used in the analyses in various ways: fixation data can be filtered based on
the duration and number of fixations enclosed within each area.
Figure 6. Areas of Interest
For analyzing the data in my pilot study, I created interactive AOIs for the
subtitles. The area would activate and deactivate every time a subtitle was shown on the
90
Chapter 4. Methods
screen. This was a very time consuming task because the onset and offset time for each
subtitle had to be set manually. Furthermore, AOIs are test- and media-dependent, that
is to say, if the Clip 1 for Tests 1 and 4 in Tobii Studio were exactly the same, I would
still have to repeat the whole process twice. Apart from the time required to apply this
approach, it could also be argued that latency would then become a problem because
participants could go to the subtitle area to look for information that was no longer
there, but their intention was still to rely on the subtitles. Along the same lines, the
method would not account for the times when participants continued to fixate on the
subtitle area even when the subtitle itself had already disappeared. Kruger and Steyn
(2014:109) comment on the fact that collecting eye movement data “when there are no
subtitles on screen [can] further skew the data”. To assess the impact of this problem, I
decided to run a test with the pilot study data before processing the main experiment
data. I compared the results obtained from using both dynamic AOIs and static AOI for
all the recordings and I found an average 4% difference among all the recordings. Based
on these results, I decided to use only static AOIs for processing the data for the main
study.
Thus, the preparation stage consisted of gauging the quality of the data, applying
the appropriate filters, and defining the AOIs. Once these steps are covered, in most
cases, the data are exported from Tobii Studio to be processed with other type of
software, such as Microsoft Excel or SPSS. Tobii Studio by itself offers some data
analysis tools, however these are highly restrictive and allow little control over the way
the data is handled. I used Tobii Studio mostly for visualizing the data, and I then
combined it with MS Excel and SPSS to explore and summarize the data in different
ways.
4.2.7. Types of eye-tracking data
The previous section presented the type of standard data offered by the eye tracker.
These measurements are used in different ways depending on the nature of the research,
the purpose of the project and the literature in the specific field. In their book on
research methodology in Translation Studies, Saldanha and O’Brien (2013) refer to
three types of data provided by eye-tracking methods that can be analyzed in translation
studies: temporal data, attentional data and data related to cognitive effort. A summary
91
The Reception of (Non)Professional Subtitling
of the types of data and the methods of exploration used in this project is given in Table
1.
Table 1. Type of eye-tracking data and methods of exploration used (adapted from Saldanha and O’Brien
2013)
Type of data
Temporal data
Attentional
data
Data related to
cognitive effort
Methods of exploration
Total fixation duration
Total fixation count
Gaze plots (gaze replays)
Heat maps
Attention shifts
Skipped subtitles
Distribution of the fixations
Fixation duration
Temporal data
In reception studies, the most appropriate analysis of temporal data is what Saldanha
and O’Brien (2013:143) call a “micro-level analysis”. The eye tracker provides
information about how much time in total is spent on each of the areas of interest: the
subtitle and the image. The count and the duration of the fixations on a given area can
be automatically obtained from Tobii Studio or manually calculated in Microsoft Excel
using the data exported from Tobii Studio.
Attentional data
This is the type of data that offers most of the information for my research. Attentional
data result from exploring where the focus of attention of the participants is and how the
participants distribute their attention among the different areas of interest: most fixated
areas, order of the fixation, revisits to a given area, etc. As mentioned above, to test my
hypotheses and answer my research questions, I rely mostly on the fixation data
provided by the eye tracker.
These types of information can be extracted and visualized using different
methods. Tobii Studio allows gaze visualization replay, which provides a visual
rendering of the fixations superimposed on the element. Additionally, I also tested the
software ESE – Eye Scanpath Explorer1, version 4.3 with the data from my pilot study.
A visual inspection can tell the researcher a lot about the viewer’s attention distribution
1
Software developed by Dr. Marco Porta at the University of Pavia. Dr. Porta kindly accepted to
reconfigure it so I could use it for analyzing my data.
92
Chapter 4. Methods
by representing the flow of attention to a specific segment. Figure 7 shows the result of
a 4.6-second gaze plot, equivalent to one two-line subtitle, for P05PS in the pilot study.
It can be seen that fixations are represented as circles and their size varies depending on
their duration (the bigger the circle, the longer the duration of the fixation). The
fixations are numbered but, additionally, ESE uses different colors to indicate their
order. Thus, the brighter green indicates the first fixation and the brighter red, the last
fixation within this segment. Saccades are represented as lines joining one fixation to
the next.
Figure 7. Scanpath as displayed in ESE
Another method used to explore visual attention data is the identification of the
most fixated areas. An extended technique for this in eye-tracking software is the
production of heat maps. Tobii Studio can create heat maps based on the gaze data
collected for each recording. The main advantage of heat maps is that they can
summarize all the fixation data from one or several recordings into one image in an
intuitive way. The visual representation offered by a heat map helps the researcher to
identify areas where most people looked for a longer period of time or areas with the
higher concentrations of fixations. Nevertheless, any qualitative result or conclusion
drawn from heat maps should be taken with caution. In order to provide a broader
picture, heat maps obviate temporal and individual data, making them misleading in
some circumstances. The main application of the heat maps when using subtitled
93
The Reception of (Non)Professional Subtitling
material is the definition of the AOIs that will later be used to analyze the data at
different levels.
The heat map in Figure 8 shows the pattern of colors that are used by Tobii
Studio to create the hot spots. The most fixated areas on the image are represented in
red; the yellow areas represent the less fixated areas; and green indicates the least
fixated regions. The areas that were not fixated at all remain uncolored.
Figure 8. Heat map with colored areas representing the areas most fixated upon
The change of focus of attention is another measure that gives information about
the participants’ behavior when watching subtitled material. Attention shifts indicate
when the attention focus shifts from one AOI to another. Tobii Studio does not include
a tool to report this measurement since the analysis is normally restricted to each AOI
instead of the participants’ interactions between them. ESE software can nevertheless
process this information. It allows the researcher to draw a horizontal line along the
screen and create a report of the number and duration of fixations above and below the
line, as well as the number of times a saccade crosses the line. The analysis provided by
ESE is neat and useful, but the preparation of the material and the extraction of the
results are very time consuming. The researcher needs to input a visual representation of
each subtitle, a file with the time stamp of the subtitles for each recording and a file with
the fixations captured in each recording. For the pilot study, I created the necessary 584
screenshots for the subtitles, one for each; plus nine subtitle time code files and 24
94
Chapter 4. Methods
fixation files, three per participant. The functioning of the program with the data was
satisfactory. Figure 9 shows the summary information provided by ESE. However, the
results for each set of subtitles had to be collected manually in a spreadsheet because the
software lacks an option to automatically export the results. Considering the amount of
time that would have been required to apply the same procedure in the main experiment,
I opted for a different option. With the support of a computer scientist, I calculated the
attention shifts by exporting the data to Excel and then use a Python script that created
an additional column with the shifts between the two AOIs.
Figure 9. Close-up on ESE summary information for subtitle segment
Finally, one type of attentional data that has been commonly used in subtitling
studies involving eye-tracking is the number of skipped subtitles (or actually read
subtitles, depending on the researcher’s approach). Attentional data can help identify
exactly which subtitles were actually read by the participants and which were skipped.
One way of calculating this is by using formulas in Microsoft Excel to analyze the
shifting between AOIs. Nevertheless, it would require the creation of one AOI per
subtitle, which would also require a significant amount of time and effort. Considering
this, another Python script was used to map the participants’ fixations on the subtitles
depending on the time in the recording when the fixations occurred, along with their X
and Y coordinates.
95
The Reception of (Non)Professional Subtitling
Data related to cognitive effort
Some of the measurements provided by eye trackers can be seen as indicators of
cognitive effort. Pupil dilation is also considered a trustable but problematic indicator of
cognitive effort. For this project, I used fixation in three different manners as an
indicator of cognitive effort: mean fixation duration, Percentage of fixations per AOIs
and percentage of duration of fixations per AOIs. As mentioned in Section 4.2.5.4, the
mean fixation duration on the image and the mean fixation duration on the subtitles
vary because they represent two different types of information offered to the viewer and
they are processed in different ways.
Calculating the percentage of time out of the total time that is invested in each
AOI helps identify the areas most used by the participants and can also help reveal the
ways participants behave when watching the subtitles. For example, an exceptionally
long fixation on the subtitle area might indicate higher cognitive effort, since the fixated
element required more time to be processed. For participants who do not rely on
subtitles as their main source of information, an incidental fixation on the subtitle area
might indicate the need to confirm something that escaped the participant’s listening
abilities, something that requires an input channel additional to the main channel being
used, or a strategy to cope with missing vocabulary in the source language.
4.2.7.1. Tobii Studio log files
Although some results can be obtained directly from Tobii Studio, parts of the analysis
were carried out using other programs (Microsoft Excel, SPSS, ESE), particularly the
statistical analysis. To make the files available, I made an inspection of all the
possibilities of exporting files from Tobii Studio. The program offers a pool of different
types of data to be exported and it is necessary to make an adequate selection to ensure
the exported files include all the information needed. At the same time, this process
helps avoid redundant information since some of the types of data are very similar. If
the selection is not done carefully, the researcher could end up with incomplete data or
with unnecessarily large files. The participants’ recordings were exported by clip, so I
had three files for each participant. Data log files are organized by fixations. Each row
in the file represents a fixation identified by Tobii Studio using the IV-T filter. Files
contain 22 columns, including time stamps, X and Y coordinates, validity, AOIs and
register log information. Figure 10 shows a sample of a file exported from Tobii Studio.
96
Chapter 4. Methods
Figure 10. Example of log file exported from Tobii Studio
97
The Reception of (Non)Professional Subtitling
4.2.8. Questionnaire and interview data
Pre-experiment questionnaires were administered on paper to make it possible to collect
data from a large population at the same time. As explained in Section 4.2.1 I needed to
be with the participants while they filled out the questionnaires, which made it
impossible to use online questionnaires and motivated me to look for strategies to apply
the questionnaires to groups of people at the same time instead of individual
respondents. These questionnaires were entered into a spreadsheet and constituted a data
base completely independent from the database containing the information from the
second stage of the research, the eye-tracking experiment.
As indicated above, to prevent participants from shifting position during the eyetracking recording, the questionnaires used during the eye-tracking session were
answered verbally and recorded. The participants’ answers to these questionnaires as
well as the interviews were transcribed in their entirety. I decided to refrain from
making a phonetic or annotated transcription of the recordings because it would have
involved an extensive and probably unnecessary effort, given the type of analysis.
Aiming at fidelity and correctly representing the participants’ opinions, the quotes from
the interviews that are included in this thesis were double-checked before their
inclusion. The interviews with the participants were held in Spanish. Whenever I quote
excerpts from the interviews, I first present them in Spanish and then offer my own
translation into English.
After transcribing them, the responses to the questionnaires were tabulated and
transferred to a spreadsheet. The answers to the content cued and free recall questions
were given a “1” when right and “0” when wrong. The values of the Likert scales were
transferred directly to the spreadsheet. The results of the eye-tracking questionnaires
were tabulated and added to a database containing metadata information from the
participants and the summary of the extracted eye-tracking data per participant. The
resulting database thus included all the necessary variables required to test the
hypothesis and could be imported into SPSS.
The remaining data pertained to the open-ended questions asking for the
participants’ opinions about the dialogues, the difficulty of following the subtitles, and
their enjoyment. These data were kept in a different file, along with the interviews. The
transcriptions were intended to be analyzed qualitatively only and to lead to conclusions
about the participants’ reception in terms of the third R proposed by Gambier (2006):
98
Chapter 4. Methods
the repercussions at the personal and social levels. The transcription of the interviews
and the answers to the open-ended questions were put together in a file. Since the
interviews were semi-structured, I arranged the participants’ interventions according to
five broad categories:
1.
Opinions about different translation modalities
2.
Use and reasons to use non-professional subtitling
3.
Opinions about subtitling
4.
Benefits of using subtitling
5.
Use of audiovisual content
Once the interviews were categorized into these five groups, I re-read them and
selected the more interesting points to help me build the profile of the participants of the
project and define how they engage with audiovisual content and subtitling.
4.3.
Population and sampling
As pointed out by Gambier (2003), viewer preferences respond to different patterns and
there are various elements to be considered when defining the profile of viewers.
Defining the profile of the viewers that would be included in the study was a
fundamental problem. The age group of the participants for this project was defined
based on the key demographic most widely used for the TV shows produced in the
United Stated: prime-time audiovisual products target an audience between 18 and 49
years of age (Storey 2009). Applying only this key demographic to define the
population of the study would be problematic since age would become a prominent
variable to be considered in the analysis. To control this variable and combine the key
demographic information with the most common age group of students at the university,
participants for this study were envisaged to be university students ranging from 18 to
30 years of age. Given that the study explores the reception of audiovisual products
among regular viewers, the profile of the participant is then 18 to 30 year-old university
student who is a regular user of audiovisual material in any form and also uses the
Internet, since they were required to access the video material and the subtitles.
Convenience sampling was used to contact and select the participants. This
means that participants are chosen from a population that is approachable by the
researcher and probably willing to take part. The selection depends on the availability of
prospective participants in the location where the research takes place, and the
99
The Reception of (Non)Professional Subtitling
possibility of contacting the community and finding enough volunteers for the study.
Since the study involved the use of an eye tracker, the data-collection method
influenced greatly the decision to use convenience sampling. It was judged more
practical to request participants to come to a laboratory on the university campus than to
have different data-collection locations or take the eye-tracking equipment to other
places. Moving the eye-tracking equipment would have brought up additional issues to
deal with (Teixeira 2014) such as additional transportation and insurance expenses, time
invested in setting the eye-tracking equipment up, higher risk of data loss, and the
impossibility of maintaining a similar environment for all the sessions.
Participants were contacted during classes at the Faculty of Letters, the Faculty of
Engineering and the Faculty of Law, or in small groups or individually at the library of
the Campus Catalunya at the Universitat Rovira i Virgili. The subtitles for the material
used in the experiment were in Spanish, given the location of the university and the
market distribution policies for TV products that apply to the locale. Distribution
companies, in this case, Warner Bros Spain, rarely include translation into Catalan, so
only Spanish subtitles or dubbing are available. Catalan viewers are used to watching
TV series and films in Catalan and Spanish, as only the regional channels screen
audiovisual programs in Catalan. The participants were native speakers of Catalan
and/or Spanish. The selection process was conducted in two steps due to delays caused
by the university summer break: data from the first group of participants were collected
in May 2013 and from the second, at the beginning of September 2013.
A key issue that had to be addressed in the sampling process was how to test the
participants’ proficiency in English in order to classify them into groups, since that
would help define their suitability for the study. One of the main limitations on this type
of study is that large sample sizes cannot be used in translation-related studies using eye
tracking. The amount of time required to collect and process the data imposes a limit on
the number of participants included in the studies (O’Brien 2010); there are studies with
only five participants, most include between 10 and 25, but there are exceptions, with
studies with up to 88 participants.
The research design included both intra- and inter-subject analyses. This means
each participant would watch each clip under one condition only, so I could measure all
three types of subtitling, but did not watch any clip more than once. This was intended
to help avoid any carry-over effect and collect as much data as possible from the
100
Chapter 4. Methods
participants. Considering the two levels for the Level of L2/L3 independent variable
(HLE=High level of English and LLE=Low Level of English) and the three levels for
the Type of subtitle independent variable (PRO=professional version, NP1=nonprofessional 1 and NP2=non-professional 2), the factorial table has six different
conditions for each clip.
Consultation with a statistician indicated that at least six participants were
required for each condition in order to have statistically reliable results. According to
Saldanha and O’Brien (2013:139) a 25% data loss rate due to dubious quality or lack of
suitability of the data should be a reasonable assumption when defining the number of
participants required for studies using eye tracking. Even though in my pilot study only
the data from one out of the nine participants was eventually discarded, I decided to
follow this recommendation. Each participant would watch one clip under one
condition, without repeating clip or condition. For instance, P01 watched Clip 1 with
the non-professional version 1, Clip 2 with the non-professional version 2 and Clip 3
with the professional version of the subtitles. Thus, the design required at least eight
participants under each condition, a total of 48 participants, in order to obtain enough
good-quality data for at least 36 of them.
Table 2. Factorial table for the main experiment
Clip 1
PRO
NP1
LLE
n=8
n=8
HLE n=8
n=8
Participants: n=48
NP2
n=8
n=8
PRO
n=8
n=8
Clip 2
NP1
n=8
n=8
NP2
n=8
n=8
PRO
n=8
n=8
Clip 3
NP1
n=8
n=8
NP2
n=8
n=8
This project aims to determine how proficiency in the source language affects
viewers’ receptions, making it necessary to include participants with different levels of
proficiency in L2/L3. Taking this restriction and the project’s aim into account, I
decided to exclude from the study participants with a middle level of proficiency in
English and to compare the two extremes: participants with a high level of proficiency
in English and participants with a low level of proficiency in English. Comparing the
two groups, with high and low proficiency level, under the three subtitling conditions
required a minimum of 48 participants. The decision not to include participants with a
middle level of proficiency in English responds to a combination of reasons pertaining
to the feasibility of the project: a third group would require about 24 participants more
than initially planned. Considering that I was using three data collection methods and
101
The Reception of (Non)Professional Subtitling
taking into special consideration the amount of data generated by the eye tracker, this
would mean a significant increase of about three months of additional work in the
selection and engagement of participants and the collection and processing of additional
data.
While the decision to exclude the middle group is efficient for the project and is
ecologically valid, it is not without consequences. The first hypothesis related to the
reception of professional and non-professional subtitling could be directly affected,
since the group in the middle could have a behavior that is different from that of the
participants in the other two groups. On the other hand, this decision was the reason
why the second hypothesis became more relevant after a first analysis of the data: it was
possible to see a clear difference between the two groups and the variations in the
behavior of the participants. The data collection methods used, especially the eye
tracking, produce large amounts of data; tradeoffs of this nature are necessary to narrow
down the focus of the project and maintain its feasibility.
With these considerations in mind, it was necessary to have the appropriate tool
to classify the participants into groups. Given that the regular standardized tests are
designed to assess a wide range of language skills, I decided to develop a special
proficiency test to select the participants according to their listening-comprehension
skills alone and filter only the volunteers that would be suitable for the study. I designed
a listening-comprehension test using an excerpt from the same TV series used for the
experiment, The Big Bang Theory. As Thomas (1994:322) explains, tests designed for
specific purposes have “the advantages that if all participants are tested uniformly,
proficiency within the sample may at least have internal consistency and that subgroups
may be compared with respect to proficiency on some rational basis.” Although
understanding a foreign language involves more skills than listening, it could be argued
that audiovisual products are regularly used to assess listening comprehension. It is
accepted that listening skills are more active when understanding this type of product
and my decision to rely on a test focused exclusively on listening skills to classify the
participants is grounded on that assumption.
4.3.1. Participants
A total of 377 students from the university completed the pre-selection questionnaire
(Section 4.2.1) and took the listening-comprehension test that aimed at screening the
102
Chapter 4. Methods
population. From this sample, only 332 (193 women and 139 men) were considered
suitable for the study, since 45 of the respondents did not meet one or several of the
characteristics defined in the participants’ profile: age, mother tongue(s), audiovisual
product consumption or Internet use.
The listening-comprehension test gave 133 (40%) participants in the High Level
of English group; 142 (43%) in the Middle Level of English group, and 57 (17%) in the
Low Level of English group. Excluding the Middle level, there were 191 prospective
participants available. However, since I could only reach those participants who had
provided their correct contact details, I could only contact 151 participants to invite
them to continue with the study. In the end, 54 students volunteered to participate in the
main study, but only 52 of them were able to complete all the stages of the project. It
was impossible to calibrate the eye tracker for two of the people who volunteered. Table
3 shows the distribution of the participants in the main experiment. The final sample
was configured as follows: the High Level of English group included 21 women and 5
men (ranging from 18 to 30 years, M=20.6, SD=3.29) and the Low Level of English
group was made of 14 women and 12 men (also ranging from 18 to 30 years, M=21.7,
SD=3.23).
Table 3. Participants included in the main experiment
Clip 1
PRO
NP1
LLE
9
8
HLE
9
8
Participants: n=52
NP2
9
9
PRO
9
9
Clip 2
NP1
9
9
NP2
8
8
PRO
8
8
Clip 3
NP1
9
9
NP2
9
9
There is a clear gender imbalance in the High Level of English group.
Nevertheless, this imbalance also reflects the results of the pre-experiment
questionnaire. Men comprised 38.4% of the Low Level of English group, while only
28% of respondents placed in the High Level of English group were men. Furthermore,
among those participants who provided their contact information in the pre-experiment
questionnaire, only one third were men. This imbalance is reflected at the general
student population. According to data provided by the university’s Gender Equality
Observatory, of the students registered for the academic year 2012-2013, 7077 (60%)
were women and 5560 (40%) were men. However, most of my High-level L2/L3
sample was made of students from the BA in English, in which the gender imbalance
was more pronounced: 129 (75%) women and 44 (25%) men.
103
The Reception of (Non)Professional Subtitling
Considering the imbalance between the two groups, I decided it was also
important to test gender as a variable in the statistical analyses. Since the Low Level of
English group does have balance between the two genders, I tested this variable in them
to see if there was any effect.
The success of the study depended greatly on the number of participants that
were willing to volunteer and offer their time to complete the different stages of the
project. With this in mind, it was clear I needed to approach as many students as
possible in the first stage to make sure I would have enough participants for the second
stage. I contacted lecturers at three faculties at the Universitat Rovira i Virgili and they
allowed me to come to their classes, where about 20 minutes were allocated for their
students to complete the questionnaires. When I came to the classroom, first I explained
the broad objective of the project to the participants, without any reference to subtitling,
and then asked them to fill out the pre-experiment questionnaire. Once they all had
finished the questionnaire, I distributed the listening-comprehension test and showed the
video.
I also contacted participants directly at the library in the Campus Catalunya of the
Universitat Rovira i Virgili, where I used a tablet to show them the video. The order of
administration of the tests remained the same: they filled out the pre-experiment
questionnaire first and then took the test.
Subsequently, I processed the questionnaires and tabulated the results. I then
identified prospective participants based on the listening-comprehension test results and
the answers to the questions related to audiovisual material consumption. I contacted
these prospective participants by e-mail or mobile phone (depending on the information
they had provided) and asked if they were willing to continue with the study. I
explained the characteristics of the second stage of the study, detailing the amount of
time they would be asked to dedicate to the session, the voluntary character of their
participation, the conditions under which the experiment would develop and their rights.
To keep track of the appointments and make it easier for the participants to select one, I
created a spreadsheet in Google Docs so participants would be able to check in real time
the time slots that were available for them to come to the laboratory.
104
Chapter 4. Methods
4.4.
Materials
The selection of the material for the study was an essential aspect to be carefully
considered during the design. Depending on the type of material included, the study
could have gone different ways. Initially, one of the options for the experiment was to
use video material in a language that is unfamiliar language to the population (e.g.
Hungarian, Korean or Japanese). Nevertheless, as the decisions were made to shape the
design and narrow down the aim of the study, it became evident that one of the most
important aspects of the project was to test the everyday use that viewers make of nonprofessional subtitling. Spanish and Catalan television channels regularly carry material
produced in English and translated into Spanish or Catalan – the latter in the case of the
regional channels. Although most products are dubbed, the expansion of digital TV and
the accessibility guidelines have open the door for a growing market of subtitled
products. Unlike the case with products in unknown languages, the viewers’ familiarity
with the product and the language might in turn become one of the key aspects that can
provide ecological validity to the experiment and open up interesting research avenues.
On the one hand, non-professional subtitle production depends very much on the
popularity of the TV series, with the subtitles for the most popular TV shows being
produced more quickly because people are more interested in them (Massidda 2012).
On the other, the viewers’ attitudes and the genre also affect the reception of the product
(Gambier 2003:185). These two arguments mean that selecting a popular TV show
should probably give an evaluation that is closer to participants’ everyday life than
selecting material that is totally unknown to the viewer. The participants could relate
more easily to the content and, at the same time, to the presence of non-professional
subtitling.
Further, this also allowed the project to move towards the exploration of variation
between people with different levels of proficiency. At the beginning of the project, I
did not intend to explore this difference in the use of subtitles as a main variable, but it
gained importance as the project progressed. For similar reasons, only one audiovisual
genre (comedy) was included in the study and only one TV show, in order to avoid
adding one additional variable to the study.
Nevertheless, this approach is not without its problems. Using a TV show that is
familiar to the participants as testing material introduces a further factor that has to be
controlled. It is necessary to confirm what the participants’ degree of familiarity with
105
The Reception of (Non)Professional Subtitling
the material is. As will be detailed in the following sub-sessions, this factor was taken
into account at the level of the three clips included in the study as well as the chosen TV
series. After they were shown each clip, the participants taking part in the eye-tracking
session were asked if they had seen the episode before. The answers to this question
were included in the statistical analysis.
4.4.1. Audiovisual material
Three excerpts, of between 3:08 and 3:55 minutes, were selected from the second
season of the popular sitcom The Big Bang Theory (CBS 2007–). The show focuses on
the lives of five characters: Leonard, an experimental physicist; his flatmate Sheldon, a
theoretical physicist; their two equally geeky and socially awkward scientist friends
Howard and Raj; and their neighbor Penny, an aspiring actress who works as a waitress.
The show builds its comedy around the contrast between the geek characters’ intellect
and lack of social skills and Penny’s outgoing personality and low level of education.
Since it was first released in the United States in 2007, the show has proved to be
an international success, as indicated by the number of nominations it has received and
the large audience ratings it has maintained. Its viewership was 8.31 million viewers in
its first season and increased to 20.44 million for its seventh season (2011-2013).
Additionally, the show is also of considerable importance for the non-professional
subtitling phenomenon. Since 2010 it has always appeared consecutively in the top five
of the Most Pirated Shows list collected and published yearly by TorrentFreak. The list
does not take into account direct downloads or streaming distributions, but serves as an
indicator of the popularity of the show among the audience that uses P2P network to
access the content.
The selection of the scenes was guided by the principle of allowing comparison
between the three excerpts. Primary importance was given to the narrative of the scene
and the audiovisual language; thus the three clips correspond to full scenes in the
episode. This is what generated the disparity in the length of the excerpt included in the
study. The selected scenes were self-contained and no additional information about the
TV show or the episode was needed in order to follow the story. The selected scenes
were taken from episodes 4, The Griffin Equivalency; 11, The Bath Item Gift Hypothesis
and 15, The Maternal Capacitance. The three scenes occurred during eating time (lunch
or dinner) and took place in an interior setting where the four main male characters and
106
Chapter 4. Methods
another character participated in a discussion. In each of the excerpts the conversation
revolved about two different topics, with all characters participating in the scene.
Table 4. Duration of clips and number of words in the original dialogue
Clip
Clip 1
Clip 2
Clip 3
Duration
(seconds)
217
188
225
Uttered
words
400
477
517
Words/second
1.84
2.54
2.30
4.4.2. Subtitles
Three sets of subtitles were included in the study: one professional version and two nonprofessional versions. The professional version (PRO) was taken from the DVD
distributed in Spain (Warner Bros 2010). The two non-professional versions were taken
from the two online communities: aRGENTeaM (NP1) and TusSeries.com (NP2). These
two communities were selected because they were both active in 2008 (when the
episodes were released) and their records indicated that the subtitles were produced
internally. Additionally, both communities follow pro-am guidelines, which makes
them comparable to the professional subtitles since they are targeted at similar
audiences. The subtitles are distributed in the form of .srt files instead of embedding
them in the video file. The description of the production processes followed to create
the subtitles as well as the internal structure of these communities fall beyond the scope
of this thesis. A description of the workflow and structure of aRGENTeaM can be found
in Orrego-Carmona (2011) and a comparative account of both communities is included
in Pym et al. (Forthcoming).
Table 5 shows the subtitle density for the three clips and the three types of
subtitles. The subtitles are included in Appendix 2. As expected, there is variation in the
length of the subtitles and the time they are shown on the screen. Since segmentation
and length guidelines might not be the same, this variation is natural in subtitling.
Nevertheless, the numbers are very similar, especially when it comes to the time per
character and the number of characters.
I decided to include two non-professional subtitling versions because frequently
there is considerable variation between non-professional subtitling communities.
Furthermore, Spanish-language non-professional subtitling communities are very
clearly divided between those translating into Iberian Spanish and those translating into
107
The Reception of (Non)Professional Subtitling
Latin-American Spanish. Users can very easily be exposed to both, and this could be a
factor that sheds some light on the popularity or lack of interest in the subtitles.
Since the frame rates of the video versions are different, I decided to use two
different versions of the video instead of altering the time codes. I prepared three
versions of each clip using DVD Decrypter 3.5.4.0, FairUse Wizard 3D R2 and Total
Video Converter 3.71 to extract the subtitles; I cut the video to the correct length and
created the resolution appropriate to fit the screen: 1920×1080.
Table 5. Subtitle density per condition and clip
1693
% Subtitles
on Screen
73%
Clip 1
0.089
11.3
72
30
42
0.58
1721
68%
Clip 2
0.107
9.4
64
23
41
0.64
1591
87%
Clip 3
0.087
11.5
66
18
48
0.73
1768
66%
0.099
10.1
0.58
1790
81%
PRO
NP1
Time/
Characters
Character /Second
0.094
10.7
Subtitles
Oneliners
Twoliners
% TwoLiners
0.65
Characters
Clip 1
0.102
9.8
70
27
43
0.61
1757
79%
Clip 2
0.098
10.2
72
32
40
0.56
1720
87%
Clip 3
0.096
10.4
66
29
37
0.56
1893
78%
0.092
10.9
0.63
1714
75%
Clip 1
0.094
10.6
66
22
44
0.67
1674
73%
Clip 2
0.093
10.8
57
22
35
0.61
1634
80%
Clip 3
0.089
11.2
61
23
38
0.62
1833
73%
NP2
4.5.
Ethical considerations
It is becoming increasingly common in the humanities and social sciences to undergo an
ethical approval process for the research projects involving copyrighted material and
human participants. Considering 1) the nature of the translations under analysis in this
project and 2) the inclusion of volunteer students in the study, the design of the present
study followed the internal guidelines defined by the Intercultural Studies Group. At the
time when these issues emerged for discussion, neither the Department of English and
German Studies at the Universitat Rovira i Virgili nor the university had a regulation for
this type of research project in the humanities. An ethics protocol was approved by the
Department Board in June 2013 and the draft of a Code of Good Practices in Research
was disseminated by the university in October 2013. Both documents postdate the
beginning of my project, but the conditions of the experiment are nevertheless in line
with the indicated regulations.
108
Chapter 4. Methods
The project includes the use of copyrighted material, segments of video with their
accompanying subtitles in Spanish, as well as the non-professional subtitles created for
that content by two different collaborative subtitling communities. The use of
copyrighted material for educational and research purposes is allowed by the Spanish
intellectual property law, Legislative Decree 1/1996, amended by Law 23/2006 and
Law 21/2014, as long as the inclusion of the material is justified and the source of the
content is properly mentioned. Further, for this project, the material is used for analysis
purposes only. The video material used for the analysis is not published here or
distributed by itself nor in its entirety, and only segments of the subtitles are included as
appendices for illustrative purposes.
The project required a significant amount of personal information: the
participants were asked private questions about their eyesight and eye problems; their
voices were audio-recorded; their faces were recorded on video. In conformity with
Intercultural Studies Group practice, I designed a detailed consent form (Appendix 1)
that was presented to the participants. This included the details about the nature of the
data collected (participants’ voice and face recording, eye-tracking data) and the
conditions for the management and storage of records. It also informed the participants
about their right to withdraw from the experiment at any point, during or after the
experiment, and also the appropriate channels of complaint in case they felt their rights
were violated. All this information was also given to the participants verbally prior to
the eye-tracking and interview sessions. To make sure the participants understood the
type of data collected from them, at the end of the interview I showed them a sample of
the recording of their gaze data and their faces and they were properly debriefed about
the specific and detailed nature of the project. They were also offered the opportunity to
ask questions on different occasions during the session and enough time was schedule
between one session and the next in case any of the participants wanted to become more
acquainted with the project or the data. All of them were told the results of the project
would be shared with them once published.
4.6.
Procedure
As mentioned in Section 4.3, the first stage of the study entailed the collection of data
from the university students in order to select the participants for the second stage.
Participants were informed about the broad purpose of the research and were asked to
109
The Reception of (Non)Professional Subtitling
complete the pre-experiment questionnaire. As described above, the pre-experiment
questionnaire aimed at collecting data on the participants’ audiovisual consumption
habits and served to test their listening comprehension in English. The results from
these instruments served to determine the profile of the participants in order to choose
those that could be invited to take part in the second stage of the experiment.
The eye-tracking session and the interview combined lasted between 40 and 60
minutes per subject. Prior to the experiment, I wrote a protocol for the eye-tracking and
interview session so all participants received the same information. During the eyetracking session, the participants watched the three clips, answered the experiment
questionnaire and took part in the interview. Tobii Studio 3.2.1 was used to create the
stimulus with the video excerpts and on-screen questions, collect gaze data and record
the participants’ verbal responses. When the participants arrived, I verbally informed
them regarding their involvement in the study and their rights, and they signed the
appropriate consent form (see Appendix 1). At the beginning of the eye-tracking
session, the participants were asked about their active and passive use of English. After
that, they read a synopsis of the TV series and watched the clips. The sequence in which
the subtitles were presented was randomized so that the subtitles were presented in all
possible positions (initial, medial and final). The randomization design included six
sequences in which the subtitles were presented. The sequences were actually semirandomized because the order of clips was not altered, only that of the subtitles. The
reason for this was twofold: 1) rotating the clips would add complexity to the
experiment design and would require more participants, since there would be 54
possible sequences of administration, and 2) the selected videos had already been
assessed for comparability so that their individual differences would not affect the
analyses.
Table 6 represents the way in which I rotated the presentation sequence of the
subtitles (PRO=professional version, NP1=non-professional version 1 and NP2=nonprofessional version 2). By using this randomization and assigning the same number of
participants to each order of administration, each type of subtitles would be presented
the same number of times in each position, initial, medial and final, and all clips would
have been seen by a uniform number of participants with each type of subtitles.
110
Chapter 4. Methods
Table 6. Presentation sequence used
Order
Test 1
Test 2
Test 3
Test 4
Test 5
Test 6
4.7.
Clip 1
NP1
PRO
NP2
NP1
PRO
NP2
Clip 2
NP2
NP1
PRO
PRO
NP2
NP1
Clip 3
PRO
NP2
NP1
NP2
NP1
PRO
Pilot study2
The pilot study was carried out at the Universitat Rovira i Virgili in December 2012.
The instruments and materials I used for the pilot study were exactly the same as those I
later used for the main experiment. From a methodological point of view, the main
objectives of the pilot experiment were to validate the combination of the data collected
through the questionnaires, the eye tracker and the interviews, as well as the
appropriateness of the listening-comprehension test as a tool to categorize the
participants. From a pragmatic point of view, the pilot study served as a training
experiment for me, with the main aim being to become familiar with the eye tracker and
Tobii Studio software.
4.7.1. Participants
The group of participants for the pilot experiment comprised nine second-year
undergraduate students from the BA program in English at the Universitat Rovira i
Virgili (Spain): eight women and one man, ranging from 20 to 27 years (M=23.1,
SD=3.0). I contacted students by e-mail and 15 of them volunteered to participate in the
experiment. On the basis of the number of correct answers to the questions from the
pre-experiment questionnaire, five students were placed in the low-level of English
group (LLE), six in the mid-level group, and four in the high-level group (HLE).
Participants in the mid-level group were not included in the experiment. All of them
were L1 speakers of Catalan or Spanish and had normal vision or corrected-to-normal
vision (by wearing glasses or contact lenses). The participants were deliberately only
told that the research was related to media reception, in order to make sure their
2
This section replicates Orrego-Carmona (2014b) published in the volume Translation Research Projects
5 (Torres-Simón and Orrego-Carmona eds. 2014)
111
The Reception of (Non)Professional Subtitling
attention to the subtitle area was not mediated by their expectations about the
experiment. At the end of the experiment, I informed them about the specific purpose of
the study.
4.7.2. Procedure
Prior to the experiment, I wrote a protocol for the eye-tracking and interview session.
The experiment took place in the Aula d’Anàlisi de la Parla at the Universitat Rovira i
Virgili during two weeks in December 2012. I arranged individual appointments with
the participants. They selected a suitable time and date for the session and came to the
laboratory individually. Upon arrival, I verbally informed them regarding their
participation in the study and their rights, and they signed the appropriate consent form
(see Appendix 1).
Table 7. Sequence randomization for the pilot experiment
Participant
P1PS
P2PS
P3PS
P4PS
P5PS
P6PS
P7PS
P8PS
P9PS
English Level
Low
High
Low
Low
High
High
Low
Low
High
Sequence
1
2
3
1
5
5
2
4
6
Clip1
NP1
PRO
NP2
NP1
PRO
PRO
PRO
NP1
NP2
Clip2
NP2
NP1
PRO
NP2
NP2
NP2
NP1
PRO
NP1
Clip3
PRO
NP2
NP1
PRO
NP1
NP1
NP2
NP2
PRO
Despite the low number of participants in each group, all six possible sequences were
tested to identify any possible problem in the order of presentation or any problem
emerging from using six different tests in Tobii Studio. I seated the participants in front
of the monitor one by one, and calibrated the eye tracker to their eyes. Once the
calibration process was successfully completed, I started the video playback. I assigned
a sequence to each participant and showed them the three excerpts, alternating the three
conditions, as shown in Table 7. After each clip, they orally answered the on-screen
questions and recorded their answers with a microphone. Immediately after this session,
they took part in the interview.
112
Chapter 4. Methods
4.7.3. Results
Only the results from eight participants are included here. In accordance with Caffrey
(2009), the gaze data threshold was set at 85%. Participant P1PS was excluded from the
analysis because the gaze data collected did not meet the threshold.
4.7.3.1. Eye-tracking data
Initial testing showed the presentation order of the subtitles for this sample is not a
relevant variable for mean fixation duration (F(5,18)=325.46 p=0.545). The following
results thus present statistical analyses regardless of the low number of participants
included in this pilot experiment. Two areas of interest (AOIs) were defined for the
analyses: a rectangle surrounding the subtitles (subtitle area) at the bottom of the screen
and the rest of the screen, for the image area.
4.7.3.2. Fixation duration
The mean fixation duration for each participant, for each condition, was calculated by
dividing the sum of the duration of the fixations by the number of fixations (Table 8).
No significant difference was found between the mean durations of the fixations on the
subtitle area among the participants based on their level of English (F(1,18)=0.013
p=0.909) nor the type of subtitle (F(2,18)=0.867 p=0.437). The same occurred with the
fixations on the image area and the level of English (F(1,18)=0.394 p=0.538), and the
type of subtitle (F(2,18)=0.011 p=0.989).
Table 8. Means fixation (in milliseconds) by area of interest, group and subtitle condition (PRO, NP1 and
NP2)
Subtitle area
Mean
SD
Low Level of English (LLE)
PRO 200.97
15.84
NP1 223.87
10.92
NP2 202.32
19.46
High Level of English (HLE)
PRO 202.36
26.50
NP1 208.79
33.33
NP2 212.83
21.21
Image area
Mean
SD
352.86
369.94
342.31
63.60
55.02
21.77
383.74
371.97
385.31
135.45
118.04
136.73
4.7.3.3. Number of fixations
As shown in Figure 11, in the LLE group, 44% of all fixations are on the image, while
this number rises to 68% in the HLE group. Thus, there is a significant difference
113
The Reception of (Non)Professional Subtitling
(F(1,18)=13.39 p<0.01) in the percentages of the fixations allocated to the image
depending on the level of English, but this percentage did not differ significantly
between the subtitle conditions (F(2,18)=0.137 p=0.873). A similar correlation with
language level was found for the percentage of time allocated to the subtitle area
(F(1,18)=13.39 p<0.01). A paired t-test showed there is a significant difference between
the amount of fixations allocated to each area by the two groups, HLE (t(11)=3.31
p<0.01) and LLE (t(11)=-2.39 p<0.05).
Figure 11. Percentage of fixations on areas of interest by Level of L2/L3 and Type of subtitle
4.7.3.4. Percentage of duration of fixations
The proportion of duration of fixations on each area of interest (Figure 12) shows that
both HLE and LLE fixated more on the image (77% and 56% respectively) than on the
subtitle area (23% and 43%). As with the percentage of the number of fixations, there is
a significant difference in the percentage of duration of fixations on the image
(F(1,18)=10.83 p<0.01) and the subtitle area (F(1,18)=10.83 p<0.01) by Level of L2/L3,
but the type of subtitle does not have a statistically significant effect. Within-group
analyses also confirmed significant differences in the percentage of duration of fixations
on each area for each group, HLE (t(11)=5.26 p<0.001) and LLE (t(11)=2.43 p<0.05).
A visual inspection of Figure 11 and Figure 12, as well as the standard deviations
of mean fixations (Table 8), indicate that the data from HLE subjects have a greater
114
Chapter 4. Methods
dispersion than those from LLE participants, which is consistent with the different
levels of proficiency. LLE participants rely mostly on the subtitles, while HLE
participants might make a conscious decision about reading the subtitles or not. When
comparing the percentage of fixations to the percentage of duration of fixations on the
image and on the subtitle area, on average, participants in the LLE group have more
fixations on the subtitle area than on the image, but their fixations are longer on the
image than on the subtitles.
Figure 12. Percentage of duration of fixations on areas of interest by Level of L2/L3 and Type of subtitle
4.7.3.5. Comprehension questionnaire
Table 9 shows the percentage of correct answers by the participants under the three
conditions. As can be seen in the means, the PRO condition ranks higher than NP1 and
NP2 in both groups, with 76.7% correct answers in the LLE group and 83.3% in the
HLE group. Analysis of the answers shows differences in the scores for the Verbal
Attention questions, where the PRO subtitles correlate with better results than the other
two versions: 70% for PRO condition and 40% and 50% for NP1 and NP2 among the
LLE participants, and 87.5% for the PRO condition and 62.5% for both NP1 and NP2 in
the HLE group. In the Narrative attention and Iconic attention questions, the PRO and
NP1 subtitles give the same results in both groups.
115
The Reception of (Non)Professional Subtitling
Table 9. Percentage of correct questions by type of question and condition
Verbal
attention
Low Level of English
PRO
70.0
NP1
40.0
NP2
50.0
High Level of English
PRO
87.5
NP1
62.5
NP2
62.5
Iconic
attention
Narrative
attention
Mean
(cued recall)
Gist comprehension
(free recall)
80.0
80.0
90.0
80.0
80.0
70.0
76.7 (SD=5.7)
66.7 (SD=23.1)
70.0 (SD=20.0)
100
100
60.0
87.5
87.5
62.5
75.0
75.0
62.5
83.3 (SD=7.2)
75.0 (SD=12.5)
62.5 (SD=0.0)
75.0
100
75.0
As can be seen in Table 10, the mean self-reported comprehension ratings were
positive and very similar: 4.0 for PRO and 3.8 for NP1 and NP2 in the LLE group and
4.2 for NP1 and 4 for both, PRO and NP2 in the HLE group, on a scale from 0=none to
5=very high. These do not reflect the differences found in the analyses of the
comprehension questions.
Ratings of the difficulty of following the subtitles were also highly positive, with
the only rating below 4.0 being 3.5 (where 0=very difficult and 5=very easy) for the
NP1 in the HLE group. This difference is unexpected since participants in the HLE
group were less dependent on the subtitles, but as can be seen in Figures 1 and 2, some
participants in the HLE group did use the subtitles.
Table 10. Self-reported comprehension and Subtitle-reading effort by Type of subtitle
PRO
Self-reported comprehension*
LLE
4.0 (SD=0.71)
HLE
4.0 (SD=0.8)
Subtitle-reading effort**
LLE
4.2 (SD=0.8)
HLE
4.2 (SD=0.5)
* 0=none and 5=very high
** 0=very difficult and 5=very easy
NP1
NP2
3.8 (SD=0.4)
4.2 (SD=0.5)
3.8 (SD=0.4)
4.0 (SD=0.8)
4.6 (SD=0.5)
3.5 (SD=0.6)
4.0 (SD=0.7)
4.2 (SD=0.5)
4.7.4. Discussion
The eye-tracking data show there is significant difference in the behavior of participants
with different Level of L2/L3, with the participants in the LLE group spending around
56% of the time on the image area and participants in the HLE group spending around
77% on the image area. This difference fits in with comments made by the participants
during the interviews, when some mentioned they use the subtitles as a support rather
than as their main source of information. This also corroborates the initial assumption
116
Chapter 4. Methods
that participants engage differently with the content depending on their language skills.
The participants’ gaze behavior seems to be similar under all three subtitle conditions,
which is supported by the fact that they did not mention significant differences between
the subtitles during the interviews.
The results of the comprehension questions and the self-reported comprehension
levels suggest different circumstances. Most of the participants mentioned in the
interviews that, beyond this experiment, they had noticed low quality in nonprofessional subtitling generally available online, which relates to the widespread
assumption of low quality in amateur translations. Nevertheless, when they were asked
to assess the difficulty of following the subtitle versions included in this experiment, the
ratings for all conditions were very similar. Additionally, when asked if they noticed
any differences between the subtitles, they only mentioned subtitle speed and
appearance on the screen as relevant factors. All of the participants said they had good
comprehension of the material and the general comprehension ratings were high for
most of the conditions, but actually their answers to the questionnaire show there are
different levels of comprehension, depending on the type of translation, as seen in Table
10. This result might suggest that the participants’ expectations do not necessarily
correspond to a need to understand the content entirely. The results from the
comprehension questionnaire show that the Verbal Attention scores have an impact on
the overall result. This may be due to the small sample.
4.8.
Lessons learned from the pilot study
As can be seen from the results of the pilot study, the combinations of tools proposed
for the study worked satisfactorily. The analysis of the data suggests the proposed
methodology is suitable for studying the reception of professional and non-professional
subtitling, as well as exploring how people with different Level of L2/L3 listening
comprehension engaged with subtitles.
4.8.1. Revision of the instruments and the methods
The pilot study brought up important issues that were used to fine-tune the experiment
design and procedure. The modifications that stemmed from the pilot study can be
classified into two categories: procedural additions and questionnaire modifications.
117
The Reception of (Non)Professional Subtitling
Although the percentages of data collected were high in general and only one
participant was not considered suitable for the analysis in the pilot study, I noticed that
with some of the participants the eye-tracking data was consistently imprecise and
drifted more than expected for a Tobii X120 eye tracker. Although the data was still
usable since I was focusing on two areas of interest only, I thought this error could be
solved by adding a dummy media as a confirmation of the calibration step. Thus, for the
participants in the main experiment, I included a 40-second clip from the same TV show
and used it to confirm the accuracy of the eye-tracking data before starting the session.
Due to time constraints, I did this process intuitively, so in the cases in which the
precision was not visually satisfactory, I readjusted the eye tracker and asked the
participants to do the calibration process again.
The other addition I made to the research design was the insertion of an image
presenting the characters that were taking part in the clips included in the experiment.
Although The Big Bang Theory is a popular TV series, it became evident after the pilot
study that not all participants knew the names of the characters. Since the name was
used in the questions and some participants indicated this was a problem during the
interviews, I decided to include this additional information to provide the participants
with a safer starting point for the experiment.
Within the questionnaires, I made two changes. One of the verbal attention
questions for the first clip was problematic since it asked the participants about the
name of an object that was actually referred to in two different ways in the dialogues.
This was problematic for the analysis because, depending on how the participants
understood the question, there were two possible right answers. I replaced this question
with another one. The second change had to do with the Likert scale used for reporting
the participants’ opinion about the dialogue. Initially, the scale had “muy aburrido”
(very boring) and “muy inteligente” (very intelligent) at the two extremes. Some of the
participants mentioned this scale sounded unnatural since they did not see boring and
intelligent as opposites. Taking this into consideration, I changed “muy inteligente” for
“muy divertido” (a lot of fun), which is considered an antonym of “very boring” and
made more sense in the scale. The accompanying question in which I asked participants
to give a reason for rating the clip as boring or fun was also changed accordingly.
118
Chapter 4. Methods
4.9.
Statistical analysis framework
Collecting data in naturalistic settings or ecologically valid laboratory conditions means
dealing with a large number of variables that should be controlled or included in the
analysis in order to provide reliable results that can be used to test hypotheses.
Translation Studies scholars have mostly relied on descriptive statistics (typically means
and percentages) and specific techniques of inferential statistics (mainly t-tests and
ANOVA, as I did for the pilot study presented in this dissertation) to explore and test
hypotheses. However, recently other statistical methods that take into consideration the
variation between subsets of data have been judged better suited to process complex
naturalistic-oriented data. Researchers in Translation Studies have advocated the
adoption of mixed-effects models (Balling 2008; Hvelplund 2011; Green et al. 2013;
Teixeira 2014). Being more suitable for naturalistic approaches, these models combine a
variety of models in a unified approach that, among other practical benefits, “includes
accommodation of unbalanced designs and longitudinal (dependent) covariates” (Green
et al. 2013:444). Taking this into consideration, the present study uses descriptive
statistics as well as linear and generalized mixed-effects models to analyze the data.
To provide a more accurate interpretation of the collected data, I used mixedeffects models with repeated measurements. The statistical analysis is carried out using
the functions MIXED and GENLINMIXED in IBM SPSS Statistics version 22. The
label “mixed models” refers to the fact the model integrates random and fixed effects in
the analysis. As explained by Seltman:
[F]ixed effects have levels that are of primary interest and would be used again
if the experiment were repeated. Random effects have levels that are not of
primary interest, but rather are thought of as a random selection from a much
larger set of levels. Subject effects are almost always random effects, while
treatment levels are almost always fixed effects. (2014:358)
Fixed factors (i.e. independent variables) are repeatable and the numbers of their
levels “exhaust the number of all potential fixed factor levels that are of interest in a
given study” (Hvelplund 2011:118). On the other hand, random factors are not
repeatable and do not have a fixed number of levels. Random factors consist of a sample
of a population that is not necessarily known. The participants in this study are included
as random factors in the model. Different fixed factors can be added to the model and,
119
The Reception of (Non)Professional Subtitling
since the model explores whether there is an effect of these variables on the dependent
variable rather than requiring that the variables be controlled, it provides a more
thorough analysis and offers more information for understanding the data.
Mixed-model analysis can include a large number of variables. The way to
construct a model consists of running initial model with the variables that could have an
effect on the dependent variable, then follow a step-wise procedure to reduce the
number of variables (guided by the hypotheses and the researcher’s assumptions),
preserving only those that have a significant effect (Balling 2008; Seltman 2014). The
rationale behind the inclusion of the variables is to test whether they turn out to be
significant predictors of the dependent variable or not. In order to select the model, the
variables that are found to be non-significant can be excluded from the model on
multiple and successive tests as one moves towards the model that best describes the
data. After removing non-significant variables, the model is run again with the fixed
factors and the interactions among the fixed effects that have a significant effect on the
dependent variable.
The process of variable selection should be carried out in a careful and systematic
way. According to Balling (2008), there is ongoing debate about removal of nonsignificant variables. The consensus in exploratory data analysis seems to be to remove
the variables in order to avoid non-significant variables from hindering the effect of
significant variables (or vice versa) and to produce clearer results. However, the
decision to select one model after creating a large number of possible models also
generates theoretical problems and concerns. Following the approach recommended by
Balling (2008) and applied by Hvelplund (2011) and Teixeira (2014), my models
include only the variables that have a significant effect, with some specific exceptions: I
keep the Type of subtitle or Level of L2/L3 variables to illustrate their non-significant
effect.
Like other linear models, linear mixed models assumed a normal distribution of
residuals. Residuals are the difference between the data point and the value estimated by
the model. A residual exploration needs to be included as a validation of the adequacy
of the model. The model residuals can be plotted in a histogram as a way to confirm if
the residuals distribution follows the bell-shape curve with the majority of them
concentrated symmetrically around the mean. If residuals do not follow a normal
120
Chapter 4. Methods
distribution, the analysis could be misleading due to outliers and non-normally
distributed data.
In the case of binomial variables, instead of using linear mixed models I used
generalized linear mixed models. Generalized mixed models are an extension of linear
mixed models that allow non-normal outcomes, including response variables with
binomial distributions. As is the case in linear mixed models, the independent variables
and interactions are added and tested in the variable selection process. The model is
refined until only the variables and interaction with a significant effect on the target are
kept. Once these variables are identified, the final model is constructed.
As can be seen in Section 4.1.4, the specific hypotheses postulated for this study
explore the effect of one independent variable on one dependent variable. Nevertheless,
the exploratory options offered by mixed-effects modelling foster further discussions
and create room for the analysis of more complex relationships among different
variables. Mixed models allow for the exploration of the effects of one or more
independent variables as well as the interactions that occur among two or more of those
variables.
These results of the model determine the significance of the factors in the model,
but do not offer information about how the levels of the factors themselves differ. For
each significant effect or interaction, it is necessary to run more tests to see how the
different levels of the variable interact among them. Once a final model is reached, the
estimated marginal means of the levels of each factor that yield statistically significant
results are calculated and compared to the other levels of the variable. This analysis
produces comparisons that show the differences in the estimated means predicted by the
model for each pair of factor levels. The estimated marginal means obtained with these
analyses are the mean response for the dependent variable, adjusted to the variables
included in the model. Further, the estimated means are used to create boxplots that
offer a graphical representation of the data, including the median and the intervals of
confidence.
121
Chapter 5. Quantitative results
In this chapter I present the results of the statistical analysis of the data, concentrating
only on the quantitative data collected. Section 5.1 uses descriptive statistics to analyze
the data collected in the pre-experiment questionnaire. A general overview of the data
collected in the main experiment is included in Section 5.2. The purpose of Section 5.3
is to present the results of the statistical analysis carried out for each dependent variable.
This section starts with the preparation that was necessary for setting up the mixedeffects models and moves on to present the models created for identifying the
significant effects and interactions of each variable. Finally, Section 5.4 summarizes the
findings from the quantitative data and sets the stage for the discussion to follow in
Chapter 6.
5.1.
Pre-experiment questionnaire data3
The pre-experiment stage was based on a sample of 332 participants (193 women and
139 men) who provided complete and valid answers to the pre-experiment questionnaire
and the listening comprehension tests. These participants had a mean age of 20.68 years
(SD=2.8). The questionnaire aimed at screening the population in terms of age, mother
tongue(s), and listening-comprehension level in English in order to determine whether
they were suitable for the experimental stage. Additionally, the questionnaire collected
data related to their means of access to audiovisual products, their audiovisual product
consumption habits, and their regular Internet use.
5.1.1. Linguistic knowledge
All participants had Spanish or Catalan as their mother tongue. As explained in sections
4.2.2.1 and 4.3.1, at this stage participants were classified into three groups depending
on their results on a listening-comprehension test. From the sample, 57 (17%)
participants were placed in the high level of English (HLE) group, 142 (43%) obtained a
3
This section partially reproduces Orrego-Carmona (2014a).
123
The Reception of (Non)Professional Subtitling
score that put them in the middle level group, and the remaining participants, 133
(40%), were assigned to the low level of English (LLE) group. As discussed in Section
4.3.1, the participants for the second stage of the research were taken from the high and
low level of English groups. In each group, 26 participants took part in the eye-tracking
session and interview.
5.1.2. Audiovisual consumption
This initial questionnaire also collected general information on the participants’ habits
related to the consumption of audiovisual materials. The participants were asked, in
Spanish, the following question: “On average, how many hours per week do you watch
audiovisual material (TV series, movies, documentaries, music videos, video games)?”.
The answers were collected independently for each way of accessing the content:
television, Internet, DVD/Blu-ray and cinema. Table 11 shows the frequencies of the
responses to this question for all participants in the pre-experiment questionnaire.
Television and the Internet were the most popular choices as means to access content,
whereas going to the cinema or using DVDs and Blu-rays were not that popular. While
the majority of participants who said they mostly use television to access content watch
between 1 and 6 hours of television per week, heavy consumers were more inclined
towards the Internet as their means to access content. About 50% of all participants
declared that they watch more than 7 hours of audiovisual content on the Internet.
Table 11. Weekly consumption of audiovisual products in terms of time and means of access (preexperiment questionnaire)
Television
Internet
DVD/Blu-ray
Cinema
Less than 1 hour
1-3 hours
4-6 hours
7-9 hours
16.27% (54)
8.73% (29)
74.70% (248)
62.05% (206)
31.02% (103)
18.98% (63)
21.69% (72)
31.63% (105)
28.61% (95)
23.19% (77)
2.71% (9)
5.72% (19)
15.36% (51)
22.59% (75)
0.30% (1)
0.60% (2)
More than 10
hours
8.73% (29)
26.51% (88)
0.60% (2)
0% (0)
Table 12 shows the results of this question only for the subset of data of the
participants that were included in the experimental part of the study. As can be seen,
television and the Internet remain the main ways of accessing audiovisual content and
the Internet maintains the lead among those participants who watch more than 7 hours
of audiovisual content per week.
124
Chapter 5. Quantitative results
Table 12. Weekly consumption of audiovisual products in terms of time and means of access (main
experiment)
Television
Internet
DVD/Blu-ray
Cinema
Less than 1 hour
26.9% (14)
1.9% (1)
78.8% (41)
67.3% (35)
1-3 hours
25% (13)
26.9% (14)
15.4% (8)
32.7% (17)
4-6 hours
30.8% (16)
25% (13)
5.8% (3)
-
7-9 hours
11.5% (6)
21.2% (11)
-
More than 10 hours
5.8% (3)
25% (13)
-
5.1.3. Audiovisual translation modalities
The questionnaire asked the participants about their consumption of translated
audiovisual material over the prior six months. As shown in Table 13, 66% of the
respondents say they use dubbing very frequently or always, which concords with the
general dominance of this translation mode in Spain. Nevertheless, 33% percent of the
respondents indicated they use subtitling occasionally, while 36% said they use it very
frequently or always. Voice-over is less used, although 40% of the participants said they
use it occasionally (23%), very frequently (17%), or always (1%). Given that voice-over
is characteristic of documentaries and reality shows, this might also offer information
about the type of material being watched. Closed captioning ranked in the last position,
as would be expected since it is primarily targeted to deaf and hard-of-hearing people.
The results of this question for the group of participants in the experimental stage are
shown in Table 14.
Table 13. Use of audiovisual translation modalities for the pre-experiment questionnaire
Voice-over
Dubbing
Subtitling
Closed caption
Never
18.07% (60)
3.92% (13)
5.72% (19)
57.53% (191)
Rarely
40.66% (135)
12.35% (41)
25% (83)
25.60% (85)
Occasionally
23.19% (77)
17.77% (59)
33.13% (110)
12.05% (40)
Very frequently
17.17% (57)
46.08% (153)
30.12% (100)
4.52% (15)
Always
0.90% (3)
19.88% (66)
6.02% (20)
0.30% (1)
Table 14. Use of audiovisual translation modalities for the main experiment
Voice-over
Dubbing
Subtitling
Closed caption
Never
21.2% (11)
5.8% (3)
9.6% (5)
69.2% (36)
Rarely
30.8% (16)
15.4% (8)
13.5% (7)
19.2% (10)
Occasionally
23.1% (12)
17.3% (9)
42.3% (22)
9.6% (5)
Very frequently
25% (13)
53.8% (28)
30.8% (16)
1.9% (1)
Always
0
7.7% (4)
3.8% (2)
0
125
The Reception of (Non)Professional Subtitling
5.1.4. Use of the Internet
Participants were asked about their daily use of the Internet. The use of Internet in this
case refers to all activities that are performed on the net: e-mail, searches, social
networking, watching videos, listening to music, etc. Table 15 shows the answers to this
question for the participants who answered the pre-experiment questionnaire and the
subset of those who were selected for the second stage of the study. The results show
that about 35% of the participants spend between three and four hours using the Internet
on a daily basis, about 21% of them use the Internet for one or two hours a day, and
nearly one fourth access the Internet for five to six hours a day. A reduced group, 15.6%
in the case of the whole sample and 17% of the participants in the experiment stage, are
on the Internet for more than 7 hours per day.
Table 15. Use of the Internet per day for participants in the pre-experiment questionnaire and the main
experiment
Use of Internet per day
Less than 1 hour
1-2 hours
3-4 hours
5-6 hours
More than 7 hours
5.2.
Pre-experiment
3.3% (11)
21.9% (73)
35.2% (117)
23.7% (79)
15.6% (52)
Main experiment
1.9% (1)
21.2% (11)
34.6% (18)
25% (13)
17.3% (9)
Experiment data exploration
As explained in Section 4.2.1, this study was carried out in two stages: a first general
questionnaire about media consumption habits, and a second experimental stage. The
second stage included a subsample of the questionnaire respondents who were invited to
take part in an eye-tracking and interview session. In this section I will describe the
results of the response variables collected during the eye-tracking session. This
information concerns eye-tracking measurements as well as the answers to the
experiment questionnaires, variables related to reception capacity and the participants’
opinion on enjoyment and subtitle-reading effort. In what follows, I will present these
variables filtered according to the levels of L2/L3, types of subtitles, and clips. This
section only includes descriptive statistics; the analyses using mixed-effects modeling
are included in Section 5.3. I provide only a brief description of the variables. The main
effects and interactions in the mixed-effects section will provide more insights into the
data.
126
Chapter 5. Quantitative results
5.2.1. Percentage of fixations
Table 16 shows the means for the percentage of fixations (see Section 4.1.3.3) on the
image and the subtitle areas of interest depending on the level of L2/L3 and types of
subtitles. On average, the participants in the LLE had about 62% of their fixations on
the subtitle area. Among participants in the HLE group, only 44% of their fixations, on
average, were on the subtitle area.
Table 16. Mean percentage of fixations by area of interest, Level of L2/L3 and Type of subtitle
Subtitle area
Mean
SD
Low Level of English (LLE)
NP1
62%
9%
NP2
64%
9%
PRO
61%
10%
High Level of English (HLE)
NP1
43%
22%
NP2
44%
23%
PRO
46%
20%
Image area
Mean
SD
38%
36%
39%
09%
09%
10%
57%
56%
54%
22%
23%
20%
The boxplot in Figure 13 gives a graphic representation of the percentages of
fixations on both areas of interest for the participants in the LLE and the HLE groups
per Type of subtitle. It can be seen that the data points for the participants in the HLE
group are more disperse, indicating greater variation in the percentages (this is also
indicated by the higher standard deviation values in Table 16). On the other hand, apart
from some outliers, the percentages for LLE participants are less disperse. This
representation hints at differences in behavior within the two groups: while the behavior
of participants in the LLE group is more consistent, there is more variation in the
behavior of participants in the HLE group since some of them obtain the verbal
information from the audio and do not follow the subtitles thoroughly.
127
The Reception of (Non)Professional Subtitling
Figure 13. Percentage of fixations by area of interest, Level of L2/L3 and Type of subtitle
5.2.2. Percentage of duration of fixations
The Percentage of duration of fixations refers to the percentage of the Total fixation
duration that the participants spend on a given area of interest. The mean values for the
percentages of each group and types of subtitles are shown in Table 17 and the
graphical representation of the data is in Figure 14. LLE participants spent about half of
the time looking at the subtitle area, while in the case of the participants in the HLE
group this value accounts for only 32% of the time.
Table 17. Mean percentage of duration of fixations by area of interest, Level of L2/L3 and Type of subtitle
Subtitle area
Mean
SD
Low Level of English (LLE)
NP1
51%
10%
NP2
54%
12%
PRO
51%
12%
High Level of English (HLE)
NP1
31%
20%
NP2
32%
19%
PRO
33%
18%
Image area
Mean
SD
49%
46%
49%
10%
12%
12%
69%
68%
67%
20%
19%
18%
Once again, the values for the percentage of duration of fixation seem to suggest
differences in the behavior of the participants. The same pattern observed for the
percentage of fixations can be observed in the percentage of duration of fixations: a high
128
Chapter 5. Quantitative results
degree of dispersion in the HLE group and a tighter grouping of the data of the LLE
group.
Figure 14. Percentage of duration of fixation by area of interest, Level of L2/L3 and Type of subtitle
5.2.3. Mean fixation duration
Mean fixation duration is commonly used in eye-tracking studies to index cognitive
effort: longer and more fixations are considered evidence of more effortful cognitive
processing. Although widely accepted, especially in research related to reading studies,
this does not mean that it should be taken altogether as a rule. Fixations have also been
proven to be task-dependent. As explained in 4.2.7, the nature of the activity (e.g.
reading, image perception, visual search) affects the duration of fixations.
Mean fixation durations were calculated independently for the subtitle area and
the image area that were drawn onto the screen. Table 18 shows the results of this
computation per area of interest as a function of Type of subtitle and Level of L2/L3. In
all cases, mean fixation durations for the subtitle and the image areas in the PRO
version of the clips are shorter than the mean fixation durations on the non-professional
versions.
The graphic representation (Figure 15) for Mean fixation duration also shows
more dispersion in the HLE group. Interestingly, the effect seems to concern the
fixation on the images more than the fixations on the subtitles.
129
The Reception of (Non)Professional Subtitling
Table 18. Mean fixation duration by area of interest, Level of L2/L3 and Type of subtitle
Subtitle area
Mean fixation
SD
Low Level of English (LLE)
NP1
191.09
32.40
NP2
209.32
60.82
PRO
176.04
32.76
High Level of English (HLE)
NP1
192.22
21.80
NP2
195.01
24.43
PRO
171.01
22.28
Image area
Mean fixation
SD
405.74
391.18
383.96
124.39
90.01
104.13
306.71
299.68
282.79
45.66
43.67
47.72
Figure 15. Mean fixation duration by area of interest, Level of L2/L3 and Type of subtitle
5.2.4. Reception capacity
Reception capacity was measured using a comprehension questionnaire. The questions
asked the participants about the verbal, narrative and iconic information in the clips
(two questions per each type of attention). There is one additional free-recall question
related to gist comprehension. In Section 5.3, the three types of attention are tested
independently using generalized mixed models with binomial distribution, while
reception capacity, as an overall variable, is tested using a linear mixed model.
As a way to bring together the overall results of these variables, Table 19 presents
the percentage of correct answers for each type of attention as well as the percentage of
correct answers for reception capacity. In general, the results obtained by the
participants with the PRO and the NP1 versions are very similar, with the results for
PRO surpassing those of NP1 only in the verbal attention questions. The NP2 version,
130
Chapter 5. Quantitative results
however, has lower results than the other two in all categories. It should be recalled here
that NP1 was produced by a Latin-American non-professional subtitling group that aims
at creating subtitles in neutral Spanish, while NP2 was made by a Spanish group
translating into Iberian Spanish.
Table 19. Percentage of correct answers for different types of attention and Reception capacity
Gist
comprehension
Low Level of English (LLE)
NP1
81%
NP2
73%
PRO
81%
High Level of English (HLE)
NP1
96%
NP2
92%
PRO
88%
Verbal
attention
Iconic
attention
Narrative
attention
Reception
capacity
87%
69%
88%
75%
71%
71%
77%
65%
75%
80%
69%
79%
85%
77%
92%
75%
75%
75%
79%
73%
77%
82%
77%
82%
5.2.5. Self-reported comprehension
Table 20 presents the results from the self-reported question that asked participants
about their overall comprehension of the clips. The participants were asked to rate their
own comprehension of each clip on a scale ranging from 0=no comprehension to 5=very
good. The total average self-reported comprehension ratings per types of subtitles were
positive and very similar: 3.69 for PRO, 3.42 for NP1 and 3.73 for NP2 in the LLE
group, and 3.96 for PRO, 3.77 for NP1 and 3.73 for NP2 in the HLE group. It should be
reminded that Antonini (2008) reports that participants tend to overestimate their
comprehension of translated audiovisual content. Results are also presented per clip
because, as will be shown in the statistical analysis per variable, the Clip variable was
found to have a significant effect on some dependent variables.
Table 20. Self-reported comprehension per Clip, Level of L2/L3 and Type of subtitle
Clip 1
Clip 2
Mean
SD
Mean
Low Level of English (LLE)
NP1
3.25
1.04
3.67
NP2
3.56
0.73
3.63
PRO
3.78
0.67
3.56
High Level of English (HLE)
NP1
3.63
0.52
3.56
NP2
3.33
0.50
3.63
PRO
4.11
0.78
3.89
* 0=no comprehension to 5=very high
SD
Mean
Clip 3
SD
Total average
Mean
SD
0.71
0.74
0.73
3.33
4.00
3.75
1.00
0.71
1.16
3.42
3.73
3.69
0.90
0.72
0.84
0.73
1.06
0.78
4.11
4.22
3.88
0.93
0.67
0.64
3.77
3.73
3.96
0.76
0.83
0.72
131
The Reception of (Non)Professional Subtitling
5.2.6. Subtitle-reading effort
The participants were also asked to give their own opinion about the difficulty of
following the subtitles while watching the clips. They were asked to choose a value on a
six-point scale from 0=very difficult to 5=very easy. Total average ratings were also
positive and similar in general, the lowest in the HLE group was 3.44 for the NP2, but
these subtitles ranked the highest in the LLE group, with 3.69. On the other hand, the
highest rated subtitles by the HLE group were the PRO, with 3.76, although these were
also the lowest rated by the LLE group (3.5).
Table 21. Subtitle-reading effort per Clip, Level of L2/L3 and Type of subtitle
Clip 1
Mean
SD
Low Level of English (LLE)
NP1
3.75
0.89
NP2
3.44
1.13
PRO
3.78
0.67
High Level of English (HLE)
NP1
3.50
1.07
NP2
3.22
0.97
PRO
4.44
0.73
* 0=very difficult and 5=very easy
Mean
Clip 2
SD
Mean
Clip 3
SD
Total average
Mean
SD
3.56
3.50
3.67
1.01
0.93
0.87
3.44
4.11
3.00
1.01
0.78
1.20
3.58
3.69
3.50
0.95
0.97
0.95
3.44
3.43
3.33
0.88
1.27
1.41
4.25
3.67
3.50
0.89
1.00
0.76
3.72
3.44
3.77
0.98
1.04
1.11
5.2.7. Audience enjoyment
After answering the comprehension questions, the participants were asked to assess
their enjoyment of each clip. They gave a score from 1 to 4, where 1=boring and
4=great fun. The average for the results per clip and the total averages are presented in
Table 22. A pattern similar to the previous variables also appears here: there is little
variation in the values, although the overall values for NP2 are the smallest in both
groups: 2.54 for the LLE group and 2.77 for the HLE group. The PRO version scores
the highest for HLE, 3.08, while NP1 has the highest score in the LLE group, of 2.58.
Table 22. Audience enjoyment per Clip, Level of L2/L3 and Type of subtitle
Clip 1
Mean
SD
Low Level of English (LLE)
NP1
2.63
0.92
NP2
2.11
0.78
PRO
2.11
0.60
High Level of English (HLE)
NP1
2.75
0.46
NP2
2.44
0.53
PRO
3.00
0.50
* 1=boring to 4=great fun
132
Mean
Clip 2
SD
Mean
Clip 3
SD
Total average
Mean
SD
2.33
2.38
2.11
1.00
0.92
0.78
2.78
3.11
2.75
0.83
0.78
0.89
2.58
2.54
2.31
0.90
0.90
0.79
2.67
3.00
2.89
0.50
0.53
0.78
3.11
2.89
3.38
0.93
0.78
0.52
2.85
2.77
3.08
0.67
0.65
0.63
Chapter 5. Quantitative results
5.3.
Quantitative analysis per variable
The following subsections present the models constructed for each dependent variable
in the study. Unless indicated otherwise, during the initial stages different models with
different combination of variables were tested in a stepwise manner in order to remove
non-significant variables and interactions and to refine the model.
As described above, the process of constructing the models consists of
introducing the variables into an initial model and reducing them in a systematic and
stepwise process. This process results in a final model that provides the tools to test the
hypotheses that were proposed for this dissertation. Section 5.3.1 details the
transformations that were necessary to apply two explanatory variables (Subtitlereading effort and Use of subtitling) prior to building the final models.
Given the large number of variables included in this study, at the first stage of
each test it was deemed appropriate to have multiple runs with slightly different models
combining different sets of variables and interactions. The initial models always
included the variables that were central to the investigation (Level of L2/L3, Type of
subtitle) and special attention was given to the variables Clip, Prior knowledge of the
clips, Order of presentation and Gender and their interactions. Table 23 shows all the
variables included in the study.
The exploration of interactions was equally important, since a variable may
perhaps not have a main effect and still participate in an interaction with a significant
effect. In summary, the final model includes those variables and interactions that were
found to be significant after the testing stage. Thus, if a variable is not included in the
final model, that means it turned out to produce non-significant results. However, in
cases in which the variable under investigation is non-significant but its inclusion in the
model does not obscure the effects of other significant variables, the variable under
investigation is kept in the final model.
133
The Reception of (Non)Professional Subtitling
Pre-experiment
Control
Recall
Selfassessment
Eye-tracking measurements
Table 23. List of variables with codes and descriptions
134
Variable
Level of L2/L3
Code
L2L3Level
Explanation/Questions
Proficiency in English
Levels/measurement
High/ Low
Type of subtitle
Subs
Subtitle version
PRO/NP1/NP2
Total fixation duration on
the subtitle area
SubFixDur
Total duration of fixations on the
subtitle area
Milliseconds
Percentage of fixations on
the subtitle area
SubFixCP
Fixations on the subtitle area/total
fixations
Percentage
Percentage of duration of
fixations on the subtitle area
SubFixDurP
Duration of fixations on the
subtitle area/total fixation duration
Percentage
Mean fixation duration on
the subtitle
SubMeanFix
Mean fixation on the subtitle area
Milliseconds
Percentage of fixations on
the Image area
ImgFixCP
Fixations on the image area/total
fixations
Percentage
Percentage of duration of
fixations on the subtitle area
ImgFixDurP
Duration of fixations on the image
area/total fixation duration
Percentage
Mean fixation duration on
the image
ImgMeanFix
Mean fixation on the image area
Milliseconds
Skipped subtitles
PSkipSubs
Unfixated subtitles/number of
subtitles per clip
Percentage
Attention shift ratio
Shifts
Number of shifts/total number of
subtitles per clip
Ratio
Subtitle-reading effort
Trans_Read
_Effort
Participants’ ratings for difficulty
to follow the subtitles.
6 levels: 0=very
difficult/5=very easy
Self-reported
comprehension
SRCompre
Self-reported comprehension
6 levels:
0=none/5=very good
Audience enjoyment
Enjoyment
Participants’ ratings for their
amusement with each clip
4 levels:
1=boring/4=great fun
Reception capacity
AllAtten
Results from comprehension
testing: seven questions
8 levels
Narrative attention
NarrAtte
Narrative attention testing
Iconic attention
IconAtte
Iconic attention testing
Verbal attention
VerbAtte
Verbal attention testing
Clip
Clip
Excerpt from the TV series and
position in the input
3 levels
Order of presentation
TestOrder
Order of administration of the
subtitle conditions
6 levels
Prior knowledge of the clips
Watched
Knowledge of the specific
segment
Yes/No
Gender
Gender
Use of Television
Television
Use of television
Use of Internet
Internet
Use of Internet to access
audiovisual material
Use of DVD
DVD
Use of DVD/Blu-Ray
Use of Cinema
Cinema
Use of cinema
Use of voiceover
Voiceover
Use of voiceover
Use of dubbing
Dubbing
Use of dubbing
Use of subtitling
Subtitling
Use of subtitling
Use of closed caption
Ccaption
Use of closed caption
Daily use of Internet
InternetD
Use of Internet per day
3 levels (2 questions
each)
Woman/Man
Less than 1 hour
1-3 hours
4-6 hours
7-9 hours
More than 10 hours
Never
Rarely
Occasionally
Very frequently/
Always
Less than 1 hour
1-2 hours
3-4 hours
5-6 hours
More than 7 hours
Chapter 5. Quantitative results
5.3.1. Preparation of variables for statistical analysis
The different trials with the models before reaching the final adjusted model are helpful
because they allow the researcher to become familiar with the variables and to produce
more educated guesses about the possible main effects and interactions that could be
found in the collected data. The process of variable selection helps define the general
panorama of the data.
After some initial trials with the model, it was possible to posit that the variables
Subtitle-reading effort and Use of subtitling could have significant effects on some of
the dependent variables, however their uneven initial distribution (with one level with
very few observations in each case) was posing problems and causing the models to be
unstable. Once the problem was detected, I considered two ways of dealing with it: the
data points could be removed from the data set, or the variables could be analyzed and
transformed so that the levels with few data points would not affect the model. In the
following paragraphs I explain how the variables were transformed before constructing
the statistical models.
5.3.1.1. Use of subtitling
The variable Use of subtitling reports on the participants’ consumption of subtitles
material over the six months prior to the experiment. The participants were offered five
options to answer, ranging from Never to Always. The category Always was selected by
two participants only (3.8% of the entire sample) and was very small in comparison to
the other categories. In the preliminary tests, the results from these participants did not
differ significantly from those obtained by participants in the immediately previous
category, Very frequently. Hence, after discussing the possibilities with the statistics
consultant, he suggested transforming the variable and combining the two levels: Very
frequently and Always. Figure 16 shows the initial distribution of the variable, with five
groups, and the resulting transformed variable, with four groups.
135
The Reception of (Non)Professional Subtitling
Figure 16. Use of subtitling, original and transformed
5.3.1.2. Subtitle-reading effort
As in the previous case, the solution for Subtitle-reading effort was to regroup the
values by adding the smallest category to the closest neighboring category. The values
for Subtitle-reading effort were obtained using a scale ranging from 0=very difficult to
5=very easy to follow. In the case of Subtitle-reading effort, the level that was causing
disturbance in the model was level 1. Nobody in the experiment selected zero as an
answer to this question and only one participant (1.9%) selected 1. Again, there were no
significant differences between this participant’s answers and those obtained by the
participants who gave 2 as an answer to the question. Thus, the variable was
transformed and the value 1 was changed to 2, creating a new category. The initial
distribution of the variable as well as the transformed variable, with four levels, are
shown in Figure 17.
136
Chapter 5. Quantitative results
Figure 17. Subtitle-reading effort, original and transformed
5.3.2. Attention allocation
This section will report on the analyses carried out to explore the variables related to
attention allocation. Attention allocation was measured using eye tracking. This variable
is actually composed of different variables reporting the results of different eye-tracking
measurements. First I will present the model used for exploring Total fixation duration
on the subtitle area. Then I will deal with Percentage of fixations on the subtitle area.
Lastly I will illustrate how Percentage of fixations on the subtitle area is correlated to
the remaining attention allocation variables, namely Percentage of duration of fixations
on the subtitle area and Percentage of fixations on the image area. Since these variables
are comparable, the adjusted model for Percentage of fixations on the subtitle area is
also fit for analyzing the other variables and produces similar results. Taking this into
consideration, only the results of Percentage of fixations on the subtitle area are
explained in detail here.
For analyzing cognitive effort, the fixations on the subtitles are explored under
the assumption that “the longer the fixation time […], the greater the cognitive effort”
(Saldanha and O’Brien 2013:144). This assumption does not apply for fixations on the
image area, since eye movements differ depending on the type of activity people
perform (reading text vs. watching a moving picture).
137
The Reception of (Non)Professional Subtitling
5.3.2.1. Total fixation duration on the subtitle area
The Total fixation duration reports the total time during the recording that participants
spent looking at the subtitle area. Different initial models were tested with variations in
the combinations of the variables. The models included Total fixation duration on the
subtitle area as target, and the control variables for Prior knowledge of the clips, Order
of presentation and Gender, as well as the Clip variable, were included as independent
variables. Each time, these variables were combined with others to test the model and
start removing those that did not have a significant effect on the dependent variable, in
this case, Total fixation duration on the subtitle area. Given the different lengths of the
clips, it was expected that this variable would have a significant effect on Total fixation
duration on the subtitle area. Nevertheless, in the process of adjusting the model, this
variable was not found to have a significant effect. Clip is thus not included in the final
model because, although it is a non-significant variable, when it was included in the
model it obscured the effects of the significant variables. This case is commented by
Balling (2008:186) in her discussion of mixed-effects modelling and she recommends
constructing the model with the significant variables only. Unless otherwise explained,
this rationale was used for constructing all the models included in this dissertation. The
main effect of Use of subtitling (F=7.81; p<0.001) and Level of L2/L3 (F=13.09;
p<0.0001) are significant, as is the interaction between these two variables (F=5.60;
p<0.01).
Use of subtitling
The estimated marginal means for the significant effect of Use of subtitling on Total
fixation duration on the subtitle area (F=7.81; p<0.001) are presented in Table 24.
Table 24. Estimated marginal means of Total fixation duration on the subtitle area (ms) with the effect of
Use of subtitling
Use of subtitling
Never
Rarely
Occasionally
Very frequently/Always
Mean
32339.50
91424.39
70766.12
72726.26
Std. Error
7685.69
6275.34
3406.32
3703.06
Figure 18 shows a graphical representation of the variable Use of subtitling per
Level of L2/L3. A quick inspection of this graph indicates that only the participants in
the HLE group declared they never use subtitles. This explains the significant
differences shown in Table 25 between the participants who declared they never use
138
Chapter 5. Quantitative results
subtitles and the participants who selected the other categories (in all cases, p<0.05).
There is yet another significant difference between the participants who rarely use
subtitling and those who do it occasionally (t=2.89; p<0.01) and also between the
participants who rarely use subtitles and the ones who use them very frequently or
always (t=2.57; p<0.05). No significant difference was found between the other two
groups, which are the biggest groups in the sample and include the participants who use
subtitles occasionally and very frequently/always.
Table 25. Pairwise contrast of estimated marginal means of Total duration of fixations on the subtitle area
(ms) with the effect of Use of subtitling
Use of subtitling
Never - Rarely
Never - Occasionally
Never - Very frequently/Always
Rarely - Occasionally
Rarely - Very frequently/Always
Occasionally - Very frequently/Always
* indicates significant results (α=0.05)
Mean difference
-59084.89
-38426.62
-40386.76
20658.26
18698.13
-1960.13
Std. Error
9922.18
8406.71
8531.26
7140.23
7286.46
5031.47
t
-5.95
-4.57
-4.73
2.89
2.57
-0.39
df
141
141
141
141
141
141
p-value
<0.0001*
0.010*
0.005*
0.004*
0.011*
0.697
Figure 18. Use of subtitling per Level of L2/L3
Level of L2/L3
The level of proficiency in English was found to have a significant effect on Total
duration of fixations on the subtitle area (F=13.09; p<0.0001). Table 26 shows the
estimated means for each level of the variable. The difference between the two
estimated means is 28469.74 ms (t=5.59, p<0.001). The result indicates that participants
139
The Reception of (Non)Professional Subtitling
in the LLE group spent over 28400 ms more looking at the subtitle area than the
participants in the HLE group.
Table 26. Estimated marginal means of Total fixation duration on the subtitle area (ms) with the effect of
Level of L2/L3
Level of L2/L3
Low
High
Mean
88007.43
59537.69
Std. Error
3729.34
3470.23
Level of L2/L3 and Use of subtitling
The interaction between Level of L2/L3 and Use of subtitling on Total duration of
fixations on the subtitle area had a significant effect (F=5.60; p<0.01). The estimated
means calculated for the combination of the two variables are presented in Table 27. It
can be seen that the means for the HLE group are smaller in all cases except in those
participants with a High level of English who declared they rarely use subtitles.
Table 27. Estimated marginal means of Total duration of fixations on the subtitle area (ms) with the
interaction effect between Level of L2/L3 and Use of subtitling.
Level of L2/L3
High
Low
Use of subtitling
Never
Rarely
Occasionally
Very frequently/Always
Rarely
Occasionally
Very frequently/Always
Mean
32339.50
95162.44
50754.42
59894.39
87686.33
90777.83
85558.12
Std. Error
7685.69
8874.67
5434.60
5031.47
8874.67
4108.17
5434.60
The comparison of the estimated means for the interaction is presented in Table
28, considering Level of L2/L3 as reference factor, and the graphical representation is
shown in Figure 19. No significant difference was found in Total fixation duration
among the participants in the LLE group and the HLE group that said they rarely use
subtitles. Finally, there is a significant 40023 ms difference between the two groups of
participants between those who use subtitles occasionally (t=5.87; p<0.001) and a
difference of 25664 ms between the participants in the HLE group and the LLE group
who use subtitles very frequently/always (t=3.46; p<0.05). These results concord with
what one might expect, since they indicate that participants with a HLE who are more
used to reading subtitles spent less time reading them than the participants who are used
to reading them but are in the LLE group. The findings also show that HLE participants
who never use subtitles relied on them very little and were able to skip them most of the
time, since they are the group that spent the least time on this area.
140
Chapter 5. Quantitative results
Figure 19. Estimated marginal means of Total duration of fixations on the subtitle area (ms) with the
interaction effect between Level of L2/L3 and Use of subtitling
Table 28. Pairwise contrasts of estimated marginal means of Total duration of fixations on the subtitle
area (ms) with the interaction effect between Level of L2/L3 and Use of subtitling
Level of L2/L3
Low - High
Use of subtitling
Rarely
Occasionally
Very frequently/Always
* indicates significant results (α=0.05)
Mean difference
-7476.11
40023.42
25663.73
Std. Error
12550.68
6812.64
7406.12
t
-0.10
5.87
3.46
df
141
141
141
p-value
0.552
0.0001*
0.001*
5.3.2.2. Percentage of fixations on the subtitle area
The Percentage of fixations on the subtitle area is calculated as the ratio of the number
of fixations on the subtitle area over the total number of fixations (subtitled area plus
image area). Using Percentage of fixations on the subtitle area as target in the model,
all the other factors are tested as independent variables. The final model for Percentage
of fixations on the subtitle area found a significant effect for the variables Level of
L2/L3 (F=12.23; p<0.01), Clip (F=14.16; p<0.001), Use of subtitling (F=4.69; p<0.01)
and of the interaction between Level of L2/L3 and Prior knowledge of the clips (F=5.11,
p<0.05). Type of subtitle did not yield significant results (F=0.95; p=3.90).
Level of L2/L3
The means resulting from the significant effect of Level of L2/L3 on Percentage of
fixations on the subtitle area (F=12.23; p<0.01) are presented in Table 29. As can be
seen, there is a 0.16 difference between the LLE group and the HLE group (t=3.50;
p<0.05). This indicated that there is a 15.7% difference in the percentage of fixations
that participants with a low level of English and participants with a high level of
141
The Reception of (Non)Professional Subtitling
proficiency in English made while looking at the subtitle area. For participants in the
LLE group, this means that 59% of all the fixations during the screening time were
made within the subtitle area.
Table 29. Estimated marginal means of Percentage of fixations with the effect of Level of L2/L3
Level of L2/L3
Low
High
Mean
.590
.433
Std. Error
.039
.031
Clip
The Clip variable was found to have a significant effect on the Percentage of fixations
on the subtitle area (F=14.16; p<0.001), the means are presented in Table 30 and the
graphical representation is shown in Figure 20. The comparisons of the estimated means
are included in Table 31. While there is no significant difference between the estimated
means for Clip 1 and Clip 3 (t=1.56; p=121), when participants were watching Clip 2,
Percentage of fixations on the subtitle area was 5.8 points higher than when they were
watching Clip 1 (t=5.17; p<0.001). Percentage of fixations on the subtitle area was also
4 points higher when they were watching Clip 2 than when they were watching Clip 3
(t=3.62; p<0.001).
Table 30. Estimated marginal means of Percentage of fixations on the subtitle area with the effect of Clip
Clip
Clip 1
Clip 2
Clip 3
Mean
.486
.544
.504
Std. Error
.028
.028
.028
Figure 20. Estimated marginal means of Percentage of fixations on the subtitle area with the effect of
Clip
142
Chapter 5. Quantitative results
Table 31. Pairwise contrasts of estimated marginal means of Percentage fixations on the subtitle area
with the effect of Clip
Clips
Mean difference
Clip 1 - Clip 2
-.058
Clip 2 - Clip 3
.040
Clip 3 - Clip 1
.018
* indicates significant results (α=0.05)
Std. Error
.011
.011
.011
t
-5.17
3.62
1.56
df
137
137
137
p-value
0.000*
0.000*
0.121
As will be discussed later, the intention of the design was to have video excerpts
that would not differ greatly. Nevertheless the nature of the audiovisual material and the
focus on maintaining an ecologically valid design come with necessary costs such as
this one. The effect of Clip in the variables altogether will be addressed in Section 6.2.3.
Use of subtitling
The variable Use of subtitling was determined to have a significant effect on Percentage
of fixations on the subtitle area (F=4.69; p<0.01). Table 32 shows the estimated means
for the four levels of the variable, while the pairwise contrasts are presented in Table 33.
Table 32. Estimated marginal means of Percentage of fixations on the subtitle area with the effect of Use
of subtitling
Use of subtitling
Never
Rarely
Occasionally
Very frequently/Always
Mean
.306
.651
.533
.555
Std. Error
.074
.059
.032
.034
A close analysis of Table 33 comparing the estimated means shows the level
Never has significant differences with respect to all the other levels (p<0.01). Referring
to Figure 18, only participants in the HLE group say they never use subtitles when
watching audiovisual material. This indicates that participants who declare to never use
subtitles are also the ones who fixated the least frequently on the subtitle area. These
results show that these participants in the Never group made about 31% of their
fixations on the subtitle area. The differences between this value and the percentage of
fixations for participants in all other groups range between 22 and 35 points.
Table 33. Pairwise contrasts of estimated marginal means of Percentage of fixations on the subtitle area
with the effect of Use of subtitling
Use of subtitling
Never - Rarely
Never - Occasionally
Never - Very frequently/Always
Rarely - Occasionally
Rarely - Very frequently/Always
Occasionally - Very frequently/Always
* indicates significant results (α=0.05)
Mean difference
-.345
-.227
-.249
.118
.096
-.022
Std. Error
.094
.082
.081
.066
.067
.046
t
-3.67
-2.78
-3.08
1.80
1.43
-0.47
df
137
137
137
137
137
137
p-value
0.000*
0.006*
0.002*
0.075
0.154
0.640
143
The Reception of (Non)Professional Subtitling
Figure 21. Estimated marginal means of Percentage of fixations with the effect of Use of subtitling
Level of L2/L3 and Prior knowledge of the clips
The results indicate a significant interaction effect between Level of L2/L3 and Prior
knowledge of the clips on Percentage of fixations on the subtitle area (F=5.11, p<0.05).
Table 34 shows the mean percentage of fixations on the subtitle area for the two groups,
depending on Prior knowledge of the clips. In the LLE group, the participants who had
seen the clip before made more fixations on the subtitle area, while the case is reversed
among the participants in the HLE group.
Table 34. Estimated marginal means of Percentage of fixations on the subtitle area with the interaction
effect between Level of L2/L3 and Prior knowledge of the clips
Level of L2/L3
Low
High
Watched
Yes
No
Yes
No
Mean
.605
.575
.395
.471
Std. Error
.049
.037
.036
.033
The comparison between the levels of the variable is shown in Table 35, taking
Prior knowledge of the clips as reference factor. There is a significant difference
(t=-2.53; p<0.05) in the HLE group, which indicates that participants who said they had
seen the clip before made fewer fixations on the subtitle area. No significant difference
was found in the LLE group (t=81; p=0.418).
Table 35. Pairwise contrasts of estimated marginal means of Percentage of fixations on the subtitle area
with the interaction effect between Level of L2/L3 and Prior knowledge of the clips
Watched
Yes - No
Level of L2/L3
Mean difference
Low
.030
High
-.076
* indicates significant results (α=0.05)
144
Std. Error
.037
.030
t
0.813
-2.53
df
137
137
p-value
0.418
0.012*
Chapter 5. Quantitative results
Figure 22. Estimated marginal means of Percentage of fixations on the subtitle area with the interaction
effect between Level of L2/L3 and Prior knowledge of the clips
5.3.2.3. Other attention-allocation variables
Attention allocation is assessed in different ways depending on the measurement and the
area of interest. I used the percentage of fixations and the percentage of the duration of
fixations as standardization methods for running the analysis, considering the different
lengths of the video excerpts. This means that each variable for the subtitle area of
interest is complementary to its corresponding equivalent for the image area. Given the
high correlation between the variables, running independent analyses for each one
would yield the same or very similar results and including those analyses in the
dissertation would be redundant. For this reason, the analysis presented above offers the
results for Percentage of fixations on the subtitle area and not the results of the model
for the variable Percentage of fixations on the image area.
As shown in Table 36, when the model constructed for Percentage of fixations on
the subtitle area is applied to test Percentage of fixations on the image area as target, it
produces the same results. The model includes significant effects of the variables Level
of L2/L3 (F=12.23; p<0.01), Clip (F=14.16; p<0.001), Use of subtitling (F=4.69;
p<0.01) and of the interaction between Level of L2/L3 and Prior knowledge of the clips
(F=5.11, p<0.05), and non-significant results for the Type of subtitle variable (F=0.95;
p=3.90).
145
The Reception of (Non)Professional Subtitling
Table 36. Final model of Percentage of fixations on the subtitle area applied to Percentage of fixations on
the image area, with the same main effects and interactions
Predictor
F
L2L3Level
12.23
Subs
0.95
Clip
14.16
Subtitling
4.69
Watched
0.89
L2L3Level * Watched
5.11
* indicates significant results (α=0.05)
Numerator df
1
2
2
3
1
1
Denominator df
137
137
137
137
137
137
p-value
0.001*
0.390
<0.0001*
0.004*
0.345
0.025*
Regarding the variables related to the Percentage of duration of fixations on each
area, I expected that the analysis could provide results different from the model for
percentage of fixations, as observed in the pilot study for the experiment (Section 4.7.3).
The pilot study showed the LLE participants had a higher percentage of fixations on the
subtitle area than on the image area, but the percentage of the duration of fixations was
higher for the image area. However, this was not the case in the main experiment, where
LLE participants had both a higher percentage in the number of fixations and in the
duration of fixations on the subtitle area (see Figure 13 and Figure 14). Percentage of
fixations on the subtitle area and Percentage of duration of fixations on the subtitle area
were highly correlated (r=0.96; p<0.001). A graphical representation of these variables
can be seen in Figure 23.
Figure 23. Percentage of duration of fixations and Percentage of fixations on the subtitle area
This high correlation indicated that the statistical analysis of Percentage of
duration of fixations on the subtitle area would yield results similar to those obtained
146
Chapter 5. Quantitative results
with the model of Percentage of fixations on the subtitle area. Table 37 presents the
final model of Percentage of fixations on the subtitle area applied to Percentage of
duration of fixations on the subtitle area as target variable. The same significant effects
were found as in the other variable: on Level of L2/L3 (F=19.41; p<0.001), Clip
(F=23.13; p<0.001), Use of subtitling (F=3.03; p<0.05), the interaction between Level of
L2/L3 and Prior knowledge of the clips (F=5.61, p<0.05). Again, the model yields nonsignificant results for Type of subtitle (F=2.20; p=0.115). With relation to Level of
L2/L3, HLE participants spent 31% of the time on the subtitle area on average, while
this average percentage is 50% for LLE participants.
Table 37. Final model of Percentage of fixations on the subtitle area applied to Percentage of duration of
fixations on the subtitle area, with the same main effects and interactions
Predictor
F
Numerator df
L2L3Level
19.41
1
Subs
2.20
2
Clip
23.13
2
Subtitling
3.03
3
Watched
0.54
1
L2L3Level * Watched
5.61
1
* indicates significant results (α=0.05)
Denominator df
137
137
137
137
137
137
p-value
<0.001*
0.115
<0.001*
0.031*
0.463
0.019*
5.3.2.4. Attention allocation - Summary
Here is a list of the conclusions that can be drawn from the analysis for Attention
allocation:
-
HLE participants who never use subtitles fixated the least on the subtitle area.
Only 31% of their fixations were made on the subtitle area.
-
The Type of subtitle does not have an effect on the participant’s attention
allocation.
-
The Total fixation duration on the subtitle area seems to confirm that LLE
participants have a more homogeneous behavior, regardless of their use of
subtitles, while HLE participants have a higher degree of variation in their
behavior. Clip 2 was the object of the greatest cognitive effort. The participants
made significantly more fixations on the subtitles in Clip 2 than on the subtitles
in the other two clips.
-
The participants in the HLE group made 43% of the fixations on the subtitle
area, while this average reached 59% among the participants in the LLE group.
Likewise, they spent 32% and 51% of the time on the subtitle area respectively.
147
The Reception of (Non)Professional Subtitling
-
It would have been expected that all participants who had already seen the clip
would fixate less on the subtitles. While this is the case for HLE, the results are
counterintuitive for LLE. HLE participants who had seen the clip before made
fewer fixations on the subtitle area and this difference is statistically significant.
On the other hand, LLE participants made more fixations on the subtitle area,
although this difference is not at a significant level.
5.3.3. Skipped subtitles
Skipped subtitles are calculated as the percentage of unfixated subtitles for each clip.
The main effects of Level of L2/L3 (F=11.66; p<0.01), Order of presentation (F=3.02;
p<0.05) and Use of subtitling (F=3.11; p<0.05) were included in the final adjusted
model. Type of subtitle is also included in the model for Skipped subtitles, even though
it had a non-significant effect (F=2.49; p=0.087). It was kept as part of the model in
order to show its effect and because its inclusion did not obscure the effects of the
significant variables.
5.3.3.1. Level of L2/L3
Level of L2/L3 has a significant effect on the percentage of Skipped subtitles, according
to the model (F=11.66; p<0.01). Table 38 shows the estimated means for the two groups
of proficiency in English in the study. While participants in the LLE group skip about
7% of the subtitles on screen, HLE participants skip about one fourth of the subtitles.
Thus, there is a 17.9-point difference between the subtitles skipped by HLE participants
and those skipped by the in the LLE group (t=3.41, p<0.05).
Table 38. Estimated marginal means of Skipped subtitles with the effect of Level of L2/L3
Level of L2/L3
Low
High
Mean
.073
.252
Std. Error
.046
.038
5.3.3.2. Order of presentation
The Order of presentation was not expected to have a significant effect on the variables.
As explained above, the subtitles were randomized and participants were also randomly
assigned to one of the six orders of presentation. Nevertheless, the model for Skipped
subtitles found the order of presentation had a significant effect (F=3.02; p<0.05). The
estimated means for each level of the variable are presented in Table 39. It can be easily
148
Chapter 5. Quantitative results
seen that the estimated mean of the Order of presentation 5 is higher than the means for
the other orders. The estimated mean of order 2 is also high, although not as high as that
of order 5.
Table 39. Estimated marginal means of Skipped subtitles with the effect of order of presentation
Order of presentation
1
2
3
4
5
6
Mean
.116
.240
.060
.103
.351
.102
Std. Error
.066
.059
.067
.071
.070
.068
Figure 24. Skipped subtitle (%) per Order of presentation and Level of L2/L3
A detailed revision of the pairwise contrasts of the means in Table 40 and Figure
25 confirms that there is a significant difference between the Order of presentation 5
and all other orders of presentation (p<0.05 in all cases). As can be seen in Figure 24,
two extreme conditions coincided by chance in the order of presentation 5: those
participants in the LLE who dedicated a large percentage of the time to reading the
subtitles, and the HLE participants who dedicated the least time to the subtitle area. In
the order of presentation, there is also high dispersion among participants in the HLE
group, while the values in the LLE group are closer together. Due to the interesting
differences in these behaviors, this topic will be addressed independently in Section
6.2.1.
149
The Reception of (Non)Professional Subtitling
Table 40. Pairwise contrasts of estimated marginal means of Skipped subtitles with the effect of Order of
presentation
Order of presentation Mean difference
1-2
-.124
1-3
.056
1-4
.013
1-5
-.235
1-6
.015
2-3
.180
2-4
.137
2-5
-.111
2-6
.139
3-4
-.043
3-5
-.291
3-6
-.041
4-5
-.248
4-6
.002
5-6
.250
* indicates significant results (α=0.05)
Std. Error
.084
.090
.090
.096
.087
.088
.090
.095
.085
.088
.090
.089
.092
.088
.095
t
-1.47
0.62
0.14
-2.45
0.17
2.04
1.53
-1.16
1.63
-0.49
-3.23
-0.46
-2.70
0.02
2.62
df
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
p-value
0.142
0.536
0.886
0.016*
0.867
0.043*
0.129
0.247
0.105
0.625
0.002*
0.643
0.008*
0.984
0.010*
Figure 25. Estimated marginal means of Skipped subtitles (%) with the effect of Order of presentation
5.3.3.3. Use of subtitling
The participants’ declared use of subtitles had a significant effect on the percentage of
Skipped subtitles (F=3.11; p<0.05). The estimated means for each of the levels are
presented in Table 41. The participants who never use subtitles skipped 38% of the
subtitles on screen. Those who rarely use subtitles relied almost completely on them and
skipped less than 1%. The rest of the participants, who are more used to subtitles,
skipped only around 13%.
150
Chapter 5. Quantitative results
Table 41. Estimated marginal means of Skipped subtitles with the effect of Use of subtitling
Use of subtitling
Never
Rarely
Occasionally
Very frequently/Always
Mean
.382
.005
.125
.136
Std. Error
.096
.078
.040
.044
Once more, the HLE participants who were the only ones who declared they
never use subtitles had a significantly higher percentage of Skipped subtitles than any
other group in the sample (p<0.05) for all comparisons between never and the other
three levels. The results of the pairwise contrasts are included in Table 42 and the
representation of the means is shown in Figure 26.
Table 42. Pairwise contrasts of estimated marginal means of Skipped subtitles with the effect of Use of
subtitling
Use of subtitling
Never - Rarely
Never - Occasionally
Never - Very frequently/Always
Rarely - Occasionally
Rarely - Very frequently/Always
Occasionally - Very frequently/Always
* indicates significant results (α=0.05)
Mean difference
.377
.257
.246
-.120
-.131
-.011
Std. Error
.126
.105
.108
.092
.085
.062
t
2.99
2.46
2.29
-1.31
-1.53
-0.172
df
136
136
136
136
136
136
p-value
0.003*
0.015*
0.024*
0.192
0.128
0.864
Figure 26. Estimated marginal means of Skipped subtitles (%) with the effect of Use of subtitling
5.3.3.4. Skipped subtitles - Summary
The conclusions that can be drawn from the analysis for Skipped subtitles are
summarized as follows:
-
Type of subtitle does not influence the percentage of Skipped subtitles.
151
The Reception of (Non)Professional Subtitling
-
HLE participants skipped 25% of the subtitles while LLE participants skipped
only 7%.
-
By chance in the assignation of participants to the orders of presentations, the
fifth order of presentation included only participants in the HLE group who
skipped more subtitles than the others. This offers an interesting point of
comparison that will be covered in Section 6.2.1.
-
HLE participants who never use subtitles skipped 38% of them, showing a clear
decision to avoid fixating on the subtitle area. Those who rarely use subtitles
skipped less than 1% of them and those more used to using subtitles skipped
about 12%.
5.3.4. Mean fixation duration on the subtitle area
The final model of Mean fixation duration on the subtitle area includes the main effects
of Type of subtitle (F=26.97; p<0.001) and Use of subtitling (F=3.35, p<0.05).
Additionally, the interaction between Clip and Type of subtitle was also found to have a
significant effect (F=5.83; p<0.01). Clip by itself and Level of L2/L3 did not have a
significant effect on Mean fixation duration on the subtitle area. Level of L2/L3 was
removed from final model.
5.3.4.1. Type of Subtitle
The Type of subtitle had a significant effect on the means fixation on the subtitle area
(F=26.97; p<0.001). Table 43 shows that PRO subtitles had the lowest mean fixation
duration, 178 ms, while the mean fixation durations on both non-professional subtitles
were more similar: NP1 had a mean fixation of 197 ms and NP2 had a mean fixation
duration of 202 ms. A graphical representation of the means can be seen in Figure 27.
Table 43. Estimated marginal means of Mean fixation duration on the subtitle area (ms) with the effect of
Type of subtitle
Type of subtitle
NP1
NP2
PRO
Mean
196.688
201.848
178.408
Std. Error
4.303
4.315
4.327
Considering that PRO subtitles have the shortest fixations, the pairwise
comparisons of the means indicates there is a 23-ms difference between NP2 version
and PRO version (t=6.99; p<0.0001) and an 18-ms difference between PRO and NP1
152
Chapter 5. Quantitative results
version (t=-5.49; p<0.0001). No significant differences were found between the two
non-professional versions (t=-1.56; p=0.122).
Table 44. Pairwise contrasts of estimated marginal means of Mean fixation duration on the subtitle area
(ms) with the effect of Type of subtitle
Type of subtitle
Mean difference
NP1 - NP2
-5.159
NP2 - PRO
23.439
PRO - NP1
-18.280
* indicates significant results (α=0.05)
Std. Error
3.311
3.353
3.327
t
-1.56
6.99
-5.49
df
135
135
135
p-value
.122
<0.0001*
<0.0001*
Figure 27. Estimated marginal means of Mean fixation duration on the subtitle area with the effect of
Type of subtitle
5.3.4.2. Use of subtitling
The Mean fixation duration on the subtitle area is significantly affected by the
participants’ regular use of subtitling (F=3.35, P<0.05). Table 45 presents the mean
fixation duration for each level of the Use of subtitle variable. The participants who
declared they Never use subtitling had the longest mean fixation duration (217 ms),
while the participants who use subtitles Very frequently or always had the shortest mean
fixation duration, 179 ms.
Table 45. Estimated marginal means of Mean fixation duration on the subtitle area (ms) with the effect of
Use of subtitling
Use of subtitling
Never
Rarely
Occasionally
Very frequently/Always
Mean
216.756
183.715
189.373
179.415
Std. Error
10.783
8.808
4.587
5.112
Table 46 presents the pairwise comparisons between the levels of Use of
subtitling. Only the participants who said they Never use subtitling have significantly
153
The Reception of (Non)Professional Subtitling
longer mean fixations on the subtitle area (p<0.05). Their mean fixation duration is
between 27 ms and 37 ms longer than the mean fixation duration for the other three
groups. These results, combined with the fact that these participants are also the ones
who skipped the most subtitles, fixated the least on the subtitle area and had the lowest
total fixation duration on the same area, indicate that these participants might go to the
subtitle area on very specific occasions and then dedicate a high amount of their
cognitive processing on that area. Few and long fixations could be indicators of a
conscious and decisive behavior of the participants, triggered by salient aspects of the
clip.
Table 46. Pairwise contrasts of estimated marginal means of Mean fixation duration on the subtitle area
(ms) with the effect of Use of subtitling
Use of subtitling
Never - Rarely
Never - Occasionally
Never - Very frequently/Always
Rarely - Occasionally
Rarely - Very frequently/Always
Occasionally - Very frequently/Always
* indicates significant results (α=0.05)
Mean difference
33.041
27.383
37.341
-5.657
4.300
9.957
Std. Error
13.953
11.694
11.994
9.975
10.146
6.902
t
2.37
2.34
3.11
-0.57
0.42
1.44
df
135
135
135
135
135
135
p-value
0.019*
0.021*
0.002*
0.572
0.672
0.151
Figure 28. Estimated marginal means of Mean fixation duration on the subtitle area (ms) with the effect
of Use of subtitling
5.3.4.3. Clip and Type of subtitle
A significant interaction effect between Clip and Type of subtitle was found in the
model of Mean fixation duration on the subtitle area (F=5.83; p<0.01). Table 47 shows
the Mean fixation durations on the subtitle area classified by Type of subtitle and Clip.
154
Chapter 5. Quantitative results
The figures show that mean fixations for NP2 are very similar in all three clips, while
the mean fixations for PRO and NP1 have more variation between clips.
Table 47. Estimated marginal means of Mean fixation duration on the subtitle area (ms) with the
interaction effect between Clip and Type of subtitle
Type of subtitle
NP1
NP2
PRO
Clip
Clip 1
Clip 2
Clip 3
Clip 1
Clip 2
Clip 3
Clip 1
Clip 2
Clip 3
Mean
197.352
206.866
185.846
203.274
203.220
199.049
181.162
162.525
191.537
Std. Error
6.025
5.670
5.934
5.983
6.002
5.699
5.758
6.042
5.980
The pairwise contrasts of the means are presented in Table 48, with Type of
subtitle as the reference factor. Again, there is no significant difference between the
mean fixation durations for NP2. In the case of NP1, the mean fixation duration in Clip
2 is 21 ms longer than the mean fixation duration in Clip 3 (t=3.03; p<0.05). The mean
fixations for PRO had more variation. There is no significant difference between the
mean fixations in Clip 1 and Clip 3 for PRO. However, PRO in Clip 2 differs
significantly from Clip 1 and Clip 3. There is an 18 ms difference between the mean
fixation duration in Clip 1 and Clip 2 (t=2.66; p<0.01) and a 29-ms difference between
Clip 2 and Clip 3 (t=-4.14; p<0.0001).
Table 48. Pairwise contrasts of estimated marginal means of Mean fixation duration on the subtitle area
(ms) with the interaction effect between Clip and Type of subtitle (Type of subtitle as reference factor)
Type of subtitle
NP1
Clip
Mean difference
Clip 1 - Clip 2
-9.514
Clip 1 - Clip 3
11.506
Clip 2 - Clip 3
21.020
NP2
Clip 1 - Clip 2
.054
Clip 1 - Clip 3
4.225
Clip 2 - Clip 3
4.171
PRO
Clip 1 - Clip 2
18.637
Clip 1 - Clip 3
-10.375
Clip 2 - Clip 3
-29.012
* indicates significant results (α=0.05)
Std. Error
6.828
7.047
6.935
7.102
6.807
6.966
7.018
7.036
6.999
t
-1.39
1.63
3.03
1.01
1.62
0.60
2.66
-1.47
-4.14
df
135
135
135
135
135
135
135
135
135
p-value
0.166
0.105
0.003*
0.994
0.536
0.550
0.009*
0.143
<0.0001*
The results of the pairwise comparisons taking Clip as the reference factor show
there are no significant differences between the measurements of mean fixation duration
in Clip 3. In all other cases, the mean fixation duration for PRO is significantly shorter
than the mean fixation duration in Clip 2 using the non-professional subtitles (p<0.05 in
all cases). It is possible that mean fixation durations are affected by the format used to
155
The Reception of (Non)Professional Subtitling
display the subtitles. As explained Chapter 6, the non-professional versions used the
same standard format included in the video player software, while the display of
professional subtitles follows the embedding process used to create the DVDs.
Figure 29. Estimated marginal means of Mean fixation duration on the subtitle area (ms) with the
interaction effect between Clip and Type of subtitle
5.3.4.4. Mean fixation duration on the subtitle area - Summary
Below is a shortlist of the conclusions that can be drawn from the analysis for Mean
fixation duration on the subtitle area:
-
Level of proficiency in L2/L3 does not affect the mean fixation duration on the
subtitle area.
-
The shortest mean fixation durations were reported when participants were
watching the professionally subtitled versions of Clip 1 and Clip 2.
-
HLE participants who never use subtitles had the longest mean fixation
durations. This might indicate their fixations on the subtitles were guided by a
conscious interest in them.
-
The mean fixation duration on Clip 1 and Clip 2 in the PRO version is
significantly shorter.
5.3.5. Mean fixation duration on the image area
The Mean fixation duration is calculated by dividing the Total fixation duration on the
image area by the number of fixations on the same area. The model includes two
156
Chapter 5. Quantitative results
variables that had a significant effect on Mean fixation duration on the image area:
Type of subtitle (F=5.52; p<0.01) and Level of L2/L3 (F=20.17; p<0.001).
5.3.5.1. Type of subtitle
The Type of subtitle had a significant effect on the Mean fixation duration on the image
(F=5.52; p<0.01). Table 49 presents the mean fixation duration on the image area for
each type of subtitle. The shortest mean fixation duration on the image area is that of the
PRO versions (333 ms) and the mean fixation duration on the image area in the NP1
versions is the longest (357 ms).
Table 49. Estimated marginal means of Mean fixation duration on the image area (ms) with the effect of
Type of subtitle
Type of subtitle
NP1
NP2
PRO
Mean
357.002
346.595
333.456
Std. Error
11.688
11.688
11.751
Table 50 shows the pairwise comparisons for the mean fixation durations on the
image area with the effect of Type of subtitle. Only one significant difference was found
between the three types of subtitles: there is a mean difference of 23 ms between PRO
and NP1 subtitles (t=3.32; p<0.01). Figure 30 shows the graphical representation of the
means.
Figure 30. Estimated marginal means of Mean fixation duration on the image area (ms) with the effect of
Type of subtitle
157
The Reception of (Non)Professional Subtitling
Table 50. Pairwise contrasts of estimated marginal means of Mean fixation duration on the image area
(ms) with the effect of Type of subtitle
Type of subtitle
Mean difference Std. Error
NP1 - NP2
10.407
6.991
NP2 - PRO
13.138
7.096
PRO - NP1
-23.546
7.096
* indicates significant results (α=0.05)
t
1.49
1.85
-3.32
df
144
144
144
p-value
0.139
0.066
0.001*
5.3.5.2. Level of L2/L3
The participants’ level of proficiency in English was also found to have a significant
effect on the mean duration of their fixations on the image area (F=20.17; p<0.001).
Table 51 shows the mean fixation duration on the image area for the two groups of
participants. The mean fixation duration on the image for LLE participants was 296 ms,
while the participants in the HLE group had a mean fixation of 395 ms. This translates
into a significant difference of 98.6 ms between the two groups (t=4.49, p<0.001).
Table 51. Estimated marginal means of Mean fixation duration on the image area (ms) with the effect of
Level of L2/L3
Level of L2/L3
Low
High
Mean
296.392
394.977
Std. Error
15.512
15.533
5.3.5.3. Mean fixation duration on the image area - Summary
The main highlights from the analysis for Mean fixation duration on the image area are:
-
The mean fixation duration on the image is also shorter with the professional
version of the subtitles. This might indicate that Type of subtitle affects not only
the behavior on the subtitle area, but also the overall viewing process.
-
HLE participants have longer mean fixations on the image than do LLE
participants.
5.3.6. Attention shift ratio
Attention shift ratio reports the number of attention shifts participants made per subtitle
in the video. A model was constructed to analyze the variables that have an effect on
Attention shift ratio. Significant effects were found in Level of L2/L3 (F=7.10; p<0.01),
Clip (F=6.66; p<0.01), Use of subtitling (F=3.48; p<0.05) and Type of subtitle (F=7.32;
p<0.01). Additionally, the interaction between Clip and Type of subtitle has a significant
effect on Attention shift ratio (F=3.08; p<0.05).
158
Chapter 5. Quantitative results
5.3.6.1. Level of L2/L3
Table 52 shows the estimated marginal means of Attention shift ratio with the
significant effect of Level of L2/L3 (F=7.10; p<0.01). The participants in the LLE group
make 2.09 shifts from the image to the subtitle or vice versa per subtitle, while the mean
for HLE participants is 1.72 shifts. This represents a difference of 0.37 shifts (t=2.66;
p<0.01).
Table 52. Estimated marginal means of Attention shift ratio with the effect of Level of L2/L3
Level of L2/L3
Low
High
Mean
2.090
1.716
Std. Error
.119
.101
5.3.6.2. Clip
Clip was also found to have a significant effect on Attention shift ratio (F=6.66;
p<0.01). The mean values per clip are shown in Table 53. While the means for Clip 1
(1.94) and Clip 3 (1.97) are very similar, the mean attention shift ratio for Clip 2 is
lower than the other two, at 1.79.
Table 53. Estimated marginal means of Attention shift ratio with the effect of Clip
Clip
Clip 1
Clip 2
Clip 3
Mean
1.944
1.791
1.974
Std. Error
.091
.091
.091
The pairwise contrasts of the mean attention shift ratio are presented in Table 54.
There is no significant difference between Clip 1 and Clip 3, but Clip 2 is significantly
different from the other two. There is a 0.15-shift difference between Clip 1 and Clip 2
(t=2.81; p<0.01) and a 0.18-shift difference between Clip 2 and Clip 3 (t=-3.41;
p<0.01). The means are also shown in Figure 31.
Table 54. Pairwise contrasts of estimated marginal Attention shift ratio with the effect of Clip
Clips
Mean difference
Std. Error
Clip 1 - Clip 2
.153
.054
Clip 2 - Clip 3
-.183
.054
Clip 3 - Clip 1
.030
.054
* indicates significant results (α=0.05)
t
2.81
-3.41
0.56
df
135
135
135
p-value
0.006*
0.001*
0.577
159
The Reception of (Non)Professional Subtitling
Figure 31. Estimated marginal means of Attention shift ratio with the effect of Clip
5.3.6.3. Use of subtitling
A significant effect of the participants’ regular Use of subtitling on Attention shift ratio
was found in the model (F=3.48; p<0.05). Table 55 shows the estimated marginal
means of Attention shift ratio for each level of Use of subtitling. The participants who
said they Never use subtitling have the lowest mean, 1.3 shifts, while the participants
who Rarely use subtitles had the highest mean at 2.2 shifts per subtitle.
Table 55. Estimated marginal means of Attention shift ratio with the effect of Use of subtitling
Use of subtitling
Never
Rarely
Occasionally
Very frequently/Always
Mean
1.303
2.258
2.082
1.969
Std. Error
.244
.191
.101
.111
Table 56 shows the results of the pairwise contrasts of the means. Only the
comparisons involving the Never level give significant results. There is a 0.95-shift
difference between Never and Rarely (t=-3.07; p<0.01) and a 0.78-shift difference
between Never and Occasionally (t=-2.89; p<0.01). In the case of Never and Very
frequently/always, the mean difference is 0.67 shifts (t=-2.49, p<0.05). The graph in
Figure 32 shows the mean attention shifts per subtitle for each level of Use of subtitling.
160
Chapter 5. Quantitative results
Table 56. Pairwise contrasts of estimated marginal Attention shift ratio with the effect of Use of subtitling
Use of subtitling
Never - Rarely
Never - Occasionally
Never - Very frequently/Always
Rarely - Occasionally
Rarely - Very frequently/Always
Occasionally - Very frequently/Always
* indicates significant results (α=0.05)
Mean difference
-.955
-.779
-.666
.175
.289
.113
Std. Error
.311
.269
.267
.217
.220
.152
t
-3.07
-2.89
-2.49
0.81
1.31
0.75
df
135
135
135
135
135
135
p-value
0.003*
0.004*
0.014*
0.420
0.192
0.455
Figure 32. Estimated marginal means of Attention shift ratio with the effect of Use of subtitling
5.3.6.4. Type of Subtitle
The different types of subtitles have a significant effect on Attention shift ratio (F=7.32;
p<0.01). The adjusted means for attention shifts with the effect of Type of subtitle are
shown in Table 57. The means for NP1 and NP2 are very similar, but the mean for PRO
is higher. As shown in Table 58, the pairwise comparisons indicate there are significant
differences between the mean of the PRO version and the means of the two nonprofessional versions. The mean attention shift ratio per subtitle in the PRO version is
0.21 shifts higher than the mean in the NP2 version (t=-3.77; p<0.001) and it is also
0.14 shifts higher than the mean in NP1 (t=2.50; p<0.05).
Table 57. Estimated marginal means of Attention shift ratio with the effect of Type of subtitle
Type of subtitle
NP1
NP2
PRO
Mean
1.880
1.811
2.016
Std. Error
.091
.091
.091
161
The Reception of (Non)Professional Subtitling
Table 58. Pairwise contrasts of estimated marginal means of Attention shift ratio with the effect of Type
of subtitle
Type of subtitle
Mean difference Std. Error
NP1 - NP2
.069
.054
NP2 - PRO
-.205
.054
PRO - NP1
.136
.054
* indicates significant results (α=0.05)
t
1.28
-3.77
2.50
df
135
135
135
p-value
0.202
0.000*
0.014*
Figure 33. Estimated marginal means of Attention shift ratio with the effect of Type of subtitle
5.3.6.5. Clip and Type of subtitle
The interaction between Clip and Type of subtitle had a significant effect on the
Attention shift ratio (F=3.08; p<0.05). Table 59 shows the mean Attention shift ratio as
a function of Type of subtitle and Clip. By looking at the values, it can be seen that the
mean attention shifts per subtitle in Clip 2 with NP2 is the lowest of all the figures.
Table 59. Estimated marginal means of Attention shift ratio with the interaction between Clip and Type of
subtitle
Type of subtitle
NP1
NP2
PRO
Clip
Clip 1
Clip 2
Clip 3
Clip 1
Clip 2
Clip 3
Clip 1
Clip 2
Clip 3
Mean
1.960
1.842
1.838
1.930
1.511
1.994
1.941
2.019
2.090
Std. Error
.115
.110
.113
.115
.113
.110
.111
.115
.114
Table 60 presents the mean values for attention shifts taking Clip as the reference
factor. In Clip 2, mean differences are significantly shorter for NP2. A graphical
representation of the means is shown in Figure 34. Considering Clip as the reference
162
Chapter 5. Quantitative results
factor, the only significant differences are found in the NP2 version of Clip 2. There is a
0.42-shift ratio difference on average between Clip 1 and Clip 2 (t=3.57; p<0.01). The
difference between Clip 2 and Clip 3 is -0.48 shifts (t=-4.18; p<0.0001).
Table 60. Pairwise contrasts of estimated marginal means of Attention shift ratio with the interaction
effect between Clip and Type of subtitle (Clip as reference factor)
Type of subtitle
Clip 1
Clip
Mean difference
NP1 - NP2
.030
NP2 - PRO
-.011
PRO - NP1
-.020
Clip 2
NP1 - NP2
.331
NP2 - PRO
-.508
PRO - NP1
.177
Clip 3
NP1 - NP2
-.155
NP2 - PRO
-.096
PRO - NP1
.252
* indicates significant results (α=0.05)
Std. Error
.117
.118
.118
.116
.117
.115
.116
.114
.118
t
0.26
-0.09
-0.17
2.86
-4.32
1.53
-1.34
-0.84
2.13
df
135
135
135
135
135
135
135
135
135
p-value
0.795
0.927
0.867
0.005*
0.0001*
0.127
0.182
0.402
0.035
Figure 34. Estimated marginal means of Attention shift ratio with the interaction between Clip and Type
of subtitle
5.3.6.6. Attention shift ratio - Summary
The main highlights resulting from the analysis of Attention shift ratio are as follows:
-
In accordance with the percentage of fixations on the subtitle area, the
participants in the HLE group made fewer attention shifts.
-
Clip 2 had significantly less attention shifts probably because the participants
followed the subtitles more closely.
-
HLE participants who had never used subtitles made significantly fewer shifts,
given that they skipped 25% of the subtitles.
163
The Reception of (Non)Professional Subtitling
-
The PRO version had significantly more attention shifts than did the other types
of subtitles.
-
Clip 2 with NP2 had the lowest mean attention fixation among all the
combinations of clips and subtitles.
5.3.7. Subtitle-reading effort
The participants in the experiment were asked to rate the difficulty they had following
the subtitles, using a scale ranging from 0=very difficult to 5=very easy. Table 61
presents the model with the variables and interactions that had a significant effect on
Subtitle-reading effort. The variables were Prior knowledge of the clips (F=9.51;
p<0.01) and Use of subtitling (F=3.06; p<0.05). The interaction between Clip and Type
of subtitle was also found to have a significant effect (F=3.10; p<0.05). The Type of
subtitle variable did not have a significant effect, and neither did Level of L2/L3. The
latter is not included in the final model because, although by itself it is not significant, it
affects the results of the significant variables.
Table 61. Final model with significant main effects and interactions of Subtitle-reading effort
Predictor
F
Numerator df
Clip
1.14
2
Subs
0.60
2
Watched
9.51
1
Subtitling
3.06
3
Clip * Subs
3.10
4
* indicates significant results (α=0.05)
Denominator df
141
141
141
141
141
p-value
0.322
0.552
0.002*
0.030*
0.018*
5.3.7.1. Prior knowledge of the clips
A significant effect was found for the participants’ previous knowledge of the clip on
the difficulty they had to read the subtitles (F=9.51; p<0.01). Table 62 shows the
estimated means for Subtitle-reading effort for the levels of Prior knowledge of the
clips. The mean is higher for the participants who have previously seen the clips. There
is a 0.59 mean difference between the two groups (t=3.08; p<0.01).
Table 62. Estimated marginal means of Subtitle-reading effort with the effect of Prior knowledge of the
clips
Watched
Yes
No
164
Mean
4.201
3.606
Std. Error
.182
.120
Chapter 5. Quantitative results
5.3.7.2. Use of subtitling
There is a significant effect of Use of subtitling on Subtitle-reading effort (F=3.06;
p<0.05). The mean values of Subtitle-reading effort for all levels of Use of subtitling are
shown in Table 63. The participants who rated the subtitles higher were those at the
Never level. As has been seen in the previous results, these are participants from the
HLE group who barely look at the subtitle area but have longer mean fixations on the
subtitles.
Table 63. Estimated marginal means of Subtitle-reading effort with the effect of Use of subtitling
Use of subtitling
Never
Rarely
Occasionally
Very frequently/Always
Mean
4.618
3.610
3.669
3.716
Std. Error
.300
.261
.151
.157
Table 64 presents the pairwise comparisons of the means of Subtitle-reading
effort with the effect of Use of subtitling. The mean of the participants at the Never level
differs by about 1 point from all other mean (p<0.05 in all cases), as shown in Figure
35.
Table 64. Pairwise contrasts of estimated marginal means of Subtitle-reading effort with the effect of Use
of subtitling
Use of subtitling
Never - Rarely
Never - Occasionally
Never - Very frequently/Always
Rarely - Occasionally
Rarely - Very frequently/Always
Occasionally - Very frequently/Always
* indicates significant results (α=0.05)
Mean difference
1.008
.949
.902
-.059
-.106
-.047
Std. Error
.391
.328
.338
.289
.298
.214
t
2.58
2.90
2.67
-.20
-0.36
-0.22
df
141
141
141
141
141
141
p-value
0.011*
0.004*
0.009*
0.839
0.722
0.825
Figure 35. Estimated marginal means of Subtitle-reading effort with the effect of Use of subtitling
165
The Reception of (Non)Professional Subtitling
5.3.7.3. Clip and Type of subtitle
The interaction between Clip and Type of subtitle was found to have a significant effect
on Subtitle-reading effort (F=3.10; p<0.05). The mean Subtitle-reading effort scores per
Clip and Type of subtitle are shown in Table 65. The means for Clip 2 are lower than 4
in all cases. In Clip 1, the lowest mean is 3.57 points in the NP2 version, while for
Clip 3 it is the PRO version the one that shows the lowest score, 3.59.
Table 65. Estimated marginal means of Subtitle-reading effort with the interaction effect between Clip
and Type of subtitle
Clip
Clip 1
Clip 2
Clip 3
Type of subtitle
NP1
NP2
PRO
NP1
NP2
PRO
NP1
NP2
PRO
Mean
4.003
3.572
4.322
3.845
3.633
3.875
4.035
4.248
3.599
Std. Error
.224
.205
.204
.209
.217
.213
.215
.210
.217
Table 66 shows the pairwise comparisons considering Clip as the reference
factor. The values in the table indicate there is a 0.75-point significant difference in Clip
1 between the NP2 and the PRO version (t=-2.85; p<0.01), with the PRO version having
a higher score. In Clip 3, there is also a 0.65-point significant difference between NP2
and PRO, but in this case the higher score is that of NP2 (t=2.39; p<0.05). The graphical
representation of these means is shown in Figure 36.
Table 66. Pairwise contrasts of estimated marginal means of Subtitle-reading effort with the interaction
effect between Clip and Type of subtitle (Clip as reference factor)
Clip
Clip 1
Type of subtitle Mean difference
NP1 - NP2
.431
NP2 - PRO
-.750
PRO - NP1
.319
Clip 2
NP1 - NP2
.212
NP2 - PRO
-.242
PRO - NP1
.031
Clip 3
NP1 - NP2
-.214
NP2 - PRO
.649
PRO - NP1
-.435
* indicates significant results (α=0.05)
Std. Error
.271
.263
.272
.281
.281
.265
.270
.271
.280
t
1.59
-2.85
1.17
0.75
-0.86
0.17
-0.79
2.39
-1.56
df
141
141
141
141
141
141
141
141
141
p-value
0.115
0.005*
0.242
0.453
0.390
0.908
0.429
0.018*
0.122
According to the pairwise comparisons of the means, and taking Type of subtitle
as reference factor, there are no significant differences for NP1 between the three clips.
In the case of the NP2 version, the subtitle-reading difficulty score for Clip 3 is
166
Chapter 5. Quantitative results
significantly different from the other two. There is a 0.62 difference between the scores
for Clip 2 and Clip 3 (t=-2.19; p<0.05) and a 0.68 difference between the scores for Clip
3 and Clip 1 (t=2.57; p<0.05). There is only one significant difference among the clips
with professional subtitles. The mean scores for Clip 3 and Clip 1 differ by 0.72 points
(t=-2.65; p<0.01).
Figure 36. Estimated marginal means of Subtitle-reading effort with the interaction effect between Clip
and Type of subtitle
5.3.7.4. Subtitle-reading effort - Summary
The conclusions that can be drawn from the analysis of Subtitle-reading effort are
summarized as follows:
-
Type of subtitle and Level of L2/L3 did not have a significant effect on Subtitlereading effort, but the interaction between Clip and Type of subtitle was
significant.
-
The participants who had already seen the clips gave higher scores (indicators of
less effort) for Subtitle-reading effort.
-
HLE participants who never use subtitles rated them almost one point higher
than all the other participants.
-
While the ratings for Clip 1 and Clip 3 range from 3.5 to 4.3, there is less
variation in the average rating for Clip 2. In all cases the average for Subtitlereading effort for Clip 2 is between 3.5 and 3.8.
167
The Reception of (Non)Professional Subtitling
5.3.8. Audience enjoyment
Audience enjoyment was measured on a scale ranging from 1=boring to 4=great fun.
Three variables were found to have a significant effect on Audience enjoyment: the final
model, shown in Table 67, includes Level of L2/L3 (F=7.17; p<0.01), Clip (F=10.22;
p<0.001) and Subtitle-reading effort (F=7.12; p<0.01). No interaction effect was found
to be significant.
Table 67. Final model with significant main effects on Audience enjoyment
Predictor
L2L3Level
Clip
Trans_Read_Effort
F
7.17
10.22
7.12
Numerator df
1
2
3
Denominator df
147
147
147
p-value
0.008*
0.000*
0.000*
5.3.8.1. Level of L2/L3
The participants’ level of proficiency in English was found to have a significant effect
on their enjoyment of the material (F=7.17; p<0.01). Table 68 shows the estimated
means for the two groups. The participants in the LLE group gave a mean enjoyment
score of 2.49, while the mean for HLE participants was 2.86 (t=2.68; p<0.01).
Table 68. Estimated marginal means of Audience enjoyment with the effect of Level of L2/L3
Level of L2/L3
Low
High
Mean
2.494
2.855
Std. Error
.098
.097
5.3.8.2. Clip
The clips also had a significant effect on the participants’ enjoyment (F=10.22;
p<0.001). As shown in Table 69, the mean for Clip 1 was 2.48, the mean for Clip 2 was
2.56 and the mean for Clip 3 was 2.97. Table 70 shows the pairwise comparisons for
these means. The mean score of Clip 3 is significantly different from the other two.
There is a 0.41-point difference between Clip 2 and Clip 3 (t=-3.48; p<0.01) and a 0.49point difference between Clip 3 and Clip 1 (F=4.23; p<0.001).
Table 69. Estimated marginal means of Audience enjoyment with the effect of Clip
Clip
Clip 1
Clip 2
Clip 3
168
Mean
2.485
2.566
2.972
Std. Error
.096
.099
.097
Chapter 5. Quantitative results
Table 70. Pairwise contrasts of estimated marginal means of Audience enjoyment with the effect of Level
of L2/L3
Clips
Mean difference
Clip 1 - Clip 2
-.081
Clip 2 - Clip 3
-.406
Clip 3 - Clip 1
.487
* indicates significant results (α=0.05)
Std. Error
.117
.117
.115
t
-0.69
-3.48
4.23
df
147
147
147
p-value
0.489
0.001*
0.0004*
5.3.8.3. Subtitle-reading effort
The Subtitle-reading effort variable had a significant effect on Audience enjoyment
(F=7.12; p<0.01). Table 71 shows the estimated marginal means for all the levels of
Subtitle-reading effort (0 and 1 are not included because they were not chosen by the
participants in their answers).
Table 71. Estimated marginal means of Audience enjoyment with the effect of Subtitle-reading effort
Subtitle-reading effort
2
3
4
5
Mean
2.401
2.419
2.695
3.182
Std. Error
.145
.120
.091
.141
Table 72 shows the pairwise contrasts between the means of Audience enjoyment
with the effect of Subtitle-reading effort. No significant difference was found between
levels 2 and 3, nor 2 and 4. The difference between 3 and 4 was 0.28 points (t=1.99;
p<0.05). Level 5 was significantly different from the other three groups. There was a
0.78 difference between 2 and 5 (t=-3.92; p<0.001). The difference between 3 and 5
was 0.76 (t=-4.23; p<0.001) and the mean difference between 4 and 5 was 0.49 (t=3.09;
p<0.01). The means are presented graphically in Figure 37.
Table 72. Pairwise contrasts of estimated marginal means of Audience enjoyment with the effect of
Subtitle-reading effort
Subtitle-reading effort
Mean difference
2–3
-.017
2–4
-.294
2–5
-.780
3–4
-.277
3–5
-.763
4–5
-.486
* indicates significant results (α=0.05)
Std. Error
.176
.162
.199
.139
.180
.157
t
-0.10
-1.82
-3.92
-1.99
-4.23
-3.09
df
147
147
147
147
147
147
p-value
0.922
0.071
<0.001*
0.048*
0.0004*
0.002*
169
The Reception of (Non)Professional Subtitling
Figure 37. Estimated marginal means of Audience enjoyment with the effect of Subtitle-reading effort
5.3.8.4. Audience enjoyment - Summary
The main highlights resulting from the analysis of Audience enjoyment are:
-
The types of subtitles did not affect self-reported Audience enjoyment.
-
HLE participants reported a higher degree of enjoyment.
-
Enjoyment depends on the clips.
-
Enjoyment is affected by Subtitle-reading effort: in cases when the participants
said the subtitles were easier to follow, they also reported a higher degree of
enjoyment.
5.3.9. Self-reported comprehension
The model constructed for analyzing the participants’ ratings of their comprehension of
the clips is shown in Table 73. The variables that have a significant effect on these
scores are Prior knowledge of the clips (F=6.70; p<0.05), Use of subtitling (F=3.78;
p<0.05), Audience enjoyment (F=3.91; p<0.05) and Narrative attention (F=7.58;
p<0.01). The interaction between Clip and Type of subtitle also has a significant effect
on Self-reported comprehension (F=2.92; p<0.05). Level of L2/L3 did not have a
significant effect.
170
Chapter 5. Quantitative results
Table 73. Final model with significant main effects and interactions of Self-reported comprehension
Predictor
F
Numerator df
Clip
0.347
2
Subs
1.54
2
Watched
6.70
1
Subtitling
3.78
3
Enjoyment
3.91
3
Narr.Atte
7.58
2
Clip * Subs
2.92
4
* indicates significant results (α=0.05)
Denominator df
138
138
138
138
138
138
138
p-value
0.707
0.218
0.011*
0.012*
0.010*
0.001*
0.024*
5.3.9.1. Prior knowledge of the clips
The fact that some of the participants had already seen the clips prior to the experiment
proved to have a significant effect on their self-reported comprehension (F=6.70;
p<0.05). The mean self-reported comprehension for those who were watching the clip
for the first time was 3.45, while the mean for the participants who had watched the clip
before was 3.86. The difference between the two means is 0.407 points (t=2.60;
p<0.05).
Table 74. Estimated marginal means of Self-reported comprehension with the effect of Prior knowledge
of the clips
Watched
Yes
No
Mean
3.857
3.450
Std. Error
.173
.128
5.3.9.2. Use of subtitling
Table 75 shows the mean Self-reported comprehension scores for all the levels of Use of
subtitling. The participants who Never use subtitles had the highest comprehension
scores, 4.28, while the participants who Rarely use subtitling had the lowest selfreported comprehension score, 3.3.
Table 75. Estimated marginal means of Self-reported comprehension with the effect of Use of subtitling
Use of subtitling
Never
Rarely
Occasionally
Very frequently/Always
Mean
4.278
3.268
3.613
3.456
Std. Error
.261
.230
.146
.157
Table 76 shows the pairwise comparisons of the means of Self-reported
comprehension with the effect of Use of subtitling. The results show that the
participants at the Never level reported significantly greater comprehension than did the
171
The Reception of (Non)Professional Subtitling
participants in all three other groups (p<0.05 in all cases). Figure 38 shows the mean
self-reported comprehension ratings for all groups.
Table 76. Pairwise contrasts of estimated marginal means of Self-reported comprehension with the effect
of Use of subtitling
Use of subtitling
Never - Rarely
Never - Occasionally
Never - Very frequently/Always
Rarely - Occasionally
Rarely - Very frequently/Always
Occasionally - Very frequently/Always
* indicates significant results (α=0.05)
Mean difference
1.010
.666
.823
-.345
-.188
.157
Std. Error
.320
.270
.278
.236
.243
.174
t
3.15
2.47
2.96
-1.46
-0.77
0.90
df
138
138
138
138
138
138
p-value
0.002*
0.015*
0.004*
0.146
0.441
0.368
Figure 38. Estimated marginal means of Self-reported comprehension with the effect of Use of subtitling
5.3.9.3. Audience enjoyment
The participants’ enjoyment was found to have a significant effect on Self-reported
comprehension (F=3.91; p<0.05). The lowest mean is 3.4 for the participants who said
the clip was boring, and the highest mean is 4.1 for the participants who thought the clip
was great fun. As shown in Figure 39, mean scores for Self-reported comprehension
grow as participants declare higher enjoyment.
Table 77. Estimated marginal means of Self-reported comprehension with the effect of Audience
enjoyment
Enjoyment
Boring
2
3
Great fun
172
Mean
3.404
3.516
3.602
4.093
Std. Error
.261
.141
.128
.173
Chapter 5. Quantitative results
The pairwise contrasts of the estimated means of the self-reported comprehension
scores are presented in Table 78. The mean difference in self-reported comprehension
scores between boring and great fun is 0.69 points (t=-2.40; p<0.05). The participants
who declared they had great fun watching the clips also had a 0.58-point difference with
respect to the participants who picked 2 (t=-3.26; p<0.01) and a difference of 0.49
points with respect to the participants who picked 3 (t=-3.06; p<0.01).
Table 78. Pairwise contrasts of estimated marginal means of Self-reported comprehension with the effect
of Use of subtitling
Enjoyment
Mean difference
Boring – 2
-.113
Boring – 3
-.198
Boring – Great fun
-.689
2–3
-.085
2 – Great fun
-.576
3 – Great fun
-.491
* indicates significant results (α=0.05)
Std. Error
.233
.243
.287
.120
.177
.160
t
-0.48
-0.81
-2.40
-0.71
-3.26
-3.06
df
138
138
138
138
138
138
p-value
0.630
0.417
0.018*
0.479
0.001*
0.003*
Figure 39. Estimated marginal means of Self-reported comprehension with the effect of Audience
enjoyment
5.3.9.4. Narrative attention
Narrative attention was assessed by means of two questions participants were asked
after watching the clip. For each question answered correctly, the participants received
one point. The scores for narrative attention were found to have an effect on the
participants’ self-reported comprehension (F=7.58; p<0.01). Table 79 shows the
estimated means of Self-reported comprehension with the effect of Narrative attention.
The lowest mean self-reported comprehension score is 3.0 for the participants who did
173
The Reception of (Non)Professional Subtitling
not answer any of the questions correctly. The scores for the participants who answered
one or two questions correctly are very similar: 4.0 and 3.8 respectively.
Table 79. Estimated marginal means of Self-reported comprehension with the effect of Narrative
attention
Narrative attention
0
1
2
Mean
3.057
4.020
3.884
Std. Error
.251
.128
.123
Table 80 presents the pairwise contrasts of the mean for each level of Narrative
attention. There are no significant differences between the participants who answered 1
or 2 questions correctly. However, the differences between the participants who did not
answer any question correctly and those who answered one question correctly is 0.96
points (t=-3.89; p<0.001) and the difference between the participants with two correct
answers and those with no correct answers at all is 0.83 points (t=3.426; p<0.01).
Table 80. Pairwise contrasts of estimated marginal means of Self-reported comprehension with the effect
of Narrative attention
Narrative attention
Mean difference
0-1
-.963
1-2
.136
2-0
.827
* indicates significant results (α=0.05)
Std. Error
.247
.109
.241
t
-3.89
1.25
3.43
df
138
138
138
p-value
0.000*
0.213
0.001*
5.3.9.5. Clip and Type of subtitle
An interaction between Clip and Type of subtitle was found for Self-reported
comprehension (F=2.92; p<0.05). The estimated means of Self-reported comprehension
scores are shown in Table 81, as well as graphically in Figure 40.
In the pairwise comparisons, the NP2 version stands out from the other versions
in two cases. When considering the clips as reference factors, in Clip 2 there is a 0.50
difference in the mean self-reported comprehension scores between the NP2 and the
PRO versions (t=-2.31; p<0.05). In Clip 3 there is a significant difference of 0.57 points
between the NP1 and NP2 versions (t=-2.67; p<0.01).
174
Chapter 5. Quantitative results
Table 81. Estimated marginal means of Self-reported comprehension with the interaction effect between
Clip and Type of subtitle
Clip
Clip 1
Clip 2
Clip 3
Type of subtitle
NP1
NP2
PRO
NP1
NP2
PRO
NP1
NP2
PRO
Mean
3.577
3.572
3.844
3.726
3.289
3.794
3.438
4.008
3.635
Std. Error
.196
.186
.184
.183
.194
.189
.200
.192
.204
Table 82. Pairwise contrasts of Self-reported comprehension with the interaction effect between Clip and
Type of subtitle (Clip as reference factor)
Clip
Clip 1
Type of subtitle Mean difference
NP1 – NP2
.005
NP2 – PRO
-.273
PRO – NP1
.268
Clip 2
NP1 – NP2
.437
NP2 - PRO
-.505
PRO - NP1
.069
Clip 3
NP1 - NP2
-.570
NP2 - PRO
.373
PRO - NP1
.197
* indicates significant results (α=0.05)
Std. Error
.219
.211
.216
.223
.219
.210
.214
.215
.220
t
0.02
-1.29
1.24
1.95
-2.31
0.33
-2.67
1.74
0.90
df
138
138
138
138
138
138
138
138
138
p-value
0.982
0.197
0.217
0.053
0.022*
0.744
0.009*
0.084
0.371
Figure 40. Estimated marginal means of Self-reported comprehension with the interaction effect between
Clip and Type of subtitle
5.3.9.6. Self-reported comprehension - Summary
Here is a list of the conclusions that can be drawn from the analysis for Self-reported
comprehension:
175
The Reception of (Non)Professional Subtitling
-
Level of L2/L3 did not have an effect on self-reported comprehension, which is a
rather counter-intuitive result.
-
As expected, the participants who have already seen the clip reported a better
comprehension of the clips.
-
When looking at the self-reported comprehension ratings, HLE participants who
Never use subtitles also had the best comprehension among all participants.
There were no significant differences between the ratings given by the other
participants.
-
Audience enjoyment correlated with Self-reported comprehension: higher ratings
for Self-reported comprehension coincide with higher ratings for enjoyment.
-
Those participants who answered the narrative attention questions correctly also
reported a higher comprehension of the content of the clip.
-
Strangely, the lowest Self-reported comprehension mean rating for NP2 occurs
in Clip 2. Among the mean self-reported comprehension scores of Clip 2, the
mean of NP2 is significantly lower than the mean of PRO. On the other hand, in
Clip 3 NP2 takes the first place and is significantly higher than NP1, which is in
the lowest place.
5.3.10. Reception capacity
Reception was measured on a scale from 1 to 7, combining the results obtained from the
questions concerning general comprehension, Iconic attention, Narrative attention and
Verbal attention (these three variables are considered merely as a support for the
Reception capacity variable and are explored individually in the subsequent sections).
The model constructed for Reception capacity is shown in Table 83. Three variables
were found to have a significant effect: Clip (F=6.66; p<0.01), Type of subtitle (F=5.85;
p<0.01) and Prior knowledge of the clips (F=13.82; p<0.001). Once again, there was no
significant effect for Level of L2/L3.
Table 83. Final model with significant main effects of Reception capacity
Predictor
F
Numerator df
Clip
6.66
2
Subs
5.85
2
Watched
13.82
1
* indicates significant results (α=0.05)
176
Denominator df
150
150
150
p-value
0.002*
0.004*
<0.001*
Chapter 5. Quantitative results
5.3.10.1. Clip
The Clip variable was found to have a significant effect on Reception capacity
(F=6.663; p<0.01). The estimated marginal means for the three clips are shown in Table
84. The highest mean score is 6.1 for Clip 1, and the lowest is 5.4 for Clip 2.
Table 84. Estimated marginal means of Reception capacity with the effect of Clip
Clip
Clip 1
Clip 2
Clip 3
Mean
6.139
5.468
5.633
Std. Error
.149
.148
.150
Table 85 shows the pairwise contrasts of the estimated marginal means. The
mean for Clip 1 is significantly different from those of the other two clips. There is a
0.67-point difference between Clip 1 and Clip 2 (t=3.50; p<0.01) and the difference
between Clip 1 and Clip 3 is 0.51 points (t=-2.64; p<0.01). A graphical representation
of the means is included in Figure 41.
Figure 41. Estimated marginal means of Reception capacity with the effect of Clip
Table 85. Pairwise contrasts of estimated marginal means of Reception capacity with the effect of Clip
Clips
Mean difference
Std. Error
Clip 1 - Clip 2
.672
.192
Clip 2 - Clip 3
-.165
.192
Clip 3 - Clip 1
-.507
.192
* indicates significant results (α=0.05)
t
3.50
-0.86
-2.64
df
150
150
150
p-value
0.001*
0.392
0.009*
5.3.10.2. Type of subtitle
A significant effect was found for Type of subtitle on Reception capacity (F=5.85;
p<0.01). The estimated means for each type of subtitle are shown in Table 86. The
177
The Reception of (Non)Professional Subtitling
lowest mean is 5.37 points for the NP2 version, and the means for PRO and NP1 are
very similar: 5.91 and 5.96 respectively.
Table 86. Estimated marginal means of Reception capacity with the effect of Type of subtitle
Type of subtitle
NP1
NP2
PRO
Mean
5.958
5.367
5.915
Std. Error
.153
.146
.149
Table 87 presents the pairwise comparisons of the mean reception capacity score
for each type of subtitle. The results for NP2 are significantly different from the results
for NP1 and PRO. There is a 0.59 difference between NP1 and NP2 (t=3.06; p<0.01),
while the difference between NP2 and PRO is 0.549 (t=-2.86; p<0.01). Figure 42 shows
a graphical representation of the means.
Figure 42. Estimated marginal means of Reception capacity with the effect of Type of subtitle
Table 87. Pairwise contrasts of Reception capacity with the effect of Type of subtitle
Type of subtitle
Mean difference
NP1 - NP2
.591
NP2 - PRO
-.549
PRO - NP1
-.043
* indicates significant results (α=0.05)
Std. Error
.193
.192
.192
t
3.06
-2.86
-0.22
df
150
150
150
p-value
0.003*
0.005*
0.824
5.3.10.3. Prior knowledge of the clips
Reception capacity scores are significantly affected by the participants’ previous
knowledge of the clips (F=13.81; p<0.001). Table 88 presents the mean reception
178
Chapter 5. Quantitative results
capacity scores according to the participants’ previous knowledge of the clips. The
participants who had seen the clips before scored 6.1 on average, while those who
watched the clip for the first time scored 5.4 (t=3.72; p<0.001).
Table 88. Estimated marginal means of Reception capacity with the effect of Prior knowledge of the clips
Watched
Yes
No
Mean
6.114
5.379
Std. Error
.170
.104
5.3.10.4. Reception capacity - Summary
The conclusions that can be drawn from the analysis of Reception capacity are
summarized as follows:
-
Surprisingly, unlike Audience enjoyment, Reception capacity was not dependent
on Level of L2/L3.
-
The understanding of the three clips varied: Clip 1 seems to be significantly
easier to understand than the other two.
-
Reception capacity is affected by Type of subtitle. The lowest mean reception
capacity score is that of NP2, the Iberian non-professional version. There was no
significant difference between the means of NP1 and PRO.
-
The participants who had seen the clip before the experiment had a better
reception capacity score.
5.3.11. Iconic attention
Given that Iconic attention (as well as the other two attention-specific questions)
consists of two questions that were asked to the participants, it was treated as a variable
with a binomial distribution using a generalized linear mixed model. Additionally, the
percentage of fixations on the subtitle area was included as a covariate in the model
because a significant main effect was detected. Only the Clip variable was found to
have a significant effect on Iconic attention (F=23.35; p<0.001).
5.3.11.1. Clip
Table 89 shows the estimated means for Iconic attention with the effect of Clip. Clip 1
and Clip 3 show high scores, with averages of 86% and 84% of correct answers
respectively. The average percentage of correct answers for Clip 2 is only 51%. Table
90 shows that the average correct answers for Clip 1 and Clip 3 are significantly
179
The Reception of (Non)Professional Subtitling
different from the average of correct answers for Clip 2. This indicates that the
participants answered only half of the Iconic attention questions of Clip 2 correctly.
Table 89. Estimated marginal means of Iconic attention with the effect of Clip
Clip
Clip 1
Clip 2
Clip 3
Mean
.856
.519
.837
Std. Error
.031
.043
.032
Table 90. Pairwise contrasts of estimated marginal means of Iconic attention with the effect of Clip
Clips
Mean difference
Std. Error
Clip 1 - Clip 2
.337
.052
Clip 2 - Clip 3
-.317
.053
Clip 3 - Clip 1
-.019
.043
* indicates significant results (α=0.05)
t
6.48
-6.00
-0.44
df
153
153
153
p-value
<0.00001*
<0.00001*
0.658
5.3.11.2. Iconic attention - Summary
The conclusions that can be drawn from the analysis for Iconic attention are
summarized as follows:
-
Neither Level of L2/L3 nor Type of subtitle have a significant effect on Iconic
attention.
-
Iconic attention was affected by the Clip variable only. Clip 2 had a significantly
lower score than the other two clips.
5.3.12. Narrative attention
The generalized linear mixed model constructed to test the significant effects and
interactions on Narrative attention shows significant effects on Prior knowledge of the
clips (F=5.10; p<0.05) and Subtitle-reading effort (F=2.78; p<0.05). The model also
includes the significant effect of Percentage of duration of fixations on the subtitle area
as a covariate (F=5.22; p<0.05).
5.3.12.1. Prior knowledge of the clips
The participants’ previous knowledge of the clips had a significant effect on their
answers to the Narrative attention questions. Table 91 shows the estimated means for
the variable. The participants who had seen the clips before answered on average 84%
of the questions correctly, while the participants who had not watched the clips before
answered only 71% of them correctly. The difference of 12.6 points turned out to be
significant (t=2.55; p<0.05).
180
Chapter 5. Quantitative results
Table 91. Estimated marginal means of Narrative attention with the effect of Prior knowledge of the clips
Watched
Yes
No
Mean
.838
.712
Std. Error
.043
.034
5.3.12.2. Subtitle-reading effort
Subtitle-reading effort also had a significant effect on the participants’ answers to the
narrative attention questions (F=2.78; p<0.05). Table 92 shows the mean values for the
variables. The participants who said the subtitles were easy to follow, with a score of 5,
answered 88% of the questions correctly. The lowest percentage of correct answers is
for participants who rated the subtitles with 4: only 68% of their answers were correct.
Strangely, the participants who gave the subtitles a 3 answered 73% of the questions
correctly, but participants who rated it with a 2 achieved 78% of correct answers. The
pairwise comparisons are shown in Table 93. There are significant differences in the
average percentage of right answers between the participants who rated the subtitles
with 5 and those who gave the subtitles a 3 (t=-2.01; p<0.05), as well as between the
participants who gave the subtitles a 4 and those who considered it was very easy to
follow, with a 5 (t=-3.18; p<0.01).
Table 92. Estimated marginal means of Narrative attention with the effect of Subtitle-reading effort
Subtitle-reading effort
2
3
4
5
Mean
.785
.735
.682
.883
Std. Error
.066
.057
.045
.044
Table 93. Pairwise contrasts of estimated marginal means of Narrative attention with the effect of
Subtitle-reading effort
Subtitle-reading effort
Mean difference
2–3
.050
2–4
.102
2–5
-.098
3–4
.052
3–5
-.148
4–5
-.200
* indicates significant results (α=0.05)
Std. Error
.078
.066
.078
.064
.074
.063
t
0.64
1.56
-1.23
0.82
-2.01
-3.18
df
137
137
137
137
137
137
p-value
0.520
0.122
0.209
0.414
0.046*
0.002*
5.3.12.3. Narrative attention - Summary
The main highlights resulting from the analysis of Narrative attention are summarized
as follows:
181
The Reception of (Non)Professional Subtitling
-
Level of L2/L3 and Type of subtitle have no significant effect on Narrative
attention.
-
The participants who had seen the clip before the experiments answered more
answers correctly than those who watched it for the first time.
-
The participants who rated the subtitles as very easy to follow (5) also had a high
Narrative attention score. Strangely, the lowest Narrative attention score is that
of the participants who rated subtitle-reading difficulty at 4.
5.3.13. Verbal attention
As in the two previous cases, a generalized linear mixed model was created for Verbal
attention. Significant effects were found on Prior knowledge of the clips (F=9.10;
p<0.01), Type of subtitle (F=9.08; p<0.001) and Clip (F=5.83; p<0.01).
5.3.13.1. Prior knowledge of the clips
The participants’ previous knowledge of the clips had a significant effect on their
Verbal attention scores. The participants who had watched the clips before scored 96%
of their answers correctly, while the participants who were watching the clips for the
first time scored only 81% of them correctly. Thus, there was a mean difference of 15
points (t=4.15 p<0.0001).
Table 94. Estimated marginal means of Verbal attention with the effect of Prior knowledge of the clips
Watched
Yes
No
Mean
.964
.813
Std. Error
.020
.033
5.3.13.2. Type of subtitle
The types of subtitles had a significant effect on the scores for verbal attention (F=9.08;
p<0.001). Table 95 shows the mean values that indicate that, after watching the videos
with NP1 and PRO subtitles, the participants scored between 93.6% and 94.8% correct
answers, while with NP2 they scored 84% of correct answers only. The pairwise
comparisons in Table 96 present the significant differences between NP1 and NP2
(t=2.91; p<0.01) and between NP2 and PRO (t=-3.07; p<0.01).
182
Chapter 5. Quantitative results
Table 95. Estimated marginal means of Verbal attention with the effect of Type of subtitle
Type of subtitle
NP1
NP2
PRO
Mean
.936
.826
.948
Std. Error
.024
.047
.021
Table 96. Pairwise contrasts of Verbal attention with the effect of Type of subtitle
Type of subtitle
Mean difference
NP1 - NP2
.110
NP2 - PRO
-.121
PRO - NP1
.011
* indicates significant results (α=0.05)
Std. Error
.038
.039
.022
t
2.91
-3.07
0.52
df
150
150
150
Adj. Sig,
0.004*
0.003*
0.607
5.3.13.3. Clip
The model for Verbal attention also found a significant effect of Clip (F=5.83; p<0.01).
Table 97 shows the mean scores of Verbal attention with the effect of Clip. On average,
the participants answered correctly 94.4% of the questions of Clip 2 and 93% of the
questions of Clip 1, but only 85% of the questions of Clip 3. In both cases, the
difference between Clip 3 and the other two clips is statistically significant (p<0.05).
Table 97. Estimated marginal means of Verbal attention with the effect of Clip
Clip
Clip 1
Clip 2
Clip 3
Mean
.930
.944
.850
Std. Error
.026
.022
.045
Table 98. Pairwise contrasts of Verbal attention with the effect of Clip
Clip
Mean difference
Clip 1 - Clip 2
-.013
Clip 2 - Clip 3
.094
Clip 3 - Clip 1
-.081
* indicates significant results (α=0.05)
Std. Error
.023
.037
.035
t
-0.57
2.55
-2.29
df
150
150
150
Adj. Sig,
0.567
0.012*
0.023*
5.3.13.4. Verbal attention - Summary
The analysis carried out to identify the main effects on Verbal attention has drawn the
following main findings:
-
Level of L2/L3 has no significant effect on Verbal attention.
-
The participants who had seen the clips before answered more questions
correctly than the participants who watched the clips for the first time.
-
NP2 subtitles, the Iberian non-professional subtitles, yielded significantly lower
Verbal attention scores than the other two types of translation.
183
The Reception of (Non)Professional Subtitling
-
The questions asking about verbal information were harder for Clip 3 than for
the other two Clips.
5.3.14. Gender
One of the main concerns raised by the sample was the imbalance between men and
women in the HLE group. While the LLE group comprised 14 women and 12 men, the
HLE group was made of 21 women and 5 men. As mentioned in Section 4.3.1, this was
mainly due to the gender imbalance among the students enrolled in the BA in English
(25% men and 75% women). Half of the HLE participants came from this degree
program. Considering this situation and in order to check for any possible effect, the
Gender variable was included in all the models in the initial stages as part of the
variable selection process. However, it did not produce a significant effect on any of the
variables.
5.4.
Quantitative analysis summary
Table 99 below presents a summary of the main effects and interactions found in the
statistical analyses. The Type of subtitle variable was found to have a significant effect
on Mean fixation duration on the image and the subtitle area and Attention shift ratio.
Mean fixation durations were shorter with the PRO version of the subtitles. Type of
subtitle also had an impact on Reception capacity, caused mainly by Verbal attention.
The participants had significantly lower scores with one of the non-professional subtitle
versions.
Level of L2/L3 was found to produce significant effects on all eye-tracking
measurements except Mean fixation duration on the subtitle area. HLE participants
made fewer fixations and looked at the subtitle area for a shorter time. They also
skipped more subtitles, thus shifting less between the image and the subtitle areas.
Further, their fixations on the image area were significantly longer. Among the selfreported variables, significant effects were found only for Audience enjoyment: the HLE
participants’ scores for Audience enjoyment were higher than the scores given by LLE
participants. Interestingly, no significant effects were found for Reception capacity or
the different types of attention questions. The differences in behavior that were found to
184
Chapter 5. Quantitative results
be dependent on the participants’ level of proficiency in the source language are
covered in Section 6.2.1.
Audience enjoyment also correlated with Subtitle-reading effort: the participants
who said they enjoyed the clip also claimed the subtitles were easier to follow. The
results indicate that the participants regular Use of subtitling is one of the most
important factors determining their behavior. Significant effects of Use of subtitling
were found for all eye-tracking variables, except for Mean fixation duration on the
image area. The group of participants who Never use subtitles, made up entirely of
participants with a HLE, was found to exhibit a behavior different from all other
participants. The significant effect found for Use of subtitling is explained in detail and
supported with interview data in Section 6.2.4. Significant effects were also found for
the Clip variable, mainly due to a higher cognitive load caused by Clip 2. The
unexpected effects for Clips are revised in Section 6.2.3.
×
×
×
×
×
Verbal attention
Narrative attention
Iconic attention
Reception capacity
Self-reported comprehension
×
×
Audience enjoyment
×
Subtitle-reading effort
×
×
Attention shift ratio
×
×
Mean fixation duration on the image area
Skipped subtitles
×
×
Mean fixation duration on the subtitle area
Percentage of fixations on the subtitle area
Main effects and interactions
Types of Subtitles
Level of L2/L3
Use of subtitling
Narrative attention
Subtitle-reading effort
Audience enjoyment
Clips
Order of presentation
Prior knowledge of the clips
Level of L2/L3 and Use of subtitling
Level of L2/L3 and Prior knowledge of the
clips
Clip and Type of subtitle
Total fixation duration on the subtitle area
Dependent variable
Table 99. Summary of dependent variables with main effects and interactions marked as ×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
185
Chapter 6. Qualitative data and discussion
In the previous chapter I described the quantitative data collected for this study and the
results obtained from the statistical analysis. In this chapter I use those results to test the
hypotheses defined in Section 4.1. The results from the hypothesis testing are covered in
Section 6.1. In Section 6.2 I draw on the eye-tracking and interview data to illustrate
some additional relevant findings of the study. This section will go deeper into specific
cases encountered during the analysis that proved to provide interesting insights into the
participants’ behavior and to complement the picture of professional and nonprofessional subtitling consumption. In Section 6.3 I turn to qualitative data to explain
the extent of the control the participants have over their experience as users. In this
section I comment on the participants’ access to audiovisual content and their use of
different kinds of audiovisual translation.4
6.1.
Hypothesis testing
6.1.1. Hypothesis 1: Type of subtitle
The first hypothesis (H1) posits that participants’ comprehension scores will be higher
with professional rather than non-professional subtitles.
This hypothesis is grounded on the fact that non-professional subtitling is
commonly deemed to be low-quality translation that cannot be compared with
professional subtitling (Dwyer and Uricaru 2009; La Forgia and Tonin 2009; Bogucki
2009). The hypothesis assumes that the differences in production conditions and quality
between professional and non-professional subtitles will have an effect on the
participants’ reading behavior and understanding of the audiovisual content.
Bearing in mind the different levels of reception on which professional and nonprofessional subtitles can be compared and contrasted, I formulated seven
sub-hypotheses. In the following subsections I address each of these sub-hypotheses and
then present the global results for H1.
4
Segments of the qualitative analysis that are included here reproduce a section published in OrregoCarmona (2014a).
187
The Reception of (Non)Professional Subtitling
6.1.1.1. H1.1: Reception capacity is higher with professional subtitles.
The Type of subtitle variable was determined to have a significant main effect on
Reception capacity (F=5.85; p<0.01). Nevertheless, the hypothesis is only partially
confirmed. The mean reception capacity with PRO was higher than the mean reception
capacity with NP2 (PRO=5.91 vs. NP2=5.37. t=-2.857; p<0.01); however, it was not
significantly higher than NP1, which had a mean of 5.96 (t=-0.22; p=0.82).
The results present a counterintuitive finding. NP1 was produced by a nonprofessional Latin-American subtitling community translating into neutral Spanish,
while NP2 was produced in Spain by a community that uses Iberian Spanish. By
reading the subtitles, it is possible to tell the geographical provenances, although the
participants did not point this out during the interviews. There are two aspects that can
be inferred from these results. First, since the results for NP1 and PRO were not
significantly different, it is possible to assume that these versions are simply clearer to
understand than NP2. Second, since the PRO version, which is also localized for the
Spanish market, produced higher scores than NP2, it could be argued that localization in
general is not what causes the lower scores. Rather, it could be the case that NP2
subtitles are somehow less accomplished than the other two and fail to convey the
message as effectively.
Reception capacity is in turn a variable composed of another set of variables:
Iconic, Narrative and Verbal attention. Among these variables, Type of subtitle has a
significant effect on Verbal attention (F=9.076; p<0.001). The same effect found on
Reception capacity is found here. When the participants answered the Verbal attention
questions after watching the PRO version, they answered them correctly in 94.8% of the
cases. In the case of NP1, they provided the correct answer in 93.6% of cases. However,
with NP2, they answered only 82.6% of the questions correctly, which is significantly
lower than the other two.
6.1.1.2. H1.2: Subtitle-reading effort is lower with professional subtitles.
Type of subtitle was not determined to have a significant main effect on Subtitle-reading
effort (F=0.60; p=0.55). The findings thus do not confirm this hypothesis. However, an
interaction effect between Clip and Type of subtitle was identified (F=3.10; p<0.05).
A similar pattern to the one found in Reception capacity was found for this
variable. NP1 did not differ significantly from the other two subtitles. When comparing
the differences between PRO (4.32) and NP2 (3.57), it can be seen that, in Clip 1, the
188
Chapter 6. Qualitative data and discussion
mean score for PRO is 0.75 points higher than the score for NP2 (p<0.01), but the case
is reversed for Clip 3, in which the score for NP2 (4.25) is 0.65 points higher than the
score for PRO (3.59).
When the clips are taken as reference factor, the best score for NP2 is found in
Clip 3, which is approximately 0.6 points higher than the score for NP2 in the other two
clips. The only additional significant difference is present in PRO. The score PRO
receives in Clip 3 is 0.72 points lower than the score it receives in Clip 1. These results
seem to indicate the importance of the audiovisual material in the study. The design of
the study aimed to select videos that were comparable. Nevertheless, given the complex
nature of audiovisual content, it is only natural that there are effects produced by the
clip. I will address these issues in an all-inclusive analysis of the audiovisual material in
Section 6.2.3.
6.1.1.3. H1.3: Self-reported comprehension is greater with professional subtitles.
This hypothesis is based on the rationale that viewers can identify bad quality
translation or at least notice the different degrees of assistance that various types of
subtitles offer. The findings from the statistical model do not provide evidence to
confirm this hypothesis, since no significant main effect of Type of subtitle on Selfreported comprehension was found (F=1.54; p=0.21).
Similarly to Subtitle-reading effort, there was a significant effect of the
interaction between Clip and Type of subtitle (F=2.92; p<0.05). However, the results
scarcely follow an identifiable pattern. There are two pairwise comparisons that show
significant differences and both of them involve NP2. In Clip 2, there is a significant
difference between NP2 and PRO (t=-2.31; p<0.05), with the PRO version receiving 0.5
more points on average than NP2. On the contrary, in Clip 3, the ratings for NP2 are
0.57 points higher than those for NP1 (t=-2.67; p<0.01).
As a general conclusion on this hypothesis, it could be assumed that the type of
subtitles did not influence the participants’ rating for their self-reported comprehension.
Other factors, such as their reported enjoyment, their use of subtitles and, intuitively,
their Prior knowledge of the clips were determined to be significant effects of Selfreported comprehension. However, self-reported measurements are known to be
troublesome due to the possibility that participants overestimate their understanding of
the audiovisual translated content (Antonini 2005; Bucaria and Chiaro 2007; Antonini
189
The Reception of (Non)Professional Subtitling
2008) or underestimate their declared understanding of the content, as shown in Caffrey
(2009).
6.1.1.4. H1.4: With professional subtitles, more attention is allocated to the image area.
As explained in Section 5.3.2.3, Percentage of fixations and Total fixation duration on
the image and on the subtitle areas are complementary and the statistical models
produce the same results. To test this hypothesis, I use the time during which
participants’ visual attention is allocated to the subtitle area, located at the bottom of the
screen. Based on the eye-tracking data, this hypothesis cannot be confirmed. Total
fixation duration on the subtitle area was not significantly shorter for the professional
subtitles. Similarly, the Percentage of fixations on the subtitle area was not significantly
shorter for the professional subtitles either. In the HLE group, the mean percentage of
fixations on the subtitle area was 46% in the PRO version, 43% in the NP1 version and
44% in the NP2 version. Among LLE participants, the mean percentage of fixations on
the subtitle area for PRO was 61%, while the NP1 and the NP2 had mean percentages of
62% and 64% respectively.
By comparing the three subtitle conditions, I tested whether any of them
demanded a higher percentage of attention allocation. Had the results shown that the
differences between the means were significant, this would have indicated that a given
set of subtitles was more cognitively demanding than the other subtitles. When
watching subtitled audiovisual products, subtitles have to compete for the attention
against the other sources of information: sound and image. Both the acoustic and visual
channels can offer verbal and non-verbal information; like those two channels, the
subtitles also demand a large share of the cognitive resources from viewers. The amount
of attention that viewers have to allocate to the subtitles in relation to the other sources
of information is considered an indicator of the cognitive effort they entail. Whenever
some subtitles demand more time or require a higher number of fixations than others, it
can be assumed that those subtitles are more cognitively demanding (Saldanha and
O’Brien 2013; Kruger et al. 2015).
6.1.1.5. H1.5: Mean fixation duration is shorter with professional subtitles.
As in the previous case, the hypothesis stated that the professional subtitles would have
shorter mean fixations than the non-professional subtitles. According to the results
presented in Section 5.3.4, this hypothesis is confirmed. The main effect of Type of
190
Chapter 6. Qualitative data and discussion
subtitle (F=26.97; p<0.001) and the interaction between Clip and Type of subtitle
(F=5.83; p<0.01) are significant for Mean fixation duration on the subtitle area.
Additionally, Type of subtitle was found to have a significant main effect on Mean
fixation duration on the image area (F=5.52; p<0.01).
The results indicate that the mean on the PRO version (178.41 ms) is significantly
shorter than the mean fixations on both NP1 and NP2 subtitles (196.69 ms and
201.84 ms). Mean fixation duration is commonly regarded as an indicator of cognitive
effort (d’Ydewalle and Bruycker 2007; Perego et al. 2010; Kruger et al. 2015) not only
in studies within Translation Studies but also in reading studies. Rayner and Pollatsek
(2006) argue that reading studies have provided solid evidence to accept that longer
fixations on words are associated with higher difficulty to understand them.
The mean fixation duration estimated with the effect of Type of subtitle concords
with the mean fixation duration on subtitles reported by d’Ydewalle and Bruycker
(2007) for adult viewers watching subtitled material: 178 ms for one liners and 179 ms
for two liners. Kruger et al. (2015) reported a mean fixation of 186.55 ms among
hearing viewers watching subtitles presented at a speed of 12 characters per second,
which is a faster speed than the speed of all the subtitles included in this study.
Interestingly enough, the mean fixations reported in other studies are still longer than
the means I found for the non-professional subtitles in my experiment. The mean
fixation reported by Caffrey (2009:163) is 237 ms for one-line subtitles and 201.43 ms
for two-line subtitles. Perego et al. (2010:259) found a 221-ms mean fixation while
Künzli and Ehrensberger-Dow (2011) reported a mean of 0.36 seconds. Among all the
studies above, only d’Ydewalle and Bruycker (2007) and Künzli and Ehrensberger-Dow
(2011) include participants who are supposed to be used to subtitling, since this is the
most widely used audiovisual translation modality in Flemish-speaking Belgium and
German-speaking Switzerland (Media Consulting Group 2011). On the other hand,
Perego et al. (2010) present an experiment carried out in Italy, primarily a dubbing
country, so their results should be more similar to mine. Although the mean fixations on
the non-professional versions in my study are very similar to theirs, the mean fixation in
the PRO version is shorter.
A more detailed analysis can be based on the interaction between Type of subtitle
and Clip. It shows that PRO differs significantly from the other types of subtitles only in
the first two clips and not in the third. This could indicate that participants become used
to the subtitles over time or their behavior stabilizes as a result of immersion in the
191
The Reception of (Non)Professional Subtitling
whole viewing experience. However, it could also be a clip-influenced effect. With only
three short clips in this experiment, it is impossible to test this any further.
The findings from the variable Mean fixation duration on the subtitle area seem
to support the connection between mean fixation duration on the subtitles and cognitive
effort. When taking the types of subtitles as reference factor, Clip 2 differs significantly
from Clip 3 in the NP1 version and from the other two clips in the PRO version. The
tests of Reception capacity, Iconic attention and Attention shift ratio show the
participants’ efficiency with Clip 2 was lower than with the other two. This hints at
greater difficulty to understand Clip 2, which causes a lower-level performance. Given
the possibilities this discussion offers, I will address it in a holistic manner in Section
6.2.3.
Interestingly, Type of subtitle also influenced the mean fixation on the image
area. Again, the shortest mean fixation was found in PRO (333.55 ms), which is
significantly different from the mean fixation duration on the image for NP1, 357 ms.
NP2 had an estimated mean fixation of 346.59 ms and is not significantly different from
any of the other two means. The fact that subtitles affect the visualization process as a
whole has already been shown in previous research. In one study, Bairstow (2011)
found that for the participants who relied on the subtitles (monolingual participants who
did not understand the original language), the subtitles acted as a facilitating factor for
the understanding of visual information. Instead of obtaining lower scores on the visual
attention questions due to the amount of time dedicated to the subtitle area, the
participants who relied on the subtitles obtained higher scores on visual information
perception than the bilingual participants who were assumed to focus more on the image
(Bairstow 2011). However, the fact that different types of subtitling affect the viewing
process in different ways highlights the importance of the subtitles. Since other eyetracking measurements are not affected by Type of subtitle, I suspect this difference in
mean fixation duration is more related to the format of the subtitles and their appearance
on the screen. I comment on this in Section 6.2.5.
6.1.1.6. H1.6: Fewer subtitles are skipped when participants are watching professional
subtitles.
The results did not provide any evidence to confirm this hypothesis. No significant
effect for Type of subtitle (F=2.49; p=0.087) was found on Skipped subtitles. Subtitle
speed imposes a cognitive demand on viewers. If the subtitles are too fast, it is more
192
Chapter 6. Qualitative data and discussion
likely that more subtitles will not be fixated upon. However, the subtitles included in
this study did not vary much in terms of subtitle speed. The PRO version has an average
speed of 10.7 characters per second, while the NP1 and NP2 version are presented at
10.1 characters per second and 10.9 characters per second on average.
6.1.1.7. H1.7: Attention shift varies depending on type of subtitle.
Attention shifts are calculated per subtitle and indicate the number of times the viewer’s
gaze shifts from the image to the subtitles and from the subtitles to the image. Type of
subtitle was found to produce a significant effect on Attention shift ratio (F=7.32;
p<0.01). The hypothesis is stated without any direction because the increase or decrease
of attention shifts does not necessarily mean the reading process is smoother.
The mean attention shift for the PRO version was 2.01 and was significantly
higher than for the other two. The means for the non-professional versions were 1.88 for
NP1 and 1.81 for NP2. Attention shifts should be understood on the premise that, in a
linear reading behavior, each subtitle should cause two attention shifts: one shift in
which the gaze moves from the image to the subtitle area and another one from the
subtitle area to the image.
Kruger et al. claim that “the fewer the gaze shifts, the more fluent the reading and
vice versa” (2015:n.p.). However this is not necessarily the only possibility. Assuming
subtitles provide viewers with necessary time to read the subtitle and check the image,
there should always be a minimum number of attention shifts required to achieve that
goal: two shifts per subtitle. A higher ratio could indicate re-reading, which is normally
taken as a sign of greater difficulty in reading (Krejtz et al. 2013). On the other hand,
lower scores could imply a large amount of Skipped subtitles.
Yet another element should be considered. The Code of Good Subtitling Practice
(Carroll and Ivarsson 1998) states that a minimum of four frames should be left between
subtitles in order to allow the eyes of the viewer to recognize there is a new subtitle on
the screen. In the PRO subtitles used in this experiment, the minimum time left between
two given subtitles was 160 ms. This spotting is probably a guideline from the studio
and might have been controlled by the use of templates, as confirmed by private
communication with a translator who has participated in the translation of The Big Bang
Theory for Spain (Warner Bros 2010). However, this rule was not followed by the nonprofessional subtitling communities included in this study. For instance, the median for
the time between one subtitle and the next was found to be as short as 20 ms for NP1 in
193
The Reception of (Non)Professional Subtitling
Clip 2, including many instances of only 10 ms between subtitles. This indicates that
subtitles virtually overlap to the human eye. Viewers who become aware of the change
can read the second overlapping subtitle without going back to the image area, thus
reducing the number of shifts and causing them to remain on the subtitle area for longer
periods of time.
Under these conditions, even if the number of attention shifts is low, the subtitles
are disturbing the regular process of simultaneously checking the image and the
subtitles, which affects the behavior of participants. Figure 43 shows an 8.6-second
representation of the gaze path of five LLE participants. The representation extends
over three subtitles, but given the little time span between subtitles, some of the
participants do not look at the image until they have read the very last subtitle. Only one
participant makes six attention shifts, and another two make two attention shifts. In this
scenario, it could be said that the PRO version, with an average attention shift ratio of
2.01, is a better indicator of a more homogeneous behavior in which participants start
reading the subtitles and “glance at the video action after they finish reading” (Jensema
et al. 2000b:284).
Figure 43. Example of attention shifts when subtitles overlap
6.1.1.8. Conclusion on H1: Type of subtitle
The results for H1 and its sub-hypotheses are summarized in Table 100. The data do not
provide enough evidence to confirm H1. Only two of the seven postulated sub-
194
Chapter 6. Qualitative data and discussion
hypotheses are confirmed: 1) The PRO version resulted in shorter mean fixation
duration on both the subtitle and the image areas, and 2) the ratio of attention shifts
varied depending on the subtitles. In fact, the attention shift ratio indicates that, on
average, professional subtitles allow participants to have a more regular reading
behavior.
The other eye-tracking measurements, the Attention allocation and the percentage
of Skipped subtitles, did not provide evidence that the reception scores were higher for
the professional version. Further, the self-reported measurements related to Subtitlereading effort and Comprehension did not have a significant effect of Type of subtitle
and seem to be more affected by the nature of the audiovisual material. Perhaps the
most surprising finding is that Reception capacity was affected by Type of subtitle but
only one of the non-professional subtitle versions, the Iberian Spanish one, was
significantly different from the professional version. The other non-professional
version, the Latin-American one, which is supposed to be formulated in neutral Spanish,
obtained results that are not significantly different from the professional version.
Table 100. Summary of the hypothesis testing for H1
Hypothesis
H1.1
Variable
Reception capacity
H1.2
Subtitle-reading effort
H1.3
H1.4
Self-reported
comprehension
Attention allocation
H1.5
Mean fixation duration
H1.6
Skipped subtitles
H1.7
Attention shifts
Confirmation
() This hypothesis is only partially confirmed. The Reception
capacity scores for the PRO version are higher than NP1,
but they were not significantly different from NP2. The
effect seems to be driven by the significant differences in
the Verbal attention questions.
Type of subtitle does not have a significant effect on
Subtitle-reading effort. Differences can be observed in the
results from the interaction between Clip and Type of
subtitle, but they do not follow a pattern.
The analyses did not find a significant effect of Type of
subtitle on self-reported comprehension.
Participants’ Total fixation duration and Percentages of
fixation duration on the subtitle area are not lower with the
professional subtitles.
Mean fixation durations are longer for the non-professional
subtitle versions.
Mean fixation durations on the image are also found to be
shorter in the PRO version.
All subtitles have very similar speed and the results do not
indicate a significant effect for Type of subtitle on the
percentage of Skipped subtitles.
The average number of Attention shifts with the PRO
version is higher, which could be an indicator of a more
homogeneous viewing process in which the participants are
able to continuously shift between the image and the
subtitles.
195
The Reception of (Non)Professional Subtitling
It cannot be said that viewers achieve higher comprehension with the professional
subtitles used in this study. However, the findings suggest there is an impact of the
subtitle spotting and their layout on the eye-tracking measurements that benefits
professional subtitling.
6.1.2. Hypothesis 2: Level of L2/L3
The second hypothesis (H2) for my study thus states: Participants with a high level of
proficiency in L2/L3 will be less dependent on subtitles than will participants with a
low level of L2/L3. It finds its grounding in previous studies (Lavaur and Nava 2008;
Lavaur and Bairstow 2011; Bairstow and Lavaur 2012) that comment on the distracting
effect that subtitles could have on viewers who have a high level of proficiency in the
source language. Likewise, these studies, like most studies on eye-tracking and
subtitling, are based on d’Ydewalle and colleagues’ conclusions about the almost
automatic task of reading subtitles (d’Ydewalle and Gielen 1992; d’Ydewalle and
Bruycker 2007).
In order to test this hypothesis, I postulated three sub-hypotheses exploring the
relation between Attention allocation, Mean fixation duration and Audience enjoyment.
6.1.2.1. H2.8: Participants with a high level of L2/L3 allocate less attention to the
subtitle area.
Attention allocation is tested through two variables: Total fixation duration on the
subtitle area and Percentage of fixations on the subtitle area. In both cases the results
show that Level of L2/L3 has a significant effect on these variables, thereby confirming
the hypothesis.
The difference between Total fixation duration on the subtitle area is highly
significant (p<0.0001). The estimated mean for the HLE participants is shorter than the
estimated mean for LLE participants (t=5.60, p<0.001). When it comes to Percentage of
fixations on the subtitle area, there is a 15.7-point difference between the estimated
means of the two groups (t=3.50; p<0.05). HLE participants made 43.3% of the
fixations on the subtitles, while the participants in the LLE group made 59% of the
fixations on the same area.
In terms of the Percentage of duration of fixations on the subtitle area, HLE
participants spent 32% of their time looking at the subtitles, while LLE did so for 51%
of the time. The mean fixation percentages of time spent on the subtitle area for the two
196
Chapter 6. Qualitative data and discussion
groups are similar to other percentages reported by previous researchers. Among studies
that report the percentage of time participants allocate to the subtitle there nevertheless
tends to be considerable variation. d’Ydewalle and Bruycker (2007) report a mean
percentage of 31% when adults were reading one-line subtitles, and 37% when they
were reading two-line subtitles. Caffrey (2009) reports that participants spent 42% of
the time on the subtitle area for one-liners and 56% for two-liners. Perego et al. (2010)
found that participants spent 67% of the time looking at the subtitles. The lowest
percentage is 31.5%, under mainstream subtitle circumstances, reported by Künzli and
Ehrensberger-Dow (2011).
6.1.2.2. H2.9: Mean fixation duration varies depending on the level of L2/L3.
Mean fixation durations were processed and tested independently for the image area and
the subtitle area. Level of L2/L3 was included in the model for Mean fixation duration
on the subtitle area at the initial stage of the model construction, but no significant
effect was found and, since it was affecting the results for other variables, it was
removed from the final model. Based on the results for the subtitle area, there is no
evidence to confirm this hypothesis.
On the other hand, when running the model for Mean fixation duration on the
image area, the effect of Level of L2/L3 was found to be significant. There is a 98.6 ms
difference between the estimated means for the HLE group, 395 ms, and the LLE, 296
ms (t=4.49, p<0.0001). Interestingly enough, it is the mean fixation duration on the
image that varies depending on Level of L2/L3, thus partially confirming the hypothesis.
The fixations of the participants in the HLE group who dedicate less time to reading the
subtitles become longer. Since their attention does not change focus as much as the
attention of LLE participants does, they have a smoother viewing process, which is
more centered on the image.
6.1.2.3. H2.10: Audience enjoyment is higher for participants with a high level of
proficiency in L2/L3.
The results show that there is a significant main effect of Level of L2/L3 on Audience
enjoyment, thus confirming this hypothesis. The estimated mean for perceived audience
enjoyment among HLE participants was 2.85 points, while the mean among LLE
participants was 2.49 points (t=2.68; p<0.01).
197
The Reception of (Non)Professional Subtitling
The interest of these results stems from the fact that HLE participants spend less
time on the subtitle area, which might mean that depending more on the subtitles could
decrease viewer’s enjoyment of content. Actually, during the interviews, it became
evident that most participants considered subtitling to be a necessary evil: they
recognize that they need them and also granted them an added value in terms of
language learning, but they affirm that subtitles have the downside of being more
cognitively demanding and tiring than dubbing (see Section 6.3.2).
There could be yet another reason for the lower ratings among LLE participants.
Bucaria and Chiaro (2007) found that Italian viewers rated translated foreign material as
less entertaining than locally produced content. This situation is, to some extent, similar
to the effect of higher proficiency in the foreign language and enjoyment. If we assume
that a higher level of proficiency in the source language also implies a closer relation
with the source culture, then the participants in the HLE group could identify more with
the TV series and the culture it represents. They find the content more entertaining than
do participants in the LLE group, who have to allocate a considerable proportion of
their cognitive resources to following the subtitles and understanding the content in
order to connect with the culture.
6.1.2.4. Conclusion on H2: Level of L2/L3
In the case of H2, as shown in Table 101, all three sub-hypotheses were confirmed, thus
confirming the main hypothesis. According to the data collected, the participants behave
differently depending on their level of proficiency in the source language of the content.
Viewers with a low level of proficiency in the language have to follow the subtitles
more closely, as expected. The viewers who are more proficient in the language follow
a less linear and more independent reading process, since they do not fully rely on the
subtitles (see Section 6.2.1).
Interestingly, the results also indicate that a higher reliance on the subtitles is
correlated with a lower feeling of enjoyment. The scores of viewers who spent more
time on the subtitles coincide with lower average enjoyment than do the scores of the
highly proficient viewers.
198
Chapter 6. Qualitative data and discussion
Table 101. Summary of the hypothesis testing for H2
Hypothesis
H2.8
Variable
Attention allocation
H2.9
Mean fixation duration
H2.10
Audience enjoyment
Confirmation
Participants who have a high Level of L2/L3 fixate less and
for shorter periods of time on the subtitle area than do
participants with a low Level of L2/L3.
() This sub-hypothesis is partially confirmed. No significant
effect for Level of L2/L3 on Mean fixation duration on the
subtitle area was found, but there is a significant difference
in the Mean fixation duration on the image area depending
on the participants’ knowledge of the source language.
The scores of participants in the HLE group indicate they
enjoy the content more than participants in the LLE group
did.
6.1.3. Hypothesis 3: Type of non-professional subtitles
The wide range of non-professional subtitling activities results in products with a high
degree of variation between them. In view of this, in this study I decided to include two
versions of non-professional subtitles in order to explore the differences between them,
as well as how they compare with professional subtitling.
The hypothesis for the comparison between the two non-professional subtitles
was: Participants’ reception scores will be higher with the non-professional subtitles
produced in Spain (their own country) than with the non-professional subtitles produced
in Latin America.
The phrasing of this hypothesis is guided by the fact that the subtitles produced in
Spain (NP2) use the regional variety of the language and are intended specifically for
the Spanish audience. The other set of subtitles (NP1) was created by a Latin-American
non-professional subtitling community that started in Argentina and translates into
“neutral” Spanish. I formulated three sub-hypotheses for assessing the effect of the
types of non-professional subtitles. These hypotheses involve the variables Reception
capacity, Subtitle-reading effort and Self-reported comprehension. All three are related
to Gambier’s second R, responses at the perceptual level, and assess the participants’
performance and perception about the subtitles.
6.1.3.1. H3.11: Reception capacity is higher with NP2.
As shown in Section 6.1.1.1, Type of subtitle was found to have a significant main effect
on Reception capacity. The estimated mean for Reception capacity with NP1 was 5.96,
while the mean for NP2 was 5.37. The difference between the two types of non-
199
The Reception of (Non)Professional Subtitling
professional subtitling is significant, but the estimated mean is higher for NP1, thus
providing evidence to reject the hypothesis.
As commented in the test for H1.5 in Section 6.1.1.1, the fact that the results
indicate a higher average Reception capacity with the neutral Spanish version comes as
a surprise. The differences between the two versions of non-professional Spanish
subtitles are not difficult to spot for Spanish speakers, but the participants did not even
comment on this difference during the interviews.
Since the PRO version achieved scores similar to NP1, probably the difference in
scores is not necessarily an effect of NP1 having a more standard version of the subtitles
but rather is due to the specificities of the subtitles themselves. In that sense, the scores
for Reception capacity help establish a ranking of the translations and serve as an
indicator of quality. The inclusion of three different sets of subtitles in the experiment
and the fact that two of them gave similar results allow us to assume that the poorer
performance of the participants when they watched the audiovisual content with NP2
could be due to the quality of the subtitles. This is supported by the fact that, among the
types of attention (Verbal, Iconic and Narrative) that comprise Reception capacity, only
Verbal attention has a significant main effect on Type of subtitle. The participants had
82.6% of their questions right when watching the videos with NP2. However, this
percentage was 93.6% with NP1, which stands closer to the 94.8% of correct answers
obtained with the PRO versions.
6.1.3.2. H3.12: Subtitle-reading effort is lower with NP2.
As explained in H1.6 in Section 6.1.1.2, no significant effect of Type of subtitle was
found on Subtitle-reading effort. This means that the ratings the participants gave to the
difficulty they had were not affected by Type of subtitle. Thus this hypothesis cannot be
confirmed. The interaction between Clip and Type of subtitle was significant (F=3.10;
p<0.05), which indicates that the combined effect of Clip and Type of subtitle affects the
perceived difficulty to follow the content. In the analysis of the results it is possible to
appreciate that, while in Clip 1 and Clip 3 there is more variation in the scores for
subtitle-reading difficulty, the scores for Clip 2 are closer together. This, combined with
the effect of the Clip variable on other dependent variables, seems to indicate that Clip 2
was more difficult to understand than the other clips (see Section 6.2.3). Further, the
pairwise comparisons for Type of subtitle taking Clip as reference factor did not yield
200
Chapter 6. Qualitative data and discussion
significant results for the differences between NP1 and NP2 in the three clips.
Significant differences only occur between PRO and NP2.
6.1.3.3. H3.13: Self-reported comprehension is greater for NP2.
No significant effect for Type of subtitle was found in the model constructed for Selfreported comprehension. This indicates that Type of subtitle did not influence the
perceived understanding of the content. Based on this, the data do not provide any
support for the hypothesis.
On the other hand, the interaction between Clip and Type of subtitle was found to
be significant for this variable (F=2.92; p<0.05). However, when taking Clip as
reference factor, the only instance in which there was a significant difference between
the two non-professional versions was Clip 3. The estimated mean score for NP2 in
Clip 3 was 4.01, while the estimated mean score for NP1 was 3.44. In the other two
instances, i.e. Clip 1 and Clip 2, there was no significant difference between the two
non-professional versions.
6.1.3.4. Conclusion on H3: Type of non-professional subtitling
As shown in the summary in Table 102, the results from the analysis of the subhypotheses do not allow for a confirmation of H3. The findings show that NP2, the
Iberian Spanish non-professional version, gave lower results than the Latin-American
version in terms of Reception capacity. The results of NP1 are in fact similar to those of
the professional subtitle version. Additionally, Type of subtitle did not have an effect on
the self-reported measurements assessing comprehension and Subtitle-reading effort.
Table 102. Summary of the hypothesis testing for H3
Hypothesis
H3.11
Variable
Reception capacity
H3.12
Subtitle-reading effort
H3.13
Self-reported
comprehension
Confirmation
Participants have higher reception capacity with the LatinAmerican version.
There is no variation in the scores for Subtitle-reading
effort resulting from the effect of Type of subtitle.
The scores for Self-reported comprehension are not
significantly different for the two types of subtitle.
6.1.4. Hypotheses based on qualitative data
Two additional hypotheses were postulated regarding the participants’ opinions about
the non-professional subtitles included in the study. It should be recalled that the
participants were not told about the production conditions of the subtitles until the end
201
The Reception of (Non)Professional Subtitling
of the experiment. Only at the end of the interview were they informed about the
differences between the subtitles.
6.1.4.1. H14: Participants will be more positively inclined towards the professional
rather than non-professional subtitles.
During the interview session, some of the participants commented on the fact that they
use non-professional subtitles, but no one hinted at the possibility that the subtitles
included in the experiment were taken from Internet communities or were done by nonprofessional subtitlers. When asked if they had noticed any difference between them,
sometimes participants commented on spelling mistakes or on the appearance of the
subtitles (see Section 6.2.5), but not on the provenance or production conditions of the
subtitles. Given that the participants did not comment on this issue, there is not enough
information to confirm this hypothesis.
6.1.4.2. H15: Participants will be more positively inclined towards the Iberian rather
than Latin-American subtitles.
As mentioned in the previous hypothesis, when the participants were asked if they
noticed any difference between the subtitles, they did not mention any difference related
to the provenance of the subtitles. With the material collected during the interviews, it
was not possible to support this hypothesis. Some of the participants did mention that
there are normally Iberian and Latin-American versions of the subtitles on the Internet
and that sometimes it is annoying because they have to make sure which one is
available. Nevertheless, none of them said that one of the versions was Latin-American.
Participant 35, with a low level of English, commented that she felt the subtitles had
variations in terms of vocabulary, but she was not able to give an example or explain the
reasons for her feeling.
(1)
Participant 35: Yo creo que [los subtítulos] son diferentes. No sabría decir [por
qué], es una impresión que me da […] La primera y la segunda [NP1, NP2]
están más o menos igual y la tercera diría que la ha hecho otra persona [PRO].
I think the subtitles are different. I can’t explain why, it’s just the impression that
I got. I think the first and the second one are more or less the same [NP1, NP2],
but I’d say the third [PRO] was done by somebody else.
202
Chapter 6. Qualitative data and discussion
One of the reasons for this could be the short exposure the participants had to
each type of subtitle. As mentioned above, given some salient differences between
Iberian and neutral Spanish, it is not hard for native Spanish speakers to identify the
geographical imprint in the texts. For instance, the Iberian version used vosotros for the
second person plural, while the Latin-American version used ustedes. Since each clip
was only about three minutes long and participants watched one clip after the other,
they may not have had time to pay much attention to the sociolinguistic aspects. As
pointed out in previous studies (Secară 2011; Künzli and Ehrensberger-Dow 2011),
results could be different with longer exposure to the subtitles. However, an experiment
exploring the reception of an entire feature film could hardly include eye-tracking data,
at least until we have access to more specialized tools.
6.1.4.3. Conclusion on Hypotheses 14 and 15
Surprisingly, the participants did not notice differences between the subtitles. I was
convinced some of them would point out that one set of subtitles was Latin-American,
but this did not transpire during the interviews. Considering the participants’ comments,
as shown in Table 103, it is not possible to confirm these hypotheses.
Table 103. Summary of the hypothesis testing for H14 and H15
Hypothesis
H14
H15
6.2.
Variable
Inclination towards
professional subtitles
Inclination towards
Iberian nonprofessional subtitles
Confirmation
The participants did not comment on the linguistic
differences between the subtitles during the interviews.
No evidence was found to confirm this hypothesis.
Findings from the qualitative data
6.2.1. Level of L2/L3
The results of the eye-tracking data indicated that there is a clear difference in terms of
attention allocation between the participants with a high level of English and the
participants with a low level of English. As expected, the participants in the HLE group
allocated significantly less time to the subtitle area than did the participants in the LLE
group. Additionally, they also skipped more subtitles on average. While in previous
studies (d’Ydewalle and Bruycker 2007; Caffrey 2009), the percentage of Skipped
subtitles was lower than 4%, HLE participants in my study had an average of 25%
unfixated subtitles. This high average and the percentage difference in the duration on
203
The Reception of (Non)Professional Subtitling
the subtitle area hint at the existence of different reading behaviors that could be partly
explained by the level of proficiency in the source language.
One of the most robust findings in eye-tracking studies is that the reading of
subtitles is a more or less automatic behavior (d’Ydewalle et al. 1987; d’Ydewalle and
Bruycker 2007; Kruger et al. 2015). This was reported by d’Ydewalle et al. (1987) in an
experiment that included participants who knew the source language of the audiovisual
products and watched them with subtitles in their mother tongue. They found that “the
subjects who know very well the spoken language still read the subtitles as much as the
other subjects” (1987:320). The findings from my study challenge that conception, since
some of the participants managed to avoid fixating on the subtitle for most of the video
runtime. It is important to point out that the constant exposure to subtitles might have
caused the participants in d’Ydewalle et al. to become so used to reading the subtitles
that they cannot avoid fixating on them. On the other hand, Spanish viewers who are
more exposed to dubbed content may be instinctively more used to relying on the image
than to depending on the subtitles. d’Ydewalle et al. (1991) actually showed there is
some degree of control in subtitle reading, but it is still generally commented in the
literature that the semi-automatization of subtitle reading is the most common
characteristic. To some extent, the decision not to read the subtitles could be related to
the Observer’s paradox (Labov 1972): the participants tried not to follow the subtitles
because they were being monitored and wanted to show they could do well without
them. However, there are two reasons that could help understand why this is not
necessarily the case: 1) in the pre-experiment questionnaire some of these participants
declared they never use subtitles, before knowing the specificities of the experiment,
and 2) their performance in the Reception capacity questions was not lower than that of
the rest of the participants, which suggests they did understand the content of the clips.
The participants in the HLE group seemed to have developed autonomous
reading processes. Reading behavior has been shown to be affected by the reading
conditions or tasks. Rayner and Fischer (1996) and other researchers have shown there
are differences in eye movements in different types of activities, such as reading, the
scanning texts or visual search. Additionally, it has also been proven that there are
viewer-related characteristics that affect this behavior. The hearing capacity of the
viewers affects the subtitle reading performance (Szarkowska et al. 2011; Kruger et al.
2015): deaf and hard-of-hearing viewers are slower readers than hearing viewers,
making more fixations on the subtitles and having longer mean duration fixations. The
204
Chapter 6. Qualitative data and discussion
level of proficiency in the source language could be an internal factor that influences
and shapes the subtitle-reading behavior.
So far, knowledge of the source language has not been thoroughly explored as a
factor in studies on subtitle reception. Apart from studies with a focus on language
learning, researchers analyzing the effects of subtitling have been more interested in
exploring the reading behavior when participants depend completely on the subtitles.
That is the case of d’Ydewalle and Bruycker (2007), who use a Swedish film, and
Perego et al. (2010), who include a Hungarian film in their experiments. However,
given the current globalizing exposure to English, and especially the multilingualism in
Europe, it is safe to assume that the standard situation is not necessarily one in which
the source content is totally foreign (Pym 2013). Most of the audiovisual material
translated into Spanish in Spain comes from English-speaking markets and the
audience, although not always proficient in the language, is relatively well-acquainted
with it. The initial questionnaire included in this study shows that 60% of the surveyed
university population has a middling or high listening-comprehension proficiency in
English (Section 5.1.1), which would suggest a high level of contact with English as a
foreign language. As noted, students in all undergraduate programs at the university are
required to pass a test equivalent to the B2 proficiency level in English before they
graduate.
In view of this apparent difference in behavior, here I will present the results of
the participants who were placed in the Order of presentation 5 in my experiment, in
order to illustrate the different behaviors that I encountered. As shown in Section
5.3.3.2, the Order of presentation variable had a significant effect on Skipped subtitles.
The participants in Order of presentation 5 skipped 35% on average of the subtitles, a
percentage that is significantly higher than the estimated means of the other orders.
The difference in the scores for Order of presentation 5 are due to the fact that
this group had four of the participants in the HLE group who allocated the least time to
the subtitle area: Participants 12, 21, 39 and 44 (only Participant 21 is a man, the other
three are women). The difference in the percentage of fixations on the subtitle area
between these participants and the LLE participants in the same order of presentation is
greater than the difference between the two levels in any other of the groups. Participant
21 made less than 1% of the fixations on the subtitle area and skipped 98% of the
subtitles. Participant 12 skipped 84% of the subtitles and made about 3% of the
205
The Reception of (Non)Professional Subtitling
fixations on the subtitle area. The other two participants each made 11% of their
fixations on the subtitle area and skipped 53% of the subtitles.
Heat maps give a graphical representation of the difference in the number of
fixations between LLE and HLE participants. Figure 44 shows the heat maps resulting
from the data of the participants in Order 5. The column on the left shows the heat maps
for the LLE participants per clip; the heat maps on the right represent the results for
HLE participants. As can be seen, in the LLE heat maps, the higher concentration of
fixations is on the subtitle area, while the case is reversed for the HLE participants, who
have more fixations at the center of the screen on the image area.
Figure 44. Heat maps of the fixations of HLE and LLE participants in Order of presentation 5
HLE participants
Clip 3
Clip 2
Clip 1
LLE participants
There are some interesting aspects about the behavior of these participants. It is
important to point out here that these behavior patterns are not restricted to the
206
Chapter 6. Qualitative data and discussion
participants in Order 55; I am merely referring to this group as an example because it
unexpectedly includes a subsample of participants whose behavior clearly describes the
particularity I noticed among some of the HLE participants. The salient features can be
summarized as follows:
-
There are viewers with a high level of proficiency in the source language who
are able to adapt their subtitle-reading behavior to their needs and go to the
subtitle area only on specific occasions.
-
In most cases, the viewers do not go to the subtitles at the onset, when the
subtitles first appear, but move their gaze to the subtitle area only after they have
identified something in the audio track that sounds unfamiliar to them.
-
Due to this behavior, there is a latency effect. Latency refers to the time between
the onset of the subtitle presentation on the screen and the first fixation of the
viewer on the subtitle (d’Ydewalle and Bruycker 2007). In some cases, the gaze
reaches the subtitle area when the subtitle has already changed or disappeared,
probably leaving the viewer without the information they required and causing a
greater latency effect. However, this remark should be taken carefully. I did not
calculate the latency time; this is merely a recurrent feature I noticed from the
recordings.
-
It is frequently the case that participants do not go to the beginning of the
subtitles when they are looking for information. They move their gaze to the
center of the subtitle area – as previously pointed out by Jensema et al.
(2000b) – or towards the right side. Figure 45 presents one example in which
three of the four participants fixated on the subtitle area. The image shows a
higher concentration of fixations on the right half of the subtitles.
-
Although I did not carry out a detailed analysis of the specific elements that
caused HLE participants to look at the subtitle area, I could identify some
elements that tend to provoke fixations on the subtitle area: long noun
compounds, proper nouns, and low-frequency words or specialized terms.
Again, the subtitle in Figure 45 serves as an example of this phenomenon. The
5
As explained in Section 4.2.5.2, I had to create six different tests in Tobii Studio for the six orders of
presentation. Since Tobii Studio at the moment does not support across-test analysis, I can only provide
visualizations per test and not for the complete sample of all participants per Level of L2/L3.
207
The Reception of (Non)Professional Subtitling
subtitle includes the name of actress Ellen Page and her film Juno, as well as a
complex noun phrase “star of the charming independent film Juno”.
-
The differences in behavior between LLE and HLE participants are also
reflected on the participants’ enjoyment of the audiovisual content. The analysis
found that HLE participants reported higher ratings for Audience enjoyment.
Along with this finding, it should also be considered that higher ratings of
Audience
enjoyment
correlate
with
higher
ratings
of
Self-reported
comprehension. Essentially, the HLE participants consider the content to be
more enjoyable and the participants who give higher Enjoyment scores also
claim higher Self-reported comprehension.
Figure 45. Example of HLE participants’ fixations
6.2.2. Subtitling and L2/L3
The benefits of subtitling for language learning have been the subject of a constant
debate in Translation Studies, as well as in foreign-language acquisition research
(d’Ydewalle and Pavakanun 1995; d’Ydewalle 2002; Bisson et al. 2014). The idea that
subtitles enhance language learning was also voiced by the participants in my study.
Some of them specifically pointed out the relation between subtitling and language
learning as a benefit of and motivation for using subtitles. During the interviews, it was
evident that most of the participants, regardless of their level of English, assumed a
direct relation between subtitling and language learning. The participants generally
208
Chapter 6. Qualitative data and discussion
pointed to the benefits of subtitling for vocabulary learning and for general familiarity
with the foreign language. Those participants who felt they had a higher level of English
mentioned that they do not follow the subtitles entirely. For them, the subtitles act as an
aid to understanding the content. The participants use the subtitles as a tool to confirm
they understand the content or as a source of information where they can look for what
they might have missed, instead of relying completely on them. Interestingly enough,
this points towards a cognitive process that allows participants to watch subtitles in a
selective manner, similar to what Künzli and Ehrensberger-Dow (2011) and Ramos
Pinto (2013) found in their studies analyzing reactions to surtitles. Participant 43 (from
the HLE group) explains how she uses subtitles only as a source of confirmation:
(2)
Participant 43: Más que nada para mí [los subtítulos] sirven para ver si lo que
has oído es lo que has entendido. Yo preferiría que [los subtítulos] estuviesen en
inglés, porque en español me distraigo y tengo la tentación de seguir en español
porque es más cómodo.
For me, the subtitles work mainly as a way to confirm whether you understand
what you hear. I prefer to have the subtitles in English. If they are in Spanish,
then I become distracted and I am tempted to follow the Spanish subtitles only,
because it is more comfortable.
For all of the participants it was nevertheless evident that the feeling of reading
subtitles while watching a film is a more demanding activity than watching dubbed
films. They use subtitles mainly because they consider it to be an activity with a twofold
intention: watching a film for entertainment and making some effort to improve their
English. As participants 35 and 47 (both from the HLE group) put it:
(3)
Participant 35: Friends la veo subtitulada para practicar el inglés. Y las otras
[series] las veo dobladas. Los subtítulos son solo para practicar. Como ya he
visto Friends en español y ya más o menos entiendo lo que dicen, entonces me
es más fácil seguirlo y no tengo que estar leyendo todo el rato. Puedo practicar
más el audio.
I watch Friends with subtitles to practice my English. For other TV series, I use
the dubbed versions. I use the subtitles only as a way of practicing. As I have
already seen Friends in Spanish and I am able to follow what they say, it is
209
The Reception of (Non)Professional Subtitling
easier for me to follow the episodes and I don’t need to read all the time. I can
practice some listening.
(4)
Participant 47: En la época en la que tenía que estudiar para inglés, las veía todas
[las series] en versión original y subtituladas en inglés, pero cuando no tuve que
hacer esto, las veo normal.
When I was learning English, I used to watch all [the TV series] in the original
version with subtitles in English. But since I finished my English classes, I now
watch them the normal way [i.e. dubbed].
6.2.3. Effect of the Clip variable
This study suffered from a regular disadvantage of experimental research that has a high
degree of ecological validity. Although this approach allows one to recreate conditions
that are similar to the real-life conditions of the participants, it also adds additional
variables that should be controlled in order to identify the effects of the variables that
are actually being studied. Along these lines, when selecting the clips and the subtitles
to be included in the experiment, I took special care to ascertain that the audiovisual and
technical features of the clips were not altered. My primary interest was preserving the
integrity of the audiovisual content. The clips were self-contained, they included a
complete scene each time, and they were all taken from the same season. The same
number of characters interacted in each excerpt and the discussion was structured in a
similar fashion. Additionally, I decided not to alter the duration of the subtitles and used
three different versions of the video clip with different frame rates. By doing that, I
avoided modifying the time codes of the subtitle versions.
This process resulted in three clips that were different in length but complied with
the aforementioned requirements. The clips were thus assumed to be comparable. There
were some differences that I noticed at the beginning, but they were considered inherent
to the audiovisual material. I was suspicious about Clip 1 because a significant number
of the dialogues included a discussion between Raj, an Indian scientist with a very
strong accent, and Sheldon, a character whose speech tends to be very fast-paced. In
Clip 2, part of the conversation is at a very rapid pace. When comparing the number of
words uttered per second, Clip 2 has a higher ratio, with 2.5 words spoken per second,
while Clips 1 and 3 have ratios of 1.8 and 2.3 words per second respectively. In the end,
210
Chapter 6. Qualitative data and discussion
the accented English in Clip 1 did not seem to put additional pressure on the
participants, but it is likely that the faster speech rate in Clip 2 caused problems, as
explained below.
The results show that the participants found Clip 2 more demanding in terms of
allocation of cognitive resources and performance. One of the advantages of using
mixed-models to analyze the data is that the models account for these types of
variations and make their influence measurable. In this case, the Clip variable affected
the behavior of participants at different levels. In terms of eye-tracking measurements, it
had an effect on the percentage of fixations they made in the subtitle area and,
consequently, on their attention shifts ratio. At the level of the comprehension of the
content, it affected their Reception capacity and also the enjoyment of the clip. In order
to illustrate this, Figure 46 shows the accumulated fixation duration of all participants,
HLE and LLE, in Order of presentation 1 for the three clips. It can be seen that in the
heat maps of Clip 1 and 2 there are hot areas over the image area, while none can be
found on the image area in Clip 2, indicating that the participants fixated for much
longer on the subtitles.
Clip 3
Clip 2
Clip 1
Figure 46. Heat maps with fixations durations made by all participants in Order 1
211
The Reception of (Non)Professional Subtitling
On average, the participants fixated 54.4% of the time on the subtitle area when
they were watching Clip 2 and had an attention shift ratio of 1.7, lower than the 1.9
attention shift ratio for the other two clips. This means participants not only spent longer
periods of time looking at the subtitle area, but they also made fewer attention shifts per
subtitle. They stayed in the subtitle area during the time in-between subtitles. This
behavior implies a high degree of reliance on the subtitles: the participants recognize
that it is harder for them to understand the subtitles and then decide to allocate a higher
share of their attention to the subtitle area, even sacrificing incidental glances at the
image. This trade-off in attention allocation is evident in the results of the statistical
analysis. The reduced attention on the image area correlates with poorer scores in Iconic
attention. The Clip variable was the only one that had a significant effect on Iconic
attention: the participants obtained fewer correct answers for Clip 2 than for the other
two clips.
Figure 47. Percentage of duration of fixations per Clip and Level of L2/L3
A closer look at the data shows that this behavior affects both types of
participants, those with a high and those with a low level of English indistinctively.
Figure 47 shows the percentage of durations of fixation per Clip and Level of L2/L3. As
in other cases, there is more dispersion in the values for the participants in the LLE
group in all three clips. However, it can be seen that the mean percentage of fixations
increases for all participants in Clip 2. The mean percentages of duration of fixation on
212
Chapter 6. Qualitative data and discussion
the subtitle area for HLE and LLE in Clip 2 are 35% and 56% respectively. These
values are higher than the average 30% fixation duration HLE participants had with the
other two clips and also higher than the 49% and 51% fixation duration observed in
Clip 1 and Clip 3 respectively for the LLE group.
In terms of performance, Clip 2 produced poorer results in Reception capacity
than the other two clips. The estimated mean score for Clip 2 was significantly lower
than that of Clip 1 but the difference between Clip 2 and Clip 3 was not significant. The
Clip variable also had an effect on Audience enjoyment. Interestingly enough, Clip 1,
which produced the best score in terms of Reception capacity, was also judged to be the
most boring clip among all three.
The variability among the clips allows for further considerations. Given that the
Clip variable did not have a significant effect on Mean fixation duration on the subtitle
area, it could be claimed that the length of individual fixations as such is not affected by
the increase in the dialogue speed. Only the number of fixations a viewer makes on the
subtitle area actually increases, which consequently affects total fixation duration. This
supports the idea of using the number of fixations on an area as an indicator of cognitive
effort.
The results also provide insights into the effectiveness of subtitles. In all three
subtitling conditions, Clip 2 had subtitles on the screen for the longest period of time. In
the case of PRO and NP1, subtitles were on the screen for 87% of the segment runtime.
In the case of NP2, subtitles were shown 80% of the time. This contrasts with the other
clips, in which subtitles where shown from 66% to 79% of the time. The results are thus
rather contradictory, if we consider that the participants stayed in the subtitle area
longer, sacrificing the possibility of watching more of the image. It seems a rather
paradoxical outcome that they had the lowest score in Reception capacity when they
stayed longest on the subtitles. Naturally, there can be other reasons for the participants
to stay for longer on the subtitle area, but given that they face the same situation under
all three subtitle conditions, the subtitle effectiveness seems to be a plausible
explanation.
6.2.4. Viewers’ familiarity with subtitles
Another variable that provided thought-provoking results at the analysis stage was the
familiarity of participants with the subtitles. Spain is traditionally considered a dubbing
213
The Reception of (Non)Professional Subtitling
country, but, as it has been explained, the traditional divisions of audiovisual translation
have been adjusting to accommodate more active users. Taking that into consideration, I
included variables to explore the effects of the participants’ use of dubbing, subtitling,
voice over and closed captioning.
Little has been done to compare how different levels of familiarity with an
audiovisual translation modality affect the viewer’s behavior. It is commonly assumed
that the country where the viewer lives or grew up serves as a categorization
(d’Ydewalle et al. 1991; Perego et al. 2010; Künzli and Ehrensberger-Dow 2011).
People coming from dubbing countries will be more used to dubbing than will people
coming from subtitling countries. While this assumption is likely to hold for
environments where there is only a reduced number of sources of audiovisual content,
under the current network of media flows and content circulation, taking this broad
categorization as a standard is very rough at best. Subtitles currently serve diverse
audiences who have different degrees of familiarity with subtitle reading. The viewer’s
use of subtitling, in this case, becomes an internal-viewer factor that could affect the
way people engage with the content and could also shape their behavior.
The Use of subtitling variable had a significant effect on all but one of the eyetracking measurements included in the study (the exception was the Mean fixation
duration on the image), as well as on Subtitle-reading effort and Self-reported
comprehension. According to their use of subtitling in the six months prior to the
experiment, the participants were grouped in five levels, ranging from those who say
they never use subtitles to those who claim they always use subtitles. As detailed in
Section 5.3.1.1, the initial five categories were reduced to four to make the statistical
analysis possible: Never, Rarely, Occasionally, and Very frequently/Always. The results
show that there are differences in the behavior of participants who have different levels
of familiarity with subtitles. As partly commented on in Section 6.1.4, there were
participants in my study who managed to fixate on the subtitle area for a very short
amount of time and looked at the image area for more than 90% of the runtime. As a
general result, the Never group was significantly different from the other three in all
occasions, while Rarely was different from all other groups although the differences did
not reach statistical significance.
The subgroup of participants who never use subtitles comprised four HLE
participants. This is the group that differs the most from all the others: in all cases the
difference with every other level of familiarity is statistically significant. This group of
214
Chapter 6. Qualitative data and discussion
participants had the longest estimated mean fixation on the subtitle area, 216.76 ms,
surpassing the other groups by between 27 and 37 ms. On average they only spent 30%
of the time on the subtitle area, which contrasts with the average 43% that participants
in the HLE group allocated to the subtitles. Additionally, they had the lowest attentionshift ratio and skipped 38% of the subtitles, while the group of occasional and frequent
users skipped about 30% only.
These participants’ scores also indicate their self-reported comprehension was
better than that of the rest of participants and they reported the lowest scores for
Subtitle-reading effort. However, no significant effect of the Use of subtitling variable
was found on Reception capacity, which indicates that these data do not support the
participants’ impression that they comprehended better. The longer fixations, combined
with the reduced amount of time they spent on the subtitle area, means HLE participants
who had never used subtitles did not follow a completely linear reading behavior when
they were watching subtitled material. However, the participants did fixate upon 62% of
the subtitles, which is indeed a high percentage, and their scores for Subtitle-reading
effort were low. This means the participants found the translation useful and easy to
follow. The results come as a surprise. By inspecting the eye-tracking recordings, there
seems to be a longer latency effect among these participants. They fixate on the subtitle
after identifying something in the audio that puzzles them and that triggers a deflection
to the subtitle area. As noted in Section 6.2.1 above, in many cases, this means their
gaze fixates on the subtitle shortly before it disappears, presumably leaving them little
time to process the information. However, it could also be the case that the confirmation
they are looking for is sufficiently supported in the time in which they can try to read
the subtitle.
In a study carried out in Belgium using intralingual subtitles, d’Ydewalle et al.
(1991) found that participants from the United States who are not used to subtitling (due
to the scarce use of subtitles in the US) spent up to 27.07% of the subtitled time on the
subtitle area, even though the content was available to them in their own language
(intralingual subtitles). Unfortunately, the study does not report the number of skipped
subtitles, so we cannot make comparisons. However, the American participants spent
less time on the subtitle area than the average time of the HLE group in my experiment.
The authors do recognize that some type of control can be exerted by viewers: “Reading
subtitles seems to be at least partially under the subject’s control. It should be noted,
215
The Reception of (Non)Professional Subtitling
however, that this personal control does not imply the absence of automatic processes”
(d’Ydewalle et al. 1991:656–657).
Once more, I rely on the participants from Order of presentation 5 to show the
difference between viewers with different levels of familiarity with the subtitles. In
Figure 48, the image on the left shows the heat map of Clip 3 created with the
accumulated fixation durations from Participants 12 and 39, who say they never use
subtitles, and the image on the right shows the heat map of the same clip resulting from
the accumulated fixation durations of HLE and LLE participants in the same group,
who say they use subtitles occasionally. The hottest areas are totally reversed, with the
participants who never use subtitles focusing almost restrictively on the image and the
other participants spending more time on the subtitles but still looking at the image for
some time.
Figure 48. Heat maps for participants who use subtitles never (left) and occasionally (right)
The interview data from HLE participants offer some additional information to
complement the picture of their behavior. Participant 18 says the subtitles distract her
from the action on the screen and that this bothers her. She realizes that even if she is
not relying on them to access the content, it is inevitable to fixate on the subtitle area.
Participant 4, who spent on average only 6% of the time on the subtitle area, also sees
the subtitles as a distraction, while Participant 12, who looked at the subtitle area for 3%
of the time, says she uses the subtitles as a support:
(5)
Participant 04: Sí, [miro los subtítulos] por inercia. Sin pensarlo muchas veces
porque estoy entendiendo lo que están diciendo, pero como están los subtítulos,
los ojos se van a los subtítulos aunque intento no hacerlo porque me distrae de lo
que están diciendo.
Yes, I look at the subtitles out of habit. I do it unconsciously since I am able to
understand what they are saying, but as the subtitles are there, my eyes go there
216
Chapter 6. Qualitative data and discussion
automatically. I try not to do it because it distracts me from what the characters
are saying.
(6)
Participant 12: Para mí no son necesarios los subtítulos en general, pero si hay
alguna palabra que no entiendo, sí miro directamente los subtítulos. No me
molestan, cuando me acostumbro a ello, porque cuando veo series en inglés no
me hace falta mirarlos, pero si veo alguna cosa en otro idioma, a veces, tal vez
porque no estoy acostumbrada, pero me cuesta un poco. No me cuesta leer y
mirar, pero siento que me pierdo un poco de la imagen. No me distraen cuando
aparecen en pantalla. Los uso como referencia.
In general, I really don’t need the subtitles, when there are words I don’t
understand, then I look at the subtitles directly. Once I get used to them, they
don’t bother me. When I’m watching TV series in English I don’t need to look at
them, but if I watch something in a different language, sometimes, maybe
because I am not used to them, it is harder for me. It’s not that reading the
subtitles and watching the video at the same time is harder for me, but I feel I’m
missing some of the image. The subtitles don’t bother me when they are on the
screen. I use them as a reference.
Apart from the results for the group who never uses subtitles, the participants
who rarely use subtitles had a behavior pattern that is hard to explain. While the
participants who are more used to subtitling (those who use it occasionally and very
frequently/always) spent 54% of the time on the subtitle area, the participants who
rarely use it spent 65% of the time on the subtitle area: they read almost all the subtitles,
skipping less than 1%. This group also makes the most attention shifts and reports the
lowest mean comprehension score. None of these differences reach statistical
significance when compared to the results of the Rarely and Very frequently/always
groups. However, the picture the results show is interesting. As can be seen in the
graphs in Chapter 5 (e.g. Figure 21, Figure 26, Figure 38), the curves created for Use of
subtitling are always interrupted by the results of the level rarely. As shown above,
those participants who never use subtitles have a completely different behavior from the
participants who have a more regular use of subtitles. In that sense, the Never group is
made of viewers that have developed an independent reading pattern. It could be said
that the participants who rarely use subtitles are the ones who have decided they need
217
The Reception of (Non)Professional Subtitling
the subtitles and, as part of the process of becoming used to them, spend more time on
the subtitle area than do the participants who are already more used to the subtitles. The
Rarely group should actually be considered to be at the base of the learning curve. As
viewers become more used to the subtitles, their behavior sharpens and somehow
becomes more homogeneous, as can be seen by the very similar results for participants
who occasionally or very frequently/always use subtitles.
6.2.5. Quality from the viewer’s perspective
Once the interview was almost completed, the participants were told the specific
purpose of the research and the details of the audiovisual material. Once again, they
were asked if they noticed any difference between the subtitles or if they thought the
same person could have made them all. This time, some of them said that they had
noticed differences but that it was difficult to pinpoint any specific aspect or to indicate
in which clip the differences occurred. In general, the participants said the quality was
good. Most of the participants were able to recall the color of the subtitles (white) and
their position on the screen. About half of the participants answered in the affirmative
when asked if they thought the same person did all three translations.
One aspect that called the participants’ attention as a sign of low quality was the
layout of the subtitles. Figure 49 shows the layout of the three types of subtitles on the
screen as seen by the participants. The professional version (at the top) looks blurrier
than the other two and, although the subtitles are white, they are not as bright as the
non-professional versions.
Although all three versions use proportional fonts, the font size is different. The
professional version looks more spread-out across the screen. In the example, the
professional subtitle has 27 characters and the NP2 version has 30, but the professional
looks longer. The blurriness caused some of the participants to assume the professional
version was actually a non-professional version, because it was “more difficult to read”.
They also pointed out that the professional subtitles were “longer” because they
occupied more space on the screen, as in Figure 43. For some participants, these
features signaled that the subtitles were of low quality; they assumed they were the nonprofessional versions. As pointed out above, the Mean fixation duration on the image
and the subtitle areas vary depending on the Type of subtitle. In both cases, the mean
fixation durations in the PRO versions were shorter than the means in the other two
218
Chapter 6. Qualitative data and discussion
versions, which at the same time were more similar between them. Although it is hard
to test if this is to some extent caused by the appearance of the subtitles on the screen, it
is worth commenting that the two non-professional versions have the same appearance
while the professional version is different. It is possible that this difference in format
had an effect on the viewer’s behavior, but it would be necessary to carry out a different
study in order to test whether this is true.
Figure 49. Layout of the subtitles on the screen
PRO
NP1
NP2
Participant 39 (with a high level of English) said the first video she watched
(PRO) was different from the other two, and that it was harder to read. When de-briefed,
she was actually surprised those were the professional subtitles. Participant 10 (with a
low level of English) commented on the subtitles even before I told him about the
purpose of the research:
(7)
Participant 10: […] los dos primeros [subtítulos, NP1 and NP2], han sido
bastante similares. […] Cuando tenía alguna duda, bajaba la vista y se podía leer
fácilmente sin desviar mucho la vista a la hora de ver el vídeo. En cambio en el
último [PRO] eran quizás frases muy alargadas que iban de punta a punta de la
pantalla y al pasar abajo tenías que desplazar tanto la vista que directamente no
219
The Reception of (Non)Professional Subtitling
podías estar mirándolo [el vídeo]. En este caso el subtítulo del tercer vídeo era
incómodo y a la vez un poco difícil de leer.
The first two [subtitles, NP1, NP2] were very similar. […] Whenever I had a
doubt, I’d look down and they were easy to read without losing total sight of the
video. With the last subtitle [PRO], on the other hand, the sentences seemed to
stretch from one side of the screen to the other and whenever I needed to go
down the screen, I had to shift my gaze so much that it was impossible to
continue watching the video at the same time. In that sense, the subtitles for the
third video were more uncomfortable and, at the same time, harder to read.
Another feature that raised flags for the participants was spelling. In one of the
non-professional subtitles, the name of the actor Zac Efron was written with a ph as
Ephron. Some participants pointed this as a mistake during the interviews and said this
could be an indicator of non-professional subtitling. This assumption was not only
restricted to mistakes. In the third clip, one of the characters mimics the accent of Jar
Jar Binks, the fictional character from Star Wars. Subtitlers, both professional and nonprofessional, tried to adapt the subtitles to mimic the character’s accent. However, the
participants judged this to be a poor decision by the translators. This concurs with the
results of Gottlieb’s (1995) experiment in which some participants judged normal
subtitling solutions (such as condensation and reduction) to be mistakes.
The above-mentioned aspects show one dimension of what viewers could
consider to be indicator of quality in translation. As can be seen, the aspects that
viewers assume to be indicators of quality are not necessarily the same ones that
translators would take as features signaling the quality of a translation. Evidently, it is
difficult to reconcile these two aspects: translators are responsible for providing
translations that are suitable for the audience and many of their decisions and options
are the results of complex thought processes, while receivers commonly reduce
translation to a one-to-one replacement process. However, knowing what is relevant to
the audience could help translators in their decision-making processes. Empirical data
could help them decide among a multiplicity of options. The discussion, from a
Translation Studies perspective, should not stay at the level of what is better or worse,
but rather focus on the possibilities offered by the newly acquired knowledge.
Prosumers and the other members of the participatory culture are providing new
220
Chapter 6. Qualitative data and discussion
information and shaping new behaviors that should be taken into account, since they are
the ones using the translations. For instance, Participant 21, from the HLE group and
assigned to Order of presentation 5, comments on how he uses the subtitles:
(8)
Participant 21: En general no los he mirado [los subtítulos]. Creo recordar haber
hecho cuatro o cinco movimientos como buscando alguna expresión en concreto
o algo que me sorprendió […] fue una mirada muy rápida como diciendo “¿Y
esto cómo lo van a traducir?” porque a veces me gusta mirar qué patrón siguen y
qué parecido hay entre los subtítulos y lo que traducen, porque a veces no tiene
nada que ver y a veces está bien y hay cosas que no se pueden traducir, que las
tienes que ver en versión original o no funciona. Pero claro entonces tienes que
dejar la película o la serie aparte y fijarte sólo en la curiosidad de ver hasta qué
cierto punto los subtítulos tienen o no fidelidad respecto al texto.
I did not look at all the subtitles. I’d say I looked at them four or five times
searching for a specific expression or something that surprised me. It was a
quick glance, I just wanted to check how they translated it. From time to time I
enjoy searching for the pattern they follow and how the subtitles compare to
what is being translated. Sometimes the subtitles don’t have anything to do with
the original and sometimes the translation is good. Besides, there are things that
can’t be translated, you have to access the original or it doesn’t work. However,
to do the comparison you have to stop looking at the film or the show and, out of
curiosity, fixate on the subtitles to see how faithful the subtitles are to the text.
During the interview I also asked participants directly about their feelings related
to non-professional subtitling. The participants have, in general, very different opinions
about the subtitles available on the Internet. From their comments it is possible to see
that they are aware of variations in terms of quality but, considering the aspects
mentioned above, they have also learned to interact with the subtitles, classify them, and
decide when they can use them or how to come to terms with the complex situation.
Participant 48 mentioned that, for her, having the possibility of watching the series is
more important than having perfect subtitles. Participant 13 explained how he tries to
look for the best subtitles available:
221
The Reception of (Non)Professional Subtitling
(9)
Participant 13: A veces hay muchos errores y lo notas o incluso se saltan trozos,
o la traducción no es buena. [Los uso] para asegurarme que entienda [sic] lo que
están diciendo. A veces si fueran muy diferentes, muy malos, o los saco o busco
otros de otra persona. Además también a veces hay latino y a veces español.
Sometimes there are many mistakes and you can spot them, or they even skip
some parts, or the translation is not good. I use them to make sure that I
understand what they are saying. Sometimes when they look very different from
the original or the quality is too bad, I look for another subtitle file made by a
different person. Also, sometimes there is also the Latin-American version and
the Spanish version.
Participant 24 mentioned that variations in the subtitles depended on the source
language of the material:
(10) Participant 24: [En cuanto a la calidad de los subtítulos de Internet] Depende…
hay mucha variedad, depende de dónde los saques. Generalmente, algunos están
muy desfasados. Por ejemplo, en inglés no lo cojo tanto, pero cuando cojo
películas de Brasil, la traducción suele ser bastante mala.
The quality of subtitles on the Internet depends… there are many options, it
depends on where you get them. Normally they are completely out-of-sync. For
instance, I do not notice it that much in English, but when I watch Brazilian
films, the translation is usually very bad.
6.3.
Audience control over the viewing experience
One of the characteristics of the new media consumption habits is that users have the
possibility of exercising more control over the viewer experience. They can decide what
to watch, when to watch it, and how. In that sense, media consumption has become
more flexible. The interview questions aimed at going deeper into the aspects that were
previously explored in the questionnaires. During the interviews, various indications
were given as to the factors influencing the participants’ choice of different translation
modalities. In the following sections I will cover some aspects that exemplify how the
participants are accessing media content and integrating it into their routines.
222
Chapter 6. Qualitative data and discussion
6.3.1. Foreign audiovisual content and distribution delays
One of the most interesting results from the interview stage had to do with the amount
of non-local content that is consumed by the participants in the study. Most of the
participants mentioned that half or more than half of all the audiovisual content they
consume is not produced in Spain. This is consistent with what is reported in the VideoOver-Internet Consumer Survey 2013 (Accenture 2013), which states that in Spain 39%
of consumers use local/national online video providers while 61% of them access
content using international online video providers.
The time between the original release in the United States and the availability of
the content in the Spanish market is still the main reason why the participants decided to
use non-professional subtitles. This was one of the reasons for the emergence of
fansubbing and the trend is continuing; people want to access the content as soon as it is
released in the United States. Additionally, it should be recalled that heavy users of
audiovisual content rely mostly on the Internet as their source of films and TV series, as
pointed out in the results of the pre-experiment questionnaire. Some of the participants
said during the interview they watch subtitled content because it was available sooner
than the dubbed version that was released in Spain. Here are the opinions of two of the
participants:
(11) Participant 8: [Veo] muchas, muchas series. Veo bastantes de la BBC. Veo
Doctor Who, Sherlock, Downton Abbey. Después de americanas también veo
Orphan Black… no sé, bastantes. Así de comedia también veo New Girl…
Todas en Internet, pero si las pasan al poco tiempo de haberlas sacado allí, las
veo por la tele. Pero, pocas veces.
[I watch] lots and lots of TV series. I watch many from the BBC. I watch Doctor
Who, Sherlock, Downton Abbey. When it comes to American TV series, I watch
Orphan Black… I don’t know, lots of them… As for comedies, I watch New Girl
and… I watch all of them on the Internet, but if they are on TV soon after they
were released in the United States or the United Kingdom, then I watch them on
TV, but that almost never happens.
(12) Participant 48: [Sigo] Falling Skies, Juego de Tronos, The Walking Dead y
también películas. Todas en Internet. Si salen antes las dobladas, las dobladas, y
223
The Reception of (Non)Professional Subtitling
si no han salido todavía, pues me tengo que mirar las que tienen subtítulos. [Con
el doblaje] puedo estar más atenta a la imagen.
[I follow] Falling Skies, Game of Thrones, The Walking Dead and also many
films. I watch all of them on the Internet. If the dubbed version is available first,
then I watch it, but if it is not ready yet, then I have to watch the version with
subtitles.
They see fast production of subtitles as the best way to access content and keep
up with the releases in the United States. In some cases, subtitles are mentioned as a
necessary evil. People want to access the content, but find themselves confronted with a
linguistic barrier. Although some of them would rather watch the dubbed version, they
decide to put up with the subtitled one in order to watch the content. Even in those
cases, participants said that they enjoyed the content, and that was the reason why they
decided to continue using subtitles. They recognize the value of subtitling as a way to
keep watching the series. For instance, Participant 30 said she preferred to watch the
material dubbed into Spanish because it is easier for her than reading the subtitles and
watching the image at the same time; nevertheless she watches Orphan Black and New
Girl online with subtitles because they are usually dubbed and distributed much later in
Spain.
6.3.2. Making decisions about dubbing or subtitling
6.3.2.1. The use of different translation modalities depending on the audiovisual content
Surprisingly, a recurrent issue mentioned by participants was the classification of
translation modalities and their preference for them depending on the type of content
they watch. In some cases, participants provided information on how they were inclined
towards a given translation mode, and they did so in an informed and structured way.
Some participants said they preferred watching dubbed films because films are mostly
made of only one instalment or just a few of them in the case of sequels and trilogies.
For the participants, watching a film (rather than a series) involves a lower degree of
personal engagement with the content. On the other hand, when it comes to TV series,
they would be more inclined towards the use of subtitles. Since TV series normally have
more than eight episodes per season (with the exception of mini-series) and possibly
several seasons, participants are able to acquire more knowledge about the characters,
224
Chapter 6. Qualitative data and discussion
the way they interact, their behavior and especially their speech. Listening to them on a
more regular basis allows the participants to anticipate characters’ behavior and to use
their prior knowledge to compensate for any information that might possibly go
unnoticed or that they do not fully understand. They become what Allen (1985) call
expert viewers, since they already know the network of the characters and the plot.
Participants pointed out that they understand subtitling as good enough for their purpose
of keeping up-to-date with the events and being able to watch the show as it is broadcast
in the United States. For instance, Participant 46 explained how she decides about
translation modalities based on the genre of the content:
(13) Participant 46: Veo [las series] en Internet subtituladas. Prefiero verlas en inglés
que dobladas y no que no haya subtítulos […] En cambio, las películas,
normalmente las miro en español. Siempre las veo dobladas, no lo sé, es una
costumbre. Supongo que porque cuando sigues una serie, más o menos conoces
a los personajes, lo que van a decir, el tema y voy aprendiendo sobre cada tema
un poco escuchándolo en inglés. En cambio una película que son solo dos horas
quizás debería estar demasiado concentrada y prefiero relajarme y verla
tranquilamente sin tener que esforzarme para comprender.
I watch the subtitled [TV series] on the Internet. I prefer watching them in
English than dubbed and without subtitles […] But when it comes to films, I
mainly watch them [dubbed] in Spanish. I always watch dubbed films, I don’t
know why, maybe because I am used to it. I guess it’s because when you follow a
TV series, you somehow know the characters, what they’re going to say, the
topic and I also learn about each topic if I listen to it in English. But, as films go
for about two hours, then perhaps I would need to be very focused on them and I
prefer to relax and watch them without any pressure to understand.
6.3.2.2. The integration of viewing time into participants’ routine
In terms of time management, most of the participants pointed out that when it comes to
multi-tasking, dubbing is without any doubt the only possible solution. If they are doing
chores at home or are busy on the computer, for instance, then they would immediately
select the dubbed version as the best option. Additionally, the time of the day when the
participants watch the material also affects their decision. Participant 20 commented on
the benefits of watching shows on the Internet:
225
The Reception of (Non)Professional Subtitling
(14) Participant 20: [Veo las series en Internet] porque a la hora que la ponen por la
tele o los días que la dan, pues no la puedo ver por la tele porque no se
compagina con mi horario. En cambio, por Internet puedes verlo cuando a ti te
vaya bien.
[I watch TV series on the Internet] because when they show them on TV, I can’t
watch them because it doesn’t fit in with my schedule. But on the Internet you
can watch them whenever you want.
Another circumstance that immediately prompts this decision is related to the
company they have when they are watching the video. Even participants who prefer
subtitling would easily adapt and watch the dubbed version if the people who they are
watching the product with do not like subtitles, feel incapable of following them, or
simply prefer the dubbed version. In some cases, however, the influence of company
also works to the benefit of popularizing subtitles. About half of the participants
mentioned they decide to watch a certain TV series or film based on their friends’
recommendations. Although they also enjoy the content, watching a certain type of
product also becomes a way of interacting with their group of friends. By following the
influence of others, they adopt translation modalities and video service providers that
would not necessarily be their first option if they looked for audiovisual content by
themselves. Another claim that was mentioned during the interviews was that being able
to watch a TV series or a film in the original language with subtitles indicates that they
are able to understand the film and also to appreciate the content in its original form,
which grants them additional prestige among their peers. When asked how he decided
which translation mode he used for a given product, Participant 10 mentioned:
(15) Participant 10: Si me recomiendan que es mejor en versión original, la veo en
versión original. Si me lo recomiendan que tiene una buena traducción, la veo
con traducción.
If [my friends] recommend the original version because it is better, then I watch
the original version. If they say that it has a good translation, then I watch the
translated version.
Following the same line of argumentation, some participants mentioned that
reading becomes tiresome since it requires that their attention be fully directed to what
226
Chapter 6. Qualitative data and discussion
is happening on the screen. This makes it impossible for them to watch subtitled
material when they are feeling tired after work, for instance. The participants say they
prefer to watch subtitled material when they are totally relaxed and are willing to read
the subtitles and follow the action. Participants 35 and 51 commented as follows on the
shortcomings they saw in subtitling, considering the cognitive effort it demands:
(16) Participant 35: Bueno, le puedes poner los audios en idioma original, los
subtítulos. Y a veces lo hago. Hay días que no tengo ganas y digo, hoy en
español, porque no quiero leer. Pero hay días que sí.
Well, you can play it with the original soundtrack and the subtitles. I sometimes
do. There are some days when I don’t feel like doing that and I’d rather just
watch it [dubbed] in Spanish, because I do not want to read. Other days I do it.
(17) Participant 51: Es mejor si lo ves y entiendes sin subtítulos a la primera, porque
el hecho de estar para abajo y para arriba es un poco incómodo, pero no me
molesta. [De todas formas] cuando estás leyendo pierdes bastante la imagen.
It’s better when you can watch it and understand it without reading the subtitles.
Looking up and down the screen can be inconvenient, but it doesn’t bother me.
Anyway, when you are reading, you lose a lot of what is happening in the image.
6.4.
Discussion summary
Three initial main hypotheses were postulated at the beginning of this study. After
putting them to test, the results can be summarized as follows:
-
H1: Participants’ comprehension scores will be higher with professional rather
than non-professional subtitles.
There was no support for this hypothesis. Although there were differences
between one of the non-professional subtitle versions and the professional
version in the performance measurements, the non-professional Latin-American
version obtained results similar to those of the professional versions. In terms of
technical aspects, eye-tracking measurements seem to confirm that the
professional version allows for a smoother subtitle-reading process by allowing
the participants to alternate between the image and the subtitles.
227
The Reception of (Non)Professional Subtitling
-
H2: Participants with a high level of proficiency in L2/L3 will be less dependent
on subtitles than will participants with a low level of L2/L3.
The second hypothesis was confirmed. As expected, the participants with a low
level of L2/L3 followed a more linear subtitle-reading behavior. HLE
participants fixated less on the subtitle area and exhibited different types of
behaviors: some of them relied on the subtitles as much as LLE participants did,
but some others resorted to the subtitles as support on specific occasions only.
Interestingly, some of the HLE participants avoided the subtitles almost
completely. However, on the occasions when HLE participants who never use
subtitles did look at the subtitle area, their mean fixation was longer, indicating a
higher cognitive effort.
-
H3: Participants’ reception scores will be higher with the non-professional
subtitles produced in Spain (their own country) than with the non-professional
subtitles produced in Latin America.
The participants were not positively inclined either towards the professional
subtitles, or towards the non-professional Iberian subtitles. When asked directly
about the quality of non-professional subtitles, they did claim some of them have
a very low quality. However, they did not identify the subtitles in this
experiment as non-professional.
Additionally, recapitulating the additional information obtained from the
interviews, the participants explained they think subtitles are more demanding.
However there are reasons for the participants to put up with them. For instance, they do
it because they want to access the original content before it is distributed in Spain or
because they think it constitutes a better experience. Also, they see subtitling as an
opportunity to improve their foreign language skills. Instead of favoring dubbed
products exclusively, there seems to be a cohabitation of subtitling and dubbing in the
participants’ consumption of audiovisual content (Chaume 2013; Casarini 2014).
Depending on different consumption conditions, they can decide which translation
modality to use.
228
Chapter 7. Conclusions
This study on reception was grounded on the orthodox assumption that non-professional
subtitles generally have lower quality than do professional subtitles. Another feature
that was explored was the viewers’ level of proficiency in the source language of the
content. The main questions guiding the study were:
-
Does audience reception indicate any difference between the professional and
the non-professional subtitles?
-
Do better reception scores correlate with professional subtitles?
-
Does the users’ level of proficiency in L2/L3 affect the reception of
professionally and non-professionally subtitled audiovisual material?
-
Do participants notice any difference between professional and non-professional
subtitles?
In order to answer these questions, the Type of subtitle and Level of L2/L3
variables as well as the variables resulting from the operationalization of reception were
brought into relation to postulate three main hypotheses:
-
H1: Participants’ comprehension scores will be higher with professional rather
than non-professional subtitles.
-
H2: Participants with a high level of proficiency in L2/L3 will be less dependent
on subtitles than will participants with a low level of L2/L3.
-
H3: Participants’ reception scores will be higher with the non-professional
subtitles produced in Spain (their own country) than with the non-professional
subtitles produced in Latin America.
These hypotheses were further refined and became fifteen sub-hypotheses that
were put to the test in the study (see Section 4.1.4). The sub-hypotheses make
assumptions on the relation of Type of subtitle and Level of L2/L3 and the variables
defined to assess reception. The sub-hypotheses were put to test and results are
presented in the following sections to provide an answer to the research questions.
In general terms, Type of subtitle did not affect the participants’ performance.
Level of L2/L3 proved to be a factor affecting eye-tracking measurements at almost all
levels. However, no effects were found on Reception capacity testing. Surprisingly,
neither the qualitative nor the quantitative data indicate an inclination of the participants
towards the Iberian non-professional subtitles.
229
The Reception of (Non)Professional Subtitling
7.1.
Research questions answered
Does audience reception indicate any difference between the professional and the nonprofessional subtitles?
Do better reception scores correlate with professional subtitles?
The first research question asks about the correlation between Reception capacity and
Type of subtitle, the second asks whether the performance scores of the participants will
be better when they watch the professional version.
These questions, related to the type of subtitle, were answered based on H1:
Participants’ comprehension scores will be higher with professional rather than nonprofessional subtitles. It was tested through seven sub-hypotheses involving Attention
allocation, Mean fixation duration, Skipped subtitles, Attention shifts, Reception
capacity, Subtitle-reading effort, and Self-reported comprehension. In postulating the
sub-hypothesis, I contend professional subtitles result in higher comprehension scores,
and that this can be supported with eye-tracking measurements. This assumes
professional subtitles are created following professional standards that provide
instructions for subtitles which are more appropriate for reading. Among these subhypotheses, only two were confirmed (Mean fixation duration and Attention shifts) and
one was partially confirmed (Reception capacity).
Table 104. Confirmation of sub-hypotheses for H1: Type of subtitle
H1.1
H1.2
H1.3
H1.4
H1.5
H1.6
H1.7
Variable
Reception capacity
Subtitle-reading effort
Self-reported comprehension
Attention allocation
Mean fixation duration
Skipped subtitles
Attention shifts
Confirmation
()
H1: not confirmed
Considering these results, it is safe to assume that the reception of professional
and non-professional subtitles in this study is not different at all levels of reception.
There was no evidence that differences in the overall reading behavior correlated with
the types of subtitle. Furthermore, the results for the self-reported measurements did not
indicate that the type of subtitle was a relevant factor from the participants’ perspective.
The mean fixation duration for the professional subtitles was significantly shorter
than for the others. However, these differences might have been triggered by the layout
230
Chapter 7. Conclusions
of the subtitles on the screen. Since the other eye-tracking measurements did not vary
significantly, I consider this to be a plausible explanation. Further research will be
needed in order to test this hypothesis.
The significant difference between the professional and non-professional versions
in terms of the number of attention shifts between the text and the image supports the
professionally accepted guideline that a time span of four frames is needed between
subtitles. According to Carroll and Ivarsson (1998), this span allows the human eye to
realize there has been a change. The eye-tracking data in this study indicate that the time
span between subtitles makes it possible for viewers to have a smoother reading
process, alternating their fixations between the image and the subtitles. On the other
hand, a shorter period between subtitles caused the participants to remain on the subtitle
area for longer periods of time.
Surprisingly, the results for Reception capacity varied, with poorer score
performances for the Iberian non-professional version only. The scores for the
participants when watching the two other versions of the subtitles were similar, but they
gave fewer correct answers to the content questionnaire after watching the Iberian nonprofessional subtitles. This means the hypothesis was only partially confirmed. Taking a
closer look at the results, only the Verbal attention questions were lower: the Narrative
and Iconic attention variables did not vary significantly according to the type of subtitle.
This suggests the difference in scores is indeed related to the content of the subtitles and
not to the other factors that constitute Reception capacity. Further, these results offer
thought-provoking initial insights into viewers’ performance as an indicator of quality.
Additionally, one main hypothesis was proposed based on the comparison
between the two non-professional versions. H3 stated: Participants’ reception scores
will be higher with the non-professional subtitles produced in Spain (their own country)
than with the non-professional subtitles produced in Latin America. Three subhypotheses, related to self-reported questions and recall testing, were formulated guided
by the assumption that the participants, being speakers of Iberian Spanish, would
identify more and obtain better results with the Iberian version than with the LatinAmerican version.
Surprisingly, the results were the opposite: the scores for the Latin-American
neutral version were higher than those for the Iberian version, and there were no
significant differences in terms of the self-reported measurements, namely Subtitlereading effort and Self-reported comprehension.
231
The Reception of (Non)Professional Subtitling
Table 105. Confirmation of sub-hypotheses for H3: Type of non-professional subtitles
H3.11
H3.12
H3.13
Variable
Reception capacity
Subtitle-reading effort
Self-reported comprehension
Confirmation
H3: not confirmed
Does the users’ level of proficiency in L2/L3 affect the reception of professionally and
non-professionally subtitled audiovisual material?
This question was related to H2: Participants with a high level of proficiency in L2/L3
will be less dependent on subtitles than will participants with a low level of L2/L3. In
order to provide an answer, three sub-hypotheses were postulated based on the rationale
that the participants with a high level of proficiency in English would look less at the
subtitle area. The sub-hypotheses involved the variables Attention allocation, Mean
fixation duration, and Audience enjoyment. The Attention allocation and Audience
enjoyment hypotheses were confirmed, while only the Mean fixation duration on the
image was significantly affected. Thus, based on the data for this research, the level of
proficiency in L2/L3 does affect the reception of subtitled content, regardless of the
subtitles, as suggested by the results for H1.
Table 106. Confirmation of sub-hypotheses for H2: Level of L2/L3
H2.8
H2.9
H2.10
Variable
Attention allocation
Mean fixation duration
Audience enjoyment
Confirmation
()
H2: confirmed
The participants in the high level of English and low level of English groups had
very different behaviors. The participants with a low level of English fixated more and
for longer on the subtitle area, as expected, and they skipped almost no subtitles.
Additionally, these participants had a more homogeneous behavior. On the other hand,
the participants in the high level of English group had more variation in their reading,
with some participants exhibiting a behavior similar to that of the low level of English
participants, and others displaying a completely opposite behavior, spending almost no
time on the subtitles. In general, they had a lower average of fixations and duration of
fixations on the subtitle area and they skipped almost a quarter of the subtitles.
However, this did not affect the mean fixation duration, which did not vary
significantly between the two groups. One of the most relevant findings is that the level
of L2/L3 affects the mean fixation duration on the image area but not that on the subtitle
232
Chapter 7. Conclusions
area. Since they fixate mostly on the image, the gaze of the participants with a high
level of English remain at the center of the screen. They make more fixations but also
allocate more time to the image area. On the other hand, the participants with a low
level of English are constantly shifting their focus, which indicates a more segmented
visualization. This points to one additional effect of subtitle reading: when participants
fixate for longer on the subtitles, they make significantly shorter fixations on the image.
Although the participants’ level of proficiency in English did not affect their selfreported comprehension, it did affect enjoyment. The scores for audience enjoyment are
lower for participants in the low level of English group. Reading subtitles is seen as a
bothersome action in this group and reduced the participants’ enjoyment. During the
interviews, the participants commented that they think the subtitles were cognitively
demanding.
A subgroup of the high level of English group behaved in a manner different
from all other participants. They managed to skip a large amount of the subtitles and
looked at the subtitle area only on specific occasions, possibly to gather information that
was undiscernible to them from the audio track. This group of viewers showed there is a
degree of control on the viewer’s side (d’Ydewalle et al. 1989). Interestingly, the group
comprised participants who said they never used subtitles when watching audiovisual
materials.
Do participants notice any difference between professional and non-professional
subtitles?
Two sub-hypotheses were postulated based on the assumption that participants would
feel more inclined towards the professional version and, when comparing the nonprofessional versions only, towards the Iberian non-professional version. One of the
most surprising outcomes from the study was the participants’ inability to identify the
provenance of the subtitles. In terms of language, the Iberian versions (both professional
and non-professional) and the Latin-American versions are clearly marked as belonging
to the two language varieties. For instance, the Iberian version uses vosotros for
translating you, while the Latin-American version uses ustedes. However, the
participants did not point this out during the interviews. Some of them did comment that
there are different versions of subtitles in Spanish on the Internet, namely the Latin-
233
The Reception of (Non)Professional Subtitling
American version and the Iberian version, but they did not liken that situation to the
subtitles they had just seen.
Table 107. Confirmation of Quantitative data sub-hypotheses: H14 and H15
Variable
Inclination towards professional subtitles
Inclination towards Iberian non-professional subtitles
H14
H15
7.2.
Confirmation
H14: not confirmed
H15: not confirmed
Complementary findings
Other quantitative variables included in the study as well as the qualitative data from the
interviews provided additional results that could help advance not only research on nonprofessional subtitling, but also the field of reception studies in audiovisual translation.
First of all, this research has shown there are different type of subtitle-reading
behaviors related to different variables that affect the way in which the participants read
subtitles. All of these factors help shape the subtitle reading behavior. The most relevant
one, as discussed above, is the level of proficiency in the source language. However,
one element that also provided interesting insights was the participants’ use of
subtitling. As shown, the frequency with which participants used subtitles in everyday
life correlated with different patterns of eye movements. Up to now, studies on
reception resort to a crude classification of participants into two groups: viewers who
are used to subtitles and viewers who are used to dubbing. The findings of this study
indicate that a more detailed analysis of these variables could help explain participants’
behavior.
The eye-tracking measurements also indicate differences in behavior related to
the audiovisual input. Interestingly, the analyzed variables consistently showed the
participants had more difficulty understanding one of the clips than the other two. This
difficulty was reflected in an increased allocation of attention to the subtitle area.
Among participants with a high level of English, those who never use subtitles
and fixated the least on the subtitle area had longer fixations when they looked at the
subtitles. Most of the time their gaze was fixated on the image, but their deflections to
the subtitle area, probably prompted by a difficulty in understanding the audio track,
accounted for longer mean fixation durations. This might support the assumption that
longer and more fixations can be used as indicators of cognitive effort (Saldanha and
O’Brien 2013; Kruger et al. 2015).
234
Chapter 7. Conclusions
In line with the new patterns of behavior in the consumption of audiovisual
material, this study has shown that the traditional division between dubbing and
subtitling countries no longer holds, as previously pointed out by Chaume (2013) and
Casarini (2014). Although in most cases the participants said that subtitling was not
their preferred audiovisual translation modality, they were willing to put up with it in
daily life due to a set of social and personal conditions, such as availability of the
content, interest in language learning, or social engagement. Their decision-making
process when choosing which translation mode to use has become more complex;
having access to both dubbed and subtitled products, they are now more conscious of
how to use them to their advantage or according to their needs.
Most of the participants had a negative image of non-professional subtitling but
recognized there are varying degrees of quality. Some even mentioned they would try
different subtitles if the ones they found first had spelling mistakes or were badly
synchronized. It could thus be said that these viewers have some of the characteristics
described as being typical of more active and proficient users of audiovisual content,
and that this is affecting their general consumption habits.
7.3.
Limitations of the research
Defining the research design is always a trade-off process. Including or excluding a
method or a variable can help the researcher have a more solid design, while at the same
time limit the generalizability of the findings. The scope of this study and the selected
methodology constitute the first limitations of the study. This research explores the
reception of only one type of subtitling: professional-amateur (pro-am) subtitles. The
wide range of non-professional subtitling practices and the different types of
communities make it almost impossible to generalize the findings. As shown by the
subtitles in this study, one non-professional version achieved results similar to those of
professional subtitles, while the other was ranked lower.
I decided to exclude the participants with a mid-level of proficiency in English in
order to compare the extremes and reduce the number of people required for the study.
However, looking back and considering the interesting findings provided by the
differences in behavior, I can but wonder how those participants in the middle would
have compared to those at the extremes. I nevertheless consider that this self-imposed
limitation allowed me to see more clearly the differences in the behavior of the two
235
The Reception of (Non)Professional Subtitling
extreme types of participants. Another element that could have yielded interesting
results is the inclusion of a control group of native speakers of English. This would
provide a point of comparison when analyzing eye movements, comprehension of
content and degree of enjoyment.
Both the number of participants and the analyses performed are affected by the
decision to include eye tracking among the methods. As thoroughly explained in
Section 4.2, eye-tracking data provide detailed and mostly accurate measurements of the
participants’ eye movements. Such comprehensive logging of gaze information also
means an onerous amount of raw data. The manipulation and processing of that data are
challenging and time-consuming. Given that in audiovisual translation we still do not
have specific software to assist in this process, most of the data manipulation,
processing and analysis has to be done manually. Eye-tracking software is more
advanced for the analysis of static input such as text, than dynamic input such as films.
The demands of the data processing affect the number of participants that can be
included in a given study, since the larger the sample, the larger the data the researcher
has to process.
Considering the number of participants, I managed to include a fairly large
sample. When compared with other eye-tracking studies in the field, it is one of the
largest samples to date. Additionally, the data quality of the eye-tracking data was in
general good, since I only had to exclude about 5% of the eye-tracking data due to poor
quality. Nevertheless, here again the trade-off problem surfaces: in order to process the
data for all my participants, I decided to rely on the distribution of attention
measurements only. This is criticized by Kruger and Steyn (2014) since it lacks a
detailed analysis of the individual subtitles.
Attention allocation measurements are based on the assumption that the fixations
within the subtitle area should be considered attention dedicated to the subtitles. With
this approach, the researcher draws two areas on the screen, one for the image and one
for the subtitles. Both of them are kept active from the beginning to the end of the
recordings. Kruger and Steyn (2014) disapprove of this because in most cases it
involves a crude definition of the areas of interest and does not exclude the
measurements obtained from the subtitle area when no subtitles are shown on the
screen. Taking this into account, I ruled out the possible misleading effects of
maintaining the subtitle area always active by testing the differences in the results
between a subtitle area which is active from the beginning to the end of the recording
236
Chapter 7. Conclusions
and one that is activated and deactivated with the onset and offset of every subtitle. The
differences between the two options were minimal. Further, due to parameters such as
re-reading and latency, even if there is no subtitle shown on the screen, in some
circumstances it could be argued that the viewer’s gaze is directed to the subtitle area
with the intention of reading the subtitles or that the gaze remained on the subtitle area
waiting for the next subtitle to appear. More research is needed to test how this affects
the measurements and to define whether it constitutes a limitation or not.
The analysis of individual subtitles was not considered viable for this study. As
reported again by Kruger and Steyn (2014), this kind of analysis entails incredibly
demanding manual work. Even checking a subsample of subtitles per participant could
take a considerable amount of time and effort. Although it could yield interesting
results, it is unfeasible in a project with a large sample and only one researcher.
Hopefully, in the future, better tools will help explore specific features and make more
detailed analyses, such as isolating those subtitles that are read by the participants with a
high level of English in order to search for a pattern in their deflections.
In order to keep the Clip variable as controlled as possible, I decided to focus on
one audiovisual genre only. As shown by the results, this goal was actually threatened
by the fact that one of the clips turned out to be more cognitively demanding for the
participants. This shows that even the selection of excerpts from the same product does
not imply straightforward comparability. I still think that including excerpts from
different genres or products would entail even more disparity and could pose a
challenge when interpreting the results. Based on the outcome for the Clip variable, I
now think a stricter selection process is necessary. Perhaps a set of pre-selected clips
could be tested with people who fit the profile of the participants of the study. The
participants in this pre-testing would watch the clips and rate how difficult they are to
understand. Additionally, they could describe the content of the clip and pinpoint the
salient narrative, verbal and iconic elements they can recall. These initial results would
offer the researcher initial data to make a more informed decision when selecting videos
for the experiment and preparing comprehension questionnaires.
The number of recall questions included in the questionnaire is also a limiting
factor. Each type of information, namely Narrative, Verbal and Iconic, was tested based
on two questions only, one open-recall question and one cued-recall question. The
reduced number of questions responded mainly to the time constraints imposed. Each
session combining the eye-tracking experiment and the interview lasted for about 45
237
The Reception of (Non)Professional Subtitling
minutes per participant. Including a longer questionnaire for each clip would have
increased the duration of the session. Since the participants did not receive any
payment, it would have been difficult to achieve the number of participants required had
I asked them to give me more than one hour of their time.
The aim of this research was to approach the reception of the subtitles from the
participants’ point of view. It did not include an assessment of the subtitles according to
professional standards or theoretical considerations. This choice could be seen as a
limitation, depending on where you stand. In my case, I was more interested in
discovering how the participants would assess and react to the translations. My goal was
to draw some conclusions based on the results of the participants’ performance and
assessment.
Finally, it is necessary to recognize the inherent limitation of this type of
experiment, the “observer’s paradox” (Labov 1972). For the design of the project I
wanted to test the use of subtitling in conditions similar to the participants’ experience.
In doing so, I decided to use a popular TV series that could be familiar to the
participants. However, it is only safe to assume some participants do not feel at ease
under experimental conditions. This was one of the reasons for randomizing the
presentation of the subtitles. Additionally, after the pilot study and considering the
feedback indicating a potentially higher awareness of the experimental conditions at the
beginning of the session, I decided to include a short video clip from the same series
with a twofold purpose: checking the calibration of the eye tracker and giving the
participants some time to become used to the setting. The participants did not mention
the setting or the eye tracker as stressful elements. Further, the main differences in
measurement were caused by the excerpt that was in the second position, not the first
one.
7.4.
Applicability
This research aligns in various ways with Tymoczko’s (2005) predictions about the
trajectory of research in Translation Studies: it assumes the concept of translation can be
extended to non-professional practices of translation that were fueled by the expansion
and democratization of technology, while at the same time looks at translation as a
result of an international social phenomenon. With that in mind, I hope to have
contributed to Translation Studies in several ways:
238
Chapter 7. Conclusions
-
Based on previous research, I have adopted and tested a methodology that allows
for a holistic exploration of the reception of subtitled material. The addition of
interviews to the more traditional combination of eye tracking and
questionnaires has proved to provide insights into the social conditions that
surround the consumption of translated audiovisual products. The data collected
through the interviews support or question the findings from the other methods,
which makes the overall conclusions more robust.
-
Again in terms of methodological aspects, I have tried to be as meticulous as
possible in the description of the methods deployed for the preparation,
collection and processing of the eye-tracking data. One of the biggest problems I
had was the lack of studies explaining the data collection and processing
methods in detail. In order to fill this gap, I have covered the technical aspects
extensively, with special emphasis on the eye-tracking quality assessment
techniques and their application to research in audiovisual translation.
-
The study has shown that professional and non-professional subtitles can be
compared and that there is variation between non-professional subtitles, at least
from the perspective of the users of the translation. These results come to
complement the already growing literature that supports a range of applications
of non-professional subtitling features to mainstream translation, translator
training and Translation Studies. These applications range from translator
training and the possibility of adapting and integrating non-professional
mechanisms to professional translation practices, to the re-definition of the
translation profession and of translation practice.
-
Companies have decided to use optional glosses in anime DVDs to explain
culture-bound terms for those users who are interested in them. In the same way,
knowing that there are people with high proficiency in the source language who
are capable of identifying what type of information they want to recover from
the subtitles, it becomes possible to create different types of subtitles depending
on the users’ needs.
239
The Reception of (Non)Professional Subtitling
7.5.
Implications and avenues for future research
The present study constitutes the first attempt to explore the reception of professional
and non-professional subtitles using a triangulation of methods. In that sense, it offers a
new approach to non-professional subtitling and adds another layer of understanding. At
the same time, it enlarges the field of reception studies.
I hope the findings motivate more systematic analyses of non-professional
subtitling practices. The non-professional subtitling panorama is wide (O’Hagan 2009,
2012; Fernández Costales 2012; O’Hagan 2013) and includes many areas that are still
uncharted. One of them is the adaptation of non-professional subtitling mechanisms and
tools for translator training (Orrego-Carmona 2013). The development of translator
competence could also benefit from the study of non-professional subtitling. Given that
one of the non-professional subtitles in this study produced results similar to those of
the professional subtitle version, future studies could analyze how non-professional
translators actually develop translator competence (Harris 1977). The difference in the
scores for the two non-professional subtitles included in this study, and the variation
within non-professional subtitling communities in general commented on by many
scholars seems to indicate that some conditions support the development of translator
competence better than others. The identification of these aspects or circumstances
could help advance our understanding of how translators can be trained.
Taking into account that the participants’ performance in the test with one version
of the non-professional subtitles was similar to their performance with the professional
version, regardless of their subjective opinion on non-professional subtitling, one
possible area for research is the exploration of the effects of people’s negative
predispositions or implicit theories. Would participants behave in the same way and
obtain the same results it they knew the subtitles were non-professional?
From a general perspective on reception, there are some aspects that could be
developed further in future studies. The differences in behavior between participants
who are highly proficient in the source language of the content and the participants who
have a low level of proficiency in the source language open up a completely new area
for research. First, as mentioned, this research does not provide information about
participants with a middling level of competence in English, even though they made up
the biggest group of my initial sample. Exploring their behavior could provide
interesting insights. Second, a detailed analysis of the fixations of the participants with a
240
Chapter 7. Conclusions
high level of English could yield information on the elements that trigger their
deflections to the subtitle area. Learning about this could help in the development of
subtitles tailored to viewers who are highly proficient in the source language but still
require some degree of support. At the same time, it could benefit language learning by
providing information on the needs of learners. This could be used to create subtitles
that are more suitable to specific learning goals. Third, while I was collecting the data, I
had the impression that participants in both groups reacted differently to the audio track.
Having learned that reading subtitles actually affected self-reported comprehension, I
think it would be interesting to explore the kind of cues that both groups take from the
audio track. Even if they do not understand the linguistic information, the mere fact that
there is some sound might have an effect on the participants. This is an unexplored area
in Translation Studies.
One solution for overcoming the limitation on the number of participants
imposed by eye tracking methods would be to combine questionnaires and eye tracking
in a reception study structured in two complementary phases: a first stage using
questionnaires with a large sample and a second stage collecting eye-tracking data from
a sub-sample. The questionnaire stage would analyze the responses and repercussions of
the material while the eye tracking would provide information on their reactions.
One variable that I did not expect would have an effect but that I now think is
important is the layout of the subtitles. On the one hand, the participants commented on
the layout of the subtitles as being one of the salient elements when judging the
professionalism of the translation. On the other, it seems the layout also influenced the
eye-tracking measurements. This could explain the cases in which the two nonprofessional versions give similar results but are different from the professional version.
As shown in Figure 49, the layout is one aspect the two non-professional translations
have in common. Future research could explore the effect of the subtitle layout on the
participants’ eye movements.
Finally, although it might look like a far-fetched assumption since it is out of the
scope of this thesis, I think the participants’ performance on reception capacity testing
could be extended to be used as a type of quality assessment. The findings indicate the
performance of participants in the verbal attention test hints at differences between the
subtitles. The results were robust enough to show differences between the three types of
subtitles the participants were exposed to. This adds yet another aspect that could be
taken into account as part of quality assurance or translation assessment: user reception.
241
The Reception of (Non)Professional Subtitling
Verbal attention was tested with only two questions, so the results should be taken with
care. However, these initial results could help shape a quality-assessment process that
involves the target users as raters of the translations. Much like the system put in place
by Facebook to test its crowdsourced translation, testing audience reactions could
confirm translation choices or decide among different options.
242
References
Accenture. 2013. Video-Over-Internet Consumer Survey 2013: Multi-tasking and
Taking Control.
http://www.accenture.com/SiteCollectionDocuments/PDF/Accenture-Video-OverInternet-Consumer-Survey-2013.pdf. Accessed May 2014.
Ahn, Luis von, Benjamin Maurer, Colin McMillen, David Abraham and Manuel Blum.
2008. “reCAPTCHA: Human-based character recognition via web security
measures”. Science 321(5895): 1465–1468.
Allen, Robert Clyde. 1985. Speaking of soap operas. Chapel Hill: University of North
Carolina Press.
Anderson, Chris. 2006. The long tail: Why the future of business is selling less of more.
New York: Hyperion.
Antonini, Rachele. 2005. “The perception of subtitled humor in Italy”. Humor International Journal of Humor Research 18(2): 209–225.
Antonini, Rachele. 2008. “The perception of dubbese: An Italian study”. In Delia
Chiaro, Christine Heiss and Chiara Bucaria (eds) Between Text and Image:
Updating Research in Screen Translation. Amsterdam and Philadelphia: John
Benjamins Publishing Company. 135–148.
Antonini, Rachele and Delia Chiaro. 2009. “The perception of dubbing by Italian
audiences”. In Jorge Díaz Cintas and Gunilla Anderman (eds) Audiovisual
Translation: Language Transfer on Screen. Great Britain: Palgrave McMillan. 97–
114.
Aris, Annet. 2011. “Managing media companies through the digital transition”. In Mark
Deuze (ed.) Managing Media Work. Thousand Oaks: Sage Publications. 265–278.
Bairstow, Dominique. 2011. “Audiovisual processing while watching subtitled films: A
cognitive approach”. In Adriana Serban, Anna Matamala and Jean-Marc Lavaur
(eds) Audiovisual translation in close-up: Practical and theoretical approaches.
Bern: Peter Lang. 205–219.
Bairstow, Dominique and Jean-Marc Lavaur. 2012. “Audiovisual information
processing by monolinguals and bilinguals: Effects of intralingual and interlingual
243
The Reception of (Non)Professional Subtitling
subtitles”. In Aline Remael, Pilar Orero and Mary Carroll (eds) Audiovisual
translation and media accessibility at the crossroads. Amsterdam: Rodopi. 273–
293.
Balling, Laura W. 2008. “A brief introduction to regression designs and mixed-effects
modelling by a recent convert”. In Susanne Göpferich, Arnt L. Jakobsen and Inger
M. Mees (eds) Looking at Eyes: Eye-tracking studies of reading and translation
processing. Frederiksberg: Samfundslitteratur. 175–192.
Barra, Luca. 2009. “The mediation is the message: Italian regionalization of US TV
series as co-creational work”. International Journal of Cultural Studies 12(5): 509–
525.
Bielby, Denise D. and C. Lee Harrington. 2008. Global TV: Exporting television and
culture in the world market. New York: New York University Press.
Bisson, Marie-Josée, Walter J. B. van Heuven, Kathy Conklin and Richard Tunney.
2014. “The role of repeated exposure to multimodal input in incidental acquisition
of foreign language vocabulary”. Language Learning 64(4): 855–877.
Bogucki, Łukasz. 2009. “Amateur subtitling on the Internet”. In Jorge Díaz Cintas and
Gunilla Anderman (eds) Audiovisual Translation: Language Transfer on Screen.
Great Britain: Palgrave McMillan. 49–57.
Brems, Elke and Sara Ramos Pinto. 2013. “Reception and translation”. In Yves
Gambier and Luc van Doorslaer (eds) Handbook of Translation Studies, vol. 4.
Amsterdam and Philadelphia: John Benjamins Publishing Company. 142–147.
Bruycker, Wim de and Géry d’Ydewalle. 2003. “Reading native and foreign language
television subtitles in children and adults”. In Ralph Radach, Jukka Hyona and
Heiner Deubel (eds) The mind’s eye: cognitive and applied aspects of eye movement
research. Elsevier Science. 671–684.
Bucaria, Chiara and Delia Chiaro. 2007. “End user perception of screen translation: The
case of Italian dubbing”. Tradterm 13(1): 91–118.
Caffrey, Colm. 2009. Relevant abuse? Investigating the effects of an abusive subtitling
procedure on the perception of TV anime using eye tracker and questionnaire. PhD
thesis. Dublin: Dublin City University.
244
References
Cañadas Osinki, Isabel and Alfonso Sánchez Bruno. 1998. “Categorías de respuesta en
escalas tipo Likert”. Psicothema 10(3): 623–631.
Carroll, Mary and Jan Ivarsson. 1998. Code of good subtitling practice.
Casarini, Alice. 2014. The perception of American adolescent culture through the
dubbing and fansubbing of a selection of US teen series from 1990 to 2013. PhD
thesis. Forlì: Università di Bologna.
Castells, Manuel. 2000. The information age: Economy, society and culture. Oxford:
Blackwell.
Cavaliere, Flavia. 2008. “Measuring the perception of the screen translation of Un Posto
al Sole: A cross-cultural study”. In Delia Chiaro, Christine Heiss and Chiara Bucaria
(eds) Between Text and Image: Updating Research in Screen Translation.
Amsterdam and Philadelphia: John Benjamins Publishing Company. 165–180.
CBS. 2007-present. The Big Bang Theory. Created by Chuck Lorre and Bill Prady.
Chaume, Frederic. 2013. “The turn of audiovisual translation: New audiences and new
technologies”. Translation Spaces 2.
Chesterman, Andrew. 2004. Functional quality. Unpublished lecture. Tarragona.
Chesterman, Andrew. 2007. “Bridge concepts in translation sociology”. In Michaela
Wolf and Alexandra Fukari (eds) Constructing a sociology of translation.
Amsterdam, Philadelphia: John Benjamins Publishing Company. 171.
Chiaro, Delia. 2004. “Investigating the perception of translated verbally expressed
humour on Italian TV”. ESP Across Cultures 1(1): 35–52.
Chiaro, Delia. 2007. “The effect of translation on humour response”. In Yves Gambier,
Miriam Shlesinger and Radegundis Stolze (eds) Doubts and Directions in
Translation Studies, vol. 72. Amsterdam and Philadelphia: John Benjamins
Publishing Company. 137–152.
Chu, Donna S. 2012. “Fanatical labor and serious leisure: A case of fansubbing in
China”. In Francis L. Lee, Louis Leung, Jack L. Qiu and Donna S. C. Chu (eds)
Frontiers in new media research. Hoboken: Taylor and Francis. 259–277.
Cornu, Jean-François. 2013. “De Barcelone aux bords du Rhin : le doublage et le soustitrage au fil des congrès”. L’écran traduit 2(automne 2013): 107–122.
245
The Reception of (Non)Professional Subtitling
Creswell, John W. and Vicki Plano Clark. 2011. Designing and conducting mixed
methods research. Los Angeles: Sage Publications.
Cronin, Michael. 2012. Translation in the digital age. London, New York: Routledge.
Cubbison, Laurie. 2005. “Anime fans, DVDs, and the authentic text”. The velvet light
trap 56(1): 45–57.
d’Ydewalle, Géry. 2002. Foreign-language acquisition by watching subtitled television
programs. http://www.kansaiu.ac.jp/fl/publication/pdf_education/04/4geryd'ydewalle.pdf. Accessed January
2015.
d’Ydewalle, Géry and Wim de Bruycker. 2007. “Eye movements of children and adults
while reading television subtitles”. European Psychologist 12(3): 196–205.
d’Ydewalle, Géry and Ingrid Gielen. 1992. “Attention allocation with overlapping
sound, image, and text”. In Keith Rayner (ed.) Eye Movements and Visual
Cognition: Scene Perception and Reading. New York, NY: Springer New York.
415–427.
d’Ydewalle, Géry, Patrick Muylle and Johan van Rensbergen. 1985. “Attention shifts in
partially redundant information situations”. In Rudolf Groner, George McConkie
and Christine Menz (eds) Eye Movements and Human Information Processing.
Amsterdam: Elsevier Science. 375–384.
d’Ydewalle, Géry and Ubolwanna Pavakanun. 1995. “Acquisition of a second/foreign
language by viewing a television program”. In Peter Winterhoff-Spurk (ed.)
Psychology of Media in Europe. Wiesbaden: VS Verlag für Sozialwissenschaften.
51–64.
d’Ydewalle, Géry, Caroline Praet, Karl Verfaillie and Johan van Rensbergen. 1991.
“Watching subtitled television: Automatic reading behavior”. Communication
Research 18(5): 650–666.
d’Ydewalle, Géry and Marijke Van de Poel. 1999. “Incidental foreign-language
acquisition by children watching subtitled television programs”. Journal of
Psycholinguistic Research 28(3): 227–244.
d’Ydewalle, Géry and Johan van Rensbergen. 1989. “Developmental studies of textpicture interactions in the perception of animated cartoons with text”. In Heinz
246
References
Mandl and Joel R. Levin (eds) Knowledge Acquisition from Text and Pictures,
vol. 58. Elsevier Science. 233–248.
d’Ydewalle, Géry, Johan van Rensbergen and Joris Pollet. 1987. “Reading a message
when the same message is available auditorily in another language: The case of
subtitling”. In J. K. O’Regan and Ariane Lévy-Schoen (eds) Eye Movements from
Physiology to Cognition. Amsterdam: Elsevier Science. 313–321.
d’Ydewalle, Géry, Luc Warlop and Johan van Rensbergen. 1989. “Television and
attention: Differences between younger and older adults in the division of attention
over different sources of TV information”. Medienpsychologie 1: 42–57.
de Linde, Zoé and Neil Kay. 1999. The Semiotics of Subtitling. Manchester: St. Jerome
Pub.
Deuze, Mark. 2006. “Participation, remediation, bricolage: Considering principal
components of a digital culture”. The Information Society 22(2): 63–75.
Deuze, Mark. 2007. Media Work. Cambridge: Polity.
Díaz Cintas, Jorge and Pablo Muñoz Sánchez. 2006. “Fansubs: Audiovisual translation
in an amateur environment”. JoSTrans, The Journal of Specialised Translation (6):
37–52.
Díaz Cintas, Jorge and Aline Remael. 2007. Audiovisual translation: Subtitling.
Manchester, UK, Kinderhook, NY: St. Jerome Pub.
Doherty, Stephen. 2012. Investigating the effects of controlled language on the reading
and comprehension of machine translated texts: A mixed-methods approach. PhD
thesis. Dublin: Dublin City University.
Drugan, Joanna. 2011. “Translation ethics wikified: How far do professional codes of
ethics and practice apply to non-professionally produced translation?”. Linguistica
Antverpiensia 10: 111–127.
Duchowski, Andrew T. 2007. Eye Tracking Methodology: Theory and Practice.
London: Springer.
Dwyer, Tessa. 2012. “Fansub dreaming on ViKi”. The Translator 18(2): 217–243.
Dwyer, Tessa and Ioana Uricaru. 2009. “Slashings and subtitles: Romanian media
piracy, censorship, and translation”. The velvet light trap 63(1): 45–57.
247
The Reception of (Non)Professional Subtitling
Farrelly, Glen. 2008. “Does Rotten Tomatoes spoil users? Examining whether social
media features foster participatory culture”. Stream: Culture/Politics/Technology
1(2): 38–48.
Feitosa, Marcos P. 2009. Legendagem comercial e legendagem pirata: um estudo
comparado. Postgraduate thesis. Minas Gerais: Faculdade de Letras da UFMG.
Fernández Costales, Alberto. 2012. “Collaborative Translation Revisited: Exploring the
Rationale and the Motivation for Volunteer Translation”. Forum 10(1): 115–142.
Fernández Costales, Alberto. 2013. “Crowdsourcing and collaborative translation: mass
phenomena or silent threat to Translation Studies?”. Hermēneus 15: 85–110.
Ferrer Simó, María R. 2005. “Fansubs y scanlations: la influencia del aficionado en los
criterios profesionales”. Puentes (5): 27–44.
Fuentes Luque, Adrián. 2003. “An empirical approach to the reception of AV translated
humour”. The Translator 9(2): 293–306.
Gambier, Yves. 2003. “Introduction”. The Translator 9(2): 171–189.
Gambier, Yves. 2006. “Multimodality and audiovisual translation” Audiovisual
Translation Scenarios (Marie Curie Euroconferences MuTra), Copenhagen, May 15. In Mary Carroll, Heidrun Gerzymisch-Arbogast and Sandra Nauert (eds).
Gambier, Yves. 2008. “Recent developments and challenges in audiovisual translation
research”. In Delia Chiaro, Christine Heiss and Chiara Bucaria (eds) Between Text
and Image: Updating Research in Screen Translation. Amsterdam and Philadelphia:
John Benjamins Publishing Company. 11–36.
Gambier, Yves. 2009. “Challenges in research on audiovisual translation”. In Anthony
Pym and Alexander Perekrestenko (eds) Translation Research Projects 2.
Tarragona: Intercultural Studies Group. 17–25.
Gambier, Yves. 2012. “The position of audiovisual translation studies”. In Carmen
Millán and Francesca Bartrina (eds) The Routledge handbook of translation studies.
Milton Park, Abingdon, NY: Routledge. 45–59.
Gambier, Yves. 2014. “Changing landscape in translation”. International Journal of
Society, Culture & Language 2(2): 1–12.
248
References
Ghia, Elisa. 2012. “The impact of translation strategies on subtitle reading”. In Elisa
Perego (ed.) Eye Tracking in Audiovisual Translation. Roma: Aracne Editrice. 155–
182.
Godin, Seth. 2008. Tribes: We need you to lead us. London: Penguin Group.
Gottlieb, Henrik. 1995. “Establishing a framework for a typology of subtitle reading
strategies - Viewer reactions to deviations from subtitling standards”. Translatio
(FIT Newsletter) 143-4: 388–409.
Green, Spence, Jeffrey Heer and Christopher D. Manning. 2013. “The efficacy of
human post-editing for language translation” CHI 2013: Changing Perspectives,
Paris, France. 439–448.
Harris, Brian. 1977. The importance of natural translation. Ottawa: Ecole de
traducteurs et d'interprètes.
Hemmungs Wirtén, Eva. 2012. “Swedish subtitling strike called off! Fan-to-fan piracy,
translation, and the primacy of authorisation”. In Dan Hunter, Ramon Lobato,
Megan Richardson and Julian Thomas (eds) Amateur Media: Social, Cultural and
Legal Perspectives. Routledge. 125–136.
Hippel, Eric von. 2005. Democratizing innovation. Cambridge, Mass: MIT Press.
Holmqvist, Kenneth. 2011. Eye Tracking: A Comprehensive Guide to Methods and
Measures. Oxford, New York: Oxford University Press.
Hvelplund, Kristian T. Forthcoming. “Eye tracking and the process of dubbing
translation”. In Jorge Díaz Cintas and Kristijan Nikolić (eds) As yet untitled volume
with contributions from the Media for All 5 conference.
Hvelplund, Kristian T. 2011. Allocation of cognitive resources in translation: An eyetracking and key-logging study. PhD thesis. Copenhagen: Copenhagen Business
School.
Hvelplund, Kristian T. 2014. “Eye tracking and the translation process: reflections on
the analysis and interpretation of eye tracking data”. In Ricardo Muñoz (ed.)
Minding Translation - MonTI Special Issue 1. 201–223.
Izwaini, Sattar. 2014. “Amateur translation in Arabic-speaking cyberspace”.
Perspectives 22(1): 96–112.
249
The Reception of (Non)Professional Subtitling
Jenkins, Henry. 2004. “The Cultural Logic of Media Convergence”. International
Journal of Cultural Studies 7.
Jenkins, Henry. 2006. Convergence culture: Where old and new media collide. New
York: New York University Press.
Jensema, Carl, Ramalinga S Danturthi and Robert Burch. 2000a. “Time spent viewing
captions on television programs”. American Annals of the Deaf 145(5): 464–468.
Jensema, Carl J. 1998. “Viewer reaction to different television captioning speeds”.
American Annals of the Deaf 143(4): 318–324.
Jensema, Carl J., Sameh el Sharkawy, Ramalinga S. Danturthi, Robert Burch and David
Hsu. 2000b. “Eye movement patterns of captioned television viewers”. American
Annals of the Deaf 145(3): 275–285.
Johnson, R. B., Anthony J. Onwuegbuzie and Lisa A. Turner. 2007. “Toward a
definition of mixed methods research”. Journal of Mixed Methods Research 1(2):
112–133.
Just, Marcel A. and Patricia A. Carpenter. 1980. “A theory of reading: from eye
fixations to comprehension”. Psychological review 87(4): 329–354.
Kayahara, Matthew. 2005. “The digital revolution: DVD technology and the
possibilities for audiovisual Translation Studies”. The Journal of Specialised
Translation (3): 45–48.
Koolstra, Cees M., Tom H. A. van der Voort and Géry d'Ydewalle. 1999. “Lengthening
the Presentation Time of Subtitles on Television: Effects on Children’s Reading
Time and Recognition”. Communications 24(4).
Kovačič, Irena. 1995. “Reception of subtitles. The non-existent ideal viewer”.
Translatio (FIT Newsletter) 143-4: 376–383.
Kreb, Joris. 2011. -=$ub©ulture$=-: Towards a description of the Dutch language
amateur Subtitling scene on the Internet. Master's thesis. Utrecht: Utrecht
University.
Krejtz, Izabela, Agnieszka Szarkowska and Krzysztof Krejtz. 2013. “The effects of shot
changes on eye movements in subtitling”. Journal of Eye Movement Research 6(5):
1–12.
250
References
Kruger, Jan-Louis. 2014. “The impact of subtitles on psychological immersion”. Paper
delivered at Languages and the Media 2014 Conferece. Berlin, 2014.
Kruger, Jan-Louis, Esté Hefer and Gordon Matthew. 2013. “Measuring the Impact of
Subtitles on Cognitive Load: Eye Tracking and Dynamic Audiovisual Texts”
Proceedings of Eye Tracking South Africa (Proceedings of Eye Tracking South
Africa), Cape Town, August 29-31. 62–66.
Kruger, Jan-Louis and Faans Steyn. 2014. “Subtitles and eye tracking: Reading and
performance”. Reading Research Quarterly 49(1): 105–120.
Kruger, Jan-Louis, Agnieszka Szarkowska and Izabela Krejtz. 2015. “Subtitles on the
moving image: An overview of eye tracking studies”. Refractory: a Journal of
Entertainment Media 25: np.
Künzli, Alexander and Maureen Ehrensberger-Dow. 2011. “Innovative subtitling: A
reception study”. In Cecilia Alvstad, Adelina Hild and Elisabet Tiselius (eds)
Methods and Strategies of Process Research: Integrative approaches in Translation
Studies. Amsterdam and Philadelphia: John Benjamins Publishing Company. 187–
200.
La Forgia, Francesca and Raffaella Tonin. 2009. “In un tranquillo week-end di paura,
un Esorcista volò sul nido del…: Un case study sui rimandi intertestuali nel
sottotitolaggio e doppiaggio italiano e spagnolo della serieSupernatural”. inTRAlinea
11: n.p.
Labov, William. 1972. Sociolinguistic patterns. Oxford: Blackwell.
Lavaur, Jean-Marc and Dominique Bairstow. 2011. “Languages on the screen: Is film
comprehension related to the viewers’ fluency level and to the language in the
subtitles?”. International Journal of Psychology 46(6): 455–462.
Lavaur, Jean-Marc and Sophie Nava. 2008. “Interférences liées au sous-titrage
intralangue sur le traitement des images d'une séquence filmée” Actes du colloque
du Congrès 2007 de la Société Française de Psychologie. In Jean M. Hoc and Yves
Corson (eds), 59-64. Nantes.
Leonard, Sean. 2005. “Progress against the law: Anime and fandom, with the key to the
globalization of culture”. International Journal of Cultural Studies 8(3): 281–305.
251
The Reception of (Non)Professional Subtitling
Lindgren, Simon. 2013. “Sub*culture: Exploring the dynamics of a networked public”.
Transformative Works and Cultures 14: n.p.
Luczaj, Kamil, Magdalena Holy-Luczaj and Karolina Cwiek-Rogalska. 2014.
“Fansubbers: The case of the Czech Republic and Poland”. Journal of Comparative
Research in Anthropology and Sociology – Compaso 5(2): 175–198.
Massidda, Serenella. 2012. The Italian fansubbing phenomenon. PhD Thesis. Italy:
Università degli Studi di Sassari.
Matelski, Marilyn J. 1999. Soap operas worldwide: Cultural and serial realities.
Jefferson, N.C: McFarland & Co.
McDonough Dolmaya, Julie. 2011a. “The ethics of crowdsourcing”. Linguistica
Antverpiensia 10: 97–110.
McDonough Dolmaya, Julie. 2011b. Wikipedia survey IV (Motivations). Some thoughts
on translation research and teaching. http://mcdonoughdolmaya.ca/2011/08/24/wikipedia-survey-iv-motivations/.
McDonough Dolmaya, Julie. 2012. “Analyzing the crowdsourcing model and its impact
on public perceptions of translation”. The Translator 18(2): 167–191.
Media Consulting Group. 2011. Study on the use of subtitling: The potential of
subtitling to encourage foreign language learning and improve the mastery of
foreign languages EACEA/2009/01. Final Report.
http://www.mcu.es/cine/docs/Novedades/Study_on_use_subtitling.pdf. Accessed
December, 2014.
Media Consulting Group and Peacefulfish. 2007. Study on dubbing and subtitling needs
and practices in the European audiovisual industry. Final Report.
http://ec.europa.eu/translation/LID/index.cfm?fuseaction=main.PublicationDetail&P
BL_ID=480&theme_selector=normal. Accessed December, 2014.
Mesipuu, Marit. 2012. “Translation crowdsourcing and user-translator motivation at
Facebook and Skype”. Translation Spaces 1: 33–53.
Moran, Siobhan. 2012. “The effect of linguistic variation on subtitle reception”. In Elisa
Perego (ed.) Eye Tracking in Audiovisual Translation. Roma: Aracne Editrice. 183–
222.
252
References
Moura, Hudson. 2011. “Subtitling Jia Zhangke's films: Intermediality, digital
technology, and the varieties of foreignness in global cinema”. In Radhika Gajjala
and Rohit Chopra (eds) Global Media, Culture, and Identity: Theory, Cases, and
Approaches. New York: Routledge. 188–202.
Napier, Susan Jolliffe. 2001. Anime from Akira to Princess Mononoke: Experiencing
contemporary Japanese animation. New York: Palgrave.
Nielsen, Jakob. 2006. The 90-9-1 rule for participation inequality in social media and
online communities. http://www.nngroup.com/articles/participation-inequality/.
Accessed November 27, 2014.
Nornes, Abé M. 1999. “For an abusive subtitling”. Film Quaterly 52(3): 17–34.
Nornes, Abé Mark. 2007. Cinema Babel: Translating Global Cinema. Minneapolis:
University of Minnesota Press.
O’Brien, Sharon. 2010. “Eye tracking in translation process research: methodological
challenges and solutions”. Copenhagen Studies In Language 38: 251–266.
O’Hagan, Minako. 2008. “Fan translation networks: An accidental translator training
environment?”. In John Kearns (ed.) Translator and Interpreter Training: Issues,
Methods and Debates. New York: Continuum.
O’Hagan, Minako. 2009. “Evolution of user-generated translation: Fansubs, translation
hacking and crowdsourcing”. The Journal of Internationalisation and Localisation
(1): 94–121.
O’Hagan, Minako. 2011. “Community translation: Translation as a social activity and
its possible consequences in the advent of Web 2.0 and beyond”. Linguistica
Antverpiensia 10: 11–23.
O’Hagan, Minako. 2012. “The impact of new technologies on translation atudies: A
technological turn?”. In Carmen Millán and Francesca Bartrina (eds) The Routledge
handbook of translation studies, 503-518. Milton Park, Abingdon, NY: Routledge.
O’Hagan, Minako. 2013. “Understanding fan-translation: pop culture and geeks too
cool for translation schools?”. In Susana Bayó Belenguer, Eiléan Ní Chuilleanáin,
Cormac Ó Cuilleanáin and Giulia Zuodar (eds) Translation Right or Wrong. 230–
245.
253
The Reception of (Non)Professional Subtitling
Ochoa, Xavier and Erik Duval. 2008. “Quantitative analysis of user-generated content
on the web” Proceedings of the First International Workshop on Understanding
Web Evolution (WebEvolve2008), Beijing, April 22, 2008. 19–26.
Olohan, Maeve. 2014. “Why do you translate? Motivation to volunteer and TED
translation”. Translation Studies 7(1): 17–33.
Orrego-Carmona, David. 2011. The empirical study of non-professional subtitling: A
descriptive approach. Master's minor dissertation. Tarragona: Universitat Rovira i
Virgili.
Orrego-Carmona, David. 2013. “Using non-professional subtitling platforms for
translator training”. Rivista Internazionale di Tecnica della Traduzione 15: 129–144.
Orrego-Carmona, David. 2014a. “Subtitling, video consumption and viewers: The
impact of the young audience”. Translation Spaces 3: 51–70.
Orrego-Carmona, David. 2014b. “Where is the audience? Testing the audience
reception of non-professional subtitling”. Torres-Simón and Orrego-Carmona 2014.
Tarragona: Intercultural Studies Group. 77–92.
Perego, Elisa, Fabio Del Missier and Sara Bottiroli. 2014. “Dubbing versus subtitling in
young and older adults: cognitive and evaluative aspects”. Perspectives: 1–21.
Perego, Elisa, Fabio Del Missier, Marco Porta and Mauro Mosconi. 2010. “The
Cognitive Effectiveness of Subtitle Processing”. Media Psychology 13.
Pérez-González, Luis. 2006. “Fansubbing anime: Insights into the 'butterfly effect' of
globalisation on audiovisual translation”. Perspectives: Studies in Translatology
14(4): 260–277.
Pérez-González, Luis. 2007. “Intervention in new amateur subtitling cultures: a
multimodal account”. Linguistica Antverpiensia 7: 67–80.
Pérez-González, Luis. 2012. “Co-creational subtitling in the digital media:
Transformative and authorial practices”. International Journal of Cultural Studies
16(1): 3–21.
Pérez-González, Luis. 2013. “Amateur subtitling as immaterial labour in digital media
culture: An emerging paradigm of civic engagement”. Convergence: The
International Journal of Research into New Media Technologies 19(2): 157–175.
254
References
Polonska-Kimunguyi, Eva and Patrick Kimunguyi. 2011. “The making of the
Europeans: Media in the construction of pan-national identity”. International
Communication Gazette 73(6): 507–523.
Pope Benedict XVI. 2013. Social networks: Portals of truth and faith; new spaces for
evangelization.
http://www.vatican.va/holy_father/benedict_xvi/messages/communications/docume
nts/hf_ben-xvi_mes_20130124_47th-world-communications-day_en.html. Accessed
December, 2014.
Pym, Anthony. 2011. “Translation research terms: a tentative glossary for moments of
perplexity and dispute”. In Anthony Pym (ed.) Translation Research Projects 3.
Tarragona: Intercultural Studies Group.
Pym, Anthony. 2013. “Inculturation as elephant: On translation and the spread of
literary modernity”. In Brian Nelson and Brigid Maher (eds) Perspectives on
Literature and Translation: Creation, Circulation, Reception. 87–104.
Pym, Anthony, David Orrego-Carmona and Ester Torres-Simón. Forthcoming. “Status
and technology in the professionalization of translators. Market disorder and the
return of hierarchy”. JoSTrans, The Journal of Specialised Translation.
Ramos Pinto, Sara. 2013. “How accessible are audiovisual products: A reception study
of subtitled film”. Paper presented to the 5th International Conference Media for
All: Audiovisual Translation, Expanding Borders. Dubrovnik, 2013.
Rayner, Keith. 1998. “Eye movements in reading and information processing: 20 years
of research”. Psychological Bulletin 124(3): 372–422.
Rayner, Keith and Martin H. Fischer. 1996. “Mindless reading revisited: Eye
movements during reading and scanning are different”. Perception & Psychophysics
58(5): 734–747.
Rayner, Keith and Alexander Pollatsek. 2006. “Eye-movement control in reading”. In
Matthew J. Traxler and Morton A. Gernsbacher (eds) Handbook of
Psycholinguistics. Amsterdam, Boston: Elsevier/Academic Press. 613–658.
Sajna, Mateusz. 2013. “Amateur subtitling - selected problems and solutions”. T21N Translation in Transition 3: 1–18.
255
The Reception of (Non)Professional Subtitling
Saldanha, Gabriela and Sharon O’Brien. 2013. Research Methodologies in Translation
Studies. New York: Routledge.
Secară, Alina. 2011. “R U ready 4 new subtitles? Investigating the potential of social
translation practices and creative spellings”. Linguistica Antverpiensia 10: 153–171.
Seltman, Howard J. 2014. “Experimental design and analysis”, 2014.
Shirky, Clay. 2010. Cognitive Surplus: Creativity and Generosity in a Connected Age.
New York: Penguin Press.
Soluch, Paweł and Adam Tarnowski. 2013. “Eye-Tracking Methods and Measures”. In
Sambor Grucza, Monika Pluzyczka and Justyna Zajac (eds) Translation Studies and
Eye-Tracking Analysis. 85–105.
Storey, Michael. 2009. “Not in 18-49 age group? TV execs write you off”. Arkansas
Online 04/23/2009.
Surowiecki, James. 2005. The Wisdom of Crowds. New York: Anchor Books.
Szarkowska, Agnieszka, Izabela Krejtz, Zuzana Klyszejko and Anna Wieczorek. 2011.
“Verbatim, standard, or edited? Reading patterns of different captioning styles
among deaf, hard of hearing, and hearing viewers”. American Annals of the Deaf
156(4): 363–378.
Szarkowska, Agnieszka, Izabela Krejtz, Krzysztof Krejtz and Andrew T. Duchowski.
2013. “Harnessing the potential of eyetracking for media accessibility”. In Sambor
Grucza, Monika Pluzyczka and Justyna Zajac (eds) Translation Studies and EyeTracking Analysis. 153–183.
Tapscott, Don and Anthony D. Williams. 2006. Wikinomics: How Mass Collaboration
Changes Everything. New York: Portfolio.
Taylor, Christopher J. 2003. “Multimodal transcription in the analysis, translation and
subtitling of Italian films”. The Translator 9(2): 191–205.
Teixeira, Carlos S. C. 2014. The impact of metadata on translator performance: how
translators work with translation memories and machine translation. PhD thesis.
Tarragona: Universitat Rovira i Virgili.
256
References
Thomas, Margaret. 1994. “Assessment of L2 proficiency in second language acquisition
research”. Language Learning – A Journal of Research in Language Studies 44(2):
307–336.
Tobii Technology. 2010. “Tobii Eye Tracking: An introduction to eye tracking and
Tobii Eye Trackers”. White paper, 2010.
Tobii Technology. 2012. The Tobii I-VT fixation filter: Algorithm description.
Torres-Simón, Esther and David Orrego-Carmona (eds). 2014. Translation Research
Projects 5. Tarragona: Intercultural Studies Group.
Trochim, William M. 2006. The Research Methods Knowledge Base, 2nd Edition.
http://www.socialresearchmethods.net/kb. Accessed Juanuary, 2015.
Tuominen, Tiina. 2012. The art of accidental reading and incidental listening: An
empirical study on the viewing of subtitled films. PhD thesis. Tampere: University of
Tampere.
Tymoczko, Maria. 2005. “Trajectories of Research in Translation Studies”. Meta:
Journal des traducteurs 50(4): 1082–1097.
Van de Poel, Marijke and Géry d’Ydewalle. 2001. “Incidental foreign-language
acquisition by children watching subtitled television programs”. In Yves Gambier
and Henrik Gottlieb (eds) (Multi)Media Translation: Concepts, Practices, and
Research. Philadelphia: John Benjamins Publishing Company. 259–274.
Warner Bros, cop. 2010. The Big bang theory. Created by Chuck Lorre and Bill Prady.
Madrid.
Wilcock, Simone. 2013. A comparative analysis of fansubbing and professional DVD
subtitling. Master's Thesis. Johannesburg: University of Johannesburg.
Wilk, Richard R. 2002. “Television, time, and the national imaginary in Belize”. In
Faye D. Ginsburg, Lila Abu-Lughod and Brian Larkin (eds) Media Worlds:
Anthropology on New Terrain. Berkeley: University of California Press. 171–186.
Williams, Jenny and Andrew Chesterman. 2002. The Map: A Beginner's Guide to
Doing Research in Translation Studies. Manchester, U.K, Northampton, Mass: St.
Jerome Pub.
257
The Reception of (Non)Professional Subtitling
Wissmath, Bartholomäus, David Weibel and Rudolf Groner. 2009. “Dubbing or
subtitling?”. Journal of Media Psychology: Theories, Methods, and Applications 21(3):
114–125.
258
Appendices
Appendix 1.
Consent form
Estudio sobre recepción de material audiovisual
Consentimiento informado del participante en la investigación
Código de Ética
El presente documento presenta en líneas generales el contexto del proyecto de investigación,
así como los derechos de los participantes que aceptan colaborar con en el estudio. También
expone las normas de recolección, manejo y archivo de los datos para que el participante tenga
asegurados sus derechos fundamentales de confidencialidad.
Persona responsable del proyecto: David Orrego Carmona
Contacto: david.orrego.carmona@gmail.com, +34 633 770 930
Contexto general del proyecto
El proyecto de investigación para el cual pedimos su colaboración tiene una duración estimada
de tres años. Corresponde a la investigación doctoral cuyos resultados serán presentados en
forma de una tesis y artículos que se desprendan de ella. La investigación se desarrolla dentro
del Intercultural Studies Group del Departamento de Estudios Ingleses y Alemanes de la
Universitat Rovira i Virgili.
Objetivo del proyecto
Este proyecto tiene por objetivo el estudio del comportamiento de los espectadores en relación
con material audiovisual producido en los Estados Unidos y distribuido ampliamente en España.
Con este fin, los datos se recolectarán por medio de la monitorización y grabación de los
movimientos oculares de los espectadores, la grabación de las respuestas a unos cuestionarios
de comprensión y la grabación de una entrevista posterior al experimento.
Confidencialidad y condiciones de archivo de los datos
Datos personales: Los datos personales (nombre, edad, contacto, residencia) de los participantes
sólo serán gestionados por David Orrego Carmona y únicamente para efectos de contacto
urgente. Tales datos serán salvaguardados en todo momento y no se compartirán con ninguna
persona ajena al proyecto o institución. Para efectos de procesamiento y análisis de los datos,
los participantes serán identificados con una referencia numérica. Dichas referencias se
mantendrán en archivos independientes.
259
The Reception of (Non)Professional Subtitling
Grabación del experimento: La grabación del experimento siempre se mantendrá en su formato
original y no será divulgada por ningún otro medio (como la Internet) o compartida con ninguna
otra persona ajena al proyecto o institución.
Utilización de los datos
Los datos serán recogidos únicamente con fines científicos y sólo serán utilizados para efectos
de la investigación científica en el marco del proyecto que se acaba de describir.
Derechos del participante
- El participante tiene el derecho a retirarse del experimento en cualquier momento, tanto
durante como después de la recolección de datos.
- El participante tiene derecho a acceder a los datos del experimento en cualquier momento y a
solicitar una copia de todos los documentos (reportes, artículos, carteles, etc.) producidos con
base en los datos recolectados.
- El participante tiene derecho a presentar una queja oficial en caso de que se sienta afectado o
lesionado en cualquiera de los experimentos o por la conducta del investigador. Para ello, el
participante debe ponerse en contacto con David Orrego Carmona por correo electrónico
(david.orrego.carmona@gmail.com). En caso de que quiera presentar una queja relativa a la
conducta de David Orrego Carmona, el participante tiene derecho a dirigirse por escrito al
profesor Dr. Anthony Pym (anthony.pym@urv.cat)
_____________________________________________________________________________
Comprendo y acepto todos los parámetros expuestos en este documento, por lo que acepto
participar en el experimento de recogida de datos para el estudio sobre recepción de productos
audiovisuales.
Firmando este documento consiento mi participación en el estudio.
Tarragona, _____ de ____________ de ____
Nombre:
________________________________
Firma:
________________________________
260
Appendices
Appendix 2.
Subtitles used for the eye-tracking session
Clip 1: The Big Bang Theory, season 2, episode 4 “The Griffin Equivalency”
Transcription
Let’s see, Raj was the kung
pao chicken.
-I’m the dumplings.
-Yes, you are.
Creepy, Howard.
Creepy good or creepy bad?
DVD version
Para Raj el pollo Kun Pao...
-Who was the shrimp with
lobster sauce?
-That would be me.
- El cerdo para mí.
- Cuando tú quieras.
Me das escalofríos.
¿En el buen sentido o en el
malo?
¿Para quién son las gambas en
salsa de langosta?
Son para mí.
Come to papa, you unkosher delight.
I’m not necessarily talking
to the food.
Sit over there.
Sit over there.
-Baby wipe?
-Why do you have?
Venid con papá, delicias no
kosher.
No creas que solo me estoy
refiriendo a la comida.
Siéntate allí.
Siéntate aquí.
- ¿Toallita de bebé?
- ¿Por qué haces...?
-No, don’t ask.
-No, don’t, don’t.
I’ll tell you why.
I had to sanitize my hands
because the university...
- No, no le...
- ¡No, no, no!
Te lo diré.
Tengo que desinfectarme las
manos porque en la universidad
...replaced the paper towels
in the restrooms with hot-air
blowers.
...han sustituido el papel del
lavabo por secadores de aire.
I thought the blowers were
more sanitary.
-Why?
-Really, don’t.
Hot-air blowers are
incubators and spewers of
bacteria and pestilence.
Creía que los secadores eran
más higiénicos.
- Por favor, no...
- ¿Por qué sigues?
Los secadores son
incubadoras...
...que escupen ciertos de
bacterias y pestilencia.
La verdad, sería más higiénico
que un mono con la peste...
Frankly, it’d be more
hygienic if they had a
plague-infested gibbon...
...sneeze my hands dry.
Hey, guys, I just got the
most amazing new-Gosh, Raj do you think
you’ll ever be able to talk in
front of me without being
drunk?
...me secara la mano a
estornudos.
Hola, chicos.
Acabo de oír una cosa increíble.
¿Crees que algún día podrás
hablar delante de mí...
...sin tener que estar borracho?
Argenteam Version
Veamos, Raj tenía el pollo
Kung Pao.
- Yo tenía los bollitos.
- Sí que los tienes.
Das miedo, Howard.
¿Miedo bueno o miedo malo?
TusSeries Version
Veamos, el pollo kung pao era
para Raj.
- Los raviolis chinos son míos.
- Sí que los son.
Asqueroso, Howard.
¿Para bien o para mal?
¿Quién tenía camarones
con salsa de langosta?
Yo. Ven con papá,
deleite anti kosher.
¿Para quién eran las gambas
con salsa de langosta?
Para mí. Ven con papá,
delicia inapropiada.
No necesariamente estoy
hablándole a la comida.
Siéntate allá.
Siéntate allá.
¿Toallas para bebé?
No estoy hablando
necesariamente con la comida.
Siéntate ahí.
Siéntate ahí.
- ¿Una toallita de bebé?
- ¿Por qué tienes...?
- ¿Por qué tienes-- ¡No, no preguntes!
Te diré por qué.
Tuve que higienizarme las
manos
porque la universidad...
...cambió las toallas de papel de
los
baños por secadores de aire
caliente.
Pensé que los secadores
eran más higiénicos.
- En serio, no lo hagas.
- ¿Por qué?
Los secadores de aire caliente
son incubadoras que escupen...
...bacterias y pestilencia.
- ¡No preguntes!
- ¡No, no, no!
Te diré por qué.
Tuve que desinfectar las manos
porque la universidad
reemplazó las toallas de papel
de los servicios
por secadores de aire caliente.
Creía que los secadores
eran más higiénicos.
- De verdad, no...
- ¿Por qué...?
Los secadores de aire caliente
son incubadores y lanzadores
de bacterias y pestilencia.
Sería más higiénico si hicieran
que un simio apestado...
Francamente, sería más
higiénico
...estornude para
secarme las manos.
Hola, chicos, tengo una
noticia increíbl-Dios, Raj, ¿crees que alguna
vez podrás
hablar frente a mí sin estar
ebrio?
si tuviesen para secar las manos
el estornudo de un gibón.
Hola, chicos, acabo de
recibir la noticia más increíble...
Dios, Raj, ¿crees que alguna
vez
serás capaz de hablar delante de
mí
sin estar borracho?
Okay, well, I’ll just go eat
by myself.
-Penny, you don’t have to
do that.
-Oh, it’s okay.
Bueno, entonces me iré a comer
a mi casa.
- No tienes por qué hacer eso.
- No, tranquilo.
Between him not talking,
Entre él que no habla, él que sí
Bien, me iré a comer sola.
Penny, no hace falta.
Vale, bueno, me voy a comer
sola.
Penny, no tienes que hacerlo.
Está bien, entre él que no habla,
él que habla y él...
No, está bien, entre que
él no habla, él habla y él...
261
The Reception of (Non)Professional Subtitling
him talking, and him...
...I’m better off alone.
So goodbye, you poor,
strange little man.
-She’s so considerate.
-So, what’s your news?
habla y él...
...estaré mejor sola.
Adiós, pobre y extraño
pequeño.
- Es muy considerada.
- ¿Qué has oído?
Remember that little object
I spotted beyond the Kuiper
belt?
¿Recordáis el objeto planetario
que vi más allá del cinturón de
Kuiper?
Oh, yeah, 2008-NQ sub-17.
Or as I call it, Planet
Bollywood.
Anyway, because of my
discovery...
Ah, sí. El 2008-NQ sub-17
Yo lo llamo "Planet
Bollywood".
A causa de ese
descubrimiento...
Sí, 2008-NQ Sub-17.
O, como yo lo llamo,
"Planeta Bollywood".
En fin, por mi descubrimiento,
la revista People me nombrará...
...People Magazine is
naming me one of their 30
Under 30 to Watch.
-Raj, that’s incredible.
-Congratulations.
Excuse me, the 30 what
under 30 what to watch
what?
...la revista People me tiene
entre los 30 de menos de 30
para observar.
- ¡Raj, es estupendo!
- ¡Enhorabuena!
Disculpa. ¿Los treinta qué de
menos de treinta qué para
observar qué?
...uno de los 30 de menos de 30,
para observar.
que avisté más allá
del Cinturón de Kuiper?
Sí, 2008-NQ Sub-17.
O como yo lo llamo, Planet
Bollywood.
Bueno, por mi descubrimiento,
la revista People va a
nombrarme
uno de sus 30 por debajo de 30
a vigilar.
- Felicitaciones.
- Eso es increíble.
Disculpa...
- Felicidades.
- Es increíble.
Perdona,
30 visionaries under 30
years of age to watch
...as they challenge the
preconceptions of the fields.
Treinta visionarios de menos de
treinta años para observar...
...cómo desafían las
ideas preconcebidas de su
campo.
Ni en un millón de años
lo habría adivinado.
...¿30 qué de menos de 30 qué
para observar qué?
30 visionarios de menos
de 30 años...
...para observar como desafían
las ideas preconcebidas de sus
campos.
De un millón de posibilidades
nunca habría adivinado eso.
¿30 qué por debajo
de 30 qué a vigilar qué?
30 visionarios por debajo
de 30 años de edad a vigilar
cómo desafían las
preconcepciones de sus campos.
It’s pretty cool. They’ve got
me in with a guy doing
something about hunger in
Indonesia...
Y es muy guay, estaré con un
tío que
intenta eliminar el hambre en
Indonesia...
Es genial, comparto con un
tipo...
...que hace algo por
el hambre en Indonesia...
...and a psychotherapist
who’s using dolphins to
rehabilitate prisoners...
...and Ellen Page, star of the
charming independent film
Juno.
...y con un psicólogo que usa
delfines
para rehabilitar a los presos...
...y con Ellen Page, la estrella
de la
preciosa película independiente
Juno.
Me encantaría hacérselo.
Tú se lo harías a los delfines.
...y un psicoterapeuta que usa
delfines para rehabilitar
prisioneros...
...y Ellen Page...
...estrella de la encantadora
película independiente "Juno."
¿Mencionarás que...
¿Tendré una mención
honorífica por diseñar...
...el soporte para la cámara
del telescopio que usaste?
If I had a million guesses, I
never would have gotten
that.
I’d so do her.
You’d do the dolphins.
Do I get an honorable
mention
for designing the telescope
camera mounting bracket
you used?
Sorry, it’s not part of my
heartwarming and personal
narrative...
...in which a humble boy
from New Delhi...
...overcame poverty and
prejudice...
262
...yo diseñé el soporte en el que
va
montado el telescopio que
usaste?.
Lo siento, no es parte de
mi desgarradora historia
personal...
...en la que un niño
humilde de Nueva Delhi...
...supera la pobreza y los
prejuicios...
...estoy mejor sola.
Adiós, pobre hombrecito raro.
Es tan considerada.
¿Cuál es la noticia?
¿Recuerdan ese objeto
planetario que
descubrí cerca del Cinturón de
Kuiper?
Yo se lo haría.
Tú se lo harías a los delfines.
Lo siento pero no entra en mi
enternecedora historia
personal...
...en la cual un chico
humilde de Nueva Delhi...
...superó la pobreza
y los prejuicios...
Estoy mucho mejor sola.
Adiós, pobre y extraño
hombrecito.
Qué considerada.
¿Y qué noticias tienes?
¿Recordáis aquel
pequeño objeto planetario
Si lo hubiera intentado un
millón
de veces, nunca se me habría
ocurrido eso.
Está chulísimo.
Me metieron con un tipo que
está haciendo
algo por el hambre en
Indonesia,
y una psicoterapeuta que usa
delfines para rehabilitar
prisioneros,
y Ellen Page, protagonista de la
encantadora película
adolescente, Juno.
Me encantaría tirármela.
Tú hasta te tirarías a los
delfines.
¿Voy a tener una mención
de honor por diseñar
el soporte de montaje de la
cámara
del telescopio que usaste?
Lo siento, no es parte de mi
alentadora y personal historia
en la cual un humilde chico
de Nueva Delhi superó la
pobreza
Appendices
...and journeyed to America
to reach for the stars.
...y viaja hasta Estados Unidos
para alcanzar las estrellas.
...y viajó a Estados Unidos
para alcanzar las estrellas.
y los prejuicios y viajó a
Estados Unidos
para alcanzar las estrellas.
Poverty? Your father’s a
gynecologist. He drives a
Bentley.
¿Pobreza?. Tu padre es
ginecólogo y tiene un Bentley.
¿Pobreza?
Tu padre es ginecólogo...
¿Pobreza? Tu padre es
ginecólogo.
Conduce un Bentley.
It’s a lease.
I’m confused. Was there
some sort of peer-review
committee...
Es un leasing.
Estoy confuso. ¿No hubo una
especie de comité de sabios...
...to determine which
scientists would be
included?
Peer review? It’s People
Magazine. People picked
me.
... para
decidir qué científicos se
incluirían?.
¿Comité de sabios?. Es la
revista People. Me eligieron
ellos.
-What people?
-The people from People.
What? Yeah, but exactly
who are these people?
What are their credentials?
How are they qualified?
- ¿Ellos, quiénes?.
- Las personas de People.
Pero ¿quienes son
esas personas exactamente?.
¿Y cuáles son sus credenciales?.
¿Están cualificadas?.
What makes accidentally
noticing a hunk of rock...
¿Cómo es que ver
accidentalmente un trozo de
roca...
... que lleva cruzando el
sistema solar millones de años...
...that’s been traipsing
around the solar system for
billions of years...
...more noteworthy than any
other scientific
accomplishment made by
someone under 30?
...y tiene un Bentley.
Es alquilado.
Estoy confundido.
Es alquilado.
Estoy confuso.
¿Hubo algún tipo de
comité científico...
...para determinar qué
científicos serían incluidos?
¿Había algún tipo de
jurado de notables
para determinar qué científicos
debían ser incluidos?
¿Comité científico?
Es la revista People.
La gente me seleccionó.
- ¿Qué gente?
- La gente de People.
Sí, pero ¿quién es esa gente?
¿Jurado de notables?
Es la revista Gente. Me escogió
la gente.
¿Cuáles son sus referencias?
¿Cómo están calificados?
¿Por qué el hallazgo casual
de un pedazo de roca...
- ¿Qué gente?
- La gente de Gente.
Sí, ¿pero quién es
exactamente esta gente?
¿Cuáles son sus credenciales?
¿Cómo están cualificados?
¿Qué hace accidentalmente
destacable un trozo de roca
...que lleva millones de años
dando
vueltas por el sistema solar...
...se destaca más que cualquier
otro logro científico...
que ha estado dando vueltas
alrededor del Sistema Solar
...de alguien de menos de 30?
Boy, I’ll bet Ellen Page’s
friends aren’t giving her this
kind of crap.
...otro descubrimientno
científico
hecho por otro menor de 30
años?.
¡Vaya! Seguro que los amigos
de Ellen
Page no le irán con gilipolleces.
You proud of yourself?
In general, yes.
¿Estás orgulloso de ti mismo?.
En general, sí.
¿Estás orgulloso de ti mismo?
En general, sí.
que cualquier otro logro
científico
conseguido por alguien menor
de 30 años?
Tío, apuesto a que los amigos
de
Ellen Page no le están dando
este coñazo.
¿Estás orgulloso de ti?
Por lo general, sí.
...es más importante que
cualquier...
Seguro que los amigos
de Ellen Page no la tratan así.
durante miles de millones de
años
más digno de mención
263
The Reception of (Non)Professional Subtitling
Clip 2: The Big Bang Theory, season 2, episode 11 “The Bath Item Gift Hypothesis”
Transcription
Your argument is lacking
in all scientific merit.
It is well established
DVD Version
Tu argumento carece
de cualquier mérito científico.
Ha quedado demostrado...
...Superman cleans his
uniform by flying into
Earth’s yellow sun...
... que Superman limpia su
uniforme
volando hacia nuestro amarillo
sol...
...que incinera la materia
contaminante...
...dejando el invulnerable tejido
kryptoniano ileso y con olor a
fresco.
...which incinerates any
contaminate matter...
...and leaves the
invulnerable Kryptonian
fabric unharmed and daisy
fresh.
What if he gets something
Kryptonian on it?
-Like what?
-I don’t know. Kryptonian
mustard.
I think we can safely
assume that all Kryptonian
condiments were
destroyed...
...when the planet Krypton
exploded.
Or it turned into mustard
Kryptonite.
The only way to destroy
a rogue Kryptonian hot dog
threatening Earth.
Raj, please, let’s stay
serious here.
Superman’s body is
Kryptonian,
therefore his sweat is
Kryptonian.
Yeah, what about
Kryptonian pit stains?
Superman doesn’t sweat on
Earth.
Okay. He’s invited for
dinner
in the Bottle City of
Kandor.
He miniaturizes himself,
enters the
city where he loses his
superpowers.
Now, before dinner, his host
says, "Who’s
up for a little Kryptonian
tetherball"?
Superman says "Sure",
works up
a sweat, comes back to
Earth,
his uniform now stained
with
indestructible Kryptonian
perspiration.
Boo ya.
264
Argenteam Version
A tu argumento le falta
mérito científico.
Está establecido que Superman
limpia
su uniforme volando hacia el
Sol...
TusSeries
Tus razones carecen
totalmente de mérito científico.
Está bien establecido que
Superman limpia su uniforme
...que incinera cualquier
materia contaminante...
...y deja la tela kryptoniana
impecable y suave.
que incinera cualquier
materia contaminante
y deja la invulnerable tela
kryptoniana
indemne y oliendo a rosas.
¿Y si se mancha con algo
kryptoniano?
- ¿Como qué?
- ¿Qué sé yo? Mostaza
kryptoniana.
Podemos suponer que los
condimentos
kryptonianos se destruyeron...
¿Y qué si tiene algo
kryptoniano en ella?
¿Cómo qué?
¿Y si le cae algo kryptoniano
encima?
¿Como qué?
No lo sé. Mostaza kryptoniana.
No sé. Mostaza kryptoniana.
Creo que podemos asumir que
todos
los condimentos kryptonianos...
Creo que podemos asumir
sin temor a equivocarnos
...al explotar el planeta Kryptón.
...fueron destruidos cuando
el planeta Krypton explotó.
O se volvió kryptonita
mostaza, la única forma...
...de destruir a una salchicha
kryptoniana amenazando la
Tierra.
O se convertiría en mostaza
kryptoniana.
La única forma de destruir a un
perrito
caliente kryptoniano que nos
amenazara.
Raj, por favor,
estamos hablando en serio.
Superman es kryptoniano,
por lo que su sudor es
kryptoniano.
volando al interior
del sol amarillo de la Tierra,
que todos los condimentos
kryptonianos fueron destruidos
cuando el planeta Krypton
explotó.
O se convirtió en
kryptonita de mostaza,
la única forma de destruir
un perrito caliente kryptoniano
hostil
que amenaza la Tierra.
Raj, por favor, seamos serios.
Sí. ¿Y las manchas de los
sobacos?
Raj, por favor,
mantengámonos serios.
El cuerpo de Superman es
kryptoniano,
por lo que su sudor es
kryptoniano.
¿Qué hay de las manchas
de sudor kryptoniano?
Superman no suda en la Tierra.
Superman no suda en la Tierra.
Vale, va a cenar a la ciudad
embotellada de Kándor.
Lo invitan cenar en la
Ciudad Botella de Kandor.
Vale, lo invitan a cenar
a la Ciudad Embotellada de
Kandor.
Se hace diminuto, entra en la
botella
y pierde sus súper poderes.
Se miniaturiza, entra a la ciudad
y
pierde sus superpoderes.
Antes de cenar, el anfitrión
dice,
"¿Un partido de "kryptoból? "
Antes de cenar, su anfitrión
pregunta
si alguien quiere jugar pelota.
Superman juega, suda y vuelve
a la Tierra con el traje
manchado...
Superman acepta, suda un
poco, regresa a la Tierra...
Se miniaturiza a sí mismo, entra
en
la ciudad donde pierde sus
superpoderes.
Ahora, antes de cenar, su
anfitrión dice:
¿Quién se apunta a un poco de
tetherball?
Superman dice: "Claro",
suda, vuelve a la Tierra,
...de indestructible sudor
kryptoniano.
...con el uniforme sudado con
indestructible sudor
kryptoniano.
El cuerpo de Superman es
kryptoniano,
por tanto su sudor es
kryptoniano.
Sí, ¿qué pasa con
las manchas de sudor en las
axilas?
Superman no suda en la Tierra.
ahora su uniforme está
manchado con
transpiración kryptoniana
indestructible.
Chúpate ésa.
Appendices
Superman would have taken
his uniform to a Kandorian
dry-cleaner...
...before he left the Bottle.
Superman llevaría su traje
a una tintorería andorina...
Kandorian dry-cl... I give
up. You can’t have a
rational argument with this
man.
¿A una tintorería kandori...? No
se puede
tener una discusión racional con
él.
Hey, isn’t that the guy who
won the
MacArthur Genius Grant
last year?
No, not all at once.
- Then how?
- Leonard.
Now, Raj.
Now, Sheldon.
I didn’t get a good
look. Can I go again?
No.
-It’s David Underhill. So
what?
-So what?
His observation of
high-energy positrons has
provided the first
conclusive evidence...
Oye, ¿no es el tío que ganó el
premio
McArthur al genio el año
pasado?
¡No, no le miréis todos!
- ¿Y qué hacemos?
- Leonard.
Ahora Raj.
Y ahora Sheldon.
- No le he visto bien, ¿puedo
repetir?
- No.
- Es David Underhill. ¿Y qué?
- ¿Y qué?
...antes de salir de la ciudad.
Superman habría llevado su
uniforme
a una lavandería kandoriana...
...antes de dejar la botella.
Lavandería kando-Me rindo.
Superman habría llevado su
uniforme
a una tintorería kandoriana
antes de abandonar la Botella.
Tintorería kand...
Me rindo.
No se puede tener una discusión
racional con este hombre.
¿No es ese el tipo que ganó la
beca MacArthur el año pasado?
No se puede tener una discusión
racional con este hombre.
Oye, ¿ése no es el tío que ganó
la Beca Genio MacArthur el
año pasado?
No, no todos a la vez.
- ¿Entonces cómo?
- Leonard.
Ahora, Raj.
Ahora, Sheldon.
No lo vi bien.
¿Puedo ir de nuevo?
No.
Es David Underhill. ¿Y qué?
No, todos a la vez no.
- ¿Entonces cómo?
- Leonard.
Ahora, Raj.
Ahora, Sheldon.
No lo vi bien.
¿Puedo volver a mirar?
No.
Es David Underhill. ¿Y qué?
¿Y qué?
¿Y qué?
Su observación de los
positrones
de alta energía proporcionó la
prueba...
Su observación de los
positrones de alta
energía fue la primera
evidencia...
Su observación
de positrones de alta energía
...for the existence of
galactic dark matter.
I have two words for you.
The
first is "big," the other’s
"whoop."
It is a big whoop. It made
almost all the work you’ve
done...
...since you’ve been here
completely useless.
- Did not.
- Did, too.
Did... Okay, maybe some of
it, but...
Look, the guy was just in
the
right place, at the right
time...
with the right paradigmshifting
reinterpretation of the
universe.
He got lucky.
In more ways than one.
He’s a very handsome man.
Doesn’t do anything for me.
If I was gonna go that way,
I’m more of a Zac Efron
kind of guy.
Oh, yeah, like you have
a shot with Zac Efron.
...de que hay materia oscura
galáctica.
Solo diré dos palabras.
Una es "vaya" y la otra "
bobada".
...conclusiva de la existencia de
materia oscura galáctica.
Tengo dos palabras para ti:
"gran" y "cosa".
de la existencia
de materia oscura galáctica.
Tengo dos palabras para ti.
La primera es "qué", la otra es
"pasada".
No es una bobada. Hizo que
casi todo
tu trabajo resultara
prácticamente inútil.
Es una gran cosa.
Hizo que casi todo el
trabajo que has hecho...
...desde que estás aquí
sea complemente inútil.
- No lo hizo.
- Sí lo hizo.
No lo... Bueno, quizás
en parte, pero...
El tipo estaba en el lugar
indicado
en el momento indicado...
Sí que es una pasada.
Hizo que casi todo
el trabajo que has hecho
desde que estás aquí
sea completamente inútil.
- No.
- Sí.
No... Vale, puede que un poco,
pero...
Mirad, el tío estaba en el lugar
adecuado,
en el momento adecuado,
Tuvo suerte.
Y no solo eso.
- Es un hombre muy guapo.
- A mí no me gusta nada.
Si me decidiera por eso,
me iría más Zack Efron, la
verdad.
Como si Zack Efron fuera a
mirarte.
...con la reinterpretación del
universo
que cambiaría el paradigma
indicada.
Tuvo suerte.
En más de una forma.
Es un hombre muy apuesto.
No me hace nada.
Si fuese por ese camino, me
gusta más el estilo de Zac
Ephron.
Claro, como si tuvieses una
oportunidad con Zac Ephron.
con la reinterpretación del
universo
que cambia un paradigma
adecuada.
Tuvo suerte.
En más de un sentido.
Es un hombre muy guapo.
No es mi tipo.
Si fuera de esos,
soy más del tipo de Zac Ephron.
Excuse me. Are you
Leonard Hofstadter?
Disculpad.
¿Eres Leonard Hofstadter?
Disculpa, ¿eres Leonard
Hofstadter?
- No.
- Claro que sí.
Vale, puede que un poquito,
pero...
Estaba en el sitio adecuado
en el momento justo...
...con la correcta
reinterpretación
paradigmática del universo.
ha proporcionado
la primera prueba concluyente
Sí, como que fueses a tener
una oportunidad con Zac
Ephron.
Disculpa... ¿Eres Leonard
Hofstadter?
265
The Reception of (Non)Professional Subtitling
- Yeah.
- I’m David Underhill.
- Yeah. Heh, heh.
Yeah, Dr. Gablehauser said
if I wanted to...
- Sí
- Soy David Underhill.
Sí.
El Dr. Gablehauser me dijo que
si quería
preparar algo en el laboratorio...
...fotomultiplicador, me podrías
dar una mano.
- Sí.
- Soy David Underhill.
Sí...
Sí, el Dr. Gablehauser dijo que
si quería
- ¿Quieres trabajar conmigo?
- Si tienes tiempo, sí.
¿Quieres trabajar conmigo?
Si tienes un poco
de tiempo, claro.
¿Quieres trabajar conmigo?
Bueno, si tienes un poco de
tiempo, sí.
Wow, yeah, sure. Yeah, no
problem.
Here’s my home number.
Here’s my cell.
Here’s my office.
¡Vaya! Claro. No hay problema.
Sí, claro. Sí, no hay problema.
Sí, claro. Sí, sin problema.
Ten el teléfono de mi casa
y mi móvil, el de mi despacho...
Éste es el número de mi casa.
Éste es el de mi móvil.
Éste es el de mi despacho.
Here’s my parents’
number up in New Jersey.
They always know how to
reach me.
- So...
- Okay.
Congratulations on the
MacArthur
Grant, by the way. Big fan.
Thanks. I’ll call you.
Okay. Bye-bye!
What are you looking at?
You’ve
never seen a hypocrite
before?
...el teléfono de mis padres
en Nueva Jersey.
- Saben dónde encontrarme.
- Vale.
Este es el número de mi casa. El
de
mi teléfono móvil. El de mi
oficina.
El de mis padres en Nueva
Jersey.
Siempre saben dónde
encontrarme.
Enhorabuena por el premio
McArthur,
por cierto. Soy tu admirador.
Gracias. Te llamaré.
Vale. Hasta pronto.
¿Qué miráis? ¿Es que nunca
habéis visto a un hipócrita?
Felicitaciones por la beca
MacArthur.
Soy un gran admirador.
Gracias. Te llamaré.
Bien. ¡Adiós!
¿Qué? ¿Nunca han visto
a un hipócrita?
...set something up in
the photomultiplier lab,
...that you’d be able to give
me a hand?
-You wanna work with me?
Well, if you have a little
time, yeah.
266
- Sí.
- Soy David Underhill.
Claro.
El doctor Gablehauser dijo
que si quería trabajar en el
laboratorio...
... podrías echarme una mano.
montar algo en el laboratorio
del tubo fotomultiplicador
que me echarías una mano.
Éste es el número
de mis padres en Nueva Jersey.
Siempre saben cómo
encontrarme.
- Así que...
- Vale.
A propósito, felicidades por
la Beca MacArthur. Soy un
gran admirador.
Gracias. Ya te llamaré.
Vale. ¡Adiós!
¿Qué estáis mirando?
¿Nunca habíais visto antes un
hipócrita?
Appendices
Clip 3: The Big Bang Theory, season 2, episode 15 “The Maternal Capacitance”
Transcription
So, Dr. Hofstadter,
Leonard rarely talks about
his incredibly successful
brother and sister.
Please, don’t go there,
Howard.
I understand that, unlike
Leonard,
they’re at the top
of their respective fields.
DVD Version
Bueno, doctora Hofstadter...
...Leonard rara vez habla del
increíble éxito que tienen sus
hermanos.
- Por favor, no, Howard.
- Yo tengo entendido que...
Argenteam Version
Así que, Dra. Hofstadter,
Leonard raramente habla...
TusSeries Version
Dra. Hofstadter, Leonard habla
poco
...sobre sus increíblemente
exitosos hermanos.
Por favor, no vayas allí
Howard.
Entiendo que...
de sus increíblemente exitosos
hermanos.
Por favor, no sigas, Howard.
Entiendo que, a diferencia de
Leonard,
son punteros en sus campos.
...a diferencia de Leonard, están
en
la cumbre de sus respectivas
carreras.
Me das asco.
Bueno, su hermano pequeño,
Michael,
es catedrático de derecho en
Harvard...
...a diferencia de Leonard, están
en
lo más alto de sus respectivos
campos.
Vaya, apestas.
El hermano menor de Leonard,
Michael...
and his sister just
successfully grew a
human pancreas in an
adolescent gibbon.
...y su hermana acaba de hacer
crecer un
páncreas humano en un gibón.
...y su hermana acaba de
hacer crecer con éxito...
So, she’s close to curing
diabetes?
Why else would you grow a
pancreas in a tenaged
gibbon?
Entonces está a punto
de curar la diabetes.
¿Por qué iba a hacer crecer
un páncreas en un gibón si no?
Wow, you must be very
proud.
Why? They’re not my
accomplishments.
Vaya, estará usted muy
orgullosa.
¿Por qué? Sus éxitos no son
míos.
I have to urinate.
Tengo que ir a orinar.
Tengo que orinar.
Tengo que orinar.
Why are you doing this?
- ¿Por qué me estás haciendo
esto?
- Sabes las reglas...
- ¿Por qué estás haciendo esto?
- Ya conoces las reglas.
¿Por qué haces esto?
...si traes a tu madre al trabajo
debes sufrir.
No tenía ni idea de que tus
hermanos
tuvieran mucho más éxito que
tú.
Sí, eres como el Jar Jar Binks
de la familia Hofstadter.
Trajiste a tu madre al trabajo.
¡Debes sufrir!
No tenía idea que tus hermanos
fueran tanto más exitosos que
tú.
Sí. Eres como el Jar Jar Binks
de la familia Hofstadter.
Sí, eres como el Jar Jar Binks
de la familia Hofstadter.
Misa piensa tusa
mucho triste estás.
En lugar de burlarse...
Oh, misa cree que
tú tives mu mu triste.
En vez de burlarse de mí, mis
amigos podrían darse cuenta...
Misa piensa que tusa pareces
muy muy triste.
Mejor que burlarse, mis amigos
podrían
darse cuenta que esto es difícil
...que esto es difícil y tratar
de ayudarme a atravesarlo.
No, creo que burlarnos
de ti es más divertido.
y ayudarme a superarlo.
Boy, you suck.
Well, Leonard’s younger
brother, Michael,
is a tenured law professor at
Harvard,
You know the rules. You
brought
your mom to work. You
must suffer.
Leonard, I had no idea your
siblings
were so much more
successful than you.
Yeah, you’re like the Jar Jar
Binks of the Hofstadter
family.
Meesa think yousa lookin’
so, so sad.
You know, rather than
mock me, my friends
might realize that this is
difficult
and try to help me through
it.
Nope, I think mocking you
is more fun.
...mis amigos deberían ver lo
difícil que
es esto para mí e intentar
ayudarme.
No, burlarse es mucho más
divertido.
...es profesor titular
de leyes en Harvard...
...un páncreas humano
en un simio adolescente.
¿Entonces está cerca
de curar la diabetes?
¿Por qué otro motivo harías
crecer un
páncreas humano en un simio
adolescente?
Debe estar muy orgullosa.
¿Por qué?
No son mis logros.
Tío, das asco.
Bueno, Leondard es el pequeño,
Michael es profesor titular de
Harvard,
y su hermana acaba de crear
con éxito un
páncreas humano en un gibón
adolescente.
¿Así que está cerca
de curar la diabetes?
¿Por qué si no haría crecer un
páncreas
en un gibón adolescente?
Vaya, tiene que estar muy
orgullosa.
¿Por qué? No son mis logros.
Conoces las reglas. Trajiste a tu
madre
al trabajo. Tienes que sufrir.
Leonard, no tenía ni idea que
tus hermanos
tenían mucho más éxito que tú.
No, creo que reírse de ti es
más divertido.
267
The Reception of (Non)Professional Subtitling
Next time, don’t yousa
bring mama to work. Okeeday?
Otra vez tusa no traes mamá al
trabajo,
¿deee acueeerdo?
Próxima vez, vosa no traigasi
madre al trabajo. ¿Di acuerdo?
La próxima vez, tusa no traigas
a mamá al trabajo. ¿Vale?
That was fast.
The middle stall was
occupied.
I’ll have to try again later.
Qué rapidez.
El retrete del medio estaba
ocupado,
tendré que volver luego.
Eso fue rápido.
El compartimiento del centro
estaba ocupado.
Fuiste rápida.
El compartimento del medio
estaba
ocupado. Lo intentaré luego.
That’s totally
understandable.
In bladder voiding, as in
real estate,
it’s location, location,
location.
Es comprensible. Al vaciar la
vejiga,
como en el negocio
inmobiliario...
...Io fundamental es situación,
situación, situación.
So, where were we?
¿Por dónde íbamos?
Howard lives with his
mother
and Raj can’t speak
to women unless he’s
drunk.
Go.
That’s fascinating. Selective
mutism is quite rare.
Howard vive con su madre
y Raj no puede...
Tendré que intentarlo
de nuevo más tarde.
Es totalmente comprensible.
En el vaciado de vejiga,
así como en bienes raíces...
...todo es ubicación,
ubicación, ubicación.
- Bien, ¿en qué estábamos?
- Howard vive con su madre...
...y Raj no puede hablarle a las
mujeres
a menos que esté ebrio. Ve.
...hablar con una mujer sin
estar borracho.
Es totalmente comprensible.
En una vejiga vacía, como en
bienes
inmuebles, posición, posición,
posición.
¿Dónde estábamos?
Howard vive con su madre
y Raj no puede hablarle con
las mujeres si no está borracho.
Adelante.
Es fascinante. El mutismo
selectivo
es algo poco frecuente.
Por otra parte, el que un judío
adulto
viva con su madre es tan
común...
... que es casi un cliché
sociológico.
Es temporal y pago un alquiler.
Eso es fascinante.
El mutismo selectivo es muy
raro.
Por otro lado, un hombre judío
adulto
viviendo con su madre es tan
común...
...que bordea el cliché
sociológico.
Es temporal, pago renta.
Esto es fascinante.
El mutismo selectivo es muy
raro.
Por otro lado, un macho judío
adulto
viviendo con su madre
Vive en el mismo cuarto
donde estuvo su cuna.
Tanto el mutismo selectivo
como la
incapacidad de separarse de una
madre...
...pueden provenir de un miedo
patológico a las mujeres.
Vive en la misma habitación
donde estuvo su cuna.
Tanto el mutismo selectivo...
Vive en la misma habitación
donde tiene su moisés.
Tanto el mutismo selectivo
como la
incapacidad de separarse de la
madre
pueden provenir de un miedo
patológico a las mujeres.
It might explain why the
two of you have
created an ersatz
homosexual marriage...
...to satisfy your need for
intimacy
Eso explicaría por qué habéis
creado
una especie de matrimonio
homosexual...
...para satisfacer la
necesidad de intimidad.
Eso podría explicar por qué han
creado
una réplica de matrimonio
homosexual...
...para satisfacer
su necesidad de intimidad.
Say what?
That’s basically what I just
said.
You brought your husband
to work. You know the
rules.
Meesa thinking yousa
looking
pretty sad now too, betcha,
betcha.
Leonard, it’s 1:00.
¿Qué ha dicho?
Eso es lo que yo he preguntado.
¿Qué dijo?
Es básicamente lo que dije.
Has traído a tu esposo al
trabajo,
conoces las reglas
Misa piensa tusa mucho triste
estás ahora, ¿verdad que sí?
Trajiste a tu marido al trabajo.
Conoces las reglas.
Misa cree que ustedes se ven
muy tristes ahora, seguri,
seguri.
Eso podría explicar por qué
habéis creado
un matrimonio homosexual
artificial
para satisfacer vuestra
necesidad
de tener relaciones íntimas.
¿Qué dices?
Eso es básicamente lo que he
dicho.
Has traído a tu marido al
trabajo.
Conoces las reglas.
Misa piensa que tusa también
parece
bastante triste ahora, a que sí.
Leonard, es la una en punto.
Leonard, es la 1:00. ¿No ibas a
Leonard, es la una.
On the other hand, an adult
Jewish male living with his
mother
is so common it borders
on sociological cliché.
It’s just temporary. I pay
rent.
He lives in the same room
where his bassinet was.
You know, both selective
mutism and an
inability to separate from
one’s mother...
...can stem from a
pathological fear of women.
268
...como la incapacidad para
separarse de la madre...
...pueden provenir de un miedo
patológico a las mujeres.
es tan común que raya
el cliché sociológico.
Es temporal. Pago un alquiler.
Appendices
Weren’t you going to show
me your laboratory at 1:00?
There’s no hurry, Mother.
Tell them more about their
secret love for each other.
But it’s 1:00. You were
going to
show me your laboratory at
1:00.
Her reasoning is
unassailable. It is 1:00.
Fine. Let’s go.
I think you’ll find my
work pretty interesting.
I’m attempted to replicate
the dark matter signal
found in sodium iodide
crystals by the Italians.
So, no original research?
¿No ibas a
enseñarme tu laboratorio a la
una?
No hay prisa, madre, háblales
más
de su secreto amor mutuo.
Pero ya es la una, e ibas a
enseñarme
tu laboratorio a la una en punto.
mostrarme tu laboratorio a la
1:00?
¿No me ibas a enseñar tu
laboratorio a la una?
No hay apuro. Dime más sobre
el amor secreto de ellos dos.
No tenemos prisa, Madre.
Diles algo más sobre su amor
secreto.
Pero es la una. Ibas a enseñarme
tu laboratorio a la una.
Su razonamiento es
indiscutible,
ya es la una en punto.
Vale, vamos.
Encontrarás mi trabajo muy
interesante.
Intento replicar los destellos
de materia oscura...
...hallados con los cristales de
yoduro
de sodio por los italianos.
¿No haces estudios originales?
Su razonamiento es
irrebatible. Es la 1:00.
Su razonamiento es irrefutable.
Es la una.
Bien, vamos.
Creo que encontrarás mi
trabajo bastante interesante.
Estoy intentando replicar
la señal de materia oscura...
...que los italianos encontraron
en los cristales yodados de
sodio.
¿Así que no es una
investigación original?
Vale. Vamos.
Creo que encontrarás mi
trabajo bastante interesante.
Estoy intentando reproducir la
señal de la materia oscura
que los italianos encontraron en
cristales de yoduro de sodio.
Pero es la 1:00. Ibas a
mostrarme
tu laboratorio a la 1:00.
No.
Well, what’s the point of
my seeing it?
I could just read the
paper the Italians wrote.
No.
Y, ¿para qué voy a verlo?
¿Para qué voy a verlo entonces?
Podría leer el trabajo de esos
italianos.
Podría sólo leer el artículo que
escribieron los italianos.
Just for the record, we’re
not in
an ersatz homosexual
relationship.
Well, then why didn’t you
say that to her?
Why is it always my
responsibility?
It’s not always your
responsibility.
I swear, this is the same
thing you
did at the comic book store
last week.
I can’t believe you’re
bringing that up.
I didn’t bring it up. You did.
We’ll talk about this later.
You always say that, but we
never do.
You went to the comic
book store without me?
Para que conste, no tenemos
una especie de matrimonio
homosexual.
Para que conste, no estamos en
una réplica de relación
homosexual.
Y, ¿por qué no se lo has dicho a
ella?
- ¿Por qué todo es
responsabilidad mía?
- No todo es responsabilidad
tuya.
Esto es lo mismo que hiciste
en la tienda de cómics el otro
día.
¿Entonces por qué
no se lo dijiste a ella?
¿Por qué es siempre mi
responsabilidad?
No es siempre tu
responsabilidad.
Lo juro, es lo mismo que hiciste
en la
tienda de historietas la semana
pasada.
No puedo creer que menciones
eso.
- No fui yo. Tú lo hiciste.
- Hablaremos de eso después.
Siempre dices eso pero nunca lo
hacemos.
¿Fueron a la tienda
de historietas sin mí?
Ya estás sacando ese tema otra
vez.
- No lo he sacado yo.
- Ya hablaremos de eso luego.
Eso dices siempre, pero nunca
hablamos.
¿Fuisteis a la tienda de cómics
sin mí?
¿No es una investigación
original?
No.
¿Qué sentido tiene que lo vea?
Simplemente puedo leer el
artículo
que han escrito los italianos.
Solo para que conste, no
estamos
en una relación homosexual
artificial.
¿Entonces por qué
no se lo has dicho a ella?
¿Por qué siempre es mi
responsabilidad?
No siempre es tu
responsabilidad.
Lo juro, es lo mismo que me
hiciste
en la tienda de cómics la
semana pasada.
No me puedo creer
que saques a relucir eso.
No lo saco a relucir. Lo hiciste.
Hablaremos de esto luego.
Siempre dices lo mismo,
pero nunca lo hacemos.
¿Fuisteis a la tienda de cómics
sin mí?
269
The Reception of (Non)Professional Subtitling
Appendix 3.
Pre-experiment questionnaire
Pág.1.- ¡Bienvenido a esta encuesta!
----------------------------------------------------------------------Muchas gracias por participar en esta investigación. El estudio sobre la recepción de material audiovisual es
parte de la investigación para mi tesis doctoral y se desarrolla dentro del Intercultural Studies Group de la
Universitat Rovira i Virgili. Esta encuesta indagará sobre tus conocimientos en idiomas y tus hábitos
audiovisuales. En total, te tomará unos 10 minutos aproximadamente. Toda la información recolectada es
estrictamente confidencial y no se usará para ningún otro propósito diferente al de la presente investigación.
Si tienes alguna duda o comentario, no dudes en contactarme a la dirección de correo
davidorregocarmona@gmail.com
De nuevo, gracias por tu participación. David Orrego Carmona
Pág.2.- Datos personales
Gracias por aceptar participar en nuestro estudio. Por favor, responde las siguientes preguntas de la forma
más precisa posible.
Preg.1.- Información personal
(* Esta pregunta es obligatoria )
Nombre y apellidos : _________________________________________________________
Ciudad de residencia : _________________________________________________________
Lugar de nacimiento : _________________________________________________________
Grado :
_________________________________________________________
E-mail :
________________________________________________________
Móvil :
_________________________________________________________
Preg.2.- Sexo
(* Esta pregunta es obligatoria )
(* Marque una sola opción)
Femenino
Masculino
Preg.3.- Edad
______
Preg.4.- Nivel universitario
(* Esta pregunta es obligatoria )
(* Marque una sola opción)
Primer curso de grado
Segundo curso de grado
Tercer curso de grado
Cuarto curso de grado
Máster
Preg.5.- Idiomas hablados (enuméralos en orden del que más usas al que menos usas):
1. : _________________________________________________________
2. : _________________________________________________________
3. : _________________________________________________________
4. : _________________________________________________________
5. : _________________________________________________________
6. : _________________________________________________________
270
Appendices
Preg.6.- ¿Cuál de los idiomas que acabas de mencionar consideras tu lengua materna? En caso
de considerarte bilingüe, escribe los dos idiomas.
1. : _________________________________________________________
2. : _________________________________________________________
Pág.3.- Parte 2: Uso de material audiovisual
----------------------------------------------------------------------Gracias por aceptar participar en nuestro estudio. Por favor, responde las siguientes preguntas de la forma
más precisa posible.
Preg.7.- En promedio, ¿cuántas horas a la semana ves material audiovisual (series, películas,
documentales, comerciales, vídeos musicales, videojuegos) usando los siguientes medios?
(* Esta pregunta es obligatoria )
(* Marque una sola opción por fila)
Menos de 1
hora
Televisión
Internet (descargas y en
línea)
DVD/Blu-ray
1-3
horas
4-6
horas
7-9
horas
Más de
10 horas
Cine
Preg.8.- Indica la frecuencia con la que has usado los siguientes tipos de traducción en los
últimos seis (6) meses al ver material audiovisual producido originalmente en idiomas
extranjeros:
(* Esta pregunta es obligatoria )
(* Marque una sola opción por fila)
Nunca
Alguna vez
A veces
Muy a
menudo
Siempre
Voces superpuestas (típico de los
documentales y se escuchan la voz
original y la traducción)
Doblaje (las voces de los personajes se
doblan al idioma del país/región)
Subtitulación (texto insertado en la
parte inferior de la pantalla)
Closed caption (texto que aparece en la
parte inferior de la pantalla que presenta
los diálogos y todos los demás sonidos
del vídeo)
Preg.9.- En promedio, ¿cuántas horas al día usas la Internet (correo electrónico, consultas en
línea, redes sociales, música, videos, etc.)?
(* Esta pregunta es obligatoria )
(* Marque una sola opción)
Menos de una hora
1-2 horas
3-4 horas
5-6 horas
Más de 7 horas
271
The Reception of (Non)Professional Subtitling
Appendix 4.
Listening-comprehension test
Encuesta: Cuestionario previo al experimento
Datos personales
----------------------------------------------------------------------Gracias por aceptar participar en nuestro estudio.
Preg.1.- Información personal
(* Esta pregunta es obligatoria )
Nombre : _________________________________________________________
Ahora que ya has visto el vídeo, deberás contestar las preguntas que aparecen a continuación. Por
favor, responde las siguientes preguntas de la forma más precisa posible.
Test de comprensión
----------------------------------------------------------------------Gracias por aceptar participar en nuestro estudio. Por favor, responde las siguientes preguntas de la forma
más precisa posible.
Preg.2.- ¿Por qué no usan un ordenador para resolver el problema?
(* Esta pregunta es obligatoria )
(* Marque una sola opción)
El ordenador tardaría mucho tiempo porque hay muchas variables
No tienen todas las variables necesarias
No les ayudaría a resolver el problema
Prefieren solucionarlo ellos mismos
NS/NC
Preg.3.- ¿Por qué Sheldon rechaza la primera propuesta de Raj?
(* Esta pregunta es obligatoria )
(* Marque una sola opción)
La calidad de la imagen
La calidad de los asientos
La calidad del sonido
La disponibilidad de comidas y bebidas
NS/NC
Preg.4.- ¿Cuánto tiempo dicen que les tomará cenar?
(* Esta pregunta es obligatoria )
(* Marque una sola opción)
Media hora
Menos de una hora
Por lo menos una hora
Mucho más de una hora
NS/NC
Preg.5.- ¿Por qué no pueden cenar después de haber ido a cine?
(* Esta pregunta es obligatoria )
(* Marque una sola opción)
Sheldon llegaría tarde al trabajo el día siguiente.
Sheldon no podría cumplir su rutina en la mañana.
Sheldon tiene hambre y los sonidos de su estómago no lo dejarían concentrarse en la película.
Sheldon no podría ir al trabajo en la mañana.
NS/NC
272
Appendices
Preg.6.- Raj propone:
(* Esta pregunta es obligatoria )
(* Marque una sola opción)
Comprar las bebidas y tomarlas en casa antes de marchar.
Tomar las bebidas en el restaurante.
Comprar las bebidas que venden en el cine, aunque sean de otra marca.
Entrar las bebidas a la sala de cine sin que nadie lo note.
NS/NC
Preg.7.- ¿Qué hace Sheldon cuando los demás se van?
(* Esta pregunta es obligatoria )
(* Marque una sola opción)
Sigue analizando el problema.
Encuentra opción posible.
Les da la razón.
Dice que él tiene la razón.
NS/NC
Preg.8.- ¿Cuál es la película que quieren ver?
(* Esta pregunta es obligatoria )
(* Marque una sola opción)
The Time Machine
Red Line
The X-men
No se menciona
NS/NC
¡Muchas gracias por tu participación!
-----------------------------------------------------------------------
273
The Reception of (Non)Professional Subtitling
Appendix 5.
Experiment questionnaire
1. SEK01 ¿Cuál dirías que es tu nivel de inglés?
0
Nulo
1
Básico
2
Inicial
3
Intermedio
4
Avanzado
5
Experto
2. SEK02 ¿Con qué frecuencia te comunicas activamente en inglés (por ejemplo en
clase, en conversaciones, por teléfono, por e-mail, en el chat, etc.)?
a) diariamente
b) varias veces a la semana
c) varias veces al mes
d) varias veces al año
e) casi nunca
f) nunca
3. SEK03 ¿Con qué frecuencia usas el inglés de forma pasiva (por ejemplo al leer
libros, ver películas, leer en Internet, etc.)?
a) diariamente
b) varias veces a la semana
c) varias veces al mes
d) varias veces al año
e) casi nunca
f) nunca
3. On-screen short description of the show, taken from the DVD case.
Los físicos Leonard y Sheldon son capaces de comprender todo sobre la ineludible fuerza gravitacional de
un agujero negro en la compleja estructura del átomo. Pero es coger esos átomos y unirlos conformando
una mujer, y su comprensión y facultades chirrían hasta quedar bloqueadas. Es entonces cuando Penny,
una mujer que aúna todos esos átomos colocados en todos y cada uno de los sitios más perfectos donde
pueden estar, se muda al apartamento de enfrente y hace que, el universo cuántico de Leonard y Sheldon
comienza a expandirse hacia límites insospechados que jamás hubieran soñado imaginar.
Question codes: VAQ1.2.5=VAQ (Verbal Attention Question) 1 (clip number) 2 (question number within the category) 5 (question
number for clip) Pre-experiment questions: Self-reported English Knowledge
274
Appendices
Clip 1
Segment: TBBT 0204_1_intro
Code
GEQ1.1.1
General Questions
¿Cómo es tu compresión del vídeo que acabas
de ver?
GEQ1.2.2
GEQ1.3.3
¿Ya habías visto este episodio antes?
¿De qué trata el vídeo?
Verbal Attention Questions
¿Por qué a Sheldon no le gustan los secadores
para las manos de los lavabos?
VAQ1.1.4
VAQ1.2.5
IAQ1.2.7
¿Cómo se llama la revista que se menciona en
la discusión?
Iconic Attention Questions
¿Qué están haciendo los personajes cuando
llega Raj?
¿Qué tipo de comida están comiendo?
NAQ1.1.8
Narrative Attention Questions
¿Cuál es la noticia que tiene Raj?
IAQ1.1.6
NAQ1.2.9
GEQ1.4.10
¿Cómo reacciona Sheldon ante la noticia de
Raj?
¿Cómo calificarías el segmento que acabas de
ver?
GEQ1.5.11
¿Qué te parece la imagen del vídeo (colores,
iluminación)?
GEQ1.6.12
¿Por qué te parece agradable/desagradable?
¿Cómo te resultó seguir la traducción del
vídeo?
GEQ1.7.13
GEQ1.8.14
GEQ1.9.15
GEQ1.10.16
¿Por qué te parece que la traducción es fácil o
difícil?
¿Cómo te parece el diálogo del vídeo?
¿Por qué te parece que el diálogo es aburrido o
divertido?
Season: 2
Episode: 4
6-point Likert Scale
0=Nula
5=Muy buena
Open-ended question
Open-ended question
a) Consumen mucha energía
b) El aire que expulsan es demasiado
caliente
c) No son higiénicos
d) No secan bien
e) No lo sé
Open-ended question
Open-ended question: eating/having lunch
a) Comida árabe
b) Comida tailandesa
c) Comida italiana
d) Comida francesa
e) No lo sé
a) Acaban de regalarle una revista
b) Saldrá en una revista
c) Escribirá un artículo para una revista
d) Trabajará en una revista
e) No lo sé
Open-ended question: He questions the
selection process
Aburrido
2
3
Muy divertido
6-point Likert Scale
0=Muy desagradable
5=Muy agradable
Open-ended question
6-point Likert Scale
0=Muy difícil
5=Muy fácil
Open-ended question
6-point Likert Scale
0=Muy aburrido
5=Muy divertido
Open-ended question
275
The Reception of (Non)Professional Subtitling
Clip 2
Segment: TBBT_0211_1
GEQ2.1.1
General Questions
¿Cómo es tu compresión del vídeo que acabas de
ver?
GEQ2.2.2
GEQ2.3.3
VAQ2.1.4
VAQ2.2.5
¿Ya habías visto este episodio antes?
¿De qué trata el vídeo?
Verbal Attention Questions
¿Qué dice Sheldon sobre Superman al principio
del video?
¿Qué números de contacto le entrega Leonard al
científico que acaba de llegar?
IAQ2.1.6
Iconic Attention Questions
Según la ambientación del vídeo, ¿qué festividad
se celebra o qué época del año es?
IAQ2.2.7
¿Qué están haciendo los personajes?
NAQ2.1.8
Narrative Attention Questions
Según la discusión inicial de los personajes, se
puede decir que
NAQ2.2.9
GEQ2.4.10
Después de hablar con David (el nuevo
científico), ¿cuál es la actitud de Leonard?
¿Cómo calificarías el segmento que acabas de
ver?
GEQ2.5.11
¿Qué te parece la imagen del vídeo (colores,
iluminación)?
GEQ2.6.12
¿Por qué te parece agradable/desagradable?
¿Cómo te resultó seguir la traducción del vídeo?
GEQ2.7.13
GEQ2.8.14
GEQ2.9.15
GEQ2.10.1
6
276
¿Por qué te parece que la traducción es fácil o
difícil?
¿Cómo te parece el diálogo del vídeo?
¿Por qué te parece que el diálogo es aburrido o
divertido?
Season: 2
Episode: 11
6-point Likert Scale
0=Nula
5=Muy buena
Open-ended question
Open-ended question
a) Superman no suda en Kriptón
b) Superman no suda en la Tierra
c) Superman suda en la Tierra
d) Superman solo suda en el sol
e) No lo sé
Open-ended question: Home number,
cell phone number, office number,
parent’s number in New Jersey
Open-ended question: Christmas
a) Desayunando
b) Comiendo
c) Merendando
d) Cenando
e) No lo sé
a) Sheldon está de acuerdo con lo que
dicen los demás
b) Raj y Leonard prefieren a Batman
c) Todos se toman en serio el universo
de Superman
d) Todos son fanáticos de todos los
superhéroes
e) No lo sé
Open-ended question: He really wants to
work with him.
Aburrido
2
3
Muy divertido
6-point Likert Scale
0=Muy desagradable
5=Muy agradable
Open-ended question
6-point Likert Scale
0=Muy difícil
5=Muy fácil
Open-ended question
6-point Likert Scale
0=Muy aburrido
5=Muy divertido
Open-ended question
Appendices
Clip 3
Segment: TBBT_0215_1
GEQ3.1.1
General Questions
¿Cómo es tu compresión del vídeo que acabas de
ver?
GEQ3.2.2
GEQ3.3.3
VAQ3.1.4
VAQ3.2.5
IAQ3.1.6
IAQ3.2.7
NAQ3.1.8
NAQ3.2.9
GEQ3.4.10
¿Ya habías visto este episodio antes?
¿De qué trata el vídeo?
Verbal Attention Questions
¿Cuántos hermanos tiene Leonard según lo que
dice su madre?
¿Cómo se siente la madre de Leonard respecto a
los logros de sus hijos?
Iconic Attention Questions
Según la ambientación del vídeo, ¿en qué lugar se
encuentran los personajes?
En la escena, ¿qué hacen los personajes mientras
hablan?
Narrative Attention Questions
Durante la discusión, Howard y Leonard imitan la
forma de hablar de
¿Con qué tipo de relación asociarías la discusión
final entre Howard y Raj?
¿Cómo calificarías el segmento que acabas de
ver?
GEQ3.5.11
¿Qué te parece la imagen del vídeo (colores,
iluminación)?
GEQ3.6.12
¿Por qué te parece agradable/desagradable?
¿Cómo te resultó seguir la traducción del vídeo?
GEQ3.7.13
GEQ3.8.14
GEQ3.9.15
GEQ3.10.1
6
¿Por qué te parece que la traducción es fácil o
difícil?
¿Cómo te parece el diálogo del vídeo?
¿Por qué te parece que el diálogo es aburrido o
divertido?
Season: 2
Episode: 15
6-point Likert Scale
0=Nula
5=Muy buena
Open-ended question
Open-ended question
a) dos hermanas
b) dos hermanos varones
c) un hermano y una hermana
d) tres hermanos
e) No lo sé
Open-ended question: She does not feel
proud since these are not her
accomplishments.
Open-ended question: University
cafeteria
a) Van a empezar a comer
b) Están comiendo el postre
c) Están tomando el café
d) Están comiendo helado
e) No lo sé
a) un personaje de La Guerra de las
Galaxias
b) un personaje de los X-Men
c) un personaje de Star Trek
d) un personaje de El Señor de los
Anillos
e) No lo sé
Open-ended question: A couple’s
argument
Aburrido
2
3
Muy divertido
6-point Likert Scale
0=Muy desagradable
5=Muy agradable
Open-ended question
6-point Likert Scale
0=Muy difícil
5=Muy fácil
Open-ended question
6-point Likert Scale
0=Muy aburrido
5=Muy divertido
Open-ended question
277
The Reception of (Non)Professional Subtitling
Eye-tracking data quality
P01
P02
P03
P04
P05
P06
P07
P08
P09
P10
P11
P12
P13
P14
P15
P16
P17
P18
P19
P20
P21
P22
P23
P24
P25
P26
P27
P28
P29
P30
P31
P32
P33
P34
P35
P36
278
M.FixIma
M.FixSub
GTS
M.FixIma
Clip 3
M.FixSub
M.FixIma
Clips 2
M.FixSub
GTS
Tobii
Sample
Rate
Level
Participant
Clip 1
GTS
Appendix 6.
High 93%
Low 94%
94.0
170.11 268.12 94.1
170.44 260.07 93.4
173.82 289.25
93.2
225.23 288.17 96.3
231.08 293.26 97.4
217.36 261.81
High 76%
High 94%
90.6
121.39 309.83 87.5
155.04 361.09 41.4
195.14 365.97
98.8
187.11 532.62 97.6
268.89 523.67 96.3
198.39 484.82
Low 93%
Low 89%
90.3
141.11 177.26 91.6
151.54 185.18 91.5
157.69 188.55
94.0
232.71 276.86 94.2
178.02 216.37 92.5
214.26 253.91
Low 92%
High 95%
91.6
179.57 374.96 92.0
151.19 324.00 93.2
170.87 324.23
97.8
215.25 452.73 98.8
173.99 558.83 98.4
199.41 802.80
Low 91%
Low 88%
97.2
202.64 309.24 96.8
227.10 325.97 96.0
195.57 301.22
91.7
163.31 256.41 95.9
165.04 335.79 94.3
162.51 282.84
High 91%
High 89%
96.9
178.79 321.02 97.4
140.94 311.06 94.4
191.17 304.86
88.1
190.37 572.35 90.1
268.63 582.77 86.2
211.57 661.70
High 94%
Low 97%
97.8
201.14 374.91 98.2
211.33 358.54 98.1
187.75 341.98
98.2
201.50 301.02 99.8
211.01 268.20 99.6
205.51 277.93
High 90%
High 88%
93.9
156.75 286.30 93.8
178.15 317.28 98.1
172.82 342.73
87.8
134.54 336.55 86.2
125.07 259.32 88.8
129.25 316.63
High 87%
High 67%
80.5
162.95 387.46 91.8
186.61 405.19 89.6
179.10 348.87
79.6
215.50 333.11 82.1
145.50 321.88 75.2
169.69 366.88
High 75%
Low 76%
88.7
185.95 333.83 88.9
133.23 328.88 90.5
191.18 400.45
86.5
162.19 282.53 98.0
204.62 301.17 97.9
194.36 303.61
High 93%
Low 80%
99.7
274.75 601.58 99.1
477.33 534.62 98.9
117.00 483.91
78.0
154.94 313.06 91.0
135.80 381.03 84.8
146.80 334.70
Low 79%
Low 95%
89.4
190.58 261.51 87.7
167.91 209.41 89.1
214.50 232.66
96.6
150.00 251.57 98.0
174.01 291.24 98.1
171.44 324.72
Low 89%
Low 81%
98.8
234.90 382.81 96.4
223.75 337.52 95.5
208.20 299.39
91.2
199.87 307.26 94.8
197.94 296.30 94.9
172.82 315.12
Low 88%
Low 95%
97.5
168.46 269.31 98.9
195.13 284.05 98.1
186.04 277.69
99.3
222.11 343.71 99.2
182.25 317.45 99.1
227.80 345.85
Low 88%
High 88%
94.2
163.15 232.21 94.3
136.24 228.70 94.8
167.49 264.34
93.8
228.16 444.77 95.0
243.43 412.73 92.1
207.73 405.31
Low 88%
High 83%
91.4
186.94 275.35 95.6
227.28 303.54 90.9
217.86 302.21
91.9
210.06 494.30 92.2
209.07 488.00 95.2
184.63 504.89
High 91%
Low 94%
97.7
215.58 264.43 98.3
190.96 264.62 99.1
208.54 272.26
97.6
177.56 305.13 98.5
182.34 293.07 97.8
172.66 305.54
Low 95%
Low 93%
97.3
203.20 354.95 98.1
204.87 311.75 98.4
186.47 320.52
97.7
168.23 323.92 98.5
197.98 324.57 88.1
190.48 322.32
Appendices
P37
P38
P39
P40
P41
P42
P43
P44
P45
P46
P47
P48
P49
P50
P51
P52
High 92%
Low 90%
96.2
153.89 347.40 95.5
130.31 312.26 95.0
175.45 367.51
97.2
177.80 307.36 98.3
146.08 288.78 96.1
157.56 312.12
High 94%
High 97%
94.2
199.67 383.84 96.3
227.21 418.46 97.2
205.56 397.04
97.6
215.28 393.15 98.2
234.72 353.91 97.6
215.38 307.04
High 88%
Low 86%
90.1
182.51 307.11 93.0
178.60 282.81 90.9
182.16 315.81
93.6
212.91 353.27 92.5
181.63 262.64 90.4
196.84 305.57
High 96%
High 77%
98.9
214.46 451.10 98.5
163.52 365.62 98.1
191.41 360.36
85.5
165.60 336.03 86.2
184.60 359.78 89.3
179.44 339.00
High 88%
High 96%
93.1
184.33 373.06 94.1
136.16 337.61 94.8
176.23 366.00
97.0
207.52 572.64 98.0
196.55 606.29 98.1
184.80 528.74
High 95%
High 93%
96.2
173.87 305.89 96.5
190.45 338.16 95.5
188.30 341.98
96.6
220.61 374.88 96.4
196.82 386.12 97.0
220.28 407.13
Low 91%
Low 96%
97.0
144.81 312.26 90.4
161.96 317.86 90.7
167.01 309.42
96.3
185.12 382.62 98.1
194.52 383.32 98.3
176.99 380.57
Low 89%
Low 96%
97.4
175.61 252.82 96.7
141.00 219.16 96.8
178.99 252.14
99.1
164.09 357.99 99.1
184.07 348.06 98.8
192.49 357.28
279
© The copyright of the video used to create the screenshots from The Big Bang Theory included in the
front cover as well as throughout the thesis belongs to Warner Bros, Spain.