|
" L1/L2 eye movement reading of closed captioning: A multimodal analysis of multimodal use "
Elizabeth A. Specker
L. B. Waugh, Thomas
| Document Type
|
:
|
Latin Dissertation
|
| Language of Document
|
:
|
English
|
| Record Number
|
:
|
54871
|
| Doc. No
|
:
|
TL24825
|
| Call number
|
:
|
3315741
|
| Main Entry
|
:
|
Elizabeth A. Specker
|
| Title & Author
|
:
|
L1/L2 eye movement reading of closed captioning: A multimodal analysis of multimodal use\ Elizabeth A. Specker
|
| College
|
:
|
The University of Arizona
|
| Date
|
:
|
2008
|
| Degree
|
:
|
Ph.D.
|
| student score
|
:
|
2008
|
| Page No
|
:
|
300
|
| Abstract
|
:
|
Learning in a multimodal environment entails the presentation of information in a combination of more than one mode, such as in written words, illustrations, and sound. Research conducted across many disciplines (e.g. education, psycholinguistics, cognitive science) shows that the multimodal presentation of information can be beneficial to learners. This has been demonstrated in past studies that have included both school age children and adult learners (e.g. Koolstra, van der Voort & d'Ydewalle, 1999; Neumen & Koskinen, 1992). Other studies have described the benefit for both native and non-native language learners (e.g. d'Ydewalle & Gielen, 1992; Kothari et al., 2002). Production by learners of multimodal texts (Kress & van Leeuwen, 2001; Hull & Nelson, 2005) as well as the interpretation by learners of multimodal texts (Scollon, 1999; van Leeuwen, 2004) have recently been focal points of inquiry. Building upon past studies, my research interests in multimodality instead focus on the processes involved in the combination of various modalities and how this combination is used by learners of differing proficiencies in English to gain better comprehension (cf. Mayer, 1997, 2005; Graber, 1990; Slykhuis et al., 2005). As part of this process, my dissertation focuses on the addition of the written mode (closed captioning) to the already multimodal environment that exists in film and video presentations, and in so doing I have presented the framework of a Multimodal Multimedia Communicative Event in which to situate the language learner. Research questions focus on the eye movements of the participants as they read moving text both with and without the audio and video modes of information. Small case studies also give a context to four of the participants by bringing their individual backgrounds and observations regarding closed captioning and subtitles to bear on the use of multimodal texts as language learning tools in a second or foreign language learning environment. It was found that bilinguals and lower proficiency Non Native English Speakers (NNS) (native speakers of Arabic) show longer eye movement patterns in reading moving text (closed captioning) which is similar to those found with static text. Native Speakers of English (NS) tend to have quicker eye movements when reading closed captioning. When closed captioning was included with audio and video, the multimodal environment was shown to be used differently by the two groups: NNS looked longer at the closed captioning and NS were more able to quickly navigate the text presentation. While associative activation between the audio and print modalities was not found to alter the eye movement patterns of the NNS, participants were found to alternate between the modalities in search of additional information. Other research using closed captioning and subtitling as an additional modality have shown that viewing a video program with written text added turns the activity into a reading activity (Jensema, 2000; d'Ydewalle, 1987). The current study found this to be the case, but the results differed in regard to proficiency and strategy. Enabling the closed captioning while viewing multimedia has been shown to help children improve their reading skills (Linebarger, 2001) and it is hoped that this study will further knowledge of reading and also contribute to the second language acquisition and language learning body of research regarding the use of multimodal texts.
|
| Subject
|
:
|
Education; Language, literature and linguistics; Closed captioning; ESL; English as a second language; Eye movement; Learning strategies; Reading; Linguistics; Literacy; Reading instruction; 0535:Literacy; 0535:Reading instruction; 0290:Linguistics
|
| Added Entry
|
:
|
L. B. Waugh, Thomas
|
| Added Entry
|
:
|
The University of Arizona
|
| |