Skip to main content
Humanities LibreTexts

2.4: Multimodality in Subtitling for the Deaf and the Hard-of-Hearing Education in Brazil

  • Page ID
    57793
    • Charles Bazerman, Chris Dean, Jessica Early, Karen Lunsford, Suzie Null, Paul Rogers, & Amanda Stansell
    • WAC Clearinghouse

    \( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

    \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)

    \( \newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\)

    ( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\)

    \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)

    \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\)

    \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)

    \( \newcommand{\Span}{\mathrm{span}}\)

    \( \newcommand{\id}{\mathrm{id}}\)

    \( \newcommand{\Span}{\mathrm{span}}\)

    \( \newcommand{\kernel}{\mathrm{null}\,}\)

    \( \newcommand{\range}{\mathrm{range}\,}\)

    \( \newcommand{\RealPart}{\mathrm{Re}}\)

    \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)

    \( \newcommand{\Argument}{\mathrm{Arg}}\)

    \( \newcommand{\norm}[1]{\| #1 \|}\)

    \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)

    \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\AA}{\unicode[.8,0]{x212B}}\)

    \( \newcommand{\vectorA}[1]{\vec{#1}}      % arrow\)

    \( \newcommand{\vectorAt}[1]{\vec{\text{#1}}}      % arrow\)

    \( \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

    \( \newcommand{\vectorC}[1]{\textbf{#1}} \)

    \( \newcommand{\vectorD}[1]{\overrightarrow{#1}} \)

    \( \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} \)

    \( \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} \)

    \( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

    \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)

    \(\newcommand{\avec}{\mathbf a}\) \(\newcommand{\bvec}{\mathbf b}\) \(\newcommand{\cvec}{\mathbf c}\) \(\newcommand{\dvec}{\mathbf d}\) \(\newcommand{\dtil}{\widetilde{\mathbf d}}\) \(\newcommand{\evec}{\mathbf e}\) \(\newcommand{\fvec}{\mathbf f}\) \(\newcommand{\nvec}{\mathbf n}\) \(\newcommand{\pvec}{\mathbf p}\) \(\newcommand{\qvec}{\mathbf q}\) \(\newcommand{\svec}{\mathbf s}\) \(\newcommand{\tvec}{\mathbf t}\) \(\newcommand{\uvec}{\mathbf u}\) \(\newcommand{\vvec}{\mathbf v}\) \(\newcommand{\wvec}{\mathbf w}\) \(\newcommand{\xvec}{\mathbf x}\) \(\newcommand{\yvec}{\mathbf y}\) \(\newcommand{\zvec}{\mathbf z}\) \(\newcommand{\rvec}{\mathbf r}\) \(\newcommand{\mvec}{\mathbf m}\) \(\newcommand{\zerovec}{\mathbf 0}\) \(\newcommand{\onevec}{\mathbf 1}\) \(\newcommand{\real}{\mathbb R}\) \(\newcommand{\twovec}[2]{\left[\begin{array}{r}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\ctwovec}[2]{\left[\begin{array}{c}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\threevec}[3]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\cthreevec}[3]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\fourvec}[4]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\cfourvec}[4]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\fivevec}[5]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\cfivevec}[5]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\mattwo}[4]{\left[\begin{array}{rr}#1 \amp #2 \\ #3 \amp #4 \\ \end{array}\right]}\) \(\newcommand{\laspan}[1]{\text{Span}\{#1\}}\) \(\newcommand{\bcal}{\cal B}\) \(\newcommand{\ccal}{\cal C}\) \(\newcommand{\scal}{\cal S}\) \(\newcommand{\wcal}{\cal W}\) \(\newcommand{\ecal}{\cal E}\) \(\newcommand{\coords}[2]{\left\{#1\right\}_{#2}}\) \(\newcommand{\gray}[1]{\color{gray}{#1}}\) \(\newcommand{\lgray}[1]{\color{lightgray}{#1}}\) \(\newcommand{\rank}{\operatorname{rank}}\) \(\newcommand{\row}{\text{Row}}\) \(\newcommand{\col}{\text{Col}}\) \(\renewcommand{\row}{\text{Row}}\) \(\newcommand{\nul}{\text{Nul}}\) \(\newcommand{\var}{\text{Var}}\) \(\newcommand{\corr}{\text{corr}}\) \(\newcommand{\len}[1]{\left|#1\right|}\) \(\newcommand{\bbar}{\overline{\bvec}}\) \(\newcommand{\bhat}{\widehat{\bvec}}\) \(\newcommand{\bperp}{\bvec^\perp}\) \(\newcommand{\xhat}{\widehat{\xvec}}\) \(\newcommand{\vhat}{\widehat{\vvec}}\) \(\newcommand{\uhat}{\widehat{\uvec}}\) \(\newcommand{\what}{\widehat{\wvec}}\) \(\newcommand{\Sighat}{\widehat{\Sigma}}\) \(\newcommand{\lt}{<}\) \(\newcommand{\gt}{>}\) \(\newcommand{\amp}{&}\) \(\definecolor{fillinmathshade}{gray}{0.9}\)

    Vera Lúcia Santiago Araújo

    Universidade Estadual do Ceará

    Since the pioneer work of the 90s, research on multimodality has been developing so as to include analysis of web page and film texts and genres, along with printed pages and static images.1 Multimodal transcription (MT) is one of the methodologies devised for the examination of film texts and genres, and it has been applied successfully in much of the research on multimodality. Audiovisual translation (AVT), also known as Screen Translation, is the area in Translation Studies which deals with the translation meant for the mass media, with five different modes: subtitling, dubbing, voice-over, interpreting, and audiodescription. An outstanding sub-field in AVT is audiovisual accessibility, which deals with the translation of audiovisual products aimed at those with sensory disability: the blind (audiodescription), and the deaf (subtitling for the deaf and the hard-of-hearing—SDH).

    Despite the obvious interface between multimodality and AVT, little has been done in terms of methodology to join the two approaches. So far we are aware of only one investigation that uses MT of films in order to find a better way through novice subtitlers’ training. Taylor (2003) proposed a model which joins MT and subtitling studies. MT involves “the breaking down of a film into single frames/shots/ phases,” and the analysis “of all the semiotic modalities operating in each frame/shot/phase.” (Taylor, 2003: 191)

    The present work-in-progress aims at replicating Taylor’s study with novice subtitlers for the deaf and the hard-of-hearing in Brazil. Research on SDH has been done at the State University of Ceara (Brazil) since 2000. Three studies were carried out in order to find parameters that meet the needs of the Brazilian deaf audience. Now that these parameters are being tested in Brazil’s five regions, we felt it was time to train future subtitlers. This action is justified because the SDH produced in the country does not follow the ordinary procedures used for the hearing audience. The first procedure is the use of a subtitle rate of 145, 160 or 180 words per minute (wpm). According to D’Ydewalle et al. (1987), these rates are the ones that allow viewers to have a good reception, because they harmonize subtitle, speech and image. The second is the number of lines. Readability is guaranteed if the subtitle has two lines at maximum. The third is the subdivision of the speech into one or more subtitles (segmentation or line breaks). In order to achieve this, there are some criteria to be followed for the subtitle to be read comfortably. Finally, condensation of the speech is sometimes needed, because of the subtitle rates mentioned previously. The three studies suggest that these parameters plus the provision of additional information, such as identification of speaker and sound effects, are also to be used in SDH.

    This chapter reports on an ongoing research project about the use of multimodal transcription in translators’ training on SDH. The idea came out, after Fronteira (Frontier), a feature film, was subtitled by students from the Federal University of Minas Gerais and the State University of Ceará. At the time, it was hard to explain to the team members in which situations we should identify the speaker and how we should translate a sound into words. This explanation could have been easier to do if we had used MT. Our hypothesis is that MT can be a tool to teach future expert translators to handle text analysis and multimedia technology for successful SDH.

    This chapter describes this training research in two steps. The first focuses on describing characteristics of Subtitling for the Deaf and the Hard-of-Hearing (SDH), following the parameters proposed by research on the topic carried out at the State University (cf. Araújo, 2004a, 2005, 2007; Franco & Araújo, 2003), and the patterns aimed at a hearing audience (cf. Araújo, 2004b; Diaz Cintas & Remael, 2007; D’Ydewalle et al., 1987; Ivarsson & Carol 1998; and Perego, 2003, 2008, 2009). The second step provides a multimodal transcription of an excerpt of Frontier in order to analyse its SDH as an element of a multimodal text. The final purpose is to devise a model to be used in SDH education that combines multimodality and AVT.

    Besides the introduction, this chapter is divided into three parts. It starts with a brief account of our theoretical framework, which draws on multimodal transcription and AVT. Then the methodology used is outlined, and the analysis of the subtitling of an excerpt of the Frontier is presented and discussed. Finally, some conclusions are drawn and avenues for further research are pointed out.

    Theoretical Framework: SDH in Brazil

    Subtitling aimed at hearing viewers and subtitling produced for deaf and hard-of-hearing audiences seem to be viewed differently by Brazilian audiovisual producers and by the government because the production of both types of subtitles is different. The first type is regarded as a translation activity and follows the norms described by AVT researchers (cf. Diaz Cintas & Remael 2007; D’Ydewalle et al., 1987; Ivarsson & Carol 1998; and Perego, 2008). The second is seen as a transcription of speech and is not conceived as translation. The law itself corroborates this view, as Bill 310, which regulates the use of SDH, audiodescription and Sign Language Interpreting on TV, by defining SDH as the “transcription in Portuguese of dialogues, sound effects and other information which could not be perceived or understood by the hearing impaired.”2 Intralingual subtitling is included in translation studies because the field recognizes three types of translation: intralingual (within the same language), interlingual (between two different languages) and intersemiotic (between two different semiotic modes, for example, from the visual to the verbal and vice versa)

    Subtitles for hearing viewers are made with the aid of software that allows spotting3 , translation, revision, and preview. The parameters followed by Brazilian subtitling companies are similar when creating this kind of subtitle in many aspects to those used in Europe. According to these parameters, subtitles normally have no more than two lines containing approximately 145 or 150 words per minute, with a screening duration of four seconds to 64 characters. Although Brazilians do not follow the European six-second rule (cf. Diaz Cintas & Remael 2007; D’Ydewalle et al., 1987; Ivarsson & Carrol 1998), the maximum number of spaces available for four seconds, the Brazilian standard, is very close to the spaces provided by Diaz Cintas and Remael (2007, p. 97). Table 1 summarizes the number of characters per line for the subtitle rate of 145wpm, as higher reading speeds (160 and 180 wpm) are not common here.

    Brazilian subtitles for hearing audiences also tend to be condensed to make subtitles readable in the time available. This shortening allows synchrony across subtitle, speech, and image, which is essential in order to facilitate the viewer’s reception. A viewer must have enough time to read the subtitles, see the images, hear the source audio, and enjoy the programme comfortably. Condensing the content can be achieved through the elimination of redundant and nonrelevant ideas and by the omission of some source text words. These deletions are important in order to achieve subtitle-speech-image synchronism. Words frequently omitted are: (i) repeated words; (ii) conversational markers such, as “you know,” “I mean,” “right,” etc.; (iii) interjections; (iv) tag questions; (v) clichés or routine formulae; (vi) cognate words; (vii) words related to people or things visible on the screen (Diaz Cintas, 2003, pp. 209-211).

    Spotting or line breaking refers to the breaking down of dialogues into one or more units. Gottlieb (1994, pp. 109-110), quoting Helene Reid (1990), points out three criteria to be adopted when spotting: the visual (whenever a cut or a camera movement is present, a different subtitle is advisable); the rhetorical, (subtitles should follow speech rhythm: when speakers pause to breathe, subtitles should end); and the grammatical, (in the absence of cuts and breathing pauses, the grammatical parameter is adopted). Every subtitle must be a coherent whole, that is, semantic units should remain in the same subtitle.

    Subtitle legibility is very important and for this reason, formatting is a key issue in the production and analysis of a subtitle. Apart from the number of lines, subtitle rate, condensing of information, spotting and subtitle duration (four or six seconds), there are a number of other elements relevant to subtitle readability: location, font type, and position. Subtitles are normally located at the bottom of the screen. Fonts with varied sizes and without serif are preferred, because they solve legibility problems. As to the position, subtitles are normally centre or left-aligned, but the centre-aligned form is regarded by most analysts as the best choice (Diaz Cintas & Remael, 2007, p. 84).

    The same procedure is not adopted to create SDH. The professionals involved are not subtitlers, but stenocaptioners who operate a special keyboard— stenotype—linked to a computerised machine called a stenograph. The stenotype allows for very fast typing speeds and it is normally used to transcribe congressional and business meetings as well as courtroom sessions. Now it is also being used to subtitle pre-recorded and live TV programmes. Table 2 shows one example of these differences of subtitling (Araújo, 2009:166):

    The subtitle exhibited on TV does not have a complete thought, as some word groups are separated (“the figures” and “of,” for example), the number of characters exceed the 145-wpm-subtitle rate (see Table 1). Because of the above mentioned parameters, the four subtitles were transformed into three in order to meet the number-of-character, condensation, and segmentation criteria.

    Table \(\PageIndex{1}\): Maximum spaces for a 145-word-per-minute reading speed

    Apart from these parameters, SDH will also be approached by means of an ongoing research project which aims at designing a model of SDH for Brazilian deaf and hard-of-hearing audiences. This model was developed by the State University research team with twelve deaf people from the Ceara Institute of Education for the Deaf. This project has its basis in the standards outlined in Table 1, and three studies were carried out at the university. Although more conclusive results are required in order to consider the model as capable of meeting the needs of Brazilian deaf and hard-of-hearing viewers, it may indicate the elements considered relevant to produce an efficient SDH service across Brazil. This preliminary model is being tested in Brazil’s five regions (North, South, Southeast, Northeast, and Centre East). From each region, five deaf people from two states will test the model by watching four subtitled short-feature films. This study will not be described here, but some insights of the data collected so far will be referred to for the sake of clarification of some aspects of the third study.

    The first study we carried out analysed SDH provided by the Globo TV network, the most popular in the country. At the time (2002), it was the only network offering SDH in Brazil. The participants (15 deaf-born students from the Ceara Institute of Education for the Deaf, based in Fortaleza, and a control group of 13 hearing students) were exposed to different genres of TV programming. Four hypotheses were formulated: (1) Whenever speech comprehension did not depend on images, lack of speech-subtitle-image synchronism would not impair the reception of the subtitles and the understanding of their content; (2) Whenever speech comprehension depended on images, lack of speech-subtitle-image synchronism would impair reception; (3) Whenever there was speech-subtitle-image synchronism, reception would be facilitated; (4) When orality and acoustic markers considered to be (in)dispensable by the deaf participants were present (or lacking, as the case may be), reception would be compromised.

    Table \(\PageIndex{2}\): Closed caption versus SDH parameters

    Hypotheses 1, 2 and 4 were confirmed, suggesting that speech-image synchronized subtitles facilitate reception on the part of the viewer (Franco and Araújo, 2003). However, hypothesis 3 was found unsustainable and this signals that perhaps speech-subtitle-image synchronism and condensing of content are not sufficient to facilitate efficient reception. The films shown had synchronisation of speech and image as well as condensed subtitles, but these proved challenging for participants. One possible explanation is that the subtitles used were less condensed than the subtitles directed at hearing viewers—as the editing did not follow parameters outlined earlier and was carried out only for imagesubtitle synchronisation (Araújo, 2004a). Because deaf participants reported that they did not understand film content, we assumed that maybe the subtitles should be further condensed to reduce the time required to read them (Franco & Araújo, 2003). We are aware that further research is needed to test this assumption. Bearing in mind these results, we decided to re-subtitle the same programs for our second study in which we used only pop-on subtitles that adopted the most common parameter employed by subtitling companies in Brazil. As the 160-wpm-parameter was suitable for hearing viewers, we thought that it would also be efficient for deaf viewers.

    The second study tested the condensed pop-on subtitles with the same group of deaf and hearing participants. After they watched the same programmes on a TV set, the participants answered written questionnaires, composed of open and closed questions,4 a more efficient tool than the multiple choice questionnaires we had previously used. The new questionnaire consisted of “concept, “detail,” and “picture” questions that related to the understanding of the content and the integration of images and subtitles. This time the participants’ performance was better, but the results were still inconclusive (Araújo, 2004a). Sometimes the participants succeeded in understanding the main subject, but could not comment on the image and/or the clip’s secondary ideas. As far as editing is concerned, the amount of condensing of information that occurred was regarded as uncomfortable (Araújo, 2004a). For this reason, condensing and editing were the focus of the third study.

    The third study tried to find out what level of condensing would satisfy Brazilian deaf people’s needs (Araújo, 2007). Moreover, we investigated what the ideal format would be, taking into account technical considerations as well as style, conventions and punctuation used. A group of twelve deaf students, comprising university (nine) and high school students (three), had monthly meetings with the research team during eight months. They watched different clips of the same programme with two different sets of SDH: the pre-existing content and the research team’s proposal. The latter was based on the standards outlined in Table 1 which are directed at hearing viewers plus bracketing to provide additional information. After each viewing, participants were asked to talk about the content of the programme and to access the SDH. These recall protocols occurred in Brazilian Sign Language (LIBRAS) with the interpretation in Brazilian Portuguese filmed for further analysis.

    The clips were subtitled with different reading speeds: 145 (see Table 1), 160, and 180wpm. Although there was not much difference between the three speeds, the deaf participants preferred the 145 word-per-minute rate, as this proved to be more comfortable to watch. This suggests that for Brazilian deaf people, a greater degree of editing is needed to facilitate the enjoyment of a TV programme.

    The suggestions regarding format were for the use of brackets when signalling identification of speakers and sound effects. They rejected the European color system. Figure 1 displays the two parameters.

    There is another interesting aspect related to speaker identification we would like to point out. Deaf participants said it was very difficult to distinguish who speaks in a scene. At first, they suggested the speakers should be named in every subtitle. When argued that a great deal of space would be lost on screen, they agreed that speaker identification should be present every time a different person speaks. These changes continue to be discussed in the current study, as some deaf people from other parts of Brazil are not identifying the characters properly, because it was our choice not to name the speaker when there was no other visible character on the screen. A good example can be seen in Figure 1. The character’s name Romanza was not recognized by most deaf participants in the current research. She was always referred to as ‘the little girl” and “the granddaughter.”

    We are beginning to realize that we should pay more attention to this issue. Therefore, we think we should try to address this subject in our subtitlers’ training. The description of all multimodal elements by means of MT may help them decide in which situation this identification is needed.

    MT may also be useful when we translate the sound track of a film. The current research has shown that translating all the sounds may not be an efficient strategy. Nearly all participants who have been exposed to the subtitles so far only seem to recognize the sound if it is linked to the film plot. For example, the deaf were able to recognize the sound: [Sad song]. This sound effect announced a dead character. The effect could be described as [sound of a bell]. This proved to be a good strategy, as the deaf did not recognize other songs in which we tried to characterize the sound like [Instrumental music] and [Drums], for example.

    Multimodal Transcription

    Baldry and Thibault (2000), cited in Taylor (2003), are the authors of multimodal transcription (MT), a tool to describe and analyze film texts. Baldry and Thibault’s MT is a grid with six columns that breaks down and thoroughly describes an audiovisual film text in terms of each semiotic mode it is made of. The grid contains six columns: (1) TIME in seconds; (2) visual frame—the static image; (3) visual image—description of scenario and participants displayed according to the camera position using the components described in Figure 2; (4) KINESIC ACTION of the participants; (5) SOUNDTRACK— dialogues, ambient sounds, music; and (6) METAFUNCTIONAL INTERPRETATION, where the movie is divided in phases and subphases defined by the identification of a pattern of semiotic modes in action, that will temporally make the transition into a distinct pattern of semiotic modes.

    Figure \(\PageIndex{1}\): Identification of speaker: Brazil and Europe
    Figure \(\PageIndex{2}\): Components of the multimodal transcription table

    Taylor (2003) adapted Bladry and Thibaut’s grid in his subtitling studies, claiming that it may be used as a tool for novice subtitlers to learn how to use the image to decide which information will be prioritized when adding or deleting, and which strategies will be needed to condense or edit a subtitle. The author claims that:

    (MT) As a tool for the professional it is, as so far developed, time-consuming and not commercially viable on a cost-benefit basis, but this article attempts to show that as an instrument for sensitizing translation students to the particular demands of multimodal translation, it takes us a step further along the road to optimizing subtitling strategies. (Taylor, 2003, p. 191)

    Taylor made three considerable changes in Bauldry and Tibaut’s model, because he considered the original table over detailed and refined it for the design of subtitles (Taylor, 2004). The author thus fused the third column with the fourth, deleted the last column disregarding the breakdown in phases and subphases, and then inserted a column with subtitles, as illustrated by Figure 3.

    Figure \(\PageIndex{3}\): Taylor’s multimodal transcription of an episode of a BBC comedy series. Source: TAYLOR, 2003, p. 162
    Figure \(\PageIndex{4}\): Multimodal transcription used here.

    In accordance with Taylor’s (2004) opinion, two other important alterations had to be made. The first change is that column (1) —TIME in seconds—was replaced with Time Code Reading (TCR), which is a method of accounting for video footage and frames that reads HOURS: MINUTES: SECONDS: FRAMES (00:00:12:20). This was done because the duration of the scene, to which the titles must be inserted, is of crucial importance, for it determines the number of characters per second that should be on screen, to ensure readability. Each TCR presented on this chapter’s MT table, was retrieved from the subtitling software Subtitle Workshop©, which will be explained in the next part of this chapter.

    The other change is that the subtitle column became the SDH column where this chapter’s final subtitling suggestion will be inserted. Figure 4 shows the complete MT structure.

    The Training Course

    As part of the two universities’ cooperation project, subtitlers engaged in research and in audiovisual accessibility are taught subtitling procedures by means of a training course. As it was said previously, it was very difficult for the students to decide when a speaker should be identified and how to translate a sound into words. To facilitate student’s choices, we thought the description of all semiotic elements involved in a scene provided by MT would be helpful. So, we are beginning to introduce this tool in the course.

    The course outline is the following: a) theoretical issues related to subtitling; b) presentation of the subtitling software Subtitle Workshop©; c) subtitling practice. The freeware Subtitle Workshop (SW), developed by URUsoft—http:// www.urusoft.net, enables us to work out all phases of the subtitling process: 1) SPOTTING OR CUEING; 2) TRANSLATION; 3 REVIEW (see Figure 5).

    Figure \(\PageIndex{5}\): Subtitle Workshop

    In the menu MOVIE we can load the film and in the menu FILE we can load preexisting subtitles or create new ones. The subtitled film can be viewed while the subtitles are being prepared, and we can choose the subtitles’ format (color, font, size, etc.) in the menu EDIT. Besides, in and out times and duration can be visualized on the left side of SW. The subtitle text is written at the bottom of the screen.

    In the following pages, we provide an example of how MT will be used in the training of novice subtitlers, and it will discuss the subtitling of an excerpt of Frontier. This movie was chosen to be our focus because it is an author film with limited dialogue and many sounds that helped to create the mysterious atmosphere.

    Subtitling of Frontier

    Frontier (2008) was directed by Rafael Conde, who works for the Federal University. The DVD version, subtitled and audiodescribed by members of the project was out last year. It tells a love story, full of faith and mystery. It was shot in an old house where a young lady (Maria Santa), considered saint by the people from Minas Gerais, lives. The arrival of the Traveller, Maria Santa’s lover, and Aunt Emiliana (an old lady), who is preparing the great miracle, will change Maria Santa’s life forever

    Figure \(\PageIndex{6}\):Front cover the DVD

    As it has been said, the movie does not have a lot of dialogue, but a sound track that is necessary to create the mysterious atmosphere. Thus, identifying the speaker and the sound effects are essential to follow the plot. For this reason, we had students to pay close attention to what the images reveal so that they knew when to translate the two features.

    On the scene focused here, Maria Santa and the Traveller are talking about a secret they share. The subtitling of this two-second scene requires that 29 spaces are used to achieve the rate of 145wpm (see Table 1). Although there was enough space for the translation of the sound (crying), the analysis of the shot by means of MT shows that Maria Santa was crying before she said “I am sorry for us.” The first frame in Figure 7 displays clearly that the character is crying. So the decision was not to subtitle this information because the image tells that to the audience. The previous shot, which lasted more than three seconds shows that Maria Santa was very sad and that her crying expressed her sorrow. In this case, it was an easy decision because there was no need to condense the text. However, in the other examples, MT proved to be an efficient aid to novice subtitlers.

    Figure \(\PageIndex{7}\): Multimodal transcription 1

    The shot brings Maria Santa, the Traveler, and Aunt Emiliana. When the old lady joins the couple, it is necessary to identify her, because from a median distance, it is hard to realize which person is talking (see Figure 8). That is why the subtitle with the name EMILIANA was produced. The scene lasts two seconds and four frames and the translation should have 32 characters at the maximum (see Table 1). So, apart from EMILIANA in brackets (10 characters), the sentence Estive rezando minhas ladainhas (32 characters) also has to be included in the subtitle. Therefore, a shortened version was written, Rezei minhas ladainhas (23 characters). At the time, it took students a long time to subtitle the film, as it had many situations like that. We presume that if MT had been used, subtitling would have been easier.

    The same kind of difficulty did not happen in the next subtitle pointed out in Figure 8, because, as one can see in the MT, the situation remains the same and the viewer can see that the old lady continues her speech. So it was not necessary to name her, and her speech, E agora vou descansar um pouco (1:18, 25 characters), could be fully subtitled and did not have to be edited to meet the 145wpm parameter (see Table 1).

    The last subtitles to be discussed here are related to two situations involving Aunt Emiliana’s anger, an anger perceived because she shouts while she is speaking. In both subtitles, it was not necessary to translate the information by adding [shouting] to the subtitle, as the images showed clearly the old lady’s emotional reactions (see Figure 9).

    This strategy was helpful in the second subtitle because it had to be edited. Without the inclusion of the sound effect, only the message’s addressee (você) had to be removed. If [shouting] had been included, the subtitle would have to be more condensed, as only 40 characters were allowed in order to meet the convenient subtitle rate (see Table 1).

    Figure \(\PageIndex{8}\): Multimodal transcription 2
    Figure \(\PageIndex{9}\): Multimodal transcription 3

    Final Remarks

    Although we have used MT to produce and to analyse SDH and audiodescription, it has not been tested yet. It is our aim to carry out a research study in which two groups of trainees will be formed. MT will only be used in the experimental group in order to find out whether MT really makes a difference.

    Another aspect that came out in our observations related to the use of MT in subtitling is that transcription focusing only on salient items (VS) will be especially helpful in the condensation of long speech. It is our presupposition that it can be used in professional subtitling.

    Notes

    1. The author wishes to thank the Brazilian Government Agency FUNCAP for financial support.
    2. Bill 310 (www.mc.gov.br/o-ministerio/le...rtaria-310.pdf) regulates audiovisual accessibility on television, complementing bill 5296 from December 2, 2004 (www.planalto.gov.br/ccivil_03...reto/D5296.htm) which rules over the rights of people with any kind of physical, auditorial, mental, or visual impairment.
    3. Spotting is the action to define at which moment of the film a subtitle starts and ends.
    4. The communication with the team was mediated by a sign language interpreter.

    References

    Araújo, V. L. S. (2004a). Closed subtitling in Brazil. In P. Orero (Ed.), Topics in audiovisual translation (pp. 199-212). Amsterdam/Philadelphia: John Benjamins.

    Araújo, V. L. S. (2004b) To be or not to be natural: Clichés of emotion in screen translation. Meta, 49(1), 161-171.

    Araújo, V. L. S. (2005). A legendagem para surdos no Brasil. In Questões de Lingüística Aplicada: Miscelânea. Fortaleza, Brazil:EdUECE,.

    Araújo, V. L. S. (2007). Subtitling for the deaf and hard-of-hearing in Brazil. In J. D. Cintas, P. Orero, A. Remael (Eds.), Media for All: Subtitling for the Deaf, Audio Description, and Sign Language (pp. 99-107). Amsterdam; New York: Rodopi.

    Araújo, V. L. S. (2009). In search of SDH parameters for Brazilian party political broadcasts The Sign Language Translator and Interpreter, 3(2). 157-167

    Baldry, A. (2000). Multimodality and Multimediality in the distance learning age. J. Diaz Cintas (Ed.). Campobasso: Palladino.

    D’ydewalle, G. et al. (1987). Reading a message when the same message is available auditorily in another language: The case of subtitling”. In: K. K. O’Regan, & A. Lévy-Schoen (Eds.), Eye movements: From physiology to Cognition (pp. 313-321). Amsterdam and New York: Elsevier.

    Franco, E., & Araújo, V. L. S. (2003). Reading Television: Checking deaf people›s Reactions to Closed Subtitling in Fortaleza, Brazil. The Translator, 9(2), 249-267.

    Ivarsson, J., & Carrol, M. (1998). Subtitling. Simrishamn, Sweden: GraphoTryck AB.

    Perego, E. (2003). Evidence of explicitation in subtitling: Towards a characterization. Across Languages and Cultures, 4(1), 63-88.

    Perego, E. (2008). What would we read best? Hypotheses and suggestions for the location of line breaks in film subtitles. The Sign Language Translator and Interpreter, 2(1), 35-63.

    Perego, E. (2009). The codification of non-verbal information in subtitled texts. In J. D. Cintas (Ed.), New trends in audiovisual translation (pp. 58-69). Bristol,UK; Multilingual Matters.

    Remael, A. (2007). Audiovisual translation: Subtitling. Manchester: St. Jerome.

    Taylor, C. J. (2003). Multimodal transcription in the analysis, translation and subtitling of Italian films. The Translator, 9(2), 249-267.

    Taylor, C. J. (2004). Multimodal Text Analysis and Subtitling. In E. Ventola, C. Charles, & M. Kaltenbacher (Eds.), Perspectives on Multimodality (pp. 153-172). Amsterdam; New York: John Benjamins.

    Taylor, C. J. (2003). Multimodal Transcription in the Analysis, Translation and Subtitling of Italian Films. The Translator, 9(2), 249-267.