Automation in the Intralingual Subtitling Process

Exploring Productivity and User Experience

Abstract

The demand for intralingual subtitles for television and video content is increasing. In Finland, major broadcasting companies are required to provide intralingual subtitles for all or a portion of their programming in Finnish and Swedish, excluding certain live events. To meet this need, technology could offer solutions in the form of automatic speech recognition and subtitle generation. Although fully automatic subtitles may not be of sufficient quality to be accepted by the target audience, they can be a useful tool for the subtitler. This article presents research conducted as part of the MeMAD project, where automatically generated subtitles for Finnish were tested in professional workflows with four subtitlers. We discuss observations regarding the effect of automation on productivity based on experiments where participants subtitled short video clips from scratch, by respeaking and by post-editing automatically generated subtitles, as well as the subtitlers’ experience based on feedback collected with questionnaires and interviews.

Lay summary

This article discusses how technology can help create subtitles for television programmes and videos. Subtitles in the same language as the content help the Deaf and the hard-of-hearing to access television programmes and videos. They are also useful for example for language learning or watching videos in noisy places. Demand for subtitles is growing and many countries also have laws that demand same-language subtitles. For example, major broadcasters in Finland must offer same-language subtitles for some programmes in Finnish and Swedish. However, broadcasters usually have limited time and money for subtitling. One useful tool could be speech recognition technology, which automatically converts speech to text. Subtitles made with speech recognition alone are not good enough yet, and need to be edited. We used speech recognition to automatically produce same-language subtitles in Finnish. Four professional subtitlers edited them to create subtitles for short videos. We measured the time and the number of keystrokes they needed for this task and compared whether this made subtitling faster. We also asked how the participants felt about using automatic subtitles in their work. This study shows that speech recognition can be a useful tool for subtitlers, but the quality and usability of technology are important.