short-paper

Subtitles in 360-degree Video

Authors:

Jake Patterson,

Anastasia Schmitz,

Mike Armstrong,

Maxine GlancyAuthors Info & Claims

TVX '17 Adjunct: Adjunct Publication of the 2017 ACM International Conference on Interactive Experiences for TV and Online Video

Pages 3 - 8

https://doi.org/10.1145/3084289.3089915

Published: 14 June 2017 Publication History

Abstract

Currently there exists no agreed-upon user experience guidelines regarding subtitling (closed captions) in immersive 360-degree video experiences. It is not clear how subtitles might be acceptably displayed within this context, namely to support the balance between comprehension, freedom to look around the scene, and immersion. This work-in-progress describes four subtitle behaviours that we have designed and implemented in order to perform user-testing. We describe our rationale for each behaviour and discuss our initial hypotheses surrounding a full empirical investigation.

Supplementary Material

suppl.mov (tvxwp0101-file3.mov)

Supplemental video

Download
56.47 MB

References

[1]

Klin A, Jones W, Schultz R, Volkmar F, and Cohen D. 2002. Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. Archives of General Psychiatry 59, 9 (2002), 809--816.

[2]

Andy Brown, Rhia Jones, Mike Crabb, James Sandford, Matthew Brooks, Mike Armstrong, and Caroline Jay. 2015. Dynamic Subtitles: The User Experience. In Proc. TVX '15.

Digital Library

[3]

Julie N. Buchan, Martin Parï£¡, and Kevin G. Munhall. 2007. Spatial statistics of gaze fixations during dynamic face processing. Social Neuroscience 2, 1 (2007), 1--13. 18633803.

[4]

V. Charissis and M. Naef. 2007. Evaluation of Prototype Automotive Head-Up Display Interface: Testing Driver's Focusing Ability through a VR Simulation. In 2007 IEEE Intelligent Vehicles Symposium. 560--565.

[5]

British Broadcasting Corporation. 2016. Subtitle Guidelines v1.1.4. (2016). http://bbc.github.io/subtitle-guidelines

[6]

Antoine Coutrot and Nathalie Guyader. 2014. How saliency, faces, and sound influence gaze in dynamic social scenes. Journal of Vision 14, 8 (2014), 5.

[7]

Michael Crabb, Rhianne Jones, and Mike Armstrong. 2015. The Development of a Framework for Understanding the UX of Subtitles. In Proc. ASSETS '15. ACM.

Digital Library

[8]

A Foerster. 2010. Towards a creative approach in subtitling: a case study. In New Insights into Audiovisual Translation and Media Accessibility, J.D. Cintas, A. Matamala, and J. Neves (Eds.). Rodopi, 81--98.

[9]

Tom Foulsham and Lucy Anne Sanderson. 2013. Look who's talking? Sound changes gaze behaviour in a dynamic social scene. Visual Cognition 21, 7 (2013), 922--944.

[10]

Ian Hamilton. 2015. How to do subtitles well -- basics and good practices. (2015). http://www.gamasutra.com/ blogs/IanHamilton/20150715/248571/How_to_do_subtitles_ well__basics_and_good_practices.php.

[11]

Richang Hong, Meng Wang, Xiao-Tong Yuan, Mengdi Xu, Jianguo Jiang, Shuicheng Yan, and Tat-Seng Chua. 2011. Video Accessibility Enhancement for Hearing-impaired Users. ACM Trans. Multimedia Comput. Commun. Appl. 7S, 1, Article 24 (Nov. 2011), 19 pages.

Digital Library

[12]

Yongtao Hu, Jan Kautz, Yizhou Yu, and Wenping Wang. 2015. Speaker-Following Video Subtitles. ACM Trans. Multimedia Comput. Commun. Appl. 11, 2, Article 32 (Jan. 2015), 17 pages.

Digital Library

[13]

Charissa R. Lansing and George W. McConkie. 2003. Word identification and eye fixation locations in visual and visual-plus-auditory presentations of spoken sentences. Perception & Psychophysics 65, 4 (2003), 536--552.

[14]

Joseph J. LaViola, Jr. 2000. A Discussion of Cybersickness in Virtual Environments. SIGCHI Bull. 32, 1 (Jan. 2000), 47--56.

Digital Library

[15]

Ofcom. 2006. Television access services. Review of the Code and guidance. (2006). https://www.ofcom.org.uk/__data/assets/pdf_file/0016/42442/access.pdf, last accessed March 2017.

[16]

Jason Orlosky, Kiyoshi Kiyokawa, and Haruo Takemura. 2014. Managing Mobile Text in Head Mounted Displays: Studies on Visual Preference and Text Placement. SIGMOBILE Mob. Comput. Commun. Rev. 18, 2 (June 2014), 20--31.

Digital Library

[17]

Peter J. Passmore, Maxine Glancy, Adam Philpot, Amelia Roscoe, Andrew Wood, and Bob Fields. 2016. Effects of Viewing Condition on User Experience of Panoramic Video. In Proc. ICAT-EGVE '16.

Digital Library

[18]

Adam Philpot, Maxine Glancy, Peter J Passmore, Andrew Woods, and Bob Fields. User Experience of Panoramic Video in CAVE-like and Head Mounted Display Viewing Conditions. In Accepted for TVX '17.

Digital Library

[19]

Raisa Rashid, Jonathan Aitken, and DeborahI. Fels. 2006. Expressing Emotions Using Animated Text Captions. In Computers Helping People with Special Needs, Klaus Miesenberger, Joachim Klaus, WolfgangL. Zagler, and ArthurI. Karshmer (Eds.). Lecture Notes in Computer Science, Vol. 4061. Springer Berlin Heidelberg, 24--31.

Digital Library

[20]

Gary E. Riccio and Thomas A. Stoffregen. 1991. An ecological Theory of Motion Sickness and Postural Instability. Ecological Psychology 3, 3 (1991), 195--240.

[21]

Alina Secarâ. 2013. R U ready 4 new subtitles? Investigating the potential of social translation practices and creative spellings. Linguistica Antverpiensia, New Series' Themes in Translation Studies 0, 10 (2013). https: //lans.ua.ac.be/index.php/LANS-TTS/article/view/282

[22]

Sarah Sharples, Sue Cobb, Amanda Moody, and John R Wilson. 2008. Virtual reality induced symptoms and effects (VRISE): Comparison of head mounted display (HMD), desktop and projection display systems. Displays 29, 2 (2008), 58--69.

[23]

Alia Sheikh, Andy Brown, Zillah Watson, and Michael Evans. Directing attention in 360-degree video. In Proc. IBC '16.

[24]

Guanghan Song, Denis Pellerin, and Lionel Granjon. 2013. Different types of sounds influence gaze differently in videos. Journal of Eye Movement Research 6, 4 (2013). https://bop.unibe.ch/index.php/JEMR/article/view/2360

[25]

The New York Times. 2015. The Displaced | 360 VR Video | The New York Times. (2015). https://www.youtube.com/watch?v=ecavbpCuvkI

[26]

T. Vigier, Y. Baveye, J. Rousseau, and P. Le Callet. 2016. Visual attention as a dimension of QoE: Subtitles in UHD videos. In 2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX). 1--6.

Cited By

Yu MBi C(2023)Timeline Exploration in 360° VideoProceedings of the 2023 ACM Symposium on Spatial User Interaction10.1145/3607822.3614533(1-12)Online publication date: 13-Oct-2023
https://dl.acm.org/doi/10.1145/3607822.3614533
Mack NHeun MRose M(2023)Head-anchored text placements and cognitive load in information-rich virtual environmentsProceedings of Mensch und Computer 202310.1145/3603555.3603575(27-36)Online publication date: 3-Sep-2023
https://dl.acm.org/doi/10.1145/3603555.3603575
Nevsky ANeate TSimperl EVatavu R(2023)Accessibility Research in Digital Audiovisual Media: What Has Been Achieved and What Should Be Done Next?Proceedings of the 2023 ACM International Conference on Interactive Media Experiences10.1145/3573381.3596159(94-114)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3573381.3596159
Show More Cited By

Index Terms

Subtitles in 360-degree Video

Recommendations

Adaptive Subtitles: Preferences and Trade-Offs in Real-Time Media Adaption
CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

Subtitles can help improve the understanding of media content. People enable subtitles based on individual characteristics (e.g., language or hearing ability), viewing environment, or media context (e.g., drama, quiz show). However, some people find ...
Dynamic Subtitles in Cinematic Virtual Reality
TVX '18: Proceedings of the 2018 ACM International Conference on Interactive Experiences for TV and Online Video

Cinematic Virtual Reality has been increasing in popularity in recent years. Watching 360° movies with a Head Mounted Display, the viewer can freely choose the direction of view, and thus the visible section of the movie. Therefore, a new approach for ...
Dynamic Subtitles: The User Experience
TVX '15: Proceedings of the ACM International Conference on Interactive Experiences for TV and Online Video

Subtitles (closed captions) on television are typically placed at the bottom-centre of the screen. However, placing subtitles in varying positions, according to the underlying video content (`dynamic subtitles'), has the potential to make the overall ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

TVX '17 Adjunct: Adjunct Publication of the 2017 ACM International Conference on Interactive Experiences for TV and Online Video

June 2017

144 pages

ISBN:9781450350235

DOI:10.1145/3084289

General Chairs:
Judith Redi
Delft University of Technology, The Netherlands
,
Omar Niamut
TNO, The Netherlands
,
Dick Bulterman
Amsterdam Free University, The Netherlands
,
Program Chairs:
Wendy van den Broeck
Vrije Universiteit Brussel, Belgium
,
Mike Darnell
Samsung, USA
,
Roger Zimmermann
NUS, Singapore

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 June 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

TVX '17

Sponsor:

SIGCHI

TVX '17: ACM International Conference on Interactive Experiences for TV and Online Video

June 14 - 16, 2017

Hilversum, The Netherlands

Acceptance Rates

Overall Acceptance Rate 69 of 245 submissions, 28%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

34
Total Citations
View Citations
673
Total Downloads

Downloads (Last 12 months)64
Downloads (Last 6 weeks)7

Reflects downloads up to 12 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yu MBi C(2023)Timeline Exploration in 360° VideoProceedings of the 2023 ACM Symposium on Spatial User Interaction10.1145/3607822.3614533(1-12)Online publication date: 13-Oct-2023
https://dl.acm.org/doi/10.1145/3607822.3614533
Mack NHeun MRose M(2023)Head-anchored text placements and cognitive load in information-rich virtual environmentsProceedings of Mensch und Computer 202310.1145/3603555.3603575(27-36)Online publication date: 3-Sep-2023
https://dl.acm.org/doi/10.1145/3603555.3603575
Nevsky ANeate TSimperl EVatavu R(2023)Accessibility Research in Digital Audiovisual Media: What Has Been Achieved and What Should Be Done Next?Proceedings of the 2023 ACM International Conference on Interactive Media Experiences10.1145/3573381.3596159(94-114)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3573381.3596159
Brescia-Zapata MKrejtz KDuchowski AHughes COrero P(2023)Subtitles in VR 360° video. Results from an eye-tracking experimentPerspectives10.1080/0907676X.2023.2268122(1-23)Online publication date: 13-Nov-2023
https://doi.org/10.1080/0907676X.2023.2268122
Allman SCordy JHall JKleanthous VLander E(2022)Exploring the Perception of Additional Information Content in 360° 3D VR Video for Teaching and LearningVirtual Worlds10.3390/virtualworlds10100011:1(1-17)Online publication date: 13-May-2022
https://doi.org/10.3390/virtualworlds1010001
Anderton C(2022)Investigating Sign Language Interpreter Rendering and Guiding Methods in Virtual Reality 360-Degree ContentProceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3517428.3563373(1-6)Online publication date: 23-Oct-2022
https://dl.acm.org/doi/10.1145/3517428.3563373
Alonzo OShin HLi D(2022)Beyond Subtitles: Captioning and Visualizing Non-speech Sounds to Improve Accessibility of User-Generated VideosProceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3517428.3544808(1-12)Online publication date: 23-Oct-2022
https://dl.acm.org/doi/10.1145/3517428.3544808
Jing CDuc TTan PKamioka E(2022)Subtitle-based Viewport Prediction for 360-degree Virtual Tourism Video2022 13th International Conference on Information, Intelligence, Systems & Applications (IISA)10.1109/IISA56318.2022.9904420(1-8)Online publication date: 18-Jul-2022
https://doi.org/10.1109/IISA56318.2022.9904420
Hughes C(2022)Universal access: user needs for immersive captioningUniversal Access in the Information Society10.1007/s10209-021-00828-w21:2(393-403)Online publication date: 1-Jun-2022
https://dl.acm.org/doi/10.1007/s10209-021-00828-w
Soe TGuribye FSlavkovik M(2021)Evaluating AI assisted subtitlingProceedings of the 2021 ACM International Conference on Interactive Media Experiences10.1145/3452918.3458792(96-107)Online publication date: 21-Jun-2021
https://dl.acm.org/doi/10.1145/3452918.3458792
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents