research-article

Navigable Videos for Presenting Scientific Data on Affordable Head-Mounted Displays

Authors:

Jacqueline Chu,

Leonardo Ferrer,

Kwan-Liu MaAuthors Info & Claims

MMSys'17: Proceedings of the 8th ACM on Multimedia Systems Conference

Pages 250 - 260

https://doi.org/10.1145/3083187.3084015

Published: 20 June 2017 Publication History

Abstract

Immersive, stereoscopic visualization enables scientists to better analyze structural and physical phenomena compared to traditional display mediums. Unfortunately, current head-mounted displays (HMDs) with the high rendering quality necessary for these complex datasets are prohibitively expensive, especially in educational settings where their high cost makes it impractical to buy several devices. To address this problem, we develop two tools: (1) An authoring tool allows domain scientists to generate a set of connected, 360° video paths for traversing between dimensional keyframes in the dataset. (2) A corresponding navigational interface is a video selection and playback tool that can be paired with a low-cost HMD to enable an interactive, non-linear, storytelling experience. We demonstrate the authoring tool's utility by conducting several case studies and assess the navigational interface with a usability study. Results show the potential of our approach in effectively expanding the accessibility of high-quality, immersive visualization to a wider audience using affordable HMDs.

References

[1]

Supported Media Formats --- Android Devleopers. https://developer.android.com/guide/topics/media/media-formats.html. (2016).

[2]

Michael J Ackerman. 1998. The visible human project. Proc. IEEE 86, 3 (1998), 504--511.

[3]

Aseem Agarwala, Ke Colin Zheng, Chris Pal, Maneesh Agrawala, Michael Cohen, Brian Curless, David Salesin, and Richard Szeliski. 2005. Panoramic video textures. In ACM Transactions on Graphics (TOG), Vol. 24. 821--827.

Digital Library

[4]

Hiroshi Akiba, Chaoli Wang, and Kwan-Liu Ma. 2010. AniViz: A template-based animation tool for volume visualization. Computer Graphics and Applications, IEEE 30, 5 (2010), 61--71.

Digital Library

[5]

Xavier Amatriain, JoAnn Kuchera-Morin, Tobias Hollerer, and Stephen Travis Pope. 2009. The allosphere: Immersive multimedia for scientific discovery and artistic exploration. IEEE Multimedia 16, 2 (2009), 0064--75.

Digital Library

[6]

Kristoffer Brady and Richard Emms. 2015. Oculus Connect 2:Navigating New Worlds: Designing UI and UX in VR. https://www.youtube.com/watch?v=braV_c4M8oI. (24 September 2015).

[7]

Fanny Chevalier, Nathalie Henry Riche, Catherine Plaisant, Amira Chalbi, and Christophe Hurter. 2016. Animations 25 Years Later: New Roles and Opportunities. (2016).

[8]

Cagatay Demiralp, Cullen D Jackson, David B Karelitz, Song Zhang, and David H Laidlaw. 2006. Cave and fishtank virtual-reality displays: A qualitative and quantitative comparison. IEEE transactions on visualization and computer graphics 12, 3 (2006), 323--330.

Digital Library

[9]

Margaret Drouhard, Chad A Steed, Steven Hahn, Thomas Proffen, Jamison Daniel, and Michael Matheson. 2015. Immersive visualization for materials science data analysis using the Oculus Rift. In Big Data (Big Data), 2015 IEEE International Conference on. 2453--2461.

Digital Library

[10]

David S Ebert, Christopher D Shaw, Amen Zwa, and Cindy Starr. 1996. Two-handed interactive stereoscopic visualization. In Proceedings of the 7th Conference on Visualization'96. IEEE Computer Society Press, 205--ff.

Digital Library

[11]

Thomas A Funkhouser and Carlo H Séquin. 1993. Adaptive display algorithm for interactive frame rates during visualization of complex virtual environments. In Proceedings of the 20th annual conference on Computer graphics and interactive techniques. ACM, 247--254.

Digital Library

[12]

Desmond Germans, Hans JW Spoelder, Luc Renambot, and Henri E Bal. 2001. VIRPI: a high-level toolkit for interactive scientific visualization in virtual reality. In Immersive Projection Technology and Virtual Environments 2001. Springer, 109--120.

Digital Library

[13]

Nahum Gershon and Ward Page. 2001. What storytelling can do for information visualization. Commun. ACM 44, 8 (2001), 31--37.

Digital Library

[14]

Duke Gledhill, Gui Yun Tian, Dave Taylor, and David Clarke. 2003. Panoramic imaginga review. Computers & Graphics 21, 3 (2003), 435--445.

[15]

Stefan Göbel, Luca Salvatore, and Robert Konrad. 2008. StoryTec: A digital storytelling platform for the authoring and experiencing of interactive and nonlinear stories. In Automated solutions for Cross Media Content and Multi-channel Distribution, 2008. AXMEDIS'08. International Conference on. Ieee, 103--110.

Digital Library

[16]

Google. 2015. Rendering Omnidirectional Stereo Content. https://developers.google.com/cardboard/jump/rendering-ods-content.pdf. (2015).

[17]

Samuel Gratzl, Alexander Lex, Nils Gehlenborg, Nicola Cosgrove, and Marc Streit. 2016. From Visual Exploration to Storytelling and Back Again. bioRxiv (2016), 049585.

[18]

Claudia Hänel, Benjamin Weyers, Bernd Hentschel, and Torsten W Kuhlen. 2016. Visual Quality Adjustment for Volume Rendering in a Head-Tracked Virtual Environment. IEEE transactions on visualization and computer graphics 22, 4 (2016), 1472--1481.

Digital Library

[19]

Jeffrey Heer and George Robertson. 2007. Animated transitions in statistical data graphics. IEEE transactions on visualization and computer graphics 13, 6 (2007), 1240--1247.

Digital Library

[20]

Wei-Hsien Hsu, Yubo Zhang, and Kwan-Liu Ma. 2013. A multi-criteria approach to camera motion design for volume data animation. Visualization and Computer Graphics, IEEE Transactions on 19, 12 (2013), 2792--2801.

Digital Library

[21]

David Kanter. 2015. Graphics Processing Requirements for Enabling Immersive VR. AMD White Paper (2015).

[22]

Joe Kniss, Patrick McCormick, Allen McPherson, James Ahrens, James Painter, Alan Keahey, and Charles Hansen. 2001. Interactive texture-based volume rendering for large data sets. IEEE Computer Graphics and Applications 21, 4 (2001), 52--61.

Digital Library

[23]

Nick Kraakman. 2015. The Best Encoding Settings for Your 4K 360 3D VR Videos. http://www.purplepillvr.com/best-encoding-settings-resolution-for-4k-360-3d-vr-videos/. (1 December 2015).

[24]

Evgeny Kuzyakov and David Pio. 2016. Next-generation video encoding techniques for 360 video and VR. https://code.facebook.com/posts/1126354007399553/next-generation-video-encoding-techniques-for-360-video-and-vr/. (21 January 2016).

[25]

Bireswar Laha, Kriti Sensharma, James D Schiffbauer, and Doug A Bowman. 2012. Effects of immersion on visual analysis of volume data. Visualization and Computer Graphics, IEEE Transactions on 18, 4 (2012), 597--606.

Digital Library

[26]

Isaac Liao, Wei-Hsien Hsu, and Kwan-Liu Ma. 2014. Storytelling via Navigation: A Novel Approach to Animation for Scientific Visualization. In Smart Graphics. Springer, 1--14.

[27]

Ajay Limaye. 2012. Drishti: a volume exploration and presentation tool. In SPIE Optical Engineering + Applications. International Society for Optics and Photonics, 85060X--85060X.

[28]

Florian Lindemann and Timo Ropinski. 2011. About the influence of illumination models on image comprehension in direct volume rendering. IEEE Transactions on Visualization and Computer Graphics 17, 12 (2011), 1922--1931.

Digital Library

[29]

Kwan-Liu Ma, Isaac Liao, Jennifer Frazier, Helwig Hauser, and H-N Kostis. 2012. Scientific storytelling using visualization. Computer Graphics and Applications, IEEE 32,1 (2012), 12--19.

Digital Library

[30]

Stefan Marks, Javier E Estevez, and Andy M Connor. 2014. Towards the Holodeck: fully immersive virtual reality visualisation of scientific and engineering data. In Proceedings of the 29th International Conference on Image and Vision Computing New Zealand. ACM, 42--47.

Digital Library

[31]

Britta Meixner and Harald Kosch. 2012. Interactive non-linear video: definition and XML structure. In Proceedings of the 2012 ACM symposium on Document engineering. 49--58.

Digital Library

[32]

Britta Meixner, Beate Siegel, Günther Hölbling, Franz Lehner, and Harald Kosch. 2010. SIVA suite: authoring system and player for interactive non-linear videos. In Proceedings of the 18th ACM international conference on Multimedia. 1563--1566.

Digital Library

[33]

Jose Noguera and J Roberto Jimenez. 2016. Mobile Volume Rendering: Past, Present and Future. (2016).

[34]

Michal Ponder, Bruno Herbelin, Tom Molet, Sebastien Schertenlieb, Branislav Ulicny, George Papagiannakis, Nadia Magnenat-Thalmann, and Daniel Thalmann. 2003. Immersive VR decision training: telling interactive stories featuring advanced virtual human simulation technologies. In Proceedings of the workshop on Virtual environments 2003. ACM, 97--106.

Digital Library

[35]

Michael Prince. 2004. Does active learning work? A review of the research. Journal of engineering education 93, 3 (2004), 223--231.

[36]

Khairi Reda, Aaron Knoll, Ken-ichi Nomura, Michael E Papka, Andrew E Johnson, and Jason Leigh. 2013. Visualizing large-scale atomistic simulations in ultra-resolution immersive environments. In LDAV. 59--65.

[37]

George Robertson, Roland Fernandez, Danyel Fisher, Bongshin Lee, and John Stasko. 2008. Effectiveness of animation in trend visualization. Visualization and Computer Graphics, IEEE Transactions on 14, 6 (2008), 1325--1332.

Digital Library

[38]

Maria Roussou. 2000. Immersive interactive virtual reality and informal education. In Proceedings of User Interfaces for All: Interactive Learning Environments for Children.

[39]

Maria Roussou. 2004. Learning by doing and learning through play: an exploration of interactivity in virtual environments for children. Computers in Entertainment (CIE) 2, 1 (2004), 10--10.

Digital Library

[40]

Hans-Jörg Schulz and Christophe Hurter. 2013. Grooming the hairball-how to tidy up network visualizations?. In INFOVIS 2013, IEEE Information Visualization Conference.

[41]

Edward Segel and Jeffrey Heer. 2010. Narrative visualization: Telling stories with data. Visualization and Computer Graphics, IEEE Transactions on 16, 6 (2010), 1139--1148.

Digital Library

[42]

Sarah Sharpies, Sue Cobb, Amanda Moody, and John R Wilson. 2008. Virtual reality induced symptoms and effects (VRISE): Comparison of head mounted display (HMD), desktop and projection display systems. Displays 29, 2 (2008), 58--69.

[43]

Frank Shipman, Andreas Girgensohn, and Lynn Wilcox. 2003. Hyper-Hitchcock: Towards the easy authoring of interactive video. In Human-Computer Interaction INTERACT, Vol. 3. 33--40.

[44]

Julie Steele and Noah Iliinsky. 2010. Beautiful visualization: looking at data through the eyes of experts. " O'Reilly Media, Inc.".

Digital Library

[45]

John E Stone, William R Sherman, and Klaus Schulten. 2016. Immersive molecular visualization with omnidirectional stereoscopic ray tracing and remote rendering. In 2016 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW). IEEE, 1048--1057.

[46]

Andries Van Dam, Andrew S Forsberg, David H Laidlaw, Joseph J LaViola Jr, and Rosemary M Simpson. 2000. Immersive VR for scientific visualization: A progress report. Computer Graphics and Applications, IEEE 20, 6 (2000), 26--52.

Digital Library

[47]

Michael Wohlfart. 2006. Story Telling Aspects in Medical Applications. In Central European Seminar on Computer Graphics.

[48]

Michael Wohlfart and Helwig Hauser. 2007. Story telling for presentation in volume visualization. In Proceedings of the 9th Joint Eurographics/IEEE VGTC conference on Visualization. Eurographics Association, 91--98.

Digital Library

[49]

Dongsong Zhang, Lina Zhou, Robert O Briggs, and Jay F Nunamaker. 2006. Instructional video in e-learning: Assessing the impact of interactive video on learning effectiveness. Information & management 43, 1 (2006), 15--27.

Digital Library

[50]

Song Zhang, Cagatay Demiralp, Daniel F Keefe, Marco DaSilva, David H Laidlaw, BD Greenberg, Peter J Basser, Carlo Pierpaoli, Ennio Antonio Chiocca, and Thomas S Deisboeck. 2001. An immersive virtual environment for DT-MRI volume visualization applications: a case study. In Visualization, 2001. VIS'01. Proceedings. IEEE, 437--584.

Digital Library

Cited By

Coelho HMonteiro PGonçalves GMelo MBessa M(2024)Immersive Creation of Virtual Reality Training ExperiencesIEEE Access10.1109/ACCESS.2024.341333712(85773-85782)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3413337
You WCheng ZMa ZYang GZhou ZSun L(2023)HierVid: Lowering the Barriers to Entry of Interactive Video Making with a Hierarchical Authoring SystemInternational Journal of Human–Computer Interaction10.1080/10447318.2023.226785940:22(7688-7703)Online publication date: 3-Nov-2023
https://doi.org/10.1080/10447318.2023.2267859
Zheng CYin JWei FGuan YGuo ZZhang X(2022)STC: FoV Tracking Enabled High-Quality 16K VR Video Streaming on Mobile PlatformsIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2021.309490932:4(2396-2410)Online publication date: Apr-2022
https://doi.org/10.1109/TCSVT.2021.3094909
Show More Cited By

Index Terms

Navigable Videos for Presenting Scientific Data on Affordable Head-Mounted Displays
1. Applied computing
  1. Education
    1. Interactive learning environments
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Virtual reality
  2. Visualization
    1. Visualization application domains
      1. Scientific visualization

Recommendations

Natural Perspective Projections for Head-Mounted Displays

The display units integrated in today's head-mounted displays (HMDs) provide only a limited field of view (FOV) to the virtual world. In order to present an undistorted view to the virtual environment (VE), the perspective projection used to render the ...
Estimation of virtual interpupillary distances for immersive head-mounted displays
APGV '10: Proceedings of the 7th Symposium on Applied Perception in Graphics and Visualization

Head-mounted displays (HMDs) allow users to observe virtual environments (VEs) from an egocentric perspective. In order to present a realistic stereoscopic view, the rendering system has to be adjusted to the characteristics of the HMD, e. g., the ...
Enhancing Presence in Head-Mounted Display Environments by Visual Body Feedback Using Head-Mounted Cameras
CW '09: Proceedings of the 2009 International Conference on CyberWorlds

A fully-articulated visual representation of a user in an immersive virtual environment (IVE) can enhance the user's subjective sense of feeling present in the virtual world. Usually this requires the user to wear a full-body motion capture suit to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MMSys'17: Proceedings of the 8th ACM on Multimedia Systems Conference

June 2017

407 pages

ISBN:9781450350020

DOI:10.1145/3083187

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 June 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

MMSys'17

Sponsor:

SIGMM

MMSys'17: Multimedia Systems Conference 2017

June 20 - 23, 2017

Taipei, Taiwan

Acceptance Rates

MMSys'17 Paper Acceptance Rate 13 of 47 submissions, 28%;

Overall Acceptance Rate 176 of 530 submissions, 33%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
243
Total Downloads

Downloads (Last 12 months)16
Downloads (Last 6 weeks)1

Reflects downloads up to 27 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Coelho HMonteiro PGonçalves GMelo MBessa M(2024)Immersive Creation of Virtual Reality Training ExperiencesIEEE Access10.1109/ACCESS.2024.341333712(85773-85782)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3413337
You WCheng ZMa ZYang GZhou ZSun L(2023)HierVid: Lowering the Barriers to Entry of Interactive Video Making with a Hierarchical Authoring SystemInternational Journal of Human–Computer Interaction10.1080/10447318.2023.226785940:22(7688-7703)Online publication date: 3-Nov-2023
https://doi.org/10.1080/10447318.2023.2267859
Zheng CYin JWei FGuan YGuo ZZhang X(2022)STC: FoV Tracking Enabled High-Quality 16K VR Video Streaming on Mobile PlatformsIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2021.309490932:4(2396-2410)Online publication date: Apr-2022
https://doi.org/10.1109/TCSVT.2021.3094909
Coelho HMonteiro PGonçalves GMelo MBessa M(2022)Authoring tools for virtual reality experiences: a systematic reviewMultimedia Tools and Applications10.1007/s11042-022-12829-981:19(28037-28060)Online publication date: 29-Mar-2022
https://doi.org/10.1007/s11042-022-12829-9
Wang ZQiu SChen QTrayan NRinglein ADorsey JRushmeier H(2019)AniCodeThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-019-01681-y35:6-8(885-897)Online publication date: 1-Jun-2019
https://dl.acm.org/doi/10.1007/s00371-019-01681-y
Ren DLee BHöllerer TMorishima SItoh YShiratori TYue YLindeman R(2018)XRCreatorProceedings of the 24th ACM Symposium on Virtual Reality Software and Technology10.1145/3281505.3283400(1-2)Online publication date: 28-Nov-2018
https://dl.acm.org/doi/10.1145/3281505.3283400
Nguyen CDiVerdi SHertzmann ALiu FMandryk RHancock MPerry MCox A(2018)Depth Conflict Reduction for Stereo VR Video InterfacesProceedings of the 2018 CHI Conference on Human Factors in Computing Systems10.1145/3173574.3173638(1-9)Online publication date: 21-Apr-2018
https://dl.acm.org/doi/10.1145/3173574.3173638

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten