Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3196709.3196719acmconferencesArticle/Chapter ViewAbstractPublication PagesdisConference Proceedingsconference-collections
research-article

Scale Impacts Elicited Gestures for Manipulating Holograms: Implications for AR Gesture Design

Published: 08 June 2018 Publication History

Abstract

Because gesture design for augmented reality (AR) remains idiosyncratic, people cannot necessarily use gestures learned in one AR application in another. To design discoverable gestures, we need to understand what gestures people expect to use. We explore how the scale of AR affects the gestures people expect to use to interact with 3D holograms. Using an elicitation study, we asked participants to generate gestures in response to holographic task referents, where we varied the scale of holograms from desktop-scale to room-scale objects. We found that the scale of objects and scenes in the AR experience moderates the generated gestures. Most gestures were informed by physical interaction, and when people interacted from a distance, they sought a good perspective on the target object before and during the interaction. These results suggest that gesture designers need to account for scale, and should not simply reuse gestures across different hologram sizes.

References

[1]
Apple. ARKit. Accessed September 9th, 2017. https://developer.apple.com/arkit/
[2]
Ilhan Aslan, Martin Murer, Verena Fuchsberger, Andrew Fugard, and Manfred Tscheligi. 2013. Drag and drop the apple: the semantic weight of words and images in touch-based interaction. In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction (TEI '13). ACM, New York, NY, USA, 159--166.
[3]
Hrvoje Benko, Ricardo Jota, and Andrew Wilson. 2012. MirageTable: freehand interaction on a projected augmented reality tabletop. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, New York, NY, USA, 199208.
[4]
Mark Billinghurst, Adrian Clark, and Gun Lee. 2015. A Survey of Augmented Reality. Found. Trends Hum.Comput. Interact. 8, 2--3 (March 2015), 73--272.
[5]
Mark Billinghurst, Hirokazu Kato, and Ivan Poupyrev. 2001. The MagicBook: a transitional AR interface. Computers & Graphics, 25(5), 745--753.
[6]
Oliver Bimber and Ramesh Raskar. 2005. Spatial Augmented Reality: Merging Real and Virtual Worlds. A. K. Peters, Ltd., Natick, MA, USA.
[7]
Volkert Buchmann, Stephen Violich, Mark Billinghurst, and Andy Cockburn. 2004. FingARtips: gesture based direct manipulation in Augmented Reality. In Proceedings of the 2nd international conference on Computer graphics and interactive techniques in Australasia and South East Asia (GRAPHITE '04), Stephen N. Spencer (Ed.). ACM, New York, NY, USA, 212--221.
[8]
Juliet Corbin and Anselm Strauss. 2015. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. Sage. ISBN: 9781412997461.
[9]
Directive Games. The Machines. Accessed January 5th, 2018. http://themachinesgame.com/
[10]
Bruno Fernandes and Joaquin Fernández. 2009. Bare hand interaction in tabletop augmented reality. In SIGGRAPH '09: Posters (SIGGRAPH '09). ACM, New York, NY, USA, Article 98, 1 pages.
[11]
Tiare Feuchtner and Jörg Müller. 2017. Extending the Body for Interaction with Reality. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 5145--5157.
[12]
G. W. Furnas, T. K. Landauer, L. M. Gomez, and S. T. Dumais. 1987. The vocabulary problem in humansystem communication. Commun. ACM 30, 11 (November 1987), 964--971.
[13]
Maurizio Gentilucci, Francesca Benuzzi, Luca Bertolani, Elena Daprati, and Massimo Gangitano. 2000. Language and motor control. Experimental Brain Research, 133(4), 468--490.
[14]
Scott Glover, David A. Rosenbaum, Jeremy Graham, and Peter Dixon. 2004. Grasping the meaning of words. Experimental Brain Research, 154(1), 103--108.
[15]
Gunther Heidemann, Ingo Bax, and Holger Bekel. 2004. Multimodal interaction in an augmented reality scenario. In Proceedings of the 6th international conference on Multimodal interfaces (ICMI '04). ACM, New York, NY, USA, 53--60.
[16]
Otmar Hilliges, David Kim, Shahram Izadi, Malte Weiss, and Andrew Wilson. 2012. HoloDesk: direct 3d interactions with a situated see-through display. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, New York, NY, USA, 2421--2430.
[17]
Sylvia Irawati, Scott Green, Mark Billinghurst, Andreas Duenser, and Heedong Ko. 2006. "Move the couch where?": developing an augmented reality multimodal interface. In Proceedings of the 5th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR '06). IEEE Computer Society, Washington, DC, USA, 183--186.
[18]
Robert J.K. Jacob, Audrey Girouard, Leanne M. Hirshfield, Michael S. Horn, Orit Shaer, Erin Treacy Solovey, and Jamie Zigelbaum. 2008. Reality-based interaction: a framework for post-WIMP interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '08). ACM, New York, NY, USA, 201--210.
[19]
Brett R. Jones, Hrvoje Benko, Eyal Ofek, and Andrew D. Wilson. 2013. IllumiRoom: peripheral projected illusions for interactive experiences. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, New York, NY, USA, 869--878.
[20]
Brett Jones, Rajinder Sodhi, Michael Murdock, Ravish Mehra, Hrvoje Benko, Andrew Wilson, Eyal Ofek, Blair MacIntyre, Nikunj Raghuvanshi, and Lior Shapira. 2014. RoomAlive: magical experiences enabled by scalable, adaptive projector-camera units. In Proceedings of the 27th annual ACM symposium on User interface software and technology (UIST '14). ACM, New York, NY, USA, 637--644.
[21]
Ed Kaiser, Alex Olwal, David McGee, Hrvoje Benko, Andrea Corradini, Xiaoguang Li, Phil Cohen, and Steven Feiner. 2003. Mutual disambiguation of 3D multimodal interaction in augmented and virtual reality. In Proceedings of the 5th international conference on Multimodal interfaces (ICMI '03). ACM, New York, NY, USA, 12--19.
[22]
Hirokazu Kato, Mark Billinghurst, Ivan Poupyrev, Kenji Imamoto, and Keihachiro Tachibana. Virtual object manipulation on a table-top AR environment. In Proceedings of the IEEE and ACM International Symposium on Augmented Reality (ISAR 2000). IEEE, 111--119.
[23]
Mathias Koelsch, Ryan Bane, Tobias Hoellerer, and Matthew Turk. 2006. Multimodal Interaction with a Wearable Augmented Reality System. IEEE Comput. Graph. Appl. 26, 3 (May 2006), 62--71.
[24]
Leap Motion. Accessed January 3rd, 2018. https://www.leapmotion.com/
[25]
Minkyung Lee and Mark Billinghurst. 2008. A Wizard of Oz study for an AR multimodal interface. In Proceedings of the 10th international conference on Multimodal interfaces (ICMI '08). ACM, New York, NY, USA, 249--256.
[26]
Jae Yeol Lee, Gue Won Rhee, and Dong Woo Seo. 2010. Hand gesture-based tangible interactions for manipulating virtual objects in a mixed reality environment. The International Journal of Advanced Manufacturing Technology, 51(9), 1069--1082.
[27]
Taehee Lee, and Tobias Hollerer. 2007. Handy AR: Markerless inspection of augmented reality objects using fingertip tracking. In Proceedings of 11th IEEE International Symposium on Wearable Computers (ISWC '07). IEEE, 83--90.
[28]
Taehee Lee and Tobias Höllerer. 2009. Multithreaded Hybrid Feature Tracking for Markerless Augmented Reality. IEEE Transactions on Visualization and Computer Graphics 15, 3 (May 2009), 355--368.
[29]
Blair MacIntyre, Alex Hill, Hafez Rouzati, Maribeth Gandy, and Brian Davidson. 2011. The Argon AR Web Browser and standards-based AR application environment. In Proceedings of the 2011 10th IEEE International Symposium on Mixed and Augmented Reality (ISMAR '11). IEEE Computer Society, Washington, DC, USA, 65--74.
[30]
Meredith Ringel Morris, Andreea Danielescu, Steven Drucker, Danyel Fisher, Bongshin Lee, m. c. schraefel, and Jacob O. Wobbrock. 2014. Reducing legacy bias in gesture elicitation studies. interactions 21, 3 (May 2014), 40--45.
[31]
Meredith Ringel Morris, Jacob O. Wobbrock, and Andrew D. Wilson. 2010. Understanding users' preferences for surface gestures. In Proceedings of Graphics Interface 2010 (GI '10). Canadian Information Processing Society, Toronto, Ont., Canada, Canada, 261--268.
[32]
Microsoft. Hologram Stability. Accessed March 23rd, 2018. https://docs.microsoft.com/enus/windows/mixed-reality/hologram-stability
[33]
Microsoft. Kinect for Windows. Accessed January 3rd, 2018. https://developer.microsoft.com/enus/windows/kinect
[34]
Microsoft. Skype for HoloLens. Accessed January 3rd, 2018. https://www.microsoft.com/enus/hololens/apps/skype
[35]
Alex Olwal, Hrvoje Benko, and Steven Feiner. 2003. Senseshapes: Using statistical geometry for object selection in a multimodal augmented reality. In Proceedings of The Second IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR '03). IEEE. 300--301.
[36]
Sergio Orts-Escolano, Christoph Rhemann, Sean Fanello, Wayne Chang, Adarsh Kowdle, Yury Degtyarev, David Kim, Philip L. Davidson, Sameh Khamis, Mingsong Dou, Vladimir Tankovich, Charles Loop, Qin Cai, Philip A. Chou, Sarah Mennicken, Julien Valentin, Vivek Pradeep, Shenlong Wang, Sing Bing Kang, Pushmeet Kohli, Yuliya Lutchyn, Cem Keskin, and Shahram Izadi. 2016. Holoportation: Virtual 3D Teleportation in Real-time. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). ACM, New York, NY, USA, 741--754.
[37]
Tomislav Pejsa, Julian Kantor, Hrvoje Benko, Eyal Ofek, and Andrew Wilson. 2016. Room2Room: Enabling Life-Size Telepresence in a Projected Augmented Reality Environment. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (CSCW '16). ACM, New York, NY, USA, 1716--1725.
[38]
Wayne Piekarski and Bruce H. Thomas. 2002. Using ARToolKit for 3D hand position tracking in mobile outdoor environments. In Augmented Reality Toolkit, The first IEEE International Workshop. IEEE.
[39]
Jeffrey S. Pierce, Andrew S. Forsberg, Matthew J. Conway, Seung Hong, Robert C. Zeleznik, and Mark R. Mine. 1997. Image plane interaction techniques in 3D immersive environments. In Proceedings of the 1997 symposium on Interactive 3D graphics (I3D '97). ACM, New York, NY, USA, 39-ff.
[40]
Thammathip Piumsomboon, Adrian Clark, Mark Billinghurst, and Andy Cockburn. 2013. User-Defined Gestures for Augmented Reality. In IFIP Conference on Human-Computer Interaction (INTERACT '13). Springer, Berlin, Heidelberg. 282--299.
[41]
Thammathip Piumsomboon, Adrian Clark, Atsushi Umakatsu, & Mark Billinghurst. 2012. Poster: Physically-based natural hand and tangible AR interaction for face-to-face collaboration on a tabletop. In Proceedings of IEEE Symposium on 3D User Interfaces (3DUI '02). 155--156. IEEE.
[42]
Thammathip Piumsomboon, David Altimira, Hyungon Kim, Adrian Clark, Gun Lee, and Mark Billinghurst. 2014. Grasp-Shell vs gesture-speech: A comparison of direct and indirect natural interaction techniques in augmented reality. In Proceedings of the IEEE International Symposium on Mixed and Augmented Reality (ISMAR '14). IEEE. 73--82.
[43]
Christina Pollalis, Whitney Fahnbulleh, Jordan Tynes, and Orit Shaer. 2017. HoloMuse: Enhancing Engagement with Archaeological Artifacts through Gesture-Based Interaction with Holograms. In Proceedings of the Eleventh International Conference on Tangible, Embedded, and Embodied Interaction (TEI '17). ACM, New York, NY, USA, 565--570.
[44]
Ivan Poupyrev, Mark Billinghurst, Suzanne Weghorst, and Tadao Ichikawa. 1996. The go-go interaction technique: non-linear mapping for direct manipulation in VR. In Proceedings of the 9th annual ACM symposium on User interface software and technology (UIST '96). ACM, New York, NY, USA, 79--80.
[45]
Gerhard Reitmayr, and Dieter Schmalstieg. 2004. Scalable techniques for collaborative outdoor augmented reality. In Proceedings of the 3rd IEEE and ACM international symposium on mixed and augmented reality (ISMAR'04). IEEE.
[46]
Gustavo Alberto Rovelo Ruiz, Davy Vanacken, Kris Luyten, Francisco Abad, and Emilio Camahort. 2014. Multi-viewer gesture-based interaction for omnidirectional video. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 40774086.
[47]
Jaime Ruiz, Yang Li, and Edward Lank. 2011. Userdefined motion gestures for mobile interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 197--206.
[48]
Christian Sandor, Alex Olwal, Blaine Bell, and Steven Feiner. 2005. Immersive Mixed-Reality Configuration of Hybrid User Interfaces. In Proceedings of the 4th IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR '05). IEEE Computer Society, Washington, DC, USA, 110--113.
[49]
Gerhard Schall, Erick Mendez, Ernst Kruijff, Eduardo Veas, Sebastian Junghanns, Bernhard Reitinger, and Dieter Schmalstieg. 2009. Handheld Augmented Reality for underground infrastructure visualization. Personal Ubiquitous Comput. 13, 4 (May 2009), 281291.
[50]
Teddy Seyed, Chris Burns, Mario Costa Sousa, Frank Maurer, and Anthony Tang. 2012. Eliciting usable gestures for multi-display environments. In Proceedings of the 2012 ACM international conference on Interactive tabletops and surfaces (ITS '12). ACM, New York, NY, USA, 41--50.
[51]
Shaikh Shawon Arefin Shimon, Courtney Lutton, Zichun Xu, Sarah Morrison-Smith, Christina Boucher, and Jaime Ruiz. 2016. Exploring Non-touchscreen Gestures for Smartwatches. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 38223833.
[52]
Shaikh Shawon Arefin Shimon, Sarah Morrison-Smith, Noah John, Ghazal Fahimi, and Jaime Ruiz. 2015. Exploring User-Defined Back-Of-Device Gestures for Mobile Devices. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '15). ACM, New York, NY, USA, 227232.
[53]
Adalberto L. Simeone, Andreas Bulling, Jason Alexander, and Hans Gellersen. 2016. Three-Point Interaction: Combining Bi-manual Direct Touch with Gaze. In Proceedings of the International Working Conference on Advanced Visual Interfaces (AVI '16), Paolo Buono, Rosa Lanzilotti, and Maristella Matera (Eds.). ACM, New York, NY, USA, 168--175.
[54]
Trimble, Inc. SketchUp: 3D modeling for everyone. Accessed September 9th, 2017. https://www.sketchup.com/
[55]
Giovanni Maria Troiano, Esben Warming Pedersen, and Kasper Hornbæk. 2014. User-defined gestures for elastic, deformable displays. In Proceedings of the 2014 International Working Conference on Advanced Visual Interfaces (AVI '14). ACM, New York, NY, USA, 1--8.
[56]
John Underkoffler and Hiroshi Ishii. 1999. Urp: a luminous-tangible workbench for urban planning and design. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (CHI '99). ACM, New York, NY, USA, 386--393.
[57]
Radu-Daniel Vatavu and Ionut-Alexandru Zaiti. 2014. Leap gestures for TV: insights from an elicitation study. In Proceedings of the ACM International Conference on Interactive Experiences for TV and Online Video (TVX '14). ACM, New York, NY, USA, 131--138.
[58]
Radu-Daniel Vatavu and Jacob O. Wobbrock. 2015. Formalizing Agreement Analysis for Elicitation Studies: New Measures, Significance Test, and Toolkit. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 1325--1334.
[59]
Andrew D. Wilson and Hrvoje Benko. 2010. Combining multiple depth cameras and projectors for interactions on, above and between surfaces. In Proceedings of the 23nd annual ACM symposium on User interface software and technology (UIST '10). ACM, New York, NY, USA, 273--282.
[60]
Andrew D. Wilson and Hrvoje Benko. 2017. Holograms without Headsets: Projected Augmented Reality with the RoomAlive Toolkit. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '17). ACM, New York, NY, USA, 425--428.
[61]
Jacob O. Wobbrock, Htet Htet Aung, Brandon Rothrock, and Brad A. Myers. 2005. Maximizing the guessability of symbolic input. In CHI '05 Extended Abstracts on Human Factors in Computing Systems (CHI EA '05). ACM, New York, NY, USA, 1869--1872.
[62]
Jacob O. Wobbrock, Meredith Ringel Morris, and Andrew D. Wilson. 2009. User-defined gestures for surface computing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). ACM, New York, NY, USA, 10831092.
[63]
Yan Xu, Sam Mendenhall, Vu Ha, Paul Tillery, and Joshua Cohen. 2012. Herding nerds on your table: NerdHerder, a mobile augmented reality game. In CHI '12 Extended Abstracts on Human Factors in Computing Systems (CHI EA '12). ACM, New York, NY, USA, 1351--1356.

Cited By

View all
  • (2024)Understanding Novice Users' Mental Models of Gesture Discoverability and Designing Effective OnboardingCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3678370(290-295)Online publication date: 5-Oct-2024
  • (2024)How People Prompt Generative AI to Create Interactive VR ScenesProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661547(2319-2340)Online publication date: 1-Jul-2024
  • (2024)Do I Just Tap My Headset?Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314517:4(1-28)Online publication date: 12-Jan-2024
  • Show More Cited By

Index Terms

  1. Scale Impacts Elicited Gestures for Manipulating Holograms: Implications for AR Gesture Design

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    DIS '18: Proceedings of the 2018 Designing Interactive Systems Conference
    June 2018
    1418 pages
    ISBN:9781450351980
    DOI:10.1145/3196709
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 08 June 2018

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. augmented reality
    2. gesture elicitation
    3. gestures
    4. hololens

    Qualifiers

    • Research-article

    Conference

    DIS '18
    Sponsor:

    Acceptance Rates

    DIS '18 Paper Acceptance Rate 107 of 487 submissions, 22%;
    Overall Acceptance Rate 1,158 of 4,684 submissions, 25%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)93
    • Downloads (Last 6 weeks)14
    Reflects downloads up to 22 Sep 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Understanding Novice Users' Mental Models of Gesture Discoverability and Designing Effective OnboardingCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3678370(290-295)Online publication date: 5-Oct-2024
    • (2024)How People Prompt Generative AI to Create Interactive VR ScenesProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661547(2319-2340)Online publication date: 1-Jul-2024
    • (2024)Do I Just Tap My Headset?Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314517:4(1-28)Online publication date: 12-Jan-2024
    • (2024)Exploring Visualizations for Precisely Guiding Bare Hand Gestures in Virtual RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642935(1-19)Online publication date: 11-May-2024
    • (2024)An Artists' Perspectives on Natural Interactions for Virtual Reality 3D SketchingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642758(1-20)Online publication date: 11-May-2024
    • (2024)MAR Enhanced Hiking Experience: Exploring Multimodal Experience Design in Outdoor ScenariosDistributed, Ambient and Pervasive Interactions10.1007/978-3-031-60012-8_22(347-364)Online publication date: 1-Jun-2024
    • (2023)Brave New GES World: A Systematic Literature Review of Gestures and Referents in Gesture Elicitation StudiesACM Computing Surveys10.1145/363645856:5(1-55)Online publication date: 7-Dec-2023
    • (2023)Gesture-based Interaction for AR Systems: A Short ReviewProceedings of the 16th International Conference on PErvasive Technologies Related to Assistive Environments10.1145/3594806.3594815(284-292)Online publication date: 5-Jul-2023
    • (2023)Towards a Consensus Gesture Set: A Survey of Mid-Air Gestures in HCI for Maximized Agreement Across DomainsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581420(1-24)Online publication date: 19-Apr-2023
    • (2023)Exploring Mid-Air Hand Interaction in Data VisualizationIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.333264730:9(6347-6364)Online publication date: 14-Nov-2023
    • Show More Cited By

    View Options

    Get Access

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media