research-article

Opportunistic controls: leveraging natural affordances as tangible user interfaces for augmented reality

Authors:

Steven J. Henderson,

Steven FeinerAuthors Info & Claims

VRST '08: Proceedings of the 2008 ACM symposium on Virtual reality software and technology

Pages 211 - 218

https://doi.org/10.1145/1450579.1450625

Published: 27 October 2008 Publication History

Abstract

We present Opportunistic Controls, a class of user interaction techniques for augmented reality (AR) applications that support gesturing on, and receiving feedback from, otherwise unused affordances already present in the domain environment. Opportunistic Controls leverage characteristics of these affordances to provide passive haptics that ease gesture input, simplify gesture recognition, and provide tangible feedback to the user. 3D widgets are tightly coupled with affordances to provide visual feedback and hints about the functionality of the control. For example, a set of buttons is mapped to existing tactile features on domain objects. We describe examples of Opportunistic Controls that we have designed and implemented using optical marker tracking, combined with appearance-based gesture recognition. We present the results of a user study in which participants performed a simulated maintenance inspection of an aircraft engine using a set of virtual buttons implemented both as Opportunistic Controls and using simpler passive haptics. Opportunistic Controls allowed participants to complete their tasks significantly faster and were preferred over the baseline technique.

Supplementary Material

JPG File (file137-2.jpg)

Download
11.78 KB

AVI File (file137-2.avi)

Download
19.05 MB

References

[1]

Da Pam 738--751, 1992. Functional Users Manual for The Army Maintenance Management System - Aviation (TAMMS-A). Washington D.C: U.S. Army.

[2]

Blaskó, G. and Feiner, S. 2004. An interaction system for watch computers using tactile guidance and bidirectional segmented strokes. Proc. ISWC 2004. 120--123.

Digital Library

[3]

Bleser, G. and Stricker, D. 2008. Advanced tracking through efficient image processing and visual-inertial sensor fusion. Proc. IEEE Virtual Reality Conf. (VR '08), 137--144.

[4]

Brooks, F. P., Ouh-Young, M., Batter, J. J. and Kilpatrick, P. J. 1990. Project GROPE-Haptic displays for scientific visualization. Proc. 17th Annual Conf. on Comp. Graphics and Interactive Techniques, 177--185.

Digital Library

[5]

Buxton, W., Hill, R. and Rowley, P. 1985. Issues and techniques in touch-sensitive tablet input, Proc. 12^th Annual Conf. on Comp. Graphics and Interactive Techniques, 215--224.

Digital Library

[6]

Conner, B. D., Snibbe, S. S., Herndon, K. P., Robbins, D. C., Zeleznik, R. C. and Dam, A. V. 1992. Three-Dimensional Widgets. Proc. 1992 Symp. on Interactive 3D Graphics, 183--188.

Digital Library

[7]

Fiala, M. L. 2005. ARTag, a fiducial marker system using digital techniques. Proc. 2005 IEEE Computer Society Conf. on Comp. Vision and Pattern Recognition (CVPR'05) - Volume 2, 590--596.

Digital Library

[8]

Fishkin, K. P. 2004. A taxonomy for and analysis of tangible interfaces, Personal Ubiquitous Computing 8, 5, 347--358.

Digital Library

[9]

Gibson, J. 1986. The Ecological Approach to Visual Perception. Hillsdale, N.J.: Lawrence Erlbaum Associates.

[10]

Hinckley, K., Pausch, R., Goble, J. C. and Kassell, N. F. 1994. Passive real-world interface props for neurosurgical visualization. Proc. SIGCHI Conf. on Human Factors in Computing Systems, 452--458.

Digital Library

[11]

Insko, B. E., Meehan, M. J., Whitton, M. C. and Frederick P. Brooks, J. 2001. Passive haptics significantly enhances virtual environments. Technical Report 0-493-17286-6. The University of North Carolina at Chapel Hill.

Digital Library

[12]

Ishii, H. and Ullmer, B. 1997. Tangible Bits: Towards seamless interfaces between people, bits and atoms. Proc. SIGCHI Conf. on Human Factors in Comp. Sys. 234--241.

Digital Library

[13]

Kjeldsen, R. and Kender, J. 1996. Finding skin in color images. Proc. 2nd International Conf. on Automatic Face and Gesture Recognition (FG '96), 312.

Digital Library

[14]

Klein, G. and Murray, D. 2007. Parallel tracking and mapping for small AR workspaces. Proc. International Symp. on Mixed and Augmented Reality (ISMAR'07).

Digital Library

[15]

Lindeman, R. W., Sibert, J. L. and Hahn, J. K. 1999. Hand-held windows: towards effective 2D interaction in immersive virtual environments. Proc. IEEE Virtual Reality Conference, 205--212.

Digital Library

[16]

Murray-Smith, R., Williamson, J., Hughes, S. and Quaade, T. 2008. Stane: Synthesized surfaces for tactile input. Proc. of the Twenty-sixth annual SIGCHI Conf. on Human Factors in Comp. Systems, 1299--1302.

Digital Library

[17]

Norman, D. 1988. The Psychology of Everyday Things. New York: Basic Books.

[18]

Roeber, H., Bacus, J. and Tomasi, C. 2003. Typing in thin air: The Canesta projection keyboard - a new method of interaction with electronic devices. CHI '03 Extended Abstracts on Human Factors in Computing Systems, 712--713.

Digital Library

[19]

3DV Systems, 2008. http://www.3dvsystems.com

[20]

Szalavari, Z. and Gervautz, M. 1997. The personal interaction panel - a two-handed interface for augmented reality Computer Graphics Forum 16, 3, 335--346.

[21]

Tomasi, C., Rafii, A. and Torunoglu, I. 2003. Full-size projection keyboard for handheld devices, Communications of the ACM 46, 7, 70--75.

Digital Library

[22]

Weimer, D. and Ganapathy, S. K. 1989. A synthetic visual environment with hand gesturing and voice input. Proc. SIGCHI Conf. on Human Factors in Comp. Systems, 235--240.

Digital Library

Cited By

Greenslade MClark ALukosch S(2024)Becoming Q: Using Design Workshops to Explore Everyday Objects as Interaction Devices for an Augmented Reality Spy GameProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3682096(1-11)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3677386.3682096
Dogan MGonzalez EAhuja KDu RColaço ALee JGonzalez-Franco MKim D(2024)Augmented Object Intelligence with XR-ObjectsProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676379(1-15)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676379
Sharma AIvanov ALai FGrossman TSantosa S(2024)GraspUI: Seamlessly Integrating Object-Centric Gestures within the Seven Phases of GraspingProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661551(1275-1289)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661551
Show More Cited By

Index Terms

Opportunistic controls: leveraging natural affordances as tangible user interfaces for augmented reality
1. Computing methodologies
  1. Computer graphics
    1. Graphics systems and interfaces
      1. Virtual reality
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Graphics input devices
      2. Touch screens
    2. Interaction paradigms
      1. Mixed / augmented reality
      2. Virtual reality

Recommendations

Head-mounted display with mid-air tactile feedback
VRST '15: Proceedings of the 21st ACM Symposium on Virtual Reality Software and Technology

Virtual and physical worlds are merging. Currently users of head-mounted displays cannot have unobtrusive tactile feedback while touching virtual objects. We present a mid-air tactile feedback system for head-mounted displays. Our prototype uses the ...
Flyables: Exploring 3D Interaction Spaces for Levitating Tangibles
TEI '18: Proceedings of the Twelfth International Conference on Tangible, Embedded, and Embodied Interaction

Recent advances in technology and miniaturization allow the building of self-levitating tangible interfaces. This includes flying tangibles, which extend the mid-air interaction space from 2D to 3D. While a number of theoretical concepts about ...
3D Tabletop AR: A Comparison of Mid-Air, Touch and Touch+Mid-Air Interaction
AVI '20: Proceedings of the 2020 International Conference on Advanced Visual Interfaces

This paper contributes a first comparative study of three techniques for selecting 3D objects anchored to the table in tabletop Augmented Reality (AR). The impetus for this study is that touch interaction makes more sense when the targeted objects are ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

VRST '08: Proceedings of the 2008 ACM symposium on Virtual reality software and technology

October 2008

288 pages

ISBN:9781595939517

DOI:10.1145/1450579

Conference Chairs:
Steven Feiner
Columbia University
,
Daniel Thalmann
Ecole Polytechnique Fédérale de Lausanne
,
Pascal Guitton
Université de Bordeaux
,
Program Chairs:
Bernd Fröhlich
Bauhaus-Univ. Weimar
,
Ernst Kruijff
TU Graz
,
Martin Hachet
INRIA

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 October 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

VRST08

Sponsor:

VRST08: The ACM Symposium on Virtual Reality Software and Technology

October 27 - 29, 2008

Bordeaux, France

Acceptance Rates

VRST '08 Paper Acceptance Rate 12 of 68 submissions, 18%;

Overall Acceptance Rate 66 of 254 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

66
Total Citations
View Citations
1,500
Total Downloads

Downloads (Last 12 months)109
Downloads (Last 6 weeks)16

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Greenslade MClark ALukosch S(2024)Becoming Q: Using Design Workshops to Explore Everyday Objects as Interaction Devices for an Augmented Reality Spy GameProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3682096(1-11)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3677386.3682096
Dogan MGonzalez EAhuja KDu RColaço ALee JGonzalez-Franco MKim D(2024)Augmented Object Intelligence with XR-ObjectsProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676379(1-15)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676379
Sharma AIvanov ALai FGrossman TSantosa S(2024)GraspUI: Seamlessly Integrating Object-Centric Gestures within the Seven Phases of GraspingProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661551(1275-1289)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661551
Dupré CAppert CRey SSaidi HPietriga E(2024)TriPad: Touch Input in AR on Ordinary Surfaces with Hand Tracking OnlyProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642323(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642323
Zhu QWang ZZeng WTong WLin WMa X(2024)Make Interaction Situated: Designing User Acceptable Interaction for Situated Visualization in Public EnvironmentsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642049(1-21)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642049
Boulanger PWang TRahmani Hanzaki M(2023)Improving Medical Simulation Using Virtual Reality Augmented by Haptic ProxyModern Development and Challenges in Virtual Reality10.5772/intechopen.108330Online publication date: 18-Oct-2023
https://doi.org/10.5772/intechopen.108330
Greenslade MClark ALukosch S(2023)Using Everyday Objects as Props for Virtual Objects in First Person Augmented Reality Games: An Elicitation StudyProceedings of the ACM on Human-Computer Interaction10.1145/36110527:CHI PLAY(856-875)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3611052
Jain RShi JDuan RZhu ZQian XRamani K(2023)Ubi-TOUCH: Ubiquitous Tangible Object Utilization through Consistent Hand-object interaction in Augmented RealityProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606793(1-18)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586183.3606793
Chan VFang CHung YHuang JCheng L(2023)Chandelier: Interaction Design With Surrounding Mid-Air Tangible InterfaceAdjunct Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586182.3616695(1-3)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586182.3616695
Tschakert HLang FWieland MSchmidt AMachulla T(2023)A Dataset and Machine Learning Approach to Classify and Augment Interface Elements of Household Appliances to Support People with Visual ImpairmentProceedings of the 28th International Conference on Intelligent User Interfaces10.1145/3581641.3584038(77-90)Online publication date: 27-Mar-2023
https://dl.acm.org/doi/10.1145/3581641.3584038
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents