research-article

Smartcasting: a discount 3D interaction technique for public displays

Authors:

Krzysztof Pietroszek,

Anastasia Kuzminykh,

James R. Wallace,

Edward LankAuthors Info & Claims

OzCHI '14: Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: the Future of Design

Pages 119 - 128

https://doi.org/10.1145/2686612.2686629

Published: 02 December 2014 Publication History

Abstract

We introduce and formally evaluate smartcasting: a smartphone-based Ray Casting implementation for 3D environments presented on large, public, autostereoscopic displays. By utilizing a smartphone as an input device, smartcasting enables "walk up and use" interaction with large displays, without the need for expensive tracking systems or specialized pointing devices. Through an empirical validation we show that the performance and precision of smartcasting is comparable to a Wiimote-based raycasting implementation, without requiring specialized hardware or high-precision cameras to enable user interaction.

References

[1]

Azad, A., Ruiz, J., Vogel, D., Hancock, M., and Lank, E. Territoriality and behaviour on and around large vertical publicly-shared displays. In Proc. DIS '12, ACM (2012), 468.

Digital Library

[2]

Ballagas, R., Borchers, J., Rohs, M., and Sheridan, J. G. The smart phone: A ubiquitous input device. IEEE Pervasive Computing 5, IEEE (2006), 70--71.

Digital Library

[3]

Bishop, I. D. and Karadaglis, C. Linking modelling and visualisation for natural resources management. Environment and Planning B: Planning and Design 24, 1997, 345--358.

[4]

Bowman, D., Wingrave, C., Campbell, J., and Ly, V. Using pinch gloves (tm) for both natural and abstract interaction techniques in virtual environments. HCI International, (2001), 629--633.

[5]

Bowman, D. A. and Hodges, L. F. An evaluation of techniques for grabbing and manipulating remote objects in immersive virtual environments. In Proc. SI3D '97, ACM (1997), 35--ff.

Digital Library

[6]

Brignull, H. and Rogers, Y. Enticing people to interact with large public displays in public spaces. Proceedings of INTERACT, ACM (2003).

[7]

Brynskov, M., Dalsgaard, P., Ebsen, T., Fritsch, J., Halskov, K., and Nielsen, R. Staging urban interactions with media facades. Lecture Notes in Computer Science, ACM (2009), 154--167.

Digital Library

[8]

Burns, M. and Finkelstein, A. Adaptive cutaways for comprehensible rendering of polygonal scenes. ACM Trans. Graph. 27, ACM(2008), 1.

Digital Library

[9]

Buxton, W. A. S. A Three-State Model of Graphical Input. Hum.-Comp. Interact. 90, (1990), 449--456.

Digital Library

[10]

Cao, X. and Balakrishnan, R. VisionWand: Interaction Techniques for Large Displays Using a Passive Wand Tracked in 3D. ACM Trans. Graph. 23, 3 (2004), 729.

Digital Library

[11]

Cipiloglu, Z., Bulbul, A., and Capin, T. A framework for enhancing depth perception in computer graphics. In Proc. APGV '10, ACM (2010), 141.

Digital Library

[12]

Cook, N. D., Yutsudo, A., Fujimoto, N., and Murata, M. Factors contributing to depth perception: behavioral studies on the reverse perspective illusion. Spatial vision 21, (2008), 397--405.

[13]

Davidson, P. L. and Han, J. Y. Synthesis and control on large scale multi-touch sensing displays. In Proc. ICNIME'06, (2006), 216--219.

Digital Library

[14]

Diverdi, S., Rakkolainen, I., Olwal, A., and Barbara, S. A novel walk-through 3D display. Electronic ... 6055, (2006), 428--437.

[15]

Elmqvist, N. and Tsigas, P. View projection animation for occlusion reduction. In Proc. AVI '06, ACM (2006), 471--475.

Digital Library

[16]

Elmqvist, N. and Tsigas, P. A taxonomy of 3D occlusion management for visualization. IEEE Transactions on Visualization and Computer Graphics 14, 5 (2008), 1095--109.

Digital Library

[17]

Elmqvist, N. BalloonProbe: Reducing Occlusion in 3D using Interactive Space Distortion. In Proc. VRST'05, ACM (2005), 134--137.

Digital Library

[18]

Forsberg, A., Herndon, K., and Zeleznik, R. Aperture Based Selection for Immersive Virtual Environments. In Proc UIST'96, ACM (1996), 95--96.

Digital Library

[19]

Grossman, T. and Balakrishnan, R. The design and evaluation of selection techniques for 3D volumetric displays. In. Proc. UIST'06, ACM (2006), 3.

Digital Library

[20]

Grossman, T., Wigdor, D., and Balakrishnan, R. Multi-finger Gestural Interaction with 3D Volumetric Displays. In Proc. UIST'04, ACM (2004), 61--70.

Digital Library

[21]

Ha, S., Bai, Y., and Liu, C. K. Human motion reconstruction from force sensors. In Proc. SCA '11, ACM (2011), 129.

Digital Library

[22]

Hachet, M., Bossavit, B., Cohe, A., and de la Riviere, J.-B. Toucheo: multitouch and stereo combined in a seamless workspace. In Proc. UIST'11, (2011), 587--592.

Digital Library

[23]

Hachet, M., De La Rivière, J.-B., Laviole, J., Cohé, A., and Cursan, S. Touch-Based Interfaces for Interacting with 3D Content in Public Exhibitions. Computer Graphics and Applications, IEEE 33, 2 (2013), 80--85.

Digital Library

[24]

Hart, S. G. and Staveland, L. E. Development of NASATLX (Task Load Index): Results of empirical and theoretical research. Advances in psychology 52, (1988), 139--183.

[25]

Held, R. T. and Hui, T. T. A Guide to Stereoscopic 3D Displays in Medicine. Academic Radiology 18, 2011, 1035--1048.

[26]

Hornecker, E. "I don't understand it either, but it is cool" - Visitor interactions with a multi-touch table in a museum. In Proc. TABLETOP 2008, IEEE (2008), 113--120.

[27]

Huang, E. M., Koster, A., and Borchers, J. Overcoming assumptions and uncovering practices: When does the public really look at public displays? Lecture Notes in Computer Science, Springer (2008), 228--243.

Digital Library

[28]

Huang, E. M., Mynatt, E. D., Russell, D. M., and Sue, A. E. Secrets to success and fatal flaws: The design of large-display groupware. Computer Graphics and Applications, IEEE 26, 1 (2006), 37--45.

Digital Library

[29]

Jeon, S., Hwang, J., Kim, G. J., and Billinghurst, M. Interaction with large ubiquitous displays using camera-equipped mobile phones. Personal and Ubiquitous Computing 14, (2010), 83--94.

Digital Library

[30]

Kytö, M., Mäkinen, A., Häkkinen, J., and Oittinen, P. Improving relative depth judgments in augmented reality with auxiliary augmentations. ACM Transactions on Applied Perception 10, 1 (2013), 1--21.

Digital Library

[31]

Lee, C. M. G., Travis, A. R. L., and Lin, R. Flat-panel autostereoscopic 3D display. IET Optoelectronics 2, 2008, 24.

[32]

Liang, J. and Green, M. JDCAD: A highly interactive 3 D modeling system. Computers & Graphics 18, (1994), 449--506.

[33]

Liao, H., Dohi, T., and Nomura, K. Autostereoscopic 3D Display with Long Visualization Depth Using Referential Viewing Area-Based Integral Photography. IEEE Transactions on Visualization and Computer Graphics 17, IEEE (2011), 1690--1701.

Digital Library

[34]

MacKenzie, I. S. Fitts' law as a research and design tool in human-computer interaction. Human-Computer Interaction 7, (1992), 91--139.

Digital Library

[35]

Messner, J. and Yerrapathruni, S. Using virtual reality to improve construction engineering education. Engineering Education, (2003).

[36]

Norman, D. A. The design of everyday things. Basic books, 2002.

Digital Library

[37]

Pietroszek, K. and Lank, E. Clicking Blindly: Using Spatial Correspondence to Select Targets in Multi-device Environments. In Proc. MobileHCI'12, ACM (2012), 331--334.

Digital Library

[38]

Reeves, S., Benford, S., O'Malley, C., and Fraser, M. Designing the Spectator Experience. In Proc. CHI'05, ACM (2005), 741--750.

Digital Library

[39]

Rogers, B. and Graham, M. Motion parallax as an independent cue for depth perception. Perception 8, (1979), 125--134.

[40]

Shoemaker, G., Tang, A., and Booth, K. S. Shadow Reaching: A New Perspective on Interaction for Large Wall Displays. In. Proc. UIST'07, (2007), 53--56.

Digital Library

[41]

Siek, K. A., Rogers, Y., and Connelly, K. H. Fat finger worries: How older and younger users physically interact with PDAs. Lecture Notes in Computer Science, Springer (2005), 267--280.

Digital Library

[42]

Steed, A. and Parker, C. 3D Selection Strategies for Head Tracked and Non-Head Tracked Operation of Spatially Immersive Displays. 8th International Immersive Projection Technology Workshop, (2004), 13--14.

[43]

Steinicke, F., Hinrichs, K., Schöning, J., and Krüger, A. Multi-Touching 3D Data: Towards Direct Interaction in Stereoscopic Display, Environments coupled with Mobile Devices. Advanced Visual Interfaces Workshop AVI2008, (2008), 1--4.

[44]

Szalavári, Z. and Gervautz, M. The Personal Interaction Panel - a Two-Handed Interface for Augmented Reality. Computer Graphics Forum 16, 3 (2008), C335--C346.

[45]

Travis, A. R. Autostereoscopic 3-D display. Applied optics 29, (1990), 4341--4342.

[46]

Vanacken, L., Grossman, T., and Coninx, K. Exploring the Effects of Environment Density and Target Visibility on Object Selection in 3D Virtual Environments. 2007 IEEE Symposium on 3D User Interfaces, IEEE (2007), 117--124.

[47]

Vanacken, L., Grossman, T., and Coninx, K. Multimodal selection techniques for dense and occluded 3D virtual environments. International Journal of Human-Computer Studies 67, 3 (2009), 237--255.

Digital Library

[48]

Vogel, D. and Balakrishnan, R. Distant freehand pointing and clicking on very large, high resolution displays. In Proc. UIST'05, ACM (2005), 33--42.

Digital Library

[49]

Wallace, J. R., Mandryk, R. L., and Inkpen, K. M. Comparing content and input redirection in MDEs. In Proc. CSCW '08, ACM (2008), 157.

Digital Library

[50]

Wingrave, C. A., Williamson, B., Varcholik, P. D., et al. The wiimote and beyond: Spatially convenient devices for 3D user interfaces. IEEE Computer Graphics and Applications 30, IEEE (2010), 71--85.

Digital Library

[51]

Wyss, H., Blach, R., and Bues, M. iSith-Intersection-based spatial interaction for two hands. In Proc. 3DUI'06, IEEE (2006), 59--61.

[52]

Zhai, S., Buxton, W., and Milgram, P. The 'Silk Cursor': Investigating Transparency for 3D Target Acquisition. In. Proc. CHI'04, ACM (1994), 459--464.

Digital Library

[53]

Zhai, S., Buxton, W., and Milgram, P. The partial-occlusion effect: utilizing semitransparency in 3D human-computer interaction. ACM Transactions on Computer-Human Interaction 3, 1996, 254--284.

Digital Library

Cited By

McDonald BZhang QNanzatov APeña-Castillo LMeruvia-Pastor O(2024)SmartVR Pointer: Using Smartphones and Gaze Orientation for Selection and Navigation in Virtual RealitySensors10.3390/s2416516824:16(5168)Online publication date: 10-Aug-2024
https://doi.org/10.3390/s24165168
Pietroszek K(2024)Raycasting in Virtual RealityEncyclopedia of Computer Graphics and Games10.1007/978-3-031-23161-2_180(1534-1536)Online publication date: 5-Jan-2024
https://doi.org/10.1007/978-3-031-23161-2_180
Pietroszek K(2024)Virtual Pointing Metaphor in Virtual RealityEncyclopedia of Computer Graphics and Games10.1007/978-3-031-23161-2_179(2001-2003)Online publication date: 5-Jan-2024
https://doi.org/10.1007/978-3-031-23161-2_179
Show More Cited By

Index Terms

Smartcasting: a discount 3D interaction technique for public displays
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Tiltcasting: 3D Interaction on Large Displays using a Mobile Device
UIST '15: Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology

We develop and formally evaluate a metaphor for smartphone interaction with 3D environments: Tiltcasting. Under the Tiltcasting metaphor, users interact within a rotatable 2D plane that is "cast" from their phone's interactive display into 3D space. ...
TMAR: extension of a tabletop interface using mobile augmented reality
Transactions on edutainment I

Recently, many researchers have worked on tabletop systems. One issue with tabletop interfaces is how to control the table without using conventional desktop input devices such as a keyboard or mouse. A second issue is allowing multiple users to ...
Impact of mobile tangible slider design and its reachability on pointing performance
IHM '16: Actes de la 28ième conference francophone sur l'Interaction Homme-Machine

Touch interaction is almost omnipresent on mobile devices and technological advances have made screens bigger. The users can then interact with more content displayed on screen but it is more difficult for the users to reach all the parts of the screen ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

OzCHI '14: Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: the Future of Design

December 2014

689 pages

ISBN:9781450306539

DOI:10.1145/2686612

Conference Chair:
Tuck Leong
University of Technology Sydney

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

U1: U1 PTY LTD
IDHuP: Interaction Design and Human Practice Lab
UTS-HCTDRS: The UTS Human Centred Technology Design Research Strength
CSIRO
QUT
HFESA: Human Factors and Ergonomics Society of Australia Inc.
University of Technology Sydney
IDF: The Interaction Design Foundation
CHISIG: Computer-Human Interaction Special Interest Group, Human Factors & Ergonomics Society of Australia

In-Cooperation

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 December 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Government of Canada (NSERC Discovery Grants Program, NCE-GRAND)
Ontario (ORF-RE Program)
Google Research (Faculty Fellowship Program)

Conference

OzCHI '14

Sponsor:

U1
IDHuP
UTS-HCTDRS
HFESA
IDF
CHISIG

OzCHI '14: the Future of Design

December 2 - 5, 2014

New South Wales, Sydney, Australia

Acceptance Rates

OzCHI '14 Paper Acceptance Rate 85 of 176 submissions, 48%;

Overall Acceptance Rate 362 of 729 submissions, 50%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

25
Total Citations
View Citations
411
Total Downloads

Downloads (Last 12 months)21
Downloads (Last 6 weeks)4

Reflects downloads up to 13 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

McDonald BZhang QNanzatov APeña-Castillo LMeruvia-Pastor O(2024)SmartVR Pointer: Using Smartphones and Gaze Orientation for Selection and Navigation in Virtual RealitySensors10.3390/s2416516824:16(5168)Online publication date: 10-Aug-2024
https://doi.org/10.3390/s24165168
Pietroszek K(2024)Raycasting in Virtual RealityEncyclopedia of Computer Graphics and Games10.1007/978-3-031-23161-2_180(1534-1536)Online publication date: 5-Jan-2024
https://doi.org/10.1007/978-3-031-23161-2_180
Pietroszek K(2024)Virtual Pointing Metaphor in Virtual RealityEncyclopedia of Computer Graphics and Games10.1007/978-3-031-23161-2_179(2001-2003)Online publication date: 5-Jan-2024
https://doi.org/10.1007/978-3-031-23161-2_179
Henderson JNeshati AMizobuchi SZhou WVogel DLank E(2023)Interaction Region Characteristics for Midair Barehand Targeting on a TelevisionExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585877(1-7)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585877
Babic TReiterer HHaller M(2022)Understanding and Creating Spatial Interactions with Distant Displays Enabled by Unmodified Off-The-Shelf SmartphonesMultimodal Technologies and Interaction10.3390/mti61000946:10(94)Online publication date: 19-Oct-2022
https://doi.org/10.3390/mti6100094
Li LFang ZJia WZhang F(2022)Hierarchical Pointing on Distant Displays with Smart DevicesInternational Journal of Human–Computer Interaction10.1080/10447318.2022.210855939:19(3859-3874)Online publication date: 6-Sep-2022
https://doi.org/10.1080/10447318.2022.2108559
Saidi HDubois ESerrano MKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)HoloBar: Rapid Command Execution for Head-Worn AR Exploiting Around the Field-of-View InteractionProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445255(1-17)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445255
Ruser HKirsh I(2021)“Point at It with Your Smartphone”: Assessing the Applicability of Orientation Sensing of Smartphones to Operate IoT DevicesHCI International 2021 - Late Breaking Papers: Multimodality, eXtended Reality, and Artificial Intelligence10.1007/978-3-030-90963-5_10(115-131)Online publication date: 11-Nov-2021
https://doi.org/10.1007/978-3-030-90963-5_10
Babic TPerteneder FReiterer HHaller MBernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)Simo: Interactions with Distant Displays by Smartphones with Simultaneous Face and World TrackingExtended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3334480.3382962(1-12)Online publication date: 25-Apr-2020
https://dl.acm.org/doi/10.1145/3334480.3382962
Chen YKatsuragawa KLank EBernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)Understanding Viewport- and World-based Pointing with Everyday Smart Devices in Immersive Augmented RealityProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376592(1-13)Online publication date: 21-Apr-2020
https://dl.acm.org/doi/10.1145/3313831.3376592
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents