research-article

Enabling beyond-surface interactions for interactive surface with an invisible projection

Authors:

Yi-Ping HungAuthors Info & Claims

UIST '10: Proceedings of the 23nd annual ACM symposium on User interface software and technology

Pages 263 - 272

https://doi.org/10.1145/1866029.1866072

Published: 03 October 2010 Publication History

Abstract

This paper presents a programmable infrared (IR) technique that utilizes invisible, programmable markers to support interaction beyond the surface of a diffused-illumination (DI) multi-touch system. We combine an IR projector and a standard color projector to simultaneously project visible content and invisible markers. Mobile devices outfitted with IR cameras can compute their 3D positions based on the markers perceived. Markers are selectively turned off to support multi-touch and direct on-surface tangible input. The proposed techniques enable a collaborative multi-display multi-touch tabletop system. We also present three interactive tools: i-m-View, i-m-Lamp, and i-m-Flashlight, which consist of a mobile tablet and projectors that users can freely interact with beyond the main display surface. Early user feedback shows that these interactive devices, combined with a large interactive display, allow more intuitive navigation and are reportedly enjoyable to use.

Supplementary Material

JPG File (109i-chan.jpg)

Download
9.96 KB

MOV File (109i-chan.mov)

Download
53.28 MB

References

[1]

}}Michael Aron, Gilles Simon, and Marie-Odile Berger. Handling uncertain sensor data in vision-based camera tracking. In Proc. ISMAR '04, 58--67.

Digital Library

[2]

}}Patrick Baudisch, Nathaniel Good, Victoria Bellotti, and Pamela Schraedley. Keeping things in context: comparative evaluation of focus plus context screens, overviews, and zooming. In Proc. CHI '02, 259--266.

Digital Library

[3]

}}Hrvoje Benko, Edward W. Ishak, and Steven Feiner. Collaborative mixed reality visualization of an archaeological excavation. In ISMAR '04, 132--140.

Digital Library

[4]

}}Kar Wee Chia, Adrian David Cheok, and Simon J. D. Prince. Online 6 dof augmented reality registration from natural features. In Proc. ISMAR '02, 305--313.

Digital Library

[5]

}}Daniel Cotting, Martin Naef, Markus Gross, and Henry Fuchs. Embedding imperceptible patterns into projected images for simultaneous acquisition and display. In Proc. ISMAR '04, 100--109.

Digital Library

[6]

}}Paul Dietz and Darren Leigh. Diamondtouch: multi-user touch technology. In Proc. UIST '01, 219--226.

Digital Library

[7]

}}Mark Fiala. Automatic projector calibration using self-identifying patterns. In Workshops of CVPR '05. 113--113.

Digital Library

[8]

}}Anselm Grundhöfer, Manja Seeger, Ferry Hantsch, and Oliver Bimber. Dynamic adaptation of projected imperceptible codes. In Proc. ISMAR '07, 1--10.

Digital Library

[9]

}}Anselm Grundhöfer, Manja Seeger, Ferry Hantsch, and Oliver Bimber. Dynamic adaptation of projected imperceptible codes. In ISMAR '07, 1--10.

Digital Library

[10]

}}Jefferson Y. Han. Low-cost multi-touch sensing through frustrated total internal reflection. In Proc. UIST '05, 115--118.

Digital Library

[11]

}}Otmar Hilliges, Shahram Izadi, Andrew D. Wilson, Steve Hodges, Armando Garcia-Mendoza, and Andreas Butz. Interactions in the air: adding further depth to interactive tabletops. In Proc. UIST '09, 139--148.

Digital Library

[12]

}}Chuan-Heng Hsiao, Li-Wei Chan, Ting-Ting Hu, Mon-Chu Chen, Jane Hsu, and Yi-Ping Hung. To move or not to move: a comparison between steerable versus fixed focus region paradigms in multi-resolution tabletop display systems. In Proc. CHI '09, 153--162.

Digital Library

[13]

}}Ting-Ting Hu, Yi-Wei Chia, Li-Wei Chan, Yi-Ping Hung, and Jane Hsu. i-m-top: An interactive multi-resolution tabletop system accommodating to multi-resolution human vision. In Proc. Tabletop '08, 177--180.

[14]

}}Shahram Izadi, Steve Hodges, Stuart Taylor, Dan Rosenfeld, Nicolas Villar, Alex Butler, and Jonathan Westhues. Going beyond the display: a surface technology with an electronically switchable diffuser. In Proc. UIST '08, 269--278.

Digital Library

[15]

}}Yasuaki Kakehi and Takeshi Naemura. Ulteriorscape: optical superimposing on view-dependent tabletop display and its applications. In SIGGRAPH '08: ACM SIGGRAPH 2008 new tech demos.

Digital Library

[16]

}}Rudolph Emil Kalman. A new approach to linear filtering and prediction problems. Transactions of the ASME-Journal of Basic Engineering, 82(Series D), 35--45.

[17]

}}Johnny Lee, Scott Hudson, and Pau Dietz. Hybrid infrared and visible light projection for location tracking. In Proc. UIST '07, 57--60.

Digital Library

[18]

}}Rubin G. Pelli D. Legge, G. and M. Schleske. Psychophyics of reading ii. low vision. Vision Research, 25, 2:253--266, 1985.

[19]

}}Alex Olwal. Lightsense: enabling spatially aware handheld interaction devices. In Proc. ISMAR '06, 119--122.

Digital Library

[20]

}}Alex Olwal and Steven Feiner. Spatially aware handhelds for high-precision tangible interaction with large displays. In Proc. TEI '09, 181--188.

Digital Library

[21]

}}Alex Olwal and Anders Henrysson. Lumar: A hybrid spatial display system for 2d and 3d handheld augmented reality. International Conference on Artificial Reality and Telexistence, 0, 63--70.

Digital Library

[22]

}}Johan Sanneblad and Lars Erik Holmquist. Ubiquitous graphics: combining hand-held and wall-size displays to interact with large images. In Proc. AVI '06, 373--377.

Digital Library

[23]

}}Garth B. D. Shoemaker and Kori M. Inkpen. Single display privacyware: augmenting public displays with private information. In Proc. CHI '01, 522--529.

Digital Library

[24]

}}Gilles Simon and Marie-Odile Berger. Reconstructing while registering: A novel approach for markerless augmented reality. In Proc. ISMAR '02, 285--285.

Digital Library

[25]

}}Hyunyoung Song, Tovi Grossman, George Fitzmaurice, François Guimbretiere, Azam Khan, Ramtin Attar, and Gordon Kurtenbach. Penlight: combining a mobile projector and a digital pen for dynamic visual overlay. In Proc. CHI '09, 143--152.

Digital Library

[26]

}}Hyunyoung Song, Francois Guimbretiere, Tovi Grossman, and George Fitzmaurice. Mouselight: bimanual interactions on digital paper using a pen and a spatially-aware mobile projector. In Proc. CHI '10, 2451--2460.

Digital Library

[27]

}}Michael Tsang, George W. Fitzmaurice, Gordon Kurtenbach, Azam Khan, and Bill Buxton. Boom chameleon: simultaneous capture of 3d viewpoint, voice and gesture annotations on a spatially-aware display. In Proc. UIST '02, 111--120.

Digital Library

[28]

}}Brygg Ullmer and Hiroshi Ishii. The metadesk: models and prototypes for tangible user interfaces. In Proc. UIST '97, 223--232.

Digital Library

[29]

}}Andrew D. Wilson. Playanywhere: a compactc interactive tabletop projection-vision system. In Proc. UIST '05, 83--92.

Digital Library

[30]

}}W. S. Yerazunis and M. Carbone. Privacy-enhanced displays by time-masking images. In Proc. OzCHI '01.

Cited By

Nasirinejad MReilly D(2024)mF+CInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2023.103170182:COnline publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.ijhcs.2023.103170
Shari AIbrahim SSofi INoordin MShari AFadzil M(2021)The Usability of Mobile Application for Interior Design via Augmented Reality2021 6th IEEE International Conference on Recent Advances and Innovations in Engineering (ICRAIE)10.1109/ICRAIE52900.2021.9703984(1-5)Online publication date: 1-Dec-2021
https://doi.org/10.1109/ICRAIE52900.2021.9703984
Brudy FHolz CRädle RWu CHouben SKlokmose CMarquardt NBrewster SFitzpatrick GCox AKostakos V(2019)Cross-Device TaxonomyProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300792(1-28)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300792
Show More Cited By

Index Terms

Enabling beyond-surface interactions for interactive surface with an invisible projection
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Graphical user interfaces

Recommendations

Beyond the surface: 3D interactions for tabletop systems
SIGGRAPH '10: ACM SIGGRAPH 2010 Emerging Technologies

Current tabletop systems are designed to sense 2D interactions taken place on the tabletop surface, such as finger touches and tangible objects. The ability to interact above the tabletop surface makes it possible to support 3D interactions. For example,...
Pointable: an in-air pointing technique to manipulate out-of-reach targets on tabletops
ITS '11: Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces

Selecting and moving digital content on interactive tabletops often involves accessing the workspace beyond arm's reach. We present Pointable, an in-air, bimanual perspective-based interaction technique that augments touch input on a tabletop for ...
Understanding Multi-touch Manipulation for Surface Computing
INTERACT '09: Proceedings of the 12th IFIP TC 13 International Conference on Human-Computer Interaction: Part II

Two-handed, multi-touch surface computing provides a scope for interactions that are closer analogues to physical interactions than classical windowed interfaces. The design of natural and intuitive gestures is a difficult problem as we do not know how ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '10: Proceedings of the 23nd annual ACM symposium on User interface software and technology

October 2010

476 pages

ISBN:9781450302715

DOI:10.1145/1866029

General Chair:
Ken Perlin
New York University
,
Program Chairs:
Mary Czerwinski
Microsoft Research
,
Rob Miller
MIT CSAIL

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 October 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

UIST '10

Sponsor:

UIST '10: The 23nd Annual ACM Symposium on User Interface Software and Technology

October 3 - 6, 2010

New York, New York, USA

Acceptance Rates

Overall Acceptance Rate 842 of 3,967 submissions, 21%

Upcoming Conference

UIST '24

Sponsor:
sigchi
sigchi

The 37th Annual ACM Symposium on User Interface Software and Technology

October 13 - 16, 2024

Pittsburgh , PA , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

29
Total Citations
View Citations
1,554
Total Downloads

Downloads (Last 12 months)26
Downloads (Last 6 weeks)2

Reflects downloads up to 29 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Nasirinejad MReilly D(2024)mF+CInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2023.103170182:COnline publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.ijhcs.2023.103170
Shari AIbrahim SSofi INoordin MShari AFadzil M(2021)The Usability of Mobile Application for Interior Design via Augmented Reality2021 6th IEEE International Conference on Recent Advances and Innovations in Engineering (ICRAIE)10.1109/ICRAIE52900.2021.9703984(1-5)Online publication date: 1-Dec-2021
https://doi.org/10.1109/ICRAIE52900.2021.9703984
Brudy FHolz CRädle RWu CHouben SKlokmose CMarquardt NBrewster SFitzpatrick GCox AKostakos V(2019)Cross-Device TaxonomyProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300792(1-28)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300792
Kamei IHiraki TFukushima SNaemura T(2019)PILC Projector: RGB-IR Projector for Pixel-level Infrared Light Communication2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)10.1109/VR.2019.8797714(1004-1005)Online publication date: Mar-2019
https://doi.org/10.1109/VR.2019.8797714
Kamei IHiraki TFukushima SNaemura T(2019)PILC Projector: Image Projection With Pixel-Level Infrared Light CommunicationIEEE Access10.1109/ACCESS.2019.29500687(160768-160778)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2019.2950068
Matsumoto AAbe SHiraki TFukushima SNaemura T(2019)Imperceptible AR Markers for Near-Screen Mobile InteractionIEEE Access10.1109/ACCESS.2019.29215807(79927-79933)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2019.2921580
Siriborvornratanakul T(2018)Enhancing User Experiences of Mobile-Based Augmented Reality via Spatial Augmented RealityAdvances in Multimedia10.1155/2018/81947262018Online publication date: 1-Jan-2018
https://dl.acm.org/doi/10.1155/2018/8194726
Sato TKoike HRekimoto JIgarashi TWobbrock JAvrahami D(2016)MlioLightAdjunct Proceedings of the 29th Annual ACM Symposium on User Interface Software and Technology10.1145/2984751.2985719(107-108)Online publication date: 16-Oct-2016
https://dl.acm.org/doi/10.1145/2984751.2985719
Jo KGupta MNayar S(2016)DisCoACM Transactions on Graphics10.1145/289681835:5(1-13)Online publication date: 28-Jul-2016
https://dl.acm.org/doi/10.1145/2896818
Karray MKrid MSellami D(2016)A survey of AR systems and a case study of virtual keyboard based camera projector system2016 International Image Processing, Applications and Systems (IPAS)10.1109/IPAS.2016.7880150(1-8)Online publication date: Nov-2016
https://doi.org/10.1109/IPAS.2016.7880150
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents