research-article

SideBySide: ad-hoc multi-user interaction with handheld projectors

Authors:

Karl D.D. Willis,

Scott E. Hudson,

Moshe MahlerAuthors Info & Claims

UIST '11: Proceedings of the 24th annual ACM symposium on User interface software and technology

Pages 431 - 440

https://doi.org/10.1145/2047196.2047254

Published: 16 October 2011 Publication History

Abstract

We introduce SideBySide, a system designed for ad-hoc multi-user interaction with handheld projectors. SideBySide uses device-mounted cameras and hybrid visible/infrared light projectors to track multiple independent projected images in relation to one another. This is accomplished by projecting invisible fiducial markers in the near-infrared spectrum. Our system is completely self-contained and can be deployed as a handheld device without instrumentation of the environment. We present the design and implementation of our system including a hybrid handheld projector to project visible and infrared light, and techniques for tracking projected fiducial markers that move and overlap. We introduce a range of example applications that demonstrate the applicability of our system to real-world scenarios such as mobile content exchange, gaming, and education.

Supplementary Material

JPG File (fp206.jpg)

Download
4.55 KB

MP4 File (fp206.mp4)

Download
39.69 MB

References

[1]

Beardsley, P., van Baar, J., Raskar, R. and Forlines, C. Interaction Using a Handheld Projector. IEEE Computer Graphics and Applications 25, 1 (2005), 39--43.

Digital Library

[2]

Cao, X., Forlines, C. and Balakrishnan, R., Multi-user Interaction Using Handheld Projectors. In Proc. UIST '07, ACM (2007), 43--5.

Digital Library

[3]

Cao, X., Massimi, M. and Balakrishnan, R., Flashlight Jigsaw: An Exploratory Study of an Ad-hoc Multi-player Game on Public Displays. In Proc. CSCW '08, ACM (2008), 77--86.

Digital Library

[4]

Card, S.K., Moran, T.P. and Newell, A. The Model Human Processor: An Engineering Model of Human Performance: Vol. 2 Cognitive Processes and Performance. in Boff, K.R., Kaufman, L. and Thomas, J.P. eds. Handbook of Perception and Human Performance, 1986, 1--35.

[5]

Chan, L.-W., Wu, H.-T., Kao, H.-S., Ko, J.-C., Lin, H.-R., Chen, M.Y., Hsu, J. and Hung, Y.-P., Enabling Beyond-surface Interactions for Interactive Surface with an Invisible Projection. UIST '10, ACM (2010), 263--272.

Digital Library

[6]

Grundhöfer, A., Seeger, M., Hantsch, F. and Bimber, O., Dynamic Adaptation of Projected Imperceptible Codes. In Proc. ISMAR '07, IEEE (2007), 1--10.

Digital Library

[7]

Hosoi, K., Dao, V.N., Mori, A. and Sugimoto, M., CoGAME: Manipulation Using a Handheld Projector. In Proc. SIGGRAPH '07 Emerging Technologies, ACM (2007), 2.

Digital Library

[8]

Kane, S.K., Avrahami, D., Wobbrock, J.O., Harrison, B., Rea, A.D., Philipose, M. and LaMarca, A., Bonfire: A Nomadic System for Hybrid Laptop-Tabletop Interaction. In Proc. UIST '09, ACM (2009), 129--138.

Digital Library

[9]

Köhler, M., Patel, S.N., Summet, J.W., Stuntebeck, E.P. and Abowd, G.D., TrackSense: infrastructure free precise indoor positioning using projected patterns. In Proc. Pervasive '07, Springer-Verlag (2007), 334--350.

Digital Library

[10]

Lee, J., Hudson, S. and Dietz, P., Hybrid Infrared and Visible Light Projection for Location Tracking. In Proc. UIST '07, ACM (2007), 57--60.

Digital Library

[11]

Nakazato, Y., Kanbara, M. and Yokoya, N., Localization System for Large Indoor Environments using Invisible Markers. In Proc. VRST '08, ACM (2008), 295--296.

Digital Library

[12]

Ozuysal, M., Calonder, M., Lepetit, V. and Fua, P. Fast Keypoint Recognition Using Random Ferns. IEEE Transactions on Pattern Analysis and Machine Intelligence 32, 3 (2010), 448--461.

Digital Library

[13]

Park, H. and Park, J.-I., Invisible Marker Based Augmented Reality System. In Proc. VCIP '05, SPIE (2005), 501--508.

[14]

Pinhanez, C.S., The Everywhere Displays Projector: A Device to Create Ubiquitous Graphical Interfaces. In Proc. UbiComp '01, Springer-Verlag (2001), 315--331.

Digital Library

[15]

Rekimoto, J., Pick-and-Drop: A Direct Manipulation Technique for Multiple Computer Environments. In Proc. UIST '97, ACM (1997), 31--39.

Digital Library

[16]

Rekimoto, J. and Saitoh, M., Augmented Surfaces: A Spatially Continuous Work Space for Hybrid Computing Environments. In Proc. CHI '99, ACM (1999), 378--385.

Digital Library

[17]

Rukzio, E., Holleis, P. and Gellersen, H. Personal Projectors for Pervasive Computing. Pervasive Computing (2011).

Digital Library

[18]

Rukzio, E., Schöning, J., Rohs, M., Häkkila, J. and Dachselt, R. Ubiprojection 2010: Workshop on Personal Projection. 2010, http://eis.comp.lancs.ac.uk/workshops/ubiproject2010/.

[19]

Saito, S., Hiyama, A., Tanikawa, T. and Hirose, M., Indoor Marker-based Localization Using Coded Seamless Pattern for Interior Decoration. In Proc. VR '07, IEEE (2007), 67--74.

[20]

Shirai, Y., Matsushita, M. and Ohguro, T., HIEI Projector: Augmenting a Real Environment with Invisible Information. In Proc. WISS '03 (2003), 115--122.

[21]

Sugimoto, M., Miyahara, K., Inoue, H. and Tsunesada, Y., Hotaru: Intuitive Manipulation Techniques for Projected Displays of Mobile Devices. In Proc. INTERACT '05, Springer (2005), 57--68.

Digital Library

[22]

Wagner, D. and Schmalstieg, D., ARToolKitPlus for Pose Tracking on Mobile Devices. In Proc. CVWW '07 (2007), 139--146.

[23]

Weng, D., Huang, Y., Liu, Y. and Wang, Y., Study on an Indoor Tracking System with Infrared Projected Markers for Large-Area Applications. In Proc. VRCAI '09 ACM (2009), 239--245.

Digital Library

[24]

Wienss, C., Nikitin, I., Goebbels, G., Troche, K., Göbel, M., Nikitina, L. and Müller, S., Sceptre: an infrared laser tracking system for virtual environments. In Proc. VRST '06, ACM (2006), 45--50.

Digital Library

[25]

Willis, K.D.D. A Pre-History of Handheld Projector based Interaction. Journal of Personal and Ubiquitous Computing, Theme issue on Personal Projection (2011).

Digital Library

[26]

Willis, K.D.D., Poupyrev, I. and Shiratori, T., MotionBeam: A Metaphor for Character Interaction with Handheld Projectors. In Proc. CHI '11, ACM (2011), 1031--1040.

Digital Library

[27]

Wilson, A.D., PlayAnywhere: A Compact Interactive Tabletop Projection-Vision System. In Proc. UIST '05, ACM (2005), 83--92.

Digital Library

Cited By

Xu RSato TMiyafuji SKoike HBottoni PPanizzi E(2022)OmniLantern: Design and Implementations of a Portable and Coaxial Omnidirectional Projector-Camera SystemProceedings of the 2022 International Conference on Advanced Visual Interfaces10.1145/3531073.3531126(1-5)Online publication date: 6-Jun-2022
https://dl.acm.org/doi/10.1145/3531073.3531126
Dogan MTaka ALu MZhu YKumar AGupta AMueller S(2022)InfraredTags: Embedding Invisible AR Markers and Barcodes Using Low-Cost, Infrared-Based 3D Printing and Imaging ToolsProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501951(1-12)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3501951
Jacob JNóbrega R(2021)Collaborative Augmented Reality for Cultural Heritage, Tourist Sites and Museums: Sharing Visitors’ Experiences and InteractionsAugmented Reality in Tourism, Museums and Heritage10.1007/978-3-030-70198-7_2(27-47)Online publication date: 25-Apr-2021
https://doi.org/10.1007/978-3-030-70198-7_2
Show More Cited By

Index Terms

SideBySide: ad-hoc multi-user interaction with handheld projectors
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Motionbeam: a metaphor for character interaction with handheld projectors
CHI '11: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

We present the MotionBeam metaphor for character interaction with handheld projectors. Our work draws from the tradition of pre-cinema handheld projectors that use direct physical manipulation to control projected imagery. With our prototype system, ...
Multi-user interaction using handheld projectors
UIST '07: Proceedings of the 20th annual ACM symposium on User interface software and technology

Recent research on handheld projector interaction has expanded the display and interaction space of handheld devices by projecting information onto the physical environment around the user, but has mainly focused on single-user scenarios. We extend this ...
SideBySide: multi-user gestural interaction with handheld projectors
MobileHCI '12: Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services companion

SideBySide is a system designed for ad-hoc multi-user interaction with handheld projectors. SideBySide does not require instrumentation of the environment and can be used almost anywhere. This paper examines the diverse ways that children interact with ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '11: Proceedings of the 24th annual ACM symposium on User interface software and technology

October 2011

654 pages

ISBN:9781450307161

DOI:10.1145/2047196

General Chair:
Jeff Pierce
IBM Research, USA
,
Program Chairs:
Maneesh Agrawala
University of California, Berkeley, USA
,
Scott Klemmer
Stanford University, USA

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 October 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

UIST '11

Sponsor:

UIST '11: The 24th Annual ACM Symposium on User Interface Software and Technology

October 16 - 19, 2011

California, Santa Barbara, USA

Acceptance Rates

UIST '11 Paper Acceptance Rate 67 of 262 submissions, 26%;

Overall Acceptance Rate 842 of 3,967 submissions, 21%

Upcoming Conference

UIST '24

Sponsor:
sigchi
sigchi

The 37th Annual ACM Symposium on User Interface Software and Technology

October 13 - 16, 2024

Pittsburgh , PA , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

65
Total Citations
View Citations
1,605
Total Downloads

Downloads (Last 12 months)29
Downloads (Last 6 weeks)8

Reflects downloads up to 13 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Xu RSato TMiyafuji SKoike HBottoni PPanizzi E(2022)OmniLantern: Design and Implementations of a Portable and Coaxial Omnidirectional Projector-Camera SystemProceedings of the 2022 International Conference on Advanced Visual Interfaces10.1145/3531073.3531126(1-5)Online publication date: 6-Jun-2022
https://dl.acm.org/doi/10.1145/3531073.3531126
Dogan MTaka ALu MZhu YKumar AGupta AMueller S(2022)InfraredTags: Embedding Invisible AR Markers and Barcodes Using Low-Cost, Infrared-Based 3D Printing and Imaging ToolsProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501951(1-12)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3501951
Jacob JNóbrega R(2021)Collaborative Augmented Reality for Cultural Heritage, Tourist Sites and Museums: Sharing Visitors’ Experiences and InteractionsAugmented Reality in Tourism, Museums and Heritage10.1007/978-3-030-70198-7_2(27-47)Online publication date: 25-Apr-2021
https://doi.org/10.1007/978-3-030-70198-7_2
Ahuja KGoel MHarrison C(2020)BodySLAM: Opportunistic User Digitization in Multi-User AR/VR ExperiencesProceedings of the 2020 ACM Symposium on Spatial User Interaction10.1145/3385959.3418452(1-8)Online publication date: 31-Oct-2020
https://dl.acm.org/doi/10.1145/3385959.3418452
Tone DIwai DHiura SSato K(2020)FibAR: Embedding Optical Fibers in 3D Printed Objects for Active Markers in Dynamic Projection MappingIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2020.297344426:5(2030-2040)Online publication date: May-2020
https://doi.org/10.1109/TVCG.2020.2973444
Kagami SHashimoto K(2019)Animated Stickies: Fast Video Projection Mapping onto a Markerless Plane through a Direct Closed-Loop AlignmentIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2019.293224825:11(3094-3104)Online publication date: Nov-2019
https://doi.org/10.1109/TVCG.2019.2932248
Cortes GMarchand EBrincin GLécuyer A(2018)MoSART: Mobile Spatial Augmented Reality for 3D Interaction With Tangible ObjectsFrontiers in Robotics and AI10.3389/frobt.2018.000935Online publication date: 20-Aug-2018
https://doi.org/10.3389/frobt.2018.00093
Siriborvornratanakul T(2018)Enhancing User Experiences of Mobile-Based Augmented Reality via Spatial Augmented RealityAdvances in Multimedia10.1155/2018/81947262018Online publication date: 1-Jan-2018
https://dl.acm.org/doi/10.1155/2018/8194726
Malinverni LValero CSchaper MPares NGiannakos MJaccheri LDivitini M(2018)A conceptual framework to compare two paradigms of augmented and mixed reality experiencesProceedings of the 17th ACM Conference on Interaction Design and Children10.1145/3202185.3202750(7-18)Online publication date: 19-Jun-2018
https://dl.acm.org/doi/10.1145/3202185.3202750
Grundhöfer AIwai D(2018)Recent Advances in Projection Mapping Algorithms, Hardware and ApplicationsComputer Graphics Forum10.1111/cgf.1338737:2(653-675)Online publication date: 22-May-2018
https://doi.org/10.1111/cgf.13387
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents