research-article

Open access

WatchThru: Expanding Smartwatch Displays with Mid-air Visuals and Wrist-worn Augmented Reality

Authors:

Johannes Schöning,

Rainer MalakaAuthors Info & Claims

CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

Pages 716 - 721

https://doi.org/10.1145/3025453.3025852

Published: 02 May 2017 Publication History

Abstract

We introduce WatchThru, an interactive method for extended wrist-worn display on commercially-available smartwatches. To address the limited visual and interaction space, WatchThru expands the device into 3D through a transparent display. This enables novel interactions that leverage and extend smartwatch glanceability. We describe three novel interaction techniques, Pop-up Visuals, Second Perspective and Peek-through, and discuss how they can complement interaction on current devices. We also describe two types of prototypes that helped us to explore standalone interactions, as well as, proof-of-concept AR interfaces using our platform.

Supplementary Material

ZIP File (pn2989-file4.zip)

Download
3.14 KB

suppl.mov (pn2989-file3.mp4)

Supplemental video

Download
57.86 MB

suppl.mov (pn2989p.mp4)

Supplemental video

Download
1.64 MB

References

[1]

Amazon. 2017. Fire Phone. (3 January 2017). Retrieved January 3, 2017 from https://amazon.com/Fire-Phone/.

[2]

Eric A. Bier, Maureen C. Stone, Ken Pier, William Buxton, and Tony D. DeRose. 1993. Toolglass and Magic Lenses: The See-through Interface. In Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '93). ACM, New York, NY, USA, 73--80.

Digital Library

[3]

Jesse Burstyn, Paul Strohmeier, and Roel Vertegaal. 2015. DisplaySkin: Exploring Pose-Aware Displays on a Flexible Electrophoretic Wristband. In Proceedings of the 9th International Conference on Tangible, Embedded, and Embodied Interaction (TEI '15). ACM, New York, NY, USA, 165--172.

Digital Library

[4]

George W. Fitzmaurice. 1993. Situated Information Spaces and Spatially Aware Palmtop Computers. Commun. ACM 36, 7 (July 1993), 39--49.

Digital Library

[5]

Google. 2017. Tango. (3 January 2017). Retrieved January 3, 2017 from https://get.google.com/tango/.

[6]

Jens Grubert, Matthias Heinisch, Aaron Quigley, and Dieter Schmalstieg. 2015. MultiFi: Multi Fidelity Interaction with Displays On and Around the Body. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 3933--3942.

Digital Library

[7]

Shahram Izadi, David Kim, Otmar Hilliges, David Molyneaux, Richard Newcombe, Pushmeet Kohli, Jamie Shotton, Steve Hodges, Dustin Freeman, Andrew Davison, and Andrew Fitzgibbon. 2011. KinectFusion: Real-time 3D Reconstruction and Interaction Using a Moving Depth Camera. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST '11). ACM, New York, NY, USA, 559--568.

Digital Library

[8]

Georg Klein and David Murray. 2007. Parallel Tracking and Mapping for Small AR Workspaces. In Proceedings of the 6th IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR '07). IEEE, Washington, DC, USA, 225--234.

Digital Library

[9]

Gierad Laput, Robert Xiao, Xiang 'Anthony' Chen, Scott E. Hudson, and Chris Harrison. 2014. Skin Buttons: Cheap, Small, Low-powered and Clickable Fixed-icon Laser Projectors. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST '14). ACM, New York, NY, USA, 389--394.

Digital Library

[10]

Kent Lyons, David Nguyen, Daniel Ashbrook, and Sean White. 2012. Facet: A Multi-segment Wrist Worn System. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology (UIST '12). ACM, New York, NY, USA, 123--130.

Digital Library

[11]

Jon Moeller and Andruid Kerne. 2012. ZeroTouch: An Optical Multi-touch and Free-air Interaction Architecture. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, New York, NY, USA, 2165--2174.

Digital Library

[12]

Alessandro Mulloni, Hartmut Seichter, and Dieter Schmalstieg. 2011. Handheld Augmented Reality Indoor Navigation with Activity-based Instructions. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI '11). ACM, New York, NY, USA, 211--220.

Digital Library

[13]

Niantic. 2017. Pokémon GO. (3 January 2017). Retrieved January 3, 2017 from https://pokemongo.com/.

[14]

Alex Olwal, Christoffer Lindfors, Jonny Gustafsson, Torsten Kjellberg, and Lars Mattsson. 2005. ASTOR: An Autostereoscopic Optical See-through Augmented Reality System. In Proceedings of the 4th IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR '05). IEEE, Washington, DC, USA, 24--27.

Digital Library

[15]

Henning Pohl, Justyna Medrek, and Michael Rohs. 2016. ScatterWatch: Subtle Notifications via Indirect Illumination Scattered in the Skin. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '16). ACM, New York, NY, USA, 7--16.

Digital Library

[16]

Jun Rekimoto and Katashi Nagao. 1995. The World Through the Computer: Computer Augmented Interaction with Real World Environments. In Proceedings of the 8th Annual ACM Symposium on User Interface and Software Technology (UIST '95). ACM, New York, NY, USA, 29--36.

Digital Library

[17]

Dieter Schmalstieg and Daniel Wagner. 2007. Experiences with Handheld Augmented Reality. In Proceedings of the 6th IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR '07). IEEE, Washington, DC, USA, 3--18.

Digital Library

[18]

Teddy Seyed, Xing-Dong Yang, and Daniel Vogel. 2016. Doppio: A Reconfigurable Dual-Face Smartwatch for Tangible Interaction. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 4675--4686.

Digital Library

[19]

Damion Shelton, George Stetten, and Wilson Chang. 2002. Ultrasound Visualization with the Sonic Flashlight. In ACM SIGGRAPH 2002 Conference Abstracts and Applications (SIGGRAPH '02). ACM, New York, NY, USA, 82--82.

Digital Library

[20]

Georg Stetten, Vikram Chib, Daniel Hildebrand, and Jeannette Bursee. 2001. Real Time Tomographic Reflection: Phantoms for Calibration and Biopsy. In Proceedings of the IEEE/ACM International Symposium on Augmented Reality (ISAR '01). 11--19.

[21]

Ka-Ping Yee. 2003. Peephole Displays: Pen Interaction on Spatially Aware Handheld Computers. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '03). ACM, New York, NY, USA, 1--8.

Digital Library

Cited By

Cheng QTian XLiao FChen TChen XWang ZDong YLiu Y(2024)Enhancing Mobile Interaction: Practical Insights from Smartphone and Smartwatch IntegrationAdjunct Proceedings of the 26th International Conference on Mobile Human-Computer Interaction10.1145/3640471.3680451(1-9)Online publication date: 21-Sep-2024
https://dl.acm.org/doi/10.1145/3640471.3680451
Grubert JWitzani LOtte AGesslein TKranz MKristensson P(2024)Text Entry Performance and Situation Awareness of a Joint Optical See-Through Head-Mounted Display and Smartphone SystemIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.330931630:8(5830-5846)Online publication date: Aug-2024
https://doi.org/10.1109/TVCG.2023.3309316
Zeng XWang XGou ZChen YZhang T(2023)WebJump: AR-facilitated Distributed Display of Web PagesExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585669(1-6)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585669
Show More Cited By

Index Terms

WatchThru: Expanding Smartwatch Displays with Mid-air Visuals and Wrist-worn Augmented Reality

Recommendations

Natural group binding and cross-display object movement methods for wearable devices
MobileHCI '16: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services

As wearable devices become more popular, situations where there are multiple persons present with such devices will become commonplace. In these situations, wearable devices could support collaborative tasks and experiences between co-located persons ...
Investigating current techniques for opposite-hand smartwatch interaction
MobileHCI '17: Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services

The small display size of smartwatches creates a challenge for touch input, which is still the interaction technique of choice. Researchers and producers have started to investigate alternative interaction techniques. Apple and Samsung, for example, ...
Bezel-Tap gestures: quick activation of commands from sleep mode on tablets
CHI '13: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

We present Bezel-Tap Gestures, a novel family of interaction techniques for immediate interaction on handheld tablets regardless of whether the device is alive or in sleep mode. The technique rests on the close succession of two input events: first a ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

May 2017

7138 pages

ISBN:9781450346559

DOI:10.1145/3025453

General Chairs:
Gloria Mark
University of California Irvine
,
Susan Fussell
Cornell University
,
Program Chairs:
Cliff Lampe
University of Michigan
,
m.c. schraefel
University of Southampton
,
Juan Pablo Hourcade
University of Iowa
,
Caroline Appert
Université Paris-Sud
,
Daniel Wigdor
University of Toronto

Copyright © 2017 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 May 2017

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Volkswagen Foundation

Conference

CHI '17

Sponsor:

SIGCHI

CHI '17: CHI Conference on Human Factors in Computing Systems

May 6 - 11, 2017

Colorado, Denver, USA

Acceptance Rates

CHI '17 Paper Acceptance Rate 600 of 2,400 submissions, 25%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

30
Total Citations
View Citations
2,601
Total Downloads

Downloads (Last 12 months)162
Downloads (Last 6 weeks)15

Reflects downloads up to 17 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Cheng QTian XLiao FChen TChen XWang ZDong YLiu Y(2024)Enhancing Mobile Interaction: Practical Insights from Smartphone and Smartwatch IntegrationAdjunct Proceedings of the 26th International Conference on Mobile Human-Computer Interaction10.1145/3640471.3680451(1-9)Online publication date: 21-Sep-2024
https://dl.acm.org/doi/10.1145/3640471.3680451
Grubert JWitzani LOtte AGesslein TKranz MKristensson P(2024)Text Entry Performance and Situation Awareness of a Joint Optical See-Through Head-Mounted Display and Smartphone SystemIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.330931630:8(5830-5846)Online publication date: Aug-2024
https://doi.org/10.1109/TVCG.2023.3309316
Zeng XWang XGou ZChen YZhang T(2023)WebJump: AR-facilitated Distributed Display of Web PagesExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585669(1-6)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585669
Grubert J(2023)Mixed Reality Interaction TechniquesSpringer Handbook of Augmented Reality10.1007/978-3-030-67822-7_5(109-129)Online publication date: 1-Jan-2023
https://doi.org/10.1007/978-3-030-67822-7_5
Di Gioia FBrasier EPietriga EAppert C(2022)Investigating the Use of AR Glasses for Content Annotation on Mobile DevicesProceedings of the ACM on Human-Computer Interaction10.1145/35677276:ISS(430-447)Online publication date: 14-Nov-2022
https://dl.acm.org/doi/10.1145/3567727
Terenti MVatavu R(2022)VIREO: Web-based Graphical Authoring of Vibrotactile Feedback for Interactions with Mobile and Wearable DevicesInternational Journal of Human–Computer Interaction10.1080/10447318.2022.210958439:20(4162-4180)Online publication date: 18-Aug-2022
https://doi.org/10.1080/10447318.2022.2109584
Vock KHubenschmid SZagermann JButscher SReiterer H(2021)IDIAR: Augmented Reality Dashboards to Supervise Mobile Intervention StudiesProceedings of Mensch und Computer 202110.1145/3473856.3473876(248-259)Online publication date: 5-Sep-2021
https://dl.acm.org/doi/10.1145/3473856.3473876
Heller FTodi KLuyten K(2021)An Interactive Design Space for Wearable DisplaysProceedings of the 23rd International Conference on Mobile Human-Computer Interaction10.1145/3447526.3472034(1-14)Online publication date: 27-Sep-2021
https://dl.acm.org/doi/10.1145/3447526.3472034
Brasier EPietriga EAppert C(2021)AR-enhanced Widgets for Smartphone-centric InteractionProceedings of the 23rd International Conference on Mobile Human-Computer Interaction10.1145/3447526.3472019(1-12)Online publication date: 27-Sep-2021
https://dl.acm.org/doi/10.1145/3447526.3472019
Lee DKim JOakley IKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)FingerText: Exploring and Optimizing Performance for Wearable, Mobile and One-Handed TypingProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445106(1-15)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445106
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten