research-article

Replicate and Reuse: Tangible Interaction Design for Digitally-Augmented Physical Media Objects

Authors:

Daniel VogelAuthors Info & Claims

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Pages 1 - 12

https://doi.org/10.1145/3313831.3376139

Published: 23 April 2020 Publication History

Abstract

Technology has transformed our physical interactions into infinitely more scalable and flexible digital ones. We can peruse an infinite number of photos, news articles, and books. However, these digital experiences lack the physical experience of paging through an album, reading a newspaper, or meandering through a bookshelf. Overlaying physical objects with digital content using augmented reality is a promising avenue towards bridging this gap. In this paper, we investigate the interaction design for such digital-overlaid physical objects and their varying levels of tangibility. We first conduct a user evaluation of a physical photo album that uses tangible interactions to support physical and digital operations. We further prototype multiple objects including bookshelves and newspapers and probe users on their usage, capabilities, and interactions. We then conduct a qualitative investigation of three interaction designs with varying tangibility that use three different input modalities. Finally, we discuss the insights from our investigations and recommend design guidelines.

Supplemental Material

MP4 File

Preview video

Download
9.79 MB

MP4 File

Supplemental video

Download
179.14 MB

References

[1]

Anand Agarawala and Ravin Balakrishnan. 2006. Keepin' It Real: Pushing the Desktop Metaphor with Physics, Piles and the Pen. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '06). ACM, NY, NY, USA, 1283--1292.

Digital Library

[2]

W Ark, Christopher Dryer, T Selker, and Shumin Zhai. 2018. Representation Matters: The Effect of 3 Dimensional Objects and Spatial Metaphor in a Graphical User Interface. (04 2018).

[3]

Oriel Bergig, Eyal Soreq, Nate Hagbi, Kirill Pevzner, Nati Levi, Shoham Blau, Yulia Smelansky, and Jihad El-Sana. 2011. Out of the Cube: Augmented Rubik's Cube. Int. J. Comput. Games Technol. 2011 (2011), 1--7.

Digital Library

[4]

Mark Billinghurst, Hirokazu Kato, and Ivan Poupyrev. 2001. The MagicBook: a transitional AR interface. Comput. Graph. 25, 5 (oct 2001), 745--753.

[5]

Mark Billinghurst, Hirokazu Kato, Ivan Poupyrev, and others. 2008. Tangible augmented reality. ACM SIGGRAPH ASIA 7 (2008).

[6]

M. Billinghurst, S. Weghorst, and T. Furness. 1998. Shared space: An augmented reality approach for computer supported collaborative work. Virtual Reality 3, 1 (01 Mar 1998), 25--36.

Digital Library

[7]

John Brooke and others. 1996. SUS-A quick and dirty usability scale. Usability evaluation in industry 189, 194 (1996), 4--7.

[8]

VP Curtis. 2011. Photographic memory: The album in the age of photography. (2011). https://scholar.google.ca/scholar?hl=en

[9]

R Dachselt and S Al-Saiegh. 2011. Interacting with printed books using digital pens and smart mobile projection. Proc. Work. Mob. ... (2011). http://old-v1.imld.de/forschung/publikationen/data/ 2011/2011-CHI-ProjectiveAugmentedBooks.pdf

[10]

Pierre Dragicevic. 2004. Combining Crossing-based and Paper-based Interaction Paradigms for Dragging and Dropping Between Overlapping Windows. In Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology (UIST '04). ACM, NY, NY, USA, 193--196.

Digital Library

[11]

M. Fjeld and B. M. Voegtli. 2002. Augmented Chemistry: an interactive educational workbench. In Proceedings. International Symposium on Mixed and Augmented Reality. 259--321.

[12]

Raphael Grasset, Andreas Duenser, Hartmut Seichter, and Mark Billinghurst. 2007. The mixed reality book. In CHI '07 Ext. Abstr. Hum. factors Comput. Syst. - CHI '07. ACM Press, New York, New York, USA, 1953.

Digital Library

[13]

J. Han, S. Heo, H. E. Lee, and G. Lee. 2014. The IrPen: A 6-DOF Pen for Interaction with Tablet Computers. IEEE Computer Graphics and Applications 34, 3 (May 2014), 22--29.

[14]

Anuruddha Hettiarachchi and Daniel Wigdor. 2016. Annexing Reality: Enabling Opportunistic Use of Everyday Objects As Tangible Proxies in Augmented Reality. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, NY, NY, USA, 1957--1967.

Digital Library

[15]

Terje Hillesund. 2010. Digital reading spaces: How expert readers handle books, the Web and electronic paper. First Monday 15, 4 (apr 2010).

[16]

David Holman, Roel Vertegaal, Mark Altosaar, Nikolaus Troje, and Derek Johns. 2005. Paper Windows: Interaction Techniques for Digital Paper. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '05). ACM, NY, NY, USA, 591--599.

Digital Library

[17]

Hyun-Tae Jeong, Dong-Woo Lee, Gi-Su Heo, and Cheol-Hoon Lee. 2012. Live Book: A mixed reality book using a projection system. In 2012 IEEE Int. Conf. Consum. Electron. IEEE, 680--681.

[18]

Sylvia Irawati, Scott Green, Mark Billinghurst, Andreas Duenser, and Heedong Ko. 2006. An Evaluation of an Augmented Reality Multimodal Interface Using Speech and Paddle Gestures. Springer, Berlin, Heidelberg, 272--283.

Digital Library

[19]

Maria Karam and M. C. Schraefel. 2005. A taxonomy of gestures in human computer interactions. Technical Report.

[20]

H. Kato, M. Billinghurst, I. Poupyrev, K. Imamoto, and K. Tachibana. 2000. Virtual object manipulation on a table-top AR environment. In Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000). 111--119.

[21]

KAWASHIMA and T. 2001. Magic Paddle : A Tangible Augmented Reality Interface for Object Manipulation. Proc. 2nd Int. Symp. Mix. Reality, 2001 (2001). https://ci.nii.ac.jp/naid/10022426812

[22]

Eike Kleiner, Roman Rädle, and Harald Reiterer. 2013. Blended shelf. In CHI '13 Ext. Abstr. Hum. Factors Comput. Syst. - CHI EA '13. ACM Press, New York, New York, USA, 577.

Digital Library

[23]

Gun A. Lee, Mark Billinghurst, and Gerard Jounghyun Kim. 2004. Occlusion based interaction methods for tangible augmented reality environments. In Proc. 2004 ACM SIGGRAPH Int. Conf. Virtual Real. Contin. its Appl. Ind. - VRCAI '04. ACM Press, New York, New York, USA, 419.

Digital Library

[24]

Gun A. Lee, Gerard J. Kim, and Mark Billinghurst. 1. Interaction Design for Tangible Augmented Reality Applications. In Emerg. Technol. Augment. Real. IGI Global, 261--282.

[25]

David Lindlbauer, Jörg Mueller, and Marc Alexa. 2017. Changing the Appearance of Real-World Objects By Modifying Their Surroundings. In Proc. 2017 CHI Conf. Hum. Factors Comput. Syst. - CHI '17. ACM Press, New York, New York, USA, 3954--3965.

Digital Library

[26]

Julian Conrad Alan Looser. 2007. Ar magic lenses: Addressing the challenge of focus and context in augmented reality. (2007).

[27]

George Margetis, Xenophon Zabulis, Panagiotis Koutlemanis, Margherita Antona, and Constantine Stephanidis. 2013. Augmented interaction with physical books in an Ambient Intelligence learning environment. Multimed. Tools Appl. 67, 2 (nov 2013), 473--495.

Digital Library

[28]

Chris McDonald and Gerhard Roth. 2003. Replacing a mouse with hand gesture in a plane-based augmented reality system. National Research Council of Canada.

[29]

H Mehmet. 2016. The classification of augmented reality books: a literature review. In Proceedings of INTED2016 Conference. 4110--4118.

[30]

Meta 2. 2018. Augmented Reality Glasses. (2018). https://www.metavision.com/

[31]

Thomas Pietrzak, Sylvain Malacria, and Éric Lecolinet. 2012. S-Notebook: Augmenting Mobile Devices with Interactive Paper for Data Management. In Proceedings of the International Working Conference on Advanced Visual Interfaces (AVI '12). ACM, NY, NY, USA, 733--736.

Digital Library

[32]

Ben Piper, Carlo Ratti, and Hiroshi Ishii. 2002. Illuminating clay. In Proc. SIGCHI Conf. Hum. factors Comput. Syst. Chang. our world, Chang. ourselves - CHI '02. ACM Press, New York, New York, USA, 355.

Digital Library

[33]

POUPYREV and I. 2001. Tiles : A Mixed Reality Authoring Interface. Human-Computer Interact. INTERACT'01 (2001). https://ci.nii.ac.jp/naid/10020982585/

[34]

A. Rauber and H. Bina. 1999. A metaphor graphics based representation of digital libraries on the World Wide Web: using the libViewer to make metadata visible. In Proceedings. Tenth Int. Work. Database Expert Syst. Appl. DEXA 99. IEEE, 286--290.

[35]

ShelvAR. 2013. ShelvAR. (2013). http://www.shelvar.com/

[36]

Jinwook Shim, Yoonsik Yang, Nahyung Kang, Jonghoon Seo, and Tack-Don Han. 2016. Gesture-based interactive augmented reality content authoring system using HMD. Virtual Reality 20, 1 (2016), 57--69.

Digital Library

[37]

Carol Tenopir, Donald W. King, Sheri Edwards, and Lei Wu. 2009. Electronic journals and changes in scholarly article seeking and reading patterns. Aslib Proc. 61, 1 (jan 2009), 5--32.

[38]

Lucia Terrenghi, David Kirk, Abigail Sellen, and Shahram Izadi. 2007. Affordances for manipulation of physical versus digital media on interactive surfaces. In Proc. SIGCHI Conf. Hum. factors Comput. Syst. - CHI '07. ACM Press, New York, New York, USA, 1157.

Digital Library

[39]

E. J. Umlauf, H. Piringer, G. Reitmayr, and D. Schmalstieg. 2002. ARLib: the augmented library. In The First IEEE International Workshop Agumented Reality Toolkit,. 2 pp.--.

[40]

Nancy A. Van House. 2011. Personal photography, digital technologies and the uses of the visual. Vis. Stud. 26, 2 (jun 2011), 125--134.

[41]

Vuforia. Vuforia | Augmented Reality. (????). https://www.vuforia.com/

[42]

Pierre Wellner. 1993. Interacting with Paper on the DigitalDesk. Commun. ACM 36, 7 (July 1993), 87--96.

Digital Library

[43]

David West, Aaron Quigley, and Judy Kay. 2007. MEMENTO: a digital-physical scrapbook for memory sharing. Personal and Ubiquitous Computing 11, 4 (01 Apr 2007), 313--328.

[44]

J. Wijnen and E.A.W.H. van den Hoven. 2011. Connected sketchbook : linking digital files to physical pages. (2011). http://repository.tue.nl/730757

[45]

Chih-Sung (Andy) Wu, Susan J. Robinson, and Ali Mazalek. 2008. Turning a page on the digital annotation of physical books. In Proc. 2nd Int. Conf. Tangible Embed. Interact. - TEI '08. ACM Press, New York, New York, USA, 109.

Digital Library

[46]

Po-Chen Wu, Robert Wang, Kenrick Kin, Christopher Twigg, Shangchen Han, Ming-Hsuan Yang, and Shao-Yi Chien. 2017. DodecaPen: Accurate 6DoF Tracking of a Passive Stylus. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (UIST '17). ACM, NY, NY, USA, 365--374.

Digital Library

Cited By

Lazaro MKim S(2025)Review of Multimodal Interaction in Optical See-Through Augmented RealityInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2442128(1-17)Online publication date: 17-Jan-2025
https://doi.org/10.1080/10447318.2024.2442128
He FHu XQian XZhu ZRamani K(2024)AdapTUI: Adaptation of Geometric-Feature-Based Tangible User Interfaces in Augmented RealityProceedings of the ACM on Human-Computer Interaction10.1145/36981278:ISS(44-69)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3698127
Perelman GSerrano MDubois E(2024)Exploiting Physical Referent Features as Input for Multidimensional Data Selection in Augmented RealityACM Transactions on Computer-Human Interaction10.1145/364861331:4(1-40)Online publication date: 19-Sep-2024
https://dl.acm.org/doi/10.1145/3648613
Show More Cited By

Index Terms

Replicate and Reuse: Tangible Interaction Design for Digitally-Augmented Physical Media Objects
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Tangible 3D tabletops: combining tangible tabletop interaction and 3D projection
NordiCHI '12: Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design

In this paper we present the tangible 3D tabletop and discuss the design potential of this novel interface. The tangible 3D tabletop combines tangible tabletop interaction with 3D projection in such a way that the tangible objects may be augmented with ...
A physical approach to tangible interaction design
TEI '07: Proceedings of the 1st international conference on Tangible and embedded interaction

The field of tangible interaction is growing in rich and diverse directions calling for new forms of understanding. In this paper I will present a view on tangible interaction that has a strong focus on movement and interaction qualities. I will ...
Physical query interface for tangible augmented tagging and interaction

Highlights Tangible tagging and interacton provide a new physical query interface. Combining visual and invisible interfaces provides more natural interactions. The approach supports physical querying, filtering and grouping of AR contents. Jena-based ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

April 2020

10688 pages

ISBN:9781450367080

DOI:10.1145/3313831

General Chairs:
Regina Bernhaupt
Eindhoven University of Technology, Netherlands
,
Florian 'Floyd' Mueller
Monash University, Australia
,
David Verweij
Newcastle University, UK
,
Josh Andres
RMIT, Australia
,
Program Chairs:
Joanna McGrenere
University of British Columbia, Canada
,
Andy Cockburn
University of Canterbury, New Zealand
,
Ignacio Avellino
University of Maryland Baltimore County, USA
,
Alix Goguey
Grenoble Alpes University, France
,
Pernille Bjørn
University of Copenhagen, Denmark
,
Shengdong (Shen) Zhao
National University of Singapore, Singapore
,
Briane Paul Samson
Future University Hakodate, Japan & De La Salle University, Philippines
,
Rafal Kocielnik
University of Washington, USA

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 April 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

CHI '20

Sponsor:

SIGCHI

CHI '20: CHI Conference on Human Factors in Computing Systems

April 25 - 30, 2020

HI, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
1,862
Total Downloads

Downloads (Last 12 months)202
Downloads (Last 6 weeks)18

Reflects downloads up to 23 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lazaro MKim S(2025)Review of Multimodal Interaction in Optical See-Through Augmented RealityInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2442128(1-17)Online publication date: 17-Jan-2025
https://doi.org/10.1080/10447318.2024.2442128
He FHu XQian XZhu ZRamani K(2024)AdapTUI: Adaptation of Geometric-Feature-Based Tangible User Interfaces in Augmented RealityProceedings of the ACM on Human-Computer Interaction10.1145/36981278:ISS(44-69)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3698127
Perelman GSerrano MDubois E(2024)Exploiting Physical Referent Features as Input for Multidimensional Data Selection in Augmented RealityACM Transactions on Computer-Human Interaction10.1145/364861331:4(1-40)Online publication date: 19-Sep-2024
https://dl.acm.org/doi/10.1145/3648613
Câmara Olim SNisi VRomão T(2024)Augmented reality interactive experiences for multi-level chemistry understandingInternational Journal of Child-Computer Interaction10.1016/j.ijcci.2024.10068142:COnline publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1016/j.ijcci.2024.100681
Chulpongsatorn NLunding MSoni NSuzuki R(2023)Augmented Math: Authoring AR-Based Explorable Explanations by Augmenting Static Math TextbooksProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606827(1-16)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586183.3606827
Jain RShi JDuan RZhu ZQian XRamani K(2023)Ubi-TOUCH: Ubiquitous Tangible Object Utilization through Consistent Hand-object interaction in Augmented RealityProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606793(1-18)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586183.3606793
Uhl JSchrom-Feiertag HRegal GHirsch LWeiss YTscheligi M(2023)When Realities Interweave: Exploring the Design Space of Immersive Tangible XRProceedings of the Seventeenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3569009.3571843(1-4)Online publication date: 26-Feb-2023
https://dl.acm.org/doi/10.1145/3569009.3571843
Monteiro KVatsal RChulpongsatorn NParnami ASuzuki R(2023)Teachable Reality: Prototyping Tangible Augmented Reality with Everyday Objects by Leveraging Interactive Machine TeachingProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581449(1-15)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581449
Chen AOdom WNeustaedter CZhong CLin H(2023)Exploring Memory-Oriented Interactions with Digital Photos In and Across Time: A Field Study of ChronoscopeProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581012(1-20)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581012
He FHu XShi JQian XWang TRamani K(2023)Ubi Edge: Authoring Edge-Based Opportunistic Tangible User Interfaces in Augmented RealityProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580704(1-14)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580704
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten