research-article

Teachable Reality: Prototyping Tangible Augmented Reality with Everyday Objects by Leveraging Interactive Machine Teaching

Authors:

Kyzyl Monteiro,

Neil Chulpongsatorn,

Ryo SuzukiAuthors Info & Claims

CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

Article No.: 459, Pages 1 - 15

https://doi.org/10.1145/3544548.3581449

Published: 19 April 2023 Publication History

Abstract

This paper introduces Teachable Reality, an augmented reality (AR) prototyping tool for creating interactive tangible AR applications with arbitrary everyday objects. Teachable Reality leverages vision-based interactive machine teaching (e.g., Teachable Machine), which captures real-world interactions for AR prototyping. It identifies the user-defined tangible and gestural interactions using an on-demand computer vision model. Based on this, the user can easily create functional AR prototypes without programming, enabled by a trigger-action authoring interface. Therefore, our approach allows the flexibility, customizability, and generalizability of tangible AR applications that can address the limitation of current marker-based approaches. We explore the design space and demonstrate various AR prototypes, which include tangible and deformable interfaces, context-aware assistants, and body-driven AR applications. The results of our user study and expert interviews confirm that our approach can lower the barrier to creating functional AR prototypes while also allowing flexible and general-purpose prototyping experiences.

Supplementary Material

MP4 File (3544548.3581449-video-preview.mp4)

Video Preview

Download
76.12 MB

MP4 File (3544548.3581449-video-figure.mp4)

Video Figure

Download
237.55 MB

MP4 File (3544548.3581449-talk-video.mp4)

Pre-recorded Video Presentation

Download
177.10 MB

References

[1]

2022. HoloBuilder. https://www.holobuilder.com/

[2]

8th Wall. 2022. Niantic Inc.https://www.8thwall.com/

[3]

A-Frame. 2022. A-Frame. https://aframe.io/

[4]

Karan Ahuja, Sujeath Pareddy, Robert Xiao, Mayank Goel, and Chris Harrison. 2019. Lightanchors: Appropriating point lights for spatially-anchored augmented reality interfaces. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology. 189–196.

Digital Library

[5]

Günter Alce, Mattias Wallergård, and Klas Hermodsson. 2015. WozARd: a wizard of Oz method for wearable augmented reality interaction—a pilot study. Advances in human-computer interaction 2015 (2015).

[6]

Jatin Arora, Aryan Saini, Nirmita Mehra, Varnit Jain, Shwetank Shrey, and Aman Parnami. 2019. Virtualbricks: Exploring a scalable, modular toolkit for enabling physical manipulation in vr. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12.

Digital Library

[7]

Narges Ashtari, Andrea Bunt, Joanna McGrenere, Michael Nebeling, and Parmit K Chilana. 2020. Creating augmented and virtual reality applications: Current practices, challenges, and opportunities. In Proceedings of the 2020 CHI conference on human factors in computing systems. 1–13.

Digital Library

[8]

Mark Billinghurst, Hirokazu Kato, Ivan Poupyrev, 2008. Tangible augmented reality. Acm siggraph asia 7, 2 (2008), 1–10.

[9]

Alberto Boem and Giovanni Maria Troiano. 2019. Non-rigid HCI: A review of deformable interfaces and input. In Proceedings of the 2019 on Designing Interactive Systems Conference. 885–906.

Digital Library

[10]

Jorge CS Cardoso and Jorge M Ribeiro. 2021. Tangible VR book: exploring the design space of marker-based tangible interfaces for virtual reality. Applied Sciences 11, 4 (2021), 1367.

[11]

Michelle Carney, Barron Webster, Irene Alvarado, Kyle Phillips, Noura Howell, Jordan Griffith, Jonas Jongejan, Amit Pitaru, and Alexander Chen. 2020. Teachable machine: Approachable Web-based tool for exploring machine learning classification. In Extended abstracts of the 2020 CHI conference on human factors in computing systems. 1–8.

[12]

Vinayak Cecil Piya. 2016. RealFusion: An interactive workflow for repurposing real-world objects towards early-stage creative ideation. In Graphics interface.

[13]

Kai-Yin Cheng, Rong-Hao Liang, Bing-Yu Chen, Rung-Huei Laing, and Sy-Yen Kuo. 2010. iCon: utilizing everyday objects as additional, auxiliary and instant tabletop controllers. In Proceedings of the SIGCHI conference on Human factors in computing systems. 1155–1164.

Digital Library

[14]

Jinsung Cho, Geunmo Kim, Hyunmin Go, Sungmin Kim, Jisub Kim, and Bongjae Kim. 2021. DeepBlock: Web-based Deep Learning Education Platform. The Journal of the Institute of Internet, Broadcasting and Communication 21, 3(2021), 43–50.

[15]

Stephanie Claudino Daffara, Anna Brewer, Balasaravanan Thoravi Kumaravel, and Bjoern Hartmann. 2020. Living Paper: Authoring AR Narratives Across Digital and Tangible Media. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. 1–10.

[16]

Christian Corsten, Ignacio Avellino, Max Möllers, and Jan Borchers. 2013. Instant user interfaces: repurposing everyday objects as input devices. In Proceedings of the 2013 ACM international conference on Interactive tabletops and surfaces. 71–80.

Digital Library

[17]

Christian Corsten, Chat Wacharamanotham, and Jan Borchers. 2013. Fillables: everyday vessels as tangible controllers with adjustable haptics. In CHI’13 Extended Abstracts on Human Factors in Computing Systems. 2129–2138.

[18]

Florian Daiber, Donald Degraen, André Zenner, Tanja Döring, Frank Steinicke, Oscar Javier Ariza Nunez, and Adalberto L Simeone. 2021. Everyday Proxy Objects for Virtual Reality. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. 1–6.

[19]

Mustafa Doga Dogan, Ahmad Taka, Michael Lu, Yunyi Zhu, Akshat Kumar, Aakar Gupta, and Stefanie Mueller. 2022. InfraredTags: Embedding Invisible AR Markers and Barcodes Using Low-Cost, Infrared-Based 3D Printing and Imaging Tools. In CHI Conference on Human Factors in Computing Systems. 1–12.

Digital Library

[20]

Ruofei Du, Alex Olwal, Mathieu Le Goc, Shengzhi Wu, Danhang Tang, Yinda Zhang, Jun Zhang, David Joseph Tan, Federico Tombari, and David Kim. 2022. Opportunistic Interfaces for Augmented Reality: Transforming Everyday Objects into Tangible 6DoF Interfaces Using Ad hoc UI. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. 1–4.

Digital Library

[21]

David Englmeier, Julia Dörner, Andreas Butz, and Tobias Höllerer. 2020. A tangible spherical proxy for object manipulation in augmented reality. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 221–229.

[22]

George W Fitzmaurice, Hiroshi Ishii, and William AS Buxton. 1995. Bricks: laying the foundations for graspable user interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems. 442–449.

Digital Library

[23]

Sergio Garrido-Jurado, Rafael Muñoz-Salinas, Francisco José Madrid-Cuevas, and Manuel Jesús Marín-Jiménez. 2014. Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recognition 47, 6 (2014), 2280–2292.

Digital Library

[24]

Ross Girshick. 2015. Fast r-cnn. In Proceedings of the IEEE international conference on computer vision. 1440–1448.

Digital Library

[25]

Terrell Glenn, Ananya Ipsita, Caleb Carithers, Kylie Peppler, and Karthik Ramani. 2020. StoryMakAR: Bringing stories to life with an augmented reality & physical prototyping toolkit for youth. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–14.

Digital Library

[26]

Google. 2022. Live AR View, Google Maps. https://arvr.google.com/ar/

[27]

Google. 2022. MediaPipe. https://mediapipe.dev/

[28]

Google. 2022. Mobile Net v3. https://tfhub.dev/google/tfjs-model/imagenet/mobilenet_v3_small_100_224/feature_vector/5/default/1

[29]

Aakar Gupta, Bo Rui Lin, Siyi Ji, Arjav Patel, and Daniel Vogel. 2020. Replicate and reuse: Tangible interaction design for digitally-augmented physical media objects. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–12.

Digital Library

[30]

Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. 2017. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision. 2961–2969.

[31]

Robert Held, Ankit Gupta, Brian Curless, and Maneesh Agrawala. 2012. 3D puppetry: a kinect-based interface for 3D animation. In UIST, Vol. 12. Citeseer, 423–434.

[32]

Anuruddha Hettiarachchi and Daniel Wigdor. 2016. Annexing reality: Enabling opportunistic use of everyday objects as tangible proxies in augmented reality. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 1957–1967.

Digital Library

[33]

Valentin Heun, James Hobin, and Pattie Maes. 2013. Reality editor: programming smarter objects. In Proceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publication. 307–310.

Digital Library

[34]

Ke Huo and Karthik Ramani. 2016. Window-Shaping: 3D Design Ideation in Mixed Reality. In Proceedings of the 2016 Symposium on Spatial User Interaction. 189–189.

Digital Library

[35]

Ke Huo, Tianyi Wang, Luis Paredes, Ana M Villanueva, Yuanzhi Cao, and Karthik Ramani. 2018. Synchronizar: Instant synchronization for spontaneous and spatial collaborations in augmented reality. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology. 19–30.

Digital Library

[36]

Adobe Inc.2022. Adobe Aero. https://www.adobe.com/in/products/aero.html

[37]

Adobe Inc.2022. Adobe XD. https://www.adobe.com/in/products/xd.html

[38]

Apple Inc.2022. Reality Composer. https://developer.apple.com/augmented-reality/reality-composer/

[39]

Gravity Sketch Inc.2017. Gravity Sketch. https://www.gravitysketch.com/

[40]

InVision Inc.2022. InVision. https://www.invisionapp.com/

[41]

Sketch Inc.2022. Sketch. https://www.sketch.com/

[42]

SketchUp Inc.2022. Sketchup. https://www.sketchup.com/page/homepage

[43]

Unreal Inc.2022. Unreal Engine. https://www.unrealengine.com/en-US

[44]

Hiroshi Ishii. 2004. Bottles: A transparent interface as a tribute to mark weiser. IEICE Transactions on information and systems 87, 6 (2004), 1299–1311.

[45]

Hiroshi Ishii and Brygg Ullmer. 1997. Tangible bits: towards seamless interfaces between people, bits and atoms. In Proceedings of the ACM SIGCHI Conference on Human factors in computing systems. 234–241.

Digital Library

[46]

Bret Jackson and Daniel F Keefe. 2016. Lift-off: Using reference imagery and freehand sketching to create 3d models in vr. IEEE transactions on visualization and computer graphics 22, 4(2016), 1442–1451.

[47]

Brett R Jones, Hrvoje Benko, Eyal Ofek, and Andrew D Wilson. 2013. IllumiRoom: peripheral projected illusions for interactive experiences. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 869–878.

Digital Library

[48]

Brian Jordan, Nisha Devasia, Jenna Hong, Randi Williams, and Cynthia Breazeal. 2021. PoseBlocks: A toolkit for creating (and dancing) with AI. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 15551–15559.

[49]

Hiroki Kaimoto, Kyzyl Monteiro, Mehrad Faridan, Jiatong Li, Samin Farajian, Yasuaki Kakehi, Ken Nakagaki, and Ryo Suzuki. 2022. Sketched Reality: Sketching Bi-Directional Interactions Between Virtual and Physical Worlds with AR and Actuated Tangible UI. arXiv preprint arXiv:2208.06341(2022).

[50]

Seokbin Kang, Ekta Shokeen, Virginia L Byrne, Leyla Norooz, Elizabeth Bonsignore, Caro Williams-Pierce, and Jon E Froehlich. 2020. ARMath: augmenting everyday life with math learning. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–15.

Digital Library

[51]

Hirokazu Kato and Mark Billinghurst. 1999. Marker tracking and hmd calibration for a video-based augmented reality conferencing system. In Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR’99). IEEE, 85–94.

[52]

Annie Kelly, R Benjamin Shapiro, Jonathan de Halleux, and Thomas Ball. 2018. ARcadia: A rapid prototyping platform for real-time tangible interfaces. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–8.

Digital Library

[53]

Jarrod Knibbe, Tovi Grossman, and George Fitzmaurice. 2015. Smart makerspace: An immersive instructional space for physical tasks. In Proceedings of the 2015 International Conference on Interactive Tabletops & Surfaces. 83–92.

Digital Library

[54]

Veronika Krauß, Michael Nebeling, Florian Jasche, and Alexander Boden. 2022. Elements of XR Prototyping: Characterizing the Role and Use of Prototypes in Augmented and Virtual Reality Design. In CHI Conference on Human Factors in Computing Systems. 1–18.

[55]

David Ledo, Steven Houben, Jo Vermeulen, Nicolai Marquardt, Lora Oehlberg, and Saul Greenberg. 2018. Evaluation strategies for HCI toolkit research. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–17.

Digital Library

[56]

Gun A Lee, Gerard J Kim, and Mark Billinghurst. 2005. Immersive authoring: What you experience is what you get (wyxiwyg). Commun. ACM 48, 7 (2005), 76–81.

Digital Library

[57]

Germán Leiva and Michel Beaudouin-Lafon. 2018. Montage: a video prototyping system to reduce re-shooting and increase re-usability. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology. 675–682.

Digital Library

[58]

Germán Leiva, Jens Emil Grønbæk, Clemens Nylandsted Klokmose, Cuong Nguyen, Rubaiat Habib Kazi, and Paul Asente. 2021. Rapido: Prototyping Interactive AR Experiences through Programming by Demonstration. In The 34th Annual ACM Symposium on User Interface Software and Technology. 626–637.

Digital Library

[59]

Germán Leiva, Cuong Nguyen, Rubaiat Habib Kazi, and Paul Asente. 2020. Pronto: Rapid augmented reality video prototyping using sketches and enaction. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.

Digital Library

[60]

Zhen Li, Michelle Annett, Ken Hinckley, Karan Singh, and Daniel Wigdor. 2019. Holodoc: Enabling mixed reality workspaces that harness physical and digital content. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–14.

Digital Library

[61]

Jian Liao, Adnan Karim, Shivesh Singh Jadon, Rubaiat Habib Kazi, and Ryo Suzuki. 2022. RealityTalk: Real-Time Speech-Driven Augmented Presentation for AR Live Storytelling. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. 1–12.

Digital Library

[62]

David Lindlbauer, Jens Emil Grønbæk, Morten Birk, Kim Halskov, Marc Alexa, and Jörg Müller. 2016. Combining shape-changing interfaces and spatial augmented reality enables extended object appearance. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 791–802.

Digital Library

[63]

Blair MacIntyre, Maribeth Gandy, Steven Dow, and Jay David Bolter. 2004. DART: a toolkit for rapid design exploration of augmented reality experiences. In Proceedings of the 17th annual ACM symposium on User interface software and technology. 197–206.

Digital Library

[64]

Michael Nebeling and Katy Madier. 2019. 360proto: Making interactive virtual reality & augmented reality prototypes from paper. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–13.

Digital Library

[65]

Michael Nebeling, Janet Nebeling, Ao Yu, and Rob Rumble. 2018. Protoar: Rapid physical-digital prototyping of mobile augmented reality applications. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–12.

Digital Library

[66]

Michael Nebeling and Maximilian Speicher. 2018. The trouble with augmented reality/virtual reality authoring tools. In 2018 IEEE international symposium on mixed and augmented reality adjunct (ISMAR-Adjunct). IEEE, 333–337.

[67]

Gary Ng, Joon Gi Shin, Alexander Plopski, Christian Sandor, and Daniel Saakes. 2018. Situated game level editing in augmented reality. In Proceedings of the Twelfth International Conference on Tangible, Embedded, and Embodied Interaction. 409–418.

Digital Library

[68]

Nintendo. 2022. Mario Kart Live. https://mklive.nintendo.com/

[69]

Nintendo. 2022. Nintendo Labo. https://www.nintendo.co.uk/Nintendo-Labo/Nintendo-Labo-1328637.html

[70]

OpenCV. 2022. OpenCV. https://opencv.org/

[71]

Youngki Park and Youhyun Shin. 2021. Tooee: A Novel Scratch Extension for K-12 Big Data and Artificial Intelligence Education Using Text-Based Visual Blocks. IEEE Access 9(2021), 149630–149646.

[72]

Shwetha Rajaram and Michael Nebeling. 2022. Paper Trail: An Immersive Authoring System for Augmented Reality Instructional Experiences. In CHI Conference on Human Factors in Computing Systems. 1–16.

[73]

Gonzalo Ramos, Christopher Meek, Patrice Simard, Jina Suh, and Soroush Ghorashi. 2020. Interactive machine teaching: a human-centered approach to building machine-learned models. Human–Computer Interaction 35, 5-6 (2020), 413–451.

[74]

Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. 2016. You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition. 779–788.

[75]

Patrick Reipschläger, Severin Engert, and Raimund Dachselt. 2020. Augmented displays: Seamlessly extending interactive surfaces with head-mounted augmented reality. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. 1–4.

Digital Library

[76]

Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, 2015. Imagenet large scale visual recognition challenge. International journal of computer vision 115, 3 (2015), 211–252.

Digital Library

[77]

Alpay Sabuncuoglu and T Metin Sezgin. 2022. Prototyping Products using Web-based AI Tools: Designing a Tangible Programming Environment with Children. In 6th FabLearn Europe/MakeEd Conference 2022. 1–6.

Digital Library

[78]

Nazmus Saquib, Rubaiat Habib Kazi, Li-Yi Wei, and Wilmot Li. 2019. Interactive body-driven graphics for augmented video performance. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12.

Digital Library

[79]

Hartmut Seichter, Julian Looser, and Mark Billinghurst. 2008. ComposAR: An intuitive tool for authoring AR applications. In 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality. IEEE, 177–178.

Digital Library

[80]

Paden Shorey and Audrey Girouard. 2017. Bendtroller: An exploration of in-game action mappings with a deformable game controller. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 1447–1458.

Digital Library

[81]

Maximilian Speicher, Katy Lewis, and Michael Nebeling. 2021. Designers, the stage is yours! medium-fidelity prototyping of augmented & virtual reality interfaces with 360theater. Proceedings of the ACM on Human-Computer Interaction 5, EICS(2021), 1–25.

Digital Library

[82]

Jürgen Steimle, Andreas Jordt, and Pattie Maes. 2013. Flexpad: highly flexible bending interactions for projected handheld displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 237–246.

Digital Library

[83]

Ryo Suzuki, Rubaiat Habib Kazi, Li-Yi Wei, Stephen DiVerdi, Wilmot Li, and Daniel Leithinger. 2020. Realitysketch: Embedding responsive graphics and visualizations in AR through dynamic sketching. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology. 166–181.

Digital Library

[84]

Three.js. 2022. Ricardo Cabello. https://threejs.org/

[85]

Tiffany Tseng, Yumiko Murai, Natalie Freed, Deanna Gelosi, Tung D Ta, and Yoshihiro Kawahara. 2021. PlushPal: Storytelling with interactive plush toys and machine learning. In Interaction design and children. 236–245.

[86]

Nicolas Villar, Daniel Cletheroe, Greg Saul, Christian Holz, Tim Regan, Oscar Salandin, Misha Sra, Hui-Shyong Yeo, William Field, and Haiyan Zhang. 2018. Project zanzibar: A portable and flexible tangible interaction platform. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–13.

Digital Library

[87]

James A Walsh, G Stewart Von Itzstein, and Bruce H Thomas. 2013. Tangible agile mapping: ad-hoc tangible user interaction definition. In AUIC. Citeseer, 3–12.

[88]

James A Walsh, Stewart Von Itzstein, and Bruce H Thomas. 2014. Ephemeral interaction using everyday objects. In Proceedings of the Fifteenth Australasian User Interface Conference-Volume 150. 29–37.

Digital Library

[89]

Tianyi Wang, Xun Qian, Fengming He, Xiyun Hu, Yuanzhi Cao, and Karthik Ramani. 2021. GesturAR: An Authoring System for Creating Freehand Interactive Augmented Reality Applications. In The 34th Annual ACM Symposium on User Interface Software and Technology. 552–567.

[90]

Tianyi Wang, Xun Qian, Fengming He, Xiyun Hu, Ke Huo, Yuanzhi Cao, and Karthik Ramani. 2020. CAPturAR: An augmented reality tool for authoring human-involved context-aware applications. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology. 328–341.

Digital Library

[91]

Zeyu Wang, Cuong Nguyen, Paul Asente, and Julie Dorsey. 2021. Distanciar: Authoring site-specific augmented reality experiences for remote environments. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–12.

Digital Library

[92]

Matt Whitlock, Jake Mitchell, Nick Pfeufer, Brad Arnot, Ryan Craig, Bryce Wilson, Brian Chung, and Danielle Albers Szafir. 2020. MRCAT: In situ prototyping of interactive AR environments. In International Conference on Human-Computer Interaction. Springer, 235–255.

Digital Library

[93]

Randi Williams, Stephen P Kaputsos, and Cynthia Breazeal. 2021. Teacher Perspectives on How To Train Your Robot: A Middle School AI and Ethics Curriculum. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 15678–15686.

[94]

Robert Xiao, Chris Harrison, and Scott E Hudson. 2013. WorldKit: rapid and easy creation of ad-hoc interactive applications on everyday surfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 879–888.

Digital Library

[95]

Hui Ye and Hongbo Fu. 2022. ProGesAR: Mobile AR Prototyping for Proxemic and Gestural Interactions with Real-world IoT Enhanced Spaces. In CHI Conference on Human Factors in Computing Systems. 1–14.

[96]

Ya-Ting Yue, Yong-Liang Yang, Gang Ren, and Wenping Wang. 2017. SceneCtrl: Mixed reality enhancement via efficient scene editing. In Proceedings of the 30th annual ACM symposium on user interface software and technology. 427–436.

Digital Library

[97]

Clement Zheng, Peter Gyory, and Ellen Yi-Luen Do. 2020. Tangible interfaces with printed paper markers. In Proceedings of the 2020 ACM designing interactive systems conference. 909–923.

Digital Library

[98]

Qian Zhou, Sarah Sykes, Sidney Fels, and Kenrick Kin. 2020. Gripmarks: Using Hand Grips to Transform In-Hand Objects into Mixed Reality Input. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–11.

Digital Library

[99]

Zhongyi Zhou and Koji Yatani. 2022. Gesture-aware Interactive Machine Teaching with In-situ Object Annotations. arXiv preprint arXiv:2208.01211(2022).

[100]

Fengyuan Zhu and Tovi Grossman. 2020. Bishare: Exploring bidirectional interactions between smartphones and head-mounted augmented reality. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–14.

Digital Library

Cited By

Hai-Jew S(2024)Using Artmaking Generative AIs to Support Augmented Reality Learning Designs With Adobe Aero AppInquiries of Pedagogical Shifts and Critical Mindsets Among Educators10.4018/979-8-3693-1078-6.ch006(132-152)Online publication date: 8-Mar-2024
https://doi.org/10.4018/979-8-3693-1078-6.ch006
Rumbelow MColes A(2024)The Promise of AI Object-Recognition in Learning Mathematics: An Explorative Study of 6-Year-Old Children’s Interactions with Cuisenaire Rods and the Blockplay.ai AppEducation Sciences10.3390/educsci1406059114:6(591)Online publication date: 30-May-2024
https://doi.org/10.3390/educsci14060591
Liao JVan KXia ZSuzuki R(2024)RealityEffects: Augmenting 3D Volumetric Videos with Object-Centric Annotation and Dynamic Visual EffectsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661631(1248-1261)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661631
Show More Cited By

Index Terms

Teachable Reality: Prototyping Tangible Augmented Reality with Everyday Objects by Leveraging Interactive Machine Teaching
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Mixed / augmented reality

Recommendations

Multimodal augmented reality: the norm rather than the exception
MVAR '16: Proceedings of the 2016 workshop on Multimodal Virtual and Augmented Reality

Augmented reality (AR) is commonly seen as a technology that overlays virtual imagery onto a participant's view of the world. In line with this, most AR research is focused on what we see. In this paper, we challenge this focus on vision and make a case ...
Interactive Augmented Live Virtual Reality Streaming: A Health Care Application
ICMHI '18: Proceedings of the 2nd International Conference on Medical and Health Informatics

Virtual Reality (VR) technology has been around for decades; however, we have probably only begun to realize the practical applications until recent years. With increase of computing power, decrease of cost and physical dimension, a new class of ...
Haptics in Augmented Reality
ICMCS '99: Proceedings of the IEEE International Conference on Multimedia Computing and Systems - Volume 2

An augmented reality system merges synthetic sensory information into a user's perception of a three-dimensional environment. An important performance goal for an augmented reality system is that the user perceives a single seamless environment. In most ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

April 2023

14911 pages

ISBN:9781450394215

DOI:10.1145/3544548

Editors:
Albrecht Schmidt
LMU Munich, Germany60028717
,
Kaisa Väänänen
Tampere University, Finland60011170
,
Tesh Goyal
Google Research, USA60006191
,
Per Ola Kristensson
University of Cambridge, UK60031101
,
Anicia Peters
University of Namibia, Namibia60072704
,
Stefanie Mueller
Massachusetts Institute of Technology, USA60022195
,
Julie R. Williamson
University of Glasgow, UK60001490
,
Max L. Wilson
University of Nottingham, UK60015138

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 April 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

MiITACS
NSERC

Conference

CHI '23

Sponsor:

SIGCHI

CHI '23: CHI Conference on Human Factors in Computing Systems

April 23 - 28, 2023

Hamburg, Germany

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
1,648
Total Downloads

Downloads (Last 12 months)1,214
Downloads (Last 6 weeks)135

Reflects downloads up to 11 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hai-Jew S(2024)Using Artmaking Generative AIs to Support Augmented Reality Learning Designs With Adobe Aero AppInquiries of Pedagogical Shifts and Critical Mindsets Among Educators10.4018/979-8-3693-1078-6.ch006(132-152)Online publication date: 8-Mar-2024
https://doi.org/10.4018/979-8-3693-1078-6.ch006
Rumbelow MColes A(2024)The Promise of AI Object-Recognition in Learning Mathematics: An Explorative Study of 6-Year-Old Children’s Interactions with Cuisenaire Rods and the Blockplay.ai AppEducation Sciences10.3390/educsci1406059114:6(591)Online publication date: 30-May-2024
https://doi.org/10.3390/educsci14060591
Liao JVan KXia ZSuzuki R(2024)RealityEffects: Augmenting 3D Volumetric Videos with Object-Centric Annotation and Dynamic Visual EffectsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661631(1248-1261)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661631
Wei XYue ZChen H(2024)Physical OOP: Spatial Program Physical Objects Interactions2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW62533.2024.00164(735-736)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VRW62533.2024.00164
Suzuki RGonzalez-Franco MSra MLindlbauer D(2023)XR and AI: AI-Enabled Virtual, Augmented, and Mixed RealityAdjunct Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586182.3617432(1-3)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586182.3617432
Chen MLiang SSmith G(2023)Stackable Music: A Marker-Based Augmented Reality Music Synthesis GameCompanion Proceedings of the Annual Symposium on Computer-Human Interaction in Play10.1145/3573382.3616071(22-28)Online publication date: 6-Oct-2023
https://dl.acm.org/doi/10.1145/3573382.3616071
Chulpongsatorn NWillett WSuzuki R(2023)HoloTouch: Interacting with Mixed Reality Visualizations Through Smartphone ProxiesExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585738(1-8)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585738

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Table of Contents