research-article

Open access

ScalAR: Authoring Semantically Adaptive Augmented Reality Experiences in Virtual Reality

Authors:

Ananya Ipsita, and

Karthik RamaniAuthors Info & Claims

CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

April 2022

Article No.: 65, Pages 1 - 18

https://doi.org/10.1145/3491102.3517665

Published: 28 April 2022 Publication History

All formats PDF

Abstract

Augmented Reality (AR) experiences tightly associate virtual contents with environmental entities. However, the dissimilarity of different environments limits the adaptive AR content behaviors under large-scale deployment. We propose ScalAR, an integrated workflow enabling designers to author semantically adaptive AR experiences in Virtual Reality (VR). First, potential AR consumers collect local scenes with a semantic understanding technique. ScalAR then synthesizes numerous similar scenes. In VR, a designer authors the AR contents’ semantic associations and validates the design while being immersed in the provided scenes. We adopt a decision-tree-based algorithm to fit the designer’s demonstrations as a semantic adaptation model to deploy the authored AR experience in a physical scene. We further showcase two application scenarios authored by ScalAR and conduct a two-session user study where the quantitative results prove the accuracy of the AR content rendering and the qualitative results show the usability of ScalAR.

Supplementary Material

MP4 File (3491102.3517665-video-figure.mp4)

Video Figure

Download
154.53 MB

MP4 File (3491102.3517665-video-preview.mp4)

Video Preview

Download
16.90 MB

MP4 File (3491102.3517665-talk-video.mp4)

Talk Video

Download
39.72 MB

References

[1]

ARCore 2021. ARCore. https://developers.google.com/ar.

[2]

ARKit 2021. ARKit. https://developer.apple.com/augmented-reality/arkit.

[3]

Eric A Bier. 1990. Snap-dragging in three dimensions. ACM SIGGRAPH Computer Graphics 24, 2 (1990), 193–204.

Digital Library

[4]

Mark Billinghurst, Adrian Clark, and Gun Lee. 2015. A survey of augmented reality. (2015).

[5]

Yuanzhi Cao, Xun Qian, Tianyi Wang, Rachel Lee, Ke Huo, and Karthik Ramani. 2020. An Exploratory Study of Augmented Reality Presence for Tutoring Machine Tasks. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.

Digital Library

[6]

Yuanzhi Cao, Tianyi Wang, Xun Qian, Pawan S Rao, Manav Wadhawan, Ke Huo, and Karthik Ramani. 2019. GhostAR: A time-space editor for embodied authoring of human-robot collaborative task with augmented reality. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology. 521–534.

Digital Library

[7]

Han Joo Chae, Jeong-in Hwang, and Jinwook Seo. 2018. Wall-based space manipulation technique for efficient placement of distant objects in augmented reality. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology. 45–52.

Digital Library

[8]

Lung-Pan Cheng, Eyal Ofek, Christian Holz, and Andrew D Wilson. 2019. Vroamer: generating on-the-fly VR experiences while walking inside large, unknown real-world building environments. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 359–366.

[9]

Yifei Cheng, Yukang Yan, Xin Yi, Yuanchun Shi, and David Lindlbauer. 2021. SemanticAdapt: Optimization-based Adaptation of Mixed Reality Layouts Leveraging Virtual-Physical Semantic Connections. In The 34th Annual ACM Symposium on User Interface Software and Technology. 282–297.

[10]

Hung-Lin Chi, Shih-Chung Kang, and Xiangyu Wang. 2013. Research trends and opportunities of augmented reality applications in architecture, engineering, and construction. Automation in construction 33 (2013), 116–122.

[11]

Subramanian Chidambaram, Hank Huang, Fengming He, Xun Qian, Ana M Villanueva, Thomas S Redick, Wolfgang Stuerzlinger, and Karthik Ramani. 2021. ProcessAR: An Augmented Reality-Based Tool to Create In-Situ Procedural 2D/3D AR Instructions. In Proceedings of the Designing Interactive Systems Conference.

Digital Library

[12]

Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. 2017. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5828–5839.

[13]

Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Sanja Fidler, Antonino Furnari, Evangelos Kazakos, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, 2018. Scaling egocentric vision: The epic-kitchens dataset. In Proceedings of the European Conference on Computer Vision (ECCV). 720–736.

[14]

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition. Ieee, 248–255.

[15]

Zhi-Chao Dong, Wenming Wu, Zenghao Xu, Qi Sun, Guanjie Yuan, Ligang Liu, and Xiao-Ming Fu. 2021. Tailored Reality: Perception-aware Scene Restructuring for Adaptive VR Navigation. ACM Transactions on Graphics (TOG) 40, 5 (2021), 1–15.

Digital Library

[16]

Ruofei Du, Eric Turner, Maksym Dzitsiuk, Luca Prasso, Ivo Duarte, Jason Dourgarian, Joao Afonso, Jose Pascoal, Josh Gladstone, Nuno Cruces, 2020. DepthLab: Real-Time 3D Interaction With Depth Maps for Mobile Augmented Reality. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology. 829–843.

Digital Library

[17]

Barrett Ens, Fraser Anderson, Tovi Grossman, Michelle Annett, Pourang Irani, and George Fitzmaurice. 2017. Ivy: Exploring spatially situated visual programming for authoring and understanding intelligent environments. In Proceedings of the 43rd Graphics Interface Conference. 156–162.

Digital Library

[18]

Barrett Ens, Eyal Ofek, Neil Bruce, and Pourang Irani. 2015. Spatial constancy of surface-embedded layouts across multiple environments. In Proceedings of the 3rd ACM Symposium on Spatial User Interaction. 65–68.

Digital Library

[19]

Mark Fiala. 2005. ARTag, a fiducial marker system using digital techniques. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), Vol. 2. IEEE, 590–596.

Digital Library

[20]

Scott Frees, G Drew Kessler, and Edwin Kay. 2007. PRISM interaction for enhancing control in immersive virtual environments. ACM Transactions on Computer-Human Interaction (TOCHI) 14, 1(2007), 2–es.

Digital Library

[21]

Yoav Freund. 2001. An adaptive version of the boost by majority algorithm. Machine Learning 43, 3 (2001), 293–318.

Digital Library

[22]

Qiang Fu, Xiaowu Chen, Xiaotian Wang, Sijia Wen, Bin Zhou, and Hongbo Fu. 2017. Adaptive synthesis of indoor scenes via activity-associated object relation graphs. ACM Transactions on Graphics (TOG) 36, 6 (2017), 1–13.

Digital Library

[23]

Yasutaka Furukawa, Brian Curless, Steven M Seitz, and Richard Szeliski. 2009. Manhattan-world stereo. In 2009 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 1422–1429.

[24]

Ran Gal, Lior Shapira, Eyal Ofek, and Pushmeet Kohli. 2014. FLARE: Fast layout for augmented reality applications. In 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 207–212.

[25]

Paul Grimm, Michael Haller, Volker Paelke, Silvan Reinhold, Christian Reimann, and R Zauner. 2002. AMIRE-authoring mixed reality. In The First IEEE International Workshop Agumented Reality Toolkit,. IEEE, 2–pp.

[26]

Saurabh Gupta, Pablo Arbeláez, Ross Girshick, and Jitendra Malik. 2015. Indoor scene understanding with rgb-d images: Bottom-up segmentation, object detection and semantic segmentation. International Journal of Computer Vision 112, 2 (2015), 133–149.

Digital Library

[27]

Taejin Ha, Woontack Woo, Youngho Lee, Junhun Lee, Jeha Ryu, Hankyun Choi, and Kwanheng Lee. 2010. ARtalet: tangible user interface based immersive augmented reality authoring tool for Digilog book. In 2010 International Symposium on Ubiquitous Virtual Reality. IEEE, 40–43.

Digital Library

[28]

Lei Han, Tian Zheng, Yinheng Zhu, Lan Xu, and Lu Fang. 2020. Live semantic 3d perception for immersive augmented reality. IEEE Transactions on Visualization and Computer Graphics 26, 5(2020), 2012–2022.

[29]

Jeremy Hartmann, Christian Holz, Eyal Ofek, and Andrew D Wilson. 2019. Realitycheck: Blending virtual environments with situated physical reality. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12.

Digital Library

[30]

Stefan Hartmann, Björn Krüger, and Reinhard Klein. 2015. Content-aware re-targeting of discrete element layouts. (2015).

[31]

Devamardeep Hayatpur, Seongkook Heo, Haijun Xia, Wolfgang Stuerzlinger, and Daniel Wigdor. 2019. Plane, ray, and point: Enabling precise spatial manipulations with shape constraints. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology. 1185–1195.

Digital Library

[32]

Alexander Hermans, Georgios Floros, and Bastian Leibe. 2014. Dense 3d semantic mapping of indoor scenes from rgb-d images. In 2014 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2631–2638.

[33]

Dominik Herr, Jan Reinhardt, Guido Reina, Robert Krüger, Rafael V Ferrari, and Thomas Ertl. 2018. Immersive modular factory layout planning using augmented reality. Procedia CIRP 72(2018), 1112–1117.

[34]

Horizon Workrooms 2021. Horizon Workrooms for VR Remote Collaboration | Meta. https://about.fb.com/news/2021/08/introducing-horizon-workrooms-remote-collaboration-reimagined/.

[35]

Ke Huo, Yuanzhi Cao, Sang Ho Yoon, Zhuangying Xu, Guiming Chen, and Karthik Ramani. 2018. Scenariot: Spatially mapping smart things within augmented reality scenes. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–13.

Digital Library

[36]

Ke Huo and Karthik Ramani. 2017. Window-shaping: 3d design ideation by creating on, borrowing from, and looking at the physical world. In Proceedings of the Eleventh International Conference on Tangible, Embedded, and Embodied Interaction. 37–45.

Digital Library

[37]

Ahmed Hussein, Mohamed Medhat Gaber, Eyad Elyan, and Chrisina Jayne. 2017. Imitation learning: A survey of learning methods. ACM Computing Surveys (CSUR) 50, 2 (2017), 1–35.

Digital Library

[38]

Hans-Christian Jetter, Roman Rädle, Tiare Feuchtner, Christoph Anthes, Judith Friedl, and Clemens Nylandsted Klokmose. 2020. ” In VR, everything is possible!”: Sketching and Simulating Spatially-Aware Interactive Spaces in Virtual Reality. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–16.

Digital Library

[39]

Dongsik Jo and Gerard Jounghyun Kim. 2016. ARIoT: scalable augmented reality framework for interacting with Internet of Things appliances everywhere. IEEE Transactions on Consumer Electronics 62, 3 (2016), 334–340.

Digital Library

[40]

Peter Kán and Hannes Kaufmann. 2017. Automated interior design using a genetic algorithm. In Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology. 1–10.

Digital Library

[41]

Peter Kán and Hannes Kaufmann. 2018. Automatic furniture arrangement using greedy cost minimization. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 491–498.

[42]

Max Krichenbauer, Goshiro Yamamoto, Takafumi Taketom, Christian Sandor, and Hirokazu Kato. 2017. Augmented reality versus virtual reality for 3d object manipulation. IEEE Transactions on Visualization and Computer Graphics 24, 2(2017), 1038–1048.

[43]

Kin Chung Kwan and Hongbo Fu. 2019. Mobi3dsketch: 3d sketching in mobile ar. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–11.

Digital Library

[44]

Yining Lang, Wei Liang, and Lap-Fai Yu. 2019. Virtual agent positioning driven by scene semantics in mixed reality. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 767–775.

[45]

Tobias Langlotz, Stefan Mooslechner, Stefanie Zollmann, Claus Degendorfer, Gerhard Reitmayr, and Dieter Schmalstieg. 2012. Sketching up the world: in situ authoring for mobile augmented reality. Personal and Ubiquitous Computing 16, 6 (2012), 623–630.

Digital Library

[46]

Florian Ledermann and Dieter Schmalstieg. 2005. APRIL: a high-level framework for creating augmented reality presentations. In Proceedings of the IEEE Virtual Reality. IEEE, 187–194.

[47]

Gun A Lee, Gerard J Kim, and Mark Billinghurst. 2005. Immersive authoring: What you experience is what you get (wyxiwyg). Commun. ACM 48, 7 (2005), 76–81.

Digital Library

[48]

Germán Leiva, Jens Emil Grønbæk, Clemens Nylandsted Klokmose, Cuong Nguyen, Rubaiat Habib Kazi, and Paul Asente. 2021. Rapido: Prototyping Interactive AR Experiences through Programming by Demonstration. In The 34th Annual ACM Symposium on User Interface Software and Technology. 626–637.

Digital Library

[49]

Germán Leiva, Cuong Nguyen, Rubaiat Habib Kazi, and Paul Asente. 2020. Pronto: Rapid augmented reality video prototyping using sketches and enaction. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.

Digital Library

[50]

Wei Liang, Xinzhe Yu, Rawan Alghofaili, Yining Lang, and Lap-Fai Yu. 2021. Scene-Aware Behavior Synthesis for Virtual Pets in Mixed Reality. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–12.

Digital Library

[51]

Chuan-en Lin, Ta Ying Cheng, and Xiaojuan Ma. 2020. ARchitect: Building Interactive Virtual Experiences from Physical Affordances by Bringing Human-in-the-Loop. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.

[52]

David Lindlbauer, Anna Maria Feit, and Otmar Hilliges. 2019. Context-aware online adaptation of mixed reality interfaces. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology. 147–160.

Digital Library

[53]

David Lindlbauer and Andy D Wilson. 2018. Remixed reality: Manipulating space and time in augmented reality. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–13.

Digital Library

[54]

Jingyang Liu. 2021. Semantic Mapping: A Semantics-based Approach to Virtual Content Placement for Immersive Environments. In 2021 17th International Conference on Intelligent Environments (IE). IEEE, 1–8.

[55]

Blair MacIntyre, Maribeth Gandy, Steven Dow, and Jay David Bolter. 2004. DART: a toolkit for rapid design exploration of augmented reality experiences. In Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology. 197–206.

Digital Library

[56]

Sebastian Marwecki and Patrick Baudisch. 2018. Scenograph: Fitting real-walking vr experiences into various tracking volumes. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology. 511–520.

Digital Library

[57]

Daniel Mendes, Filipe Relvas, Alfredo Ferreira, and Joaquim Jorge. 2016. The benefits of dof separation in mid-air 3d object manipulation. In Proceedings of the 22nd ACM Conference on Virtual Reality Software and Technology. 261–268.

Digital Library

[58]

Microsoft HoloLens 2021. Microsoft HoloLens | Mixed Reality Technology for Business. https://www.microsoft.com/en-us/hololens.

[59]

Mark Mine. 1995. ISAAC: A virtual environment tool for the interactive construction of virtual worlds. (1995).

Digital Library

[60]

MRTK Spatial Awareness 2021. Spatial Awareness - Microsoft Mixed Reality Toolkit. hhttps://docs.microsoft.com/en-us/windows/mixed-reality/mrtk-unity/features/spatial-awareness/spatial-awareness-getting-started?view=mrtkunity-2021-05.

[61]

Michael Nebeling, Katy Lewis, Yu-Cheng Chang, Lihan Zhu, Michelle Chung, Piaoyang Wang, and Janet Nebeling. 2020. XRDirector: A role-based collaborative immersive authoring system. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–12.

Digital Library

[62]

Andrew YC Nee, SK Ong, George Chryssolouris, and Dimitris Mourtzis. 2012. Augmented reality applications in design and manufacturing. CIRP annals 61, 2 (2012), 657–679.

[63]

Gary Ng, Joon Gi Shin, Alexander Plopski, Christian Sandor, and Daniel Saakes. 2018. Situated game level editing in augmented reality. In Proceedings of the Twelfth International Conference on Tangible, Embedded, and Embodied Interaction. 409–418.

Digital Library

[64]

Yinyu Nie, Xiaoguang Han, Shihui Guo, Yujian Zheng, Jian Chang, and Jian Jun Zhang. 2020. Total3dunderstanding: Joint layout, object pose and mesh reconstruction for indoor scenes from a single image. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 55–64.

[65]

Benjamin Nuernberger, Eyal Ofek, Hrvoje Benko, and Andrew D Wilson. 2016. Snaptoreality: Aligning augmented reality to the real world. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 1233–1244.

Digital Library

[66]

Oculus Quest 2 2021. Oculus Quest 2: Our Most Advanced New All-in-One VR Headset | Oculus. https://www.oculus.com/quest-2.

[67]

Oculus Store 2021. Oculus Store: VR Games, Apps, and More | Oculus. https://www.oculus.com/experiences/quest.

[68]

Viet Toan Phan and Seung Yeon Choo. 2010. Interior design in augmented reality environment. International Journal of Computer Applications 5, 5(2010), 16–21.

[69]

Wayne Piekarski and Bruce Thomas. 2002. ARQuake: the outdoor augmented reality gaming system. Commun. ACM 45, 1 (2002), 36–38.

Digital Library

[70]

Arnaud Prouzeau, Yuchen Wang, Barrett Ens, Wesley Willett, and Tim Dwyer. 2020. Corsican twin: Authoring in situ augmented reality visualisations in virtual reality. In Proceedings of the International Conference on Advanced Visual Interfaces. 1–9.

Digital Library

[71]

Francisco J Romero-Ramirez, Rafael Muñoz-Salinas, and Rafael Medina-Carnicer. 2018. Speeded up detection of squared fiducial markers. Image and vision Computing 76 (2018), 38–47.

[72]

Rafael Sacks, Amotz Perlman, and Ronen Barak. 2013. Construction safety training using immersive virtual reality. Construction Management and Economics 31, 9 (2013), 1005–1017.

[73]

Hartmut Seichter, Julian Looser, and Mark Billinghurst. 2008. ComposAR: An intuitive tool for authoring AR applications. In 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality. IEEE, 177–178.

Digital Library

[74]

Tobias Sielhorst, Marco Feuerstein, and Nassir Navab. 2008. Advanced medical displays: A literature review of augmented reality. Journal of Display Technology 4, 4 (2008), 451–467.

[75]

Spatial 2021. Spatial - Metaverse Spaces That Bring Us Together. https://spatial.io.

[76]

Misha Sra, Sergio Garrido-Jurado, and Pattie Maes. 2017. Oasis: Procedurally generated social virtual spaces from 3d scanned real spaces. IEEE Transactions on Visualization and Computer Graphics 24, 12(2017), 3174–3187.

[77]

Tomu Tahara, Takashi Seno, Gaku Narita, and Tomoya Ishikawa. 2020. Retargetable AR: Context-aware Augmented Reality in Indoor Scenes based on 3D Scene Graph. In 2020 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). IEEE, 249–255.

[78]

Tilt Brush 2021. Tilt Brush by Google. https://www.tiltbrush.com.

[79]

Shubham Tulsiani, Saurabh Gupta, David F Fouhey, Alexei A Efros, and Jitendra Malik. 2018. Factoring shape, pose, and layout from the 2d image of a 3d scene. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 302–310.

[80]

Unity3D 2021. Unity3D. https://unity.com.

[81]

Unreal 2021. Unreal. https://www.unrealengine.com.

[82]

Julien Valentin, Vibhav Vineet, Ming-Ming Cheng, David Kim, Jamie Shotton, Pushmeet Kohli, Matthias Nießner, Antonio Criminisi, Shahram Izadi, and Philip Torr. 2015. Semanticpaint: Interactive 3d labeling and learning at your fingertips. ACM Transactions on Graphics (TOG) 34, 5 (2015), 1–17.

Digital Library

[83]

Khrystyna Vasylevska, Hannes Kaufmann, Mark Bolas, and Evan A Suma. 2013. Flexible spaces: Dynamic layout generation for infinite walking in virtual environments. In 2013 IEEE Symposium on 3D User Interfaces (3DUI). IEEE, 39–42.

[84]

Manuel Veit, Antonio Capobianco, and Dominique Bechmann. 2009. Influence of degrees of freedom’s manipulation on performances during orientation tasks in virtual reality environments. In Proceedings of the 16th ACM Symposium on Virtual Reality Software and Technology. 51–58.

Digital Library

[85]

Vuforia 2021. Vuforia. https://www.ptc.com/en/products/vuforia.

[86]

Daniel Wagner and Dieter Schmalstieg. 2007. Artoolkitplus for pose tracking on mobile devices. (2007).

[87]

Tianyi Wang, Xun Qian, Fengming He, Xiyun Hu, Yuanzhi Cao, and Karthik Ramani. 2021. GesturAR: An Authoring System for Creating Freehand Interactive Augmented Reality Applications. In The 34th Annual ACM Symposium on User Interface Software and Technology. 552–567.

[88]

Tianyi Wang, Xun Qian, Fengming He, Xiyun Hu, Ke Huo, Yuanzhi Cao, and Karthik Ramani. 2020. CAPturAR: An Augmented Reality Tool for Authoring Human-Involved Context-Aware Applications. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology. 328–341.

Digital Library

[89]

Zeyu Wang, Cuong Nguyen, Paul Asente, and Julie Dorsey. 2021. DistanciAR: Authoring Site-Specific Augmented Reality Experiences for Remote Environments. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–12.

Digital Library

[90]

Hsin-Kai Wu, Silvia Wen-Yu Lee, Hsin-Yi Chang, and Jyh-Chong Liang. 2013. Current status, opportunities and challenges of augmented reality in education. Computers & education 62 (2013), 41–49.

[91]

Ting-Fan Wu, Chih-Jen Lin, and Ruby C Weng. 2004. Probability estimates for multi-class classification by pairwise coupling. Journal of Machine Learning Research 5, Aug (2004), 975–1005.

Digital Library

[92]

Haijun Xia, Sebastian Herscher, Ken Perlin, and Daniel Wigdor. 2018. Spacetime: Enabling fluid individual and collaborative editing in virtual reality. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology. 853–866.

Digital Library

[93]

Hui Ye, Kin Chung Kwan, Wanchao Su, and Hongbo Fu. 2020. ARAnimator: in-situ character animation in mobile AR with user-defined motion gestures. ACM Transactions on Graphics (TOG) 39, 4 (2020), 83–1.

Digital Library

[94]

Lap Fai Yu, Sai Kit Yeung, Chi Keung Tang, Demetri Terzopoulos, Tony F Chan, and Stanley J Osher. 2011. Make it home: automatic optimization of furniture arrangement. ACM Transactions on Graphics (TOG)-Proceedings of ACM SIGGRAPH 2011, v. 30,(4), July 2011, article no. 86 30, 4 (2011).

Digital Library

[95]

Ya-Ting Yue, Yong-Liang Yang, Gang Ren, and Wenping Wang. 2017. SceneCtrl: Mixed reality enhancement via efficient scene editing. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology. 427–436.

Digital Library

[96]

Hui Zhang. 2017. Head-mounted display-based intuitive virtual reality training system for the mining industry. International Journal of Mining Science and Technology 27, 4(2017), 717–722.

[97]

Lei Zhang and Steve Oney. 2020. FlowMatic: An Immersive Authoring Tool for Creating Interactive Scenes in Virtual Reality. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology. 342–353.

Digital Library

[98]

Egui Zhu, Arash Hadadgar, Italo Masiello, and Nabil Zary. 2014. Augmented reality in healthcare education: an integrative review. PeerJ 2(2014), e469.

Cited By

Chen JZhao LLi YXie ZWilensky UHorn M(2024)“Oh My God! It’s Recreating Our Room!” Understanding Children’s Experiences with A Room-Scale Augmented Reality Authoring ToolkitProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642043(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642043
Liu JZhang RSawabe TFujimoto YKanbara MKato H(2024)Integrating Spatial Design with Reality: An AR Game Scene Generation Approach2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW62533.2024.00154(715-716)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VRW62533.2024.00154
Yang JQiu LCorona-Moreno EShi LBui HLam MLanday J(2024)AMMA: Adaptive Multimodal Assistants Through Automated State Tracking and User Model-Directed Guidance Planning2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00108(892-902)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VR58804.2024.00108
Show More Cited By

Index Terms

ScalAR: Authoring Semantically Adaptive Augmented Reality Experiences in Virtual Reality
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Mixed / augmented reality
      2. Virtual reality
    2. Interactive systems and tools

Recommendations

Extending Virtual Reality Display Wall Environments Using Augmented Reality
SUI '19: Symposium on Spatial User Interaction

Two major form factors for virtual reality are head-mounted displays and large display environments such as CAVE®and the LCD-based successor CAVE2®. Each of these has distinct advantages and limitations based on how they’re used. This work explores ...
Read More
Haptics in Augmented Reality
ICMCS '99: Proceedings of the IEEE International Conference on Multimedia Computing and Systems - Volume 2

An augmented reality system merges synthetic sensory information into a user's perception of a three-dimensional environment. An important performance goal for an augmented reality system is that the user perceives a single seamless environment. In most ...
Read More
Speed reading on virtual reality and augmented reality
Abstract
Many virtual reality (VR) and augmented reality (AR) applications in education require speed reading. The current study aimed to explore whether the reading performance on VR and AR is different from that on traditional desktop display,...
Highlights
- We explored performance of speed reading on virtual and augmented reality.
- ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

April 2022

10459 pages

ISBN:9781450391573

DOI:10.1145/3491102

Editors:
Simone Barbosa
PUC-Rio, Brazil
,
Cliff Lampe
University of Michigan, USA
,
Caroline Appert
Université Paris-Saclay, France
,
David A. Shamma
Toyota Research Institute, USA
,
Steven Drucker
Microsoft Research, USA
,
Julie Williamson
University of Glasgow, UK
,
Koji Yatani
University of Tokyo, Japan

Copyright © 2022 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 April 2022

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

NSF (National Science Foundation)

Conference

CHI '22

Sponsor:

SIGCHI

CHI '22: CHI Conference on Human Factors in Computing Systems

April 29 - May 5, 2022

LA, New Orleans, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI PLAY '24

Sponsor:
sigchi

The Annual Symposium on Computer-Human Interaction in Play

October 14 - 17, 2024

Tampere , Finland

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
2,447
Total Downloads

Downloads (Last 12 months)976
Downloads (Last 6 weeks)66

Other Metrics

View Author Metrics

Citations

Cited By

Chen JZhao LLi YXie ZWilensky UHorn M(2024)“Oh My God! It’s Recreating Our Room!” Understanding Children’s Experiences with A Room-Scale Augmented Reality Authoring ToolkitProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642043(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642043
Liu JZhang RSawabe TFujimoto YKanbara MKato H(2024)Integrating Spatial Design with Reality: An AR Game Scene Generation Approach2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW62533.2024.00154(715-716)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VRW62533.2024.00154
Yang JQiu LCorona-Moreno EShi LBui HLam MLanday J(2024)AMMA: Adaptive Multimodal Assistants Through Automated State Tracking and User Model-Directed Guidance Planning2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00108(892-902)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VR58804.2024.00108
Kim SKim DShin JWoo W(2024)Object Cluster Registration of Dissimilar Rooms Using Geometric Spatial Affordance Graph to Generate Shared Virtual Spaces2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00099(796-805)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VR58804.2024.00099
Patil SSellent AGerndt AAlbuquerque G(2024)Improved RGB-D Indoor Semantic Segmentation using Cascaded Loss Fusion2024 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR)10.1109/AIxVR59861.2024.00024(119-127)Online publication date: 17-Jan-2024
https://doi.org/10.1109/AIxVR59861.2024.00024
Han VCho HMaeda KIon ALindlbauer D(2023)BlendMR: A Computational Method to Create Ambient Mixed Reality InterfacesProceedings of the ACM on Human-Computer Interaction10.1145/36264727:ISS(217-241)Online publication date: 1-Nov-2023
https://dl.acm.org/doi/10.1145/3626472
Auda JGruenefeld UFaltaous SMayer SSchneegass S(2023)A Scoping Survey on Cross-reality SystemsACM Computing Surveys10.1145/361653656:4(1-38)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3616536
Yu ZZeidler DVictor VMcginity M(2023)Dynascape : Immersive Authoring of Real-World Dynamic Scenes with Spatially Tracked RGB-D VideosProceedings of the 29th ACM Symposium on Virtual Reality Software and Technology10.1145/3611659.3615718(1-12)Online publication date: 9-Oct-2023
https://dl.acm.org/doi/10.1145/3611659.3615718
Chen CNguyen CHoffswell JHealey JBui TWeibel N(2023)PaperToPlace: Transforming Instruction Documents into Spatialized and Context-Aware Mixed Reality ExperiencesProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606832(1-21)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586183.3606832
Stemasov ELedo DFitzmaurice GAnderson F(2023)Immersive Sampling: Exploring Sampling for Future Creative Practices in Media-Rich, Immersive SpacesProceedings of the 2023 ACM Designing Interactive Systems Conference10.1145/3563657.3596131(212-229)Online publication date: 10-Jul-2023
https://dl.acm.org/doi/10.1145/3563657.3596131
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents