research-article

Augmented Reality Based Video Shooting Guidance for Novice Users

Authors:

Yuxin XuAuthors Info & Claims

Proceedings of the ACM on Human-Computer Interaction, Volume 6, Issue MHCI

Article No.: 215, Pages 1 - 20

https://doi.org/10.1145/3546750

Published: 20 September 2022 Publication History

Abstract

Using mobile phones to shoot video is considerably common in our daily life. However, novice users have difficulty in controlling the camera properly due to lack of professional knowledge and skill. In this paper, in order to assist novice users in learning and imitating professional camera movement from watching high quality sample videos, we propose ARCAM, an Augmented Reality (AR) based video shooting guidance method for novice users. Using AR, we visualized the concept of camera movement and embedded it into natural scene to provide real-time guidance. User can follow the guidance while shooting video by matching a calibration frame to the guidance, to achieve the desired camera movement. We conducted a user study comparing the effectiveness of ARCAM to a traditional static arrow guidance. Results showed that ARCAM was more effective in helping users understand the camera work in the sample videos and move the camera with more accuracy. Our work provides insights on designing mobile video shooting application and suggests that AR has great potential in assisting novice video shooters.

References

[1]

Alex M Andrew. 2001. Multiple view geometry in computer vision. Kybernetes (2001).

[2]

Soonmin Bae, Aseem Agarwala, and Frédo Durand. 2010. Computational Rephotography. ACM Trans. Graph., Vol. 29, 3, Article 24 (June 2010), 15 pages. https://doi.org/10.1145/1805964.1805968

Digital Library

[3]

Sebastian Büttner, Michael Prilla, and Carsten Röcker. 2020. Augmented Reality Training for Industrial Assembly Work-Are Projection-based AR Assistive Systems an Appropriate Tool for Assembly Training?. In Proceedings of the 2020 CHI conference on human factors in computing systems. Association for Computing Machinery, New York, NY, USA, 1--12. https://doi.org/10.1145/3313831.3376720

Digital Library

[4]

Fang Chen, Xiwen Cui, Boxuan Han, Jia Liu, Xinran Zhang, and Hongen Liao. 2021. Augmented reality navigation for minimally invasive knee surgery using enhanced arthroscopy. Computer Methods and Programs in Biomedicine, Vol. 201 (April 2021), 105952. https://doi.org/10.1016/j.cmpb.2021.105952

Digital Library

[5]

Fiona Draxler, Audrey Labrie, Albrecht Schmidt, and Lewis L. Chuang. 2020. Augmented Reality to Enable Users in Learning Case Grammar from Their Real-World Interactions. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). Association for Computing Machinery, New York, NY, USA, 1??2. https://doi.org/10.1145/3313831.3376537

Digital Library

[6]

Jane L. E, Ohad Fried, and Maneesh Agrawala. 2019. Optimizing Portrait Lighting at Capture-Time Using a 360 Camera as a Light Probe. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (New Orleans, LA, USA) (UIST '19). Association for Computing Machinery, New York, NY, USA, 221-32. https://doi.org/10.1145/3332165.3347893

Digital Library

[7]

Jane L. E, Ohad Fried, Jingwan Lu, Jianming Zhang, Radom'ir Mech, Jose Echevarria, Pat Hanrahan, and James A. Landay. 2020. Adaptive Photographic Composition Guidance. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1-3. https://doi.org/10.1145/3313831.3376635

Digital Library

[8]

Min Fan, Uddipana Baishya, Elgin-Skye Mclaren, Alissa N. Antle, Shubhra Sarker, and Amal Vincent. 2018. Block Talks: A Tangible and Augmented Reality Toolkit for Children to Learn Sentence Construction. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI EA '18). Association for Computing Machinery, New York, NY, USA, 1??. https://doi.org/10.1145/3170427.3188576

Digital Library

[9]

Farshid Farhat, Mohammad Mahdi Kamani, and James Z. Wang. 2022. CAPTAIN: Comprehensive Composition Assistance for Photo Taking. ACM Trans. Multimedia Comput. Commun. Appl., Vol. 18, 1, Article 14 (jan 2022), 24 pages. https://doi.org/10.1145/3462762

Digital Library

[10]

Pierre Fite-Georgel. 2011. Is there a reality in Industrial Augmented Reality?. In 2011 10th IEEE International Symposium on Mixed and Augmented Reality. 201--210. https://doi.org/10.1109/ISMAR.2011.6092387

Digital Library

[11]

Markus Funk, Thomas Kosch, and Albrecht Schmidt. 2016. Interactive Worker Assistance: Comparing the Effects of in-Situ Projection, Head-Mounted Displays, Tablet, and Paper Instructions. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (Heidelberg, Germany) (UbiComp '16). Association for Computing Machinery, New York, NY, USA, 934-39. https://doi.org/10.1145/2971648.2971706

Digital Library

[12]

John Hart. 2013. The Art of the Storyboard: A filmmaker's introduction. Taylor & Francis.

[13]

Florian Heinrich, Florentine Huettl, Gerd Schmidt, Markus Paschold, Werner Kneist, Tobias Huber, and Christian Hansen. 2021. HoloPointer: a virtual augmented reality pointer for laparoscopic surgery training. International Journal of Computer Assisted Radiology and Surgery, Vol. 16, 1 (Jan. 2021), 161--168. https://doi.org/10.1007/s11548-020-02272--2

[14]

Chuan-Shen Hu, Yi-Tsung Hsieh, Hsiao-Wei Lin, and Mei-Chen Yeh. 2019. Virtual Portraitist: An Intelligent Tool for Taking Well-Posed Selfies. ACM Trans. Multimedia Comput. Commun. Appl., Vol. 15, 1s, Article 12 (jan 2019), 17 pages. https://doi.org/10.1145/3288760

Digital Library

[15]

Yu-Hsuan Huang, Hao-Yu Chang, Wan-ling Yang, Yu-Kai Chiu, Tzu-Chieh Yu, Pei-Hsuan Tsai, and Ming Ouhyoung. 2018. CatAR: A Novel Stereoscopic Augmented Reality Cataract Surgery Training System with Dexterous Instruments Tracking Technology. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18). Association for Computing Machinery, New York, NY, USA, 1-2. https://doi.org/10.1145/3173574.3174039

Digital Library

[16]

Bernd Huber, Hijung Valentina Shin, Bryan Russell, Oliver Wang, and Gautham J. Mysore. 2019. B-Script: Transcript-Based B-Roll Video Editing with Recommendations. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1-1. https://doi.org/10.1145/3290605.3300311

Digital Library

[17]

Adam Ibrahim, Brandon Huynh, Jonathan Downey, Tobias H?llerer, Dorothy Chun, and John O'donovan. 2018. ARbis Pictus: A Study of Vocabulary Learning with Augmented Reality. IEEE Transactions on Visualization and Computer Graphics, Vol. 24, 11 (Nov. 2018), 2867--2874. https://doi.org/10.1109/TVCG.2018.2868568

[18]

Steven Douglas Katz and Steve Katz. 1991. Film directing shot by shot: visualizing from concept to screen. Gulf Professional Publishing.

[19]

Christopher Kenworthy. 2009. Master shots: 100 advanced camera techniques to get an expensive look on your low-budget movie. Michael Wiese Productions.

[20]

Joy Kim, Mira Dontcheva, Wilmot Li, Michael S. Bernstein, and Daniela Steinsapir. 2015. Motif: Supporting Novice Creativity through Expert Patterns. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (Seoul, Republic of Korea) (CHI '15). Association for Computing Machinery, New York, NY, USA, 1211-220. https://doi.org/10.1145/2702123.2702507

Digital Library

[21]

Minju Kim and Jungjin Lee. 2019. PicMe: Interactive Visual Guidance for Taking Requested Photo Composition. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1-2. https://doi.org/10.1145/3290605.3300625

Digital Library

[22]

M. Kumano, K. Uehara, and Y. Ariki. 2006. Online Training-Oriented Video Shooting Navigation System Based on Real-Time Camerawork Evaluation. In 2006 IEEE International Conference on Multimedia and Expo. 1281--1284. https://doi.org/10.1109/ICME.2006.262772

[23]

Liang Li, Jian Yang, Yakui Chu, Wenbo Wu, Jin Xue, Ping Liang, and Lei Chen. 2016. A Novel Augmented Reality Navigation System for Endoscopic Sinus and Skull Base Surgery: A Feasibility Study. PLOS ONE, Vol. 11, 1 (Jan. 2016), 1--17. https://doi.org/10.1371/journal.pone.0146996

[24]

Qifan Li and Daniel Vogel. 2017. Guided Selfies Using Models of Portrait Aesthetics. In Proceedings of the 2017 Conference on Designing Interactive Systems (Edinburgh, United Kingdom) (DIS '17). Association for Computing Machinery, New York, NY, USA, 179-90. https://doi.org/10.1145/3064663.3064700

Digital Library

[25]

Kuo-Yen Lo, Keng-Hao Liu, and Chu-Song Chen. 2013. Intelligent Photographing Interface with On-Device Aesthetic Quality Assessment. In Computer Vision - ACCV 2012 Workshops. Springer Berlin Heidelberg, Berlin, Heidelberg, 533--544. https://doi.org/10.1007/978--3--642--37484--5_43

[26]

Chen Lujun, Yao Hongxun, Sun Xiaoshuai, and Zhang Hongming. 2012. Real-Time Viewfinder Composition Assessment and Recommendation to Mobile Photographing. In Advances in Multimedia Information Processing -- PCM 2012. Springer Berlin Heidelberg, Berlin, Heidelberg, 707--714. https://doi.org/10.1007/978--3--642--34778--8_66

Digital Library

[27]

Yiwen Luo and Xiaoou Tang. 2008. Photo and Video Quality Evaluation: Focusing on the Subject. In Computer Vision -- ECCV 2009. Springer Berlin Heidelberg, Berlin, Heidelberg, 386--399.

[28]

Candice Lusk and Michael D. Jones. 2019. Cake Cam: Take Your Photo and Be in It Too. In Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services (Taipei, Taiwan) (MobileHCI '19). Association for Computing Machinery, New York, NY, USA, Article 12, 9 pages. https://doi.org/10.1145/3338286.3340123

Digital Library

[29]

Shuang Ma, Yangyu Fan, and Chang Wen Chen. 2014. Finding your spot: A photography suggestion system for placing human in the scene. In 2014 IEEE International Conference on Image Processing (ICIP). 556--560. https://doi.org/10.1109/ICIP.2014.7025111

[30]

Shuai Ma, Zijun Wei, Feng Tian, Xiangmin Fan, Jianming Zhang, Xiaohui Shen, Zhe Lin, Jin Huang, Radom'ir Mvech, Dimitris Samaras, and Hongan Wang. 2019. SmartEye: Assisting Instant Photo Taking via Integrating User Preference with Deep View Proposal Network. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1-2. https://doi.org/10.1145/3290605.3300701

Digital Library

[31]

Michael R. Marner, Andrew Irlitti, and Bruce H. Thomas. 2013. Improving procedural task performance with Augmented Reality annotations. In 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). 39--48. https://doi.org/10.1109/ISMAR.2013.6671762

[32]

Hiroko Mitarai, Yoshihiro Itamiya, and Atsuo Yoshitaka. 2013. Interactive photographic shooting assistance based on composition and saliency. In International Conference on Computational Science and Its Applications. Springer, 348--363.

[33]

K. L. Bhanu Moorthy, Moneish Kumar, Ramanathan Subramanian, and Vineet Gandhi. 2020. GAZED-Gaze-Guided Cinematic Editing of Wide-Angle Monocular Video Recordings. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1-1. https://doi.org/10.1145/3313831.3376544

Digital Library

[34]

Katsuhiko Onishi, Seiyu Fumiyama, Yohei Miki, Masahiro Nonaka, Masanao Koeda, and Hiroshi Noborio. 2020. Study on the Development of Augmented-Reality Navigation System for Transsphenoidal Surgery. In Human-Computer Interaction. Human Values and Quality of Life, Masaaki Kurosu (Ed.). Springer International Publishing, Cham, 623--638.

[35]

Yogesh Singh Rawat and Mohan S. Kankanhalli. 2015. Context-Aware Photography Learning for Smart Mobile Devices. ACM Trans. Multimedia Comput. Commun. Appl., Vol. 12, 1s, Article 19 (Oct. 2015), 24 pages. https://doi.org/10.1145/2808199

Digital Library

[36]

Yogesh Singh Rawat and Mohan S. Kankanhalli. 2017. ClickSmart: A Context-Aware Viewpoint Recommendation System for Mobile Photography. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 27, 1 (2017), 149--158. https://doi.org/10.1109/TCSVT.2016.2555658

Digital Library

[37]

Marc Ericson C Santos, Takafumi Taketomi, Goshiro Yamamoto, Ma Mercedes T Rodrigo, Christian Sandor, Hirokazu Kato, et al. 2016. Augmented reality as multimedia: the case for situated vocabulary learning. Research and Practice in Technology Enhanced Learning, Vol. 11, 1 (2016), 1--23. https://doi.org/10.1186/s41039-016-0028--2

[38]

Arthur Tang, Charles Owen, Frank Biocca, and Weimin Mou. 2003. Comparative Effectiveness of Augmented Reality in Object Assembly. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Ft. Lauderdale, Florida, USA) (CHI '03). Association for Computing Machinery, New York, NY, USA, 73-0. https://doi.org/10.1145/642611.642626

Digital Library

[39]

Alexander Vakhitov, Luis Ferraz, Antonio Agudo, and Francesc Moreno-Noguer. 2021. Uncertainty-Aware Camera Pose Estimation From Points and Lines. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19--25, 2021. Computer Vision Foundation / IEEE, 4659--4668. https://openaccess.thecvf.com/content/CVPR2021/html/Vakhitov_Uncertainty-Aware_Camera_Pose_Estimation_From_Points_and_Lines_CVPR_2021_paper.html

[40]

Christian David Vazquez, Afika Ayanda Nyati, Alexander Luh, Megan Fu, Takako Aikawa, and Pattie Maes. 2017. Serendipitous Language Learning in Mixed Reality. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI EA '17). Association for Computing Machinery, New York, NY, USA, 2172-179. https://doi.org/10.1145/3027063.3053098

Digital Library

[41]

Yan Xu, Joshua Ratcliff, James Scovell, Gheric Speiginer, and Ronald Azuma. 2015. Real-Time Guidance Camera Interface to Enhance Photo Aesthetic Quality. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (Seoul, Republic of Korea) (CHI '15). Association for Computing Machinery, New York, NY, USA, 1183-186. https://doi.org/10.1145/2702123.2702418

Digital Library

[42]

Lei Yao, Poonam Suryanarayan, Mu Qiao, James Z Wang, and Jia Li. 2012. Oscar: On-site composition and aesthetics feedback through exemplars for photographers. International Journal of Computer Vision, Vol. 96, 3 (2012), 353--383. https://doi.org/10.1007/s11263-011-0478--3

Digital Library

Cited By

Zhang QLan YGuo KWang D(2024)Lipwatch: Enabling Silent Speech Recognition on Smartwatches using Acoustic SensingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596148:2(1-29)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659614
Li WGao RXiong JZhou JWang LMao XYi EZhang D(2024)WiFi-CSI Difference ParadigmProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596088:2(1-29)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659608
He JXiong JHu WFeng CYao EWang XLiu CChen XOkoshi TKo JLiKamWa R(2024)CW-AcousLen: A Configurable Wideband Acoustic MetasurfaceProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661882(29-41)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3643832.3661882
Show More Cited By

Index Terms

Augmented Reality Based Video Shooting Guidance for Novice Users
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Marker Tracking and HMD Calibration for a Video-Based Augmented Reality Conferencing System
IWAR '99: Proceedings of the 2nd IEEE and ACM International Workshop on Augmented Reality

We describe an augmented reality conferencing system which uses the overlay of virtual images on the real world. Remote collaborators are represented on Virtual Monitors which can be freely positioned about a user in space. Users can collaboratively ...
Real-Time Composition of Stereo Images for Video See-Through Augmented Reality
ICMCS '99: Proceedings of the IEEE International Conference on Multimedia Computing and Systems - Volume 2

This paper describes a method of stereo image composition for video see-through augmented reality. In order to implement an augmented reality system, we must acquire the position and orientation of the user's viewpoint to display the composed image ...
Calibration-Free Augmented Reality in Perspective

This paper deals with video-based augmented reality and proposes an algorithm for augmenting a real video sequence with views of graphics objects without metric calibration of the video camera by representing the motion of the video camera in projective ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction

Proceedings of the ACM on Human-Computer Interaction Volume 6, Issue MHCI

MHCI

September 2022

852 pages

EISSN:2573-0142

DOI:10.1145/3564624

Editor:
Jeff Nichols
Apple Inc., United States

Issue’s Table of Contents

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 September 2022

Published in PACMHCI Volume 6, Issue MHCI

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Open Project Program of State Key Laboratory of Virtual Reality Technology and Systems, Beihang University
Fujian Science and Technology Program Guiding Project
the Natural Science Foundation of Fujian Province of China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

28
Total Citations
View Citations
391
Total Downloads

Downloads (Last 12 months)64
Downloads (Last 6 weeks)5

Reflects downloads up to 22 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhang QLan YGuo KWang D(2024)Lipwatch: Enabling Silent Speech Recognition on Smartwatches using Acoustic SensingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596148:2(1-29)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659614
Li WGao RXiong JZhou JWang LMao XYi EZhang D(2024)WiFi-CSI Difference ParadigmProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596088:2(1-29)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659608
He JXiong JHu WFeng CYao EWang XLiu CChen XOkoshi TKo JLiKamWa R(2024)CW-AcousLen: A Configurable Wideband Acoustic MetasurfaceProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661882(29-41)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3643832.3661882
Duan DSun ZNi TLi SJia XXu WLi TOkoshi TKo JLiKamWa R(2024)F2Key: Dynamically Converting Your Face into a Private Key Based on COTS Headphones for Reliable Voice InteractionProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661860(127-140)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3643832.3661860
Wang SZhong LFu YChen LRen JZhang Y(2024)UFaceProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435468:1(1-27)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643546
Wang ZWang YYang J(2024)EarSlideProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435158:1(1-29)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643515
Chang ZZhang FXiong JChen WZhang DGanesan DLane NShi W(2024)MSense: Boosting Wireless Sensing Capability Under Motion InterferenceProceedings of the 30th Annual International Conference on Mobile Computing and Networking10.1145/3636534.3649350(108-123)Online publication date: 29-May-2024
https://dl.acm.org/doi/10.1145/3636534.3649350
Zhang YHan FYang PFeng YYan YGuan R(2024)Wi-Cyclops: Room-Scale WiFi Sensing System for Respiration Detection Based on Single-AntennaACM Transactions on Sensor Networks10.1145/363295820:4(1-24)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3632958
Zhang XZhang DXie YWu DLi YZhang D(2024)WaffleProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314587:4(1-29)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631458
Duan DChen YXu WLi T(2024)EarSEProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314477:4(1-33)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631447
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents