research-article

Camera tracking in visual effects an industry perspective of structure from motion

Authors:

Alastair Barber,

Radhika PatelAuthors Info & Claims

DigiPro '16: Proceedings of the 2016 Symposium on Digital Production

Pages 45 - 54

https://doi.org/10.1145/2947688.2947697

Published: 23 July 2016 Publication History

Abstract

The 'Matchmove', or camera-tracking process is a crucial task and one of the first to be performed in the visual effects pipeline. An accurate solve for camera movement is imperative and will have an impact on almost every other part of the pipeline downstream. In this work we present a comprehensive analysis of the process at a major visual effects studio, drawing on a large dataset of real shots. We also present guidelines and rules-of-thumb for camera tracking scheduling which are, in what we believe to be an industry first, backed by statistical data drawn from our dataset. We also make available data from our pipeline which shows the amount of time spent on camera tracking and the types of shot that are most common in our work. We hope this will be of interest to the wider computer vision research community and will assist in directing future research.

References

[1]

Arri. Lens data system lds.

[2]

Autodesk. Matchmover.

[3]

Baker, S., and Matthews, I. 2004. Lucas-kanade 20 years on: A unifying framework. Int. J. Comput. Vision 56, 3 (Feb.), 221--255.

Digital Library

[4]

Bay, H., Ess, A., Tuytelaars, T., and Van Gool, L. 2008. Speeded-up robust features (surf). Comput. Vis. Image Underst. 110, 3 (June), 346--359.

Digital Library

[5]

Bregler, C., Bhat, K., Saltzman, J., and Allen, B. 2009. Ilm's multitrack: A new visual tracking framework for high-end vfx production. In SIGGRAPH 2009: Talks, ACM, New York, NY, USA, SIGGRAPH '09, 29:1--29:1.

Digital Library

[6]

Dalal, N., and Triggs, B. 2005. Histograms of oriented gradients for human detection. In Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 1 - Volume 01, IEEE Computer Society, Washington, DC, USA, CVPR '05, 886--893.

Digital Library

[7]

Dobbert, T. 2013. Mathcmoving The Invisible Art of Camera Tracking.

Digital Library

[8]

Fitzgibbon, A., and Zisserman, A. 1998. Automatic camera recovery for closed or open image sequences. In Computer Vision ECCV'98, H. Burkhardt and B. Neumann, Eds., vol. 1406 of Lecture Notes in Computer Science. Springer Berlin Heidelberg, 311--326.

Digital Library

[9]

Goulekas, K. 2010. The VES Handbook of Visual Effects. Elseveir.

[10]

Hartley, R. I., and Zisserman, A. 2004. Multiple View Geometry in Computer Vision, second ed. Cambridge University Press, ISBN: 0521540518.

Digital Library

[11]

Horn, B. K., and Schunck, B. G. 1981. Determining optical flow. Artificial Intelligence 17, 1, 185--203.

Digital Library

[12]

Hornung, E. 2010. The Art and Technique of Matchmoving: Solutions for the VFX Artist. Focal Press.

Digital Library

[13]

Huang, Y., and Essa, I. 2005. Tracking multiple objects through occlusions. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), vol. 2, 1051--1058 vol. 2.

Digital Library

[14]

Jin, H., Favaro, P., and Cipolla, R. 2005. Visual tracking in the presence of motion blur. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), vol. 2, 18--25 vol. 2.

Digital Library

[15]

Klein, G., and Drummond, T. 2004. Tightly integrated sensor fusion for robust visual tracking. Image and Vision Computing 22, 10 (September), 769--776.

[16]

Lowe, D. 1991. Fitting parameterized three-dimensional models to images. IEEE Tra (May).

Digital Library

[17]

Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60, 2 (Nov.), 91--110.

Digital Library

[18]

Okatani, T., and Deguchi, K. 2002. Robust estimation of camera translation between two images using a camera with a 3d orientation sensor. In Pattern Recognition, 2002. Proceedings. 16th International Conference on, vol. 1, 275--278 vol. 1.

Digital Library

[19]

Rosten, E., Porter, R., and Drummond, T. 2010. Faster and better: A machine learning approach to corner detection. IEEE Trans. Pattern Anal. Mach. Intell. 32, 1 (Jan.), 105--119.

Digital Library

[20]

Science D. Visions. 3dequalizer.

[21]

Shi, J., and Tomasi, C. 1994. Good features to track. 593--600.

[22]

Tola, E., Lepetit, V., and Fua, P. 2010. DAISY: An Efficient Dense Descriptor Applied to Wide Baseline Stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence 32, 5 (May), 815--830.

Digital Library

[23]

Torr, P., Fitzgibbon, A., and Zisserman, A. 1998. Maintaining multiple motion model hypotheses over many views to recover matching and structure. In SIXTH INTERNATIONAL CONFERENCE ON COMPUTER VISION, IEEE Comp Soc, 485--491. 6th International Conference on Computer Vision, BOMBAY, INDIA, JAN 04-07, 1998.

Digital Library

[24]

Tu, Z., Poppe, R., and Veltkamp, R. 2015. Estimating accurate optical flow in the presence of motion blur. Journal of Electronic Imaging 24, 5, 053018.

[25]

Vicon. Boujou.

[26]

Xue, T., Rubinstein, M., Liu, C., and Freeman, W. T. 2015. A computational approach for obstruction-free photography. ACM Transactions on Graphics (TOG) 34, 4, 79.

Digital Library

[27]

Zhang, L., Portz, T., and Jiang, H. 2012. Optical flow in the presence of spatially-varying motion blur. 2013 IEEE Conference on Computer Vision and Pattern Recognition 0, 1752--1759.

Digital Library

[28]

Zhang, Z. 2000. A flexible new technique for camera calibration. IEEE Trans. on Pattern Analysis and Machine Intelligence.

Digital Library

Cited By

Chen YWang YYu TPan Y(2024)The Effect of AI on Animation Production Efficiency: An Empirical Investigation Through the Network Data Envelopment AnalysisElectronics10.3390/electronics1324500113:24(5001)Online publication date: 19-Dec-2024
https://doi.org/10.3390/electronics13245001
Sokhandan AMonadjemi A(2024)Visual tracking in video sequences based on biologically inspired mechanismsComputer Vision and Image Understanding10.1016/j.cviu.2018.10.002239:COnline publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.cviu.2018.10.002
Karpayah ALynn-Sze N(2022)Using Video-Assisted Learning in Teaching Camera Tracking to Visual Effects Students in Malaysia – A Review2nd International Conference on Creative Multimedia 2022 (ICCM 2022)10.2991/978-2-494069-57-2_9(67-74)Online publication date: 24-Dec-2022
https://doi.org/10.2991/978-2-494069-57-2_9
Show More Cited By

Index Terms

Camera tracking in visual effects an industry perspective of structure from motion

Recommendations

Automatic scene structure and camera motion using a catadioptric system

Fully automatic methods are presented for the estimation of scene structure and camera motion from an image sequence acquired by a catadioptric system. The first contribution is the design of bundle adjustments for both central and non-central models, ...
A generic structure-from-motion framework
Special issue on omnidirectional vision and camera networks

We introduce a generic structure-from-motion approach based on a previously introduced, highly general imaging model, where cameras are modeled as possibly unconstrained sets of projection rays. This allows to describe most existing camera types ...
Efficient, causal camera tracking in unprepared environments

This paper addresses the problem of tracking the 3D pose of a camera in space, using the images it acquires while moving freely in unmodeled, arbitrary environments. A novel feature-based approach for camera tracking is proposed, intended to facilitate ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

DigiPro '16: Proceedings of the 2016 Symposium on Digital Production

July 2016

70 pages

ISBN:9781450344296

DOI:10.1145/2947688

Conference Chairs:
Andrew Pearce
DreamWorks Animation
,
Eric Enderton
NVIDIA
,
Program Chairs:
Chris Horvath
Oculus
,
Jonathan "Jono" Gibbs

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 July 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Engineering and Physical Sciences Research Council

Conference

DigiPro '16

Sponsor:

SIGGRAPH

DigiPro '16: The Digital Production Symposium

July 23, 2016

California, Anaheim

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
445
Total Downloads

Downloads (Last 12 months)27
Downloads (Last 6 weeks)4

Reflects downloads up to 12 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chen YWang YYu TPan Y(2024)The Effect of AI on Animation Production Efficiency: An Empirical Investigation Through the Network Data Envelopment AnalysisElectronics10.3390/electronics1324500113:24(5001)Online publication date: 19-Dec-2024
https://doi.org/10.3390/electronics13245001
Sokhandan AMonadjemi A(2024)Visual tracking in video sequences based on biologically inspired mechanismsComputer Vision and Image Understanding10.1016/j.cviu.2018.10.002239:COnline publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.cviu.2018.10.002
Karpayah ALynn-Sze N(2022)Using Video-Assisted Learning in Teaching Camera Tracking to Visual Effects Students in Malaysia – A Review2nd International Conference on Creative Multimedia 2022 (ICCM 2022)10.2991/978-2-494069-57-2_9(67-74)Online publication date: 24-Dec-2022
https://doi.org/10.2991/978-2-494069-57-2_9
Anantrasirichai NBull D(2021)Artificial intelligence in the creative industries: a reviewArtificial Intelligence Review10.1007/s10462-021-10039-7Online publication date: 2-Jul-2021
https://doi.org/10.1007/s10462-021-10039-7
Francisco FNührenberg PJordan A(2020)High-resolution, non-invasive animal tracking and reconstruction of local environment in aquatic ecosystemsMovement Ecology10.1186/s40462-020-00214-w8:1Online publication date: 23-Jun-2020
https://doi.org/10.1186/s40462-020-00214-w
Vyas YCampbell EAnderson SAdcock MAdcock MBednarz T(2017)A workflow for Web3D interactive outdoor scene visualisationProceedings of the 22nd International Conference on 3D Web Technology10.1145/3055624.3080880(1-4)Online publication date: 5-Jun-2017
https://dl.acm.org/doi/10.1145/3055624.3080880

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten