Article

Spatio-temporal view interpolation

Authors:

Takeo KanadeAuthors Info & Claims

EGRW '02: Proceedings of the 13th Eurographics workshop on Rendering

Pages 65 - 76

Published: 26 July 2002 Publication History

Abstract

We propose a fully automatic algorithm for view interpolation of a completely non-rigid dynamic event across both space and time. The algorithm operates by combining images captured across space to compute voxel models of the scene shape at each time instant, and images captured across time to compute the "scene flow" between the voxel models. The scene-flow is the non-rigid 3D motion of every point in the scene. To interpolate in time, the voxel models are "flowed" using an appropriate multiple of the scene flow and a smooth surface fit to the result. The novel image is then computed by ray-casting to the surface at the intermediate time instant, following the scene flow to the neighboring time instants, projecting into the input images at those times, and finally blending the results. We use our algorithm to create re-timed slow-motion fly-by movies of dynamic real-world events.

References

[1]

M. Alexa, D. Cohen-Or, and D. Levin. As-rigid-as-possible shape interpolation. In Computer Graphics Annual Conference Series (SIGGRAPH), 2000.

Digital Library

[2]

D. Breen and R. Whitaker. A level-set approach for the metamorphosis of solid models. IEEE Trans. on Visualization and Computer Graphics, 7(2):173--192, 2001.

Digital Library

[3]

S.E. Chen and L. Williams. View interpolation for image synthesis. In Computer Graphics Annual Conference Series (SIGGRAPH), pages 279--288, 1993.

Digital Library

[4]

P.E. Debevec, C.J. Taylor, and J. Malik. Modeling and rendering architecture from photographs: A hybrid geometry- and image-based approach. In Comp. Graphics Annual Conf. Series (SIGGRAPH), 1996.

Digital Library

[5]

S.J. Gortler, R. Grzeszczuk, R. Szeliski, and M.F. Cohen. The lumigraph. In Computer Graphics Annual Conference Series (SIGGRAPH), pages 43--54, 1996.

Digital Library

[6]

A. Kadosh, D. Cohen-Or, and R. Yagel. Tricubic interpolation of discrete surfaces for binary volumes. IEEE Trans. on Visualization and Computer Graphics, 2000.

Digital Library

[7]

T. Kanade, H. Saito, and S. Vedula. The 3D room: Digitizing time-varying 3D events by synchronized multiple video streams. Technical Report CMU-RI-TR-98-34, Robotics Institute, Carnegie Mellon University, 1998.

[8]

A. Lerios, C.D. Garfinkle, and M. Levoy. Feature-based volume metamorphosis. In Computer Graphics Annual Conference Series (SIGGRAPH), 1995.

Digital Library

[9]

M. Levoy and M. Hanrahan. Light field rendering. In Computer Graphics Annual Conference Series (SIGGRAPH), 1996.

Digital Library

[10]

W.E. Lorensen and H.E. Cline. Marching cubes: A high resolution 3D surface reconstruction algorithm. Computer Graphics, 21(4):163--169, 1992.

Digital Library

[11]

R.A. Manning and C.R. Dyer. Interpolating view and scene motion by dynamic view morphing. In Proc. of the Conf. on Comp. Vis. and Pattern Recognition, 1999.

[12]

P.J Narayanan, P.W. Rander, and T. Kanade. Constructing virtual worlds using dense stereo. In Proc. of the 6th Intl. Conf. on Computer Vision, 1998.

Digital Library

[13]

Y. Sato, M. Wheeler, and K. Ikeuchi. Object shape and reflectance modeling from observation. In Comp. Graphics Annual Conf. Series (SIGGRAPH), 1997.

Digital Library

[14]

E. Schechtman, Y. Capsi, and M. Irani. Increasing space-time resolution in video. In Proceedings of the 7th European Conference on Computer Vision, 2002.

Digital Library

[15]

S.M. Seitz and C.R. Dyer. View morphing. In Computer Graphics Annual Conference Series (SIGGRAPH), pages 21--30, 1996.

Digital Library

[16]

S.M. Seitz and C.R. Dyer. Photorealistic scene reconstruction by voxel coloring. International Journal of Computer Vision, 35(2):151--173, 1999.

Digital Library

[17]

G. Turk and M. Levoy. Zippered polygon meshes from range images. In Computer Graphics Annual Conference Series (SIGGRAPH), 1994.

Digital Library

[18]

G. Turk and J.F. O'Brien. Shape transformation using variational implicit functions. In Computer Graphics Annual Conference Series (SIGGRAPH), 1999.

Digital Library

[19]

S. Vedula. Image Based Spatio-Temporal Modeling and View Interpolation of Dynamic Events. PhD thesis, Robotics Institute, Carnegie Mellon University, 2001.

Digital Library

[20]

S. Vedula, S. Baker, P. Rander, R. Collins, and T. Kanade. Three-dimensional scene flow. In Proc. of the 7th IEEE Intl. Conf. on Computer Vision, 1999.

Digital Library

[21]

H. Weghorst, G. Hooper, and D.P. Greenberg. Improved computational methods for ray tracing. ACM Transactions on Graphics, 3(1):52--69, 1984.

Digital Library

[22]

Y. Wexler and A. Shashua. On the synthesis of dynamic scenes from reference views. In Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, 2000.

Cited By

Lee SKim YEisemann E(2018)Iterative Depth WarpingACM Transactions on Graphics10.1145/319085937:5(1-13)Online publication date: 23-Oct-2018
https://dl.acm.org/doi/10.1145/3190859
Yang LTse YSander PLawrence JNehab DHoppe HWilkins C(2011)Image-based bidirectional scene reprojectionACM Transactions on Graphics10.1145/2070781.202418430:6(1-10)Online publication date: 12-Dec-2011
https://dl.acm.org/doi/10.1145/2070781.2024184
Yang LTse YSander PLawrence JNehab DHoppe HWilkins C(2011)Image-based bidirectional scene reprojectionProceedings of the 2011 SIGGRAPH Asia Conference10.1145/2024156.2024184(1-10)Online publication date: 12-Dec-2011
https://dl.acm.org/doi/10.1145/2024156.2024184
Show More Cited By

Index Terms

Spatio-temporal view interpolation
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Image and video acquisition
        3D imaging
  2. Computer graphics
    1. Animation

Recommendations

Image-based spatio-temporal modeling and view interpolation of dynamic events

We present an approach for modeling and rendering a dynamic, real-world event from an arbitrary viewpoint, and at any time, using images captured from multiple video cameras. The event is modeled as a nonrigidly varying dynamic scene, captured by many ...
High-quality spatio-temporal rendering using semi-analytical visibility

We present a novel visibility algorithm for rendering motion blur with per-pixel anti-aliasing. Our algorithm uses a number of line samples over a rectangular group of pixels, and together with the time dimension, a two-dimensional spatio-temporal ...
High-quality spatio-temporal rendering using semi-analytical visibility
SIGGRAPH '11: ACM SIGGRAPH 2011 papers

We present a novel visibility algorithm for rendering motion blur with per-pixel anti-aliasing. Our algorithm uses a number of line samples over a rectangular group of pixels, and together with the time dimension, a two-dimensional spatio-temporal ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

EGRW '02: Proceedings of the 13th Eurographics workshop on Rendering

July 2002

336 pages

ISBN:1581135343

Editors:
Simon Gibson
University of Manchester
,
Paul Debevec
USC Institute for Creative Technologies

Sponsors

EUROGRAPHICS: The European Association for Computer Graphics

Publisher

Eurographics Association

Goslar, Germany

Publication History

Published: 26 July 2002

Check for updates

Qualifiers

Article

Conference

EGRW02

Sponsor:

EUROGRAPHICS

EGRW02: Eurographics Workshop on Rendering

June 26 - 28, 2002

Pisa, Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
594
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lee SKim YEisemann E(2018)Iterative Depth WarpingACM Transactions on Graphics10.1145/319085937:5(1-13)Online publication date: 23-Oct-2018
https://dl.acm.org/doi/10.1145/3190859
Yang LTse YSander PLawrence JNehab DHoppe HWilkins C(2011)Image-based bidirectional scene reprojectionACM Transactions on Graphics10.1145/2070781.202418430:6(1-10)Online publication date: 12-Dec-2011
https://dl.acm.org/doi/10.1145/2070781.2024184
Yang LTse YSander PLawrence JNehab DHoppe HWilkins C(2011)Image-based bidirectional scene reprojectionProceedings of the 2011 SIGGRAPH Asia Conference10.1145/2024156.2024184(1-10)Online publication date: 12-Dec-2011
https://dl.acm.org/doi/10.1145/2024156.2024184
Gong MSelzer JLei CYang YHealey CLank E(2007)Real-time backward disparity-based rendering for dynamic scenes using programmable graphics hardwareProceedings of Graphics Interface 200710.1145/1268517.1268557(241-248)Online publication date: 28-May-2007
https://dl.acm.org/doi/10.1145/1268517.1268557
Pons JKeriven RFaugeras O(2007)Multi-View Stereo Reconstruction and Scene Flow Estimation with a Global Image-Based Matching ScoreInternational Journal of Computer Vision10.1007/s11263-006-8671-572:2(179-193)Online publication date: 21-Apr-2007
https://dl.acm.org/doi/10.1007/s11263-006-8671-5
Einarsson PChabert CJones AMa WLamond BHawkins TBolas MSylwan SDebevec P(2006)Relighting human locomotion with flowed reflectance fieldsProceedings of the 17th Eurographics conference on Rendering Techniques10.5555/2383894.2383917(183-194)Online publication date: 26-Jun-2006
https://dl.acm.org/doi/10.5555/2383894.2383917
Yang MHuang SLo KTai WChiang C(2006)Exploiting spatial-temporal coherence in the construction of multiple perspective videosProceedings of the First Pacific Rim conference on Advances in Image and Video Technology10.1007/11949534_132(1303-1313)Online publication date: 10-Dec-2006
https://dl.acm.org/doi/10.1007/11949534_132
Choudhury PTumblin J(2005)The trilateral filter for high contrast images and meshesACM SIGGRAPH 2005 Courses10.1145/1198555.1198565(5-es)Online publication date: 31-Jul-2005
https://dl.acm.org/doi/10.1145/1198555.1198565
Starck JMiller GHilton ATerzopoulos DZordan VAnjyo KFaloutsos P(2005)Video-based character animationProceedings of the 2005 ACM SIGGRAPH/Eurographics symposium on Computer animation10.1145/1073368.1073375(49-58)Online publication date: 29-Jul-2005
https://dl.acm.org/doi/10.1145/1073368.1073375
Vedula SBaker SRander PCollins RKanade T(2005)Three-Dimensional Scene FlowIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2005.6327:3(475-480)Online publication date: 1-Mar-2005
https://dl.acm.org/doi/10.1109/TPAMI.2005.63
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

Media

Figures

Other

Tables

View Table of Contents