research-article

Novel View Synthesis by Cascading Trilinear Tensors

Authors:

Amnon ShashuaAuthors Info & Claims

IEEE Transactions on Visualization and Computer Graphics, Volume 4, Issue 4

Pages 293 - 306

https://doi.org/10.1109/2945.765324

Published: 01 October 1998 Publication History

Abstract

We present a new method for synthesizing novel views of a 3D scene from two or three reference images in full correspondence. The core of this work is the use and manipulation of an algebraic entity termed the trilinear tensor that links point correspondences across three images. For a given virtual camera position and orientation, a new trilinear tensor can be computed based on the original tensor of the reference images. The desired view can then be created using this new trilinear tensor and point correspondences across two of the reference images.

References

[1]

E.H. Adelson and J.R. Bergen, "The Plenoptic Function and the Elements of Early Vision," Computational Models of Visual Processing M. Landy and J.A. Movshon, eds., Chapter 1. Cambridge, Mass.: MIT Press, 1991.

[2]

S. Avidan T. Evgeniou A. Shashua and T. Poggio, "Image-Based View Synthesis By Combining Trilinear Tensors and Learning Techniques," VRST, 1997.

Digital Library

[3]

S. Avidan and A. Shashua, "Tensorial Transfer: On the Representation of n > 3 Views of a 3D Scene," Proc. ARPA Image Understanding Workshop, 1996.

[4]

S. Avidan and A. Shashua, "Novel View Synthesis in Tensor Space," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1997.

Digital Library

[5]

S. Avidan and A. Shashua, "Unifying Two-View and Three-View Geometry," Proc. ARPA Image Understanding Workshop, 1997.

[6]

S. Avidan and A. Shashua, "Threading Fundamental Matrices," Proc. European Conf. Computer Vision, June 1998.

Digital Library

[7]

E.B. Barrett P.M. Payton and G. Gheen, "Robust Algebraic Invariant Methods with Applications in Geometry and Imaging," Proc. SPIE Remote Sensing, July 1995.

[8]

T. Beier and S. Neely, "Feature-Based Image Metamorphosis," Proc. SIGGRAPH, 1992.

Digital Library

[9]

J.R. Bergen P. Anandan K.J. Hanna and R. Hingorani, "Hierarchical Model-Based Motion Estimation," Proc. European Conf. Computer Vision, 1992.

Digital Library

[10]

D. Beymer A. Shashua and T. Poggio, "Example Based Image Analysis and Synthesis," Technical Report A.I. Memo No. 1431, Artificial Intelligence Laboratory, Massachusetts Inst. of Technology, 1993.

Digital Library

[11]

P.J. Burt and E.H. Adelson, "The Laplacian Pyramid as a Compact Image Code," IEEE Trans. Comm., vol. 31, pp. 532-540, 1983.

[12]

S.E. Chen and L. Williams, "View Interpolation for Image Synthesis," Proc. SIGGRAPH, 1993.

Digital Library

[13]

S.E. Chen, "QuickTimeVR—An Image-Based Approach to Virtual Environment Navigation," Proc. SIGGRAPH, 1995.

Digital Library

[14]

T. Evgeniou, "Image-Based Rendering Using Algebraic Techniques," master's thesis, Massachusetts Inst. of Technology, June 1996.

Digital Library

[15]

O. Faugeras and T. Papadopoulo, "A Nonlinear Method for Estimation the Projective Geometry of 3 Views," Proc. Int'l Conf. Computer Vision, 1998.

Digital Library

[16]

O.D. Faugeras, "What Can Be Seen in Three Dimensions with an Uncalibrated Stereo Rig?" Proc. European Conf. Computer Vision, 1992.

Digital Library

[17]

O.D. Faugeras and B. Mourrain, "On the Geometry and Algebra of the Point and Line Correspondences Between N Images," Proc. Int'l Conf. Computer Vision, 1995.

Digital Library

[18]

O.D. Faugeras and L. Robert, "What Can Two Images Tell Us About a Third One?" Proc. European Conf. Computer Vision, 1994.

Digital Library

[19]

S.J. Gortler R. Grzeszczuk R. Szeliski and M. Cohen, "The Lumigraph," Proc. SIGGRAPH, pp. 43-54, 1996.

Digital Library

[20]

W.E.L. Grimson, "Why Stereo Vision Is Not Always About 3D Reconstruction," Technical Report A.I. Memo No. 1435, Artificial Intelligence Laboratory, Massachusetts Inst. of Technology, 1993.

Digital Library

[21]

C. Harris and M. Stephens, "A Combined Corner and Edge Detector," Proc. Fourth Alvey Vision Conf. Intelligence, pp. 189-192, 1988.

[22]

R. Hartley, "Lines and Points in Three Views—A Unified Approach," Proc. ARPA Image Understanding Workshop, 1994.

[23]

R. Hartley, "In Defence of the 8-Point Algorithm," Proc. Int'l Conf. Computer Vision, 1995.

Digital Library

[24]

R. Hartley, "A Linear Method for Reconstruction from Lines and Points," Proc. Int'l Conf. Computer Vision, 1995.

Digital Library

[25]

R. Hartley, "Minimizing Algebraic Error in Geometric Estimation Problems," Proc. Int'l Conf. Computer Vision, 1998.

Digital Library

[26]

A. Heyden, "Reconstruction from Image Sequences By Means of Relative Depths," Proc. Int'l Conf. Computer Vision, 1995.

Digital Library

[27]

A. Heyden and K. Astrom, "Algebraic Varieties in Multiple View Geometry," Proc. European Conf. Computer Vision, Apr. 1996.

Digital Library

[28]

W. Hoff and N. Ahuja, "Surfaces from Stereo: Integrating Feature Matching, Disparity Estimation, and Contour Detection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 11, no. 2, pp. 121-136, Feb. 1989.

Digital Library

[29]

S. Laveau and O. Faugeras, "Oriented Projective Geometry for Computer Vision," Proc. European Conf. Computer Vision, 1996.

Digital Library

[30]

S. Laveau and O.D. Faugeras, "3-D Scene Representation as a Collection of Images," Proc. Int'l Conf. Pattern Recognition, 1994.

[31]

M. Levoy and P. Hanrahan, "Light Field Rendering," Proc. SIGGRAPH, pp. 31-42, 1996.

Digital Library

[32]

B.D. Lucas and T. Kanade, "An Iterative Image Registration Technique with an Application to Stereo Vision," Proc. IJCAI, pp. 674-679, Vancouver, Canada, 1981.

[33]

L. McMillan and G. Bishop, "Plenoptic Modeling: An Image-Based Rendering System," Proc. SIGGRAPH, 1995.

Digital Library

[34]

A. Zisserman P.A. Beardsley and D.W. Murray, "Sequential Updating of Projective and Affine Structure from Motion," Int'l J. Computer Vision, vol. 23, no. 3, pp. 235-260, 1997.

Digital Library

[35]

S. Peleg and J. Herman, "Panoramic Mosaic by Manifold Projection," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1997.

Digital Library

[36]

P.H.S. Torr A. Zisserman and D. Murray, "Motion Clustering Using the Trilinear Constraint Over Three Views," Proc. Workshop Geometrical Modeling and Invariants for Computer Vision, 1995.

[37]

T. Poggio and R. Brunelli, "A Novel Approach to Graphics," Technical Report A.I. Memo No. 1354, Artificial Intelligence Laboratory, Massachusetts Inst. of Technology, 1992.

Digital Library

[38]

M. Irani R. Kumar P. Anandan J. Bergen and K. Hanna, "Representation of Scenes from Collections of Images," Proc. IEEE Workshop Representation of Visual Scenes, 1995.

Digital Library

[39]

B. Rousso S. Avidan A. Shashua and S. Peleg, "Robust Recovery of Camera Rotation from Three Frames," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1996.

Digital Library

[40]

S.M. Seitz and C.R. Dyer, "Physically-Valid View Synthesis By Image Interpolation," Proc. IEEE Workshop Representation of Visual Scenes, 1995.

Digital Library

[41]

S.M. Seitz and C.R. Dyer, "View Morphing," Proc. SIGGRAPH, pp. 21-30, 1996.

Digital Library

[42]

A. Shashua, "Algebraic Functions for Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 8, pp. 779-789, Aug. 1995.

Digital Library

[43]

A. Shashua and P. Anandan, "The Generalized Trilinear Constraints and the Uncertainty Tensor," Proc. ARPA Image Understanding Workshop, 1996.

[44]

A. Shashua and S. Avidan, "The Rank4 Constraint in Multiple View Geometry," Proc. European Conf. Computer Vision, 1996. Also in CIS report 9520, Technion, Nov. 1995.

Digital Library

[45]

A. Shashua and S.J. Maybank, "Degenerate n Point Configurations of Three Views: Do Critical Surfaces Exist?" Technical Report TR 96-19, Hebrew Univ. of Jerusalem, Nov. 1996.

[46]

A. Shashua and S. Toelg, "The Quadric Reference Surface: Applications in Registering Views of Complex 3D Objects," Proc. European Conf. Computer Vision, 1994.

Digital Library

[47]

A. Shashua and M. Werman, "On the Trilinear Tensor of Three Perspective Views and Its Underlying Geometry," Proc. Int'l Conf. Computer Vision, 1995.

Digital Library

[48]

M.E. Spetsakis and J. Aloimonos, "Structure from Motion Using Line Correspondences," Int'l J. Computer Vision, vol. 4, no. 3, pp. 171-183, 1990.

Digital Library

[49]

M.E. Spetsakis and J. Aloimonos, "A Unified Theory of Structure from Motion," Proc. ARPA Image Understanding Workshop, 1990.

[50]

G. Stein and A. Shashua, "Model Based Brightness Constraints: On Direct Estimation of Structure," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1997.

Digital Library

[51]

R. Szeliski and S.B. Kang, "Direct Methods for Visual Scene Reconstruction," Proc. IEEE Workshop Representation of Visual Scenes, 1995.

Digital Library

[52]

P.H.S. Torr and A. Zisserman, "Robust Parameterization and Computation of the Trifocal Tensor," Image and Vision Computing, vol. 15, pp. 591-607, 1997.

[53]

B. Triggs, "Matching Constraints and the Joint Image," Proc. Int'l Conf. Computer Vision, 1995.

Digital Library

[54]

J. Weng T.S. Huang and N. Ahuja, "Motion and Structure from Line Correspondences: Closed Form Solution, Uniqueness and Optimization," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 3, Mar. 1992.

Digital Library

[55]

T. Werner R.D. Hersch and V. Hlavac, "Rendering Real-World Objects Using View Interpolation," Proc. Int'l Conf. Computer Vision, 1995.

Digital Library

[56]

G. Wolberg, Digital Image Warping. Los Alamitos, Calif.: IEEE CS Press, 1992.

Digital Library

Cited By

Zhang PYang LXie XLai J(2024)Pose Guided Person Image Generation Via Dual-Task Correlation and Affinity LearningIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.328639430:8(5111-5128)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3286394
Malapelle FFusiello ARossi BFragneto P(2022)Novel View-Synthesis from Multiple Sources for Conversion to 3DSImage Analysis and Processing — ICIAP 201510.1007/978-3-319-23231-7_41(456-467)Online publication date: 10-Mar-2022
https://dl.acm.org/doi/10.1007/978-3-319-23231-7_41
Deng LPiao YWang Q(2018)Algorithm for Automatic Generation of Sparse Image Array Based on View Morphing2018 IEEE International Conference on Mechatronics and Automation (ICMA)10.1109/ICMA.2018.8484503(1145-1150)Online publication date: 5-Aug-2018
https://dl.acm.org/doi/10.1109/ICMA.2018.8484503
Show More Cited By

Index Terms

Novel View Synthesis by Cascading Trilinear Tensors
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision representations
        Image representations
  2. Computer graphics
2. Theory of computation
  1. Randomness, geometry and discrete structures

Recommendations

On norm compression inequalities for partitioned block tensors
Abstract
When a tensor is partitioned into subtensors, some tensor norms of these subtensors form a tensor called a norm compression tensor. Norm compression inequalities for tensors focus on the relation of the norm of this compressed tensor to the norm ... $(^{})$
Further results on Cauchy tensors and Hankel tensors

In this article, we present various new results on Cauchy tensors and Hankel tensors. We first introduce the concept of generalized Cauchy tensors which extends Cauchy tensors in the current literature, and provide several conditions characterizing ...
Efficient convex completion of coupled tensors using coupled nuclear norms
NIPS'18: Proceedings of the 32nd International Conference on Neural Information Processing Systems

Coupled norms have emerged as a convex method to solve coupled tensor completion. A limitation with coupled norms is that they only induce low-rankness using the multilinear rank of coupled tensors. In this paper, we introduce a new set of coupled norms ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Visualization and Computer Graphics

IEEE Transactions on Visualization and Computer Graphics Volume 4, Issue 4

October 1998

94 pages

ISSN:1077-2626

Issue’s Table of Contents

Copyright © Copyright © 1998 IEEE. All Rights Reserved.

Publisher

IEEE Educational Activities Department

United States

Publication History

Published: 01 October 1998

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

26
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhang PYang LXie XLai J(2024)Pose Guided Person Image Generation Via Dual-Task Correlation and Affinity LearningIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.328639430:8(5111-5128)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3286394
Malapelle FFusiello ARossi BFragneto P(2022)Novel View-Synthesis from Multiple Sources for Conversion to 3DSImage Analysis and Processing — ICIAP 201510.1007/978-3-319-23231-7_41(456-467)Online publication date: 10-Mar-2022
https://dl.acm.org/doi/10.1007/978-3-319-23231-7_41
Deng LPiao YWang Q(2018)Algorithm for Automatic Generation of Sparse Image Array Based on View Morphing2018 IEEE International Conference on Mechatronics and Automation (ICMA)10.1109/ICMA.2018.8484503(1145-1150)Online publication date: 5-Aug-2018
https://dl.acm.org/doi/10.1109/ICMA.2018.8484503
Sharma MChaudhury SLall B(2017)A Novel Hybrid Kinect-Variety-Based High-Quality Multiview Rendering Scheme for Glass-Free 3D DisplaysIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2016.256479827:10(2098-2117)Online publication date: 2-Oct-2017
https://dl.acm.org/doi/10.1109/TCSVT.2016.2564798
Chouvatut VMadarasmi STuceryan M(2013)3D face and motion estimation from sparse points using adaptive bracketed minimizationMultimedia Tools and Applications10.1007/s11042-011-0925-863:2(569-589)Online publication date: 1-Mar-2013
https://dl.acm.org/doi/10.1007/s11042-011-0925-8
Yeo CAhammad PRamchandran K(2011)Coding of Image Feature Descriptors for Distributed Rate-efficient Visual CorrespondencesInternational Journal of Computer Vision10.5555/2004103.200412794:3(267-281)Online publication date: 1-Sep-2011
https://dl.acm.org/doi/10.5555/2004103.2004127
Šegvić SRemazeilles ADiosi AChaumette F(2009)A mapping and localization framework for scalable appearance-based navigationComputer Vision and Image Understanding10.1016/j.cviu.2008.08.005113:2(172-187)Online publication date: 1-Feb-2009
https://dl.acm.org/doi/10.1016/j.cviu.2008.08.005
Gigengack FJiang X(2009)Improved uncalibrated view synthesis by extended positioning of virtual cameras and image quality optimizationProceedings of the 9th Asian conference on Computer Vision - Volume Part II10.1007/978-3-642-12304-7_41(438-447)Online publication date: 23-Sep-2009
https://dl.acm.org/doi/10.1007/978-3-642-12304-7_41
Duan QYu JYang XXiao S(2007)A practical framework for virtual viewing and relightingProceedings of the 6th international conference on Entertainment Computing10.5555/2394259.2394301(282-287)Online publication date: 15-Sep-2007
https://dl.acm.org/doi/10.5555/2394259.2394301
Sharma GChaudhury SSrivastava J(2007)On the view synthesis of man-made scenes using uncalibrated camerasPattern Recognition Letters10.1016/j.patrec.2006.04.01028:3(384-393)Online publication date: 1-Feb-2007
https://dl.acm.org/doi/10.1016/j.patrec.2006.04.010
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents