article

Photographing long scenes with multi-viewpoint panoramas

Authors:

Aseem Agarwala,

Maneesh Agrawala,

Richard SzeliskiAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 25, Issue 3

Pages 853 - 861

https://doi.org/10.1145/1141911.1141966

Published: 01 July 2006 Publication History

Abstract

We present a system for producing multi-viewpoint panoramas of long, roughly planar scenes, such as the facades of buildings along a city street, from a relatively sparse set of photographs captured with a handheld still camera that is moved along the scene. Our work is a significant departure from previous methods for creating multi-viewpoint panoramas, which composite thin vertical strips from a video sequence captured by a translating video camera, in that the resulting panoramas are composed of relatively large regions of ordinary perspective. In our system, the only user input required beyond capturing the photographs themselves is to identify the dominant plane of the photographed scene; our system then computes a panorama automatically using Markov Random Field optimization. Users may exert additional control over the appearance of the result by drawing rough strokes that indicate various high-level goals. We demonstrate the results of our system on several scenes, including urban streets, a river bank, and a grocery store aisle.

Supplementary Material

JPG File (p853-agarwala-high.jpg)

Download
17.11 KB

JPG File (p853-agarwala-low.jpg)

Download
14.22 KB

High Resolution (p853-agarwala-high.mov)

Download
120.43 MB

Low Resolution (p853-agarwala-low.mov)

Download
28.81 MB

References

[1]

Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., and Cohen, M. 2004. Interactive digital photomontage. ACM Transactions on Graphics 23, 3, 294--302.

Digital Library

[2]

Agarwala, A., Zheng, K. C., Pal, C., Agrawala, M., Cohen, M., Curless, B., Salesin, D. H., and Szeliski, R. 2005. Panoramic video textures. ACM Transactions on Graphics 24, 3 (Aug.), 821--827.

Digital Library

[3]

Agrawala, M., Zorin, D., and Munzner, T. 2000. Artistic multiprojection rendering. In Rendering Techniques 2000: 11th Eurographics Workshop on Rendering, 125--136.

Digital Library

[4]

Astola, J., Haavisto, P., and Neuvo, Y. 1990. Vector median filters. Proceedings of the IEEE 78, 678--689.

[5]

Bertalmio, M., Sapiro, G., Caselles, V., and Ballester, C. 2000. Image inpainting. In Proceedings of ACM SIGGRAPH 2000, Computer Graphics Proceedings, Annual Conference Series, 417--424.

Digital Library

[6]

Boykov, Y., Veksler, O., and Zabih, R. 2001. Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence 23, 11, 1222--1239.

Digital Library

[7]

Brown, M., and Lowe, D. G. 2005. Unsupervised 3D object recognition and reconstruction in unordered datasets. In 3D Imaging and Modeling (3DIM '05), 55--63.

Digital Library

[8]

Davis, J. 1998. Mosaics of scenes with moving objects. In Computer Vision and Pattern Recognition (CVPR 98), 354--360.

Digital Library

[9]

ePaperPress, 2005. http://epaperpress.com/ptlens/.

[10]

Google, 2005. http://earth.google.com.

[11]

Gupta, R., and Hartley, R. I. 1997. Linear pushbroom cameras. IEEE Transactions on Pattern Analysis and Machine Intelligence 19, 9, 963--975.

Digital Library

[12]

Hartley, R. I., and Zisserman, A. 2004. Multiple View Geometry in Computer Vision, second ed. Cambridge University Press.

Digital Library

[13]

Huber, P. 1981. Robust statistics. John Wiley.

[14]

Kang, S. B., Szeliski, R., and Chai, J. 2001. Handling occlusions in dense multi-view stereo. In Computer Vision and Pattern Recognition (CVPR 2001), vol. 1, 103--110.

[15]

Kasser, M., and Egels, Y. 2002. Digital Photogrammetry. Taylor & Francis Inc.

Digital Library

[16]

Koller, M., 2004. http://www.seamlesscity.com.

[17]

Kolmogorov, V., and Zabih, R. 2002. What energy functions can be minimized via graph cuts? In European Conference on Computer Vision (ECCV), 65--81.

Digital Library

[18]

Kubovy, M. 1986. The psychology of perspective and renaissance art. Cambridge University Press.

[19]

Kwatra, V., Schödl, A., Essa, I., Turk, G., and Bobick, A. 2003. Graphcut textures: Image and video synthesis using graph cuts. ACM Transactions on Graphics 22, 3, 277--286.

Digital Library

[20]

Levin, G., 2005. An informal catalogue of slit-scan video artworks. http://www.flong.com/writings/lists/list_slit_scan.html.

[21]

Levoy, M., Chen, B., Vaish, V., Horowitz, M., McDowall, I., and Bolas, M. 2004. Synthetic aperture confocal imaging. ACM Transactions on Graphics 23, 3 (Aug.), 825--834.

Digital Library

[22]

Lowe, D. 2004. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60, 2, 91--110.

Digital Library

[23]

Mitsunaga, T., and Nayar, S. K. 1999. Radiometric self calibration. In Computer Vision and Pattern Recognition (CVPR '99), 374--380.

[24]

Peleg, S., Rousso, B., Rav-Acha, A., and Zomet, A. 2000. Mosaicing on adaptive manifolds. IEEE Transactions on Pattern Analysis and Machine Intelligence 22, 10, 1144--1154.

Digital Library

[25]

Pérez, P., Gangnet, M., and Blake, A. 2003. Poisson image editing. ACM Transactions on Graphics 22, 3, 313--318.

Digital Library

[26]

Rademacher, P., and Bishop, G. 1998. Multiple-center-of-projection images. In Proceedings of SIGGRAPH 98, Computer Graphics Proceedings, Annual Conference Series, 199--206.

Digital Library

[27]

Rav-Acha, A., Shor, Y., and Peleg, S. 2004. Mosaicing with parallax using time warping. In 2004 Computer Vision and Pattern Recognition Workshop (CVPRW'04), vol. 11.

Digital Library

[28]

Roman, A., Garg, G., and Levoy, M. 2004. Interactive design of multi-perspective images for visualizing urban landscapes. In Proceedings of IEEE Visualization, 537--544.

Digital Library

[29]

Seitz, S. M., and Kim, J. 2003. Multiperspective imaging. IEEE Computer Graphics & Applications 23, 6, 16--19.

Digital Library

[30]

Snavely, N., Seitz, S., and Szeliski, R. 2006. Photo tourism: exploring photo collections in 3D. ACM Transactions on Graphics 25, 3, To appear.

Digital Library

[31]

Sun, J., Yuan, L., Jia, J., and Shum, H.-Y. 2005. Image completion with structure propagation. ACM Transactions on Graphics 24, 3 (Aug.), 861--868.

Digital Library

[32]

Szeliski, R., and Shum, H.-Y. 1997. Creating full view panoramic mosaics and environment maps. In Proceedings of SIGGRAPH 97, Computer Graphics Proceedings, Annual Conference Series, 251--258.

Digital Library

[33]

Uyttendaele, M., Eden, A., and Szeliski, R. 2001. Eliminating ghosting and exposure artifacts in image mosaics. In Computer Vision and Pattern Recognition (CVPR 01), 509--516.

[34]

Wexler, Y., and Simakov, D. 2005. Space-time scene manifolds. In International Conference on Computer Vision (ICCV'05), vol. 1, 858--863.

Digital Library

[35]

Wood, D. N., Finkelstein, A., Hughes, J. F., Thayer, C. E., and Salesin, D. H. 1997. Multiperspective panoramas for cel animation. In Proceedings of SIGGRAPH 97, Computer Graphics Proceedings, Annual Conference Series, 243--250.

Digital Library

[36]

Yu, J., and McMillan, L. 2004. A framework for multiperspective rendering. In Proceedings of the 15th Eurographics workshop on Rendering Techniques, 61--68.

Digital Library

[37]

Yu, J., and McMillan, L. 2004. General linear cameras. In European Conference on Computer Vision (ECCV 04), 14--27.

[38]

Zheng, J. Y. 2003. Digital route panoramas. IEEE MultiMedia 10, 3, 57--67.

Digital Library

[39]

Zomet, A., Feldman, D., Peleg, S., and Weinshall, D. 2003. Mosaicing new views: The crossed-slits projection. IEEE Transactions on PAMI 25, 6, 741--754.

Digital Library

Cited By

Hertzmann A(2024)Generative Models for the Psychology of Art and AestheticsEmpirical Studies of the Arts10.1177/02762374241288696Online publication date: 7-Oct-2024
https://doi.org/10.1177/02762374241288696
Hertzmann A(2024)Toward a theory of perspective perception in picturesJournal of Vision10.1167/jov.24.4.2324:4(23)Online publication date: 25-Apr-2024
https://doi.org/10.1167/jov.24.4.23
Song ZLi YWang HGe ZLi JXu X(2023)VR Panoramic Technology-Based Visualization System Development and Application for Flight Training On-Ground Teaching2023 International Conference on Computer Applications Technology (CCAT)10.1109/CCAT59108.2023.00058(276-281)Online publication date: 15-Sep-2023
https://doi.org/10.1109/CCAT59108.2023.00058
Show More Cited By

Index Terms

Photographing long scenes with multi-viewpoint panoramas
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Image and video acquisition
  2. Computer graphics
    1. Shape modeling
      1. Parametric curve and surface models
      2. Volumetric models
2. Theory of computation
  1. Randomness, geometry and discrete structures

Recommendations

Photographing long scenes with multi-viewpoint panoramas
SIGGRAPH '06: ACM SIGGRAPH 2006 Papers

We present a system for producing multi-viewpoint panoramas of long, roughly planar scenes, such as the facades of buildings along a city street, from a relatively sparse set of photographs captured with a handheld still camera that is moved along the ...
A unified rectification method for single viewpoint multi-camera system
AVSS '11: Proceedings of the 2011 8th IEEE International Conference on Advanced Video and Signal Based Surveillance

Stereo matching and 3D reconstruction has been studied for decades as a fundamental problem in the field of computer vision. Recent years, stereo matching and 3D reconstruction with a large field of view, especially using omnidirectional vision and ...
Multi-perspective Panoramas of Long Scenes
ICME '12: Proceedings of the 2012 IEEE International Conference on Multimedia and Expo

This paper presents a system for producing multiperspective panoramas for long scenes from dense collections of images. Multi-perspective panoramas are generated by combining different perspectives, including original and novel perspectives. The latter ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 25, Issue 3

July 2006

742 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/1141911

Issue’s Table of Contents

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 2006

Published in TOG Volume 25, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

128
Total Citations
View Citations
2,075
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)1

Reflects downloads up to 25 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Hertzmann A(2024)Generative Models for the Psychology of Art and AestheticsEmpirical Studies of the Arts10.1177/02762374241288696Online publication date: 7-Oct-2024
https://doi.org/10.1177/02762374241288696
Hertzmann A(2024)Toward a theory of perspective perception in picturesJournal of Vision10.1167/jov.24.4.2324:4(23)Online publication date: 25-Apr-2024
https://doi.org/10.1167/jov.24.4.23
Song ZLi YWang HGe ZLi JXu X(2023)VR Panoramic Technology-Based Visualization System Development and Application for Flight Training On-Ground Teaching2023 International Conference on Computer Applications Technology (CCAT)10.1109/CCAT59108.2023.00058(276-281)Online publication date: 15-Sep-2023
https://doi.org/10.1109/CCAT59108.2023.00058
Huang FLuo XLin HChen YXu ZWan KZhang MPeng PWen XZhou LLi ZWang Y(2022)Innovative Application and Improvement of Panoramic Digital Technology in Indoor Display Scenes2022 6th International Conference on Universal Village (UV)10.1109/UV56588.2022.10185467(1-6)Online publication date: 22-Oct-2022
https://doi.org/10.1109/UV56588.2022.10185467
LYU WZHOU ZCHEN LZHOU Y(2019)A survey on image and video stitchingVirtual Reality & Intelligent Hardware10.3724/SP.J.2096-5796.2018.00081:1(55-83)Online publication date: Feb-2019
https://doi.org/10.3724/SP.J.2096-5796.2018.0008
Shih YLai WLiang C(2019)Distortion-free wide-angle portraits on camera phonesACM Transactions on Graphics10.1145/3306346.332294838:4(1-12)Online publication date: 12-Jul-2019
https://dl.acm.org/doi/10.1145/3306346.3322948
Li YTofighi MMonga V(2019)Robust Alignment for Panoramic Stitching Via an Exact Rank ConstraintIEEE Transactions on Image Processing10.1109/TIP.2019.290980028:10(4730-4745)Online publication date: Oct-2019
https://doi.org/10.1109/TIP.2019.2909800
Ansari SWadhwa NGarg RChen J(2019)Wireless Software Synchronization of Multiple Distributed Cameras2019 IEEE International Conference on Computational Photography (ICCP)10.1109/ICCPHOT.2019.8747340(1-9)Online publication date: May-2019
https://doi.org/10.1109/ICCPHOT.2019.8747340
Alfarrarjeh AKim SBright AHegde VAkshansh AShahabi C(2019)Spatial Aggregation of Visual Features for Image Data Search in a Large Geo-Tagged Image Dataset2019 IEEE Fifth International Conference on Multimedia Big Data (BigMM)10.1109/BigMM.2019.00-43(48-57)Online publication date: Sep-2019
https://doi.org/10.1109/BigMM.2019.00-43
Lu JBai Y(2018)Research on Low Altitude Aerial Image Stitching2018 37th Chinese Control Conference (CCC)10.23919/ChiCC.2018.8483043(9292-9296)Online publication date: Jul-2018
https://doi.org/10.23919/ChiCC.2018.8483043
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents