article

Efficient gradient-domain compositing using quadtrees

Author:

Aseem AgarwalaAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 26, Issue 3

Pages 94 - es

https://doi.org/10.1145/1276377.1276495

Published: 29 July 2007 Publication History

Abstract

We describe a hierarchical approach to improving the efficiency of gradient-domain compositing, a technique that constructs seamless composites by combining the gradients of images into a vector field that is then integrated to form a composite. While gradient-domain compositing is powerful and widely used, it suffers from poor scalability. Computing an n pixel composite requires solving a linear system with n variables; solving such a large system quickly overwhelms the main memory of a standard computer when performed for multi-megapixel composites, which are common in practice. In this paper we show how to perform gradient-domain compositing approximately by solving an O(p) linear system, where p is the total length of the seams between image regions in the composite; for typical cases, p is O(√n). We achieve this reduction by transforming the problem into a space where much of the solution is smooth, and then utilize the pattern of this smoothness to adaptively subdivide the problem domain using quadtrees. We demonstrate the merits of our approach by performing panoramic stitching and image region copy-and-paste in significantly reduced time and memory while achieving visually identical results.

Supplementary Material

JPG File (pps093.jpg)

Download
16.82 KB

MP4 File (pps093.mp4)

Download
34.60 MB

References

[1]

Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., and Cohen, M. 2004. Interactive digital photomontage. ACM Transactions on Graphics 23, 3 (Aug.), 294--302.

Digital Library

[2]

Agarwala, A., Zheng, K. C., Pal, C., Agrawala, M., Cohen, M., Curless, B., Salesin, D. H., and Szeliski, R. 2005. Panoramic video textures. ACM Transactions on Graphics 24, 3 (Aug.), 821--827.

Digital Library

[3]

Agarwala, A., Agrawala, M., Cohen, M., Salesin, D., and Szeliski, R. 2006. Photographing long scenes with multi-viewpoint panoramas. ACM Transactions on Graphics 25, 3 (July), 853--861.

Digital Library

[4]

Agrawal, A., Raskar, R., Nayar, S. K., and Li, Y. 2005. Removing photography artifacts using gradient projection and flash-exposure sampling. ACM Transactions on Graphics 24, 3 (Aug.), 828--835.

Digital Library

[5]

Bae, S., Paris, S., and Durand, F. 2006. Two-scale tone management for photographic look. ACM Transactions on Graphics 25, 3 (July), 637--645.

Digital Library

[6]

Bolz, J., Farmer, I., Grinspun, E., and Schröder, P. 2003. Sparse matrix solvers on the GPU: Conjugate gradients and multigrid. ACM Transactions on Graphics 22, 3 (July), 917--924.

Digital Library

[7]

Dyer, C. 1982. The space efficiency of quadtrees. Computer Graphics and Image Processing 19, 4 (Aug.), 335--348.

[8]

Fattal, R., Lischinski, D., and Werman, M. 2002. Gradient domain high dynamic range compression. ACM Transactions on Graphics 21, 3, 249--256.

Digital Library

[9]

Finlayson, G., and Drew, S. H. M. 2002. Removing shadows from images. In European Conference on Computer Vision (ECCV 02), 823--831.

Digital Library

[10]

Fuhrmann, D. R. 1988. Quadtree traversal algorithms for pointer-based and depth-first representations. IEEE Transactions on Pattern Analysis and Machine Intelligence 10, 6, 955--960.

Digital Library

[11]

Georgiev, T. 2004. Photoshop healing brush: a tool for seamless cloning. In Workshop on Applications of Computer Vision (ECCV 2004), 1--8.

[12]

Goldman, D. B., and Chen, J.-H. 2005. Vignette and exposure calibration and compensation. In International Conference on Computer Vision (ICCV 05), 899--906.

Digital Library

[13]

Hays, J., and Efros, A. 2007. Scene completion using millions of photographs. ACM Transactions on Graphics 26, 3, To appear.

Digital Library

[14]

Jia, J., Sun, J., Tang, C.-K., and Shum, H.-Y. 2006. Drag-and-drop pasting. ACM Transactions on Graphics 25, 3 (July), 631--637.

Digital Library

[15]

Kwatra, V., Schödl, A., Essa, I., Turk, G., and Bobick, A. 2003. Graphcut textures: Image and video synthesis using graph cuts. ACM Transactions on Graphics 22, 3, 277--286.

Digital Library

[16]

Land, E. H. 1977. The retinex theory of color vision. Scientific American 237, 6, 108--128.

[17]

Levin, A., Zomet, A., Peleg, S., and Weiss, Y. 2004. Seamless image stitching in the gradient domain. In European Conference on Computer Vision (ECCV 04), 377--389.

[18]

Losasso, F., Gibou, F., and Fedkiw, R. 2004. Simulating water and smoke with an octree data structure. ACM Transactions on Graphics 23, 3 (Aug.), 457--462.

Digital Library

[19]

Moore, D. 1995. The cost of balancing generalized quadtrees. In Proceedings of the Third ACM Symposium on Solid Modeling and Applications, 305--312.

Digital Library

[20]

Palmer, S. E. 1999. Vision Science: Photons to Phenomenology. The MIT Press.

[21]

Pérez, P., Gangnet, M., and Blake, A. 2003. Poisson image editing. ACM Transactions on Graphics 22, 3 (July), 313--318.

Digital Library

[22]

Saad, Y. 2003. Iterative methods for sparse linear systems, 2nd ed. Society for Industrial and Applied Mathematics (SIAM).

Digital Library

[23]

Samet, H. 1990. Applications for spatial data structures: computer graphics, image processing, and GIS. Addison-Wesley.

Digital Library

[24]

Shewchuk, J. R. 1994. An introduction to the conjugate gradient method without the agonizing pain. Tech. Rep. CS-94-125, Carnegie Mellon University.

Digital Library

[25]

Simchony, T., Chellappa, R., and Shao, M. 1990. Direct analytical methods for solving Poisson equations in computer vision problems. IEEE Transactions on Pattern Analysis and Machine Intelligence 12, 5, 435--446.

Digital Library

[26]

Sun, J., Jia, J., Tang, C.-K., and Shum, H.-Y. 2004. Poisson matting. ACM Transactions on Graphics 23, 3 (Aug.), 315--321.

Digital Library

[27]

Szeliski, R., and Shum, H.-Y. 1996. Motion estimation with quadtree splines. IEEE Transactions on Pattern Analysis and Machine Intelligence 18, 12, 1199--1210.

Digital Library

[28]

Szeliski, R. 1990. Fast surface interpolation using hierarchical basis functions. IEEE Transactions on Pattern Analysis and Machine Intelligence 12, 6, 513--528.

Digital Library

[29]

Szeliski, R. 2006. Locally adapted hierarchical basis preconditioning. ACM Transactions on Graphics 25, 3 (July), 1135--1143.

Digital Library

[30]

Toledo, S. 1999. A survey of out-of-core algorithms in numerical linear algebra. In External Memory Algorithms, DIMACS Series in Discrete Mathematics and Theoretical Computer Science. 161--180.

Digital Library

[31]

Wang, H., Raskar, R., and Ahuja, N. 2004. Seamless video editing. In Proceedings of the International Conference on Pattern Recognition (ICPR), 858--861.

Digital Library

[32]

Weiss, Y. 2001. Deriving intrinsic images from image sequences. In International Conference On Computer Vision (ICCV 01), 68--75.

[33]

Zomet, A., Levin, A., Peleg, S., and Weiss, Y. 2006. Seamless image stitching by minimizing false edges. IEEE Transactions on Image Processing 15, 4, 969--977.

Digital Library

Cited By

Zhang YYuan JYan C(2024)Enhanced Seamless Video Fusion: A Convolutional Pyramid-Based 3D Integration AlgorithmSensors10.3390/s2406185224:6(1852)Online publication date: 14-Mar-2024
https://doi.org/10.3390/s24061852
Petruzza SSumma BGooch ALaney CGoulden TSchreiner JCallahan SPascucci V(2023)Interactive Visualization and Portable Image Blending of Massive Aerial Image Mosaics2023 IEEE International Conference on Big Data (BigData)10.1109/BigData59044.2023.10386428(3365-3370)Online publication date: 15-Dec-2023
https://doi.org/10.1109/BigData59044.2023.10386428
Tanathong SSmith WRemde S(2022)SurfaceView: Seamless and Tile-Based Orthomosaics Using Millions of Street-Level Images From Vehicle-Mounted CamerasIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2020.303692823:4(3482-3497)Online publication date: Apr-2022
https://doi.org/10.1109/TITS.2020.3036928
Show More Cited By

Index Terms

Efficient gradient-domain compositing using quadtrees
1. Computing methodologies

Recommendations

Efficient gradient-domain compositing using quadtrees
SIGGRAPH '07: ACM SIGGRAPH 2007 papers

We describe a hierarchical approach to improving the efficiency of gradient-domain compositing, a technique that constructs seamless composites by combining the gradients of images into a vector field that is then integrated to form a composite. While ...
Shadow matting and compositing
SIGGRAPH '03: ACM SIGGRAPH 2003 Papers

In this paper, we describe a method for extracting shadows from one natural scene and inserting them into another. We develop physically-based shadow matting and compositing equations and use these to pull a shadow matte from a source scene in which the ...
Improved deep image compositing using subpixel masks
DigiPro '15: Proceedings of the 2015 Symposium on Digital Production

We present an improved method of producing and manipulating deep pixel data which retains important surface information calculated during the execution of the rendering algorithm for later use during compositing, allowing operations normally performed ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 26, Issue 3

July 2007

976 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/1276377

Issue’s Table of Contents

Copyright © 2007 ACM.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 July 2007

Published in TOG Volume 26, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

116
Total Citations
View Citations
1,592
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)1

Reflects downloads up to 26 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang YYuan JYan C(2024)Enhanced Seamless Video Fusion: A Convolutional Pyramid-Based 3D Integration AlgorithmSensors10.3390/s2406185224:6(1852)Online publication date: 14-Mar-2024
https://doi.org/10.3390/s24061852
Petruzza SSumma BGooch ALaney CGoulden TSchreiner JCallahan SPascucci V(2023)Interactive Visualization and Portable Image Blending of Massive Aerial Image Mosaics2023 IEEE International Conference on Big Data (BigData)10.1109/BigData59044.2023.10386428(3365-3370)Online publication date: 15-Dec-2023
https://doi.org/10.1109/BigData59044.2023.10386428
Tanathong SSmith WRemde S(2022)SurfaceView: Seamless and Tile-Based Orthomosaics Using Millions of Street-Level Images From Vehicle-Mounted CamerasIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2020.303692823:4(3482-3497)Online publication date: Apr-2022
https://doi.org/10.1109/TITS.2020.3036928
Wang XZhu Y(2022)A Quadtree Based Piecewise Poisson Seamless Composition Algorithm for Image Stitching2022 International Conference on Informatics, Networking and Computing (ICINC)10.1109/ICINC58035.2022.00049(209-213)Online publication date: Oct-2022
https://doi.org/10.1109/ICINC58035.2022.00049
Metzner MReisinger DOrtmann JGrünhöfer LHandwerker ABlank AFranke J(2022)An Approach for Direct Offline Programming of High Precision Assembly Tasks on 3D Scans Using Tactile Control and Automatic Program AdaptionAnnals of Scientific Society for Assembly, Handling and Industrial Robotics 202110.1007/978-3-030-74032-0_18(215-225)Online publication date: 1-Jan-2022
https://doi.org/10.1007/978-3-030-74032-0_18
Lee GHan J(2021)Viewport Rendering Algorithm with a Curved Surface for a Wide FOV in 360° ImagesApplied Sciences10.3390/app1103113311:3(1133)Online publication date: 26-Jan-2021
https://doi.org/10.3390/app11031133
Zhu QShang QHu HYu HZhong R(2021)Structure-aware completion of photogrammetric meshes in urban road environmentISPRS Journal of Photogrammetry and Remote Sensing10.1016/j.isprsjprs.2021.02.010175(56-70)Online publication date: May-2021
https://doi.org/10.1016/j.isprsjprs.2021.02.010
Licorish CFaraj NSumma B(2020)Adaptive Compositing and Navigation of Variable Resolution ImagesComputer Graphics Forum10.1111/cgf.1417840:1(138-150)Online publication date: 30-Oct-2020
https://doi.org/10.1111/cgf.14178
Hou FSun QFang ZLiu YHu SQin HHao AHe Y(2020)Poisson Vector Graphics (PVG)IEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2018.286747826:2(1361-1371)Online publication date: 1-Feb-2020
https://doi.org/10.1109/TVCG.2018.2867478
Zhu ZLiu HLu JHu S(2020)A Metric for Video Blending Quality AssessmentIEEE Transactions on Image Processing10.1109/TIP.2019.295529429(3014-3022)Online publication date: 1-Jan-2020
https://dl.acm.org/doi/10.1109/TIP.2019.2955294
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents