research-article

Transfusive image manipulation

Authors:

Alexander Hornung,

Olga SorkineAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 31, Issue 6

Article No.: 176, Pages 1 - 9

https://doi.org/10.1145/2366145.2366195

Published: 01 November 2012 Publication History

Abstract

We present a method for consistent automatic transfer of edits applied to one image to many other images of the same object or scene. By introducing novel, content-adaptive weight functions we enhance the non-rigid alignment framework of Lucas-Kanade to robustly handle changes of view point, illumination and non-rigid deformations of the subjects. Our weight functions are content-aware and possess high-order smoothness, enabling to define high-quality image warping with a low number of parameters using spatially-varying weighted combinations of affine deformations. Optimizing the warp parameters leads to subpixel-accurate alignment while maintaining computation efficiency. Our method allows users to perform precise, localized edits such as simultaneous painting on multiple images in real-time, relieving them from tedious and repetitive manual reapplication to each individual image.

References

[1]

Agarwala, A., Hertzmann, A., Salesin, D., and Seitz, S. M. 2004. Keyframe-based tracking for rotoscoping and animation. ACM Trans. Graph. 23, 3, 584--591.

Digital Library

[2]

An, X., and Pellacini, F. 2008. AppProp: all-pairs appearance-space edit propagation. ACM Trans. Graph. 27, 3.

Digital Library

[3]

Andersen, E. D., and Andersen, K. D. 2000. The mosek interior point optimizer for linear programming: an implementation of the homogeneous algorithm. In High Performance Optimization. Kluwer Academic Publishers, 197--232.

[4]

Baker, S., and Matthews, I. 2004. Lucas-Kanade 20 years on: A unifying framework. Int. J. Comput. Vision 56, 3, 221--255.

Digital Library

[5]

Baker, S., Scharstein, D., Lewis, J. P., Roth, S., Black, M. J., and Szeliski, R. 2011. A database and evaluation methodology for optical flow. Int. J. Comput. Vision 92, 1, 1--31.

Digital Library

[6]

Barnes, C., Shechtman, E., Goldman, D. B., and Finkelstein, A. 2010. The generalized patchmatch correspondence algorithm. In Proc. ECCV.

Digital Library

[7]

Berthouzoz, F., Li, W., Dontcheva, M., and Agrawala, M. 2011. A framework for content-adaptive photo manipulation macros: Application to face, landscape, and global manipulations. ACM Trans. Graph. 30, 5, 120.

Digital Library

[8]

Bhat, P., Zitnick, C. L., Snavely, N., Agarwala, A., Agrawala, M., Curless, B., Cohen, M., and Kang, S. B. 2007. Using photographs to enhance videos of a static scene. In Proc. EGSR, 327--338.

Digital Library

[9]

Bitouk, D., Kumar, N., Dhillon, S., Belhumeur, P. N., and Nayar, S. K. 2008. Face swapping: automatically replacing faces in photographs. ACM Trans. Graph. 27, 3.

Digital Library

[10]

Brand, M., and Pletscher, P. 2008. A conditional random field for automatic photo editing. In Proc. CVPR.

[11]

Cohen-Or, D., Sorkine, O., Gal, R., Leyvand, T., and Xu, Y.-Q. 2006. Color harmonization. ACM Trans. Graph. 25, 3.

Digital Library

[12]

Farbman, Z., Fattal, R., Lischinski, D., and Szeliski, R. 2008. Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM Trans. Graph. 27, 3.

Digital Library

[13]

Farbman, Z., Fattal, R., and Lischinski, D. 2010. Diffusion maps for edge-aware image editing. ACM Trans. Graph. 29, 6.

Digital Library

[14]

Fattal, R., Carroll, R., and Agrawala, M. 2009. Edge-based image coarsening. ACM Trans. Graph. 29, 1, 1--11.

Digital Library

[15]

Fattal, R. 2009. Edge-avoiding wavelets and their applications. ACM Trans. Graph. 28, 3.

Digital Library

[16]

Finch, M., Snyder, J., and Hoppe, H. 2011. Freeform vector graphics with controlled thin-plate splines. ACM Trans. Graph. 30, 6, 166:1--166:10.

Digital Library

[17]

HaCohen, Y., Shechtman, E., Goldman, D. B., and Lischinski, D. 2011. Non-rigid dense correspondence with applications for image enhancement. ACM Trans. Graph. 30, 4.

Digital Library

[18]

Hasinoff, S. W., Jóźwiak, M., Durand, F., and Freeman, W. T. 2010. Search-and-replace editing for personal photo collections. In Proc. ICCP.

[19]

Hays, J., and Efros, A. A. 2007. Scene completion using millions of photographs. ACM Trans. Graph. 26, 3.

Digital Library

[20]

Jacobson, A., and Sorkine, O. 2012. A cotangent laplacian for images as surfaces. Tech. Rep. 757, ETH Zurich, April.

[21]

Jacobson, A., Baran, I., Popovic, J., and Sorkine, O. 2011. Bounded biharmonic weights for real-time deformation. ACM Trans. Graph. 30, 4.

Digital Library

[22]

Levin, A., Lischinski, D., and Weiss, Y. 2004. Colorization using optimization. ACM Trans. Graph. 23, 3, 689--694.

Digital Library

[23]

Li, Y., Adelson, E. H., and Agarwala, A. 2008. Scribble-boost: Adding classification to edge-aware interpolation of local image and video adjustments. Comput. Graph. Forum 27, 4.

Digital Library

[24]

Li, Y., Ju, T., and Hu, S.-M. 2010. Instant propagation of sparse edits on images and videos. Comput. Graph. Forum 29, 7.

[25]

Lischinski, D., Farbman, Z., Uyttendaele, M., and Szeliski, R. 2006. Interactive local adjustment of tonal values. ACM Trans. Graph. 25, 3, 646--653.

Digital Library

[26]

Liu, C., Yuen, J., Torralba, A., Sivic, J., and Freeman, W. T. 2008. Sift flow: Dense correspondence across different scenes. In Proc. ECCV: Part III, 28--42.

Digital Library

[27]

Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60, 2, 91--110.

Digital Library

[28]

Lucas, B. D., and Kanade, T. 1981. An iterative image registration technique with an application to stereo vision. In Proc. International Joint Conference on Artificial Intelligence.

Digital Library

[29]

Meyer, M., Desbrun, M., Schröder, P., and Barr, A. H. 2003. Discrete differential-geometry operators for triangulated 2-manifolds. In Visualization and Mathematics III. 35--57.

[30]

Photoshop. 2012. Version 12.0.4. Adobe Systems, Inc.

[31]

Pinkall, U., and Polthier, K. 1993. Computing discrete minimal surfaces and their conjugates. Experiment. Math. 2, 1.

[32]

Rav-Acha, A., Kohli, P., Rother, C., and Fitzgibbon, A. W. 2008. Unwrap mosaics: a new representation for video editing. ACM Trans. Graph. 27, 3.

Digital Library

[33]

Reinhard, E., Ashikhmin, M., Gooch, B., and Shirley, P. 2001. Color transfer between images. IEEE Comput. Graph. and Applications 21, 5, 34--41.

Digital Library

[34]

Sand, P., and Teller, S. J. 2004. Video matching. ACM Trans. Graph. 23, 3, 592--599.

Digital Library

[35]

Scharstein, D., and Szeliski, R. 2002. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vision 47, 1--3, 7--42.

Digital Library

[36]

Zimmer, H., Bruhn, A., and Weickert, J. 2011. Optic flow in harmony. Int. J. Comput. Vision 93, 3, 368--388.

Digital Library

Cited By

Ouyang WDong YYang LSi JPan X(2024)I2VEdit: First-Frame-Guided Video Editing via Image-to-Video Diffusion ModelsSIGGRAPH Asia 2024 Conference Papers10.1145/3680528.3687656(1-11)Online publication date: 3-Dec-2024
https://dl.acm.org/doi/10.1145/3680528.3687656
Bermudez LDabby NLin YHilmarsdottir SSundararajan NKar S(2021)A Learning-Based Approach to Parametric Rotoscoping of Multi-Shape Systems2021 IEEE Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV48630.2021.00082(776-785)Online publication date: Jan-2021
https://doi.org/10.1109/WACV48630.2021.00082
Liu SZhang XZhang ZZhang RZhu JRussell B(2021)Editing Conditional Radiance Fields2021 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV48922.2021.00572(5753-5763)Online publication date: Oct-2021
https://doi.org/10.1109/ICCV48922.2021.00572
Show More Cited By

Index Terms

Transfusive image manipulation
1. Applied computing
  1. Document management and text processing
    1. Document capture
      1. Graphics recognition and interpretation
2. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Computer graphics
    1. Image manipulation
    2. Rendering

Recommendations

Non-parametric 3D Shape Warping
ICPR '10: Proceedings of the 2010 20th International Conference on Pattern Recognition

A method is presented for non-rigid alignment of a source shape to a target shape through estimating and interpolating point-wise correspondences between their surfaces given as point clouds. The resulting mapping can be non-smooth and non-isometric, ...
Saliency-aware inter-image color transfer for image manipulation
Abstract
This paper proposes a novel saliency-aware inter-image color transfer method to perform image manipulation. Specifically, given the source image, the candidate images are first retrieved from a group of images with the same semantic category, and ...
Image deblurring with blurred/noisy image pairs
SIGGRAPH '07: ACM SIGGRAPH 2007 papers

Taking satisfactory photos under dim lighting conditions using a hand-held camera is challenging. If the camera is set to a long exposure time, the image is blurred due to camera shake. On the other hand, the image is dark and noisy if it is taken with a ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 31, Issue 6

November 2012

794 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/2366145

Issue’s Table of Contents

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 November 2012

Published in TOG Volume 31, Issue 6

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

29
Total Citations
View Citations
644
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ouyang WDong YYang LSi JPan X(2024)I2VEdit: First-Frame-Guided Video Editing via Image-to-Video Diffusion ModelsSIGGRAPH Asia 2024 Conference Papers10.1145/3680528.3687656(1-11)Online publication date: 3-Dec-2024
https://dl.acm.org/doi/10.1145/3680528.3687656
Bermudez LDabby NLin YHilmarsdottir SSundararajan NKar S(2021)A Learning-Based Approach to Parametric Rotoscoping of Multi-Shape Systems2021 IEEE Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV48630.2021.00082(776-785)Online publication date: Jan-2021
https://doi.org/10.1109/WACV48630.2021.00082
Liu SZhang XZhang ZZhang RZhu JRussell B(2021)Editing Conditional Radiance Fields2021 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV48922.2021.00572(5753-5763)Online publication date: Oct-2021
https://doi.org/10.1109/ICCV48922.2021.00572
Souto Ferreira LSacht L(2020)Bounded biharmonic blending of Möbius transformations for flexible omnidirectional image rectificationComputers & Graphics10.1016/j.cag.2020.09.01193(51-60)Online publication date: Dec-2020
https://doi.org/10.1016/j.cag.2020.09.011
Bau DLiu SWang TZhu JTorralba A(2020)Rewriting a Deep Generative ModelComputer Vision – ECCV 202010.1007/978-3-030-58452-8_21(351-369)Online publication date: 23-Aug-2020
https://dl.acm.org/doi/10.1007/978-3-030-58452-8_21
Jamriška OSochorová ŠTexler OLukáč MFišer JLu JShechtman ESýkora D(2019)Stylizing video by exampleACM Transactions on Graphics10.1145/3306346.332300638:4(1-11)Online publication date: 12-Jul-2019
https://dl.acm.org/doi/10.1145/3306346.3323006
Cambra AMurillo AMuñoz A(2018)A generic tool for interactive complex image editingThe Visual Computer: International Journal of Computer Graphics10.5555/3288062.328806834:11(1493-1505)Online publication date: 1-Nov-2018
https://dl.acm.org/doi/10.5555/3288062.3288068
Hennessey JLi WRussell BShechtman EMitra N(2017)Transferring image-based edits for multi-channel compositingACM Transactions on Graphics10.1145/3130800.313084236:6(1-16)Online publication date: 20-Nov-2017
https://dl.acm.org/doi/10.1145/3130800.3130842
Bonneel NTompkin JSun DWang OSunkavalli KParis SPfister H(2017)Consistent Video Filtering for Camera ArraysComputer Graphics Forum10.1111/cgf.1313536:2(397-407)Online publication date: 1-May-2017
https://dl.acm.org/doi/10.1111/cgf.13135
Wu GMasia BJarabo AZhang YWang LDai QChai TLiu Y(2017)Light Field Image Processing: An OverviewIEEE Journal of Selected Topics in Signal Processing10.1109/JSTSP.2017.274712611:7(926-954)Online publication date: Oct-2017
https://doi.org/10.1109/JSTSP.2017.2747126
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents