article

Shadow matting and compositing

Authors:

Yung-Yu Chuang,

David H. Salesin,

Richard SzeliskiAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 22, Issue 3

Pages 494 - 500

https://doi.org/10.1145/882262.882298

Published: 01 July 2003 Publication History

Abstract

In this paper, we describe a method for extracting shadows from one natural scene and inserting them into another. We develop physically-based shadow matting and compositing equations and use these to pull a shadow matte from a source scene in which the shadow is cast onto an arbitrary planar background. We then acquire the photometric and geometric properties of the target scene by sweeping oriented linear shadows (cast by a straight object) across it. From these shadow scans, we can construct a shadow displacement map without requiring camera or light source calibration. This map can then be used to deform the original shadow matte. We demonstrate our approach for both indoor scenes with controlled lighting and for outdoor scenes using natural lighting.

Supplementary Material

MP4 File (goldman_chuang_shadow.mp4)

Download
29.75 MB

References

[1]

BERTALMIO, M., SAPIRO, G., CASELLES, V., AND BALLESTER, C. 2000. Image inpainting. In Proceedings of SIGGRAPH 2000, 417--424.

Digital Library

[2]

BOUGUET, J.-Y., AND PERONA, P. 1998. 3D photography on your desk. In Proceedings of IEEE International Conference on Computer Vision (ICCV 98), 43--50.

Digital Library

[3]

CHUANG, Y.-Y., ZONGKER, D. E., HINDORFF, J., CURLESS, B., SALESIN, D. H., AND SZELISKI, R. 2000. Environment matting extensions: Towards higher accuracy and real-time capture. In Proceedings of ACM SIGGRAPH 2000, 121--130.

Digital Library

[4]

CHUANG, Y.-Y., CURLESS, B., SALESIN, D. H., AND SZELISKI, R. 2001. A Bayesian approach to digital matting. In Proceedings of Computer Vision and Pattern Recognition (CVPR 2001), vol. II, 264--271.

[5]

CHUANG, Y.-Y., AGARWALA, A., CURLESS, B., SALESIN, D. H., AND SZELISKI, R. 2002. Video matting of complex scenes. ACM Transactions on Graphics 21, 3, 243--248.

Digital Library

[6]

DEBEVEC, P. 1998. Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography. In Proceedings of SIGGRAPH 98, 189--198.

Digital Library

[7]

FINLAYSON, G. D., HORDLEY, S. D., AND DREW, M. S. 2002. Removing shadows from images. In Proceedings of European Conference on Computer Vision (ECCV 2002), vol. 2353 of LNCS, 823--836.

Digital Library

[8]

KUMAR, R., ANANDAN, P., AND HANNA, K. 1994. Direct recovery of shape from multiple views: A parallax based approach. In Twelfth International Conference on Pattern Recognition (ICPR'94), 685--688.

[9]

LORENSEN, W. E., AND CLINE, H. E. 1987. Marching cubes: A high resolution 3D surface construction algorithm. In Computer Graphics (Proceedings of ACM SIGGRAPH '90), 163--169.

Digital Library

[10]

MATSUSHITA, Y., KANG, S. B., LIN, S., SHUM, H.-Y., AND TONG, X. 2002. Lighting interpolation by shadow morphing using intrinsic lumigraphs. In Proceedings of Pacific Graphics 2002, 58--65.

Digital Library

[11]

PELLACINI, F., TOLE, P., AND GREENBERG, D. P. 2002. A user interface for interactive cinematic shadow design. ACM Transactions on Graphics 21, 3, 563--566.

Digital Library

[12]

PERONA, P., AND MALIK, J. 1990. Scale space and edge detection using anisotropic diffusion. IEEE Trans. on Pattern Analysis and Machine Intelligence 12, 7 (July), 629--639.

Digital Library

[13]

PETROVIC, L., FUJITO, B., WILLIAMS, L., AND FINKELSTEIN, A. 2000. Shadows for cel animation. In Siggraph 2000, Computer Graphics Proceedings, K. Akeley, Ed., Annual Conference Series, 511--516.

Digital Library

[14]

PORTER, T., AND DUFF, T. 1984. Compositing digital images. In Computer Graphics (Proceedings of ACM SIGGRAPH '84), 253--259.

Digital Library

[15]

RUZON, M. A., AND TOMASI, C. 2000. Alpha estimation in natural images. In Proceedings of Computer Vision and Pattern Recognition (CVPR 2000), 18--25.

[16]

SAVARESE, S., RUSHMEIER, H., BERNARDINI, F., AND PERONA, P. 2001. Shadow carving. In Proceedings of IEEE International Conference on Computer Vision (ICCV 2001), 190--197.

[17]

SMITH, A. R., AND BLINN, J. F. 1996. Blue screen matting. In Proceedings of ACM SIGGRAPH 96, 259--268.

Digital Library

[18]

SZELISKI, R., AVIDAN, S., AND ANANDAN, P. 2000. Layer extraction from multiple images containing reflections and transparency. In Proceedings of Computer Vision and Pattern Recognition (CVPR 2000), 246--253.

[19]

WEISS, Y. 2001. Deriving intrinsic images from image sequences. In Proceedings of IEEE International Conference on Computer Vision (ICCV 2001), 68--75.

[20]

WRIGHT, S. 2001. Digital Compositing for Film and Video. Focal Press.

Digital Library

[21]

ZONGKER, D. E., WERNER, D. M., CURLESS, B., AND SALESIN, D. H. 1999. Environment matting and compositing. In Proceedings of ACM SIGGRAPH 99, 205--214.

Digital Library

Cited By

Zhou YZhu Z(2024)DMAED: Dynamic Matte Aware Encoder-Decoder for Shadow RemovalProceedings of the 2024 3rd International Conference on Cyber Security, Artificial Intelligence and Digital Economy10.1145/3672919.3672966(251-256)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1145/3672919.3672966
Valença LZhang JGharbi MHold-Geoffroy YLalonde J(2023)Shadow Harmonization for Realistic CompositingSIGGRAPH Asia 2023 Conference Papers10.1145/3610548.3618227(1-12)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.1145/3610548.3618227
Liu JWang QFan HLi WQu LTang Y(2023)A Decoupled Multi-Task Network for Shadow RemovalIEEE Transactions on Multimedia10.1109/TMM.2023.325227125(9449-9463)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TMM.2023.3252271
Show More Cited By

Index Terms

Shadow matting and compositing
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks
        Scene understanding
  2. Computer graphics
    1. Image manipulation

Recommendations

Video matting of complex scenes

This paper describes a new framework for video matting, the process of pulling a high-quality alpha matte and foreground from a video sequence. The framework builds upon techniques in natural image matting, optical flow computation, and background ...
Shadow matting and compositing
SIGGRAPH '03: ACM SIGGRAPH 2003 Papers

In this paper, we describe a method for extracting shadows from one natural scene and inserting them into another. We develop physically-based shadow matting and compositing equations and use these to pull a shadow matte from a source scene in which the ...
Environment Matting and Compositing
Seminal Graphics Papers: Pushing the Boundaries, Volume 2

This paper introduces a new process, environment matting, which captures not just a foreground object and its traditional opacity matte from a real-world scene, but also a description of how that object refracts and reflects light, which we call an ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 22, Issue 3

July 2003

683 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/882262

Issue’s Table of Contents

Copyright © 2003 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 2003

Published in TOG Volume 22, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

54
Total Citations
View Citations
1,877
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)0

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhou YZhu Z(2024)DMAED: Dynamic Matte Aware Encoder-Decoder for Shadow RemovalProceedings of the 2024 3rd International Conference on Cyber Security, Artificial Intelligence and Digital Economy10.1145/3672919.3672966(251-256)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1145/3672919.3672966
Valença LZhang JGharbi MHold-Geoffroy YLalonde J(2023)Shadow Harmonization for Realistic CompositingSIGGRAPH Asia 2023 Conference Papers10.1145/3610548.3618227(1-12)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.1145/3610548.3618227
Liu JWang QFan HLi WQu LTang Y(2023)A Decoupled Multi-Task Network for Shadow RemovalIEEE Transactions on Multimedia10.1109/TMM.2023.325227125(9449-9463)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TMM.2023.3252271
Luo WXie XDeng KYang LLai J(2023)Learning Shadow Removal From Unpaired Samples via Reciprocal LearningIEEE Transactions on Image Processing10.1109/TIP.2023.328543932(3455-3464)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TIP.2023.3285439
Le HSamaras D(2022)Physics-Based Shadow Image Decomposition for Shadow RemovalIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2021.312493444:12(9088-9101)Online publication date: 1-Dec-2022
https://doi.org/10.1109/TPAMI.2021.3124934
Ohkawara MFujishiro I(2022)Illumination-aware group portrait compositorThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-022-02508-z38:12(4009-4018)Online publication date: 20-May-2022
https://dl.acm.org/doi/10.1007/s00371-022-02508-z
Chen YRong FDuggal SWang SYan XManivasagam SXue SYumer EUrtasun R(2021)GeoSim: Realistic Video Simulation via Geometry-Aware Composition for Self-Driving2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR46437.2021.00715(7226-7236)Online publication date: Jun-2021
https://doi.org/10.1109/CVPR46437.2021.00715
Raditya CRizky MMayranio SSoewito B(2021)The Effectivity of Color For Chroma-Key TechniquesProcedia Computer Science10.1016/j.procs.2021.01.007179(281-288)Online publication date: 2021
https://doi.org/10.1016/j.procs.2021.01.007
NAGOYA MKIMURA TTSUJI H(2020)A Simple Depth-Key-Based Image Composition Considering Object Movement in Depth DirectionIEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences10.1587/transfun.2020SML0005E103.A:12(1603-1608)Online publication date: 1-Dec-2020
https://doi.org/10.1587/transfun.2020SML0005
Zhang LLong CYan QZhang XXiao C(2020)CLA‐GAN: A Context and Lightness Aware Generative Adversarial Network for Shadow RemovalComputer Graphics Forum10.1111/cgf.1416139:7(483-494)Online publication date: 24-Nov-2020
https://doi.org/10.1111/cgf.14161
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents