Article

Shadow matting and compositing

Authors:

Yung-Yu Chuang,

David H. Salesin,

Richard SzeliskiAuthors Info & Claims

SIGGRAPH '03: ACM SIGGRAPH 2003 Papers

Pages 494 - 500

https://doi.org/10.1145/1201775.882298

Published: 01 July 2003 Publication History

Abstract

In this paper, we describe a method for extracting shadows from one natural scene and inserting them into another. We develop physically-based shadow matting and compositing equations and use these to pull a shadow matte from a source scene in which the shadow is cast onto an arbitrary planar background. We then acquire the photometric and geometric properties of the target scene by sweeping oriented linear shadows (cast by a straight object) across it. From these shadow scans, we can construct a shadow displacement map without requiring camera or light source calibration. This map can then be used to deform the original shadow matte. We demonstrate our approach for both indoor scenes with controlled lighting and for outdoor scenes using natural lighting.

Supplementary Material

MP4 File (goldman_chuang_shadow.mp4)

Download
29.75 MB

References

[1]

BERTALMIO, M., SAPIRO, G., CASELLES, V., AND BALLESTER, C. 2000. Image inpainting. In Proceedings of SIGGRAPH 2000, 417--424.

Digital Library

[2]

BOUGUET, J.-Y., AND PERONA, P. 1998. 3D photography on your desk. In Proceedings of IEEE International Conference on Computer Vision (ICCV 98), 43--50.

Digital Library

[3]

CHUANG, Y.-Y., ZONGKER, D. E., HINDORFF, J., CURLESS, B., SALESIN, D. H., AND SZELISKI, R. 2000. Environment matting extensions: Towards higher accuracy and real-time capture. In Proceedings of ACM SIGGRAPH 2000, 121--130.

Digital Library

[4]

CHUANG, Y.-Y., CURLESS, B., SALESIN, D. H., AND SZELISKI, R. 2001. A Bayesian approach to digital matting. In Proceedings of Computer Vision and Pattern Recognition (CVPR 2001), vol. II, 264--271.

[5]

CHUANG, Y.-Y., AGARWALA, A., CURLESS, B., SALESIN, D. H., AND SZELISKI, R. 2002. Video matting of complex scenes. ACM Transactions on Graphics 21, 3, 243--248.

Digital Library

[6]

DEBEVEC, P. 1998. Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography. In Proceedings of SIGGRAPH 98, 189--198.

Digital Library

[7]

FINLAYSON, G. D., HORDLEY, S. D., AND DREW, M. S. 2002. Removing shadows from images. In Proceedings of European Conference on Computer Vision (ECCV 2002), vol. 2353 of LNCS, 823--836.

Digital Library

[8]

KUMAR, R., ANANDAN, P., AND HANNA, K. 1994. Direct recovery of shape from multiple views: A parallax based approach. In Twelfth International Conference on Pattern Recognition (ICPR'94), 685--688.

[9]

LORENSEN, W. E., AND CLINE, H. E. 1987. Marching cubes: A high resolution 3D surface construction algorithm. In Computer Graphics (Proceedings of ACM SIGGRAPH '90), 163--169.

Digital Library

[10]

MATSUSHITA, Y., KANG, S. B., LIN, S., SHUM, H.-Y., AND TONG, X. 2002. Lighting interpolation by shadow morphing using intrinsic lumigraphs. In Proceedings of Pacific Graphics 2002, 58--65.

Digital Library

[11]

PELLACINI, F., TOLE, P., AND GREENBERG, D. P. 2002. A user interface for interactive cinematic shadow design. ACM Transactions on Graphics 21, 3, 563--566.

Digital Library

[12]

PERONA, P., AND MALIK, J. 1990. Scale space and edge detection using anisotropic diffusion. IEEE Trans. on Pattern Analysis and Machine Intelligence 12, 7 (July), 629--639.

Digital Library

[13]

PETROVIC, L., FUJITO, B., WILLIAMS, L., AND FINKELSTEIN, A. 2000. Shadows for cel animation. In Siggraph 2000, Computer Graphics Proceedings, K. Akeley, Ed., Annual Conference Series, 511--516.

Digital Library

[14]

PORTER, T., AND DUFF, T. 1984. Compositing digital images. In Computer Graphics (Proceedings of ACM SIGGRAPH '84), 253--259.

Digital Library

[15]

RUZON, M. A., AND TOMASI, C. 2000. Alpha estimation in natural images. In Proceedings of Computer Vision and Pattern Recognition (CVPR 2000), 18--25.

[16]

SAVARESE, S., RUSHMEIER, H., BERNARDINI, F., AND PERONA, P. 2001. Shadow carving. In Proceedings of IEEE International Conference on Computer Vision (ICCV 2001), 190--197.

[17]

SMITH, A. R., AND BLINN, J. F. 1996. Blue screen matting. In Proceedings of ACM SIGGRAPH 96, 259--268.

Digital Library

[18]

SZELISKI, R., AVIDAN, S., AND ANANDAN, P. 2000. Layer extraction from multiple images containing reflections and transparency. In Proceedings of Computer Vision and Pattern Recognition (CVPR 2000), 246--253.

[19]

WEISS, Y. 2001. Deriving intrinsic images from image sequences. In Proceedings of IEEE International Conference on Computer Vision (ICCV 2001), 68--75.

[20]

WRIGHT, S. 2001. Digital Compositing for Film and Video. Focal Press.

Digital Library

[21]

ZONGKER, D. E., WERNER, D. M., CURLESS, B., AND SALESIN, D. H. 1999. Environment matting and compositing. In Proceedings of ACM SIGGRAPH 99, 205--214.

Digital Library

Cited By

Niu KLiu YWu EXing G(2023)A Boundary-Aware Network for Shadow RemovalIEEE Transactions on Multimedia10.1109/TMM.2022.321442225(6782-6793)Online publication date: 2023
https://doi.org/10.1109/TMM.2022.3214422
Vasluianu FSeizinger TTimofte RCui SHuang JTian SFan MZhang JZhu LWei XWei XLuo ZGustafsson FZhao ZSjölund JSchön TDong XZhang XLi CLeng CYeo WOh WLee YRyu HLuo JJiang CHan MWu QLin WYu LLi XJiang TFan HLiu SXu SSong BChen XZhang SZhou JZhang ZZhao SZheng HGao YWei YWang BRen JLuo YKondo YMiyata RYasue FNaruki TUkita NChang HYang HChen YChiang YHuang ZChen WChen IHsieh CKuo SXianwei LFu HLiu CMa HFu BHe HWang MShe WLiu YNathan SKansal PZhang ZYang HWang YZhang YPhutke SKulkarni AKhan MMurala SVipparthi SYe HLiu ZYang XLiu SWu YJing YYu QZheng NHuang JLong YYao MZhao FZhao BYe NShen NCao YXiong TXia WLi DXia S(2023)NTIRE 2023 Image Shadow Removal Challenge Report2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW59228.2023.00179(1788-1807)Online publication date: Jun-2023
https://doi.org/10.1109/CVPRW59228.2023.00179
Dasgupta SDas AYogamani SDas SEising CBursuc ABhattacharya U(2023)UnShadowNet: Illumination Critic Guided Contrastive Learning for Shadow RemovalIEEE Access10.1109/ACCESS.2023.330557611(87760-87774)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3305576
Show More Cited By

Index Terms

Shadow matting and compositing
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks
        Scene understanding
  2. Computer graphics
    1. Image manipulation

Recommendations

Environment matting and compositing
SIGGRAPH '99: Proceedings of the 26th annual conference on Computer graphics and interactive techniques

This paper introduces a new process, environment matting, which captures not just a foreground object and its traditional opacity matte from a real-world scene, but also a description of how that object refracts and reflects light, which we call an ...
Environment matting extensions: towards higher accuracy and real-time capture
SIGGRAPH '00: Proceedings of the 27th annual conference on Computer graphics and interactive techniques

Environment matting is a generalization of traditional bluescreen matting. By photographing an object in front of a sequence of structured light backdrops, a set of approximate light-transport paths through the object can be computed. The original ...
Video matting of complex scenes
SIGGRAPH '02: Proceedings of the 29th annual conference on Computer graphics and interactive techniques

This paper describes a new framework for video matting, the process of pulling a high-quality alpha matte and foreground from a video sequence. The framework builds upon techniques in natural image matting, optical flow computation, and background ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGGRAPH '03: ACM SIGGRAPH 2003 Papers

July 2003

683 pages

ISBN:1581137095

DOI:10.1145/1201775

Conference Chair:
Alyn P. Rockwood
Colorado School of Mines

Copyright © 2003 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 2003

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

SIGGRAPH03

Sponsor:

SIGGRAPH

SIGGRAPH03: Special Interest Group on Computer Graphics and Interactive Techniques

July 27 - 31, 2003

California, San Diego

Acceptance Rates

SIGGRAPH '03 Paper Acceptance Rate 81 of 424 submissions, 19%;

Overall Acceptance Rate 1,822 of 8,601 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
1,937
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)2

Reflects downloads up to

Other Metrics

View Author Metrics

Citations

Cited By

Niu KLiu YWu EXing G(2023)A Boundary-Aware Network for Shadow RemovalIEEE Transactions on Multimedia10.1109/TMM.2022.321442225(6782-6793)Online publication date: 2023
https://doi.org/10.1109/TMM.2022.3214422
Vasluianu FSeizinger TTimofte RCui SHuang JTian SFan MZhang JZhu LWei XWei XLuo ZGustafsson FZhao ZSjölund JSchön TDong XZhang XLi CLeng CYeo WOh WLee YRyu HLuo JJiang CHan MWu QLin WYu LLi XJiang TFan HLiu SXu SSong BChen XZhang SZhou JZhang ZZhao SZheng HGao YWei YWang BRen JLuo YKondo YMiyata RYasue FNaruki TUkita NChang HYang HChen YChiang YHuang ZChen WChen IHsieh CKuo SXianwei LFu HLiu CMa HFu BHe HWang MShe WLiu YNathan SKansal PZhang ZYang HWang YZhang YPhutke SKulkarni AKhan MMurala SVipparthi SYe HLiu ZYang XLiu SWu YJing YYu QZheng NHuang JLong YYao MZhao FZhao BYe NShen NCao YXiong TXia WLi DXia S(2023)NTIRE 2023 Image Shadow Removal Challenge Report2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW59228.2023.00179(1788-1807)Online publication date: Jun-2023
https://doi.org/10.1109/CVPRW59228.2023.00179
Dasgupta SDas AYogamani SDas SEising CBursuc ABhattacharya U(2023)UnShadowNet: Illumination Critic Guided Contrastive Learning for Shadow RemovalIEEE Access10.1109/ACCESS.2023.330557611(87760-87774)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3305576
Wang XDai JChen BWei DShao Y(2023)A dynamic alignment and illumination‐aware convolution for shadow removalIET Image Processing10.1049/ipr2.1298518:3(785-795)Online publication date: 3-Dec-2023
https://doi.org/10.1049/ipr2.12985
Gao JZheng QGuo Y(2022)Towards Real-world Shadow Removal with a Shadow Simulation Method and a Two-stage Framework2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW56347.2022.00075(598-607)Online publication date: Jun-2022
https://doi.org/10.1109/CVPRW56347.2022.00075
Wang YLiu ATucker RWu JCurless BSeitz SSnavely N(2021)Repopulating Street Scenes2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR46437.2021.00507(5106-5115)Online publication date: Jun-2021
https://doi.org/10.1109/CVPR46437.2021.00507
Lu ECole FDekel TZisserman AFreeman WRubinstein M(2021)Omnimatte: Associating Objects and Their Effects in Video2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR46437.2021.00448(4505-4513)Online publication date: Jun-2021
https://doi.org/10.1109/CVPR46437.2021.00448
Risnandar Wardoyo R(2020)D-ISDET: Double Intensity of Image Shadow Detection and Elimination in Autonomous Vehicle2020 International Conference on Radar, Antenna, Microwave, Electronics, and Telecommunications (ICRAMET)10.1109/ICRAMET51080.2020.9298659(22-27)Online publication date: 18-Nov-2020
https://doi.org/10.1109/ICRAMET51080.2020.9298659
Woo APoulin P(2012)BibliographyShadow Algorithms Data Miner10.1201/b11901-9(197-238)Online publication date: 23-Aug-2012
https://doi.org/10.1201/b11901-9
Zhou YXu J(2012)Automatic non-interactive matting of dynamic targets in complex video using Graph CutsInternational Journal of Machine Learning and Cybernetics10.1007/s13042-012-0123-74:6(601-608)Online publication date: 24-Jul-2012
https://doi.org/10.1007/s13042-012-0123-7
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents