research-article

Single Image Shadow Detection via Complementary Mechanism

Authors:

Zheng-Jun ZhaAuthors Info & Claims

MM '22: Proceedings of the 30th ACM International Conference on Multimedia

Pages 6717 - 6726

https://doi.org/10.1145/3503161.3547904

Published: 10 October 2022 Publication History

Abstract

In this paper, we present a novel shadow detection framework by investigating the mutual complementary mechanisms contained in this specific task. Our method is based on a key observation: in a single shadow image, shadow regions and non-shadow counterparts are complementary to each other in nature, thus a better estimation on one side leads to an improved estimation on the other, and vice versa. Motivated by this observation, we first leverage two parallel interactive branches to jointly produce shadow and non-shadow masks. The interaction between two parallel branches is to retain the deactivated intermediate features of one branch by introducing the negative activation technique, which could serve as complementary features to the other branch. Besides, we also apply identity reconstruction loss as complementary training guidance at the image level. Finally, we design two discriminative losses to satisfy the complementary requirements of shadow detection, i.e., neither missing any shadow regions nor falsely detecting non-shadow regions. By fully exploring and exploiting the complementary mechanism of shadow detection, our method can confidently predict more accurate shadow detection results. Extensive experiments on the three widely-used benchmarks demonstrate our proposed method achieves superior shadow detection performance against state-of-the-art methods with a relatively low computational cost.

Supplementary Material

MP4 File (MM22-fp0626.mp4)

In this paper, we present a novel shadow detection framework by investigating the mutual complementary mechanisms contained in this specific task. Our method is based on a key observation: in a single shadow image, shadow regions and non-shadow counterparts are complementary to each other in nature, thus a better estimation on one side leads to an improved estimation on the other, and vice versa.

Download
8.63 MB

References

[1]

Zhihao Chen, Lei Zhu, Liang Wan, Song Wang, Wei Feng, and Pheng-Ann Heng. 2020. A multi-task mean teacher for semi-supervised shadow detection. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition. 5611--5620.

[2]

Xianyong Fang, Xiaohao He, Linbo Wang, and Jianbing Shen. 2021. Robust Shadow Detection by Exploring Effective Shadow Contexts. In Proceedings of the 29th ACM International Conference on Multimedia - ACM Multimedia 2021.

Digital Library

[3]

Zunlei Feng, Lechao Cheng, Xinchao Wang, Xiang Wang, Ya Jie Liu, Xiangtong Du, and Mingli Song. 2021. Visual Boundary Knowledge Translation for Foreground Segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 1334--1342.

[4]

Graham D Finlayson, Mark S Drew, and Cheng Lu. 2009. Entropy minimization for shadow removal. International Journal of Computer Vision 85, 1 (2009), 35--57.

Digital Library

[5]

Graham D Finlayson, Steven D Hordley, Cheng Lu, and Mark S Drew. 2005. On the removal of shadows from images. IEEE transactions on pattern analysis and machine intelligence 28, 1 (2005), 59--68.

[6]

Xavier Glorot, Antoine Bordes, and Yoshua Bengio. 2011. Deep sparse rectifier neural networks. In Proceedings of the fourteenth international conference on artificial intelligence and statistics. JMLR Workshop and Conference Proceedings, 315--323.

[7]

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. Advances in neural information processing systems (2014).

Digital Library

[8]

Ye-Peng Guan. 2008. Wavelet multi-scale transform based foreground segmentation and shadow elimination. The Open Signal Processing Journal 1, 1 (2008).

[9]

Ruiqi Guo, Qieyun Dai, and Derek Hoiem. 2011. Single-image shadow detection and removal using paired regions. In CVPR 2011. IEEE, 2033--2040.

Digital Library

[10]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition.

[11]

Sepideh Hosseinzadeh, Moein Shakeri, and Hong Zhang. 2018. Fast shadow detection from a single image using a patched convolutional neural network. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 3124--3129.

Digital Library

[12]

Qiming Hu and Xiaojie Guo. 2021. Trash or Treasure? An Interactive Dual-Stream Strategy for Single Image Reflection Separation. Advances in Neural Information Processing Systems 34 (2021).

[13]

Xiaowei Hu, Chi-Wing Fu, Lei Zhu, Jing Qin, and Pheng-Ann Heng. 2019. Direction-aware spatial context features for shadow detection and removal. IEEE transactions on pattern analysis and machine intelligence 42, 11 (2019), 2795--2808.

[14]

Xiaowei Hu, Tianyu Wang, Chi-Wing Fu, Yitong Jiang, Qiong Wang, and Pheng-Ann Heng. 2021. Revisiting shadow detection: A new benchmark dataset for complex world. IEEE Transactions on Image Processing 30 (2021), 1925--1934.

Digital Library

[15]

Xiaowei Hu, Lei Zhu, Chi-Wing Fu, Jing Qin, and Pheng-Ann Heng. 2018. Direction-aware spatial context features for shadow detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7454--7462.

[16]

Xiang Huang, Gang Hua, Jack Tumblin, and Lance Williams. 2011. What characterizes a shadow boundary under the sun and sky?. In 2011 international conference on computer vision. IEEE, 898--905.

Digital Library

[17]

Paul T. Jackway and Mohamed Deriche. 1996. Scale-space properties of the multi-scale morphological dilation-erosion. IEEE transactions on pattern analysis and machine intelligence 18, 1 (1996), 38--51.

[18]

Imran N Junejo and Hassan Foroosh. 2008. Estimating geo-temporal location of stationary cameras using shadow trajectories. In European conference on computer vision. Springer, 318--331.

Digital Library

[19]

Kevin Karsch, Varsha Hedau, David Forsyth, and Derek Hoiem. 2011. Rendering synthetic objects into legacy photographs. ACM Transactions on Graphics (TOG) 30, 6 (2011), 1--12.

Digital Library

[20]

Salman Hameed Khan, Mohammed Bennamoun, Ferdous Sohel, and Roberto Togneri. 2014. Automatic feature learning for robust shadow detection. In 2014 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 1939--1946.

Digital Library

[21]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

[22]

Philipp Krähenbühl and Vladlen Koltun. 2011. Efficient inference in fully connected crfs with gaussian edge potentials. Advances in neural information processing systems 24 (2011).

[23]

Jean-François Lalonde, AlexeiAEfros, and Srinivasa G Narasimhan. 2012. Estimating the natural illumination conditions from a single outdoor image. International Journal of Computer Vision 98, 2 (2012), 123--145.

Digital Library

[24]

Hieu Le and Dimitris Samaras. 2020. From Shadow Segmentation to Shadow Removal. In The IEEE European Conference on Computer Vision (ECCV).

Digital Library

[25]

Hieu Le and Dimitris Samaras. 2021. Physics-based shadow image decomposition for shadow removal. IEEE Transactions on Pattern Analysis & Machine Intelligence 01 (2021), 1--1.

[26]

Hieu Le, Tomas F Yago Vicente, Vu Nguyen, Minh Hoai, and Dimitris Samaras. 2018. A d net: Training a shadow detector with adversarial shadow attenuation. In Proceedings of the European Conference on Computer Vision (ECCV). 662--678.

[27]

Jingwei Liao, Yanli Liu, Guanyu Xing, Housheng Wei, Jueyu Chen, and Songhua Xu. 2021. Shadow Detection via Predicting the Confidence Maps of Shadow Detection Methods. In Proceedings of the 29th ACM International Conference on Multimedia. 704--712.

Digital Library

[28]

Vu Nguyen, Tomas F Yago Vicente, Maozheng Zhao, Minh Hoai, and Dimitris Samaras. 2017. Shadow detection with conditional generative adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision. 4510--4518.

[29]

Takahiro Okabe, Imari Sato, and Yoichi Sato. 2009. Attached shadow coding: Estimating surface normals from shadows under unknown reflectance and lighting conditions. In 2009 IEEE 12th International Conference on Computer Vision. IEEE, 1693--1700.

[30]

Alexandros Panagopoulos, Dimitris Samaras, and Nikos Paragios. 2009. Robust shadow and illumination estimation using a mixture model. In 2009 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 651--658.

[31]

Alexandros Panagopoulos, ChaohuiWang, Dimitris Samaras, and Nikos Paragios. 2012. Simultaneous cast shadows, illumination and geometry inference using hypergraphs. IEEE transactions on pattern analysis and machine intelligence 35, 2 (2012), 437--449.

[32]

Li Shen, Teck Wee Chua, and Karianto Leman. 2015. Shadow optimization from structured deep edge detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2067--2074.

[33]

Mingxing Tan and Quoc Le. 2019. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning. PMLR, 6105--6114.

[34]

Jiandong Tian, Xiaojun Qi, Liangqiong Qu, and Yandong Tang. 2016. New spectrum ratio properties and features for shadow detection. Pattern Recognition 51 (2016), 85--96.

Digital Library

[35]

Tomás F. Yago Vicente, Minh Hoai, and Dimitris Samaras. 2018. Leave-One-Out Kernel Optimization for Shadow Detection and Removal. IEEE Transactions on Pattern Analysis and Machine Intelligence 40, 3 (2018), 682--695. https://doi.org/10.1109/TPAMI.2017.2691703

[36]

Tomás F Yago Vicente, Le Hou, Chen-Ping Yu, Minh Hoai, and Dimitris Samaras. 2016. Large-scale training of shadow detectors with noisily-annotated shadow examples. In European Conference on Computer Vision. Springer, 816--832.

[37]

Tomás F Yago Vicente, Chen-Ping Yu, and Dimitris Samaras. 2013. Single Image Shadow Detection Using Multiple Cues in a Supermodular MRF. In BMVC.

[38]

Jifeng Wang, Xiang Li, and Jian Yang. 2018. Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1788--1797.

[39]

TiantianWang, Ali Borji, Lihe Zhang, Pingping Zhang, and Huchuan Lu. 2017. A stagewise refinement model for detecting salient objects in images. In Proceedings of the IEEE international conference on computer vision. 4019--4028.

[40]

Tianyu Wang, Xiaowei Hu, Chi-Wing Fu, and Pheng-Ann Heng. 2021. Single- Stage Instance Shadow Detection with Bidirectional Relation Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1--11.

[41]

Yupei Wang, Xin Zhao, Yin Li, Xuecai Hu, Kaiqi Huang, and NLPR CRIPAC. 2018. Densely Cascaded Shadow Detection Network via Deeply Supervised Parallel Fusion. In IJCAI, Vol. 5. 6.

[42]

Scott Wehrwein, Kavita Bala, and Noah Snavely. 2015. Shadow detection and sun direction in photo collections. In 2015 International Conference on 3D Vision. IEEE, 460--468.

Digital Library

[43]

Lin Wu, Xiaochun Cao, and Hassan Foroosh. 2010. Camera calibration and geolocation estimation from two shadow trajectories. Computer Vision and Image Understanding 114, 8 (2010), 915--927.

Digital Library

[44]

Pingping Zhang, Dong Wang, Huchuan Lu, Hongyu Wang, and Xiang Ruan. 2017. Amulet: Aggregating multi-level convolutional features for salient object detection. In Proceedings of the IEEE international conference on computer vision. 202--211.

[45]

Jia-Xing Zhao, Jiang-Jiang Liu, Deng-Ping Fan, Yang Cao, Jufeng Yang, and Ming-Ming Cheng. 2019. EGNet: Edge guidance network for salient object detection. In Proceedings of the IEEE/CVF international conference on computer vision. 8779--8788.

[46]

Wenda Zhao, Cai Shang, and Huchuan Lu. 2021. Self-generated Defocus Blur Detection via Dual Adversarial Discriminators. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6933--6942.

[47]

Quanlong Zheng, Xiaotian Qiao, Ying Cao, and Rynson WH Lau. 2019. Distraction-aware shadow detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5167--5176.

[48]

Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. 2016. Learning deep features for discriminative localization. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2921--2929.

[49]

Huajun Zhou, Xiaohua Xie, Jian-Huang Lai, Zixuan Chen, and Lingxiao Yang. 2020. Interactive two-stream decoder for accurate and fast saliency detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9141--9150.

[50]

Jiejie Zhu, Kegan GG Samuel, Syed Z Masood, and Marshall F Tappen. 2010. Learning to recognize shadows in monochromatic natural images. In 2010 IEEE Computer Society conference on computer vision and pattern recognition. IEEE, 223--230.

[51]

Lei Zhu, Zijun Deng, Xiaowei Hu, Chi-Wing Fu, Xuemiao Xu, Jing Qin, and Pheng-Ann Heng. 2018. Bidirectional feature pyramid network with recurrent attention residual modules for shadow detection. In Proceedings of the European Conference on Computer Vision (ECCV). 121--136.

Digital Library

[52]

Lei Zhu, Ke Xu, Zhanghan Ke, and Rynson WH Lau. 2021. Mitigating Intensity Bias in Shadow Detection via Feature Decomposition and Reweighting. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 4702--4711.

Cited By

Huang YLu XQuan YXu YJi H(2025)Image shadow removal via multi-scale deep Retinex decompositionPattern Recognition10.1016/j.patcog.2024.111126159(111126)Online publication date: Mar-2025
https://doi.org/10.1016/j.patcog.2024.111126
Zhou KFang JWei DWu WHu R(2025)Exploring better sparsely annotated shadow detectionNeural Networks10.1016/j.neunet.2024.106827181(106827)Online publication date: Jan-2025
https://doi.org/10.1016/j.neunet.2024.106827
Wang YLiu SLi LZhou WLi H(2024)SwinShadow: Shifted Window for Ambiguous Adjacent Shadow DetectionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/368880320:11(1-20)Online publication date: 27-Aug-2024
https://dl.acm.org/doi/10.1145/3688803
Show More Cited By

Index Terms

Single Image Shadow Detection via Complementary Mechanism
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
        Scene understanding

Recommendations

Shadow Detection via Predicting the Confidence Maps of Shadow Detection Methods
MM '21: Proceedings of the 29th ACM International Conference on Multimedia

Today's mainstream shadow detection methods are manually designed via a case-by-case approach. Accordingly, these methods may only be able to detect shadows for specific scenes. Given the complex and diverse shadow scenes in reality, none of the ...
Shadow Remover: Image Shadow Removal Based on Illumination Recovering Optimization
In this paper, we present a novel shadow removal system for single natural images as well as color aerial images using an illumination recovering optimization method. We first adaptively decompose the input image into overlapped patches according to the ...
New spectrum ratio properties and features for shadow detection

Successfully detecting shadows in still images is challenging yet has wide applications. Shadow properties and features are very important for shadow detection and processing. The aim of this work is to find some new physical properties of shadows and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '22: Proceedings of the 30th ACM International Conference on Multimedia

October 2022

7537 pages

ISBN:9781450392037

DOI:10.1145/3503161

General Chairs:
João Magalhães
NOVA University of Lisbon, Portugal
,
Alberto del Bimbo
University of Florence, Italy
,
Shin'ichi Satoh
National Institute of Informatics, Japan
,
Nicu Sebe
University of Trento, Italy
,
Program Chairs:
Xavier Alameda-Pineda
Inria, Grenoble, France
,
Qin Jin
Renmin University of China, China
,
Vincent Oria
New Jersey Institute of Technology, USA
,
Laura Toni
University College London, UK

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 October 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Key R&D Program of China
USTC Research Funds of the Double First-Class Initiative
Fundamental Research Funds for the Central Universities
National Natural Science Foundation of China (NSFC)
University Synergy Innovation Program of Anhui Province

Conference

MM '22

Sponsor:

SIGMM

MM '22: The 30th ACM International Conference on Multimedia

October 10 - 14, 2022

Lisboa, Portugal

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

23
Total Citations
View Citations
219
Total Downloads

Downloads (Last 12 months)70
Downloads (Last 6 weeks)3

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Huang YLu XQuan YXu YJi H(2025)Image shadow removal via multi-scale deep Retinex decompositionPattern Recognition10.1016/j.patcog.2024.111126159(111126)Online publication date: Mar-2025
https://doi.org/10.1016/j.patcog.2024.111126
Zhou KFang JWei DWu WHu R(2025)Exploring better sparsely annotated shadow detectionNeural Networks10.1016/j.neunet.2024.106827181(106827)Online publication date: Jan-2025
https://doi.org/10.1016/j.neunet.2024.106827
Wang YLiu SLi LZhou WLi H(2024)SwinShadow: Shifted Window for Ambiguous Adjacent Shadow DetectionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/368880320:11(1-20)Online publication date: 27-Aug-2024
https://dl.acm.org/doi/10.1145/3688803
Wang HWang WZhou HXu HWu SZhu LCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Language-Driven Interactive Shadow DetectionProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681192(5527-5536)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681192
Du BWang LXu S(2024)Distraction-Aware Edge Enhancement for Shadow Detection in Remote Sensing ImagesIEEE Geoscience and Remote Sensing Letters10.1109/LGRS.2024.341563721(1-5)Online publication date: 2024
https://doi.org/10.1109/LGRS.2024.3415637
Yu YXu CWang K(2024)TS-SAM: Fine-Tuning Segment-Anything Model for Downstream Tasks2024 IEEE International Conference on Multimedia and Expo (ICME)10.1109/ICME57554.2024.10688340(1-6)Online publication date: 15-Jul-2024
https://doi.org/10.1109/ICME57554.2024.10688340
Wang LHe JChen J(2024)Real-Time Object Detection in Overexposed Images Based on YOLOv52024 IEEE 4th International Conference on Electronic Technology, Communication and Information (ICETCI)10.1109/ICETCI61221.2024.10594528(324-327)Online publication date: 24-May-2024
https://doi.org/10.1109/ICETCI61221.2024.10594528
Lu XZhu YWang XLi DXiao JZhang YFu XZha Z(2024)HirFormer: Dynamic High Resolution Transformer for Large-Scale Image Shadow Removal2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW63382.2024.00651(6513-6523)Online publication date: 17-Jun-2024
https://doi.org/10.1109/CVPRW63382.2024.00651
Li DLu XZhu YWang XXiao JZhang YFu XZha Z(2024)Shadow Removal via Global Residual Free Unet and Shadow Generation2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW63382.2024.00634(6307-6316)Online publication date: 17-Jun-2024
https://doi.org/10.1109/CVPRW63382.2024.00634
Zhao LWang Q(2024)Shadow Detection and Removal Using a Hybrid Approach of CTU-Net and Subarea-Based Illumination CompensationIEEE Access10.1109/ACCESS.2024.341752312(124349-124364)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3417523
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten