Cited By
View all- Zhang JChen WLin ZWei HZhao T(2024)Underwater image restoration based on progressive guidanceSignal Processing10.1016/j.sigpro.2024.109569223(109569)Online publication date: Oct-2024
HCSD-Net: Single Image Desnowing with Color Space Transformation
Authors: 
Ting Zhang, Nanfeng Jiang, Hongxin Wu, Keke Zhang, Yuzhen Niu, Tiesong ZhaoAuthors Info & Claims
Ting Zhang, Nanfeng Jiang, Hongxin Wu, Keke Zhang, Yuzhen Niu, Tiesong ZhaoAuthors Info & ClaimsPages 8125 - 8133
Abstract
Single-image desnowing aims at depressing snowflake noises while preserving a clean background. Existing methods usually mask the locations of noises and remove them in RGB color space. In this paper, we rethink this problem by investigating the impacts of color space selection. Theoretical analysis and experiments reveal that the feature of snowflake noises exhibit different distributions in different color spaces. In particular, these noises are barely seen in Hue channel, which inspires us to recover global structure and texture information of the clean background from Hue channel. More low-frequency information is also found in the Hue channel. With these observations, we propose a novel Hybrid-Color-Space-based Desnowing Network (HCSD-Net). The proposed HCSD-Net extracts low-frequency and high-frequency features in Hue channel and RGB color space, respectively. After that, it utilizes a multi-scale fusion module to enhance high-frequency details at a small feature resolution. These details are further used to supervise and supplement the background information. Extensive experiments demonstrate that our proposed HCSD-Net outperforms state-of-the-art methods on various synthetic and real-world desnowing datasets. Codes are available at https://github.com/ttz-rainbow/HCSD-Net.
Supplementary Material
Presentation video of "HCSD-Net: Single Image Desnowing with Hybrid Color Space Transformation".
- Download
- 194.99 MB
References
[1]
Mohiuddin Ahmed, Raihan Seraj, and Syed Mohammed Shamsul Islam. 2020. The k-means algorithm: A comprehensive survey and performance evaluation. Electronics, Vol. 9, 8 (2020), 1295.
[2]
Peter C Barnum, Srinivasa Narasimhan, and Takeo Kanade. 2010. Analysis of rain and snow in frequency space. International Journal of Computer Vision, Vol. 86 (2010), 256--274.
[3]
Douglas Chai and Abdesselam Bouzerdoum. 2000. A Bayesian approach to skin color classification in YCbCr color space. In 2000 TENCON Proceedings. Intelligent Systems and Technologies for the New Millennium, Vol. 2. IEEE, 421--424.
[4]
Sixiang Chen, Tian Ye, Yun Liu, Erkang Chen, Jun Shi, and Jingchun Zhou. 2022b. SnowFormer: Scale-aware Transformer via Context Interaction for Single Image Desnowing. arXiv preprint arXiv:2208.09703 (2022).
[5]
Sixiang Chen, Tian Ye, Yun Liu, Taodong Liao, Jingxia Jiang, Erkang Chen, and Peng Chen. 2023. MSP-former: Multi-scale projection transformer for single image desnowing. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 1--5.
[6]
Weiting Chen, Haoyu Fang, Jianjiun Ding, Chengche Tsai, and Syyen Kuo. 2020. JSTASR: Joint size and transparency-aware snow removal algorithm based on modified partial convolution and veiling effect removal. In In Proceedings of the European Conference on Computer Vision. 754--770.
[7]
Weiting Chen, Haoyu Fang, Chenglin Hsieh, Chengche Tsai, I Chen, Jianjiun Ding, Syyen Kuo, et al. 2021. All snow removed: Single image desnowing algorithm using hierarchical dual-tree complex wavelet representation and contradict channel loss. In IEEE/CVF International Conference on Computer Vision. 4196--4205.
[8]
Weiting Chen, Zhikai Huang, Chengche Tsai, Haohsiang Yang, Jianjiun Ding, and Syyen Kuo. 2022a. Learning Multiple Adverse Weather Removal via Two-Stage Knowledge Learning and Multi-Contrastive Regularization: Toward a Unified Model. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 17653--17662.
[9]
Bodong Cheng, Juncheng Li, Ying Chen, Shuyi Zhang, and Tieyong Zeng. 2022. Snow mask guided adaptive residual network for image snow removal. arXiv preprint arXiv:2207.04754 (2022).
[10]
Xueyang Fu, Jiabin Huang, Delu Zeng, Yue Huang, Xinghao Ding, and John Paisley. 2017. Removing rain from single images via a deep detail network. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3855--3863.
[11]
Wenshuo Gao, Xiaoguang Zhang, Lei Yang, and Huizhong Liu. 2010. An improved Sobel edge detection. In 2010 3rd International conference on computer science and information technology, Vol. 5. IEEE, 67--71.
[12]
Shreyank N Gowda and Chun Yuan. 2019. ColorNet: Investigating the importance of color spaces for image classification. In Proceedings of the Asian Conference on Computer Vision. Springer, 581--596.
[13]
Yang Guan, Yangang Ren, Qi Sun, Shengbo Eben Li, Haitong Ma, Jingliang Duan, Yifan Dai, and Bo Cheng. 2022. Integrated decision and control: toward interpretable and computationally efficient driving intelligence. IEEE Transactions on Cybernetics, Vol. 53, 2 (2022), 859--873.
[14]
Kaiming He, Jian Sun, and Xiaoou Tang. 2010. Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 33, 12 (2010), 2341--2353.
[15]
Xiaowei Hu, Chi-Wing Fu, Lei Zhu, Jing Qin, and Pheng-Ann Heng. 2019. Direction-aware spatial context features for shadow detection and removal. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 42, 11 (2019), 2795--2808.
[16]
Nanfeng Jiang, Weiling Chen, Yuting Lin, Tiesong Zhao, and Chia-Wen Lin. 2021. Underwater image enhancement with lightweight cascaded network. IEEE Transactions on Multimedia, Vol. 24 (2021), 4301--4313.
[17]
Weisheng Lai, Jiabin Huang, Narendra Ahuja, and Minghsuan Yang. 2018. Fast and accurate image super-resolution with deep laplacian pyramid networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 41, 11 (2018), 2599--2613.
[18]
Lerenhan Li, Yunlong Dong, Wenqi Ren, Jinshan Pan, Changxin Gao, Nong Sang, and Ming-Hsuan Yang. 2019a. Semi-supervised image dehazing. IEEE Transactions on Image Processing, Vol. 29 (2019), 2766--2779.
[19]
Ruoteng Li, Robby T Tan, and Loong-Fah Cheong. 2020. All in one bad weather removal using architectural search. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3175--3185.
[20]
Zhi Li, Juan Zhang, Zhijun Fang, Bo Huang, Xiaoyan Jiang, Yongbin Gao, and Jenq-Neng Hwang. 2019b. Single image snow removal via composition generative adversarial networks. IEEE Access, Vol. 7 (2019), 25016--25025.
[21]
Liang Liao, Wenyi Chen, Jing Xiao, Zheng Wang, Chia-Wen Lin, and Shin'ichi Satoh. 2022. Unsupervised Foggy Scene Understanding via Self Spatial-Temporal Label Diffusion. IEEE Transactions on Image Processing, Vol. 31 (May 2022), 3525--3540.
[22]
Junhong Lin, Nanfeng Jiang, Zhentao Zhang, Weiling Chen, and Tiesong Zhao. 2023. LMQFormer: A Laplace-Prior-Guided Mask Query Transformer for Lightweight Snow Removal. IEEE Transactions on Circuits and Systems for Video Technology (2023), 1--1.
[23]
Yunfu Liu, Dawei Jaw, Shihchia Huang, and Jenqneng Hwang. 2018. DesnowNet: Context-aware deep network for snow removal. IEEE Transactions on Image Processing, Vol. 27, 6 (2018), 3064--3073.
[24]
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021a. Swin transformer: Hierarchical vision transformer using shifted windows. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10012--10022.
[25]
Zhihao Liu, Hui Yin, Yang Mi, Mengyang Pu, and Song Wang. 2021b. Shadow removal by a lightness-guided network with training on unpaired data. IEEE Transactions on Image Processing, Vol. 30 (2021), 1853--1865.
[26]
Timo Ojala, Matti Pietikainen, and Topi Maenpaa. 2002. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 24, 7 (2002), 971--987.
[27]
Soochang Pei, Yutai Tsai, and Chenyu Lee. 2014. Removing rain and snow in a single image using saturation and visibility features. In IEEE International Conference on Multimedia and Expo Workshops. 1--6.
[28]
Yanyun Qu, Yizi Chen, Jingying Huang, and Yuan Xie. 2019. Enhanced pix2pix dehazing network. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8160--8168.
[29]
Dhanashree Rajderkar and PS Mohod. 2013. Removing snow from an image via image decomposition. In IEEE International Conference on Emerging Trends in Computing, Communication and Nanotechnology. 576--579.
[30]
Min C Shin, Kyong I Chang, and Leonid V Tsap. 2002. Does colorspace transformation make any difference on skin detection?. In Sixth IEEE Workshop on Applications of Computer Vision, 2002. Proceedings. IEEE, 275--279.
[31]
Jeya Maria Jose Valanarasu, Rajeev Yasarla, and Vishal M Patel. 2022. Transweather: Transformer-based restoration of images degraded by adverse weather conditions. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2353--2363.
[32]
Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research, Vol. 9, 11 (2008).
[33]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, Vol. 30 (2017).
[34]
Yinglong Wang, Shuaicheng Liu, Chen Chen, and Bing Zeng. 2017. A Hierarchical Approach for Rain or Snow Removing in A Single Color Image. IEEE Transactions on Image Processing, Vol. 26, 8 (2017), 3936--3950.
[35]
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, Vol. 13, 4 (2004), 600--612.
[36]
Zhendong Wang, Xiaodong Cun, Jianmin Bao, Wengang Zhou, Jianzhuang Liu, and Houqiang Li. 2022. Uformer: A general u-shaped transformer for image restoration. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 17683--17693.
[37]
Jie Xiao, Xueyang Fu, Aiping Liu, Feng Wu, and Zheng-Jun Zha. 2022. Image de-raining transformer. IEEE Transactions on Pattern Analysis and Machine Intelligence (2022).
[38]
Jing Xu, Wei Zhao, Peng Liu, and Xianglong Tang. 2012. An Improved Guidance Image Based Method to Remove Rain and Snow in a Single Image. Journal of Computing and Information Science in Engineering, Vol. 5 (2012), 49--55.
[39]
Hong Yang, Gongrui Nan, Mingbao Lin, Fei Chao, Yunhang Shen, Ke Li, and Rongrong Ji. 2022. LAB-Net: LAB Color-Space Oriented Lightweight Network for Shadow Removal. arXiv preprint arXiv:2208.13039 (2022).
[40]
Tian Ye, Sixiang Chen, Yun Liu, Yi Ye, Jinbin Bai, and Erkang Chen. 2022. Towards real-time high-definition image snow removal: Efficient pyramid network with asymmetrical encoder-decoder architecture. In Proceedings of the Asian Conference on Computer Vision. 366--381.
[41]
Han Zhang, Ian Goodfellow, Dimitris Metaxas, and Augustus Odena. 2019. Self-attention generative adversarial networks. In International Conference on Machine Learning. PMLR, 7354--7363.
[42]
Kaihao Zhang, Rongqing Li, Yanjiang Yu, Wenhan Luo, and Changsheng Li. 2021. Deep dense multi-scale network for snow removal using semantic and depth priors. IEEE Transactions on Image Processing, Vol. 30 (2021), 7419--7431.
[43]
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. 2018. The unreasonable effectiveness of deep features as a perceptual metric. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 586--595.
[44]
Xianhui Zheng, Yinghao Liao, Wei Guo, Xueyang Fu, and Xinghao Ding. 2013. Single-image-based rain and snow removal using multi-guided filter. In Neural Information Processing. 258--265.
[45]
Xian Zhong, Shilei Zhao, Xiao Wang, Kui Jiang, Wenxuan Liu, Wenxin Huang, and Zheng Wang. 2021. Unsupervised vehicle search in the wild: A new benchmark. In Proceedings of the 29th ACM International Conference on Multimedia. 5316--5325
Index Terms
- HCSD-Net: Single Image Desnowing with Color Space Transformation
Recommendations
Color medical image enhancement based on adaptive equalization of intensity numbers matrix histogram
The enhancement technique for color medical images is conductive to improve the resolution and accuracy of the original image. A new enhancement method combining the Young-Helmholtz (Y-H) transformation with the adaptive equalization of intensity ...
A uniform illumination image enhancement via linear transformation in CIELAB color space
AbstractDigital devices such as digital cameras and smartphones are important tools in capturing color images. However, images captured under a low-light condition have low contrast, low brightness, and they lose their color information. In this paper, we ...
Histogram-based fuzzy colour filter for image restoration
A new impulse noise reduction method for colour images, called histogram-based fuzzy colour filter (HFC), is presented in this paper. The HFC filter is particularly effective for reducing high-impulse noise in digital images while preserving edge ...
Comments
Information & Contributors
Information
Published In
October 2023
9913 pages
ISBN:9798400701085
DOI:10.1145/3581783
- General Chairs:
- Abdulmotaleb El Saddik,
- Tao Mei,
- Rita Cucchiara,
- Program Chairs:
- Marco Bertini,
- Diana Patricia Tobon Vallejo,
- Pradeep K. Atrey,
- M. Shamim Hossain
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 27 October 2023
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
MM '23
Sponsor:
MM '23: The 31st ACM International Conference on Multimedia
October 29 - November 3, 2023
Ottawa ON, Canada
Acceptance Rates
Overall Acceptance Rate 995 of 4,171 submissions, 24%
Upcoming Conference
MM '24
- Sponsor:
- sigmm
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 187Total Downloads
- Downloads (Last 12 months)187
- Downloads (Last 6 weeks)12
Reflects downloads up to 01 Sep 2024
Other Metrics
Citations
Cited By
View all- Zhang JChen WLin ZWei HZhao T(2024)Underwater image restoration based on progressive guidanceSignal Processing10.1016/j.sigpro.2024.109569223(109569)Online publication date: Oct-2024
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in