Spectrum-inspired Low-light Image Translation for Saliency Detection
Article No.: 33, Pages 1 - 9
Abstract
Saliency detection methods are central to several real-world applications such as robot navigation and satellite imagery. However, the performance of existing methods deteriorate under low-light conditions because training datasets mostly comprise of well-lit images. One possible solution is to collect a new dataset for low-light conditions. This involves pixel-level annotations, which is not only tedious and time-consuming but also infeasible if a huge training corpus is required. We propose a technique that performs classical band-pass filtering in the Fourier space to transform well-lit images to low-light images and use them as a proxy for real low-light images. Unlike popular deep learning approaches which require learning thousands of parameters and enormous amounts of training data, the proposed transformation is fast and simple and easy to extend to other tasks such as low-light depth estimation. Our experiments show that the state-of-the-art saliency detection and depth estimation networks trained on our proxy low-light images perform significantly better on real low-light images than networks trained using existing strategies.
References
[1]
Radhakrishna Achanta, Sheila Hemami, Francisco Estrada, and Sabine Susstrunk. 2009. Frequency-tuned salient region detection. In 2009 IEEE conference on computer vision and pattern recognition. IEEE, 1597–1604.
[2]
Ivan Anokhin, Pavel Solovev, Denis Korzhenkov, Alexey Kharlamov, Taras Khakhulin, Aleksei Silvestrov, Sergey Nikolenko, Victor Lempitsky, and Gleb Sterkin. 2020. High-resolution daytime translation without domain labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7488–7497.
[3]
Shariq Farooq Bhat, Ibraheem Alhashim, and Peter Wonka. 2021. AdaBins: Depth estimation using adaptive bins. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4009–4018.
[4]
Jianrui Cai, Shuhang Gu, and Lei Zhang. 2018. Learning a deep single image contrast enhancer from multi-exposure images. IEEE Transactions on Image Processing 27, 4 (2018), 2049–2062.
[5]
Xiaochun Cao, Changqing Zhang, Huazhu Fu, Xiaojie Guo, and Qi Tian. 2015. Saliency-aware nonparametric foreground annotation based on weakly labeled data. IEEE transactions on neural networks and learning systems 27, 6(2015), 1253–1265.
[6]
Chen Chen, Qifeng Chen, Jia Xu, and Vladlen Koltun. 2018. Learning to see in the dark. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3291–3300.
[7]
Bo Dong, Yan Zhou, Chuanfei Hu, Keren Fu, and Geng Chen. 2021. BCNet: Bidirectional collaboration network for edge-guided salient object detection. Neurocomputing 437(2021), 58–71.
[8]
Deng-Ping Fan, Ming-Ming Cheng, Yun Liu, Tao Li, and Ali Borji. 2017. Structure-measure: A new way to evaluate foreground maps. In Proceedings of the IEEE international conference on computer vision. 4548–4557.
[9]
Deng-Ping Fan, Cheng Gong, Yang Cao, Bo Ren, Ming-Ming Cheng, and Ali Borji. 2018. Enhanced-alignment measure for binary foreground map evaluation. arXiv preprint arXiv:1805.10421(2018).
[10]
Shang-Hua Gao, Yong-Qiang Tan, Ming-Ming Cheng, Chengze Lu, Yunpeng Chen, and Shuicheng Yan. 2020. Highly efficient salient object detection with 100k parameters. In European Conference on Computer Vision. Springer, 702–721.
[11]
Stas Goferman, Lihi Zelnik-Manor, and Ayellet Tal. 2011. Context-aware saliency detection. IEEE transactions on pattern analysis and machine intelligence 34, 10(2011), 1915–1926.
[12]
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep learning. MIT press.
[13]
Chunle Guo, Chongyi Li, Jichang Guo, Chen Change Loy, Junhui Hou, Sam Kwong, and Runmin Cong. 2020. Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1780–1789.
[14]
Xiaojie Guo, Yu Li, and Haibin Ling. 2016. LIME: Low-light image enhancement via illumination map estimation. IEEE Transactions on image processing 26, 2 (2016), 982–993.
[15]
Jianming Hu, Xiyang Zhi, Wei Zhang, Longfei Ren, and Lorenzo Bruzzone. 2020. Salient Ship Detection via Background Prior and Foreground Constraint in Remote Sensing Images. Remote Sensing 12, 20 (2020), 3370.
[16]
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1125–1134.
[17]
Laurent Itti, Christof Koch, and Ernst Niebur. 1998. A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on pattern analysis and machine intelligence 20, 11(1998), 1254–1259.
[18]
Hugo Jacob, Flávio LC Pádua, Anisio Lacerda, and Adriano Pereira. 2017. A video summarization approach based on the emulation of bottom-up mechanisms of visual attention. Journal of Intelligent Information Systems 49, 2 (2017), 193–211.
[19]
Yifan Jiang, Xinyu Gong, Ding Liu, Yu Cheng, Chen Fang, Xiaohui Shen, Jianchao Yang, Pan Zhou, and Zhangyang Wang. 2021. Enlightengan: Deep light enhancement without paired supervision. IEEE Transactions on Image Processing 30 (2021), 2340–2349.
[20]
Jiwhan Kim, Dongyoon Han, Yu-Wing Tai, and Junmo Kim. 2014. Salient region detection via high-dimensional color transform. In Proceedings of the IEEE conference on computer vision and pattern recognition. 883–890.
[21]
Gábor Kovács, Yasuharu Kunii, Takao Maeda, and Hideki Hashimoto. 2019. Saliency and spatial information-based landmark selection for mobile robot navigation in natural environments. Advanced Robotics 33, 10 (2019), 520–535.
[22]
Chen-Yu Lee, Tanmay Batra, Mohammad Haris Baig, and Daniel Ulbricht. 2019. Sliced wasserstein discrepancy for unsupervised domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10285–10295.
[23]
Chuan Li and Michael Wand. 2016. Precomputed real-time texture synthesis with markovian generative adversarial networks. In European conference on computer vision. Springer, 702–716.
[24]
Nian Liu, Junwei Han, and Ming-Hsuan Yang. 2018. Picanet: Learning pixel-wise contextual attention for saliency detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3089–3098.
[25]
Yuen Peng Loh and Chee Seng Chan. 2019. Getting to know low-light images with the exclusively dark dataset. Computer Vision and Image Understanding 178 (2019), 30–42.
[26]
Jonathan Long, Evan Shelhamer, and Trevor Darrell. 2015. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3431–3440.
[27]
Nan Mu, Xin Xu, and Xiaolong Zhang. 2019. Salient object detection in low contrast images via global convolution and boundary refinement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 0–0.
[28]
Nan Mu, Xin Xu, Xiaolong Zhang, and Hong Zhang. 2018. Salient object detection using a covariance-based CNN model in low-contrast images. Neural Computing and Applications 29, 8 (2018), 181–192.
[29]
Pushmeet Kohli Nathan Silberman, Derek Hoiem and Rob Fergus. 2012. Indoor Segmentation and Support Inference from RGBD Images. In ECCV.
[30]
Mark Nixon and Alberto Aguado. 2019. Feature extraction and image processing for computer vision. Academic press.
[31]
Alan V Oppenheim, John R Buck, and Ronald W Schafer. 2001. Discrete-time signal processing. Vol. 2. Upper Saddle River, NJ: Prentice Hall.
[32]
Alan V Oppenheim and Jae S Lim. 1981. The importance of phase in signals. Proc. IEEE 69, 5 (1981), 529–541.
[33]
Taesung Park, Alexei A Efros, Richard Zhang, and Jun-Yan Zhu. 2020. Contrastive learning for unpaired image-to-image translation. In European Conference on Computer Vision. Springer, 319–345.
[34]
Houwen Peng, Bing Li, Weihua Xiong, Weiming Hu, and Rongrong Ji. 2014. Rgbd salient object detection: a benchmark and algorithms. In European conference on computer vision. Springer, 92–109.
[35]
Fabio Pizzati, Pietro Cerri, and Raoul de Charette. 2021. CoMoGAN: continuous model-guided image-to-image translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 14288–14298.
[36]
Xuebin Qin, Zichen Zhang, Chenyang Huang, Chao Gao, Masood Dehghan, and Martin Jagersand. 2019. Basnet: Boundary-aware salient object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7479–7489.
[37]
Han Wang, Chen Wang, and Lihua Xie. 2020. Online visual place recognition via saliency re-identification. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 5030–5036.
[38]
Ruixing Wang, Qing Zhang, Chi-Wing Fu, Xiaoyong Shen, Wei-Shi Zheng, and Jiaya Jia. 2019. Underexposed photo enhancement using deep illumination estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6849–6857.
[39]
Xuanhan Wang, Lianli Gao, Jingkuan Song, and Hengtao Shen. 2016. Beyond frame-level CNN: saliency-aware 3-D CNN with LSTM for video action recognition. IEEE Signal Processing Letters 24, 4 (2016), 510–514.
[40]
Xiaofang Wang and Chun Qi. 2020. Detecting action-relevant regions for action recognition using a three-stage saliency detection technique. Multimedia Tools and Applications 79, 11 (2020), 7413–7433.
[41]
Chen Wei, Wenjing Wang, Wenhan Yang, and Jiaying Liu. 2018. Deep retinex decomposition for low-light enhancement. arXiv preprint arXiv:1808.04560(2018).
[42]
Chen Wei, Wenjing Wang, Wenhan Yang, and Jiaying Liu. 2018. Deep retinex decomposition for low-light enhancement. arXiv preprint arXiv:1808.04560(2018).
[43]
Xin Xu, Nan Mu, Hong Zhang, and Xiaowei Fu. 2015. Salient object detection from distinctive features in low contrast images. In 2015 IEEE international conference on image processing (ICIP). IEEE, 3126–3130.
[44]
Xin Xu and Jie Wang. 2018. Extended non-local feature for visual saliency detection in low contrast images. In Proceedings of the European Conference on Computer Vision (ECCV). 0–0.
[45]
Xin Xu, Shiqin Wang, Zheng Wang, Xiaolong Zhang, and Ruimin Hu. 2020. Exploring Image Enhancement for Salient Object Detection in Low Light Images. arXiv preprint arXiv:2007.16124(2020).
[46]
Yanchao Yang and Stefano Soatto. 2020. Fda: Fourier domain adaptation for semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4085–4095.
[47]
Libao Zhang and Qiaoyue Sun. 2018. Saliency detection and region of interest extraction based on multi-image common saliency analysis in satellite images. Neurocomputing 283(2018), 150–165.
Index Terms
- Spectrum-inspired Low-light Image Translation for Saliency Detection
Recommendations
A Low-light Image Enhancement Method Balancing Brightness Boosting and Noise Smoothing
CSAI '22: Proceedings of the 2022 6th International Conference on Computer Science and Artificial IntelligenceImages captured in low-light or uneven lighting environments need to use low-light image enhancement techniques to boost brightness and contrast for high-quality improved results. Some existing low-light image enhancement methods over-enhance the image ...
Low-light image enhancement based on sharpening-smoothing image filter
AbstractLow-light images suffer from poor visibility, severe noise, low contrast, and low brightness. To overcome these issues, many image enhancement methods have been proposed. Few techniques solve these problems simultaneously. This paper presents a ...
Perceptual Quality Assessment of Low-light Image Enhancement
Low-light image enhancement algorithms (LIEA) can light up images captured in dark or back-lighting conditions. However, LIEA may introduce various distortions such as structure damage, color shift, and noise into the enhanced images. Despite various ...
Comments
Information & Contributors
Information
Published In
December 2022
506 pages
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 12 May 2023
Check for updates
Author Tag
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICVGIP'22
ICVGIP'22: Thirteenth Indian Conference on Computer Vision, Graphics and Image Processing
December 8 - 10, 2022
Gandhinagar, India
Acceptance Rates
Overall Acceptance Rate 95 of 286 submissions, 33%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 25Total Downloads
- Downloads (Last 12 months)12
- Downloads (Last 6 weeks)0
Reflects downloads up to 10 Nov 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format