Deep super-resolution based hashing for image retrieval
Article No.: 138, Pages 1 - 7
Abstract
Image retrieval based on deep convolutional neural networks (CNNs) has achieved promising performance in recent years. However, there are many low-resolution images in real-world retrieval tasks, and they would result in inaccurate hash representations for the CNN, which has only trained on high-resolution images. In this paper, we propose a novel framework, which is called deep superresolution based hashing (DSR-Hashing), to solve the problem. DSR-Hashing is constructed by two components: a super-resolution network and an encoding network. For low-resolution images, the super-resolution network can upscale them to their high-resolution versions, so as to provide more semantic information from pixel-wise reconstruction. Then, the up-scaled images are fed into the encoding network which consists many residual blocks. The encoding network is deep enough and adopts transfer learning strategy for better training. Extensive experiments conducted on two benchmark datasets demonstrate the state-of-the-art performance of DSR-Hashing on low-resolution image retrieval.
References
[1]
Yue Cao, Mingsheng Long, Jianmin Wang, Qiang Yang, and Philip S Yu. 2016. Deep visual-semantic hashing for cross-modal retrieval. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1445--1454.
[2]
Ken Chatfield, Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2014. Return of the devil in the details: Delving deep into convolutional nets. arXiv preprint arXiv:1405.3531 (2014).
[3]
Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yantao Zheng. 2009. NUS-WIDE: a real-world web image database from National University of Singapore. In Proceedings of the ACM international conference on image and video retrieval. ACM, 48.
[4]
Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. 2014. Learning a deep convolutional network for image super-resolution. In European conference on computer vision. Springer, 184--199.
[5]
Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. 2016. Image super-resolution using deep convolutional networks. IEEE transactions on pattern analysis and machine intelligence 38, 2 (2016), 295--307.
[6]
Weisheng Dong, Lei Zhang, Guangming Shi, and Xiaolin Wu. 2011. Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization. IEEE Transactions on Image Processing 20, 7 (2011), 1838--1857.
[7]
Yunchao Gong, Svetlana Lazebnik, Albert Gordo, and Florent Perronnin. 2013. Iterative quantization: A procrustean approach to learning binary codes for large-scale image retrieval. IEEE transactions on pattern analysis and machine intelligence 35, 12 (2013), 2916--2929.
[8]
Karol Gregor and Yann LeCun. 2010. Learning fast approximations of sparse coding. In Proceedings of the 27th International Conference on International Conference on Machine Learning. Omnipress, 399--406.
[9]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision. 1026--1034.
[10]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.
[11]
Daniel Glasner Shai Bagon Michal Irani. 2009. Super-resolution from a single image. In Proceedings of the IEEE International Conference on Computer Vision, Kyoto, Japan. 349--356.
[12]
Jiwon Kim, Jung Kwon Lee, and Kyoung Mu Lee. 2016. Deeply-recursive convolutional network for image super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1637--1645.
[13]
Alex Krizhevsky and Geoffrey Hinton. 2009. Learning multiple layers of features from tiny images. Technical Report. Citeseer.
[14]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems. 1097--1105.
[15]
Brian Kulis and Trevor Darrell. 2009. Learning to hash with binary reconstructive embeddings. In Advances in neural information processing systems. 1042--1050.
[16]
Hanjiang Lai, Yan Pan, Ye Liu, and Shuicheng Yan. 2015. Simultaneous feature learning and hash coding with deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3270--3278.
[17]
Christian Ledig, Lucas Theis, Ferenc Huszár, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, et al. 2017. Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4681--4690.
[18]
Michael S Lew, Nicu Sebe, Chabane Djeraba, and Ramesh Jain. 2006. Content-based multimedia information retrieval: State of the art and challenges. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 2, 1 (2006), 1--19.
[19]
Wu-Jun Li, Sheng Wang, and Wang-Cheng Kang. 2015. Feature learning based deep supervised hashing with pairwise labels. arXiv preprint arXiv:1511.03855 (2015).
[20]
Zhuangzi Li, Xiaobin Zhu, Lei Wang, and Peiyu Guo. 2018. Image Classification Using Convolutional Neural Networks and Kernel Extreme Learning Machines. In 2018 25th IEEE International Conference on Image Processing (ICIP). IEEE, 3009--3013.
[21]
Haomiao Liu, Ruiping Wang, Shiguang Shan, and Xilin Chen. 2016. Deep supervised hashing for fast image retrieval. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2064--2072.
[22]
Wei Liu, Jun Wang, Rongrong Ji, Yu-Gang Jiang, and Shih-Fu Chang. 2012. Supervised hashing with kernels. In 2012 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2074--2081.
[23]
Xianglong Liu, Junfeng He, Cheng Deng, and Bo Lang. 2014. Collaborative hashing. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2139--2146.
[24]
Mohammad Norouzi and David M Blei. 2011. Minimal loss hashing for compact binary codes. In Proceedings of the 28th international conference on machine learning (ICML-11). Citeseer, 353--360.
[25]
Ruslan Salakhutdinov and Geoff Hinton. 2007. Learning a nonlinear embedding by preserving class neighbourhood structure. In Artificial Intelligence and Statistics. 412--419.
[26]
Fumin Shen, Chunhua Shen, Wei Liu, and Heng Tao Shen. 2015. Supervised discrete hashing. In Proceedings of the IEEE conference on computer vision and pattern recognition. 37--45.
[27]
Wenzhe Shi, Jose Caballero, Ferenc Huszár, Johannes Totz, Andrew P Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang. 2016. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1874--1883.
[28]
Jun Wang, Sanjiv Kumar, and Shih-Fu Chang. 2012. Semi-supervised hashing for large-scale search. IEEE Transactions on Pattern Analysis and Machine Intelligence 34, 12 (2012), 2393--2406.
[29]
Jun Wang, Wei Liu, Sanjiv Kumar, and Shih-Fu Chang. 2016. Learning to hash for indexing big dataąłA survey. Proc. IEEE 104, 1 (2016), 34--57.
[30]
Jingdong Wang, Ting Zhang, Nicu Sebe, Heng Tao Shen, et al. 2018. A survey on learning to hash. IEEE transactions on pattern analysis and machine intelligence 40, 4 (2018), 769--790.
[31]
Rongkai Xia, Yan Pan, Hanjiang Lai, Cong Liu, and Shuicheng Yan. 2014. Supervised hashing for image retrieval via image representation learning. In Twenty-Eighth AAAI Conference on Artificial Intelligence.
[32]
Felix Yu, Sanjiv Kumar, Yunchao Gong, and Shih-Fu Chang. 2014. Circulant binary embedding. In International conference on machine learning. 946--954.
[33]
Sergey Zagoruyko and Nikos Komodakis. 2016. Wide residual networks. arXiv preprint arXiv:1605.07146 (2016).
[34]
Jian Zhang and Yuxin Peng. 2018. Query-adaptive image retrieval by deep-weighted hashing. IEEE Transactions on Multimedia 20, 9 (2018), 2400--2414.
[35]
Han Zhu, Mingsheng Long, Jianmin Wang, and Yue Cao. 2016. Deep hashing network for efficient similarity retrieval. In Thirtieth AAAI Conference on Artificial Intelligence.
[36]
Xiaobin Zhu, Zhuangzi Li, Xiaoyu Zhang, Haisheng Li, Ziyu Xue, and Lei Wang. 2018. Generative Adversarial Image Super-Resolution Through Deep Dense Skip Connections. In Computer Graphics Forum, Vol. 37. Wiley Online Library, 289--300.
[37]
Xiaobin Zhu, Zhuangzi Li, Xiao-Yu Zhang, Changsheng Li, Yaqi Liu, and Ziyu Xue. 2019. Residual Invertible Spatio-Temporal Network for Video Super-Resolution. In AAAI Conference on Artificial Intelligence.
[38]
Xiaobin Zhu, Jing Liu, Jinqiao Wang, Changsheng Li, and Hanqing Lu. 2014. Sparse representation for robust abnormality detection in crowded scenes. Pattern Recognition 47, 5 (2014), 1791--1799.
[39]
Xiaobin Zhu, Xinming Zhang, Xiao-Yu Zhang, Ziyu Xue, and Lei Wang. 2019. A novel framework for semantic segmentation with generative adversarial network. Journal of Visual Communication and Image Representation 58 (2019), 532--543.
Index Terms
- Deep super-resolution based hashing for image retrieval
Recommendations
Deep Super-Resolution Network for Single Image Super-Resolution with Realistic Degradations
ICDSC 2019: Proceedings of the 13th International Conference on Distributed Smart CamerasSingle Image Super-Resolution (SISR) aims to generate a high-resolution (HR) image of a given low-resolution (LR) image. The most of existing convolutional neural network (CNN) based SISR methods usually take an assumption that a LR image is only ...
Dilated projection correction network based on autoencoder for hyperspectral image super-resolution
AbstractThis paper focuses on improving the spatial resolution of the hyperspectral image (HSI) by taking the prior information into consideration. In recent years, single HSI super-resolution methods based on deep learning have achieved good ...
Highlights- We transform the single HSI SR problem into the abundance domain.
- We deploy a ...
Image Super-Resolution Using Deep Belief Networks
ICIMCS '14: Proceedings of International Conference on Internet Multimedia Computing and ServiceIn this paper, we aim at using Deep Belief Networks (DBNs) to solve the problem of image super-resolution (SR). We exploit the hierarchical structure of the DBNs to capture the non-linear mapping from low-resolution (LR) patches to their high-resolution ...
Comments
Information & Contributors
Information
Published In
May 2019
963 pages
ISBN:9781450371582
DOI:10.1145/3321408
Copyright © 2019 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 17 May 2019
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
ACM TURC 2019
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 92Total Downloads
- Downloads (Last 12 months)4
- Downloads (Last 6 weeks)0
Reflects downloads up to 12 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in