research-article

Saliency detection using Multi-layer graph ranking and combined neural networks

Authors:

Ye ZhangAuthors Info & Claims

Volume 65, Issue C

https://doi.org/10.1016/j.jvcir.2019.102673

Published: 01 December 2019 Publication History

Highlights

•

A combined net is proposed to improve salient object detection.

•

An function is constructed to diffuse saliency from borders to salient objects.

•

Region Net is adopted to generate salient map with sharp object boundary.

•

Local-Global Net is proposed to provide reliable multi-scale contextual information.

Abstract

In this paper, a new algorithm based on a combined neural network is proposed to improve salient object detection in the complex images. It consists of two main steps. The first step, an objective function which is optimized on a multi-layer graph structure is constructed to diffuse saliency from borders to salient objects, aiming to roughly estimate the location and extent salient objects of an image, meanwhile, color attribute is adopted to rapidly find a set of object-related regions in the image. The second step, establish a combined neural network with Region Net and Local-Global Net. Region Net is adopted to efficiently generate the salient map with the sharp object boundary. Then Local-Global Net based on multi-scale spatial context is proposed to provide strongly reliable multi-scale contextual information, and thus achieves an optimized performance. Experimental results and comparison analysis demonstrate that the proposed algorithm is more effective and superior than most low-level oriented prior methods in terms of precision recall curves, F-measure and mean absolute errors.

References

[1]

P.L. Rosin, A simple method for detecting salient regions, Pattern Recogn. (2009) 2363–2371.

[2]

R. Achanta, S. Hemami, F. Estrada, S. Susstrunk, Frequency-tuned salient region detection, in: IEEE Proc. CVPR, 2009, pp. 1597–1604.

[3]

G. Wang, Y.d. Zhang, J.T. Li, High-level background prior based salient object detection, in: J. Vis. Commun. Image R. 2017, pp.432–441.

[4]

F. Huang, Q. Jinqing, H. Lu, et al., Salient object detection via multiple instance learning, IEEE Trans. Image Process. (2017) 1911–1922.

[5]

J. Long, E. Shelhamer, T. Darrell, Fully convolutional networks for semantic segmentation, in: IEEE Proc. CVPR, 2015, pp. 3431–3440.

[6]

S. Xie, Z. Tu, Holistically-nested edge detection, in: IEEE Proc. ICCV, 2015, pp. 1395–1403.

[7]

C.Y. Shao, Q.H. Ding, Image segmentation fusion using weakly supervised trace-norm multi-task learning method, in: IET Image Process, 2018, pp.1079–1085.

[8]

G. Li, Y. Yu, Visual saliency based on multiscale deep features, in: IEEE Proc. CVPR, 2015, pp. 5455–5463.

[9]

H.S. Wu, Y.L. Wu, S.L. Zhang, Cartoon image segmentation based on improved SLIC superpixels and adaptive region propagation merging, in: IEEE International Conference on Signal and Image Processing, 2016, pp. 277–281.

[10]

R. Girshick, Fast R-CNN, in: IEEE Proc. ICCV, 2015, pp. 1440–1448.

[11]

H. Jiang, J. Wang, Z. Yuan, Y. Wu, N. Zheng, S. Li, Salient object detection: a discriminative regional feature integration approach, in: IEEE Proc. CVPR, 2013, pp.2083–2090.

[12]

Y.F. Ma and H.J. Zhang, Contrast-based image attention analysis by using fuzzy growing, in: ACM Multimedia, 2003, pp.374–381.

[13]

X. Li, H. Lu, L. Zhang, X. Ruan, M.-H. Yang, Saliency detection via dense and sparse reconstruction, in: IEEE Proc. CVPR, 2013, pp. 2976–2983.

[14]

Z. Li, C. Lang, S. Feng, T. Wang, Saliency ranker: a new salient object detection method, J. Vis. Commun. Image Represent. (2018) 16–26.

[15]

F. Perazzi, P. Krahenbuhl, Y. Pritch, A. Hornung, Saliency filters: contrast based filtering for salient region detection, Proc. IEEE Conf. Comput. Vis. Pattern Recog. (2012) 733–740.

[16]

H. Peng, B. Li, W. Xiong, W. Hu, R. Ji, RGBD salient object detection: a benchmark and algorithms, in: Proc. Eur. Conf. Comput. Vis, 2014, pp. 1–18.

[17]

X. Shen, Y. Wu, A unified approach to salient object detection via low rank matrix recovery, in: IEEE Proc. CVPR, 2012, 853–860.

[18]

G. Li, Y. Yu, Deep contrast learning for salient object detection, in: IEEE Proc. CVPR, 2016, pp. 543–552.

[19]

Y. Liu, M.S. Lew, Learning relaxed deep supervision for better edge detection, in: IEEE Proc. CVPR, 2016, pp. 611–622.

[20]

S. Xie, Z. Tu, Holistically-nested edge detection, in: IJCV. Springer, 2017, pp. 113–119.

[21]

X. Li, L.M. Zhao, L.N. Wei, et al., Deep saliency: multi-task deep neural network model for salient object detection, IEEE Trans. Image Process. (2016) 3919–3930.

[22]

Y. Sun, X. Wang, X. Tang, Deep convolutional network cascade for facial point detection, in: IEEE Proc. CVPR, 2013, pp. 312–324.

[23]

L.C. Kong, H. Zhang, Y.H. Zheng, Image segmentation using a hierarchical student's-t mixture model, IET Image Process. (2017) 1094–1102.

[24]

M.S. Allili, D. Ziou, Active contours for video object tracking using region, boundary and shape information, Signal Image Video Process. (2007) 101–117.

[25]

M.S. Allili, Wavelet modeling using finite mixtures of generalized Gaussian distributions: application to texture discrimination and retrieval, IEEE Trans. Image Process. (2012) 1452–1464.

[26]

X.P. Liang, D.S. Huang, H.B. Luo, Image segmentation fusion using weakly supervised trace-norm multi-task learning method, in: IET Image Process, 2018, pp. 801–809.

[27]

S. Goferman, L. Zelnik Manor, A. Tal, Context-aware saliency detection, IEEE Trans. Pattern Anal. Mach. (2012) 1915–1926.

[28]

Y. Li, J. Yosinski, J. Clune, H. Lipson, J. Hopcroft, Convergent learning: do different neural networks learn the same representations, in: Proc. ICLR, 2016, pp. 196–212.

[29]

X. Wang, H. Ma, X. Chen, Salient object detection via fast R-CNN and low-level cues, in: Proc. ICIP, 2016, pp. 1042–1046.

[30]

Maryam Imani, Hassan Ghassemian, Edge patch image-based morphological profiles for classification of multispectral and hyperspectral data, in: IET Image Process, 2016, pp. 164–172.

[31]

T. Akilan, Q.M. Wu, H. Zhang, Effect of fusing features from multiple DCNN architectures in image classification, in: IET Image Process, 2018, pp. 1102–1110.

[32]

Y. Jia Caffe, et al., Convolutional architecture for fast feature embedding, in: Proc. ACM MM, 2014, pp. 675–678.

[33]

O. Russakovsky, et al., ImageNet large scale visual recognition challenge, Int. J. Comput. (2015) 211–252.

[34]

T. Liu, Z. Yuan, J. Sun, J. Wang, N. Zheng, X. Tang, H. Shum, Learning to detect a salient object, IEEE Trans. PAMI (2011) 674–681.

[35]

C. Yang, L. Zhang, H. Lu, X. Ruan, M.-H. Yang, Saliency detection via graph-based manifold ranking, in: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2013, pp. 3166–3173.

[36]

Q. Yan, L. Xu, J. Shi, J. Jia, Hierarchical saliency detection, in: IEEE Proc. CVPR, 2013, pp, 1–8.

[37]

K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, in: ICLR, 2015.

[38]

R. Zhao, W. Ouyang, H. Li, X. Wang, Saliency detection by multi-context deep learning, in: IEEE Proc. CVPR, 2015.

[39]

J. Zhang, S. Sclaroff, Z. Lin, X. Shen, B. Price, R. Mech, Minimum barrier salient object detection at 80 fps, in: ICCV, 2015, pp. 1404–1412.

[40]

M. Zhang, Y. Wu, Y. Du, L. Fang, Y. Pang, Saliency detection integrating global and local information, J. Vis. Commun. Image Represent. (2018) 215–223.

[41]

X.Z. Chen, H.M. Ma, X. Wang, Improving object proposals with multi-thresholding straddling expansion, IEEE Proc. CVPR, 2015.

[42]

L.Y. Zhang, M.H. Tong, T. Marks, H. Shan, et al., SUN: A Bayesian framework for saliency using natural statistics, J. Vision (2008) 1–20.

[43]

Q.B. Hou, M.M. Cheng, X.W. Hu, et al., Deeply supervised salient object detection with short connections, in: IEEE Proc. CVPR, 2017.

[44]

F. Perazzi, P. Krähenbühl, Y. Pritch, A. Hornung, Saliency filters: Contrast based filtering for salient region detection, in: IEEE Proc. CVPR, 2012.

[45]

C. Yang, L.H. Zhang, H.C. Lu, Graph-regularized saliency detection with convex-hull-based center prior, in: IEEE Signal Process, 2013, pp.637–640.

[46]

Y. Wei, F. Wen, W. Zhu, J. Sun, Geodesic saliency using background priors, in: European Conference on Computer Vision, 2012, pp. 163–174.

[47]

H.W. Peng, B. Li, H.B. Ling, et al., Salient object detection via structured matrix decomposition, in: IEEE Proc. CVPR, 2017, pp. 315–325.

[48]

Y. Xu, X. Hong, X. Liu, G. Zhao, Saliency detection via bi-directional propagation, J. Vis. Commun. Image Represent. (2018) 113–121.

[49]

M. Cheng, G. Zhang, N.J. Mitra, X. Huang, S. Hu, Global contrast based salient region detection, Proc. IEEE Conf. Comput. Vis. Pattern Recog. (2011) 409–416.

[50]

W. Zhu, S. Liang, Y. Wei, J. Sun, Saliency optimization from robust background detection, in: IEEE Proc. CVPR, 2014, pp.717–729.

[51]

T. Wang, L. Zhang, S. Wang, H.C. Lu, G. Yang, X. Ruan, A. Borji, Detect globally, refine locally: a novel approach to saliency detection, IEEE Proc. CVPR, 2018.

[52]

Y. Qin, M. Yang, H.C. Lu, et al., Hierarchical cellular automata for visual saliency, Int. J. Comput. Vision (2018) 751–770.

[53]

L.H. Zhang, J.W. Ai, B.W. Jiang, H.C. Lu, Saliency detection via Absorbing markov chain with learnt transition probability, IEEE Trans. Image Process. (2018).

[54]

L.H. Zhang, C. Yang, H.C. Lu, X. Ruan, M.H. Yang, Ranking saliency, IEEE Trans. Pattern Anal. Mach. Intell. (2017) 1892–1904.

[55]

J.W. Han, H. Chen, N. Liu, C.G. Yan, X.L. Li, CNNs-based RGB-D saliency detection via cross-view transfer and multiview fusion, IEEE Trans. Cybernet. (2018) 3171–3183.

[56]

G. Li, Y. Yu, Deep contrast learning for salient object detection, in: IEEE Proc. CVPR, 2016, pp. 478–487.

[57]

Q. Hou, M.M. Cheng, X. Hu, A. Borji, Z. Tu, Deeply supervised salient object detection with short connections, in: IEEE TPAMI, 2018.

[58]

G.Y. Lee, Y.W. Tai, J.M. Kim, Deep saliency with encoded low level distance map and high level features, IEEE Conf. Comput. Vis. Pattern Recog. (2016).

[59]

M.S. Allili, D. Ziou, Object of interest segmentation and tracking by using feature selection and active contours, IEEE Conf. Comput. Vision Pattern Recogn. (2007) 1–8.

[60]

R. Margolin, A. Tal, L. Zelnik-Manor, “What makes a patch distinct?”, in: Proc. IEEE Conf. Comput. Vision Pattern Recogn., 2013, pp. 1139–1146.

Index Terms

Saliency detection using Multi-layer graph ranking and combined neural networks
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks
  2. Machine learning
    1. Machine learning approaches
      1. Neural networks

Index terms have been assigned to the content through auto-classification.

Recommendations

Saliency detection using boundary information

Efficient and robust saliency detection is a fundamental problem in computer vision field for its wide applications, such as image segmentation and image retargeting, etc. In this paper, with the aim of uniformly highlighting the salient objects and ...
Object-level saliency detection with color attributes

Recently, saliency detection has been becoming a popular topic in computer vision. In this paper we propose an object-level saliency detection algorithm which explicitly explores bottom-up visual attention and objectness cues. Firstly, some category-...
Image saliency detection using Gabor texture cues

Image saliency analysis plays an important role in various applications such as object detection, image compression, and image retrieval. Traditional methods for saliency detection ignore texture cues. In this paper, we propose a novel method that ...

Comments

Information & Contributors

Information

Published In

cover image Journal of Visual Communication and Image Representation

Journal of Visual Communication and Image Representation Volume 65, Issue C

Dec 2019

271 pages

ISSN:1047-3203

Issue’s Table of Contents

Elsevier Inc.

Publisher

Academic Press, Inc.

United States

Publication History

Published: 01 December 2019

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

0
Total Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 15 Jan 2025

Other Metrics

View Author Metrics

Citations

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents