Abstract
Despite the demonstrated success of numerous correlation filter (CF) based tracking approaches, their assumption of circulant structure of samples introduces significant redundancy to learn an effective classifier. In this paper, we develop a fast manifold regularized context-aware correlation tracking algorithm that mines the local manifold structure information of different types of samples. First, different from the traditional CF based tracking that only uses one base sample, we employ a set of contextual samples near to the base sample, and impose a manifold structure assumption on them. Afterwards, to take into account the manifold structure among these samples, we introduce a linear graph Laplacian regularized term into the objective of CF learning. Fortunately, the optimization can be efficiently solved in a closed form with fast Fourier transforms (FFTs), which contributes to a highly efficient implementation. Extensive evaluations on the OTB100 and VOT2016 datasets demonstrate that the proposed tracker performs favorably against several state-of-the-art algorithms in terms of accuracy and robustness. Especially, our tracker is able to run in real-time with 28 fps on a single CPU.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Li X, Hu W M, Shen C H, Zhang Z F, Dick A, Hengel A V D. A survey of appearance models in visual object tracking. ACM Transactions on Intelligent Systems and Technology, 2013, 4(4): 1–13
Wang H J, Ge H J. Visual tracking using discriminative representation with l 2 regularization. Frontiers of Computer Science, 2018, 12(1): 1–13
Ali A, Jalil A, Niu J W, Zhao X K, Rathore S, Ahmed J, Iftikhar M A. Visual object tracking-classical and contemporary approaches. Frontiers of Computer Science, 2016, 10(1): 167–188
Zhang K H, Liu Q S, Ynag J, Yang M H. Visual tracking via boolean map representations. Pattern Recognition, 2018, 81: 147–160
Zhang K H, Li X J, Song H H, Liu Q S, Lian W. Visual tracking using spatio-temporally nonlocally regularized correlation filter. Pattern Recognition, 2083, 185: 185–195
Bolme D S, Beveridge J R, Draper B A, Lui Y M. Visual object tracking using adaptive correlation filters. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2010, 2544–2550
Henriques J F, Caseiro R, Martins P, Batista J. High-speed tracking with kernelized correlation filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(3): 583–596
Zhang K H, Zhang L, Liu Q S, Zhang D, Yang M H. Fast visual tracking via dense spatio-temporal context learning. In: Proceedings of European Conference on Computer Vision. 2014, 127–141
Bertinetto L, Valmadre J, Golodetz S, Miksik O, Torr P H S. Staple: complementary learners for real-time tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 1401–1409
Zhang K H, Liu Q S, Wu Y, Yang M H. Robust visual tracking via convolutional networks without training. IEEE Transactions on Image Processing, 2016, 25(4): 1779–1792
Ma C, Xu Y, Ni B B, Yang X K. When correlation filters meet convolutional neural networks for visual tracking. IEEE Signal Processing Letters, 2016, 23(10): 1454–1458
Danelljan M, Häger G, Khan F, Felsberg M. Accurate scale estimation for robust visual tracking. In: Proceedings of the British Machine Vision Conference. 2014
Kristan M, Leonardis A, Matas J. The visual object tracking VOT2017 challenge results. In: Proceedings of the IEEE International Conference on Computer Vision Workshop. 2017, 1949–1972
Mueller M, Smith N, Ghanem B. Context-aware correlation filter tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017, 1396–1404
Galoogahi K H, Fagg A, Lucey S. Learning background-aware correlation filters for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision. 2017, 1135–1143
Yan Y, Nie F, Li W, Gao C Q, Yang Y, Xu D. Image classification by cross-media active learning with privileged information. IEEE Transactions on Multimedia, 2016, 18(12): 2494–2502
Yang Y, Ma Z G, Nie F P, Chang X J, Hauptmann A G. Multi-class active learning by uncertainty sampling with diversity maximization. International Journal of Computer Vision, 2015, 113(2): 113–127
Yang Y, Nie F P, Xu D, Luo J B, Zhuang Y T, Pan Y H. A multimedia retrieval framework based on semi-supervised ranking and relevance feedback. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(4): 723–742
Wu Y, Lim J W, Yang M H. Object tracking benchmark. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1834–1848
Danelljan M, Shahbaz K F, Felsberg M, Joost V W. Adaptive color attributes for real-time visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014, 1090–1097
Ma C, Huang J B, Yang X K, Yang M H. Hierarchical convolutional features for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision. 2015, 3074–3082
Danelljan M, Hager G, Khan F S, Felsberg M. Convolutional features for correlation filter based visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision Workshops. 2015, 58–66
Danelljan M, Robinson A, Khan F S, Felsberg M. Beyond correlation filters: learning continuous convolution operators for visual tracking. In: Proceedings of European Conference on Computer Vision. 2016, 472–488
Danelljan M, Bhat G, Khan F S, Felsberg M. ECO: rfficient convolution operators for tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017, 21–26
Liu S, Zhang T Z, Cao X C, Xu C S. Structural correlation filter for robust visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 4312–4320
Lukezic A, Vojír T, Zajc L C, Matas J, Kristan M. Discriminative correlation filter with channel and spatial reliability. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017, 6309–6318
Danelljan M, Hager G, Shahbaz K F, Felsberg M. Learning spatially regularized correlation filters for visual tracking. In: Proceedings of European Conference on Computer Vision. 2015, 4310–4318
Belkin M, Niyogi P, Sindhwani V. Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. Journal of Machine Learning Research, 2006, 7(Nov): 2399–2434
Chang X J, Yang Y. Semisupervised feature analysis by mining correlations among multiple tasks. IEEE Transactions on Neural Networks and Learning Systems, 2017, 28(10): 2294–2305
Yu S, Yang Y, Hauptmann A. Harry potter’s marauder’s map: localizing and tracking multiple persons-of-interest by nonnegative discretization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2013, 3714–3720
Bai Y C, Tang M. Robust tracking via weakly supervised ranking SVM. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2012, 1854–1861
Hu H W, Ma B, Shen J B, Shao L. Manifold regularized correlation object tracking. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(5): 1786–1795
Zhuang B, Lu H C, Xiao Z Y, Wang D. Visual tracking via discriminative sparse similarity map. IEEE Transactions on Image Processing, 2014, 23(4): 1872–1881
Belkin M, Niyogi P. Laplacian eigenmaps and spectral techniques for embedding and clustering. Advances in Neural Processing Systems, 2002, 585: 585–591
Ma C, Yang X K, Zhang C Y, Yang M H. Long-term correlation tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 5388–5396
Zhang J M, Ma S G, Sclaroff S. MEEM: robust tracking via multiple experts using entropy minimization. In: Proceedings of European Conference on Computer Vision. 2014, 188–203
Hare S, Golodetz S, Saffari A, Vineet V, Cheng M M, Hicks S L, Torr P H S. Struck: structured output tracking with kernels. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(10): 2096–2109
Wu Y, Lim J W, Yang M H. Online object tracking: a benchmark. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2013, 2411–2418
Song Y B, Ma C, Gong L J, Zhang J W, Lau R W H, Yang M H. Crest: convolutional residual learning for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision. 2017, 2574–2583
Zhu G, Porikli F, Li H D. Beyond local search: tracking objects everywhere with instance-specific proposals. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 943–951
Acknowledgments
This work was supported in part by NSF of Jiangsu province (BK20170040), in part by the National Natural Science Foundation of China (Grant Nos. 61872189, 61876088, 61605083), in part by the NSF of Jiangsu Higher Education Institutions of China (16KJB510023), in part by Postgraduate Research & Practice Innovation Program of Jiangsu Province (KYCX17_0903).
Author information
Authors and Affiliations
Corresponding author
Additional information
Jiaqing Fan recieved Bachelor degree from the School of Information and Control in Nanjing University of Information Science and Technology, China. He is currently pursuing the MS degree with the School of Information and Control, Nanjing University of Information Science and Technology, China. His current research interests include visual object tracking algorithms.
Huihui Song is a professor with the Jiangsu Key Laboratory of Big Data Analysis Technology, Nanjing University of Information Science and Technology, China. She received her BS degree in technology and science of electronic information from Ocean University of China in 2008, master’s degree in communication and information system from University of Science and Technology of China (USTC) in 2011, and PhD degree in geography and resource management from the Chinese University of Hong Kong, China in 2014. Her research interests include remote sensing image processing and image fusion.
Kaihua Zhang is a professor in the School of Information and Control, Nanjing University of Information Science and Technology, China. He received the BS degree in Technology and Science of Electronic Information from Ocean University of China (OUC), China in 2006, the MS degree in Signal and Information Processing from the University of Science and Technology of China (USTC), China in 2009 and PhD degree from the Department of Computing in the Hong Kong Polytechnic University, China in 2013. From August 2009 to August 2010, he worked as a Research Assistant in the Department of Computing, The Hong Kong Polytechnic University, China. His research interests include image segmentation, level sets, and visual tracking.
Qingshan Liu is a professor with the School of Information and Control, Nanjing University of Information Science and Technology, China. He received the PhD degree from the National Laboratory of Pattern Recognition, Chinese Academic of Science, China in 2003, and the MS degree from the Department of Auto Control, Southeast University, China in 2000. He was an Assistant Research Professor with the Department of Computer Science, Computational Biomedicine Imaging and Modeling Center, Rutgers, The State University of New Jersey, New Brunswick, NJ, USA, from 2010 to 2011. Before he joined Rutgers University, he was an Associate Professor with the National Laboratory of Pattern Recognition, Chinese Academic of Science, and an Associate Researcher with the Multimedia Laboratory, Chinese University of Hong Kong, China, from 2004 and 2005. He was a recipient of the President Scholarship of the Chinese Academy of Sciences, China in 2003. His current research interests are image and vision analysis, including face image analysis, graph and hypergraph-based image and video understanding, medical image analysis, and event-based video analysis.
Fei Yan is a PhD, lecturer. He graduated from the University of Chinese Academy of Sciences in the electrical and systems specialty, China, mainly engaged in image processing, information display technology. Wei Lian received the BS degree in automation from Taiyuan University of Technology, China in 2000, and the MS and PhD degrees in automatic control theory and engineering from Northwestern Polytechnical University, China in 2003 and 2007, respectively. He is an associate professor in the Department of Computer Science, Changzhi University, China. He was a research fellow in the Department of Computing, The Hong Kong Polytechnic University, China in 2018.
Electronic supplementary material
Rights and permissions
About this article
Cite this article
Fan, J., Song, H., Zhang, K. et al. Real-time manifold regularized context-aware correlation tracking. Front. Comput. Sci. 14, 334–348 (2020). https://doi.org/10.1007/s11704-018-8104-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11704-018-8104-y