Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

Visual Tracking Method Based on Siamese Network with Multi-Feature Fusion

  • Published:
Automatic Control and Computer Sciences Aims and scope Submit manuscript

Abstract

The traditional deep learning tracking method SiamFC faces performance degradation while solving issues, for instance, similar background, occlusion, target deformation, and illumination variation. This paper proposes an improved SiamFC with multi-feature fusion strategy. The proposed method first extracts the histogram of gradient and color name of the template image and search area by correlation filter. Then, the method fuses them and weights the SiamFC response map to obtain a more accurate object response position. Comparison experiments on VOT and OTB datasets prove that the improved method is more accurate and robust than the excellent tracking methods to deal with problems such as target cover, out of sight, scale variation and motion blur.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.

Similar content being viewed by others

REFERENCES

  1. Yilmaz, A., Javed, O., and Shah, M., Object tracking: A survey, ACM Comput. Surv., 2006, vol. 38, no. 4, pp. 13–45.  https://doi.org/10.1145/1177352.1177355

    Article  Google Scholar 

  2. Yang, H., Shao, L., Zheng, F., Wang, L., and Song, Z., Recent advances and trends in visual tracking: A review, Neurocomputing, 2011, vol. 74, no. 18, pp. 3823–3831.  https://doi.org/10.1016/j.neucom.2011.07.024

    Article  Google Scholar 

  3. Smeulders, A.W.M., Chu, D.M., Cucchiara, R., Calderara, S., Dehghan, A., and Shah, M., Visual tracking: An experimental survey, IEEE Trans. Pattern Anal. Mach. Intell., 2014, vol. 36, no. 7, pp. 1442–1468.  https://doi.org/10.1109/TPAMI.2013.230

    Article  Google Scholar 

  4. Li, P., Wang, D., Wang, L., and Lu, H., Deep visual tracking: Review and experimental comparison, Pattern Recognit., 2018, vol. 76, pp. 323–338.  https://doi.org/10.1016/j.patcog.2017.11.007

    Article  Google Scholar 

  5. Danelljan, M., Häger, G., Khan, F., and Felsberg, M., Accurate scale estimation for robust visual tracking, Proc. of the British Machine Vision Conference 2014, Valstar, M., French, A., and Pridmore, T., Eds., BMVA Press, 2014, pp. 1–11.  https://doi.org/10.5244/C.28.65

  6. Danelljan, M., Häger, G., Khan, F.S., and Felsberg, M., Discriminative scale space tracking, IEEE Trans. Pattern Anal. Mach. Intell., 2017, vol. 39, no. 8, pp. 1561–1575.  https://doi.org/10.1109/TPAMI.2016.2609928

    Article  Google Scholar 

  7. Zhang, J., Ma, S., and Sclaroff, S., MEEM: Robust tracking via multiple experts using entropy minimization, Computer Vision – ECCV 2014, Fleet, D., Pajdla, T., Schiele, B., and Tuytelaars, T., Eds., Lecture Notes in Computer Science, vol. 8694, Cham: Springer, 2014, pp. 188–203.  https://doi.org/10.1007/978-3-319-10599-4_13

    Book  Google Scholar 

  8. Bolme, D.S., Beveridge, J.R., Draper, B.A., and Lui, Y.M., Visual object tracking using adaptive correlation filters, IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, San Francisco, 2010, IEEE, 2010, pp. 2544–2550.  https://doi.org/10.1109/CVPR.2010.5539960

  9. Henriques, J.F., Caseiro, R., Martins, P., and Batista, J., Exploiting the circulant structure of tracking-by-detection with kernels, Computer Vision—ECCV 2012, Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., and Schmid, C., Eds., Lecture Notes in Computer Science, vol. 7575, Berlin: Springer, 2012, pp. 702–715.  https://doi.org/10.1007/978-3-642-33765-9_50

    Book  Google Scholar 

  10. Henriques, J.F., Caseiro, R., Martins, P., and Batista, J., High-speed tracking with kernelized correlation filters, IEEE Trans. Pattern Anal. Mach. Intell., 2015, vol. 37, no. 3, pp. 583–596.  https://doi.org/10.1109/TPAMI.2014.2345390

    Article  Google Scholar 

  11. Li, Y. and Zhu, J., A scale adaptive kernel correlation filter tracker with feature integration, Computer Vision— ECCV 2014 Workshops, Agapito, L., Bronstein, M., and Rother, C., Eds., Lecture Notes in Computer Science, vol. 8926, Cham: Springer, 2015, pp. 254–265.  https://doi.org/10.1007/978-3-319-16181-5_18

    Book  Google Scholar 

  12. Danelljan, M., Häger, G., Khan, F.S., and Felsberg, M., Learning spatially regularized correlation filters for visual tracking, IEEE Int. Conf. on Computer Vision (ICCV), Santiago, 2015, IEEE, 2015, pp. 4310–4318.  https://doi.org/10.1109/ICCV.2015.490

  13. Danelljan, M., Häger, G., Khan, F.S., and Felsberg, M., Convolutional features for correlation filter based visual tracking, IEEE Int. Conf. on Computer Vision Workshop (ICCVW), Santiago, 2015, IEEE, 2015, pp. 58–66.https://doi.org/10.1109/ICCVW.2015.84

  14. Lukežic, A., Vojír, T., Zajc, L.C., Matas, J., and Kristan, M., Discriminative correlation filter with channel and spatial reliability, IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Honolulu, Hawaii, 2017, IEEE, 2017, pp. 6309–6318.  https://doi.org/10.1109/CVPR.2017.515

  15. Tao, R., Gavves, E., and Smeulders, A.W.M., Siamese instance search for tracking, IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 2016, IEEE, 2016, pp. 1420–1429.  https://doi.org/10.1109/CVPR.2016.158

  16. Bertinetto, L., Valmadre, J., Henriques, J.F., Vedaldi, A., and Torr, P.H., Fully-convolutional Siamese networks for object tracking, Computer Vision—ECCV 2016 Workshops, Hua, G. and Jégou, H., Eds., Lecture Notes in Computer Science, vol. 9914, Cham: Springer, 2016, pp. 850–865.  https://doi.org/10.1007/978-3-319-48881-3_56

    Book  Google Scholar 

  17. Krizhevsky, A., Sutskever, I., and Hinton, G.E., ImageNet classification with deep convolutional neural networks, Proc. 25th Int. Conf. on Neural Information Processing Systems, Lake Tahoe, Nev., 2012, Pereira, F., Burges, C.J.C., Bottou, L., and Weinberger, K.Q., Eds., Red Hook, N.Y.: Curran Associates, 2012, vol. 1, pp. 1097–1105.

  18. Valmadre, J., Bertinetto, L., Henriques, J., and Vedaldi, A., and Torr, P.H.S., End-to-end representation learning for correlation filter based tracking, IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Honolulu, Hawaii, 2017, IEEE, 2017, pp. 2805–2813.  https://doi.org/10.1109/CVPR.2017.531

  19. Wang, Q., Gao, J., Xing, J., Zhang, M., and Hu, W., DCFNet: Discriminant correlation filters network for visual tracking, 2017. arXiv:1704.04057 [cs.CV]

  20. Li, B., Yan, J., Wu, W., Zhu, Z., and Hu, X., High performance visual tracking with Siamese region proposal network, IEEE/CVF Conf. on Computer Vision and Pattern Recognition, Salt Lake City, Utah, 2018, IEEE, 2018, pp. 8971–8980.  https://doi.org/10.1109/CVPR.2018.00935

  21. Zhu, Z., Wang, Q., Li, B., Wu, W., Yan, J., and Hu, W., Distractor-aware Siamese networks for visual object tracking, Computer Vision—ECCV 2018, Ferrari, V., Hebert, M., Sminchisescu, C., and Weiss, Y., Eds., Lecture Notes in Computer Science, vol. 11213, Cham: Springer, 2018, pp. 101–117.  https://doi.org/10.1007/978-3-030-01240-3_7

    Book  Google Scholar 

  22. Li, B., Wu, W., Wang, Q., Zhang, F., Xing, J., and Yan, J., SiamRPN++: Evolution of Siamese visual tracking with very deep networks, IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), Long Beach, Calif., 2019, IEEE, 2019, pp. 4282–4291.  https://doi.org/10.1109/CVPR.2019.00441

  23. Wang, Q., Li, Z., Bertinetto, L., Hu, W., and Torr, P.H.S., Fast online object tracking and segmentation: A unifying approach, IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), Long Beach, Calif., 2019, IEEE, 2019, pp. 1328–1338.  https://doi.org/10.1109/CVPR.2019.00142

  24. Fan, H. and Ling, H., Siamese cascaded region proposal networks for real-time visual tracking, IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), Long Beach, Calif., 2019, IEEE, 2019, pp. 7952–7961.  https://doi.org/10.1109/CVPR.2019.00814

  25. Danelljan, M., Bhat, G., Khan, F.S., and Felsberg, M., ATOM: Accurate tracking by overlap maximization, IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), Long Beach, Calif., 2019, IEEE, 2019, pp. 4660–4669.  https://doi.org/10.1109/CVPR.2019.00479

  26. Wu, Y., Lim, J., and Yang, M.-H., Online object tracking: A benchmark, IEEE Conf. on Computer Vision and Pattern Recognition, Portland, Ore., 2013, IEEE, 2013, pp. 2411–2418.  https://doi.org/10.1109/CVPR.2013.312

  27. Dalal, N. and Triggs, B., Histograms of oriented gradients for human detection, IEEE Computer Society Conf. on Computer Vision and Pattern Recognition (CVPR’05), San Diego, Calif., 2005, IEEE, 2005, vol. 1, pp. 886–893.  https://doi.org/10.1109/CVPR.2005.177

  28. Danelljan, M., Khan, F.S., Felsberg, M., and Van De Weijer, J., Adaptive color attributes for real-time visual tracking, IEEE Conf. on Computer Vision and Pattern Recognition, Columbus, Ohio, 2014, IEEE, 2014, pp. 1090–1097.  https://doi.org/10.1109/CVPR.2014.143

  29. Bertinetto, L., Valmadre, J., Golodetz, S., Miksik, O., and Torr, P.H., Staple: Complementary learners for real-time tracking, IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 2016, IEEE, 2016, pp. 1401–1409.  https://doi.org/10.1109/CVPR.2016.156

  30. Gao, T., Wang, N., Cai, J., Lin, W., Yu, X., Qiu, J., and Gao, H., Explicitly exploiting hierarchical features in visual object tracking, Neurocomputing, 2020, vol. 397, pp. 203–211.  https://doi.org/10.1016/j.neucom.2020.02.038

    Article  Google Scholar 

Download references

Funding

This research was financially supported by the TaiShan Scholar Foundation (project no. tshw201502042), Shandong Province Key Research and Development Plan (project nos. 2017CXG0607 and 2017GGX30145), and National Natural Science Foundation of China (project no. 61702295).

Author information

Authors and Affiliations

  1. Qingdao University of Science and Technology, Qingdao, Shandong, China

    Qingdang Li, Rui Xu, Mingyue Zhang & Zhen Sun

Authors
  1. Qingdang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rui Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mingyue Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhen Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhen Sun.

Ethics declarations

The authors declare that they have no conflicts of interest.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qingdang Li, Xu, R., Zhang, M. et al. Visual Tracking Method Based on Siamese Network with Multi-Feature Fusion. Aut. Control Comp. Sci. 56, 150–159 (2022). https://doi.org/10.3103/S0146411622020080

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.3103/S0146411622020080

Keywords:

Search

Navigation