Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
SpringerLink
Log in

Long-term visual tracking algorithm for UAVs based on kernel correlation filtering and SURF features

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Long-term visual target tracking of unmanned aerial vehicles (UAVs) is a challenging and basic research topic. In recent years, many visual object tracking methods have been proposed based on the kernel correlation filtering algorithm and achieved good results. These algorithms have good performance in short-term tracking, but when the target is occluded or disappears from view, the original update strategy may lead to tracker drift. Based on these issues, this paper proposes a long-term kernel correlation filtering and speeded-up robust features (KCFSURF) target tracking algorithm for UAVs in the process of long-term target tracking due to target occlusion or loss. The algorithm takes the KCF target tracking algorithm as the framework, introduces the strategy of searching and locating the target after occlusion or loss, and uses the peak side lobe (PSR) ratio to determine whether the target is covered, blocked, or lost. When the target is occluded or lost, the SURF-random sample consensus (RANSAC) target retrieval matching strategy is introduced to rematch the target and select the box. The new samples are input into the KCF algorithm to continue tracking the target. To verify the superiority and feasibility of the proposed algorithm, the OTB100, UAV123, and Temple-color-128 dataset is selected to evaluate and analyze the algorithm quantitatively and qualitatively. The evaluation results show that KCFSURF can rediscover the target after it is blocked or lost, realizing long-term stable target tracking. Finally, the effectiveness of the KCFSURF algorithm is verified in an S500 UAV target tracking scene.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Mohamed, N., Al-Jaroodi, J., Jawhar, I., Idries, A., Mohammed, F.: Unmanned aerial vehicles applications in future smart cities. Technol. Forecast Soc. Chang. 153(2), 15 (2020). https://doi.org/10.1016/j.techfore.2018.05.004

    Article  Google Scholar 

  2. Zhang, S., Zhao, X., Zhou, B.: Robust vision-based control of a rotorcraft UAV for uncooperative target tracking. Sensors (Switzerland). 20(12), 1–23 (2020). https://doi.org/10.3390/s20123474

    Article  Google Scholar 

  3. Chen, J., Hua, C., Guan, X.: Image based fixed time visual servoing control for the quadrotor UAV. IET Control Theory Appl. 13(18), 3117–3123 (2019). https://doi.org/10.1049/iet-cta.2019.0032

    Article  MathSciNet  Google Scholar 

  4. Huang, Y., Chen, J., Wang, H., Su, G.: A method of 3D path planning for solar-powered UAV with fixed target and solar tracking. Aerosp. Sci. Technol. 92, 831–838 (2019). https://doi.org/10.1016/j.ast.2019.06.027

    Article  Google Scholar 

  5. Chen, S., Sun, J., Cao, Y.-G., et al.: Target tracking based on increment deep learning. Opt. Precis. Eng. 23(4), 1161–1170 (2015). https://doi.org/10.3788/OPE.20152304.1161

    Article  Google Scholar 

  6. Zhang, K., Zhang, L., Yang, M.H.: Real-time compressive tracking. In: 12th European Conference on Computer Vision (ECCV), Florence, ITALY, 07–13 October, 2012, pp. 864–877 (2012)

  7. Zhang, J., Ma, S., Sclaroff, S.: MEEM: robust tracking via multiple experts using entropy minimization. In: 13th European Conference on Computer Vision (ECCV), Zurich, SWITZERLAND, 06–12 September, pp. 188–203 (2014). https://doi.org/10.1007/978-3-319-10599-4_13

  8. Liu, B., Huang, J., Kulikowski, C., et al.: Robust visual tracking using local sparse appearance model and K-selection. IEEE Trans. Pattern Anal. Mach. Intell. 35(12), 2968–2981 (2013). https://doi.org/10.1109/TPAMI.2012.215

    Article  Google Scholar 

  9. Jia, X., Lu, H., Yang, M.-H.: Visual tracking via adaptive structural local sparse appearance model. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, 16–21 June, 2012, pp. 1822–1829 (2012)

  10. Danelljan, M., Khan, F.S., Felsberg, M.: Adaptive color attributes for real-time visual tracking. In: 27th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, 23–28 June, 2014, pp. 1090–1097. https://doi.org/10.1109/CVPR.2014.143

  11. Henriques, J.F., Caseiro, R., Martins, P., et al.: High-speed tracking with kernelized correlation filters. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 583–596 (2015). https://doi.org/10.1109/TPAMI.2014.2345390

    Article  Google Scholar 

  12. Oron, S., Bar-Hillel, A., Levi, D., et al.: Locally orderless tracking. Int. J. Comput. Vis. 111(2), 213–228 (2014). https://doi.org/10.1007/s11263-014-0740-6

    Article  MathSciNet  MATH  Google Scholar 

  13. Henriques, J., Caseiro, R., Martins, P., Batista, J.: Exploiting the circulant structure of tracking-by-detection with kernels. In: European Conference on Computer Vision (ECCV), Florence, ITALY, 07–13 October, 2012, pp. 702–715 (2012). https://doi.org/10.1007/978-3-642-33765-9_50

  14. Dinh, T.B., Vo, N., Medioni, G.: Context tracker: Exploring supporters and distracters in unconstrained environments. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Colorado Springs, CO, 20–25 June, 2011, pp. 1177–1184 (2011). https://doi.org/10.1109/CVPR.2011.5995733

  15. Gao, J., Ling, H., Hu, W., Xing, J.: Transfer learning based visual tracking with Gaussian process regression. In: 13th European Conference on Computer Vision (ECCV), Zurich, SWITZERLAND, 06–12 September, 2014, pp. 188–203 (2014). https://doi.org/10.1007/978-3-319-10578-9_13

  16. Mbelwa, J.T., Zhao, Q., Wang, F.: Visual tracking tracker via object proposals and co-trained kernelized correlation filters. Vis. Comput. (2020). https://doi.org/10.1007/s00371-019-01727-1

    Article  Google Scholar 

  17. Yang, X., Zhu, S., Xia, S., et al.: A new TLD target tracking method based on improved correlation filter and adaptive scale. Vis. Comput. 36(9), 1783–1795 (2020). https://doi.org/10.1007/s00371-019-01772-w

    Article  Google Scholar 

  18. Weng, S.-K., Kuo, C.-M., Shu-Kang, Tu.: Video object tracking using adaptive Kalman filter. J. Vis. Commun. Image Represent. 17(6), 1190–1208 (2006)

    Article  Google Scholar 

  19. Xiao, Y., Zhou, J., Zhao, B.: Attitude dynamics aiding for three-dimensional passive target tracking of strap-down seeker based on instrumental variable Kalman filter. Trans. Inst. Meas. Control. 1, 1–15 (2020). https://doi.org/10.1177/0142331220923768

    Article  Google Scholar 

  20. Pelland, N.A., Bennett, J.S., Steinberg, J.M., et al.: Automated glider tracking of a California undercurrent eddy using the extended Kalman filter. J. Atmos. Ocean. Technol. 35(11), 1–65 (2018). https://doi.org/10.1175/JTECH-D-18-0126.1

    Article  Google Scholar 

  21. Chen, Y., Wu, Y., Zhang, W.: Survey of target tracking algorithm based on siamese network structure. Comput. Eng. Appl. 56(6), 10–18 (2020)

    Google Scholar 

  22. Moorthy, S., Choi, J.Y., Joo, Y.H.: Gaussian-response correlation filter for robust visual object tracking. Neurocomputing 411, 78–90 (2020). https://doi.org/10.1016/j.neucom.2020.06.016

    Article  Google Scholar 

  23. Bertinetto, L., Valmadre, J., Henriques, J.F., Vedaldi, A., Torr, P.H.: Fully-convolutional Siamese networks for object tracking. In: Proc. ECCV, 2016, pp. 850–865 (2016). https://doi.org/10.1007/978-3-319-48881-3_56

  24. Bo, L., Yan, J., Wei, W., et al.: High performance visual tracking with siamese region proposal network. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 8971–8980 (2018). https://doi.org/10.1109/CVPR.2018.00935

  25. Jiang, C., Xiao, J., Xie, Y., et al.: Siamese network ensemble for visual tracking. Neurocomputing 275, 2892–2903 (2018). https://doi.org/10.1016/j.neucom.2017.10.043

    Article  Google Scholar 

  26. Zhang, W., Du, Y., Chen, Z., et al.: Robust adaptive learning with Siamese network architecture for visual tracking. Vis. Comput. 37(5), 881–894 (2020). https://doi.org/10.1007/s00371-020-01839-z

    Article  Google Scholar 

  27. Huang, Z., Zhao, H., Zhan, J., et al.: A multivariate intersection over union of SiamRPN network for visual tracking. Vis. Comput. 5, 1–12 (2021). https://doi.org/10.1007/s00371-021-02150-1

    Article  Google Scholar 

  28. Gordon, D., Farhadi, A., Fox, D.: Re3: real-time recurrent regression networks for visual tracking of generic objects. IEEE Robotics Autom. Lett. (2018). https://doi.org/10.1109/LRA.2018.2792152

    Article  Google Scholar 

  29. Wang, Q., Gao, J., Xing, J., Zhang, M., Hu, W.: DCFNet: discriminant correlation fifilters network for visual tracking. http://arxiv.org/abs/1704.04057 (2017)

  30. Valmadre, J., Bertinetto, L., Henriques, J., Vedaldi, A., Torr, P.H.: End-to-end representation learning for correlation fifilter based tracking. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 5000–5008 (2017). https://doi.org/10.1109/CVPR.2017.531

  31. Qi, Y., Zhang, S., Qin, L., Yao, H., Huang, Q., Lim, J., Yang, M.H.: Hedged deep tracking. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 4303–4311 (2016). https://doi.org/10.1109/CVPR.2016.466.

  32. Fan, H., Ling, H.: Parallel tracking and verifying: a framework for real-time and high accuracy visual tracking. In: IEEE Computer Society, 2017, pp. 5487–5495 (2017). https://doi.org/10.1109/ICCV.2017.585

  33. Ma, C., Huang, J.B., Yang, X., et al.: Adaptive correlation filters with long-term and short-term memory for object tracking. Int. J. Comput. Vis. (2018). https://doi.org/10.1007/s11263-018-1076-4

    Article  Google Scholar 

  34. Bhat, G., Danelljan, M., Gool, L.V., Timofte, R: Learning discriminative model prediction for tracking. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 6182–6191 (2019)

  35. Li, B., Wu, W., Wang, Q., et al.: SiamRPN++: evolution of siamese visual tracking with very deep networks. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2020).

  36. Choi, J.: Attentional correlation filter network for adaptive visual tracking. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 4828–4837 (2017)

  37. Boudjit, K., Larbes, C.: Detection and implementation autonomous target tracking with a quadrotor AR. In: Drone,12th International Conference on Informatics in Control Automation and Robotics (ICINCO), Alsace, FRANCE, 21–23 July, 2015, pp. 223–230 (2015). https://doi.org/10.5220/0005523102230230

  38. Pestana, J., Sanchezlopez, J.L., Campoy, P., et al.: Vision based GPS-denied object tracking and following for unmanned aerial vehicles. In: IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), Linkoping, SWEDEN, 21–26 October, 2013 (2013).https://doi.org/10.1109/SSRR.2013.6719359

  39. Liu, X., Yang, Y., Ma, C., et al.: Real-time visual tracking of moving targets using a low-cost unmanned aerial vehicle with a 3-axis stabilized gimbal system. Appl. Sci. 10(15), 5064 (2020). https://doi.org/10.3390/app10155064

    Article  Google Scholar 

  40. Haag, K., Dotenco, S., Gallwitz, F.: Correlation filter based visual trackers for person pursuit using a low-cost Quadrotor. In: International Conference on Innovations for Community Services (I4CS), Nuremberg, GERMANY, 08–10 July, 2015, pp. 1–8 (2015). https://doi.org/10.1109/I4CS.2015.7294481

  41. Li, R., Pang, M., Zhao, C., et al.: Monocular long-term target following on UAVs. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, June 26–July 01, 2016, pp. 29–37 (2016). https://doi.org/10.1109/CVPRW.2016.11

  42. Gundogdu, E., Ozkan, H., Demir, H.S., Ergezer, H., Erdem, A., Pakin, S.K.: Comparison of infrared and visible imagery for object tracking: Toward trackers with superior IR performance, Boston, MA, June 07–12, 2015, pp. 1–9. https://doi.org/10.1109/CVPRW.2015.7301290

  43. Lu, R., Yang, X., Li, W., et al.: Robust infrared small target detection via multidirectional derivative-based weighted contrast measure. IEEE Geosci. Remote Sens. Lett. 1(1), 1–5 (2020). https://doi.org/10.1109/LGRS.2020.3026546

    Article  Google Scholar 

  44. Lu, R., Yang, X., Jing, X., et al.: Infrared Small Target Detection Based on Local Hypergraph Dissimilarity Measure. IEEE Geosci. Remote Sens. Lett. (2020). https://doi.org/10.1109/LGRS.2020.3038784

    Article  Google Scholar 

  45. Qiang, W., Zheng, Z.: Long-term tracking based on kernelized correlation filtering. In: 2018 4th Annual International Conference on Network and Information Systems for Computers (ICNISC), Wuhan, China, 19–21 April, 2018, pp. 171–175 (2018). https://doi.org/10.1109/ICNISC.2018.00041

  46. Wang, S., Jiang, F., Zhang, B., et al.: Development of UAV-based target tracking and recognition systems. IEEE Trans. Intell. Transp. Syst. 21(8), 3409–3422 (2020). https://doi.org/10.1109/TITS.2019.2927838

    Article  Google Scholar 

  47. Li, W., Wu, J., Lu, H., et al.: Pedestrian target tracking algorithm based on improved RANSAC and KCF. In: 2020 Chinese Control And Decision Conference (CCDC). Cloud Conference, China, 22–25 August 2020, pp. 481–486 (2020). https://doi.org/10.1109/CCDC49329.2020.9164874

  48. Li, Y., Zhu, J.: A scale adaptive kernel correlation filter tracker with feature integration. In: European Conference on Computer Vision. Zurich, SWITZERLAND, 06–12 September 2014, pp. 254–265 (2014). https://doi.org/10.1007/978-3-319-16181-5_18

  49. Zhang, L., Suganthan, P.N.: Robust visual tracking via co-trained kernelized correlation filters. Pattern Recogn. 69, 82–93 (2017). https://doi.org/10.1016/j.patcog.2017.04.004

    Article  Google Scholar 

  50. Yangping, W., Jiu, Y., Zhengping, Z., et al.: Augmented reality tracking registration based on improved KCF tracking and ORB feature detection. In: 2019 7th International Conference on Information, Communication and Networks (ICICN). Macau, China, 5–7 January 2019, pp. 230–233 (2019). https://doi.org/10.1109/ICICN.2019.8834947

  51. Tareen, S.A.K., Saleem, Z.: A comparative analysis of SIFT, SURF, KAZE, AKAZE, ORB, and BRISK. In: 2018 International Conference on Computing, Mathematics and Engineering Technologies (iCoMET). Sukkur, Pakistan, 26 April 2018, pp. 978–986 (2018). https://doi.org/10.1109/ICOMET.2018.8346440

  52. Li, Y., Yang, C., Zhang, L., Xia, R., Fan, L., Xie, W.: A novel SURF based on a unified model of appearance and motion-variation. IEEE Access 6, 31065–31076 (2018). https://doi.org/10.1109/ACCESS.2018.2832290

    Article  Google Scholar 

  53. Zhang, T., Zhao, R., Chen, Z.: Application of migration image registration algorithm based on improved SURF in remote sensing image mosaic. IEEE Access 8, 163637–163645 (2020). https://doi.org/10.1109/access.2020.3020808

    Article  Google Scholar 

  54. Bay, H.: SURF: speeded up robust features. Comput. Vis. Image Underst. 110(3), 404–417 (2006). https://doi.org/10.1016/j.cviu.2007.09.014

    Article  Google Scholar 

  55. Zhu, Q., Wang, J., Zhang, P., et al.: Mobilerobot location research based on gauss moment improved SURF algorithms. J. Instrum. Instrum. 36(11), 2451–2457 (2015). https://doi.org/10.3969/j.issn.0254-3087.2015.11.007

    Article  Google Scholar 

  56. Pui, S.T., Minoi, J.L.: Keypoint descriptors in SIFT and SURF for face feature extractions. In: International Conference on Computational Science and Technology (ICCST), Kuala Lumpur, MALAYSIA, 29–30 November, 2018, vol. 488, pp. 64–73 (2018). https://doi.org/10.1007/978-981-10-8276-4_7

  57. Suju, D.A., Jose, H.: FLANN: fast approximate nearest neighbour search algorithm for elucidating human-wildlife conflicts in forest areas. In: International Conference on Signal Processing, Communication and Networking (ICSCN), Chennai, India, 16–18 March, 2017, pp. 2–7 (2017). https://doi.org/10.1109/ICSCN.2017.8085676

  58. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981). https://doi.org/10.1145/358669.358692

    Article  MathSciNet  Google Scholar 

  59. Qi, N., Zhang, S., Cao, L., et al.: Monocular visual navigation method with 1-point RANSAC based on aided matching. Xi Tong Gong Cheng Yu Dian Zi Ji Shu/Syst. Eng. Electron. 40(5), 1109–1117 (2018). https://doi.org/10.3969/j.issn.1001-506X.2018.05.23

    Article  Google Scholar 

  60. Liu, T., Wang, G., Yang, Q.: Real-time part-based visual tracking via adaptive correlation filters. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, 07–12 June, 2015, pp. 4902–4912 (2015). https://doi.org/10.1109/CVPR.2015.7299124

  61. Bolme, D.S., Beveridge, J.R., Draper. B.A., et al.: Visual object tracking using adaptive correlation filters. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2010, pp. 2544–2550 (2010). https://doi.org/10.1109/CVPR.2010.5539960

  62. Wu, Y., Lim, J., Yang, M.H.: Object tracking benchmark. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1834–1848 (2015). https://doi.org/10.1109/TPAMI.2014.2388226

    Article  Google Scholar 

  63. Liang, P., Blasch, E., Ling, H.: Encoding color information for visual tracking: algorithms and benchmark. IEEE Trans. Image Process. 24(12), 5630–5644 (2015). https://doi.org/10.1109/TIP.2015.2482905

    Article  MathSciNet  MATH  Google Scholar 

  64. Mueller, M., Smith, N., Ghanem, B.: A benchmark and simulator for UAV tracking, pp. 445–461. Springer, Berlin. https://doi.org/10.1007/978-3-319-46448-0_27

Download references

Funding

This work was supported in part by the National Natural Science Foundation of China under Grant 61806209, in part by the Natural Science Foundation of Shaanxi Province under Grant 2020JQ-490, and in part by the Aeronautical Science Fund under Grant 201851U8012. (Corresponding author: Xiaogang Yang.)

Author information

Authors and Affiliations

  1. Rocket Force University of Engineering, Xi’an, 710025, China

    Jiwei Fan, Xiaogang Yang, Ruitao Lu, Weipeng Li & Yueping Huang

Authors
  1. Jiwei Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaogang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ruitao Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Weipeng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yueping Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

X.Y., J.F., and R.L. contributed to conceptualization; J.F. and R.L. contributed to methodology and writing—original draft preparation; J.F. provided software; W.L. and Y.H. performed investigation; Y.H. provided resources; X.Y., J.F., and W.L. performed writing—review and editing; J.F. helped with visualization; J.F. and Y.H. carried out supervision; X.Y. contributed to funding acquisition and project administration. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Xiaogang Yang.

Ethics declarations

Conflicts of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fan, J., Yang, X., Lu, R. et al. Long-term visual tracking algorithm for UAVs based on kernel correlation filtering and SURF features. Vis Comput 39, 319–333 (2023). https://doi.org/10.1007/s00371-021-02331-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-021-02331-y

Keywords

Search

Navigation