Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

Springer Nature Link
Log in

Real-time multiple object tracking using deep learning methods

  • S.I.: information, intelligence, systems and applications
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Multiple-object tracking is a fundamental computer vision task which is gaining increasing attention due to its academic and commercial potential. Multiple-object detection, recognition and tracking are quite desired in many domains and applications. However, accurate object tracking is very challenging, and things are even more challenging when multiple objects are involved. The main challenges that multiple-object tracking is facing include the similarity and the high density of detected objects, while also occlusions and viewpoint changes can occur as the objects move. In this article, we introduce a real-time multiple-object tracking framework that is based on a modified version of the Deep SORT algorithm. The modification concerns the process of the initialization of the objects, and its rationale is to consider an object as tracked if it is detected in a set of previous frames. The modified Deep SORT is coupled with YOLO detection methods, and a concrete and multi-dimensional analysis of the performance of the framework is performed in the context of real-time multiple tracking of vehicles and pedestrians in various traffic videos from datasets and various real-world footage. The results are quite interesting and highlight that our framework has very good performance and that the improvements on Deep SORT algorithm are functional. Lastly, we show improved detection and execution performance by custom training YOLO on the UA-DETRAC dataset and provide a new vehicle dataset consisting of 7 scenes, 11.025 frames and 25.193 bounding boxes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30
Fig. 31
Fig. 32
Fig. 33
Fig. 34
Fig. 35
Fig. 36
Fig. 37
Fig. 38
Fig. 39
Fig. 40
Fig. 41
Fig. 42
Fig. 43
Fig. 44
Fig. 45
Fig. 46
Fig. 47
Fig. 48
Fig. 49
Fig. 50

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Luo W, Zhao X, Kim T-K (2014) Multiple object tracking: a review. CoRR. http://arxiv.org/abs/1409.7618

  2. Ciaparrone G, Sánchez FL, Tabik S, Troiano L, Tagliaferri R, Herrera F (2020) Deep learning in video multi-object tracking: a survey. Neurocomputing 381:61–88

    Article  Google Scholar 

  3. Sanchez-Matilla R, Poiesi F, Cavallaro A (2016) Online multi-target tracking with strong and weak detections. vol 9914, pp 84–99. https://doi.org/10.1007/978-3-319-48881-3_7

  4. Sadeghian A, Alahi A, Savarese S (2017) Tracking the untrackable: learning to track multiple cues with long-term dependencies. CoRR. http://arxiv.org/abs/1701.01909

  5. Wojke N, Bewley A, Paulus D (2017) Simple online and realtime tracking with a deep association metric. CoRR. http://arxiv.org/abs/1703.07402

  6. Redmon J, Farhadi A (2018) YOLOv3: an incremental improvement. CoRR. http://arxiv.org/abs/1804.02767

  7. Bochkovskiy A, Wang C-Y, Liao H-YM (2020) YOLOv4: optimal speed and accuracy of object detection

  8. Bewley A, Ge Z, Ott L, Ramos F, Upcroft B (2016) Simple online and realtime tracking. CoRR. http://arxiv.org/abs/1602.00763

  9. Bernardin K, Stiefelhagen R (2008) Evaluating multiple object tracking performance: the CLEAR MOT metrics. EURASIP J Image Video Process 2008(1):246309. https://doi.org/10.1155/2008/246309

    Article  Google Scholar 

  10. Leal-Taixé L, Milan A, Reid ID, Roth S, Schindler K (2015) MOTChallenge 2015: towards a benchmark for multi-target tracking. CoRR. http://arxiv.org/abs/1504.01942

  11. Voigtlaender P et al (2019) MOTS: multi-object tracking and segmentation. CoRR. http://arxiv.org/abs/1902.03604

  12. Wen L et al (2015) DETRAC: a new benchmark and protocol for multi-object tracking. CoRR. http://arxiv.org/abs/1511.04136

  13. Fuchs F, Kosiorek AR, Sun L, Jones OP, Posner I (2019) End-to-end recurrent multi-object tracking and trajectory prediction with relational reasoning. http://arxiv.org/abs/1907.12887

  14. Chu P, Ling H (2019) FAMNet: joint learning of feature, affinity and multi-dimensional assignment for online multiple object tracking. CoRR. http://arxiv.org/abs/1904.04989

  15. Wang X, Cheng P, Liu X, Uzochukwu B (2018) Focal loss dense detector for vehicle surveillance. http://arxiv.org/abs/1803.01114

  16. Sun S, Akhtar N, Song X, Song H, Mian A, Shah M (2020) Simultaneous detection and tracking with motion modelling for multiple object tracking. http://arxiv.org/abs/2008.08826

  17. Chu Q, Ouyang W, Li H, Wang X, Liu B, Yu N (2017) Online multi-object tracking using CNN-based single object tracker with spatial-temporal attention mechanism. http://arxiv.org/abs/1708.02843

  18. Ristani E, Solera F, Zou RS, Cucchiara R, Tomasi C (2016) Performance measures and a data set for multi-target, multi-camera tracking. CoRR. http://arxiv.org/abs/1609.01775

  19. Redmon J, Divvala SK, Girshick RB, Farhadi A (2015) You only look once: unified, real-time object detection. CoRR. http://arxiv.org/abs/1506.02640

  20. Maiya SR (2020) abhyantrika/nanonets_object_tracking. GitHub. https://github.com/abhyantrika/nanonets_object_tracking

  21. Milan A, Leal-Taixé L, Reid ID, Roth S, Schindler K (2016) MOT16: a benchmark for multi-object tracking. CoRR. http://arxiv.org/abs/1603.00831

  22. Dendorfer P et al (2020) MOT20: a benchmark for multi object tracking in crowded scenes. http://arxiv.org/abs/2003.09003

  23. Emami P, Pardalos PM, Elefteriadou L, Ranka S (2020) Machine learning methods for data association in multi-object tracking. ACM Comput Surv (CSUR) 53(4):1–34

    Article  Google Scholar 

  24. Hou X, Wang Y, Chau LP (2019) Vehicle tracking using deep SORT with low confidence track filtering. In: 2019 16th IEEE international conference on advanced video and signal based surveillance (AVSS). IEEE, pp 1–6

  25. Wojke N, Bewley A (2018) Deep cosine metric learning for person re-identification. In: 2018 IEEE winter conference on applications of computer vision (WACV). IEEE, pp 748–756

  26. Karunasekera H, Wang H, Zhang H (2019) Multiple object tracking with attention to appearance, structure, motion and size. IEEE Access 7:104423–104434

    Article  Google Scholar 

  27. Voigtlaender P, Krause M, Osep A, Luiten J, Sekar BBG, Geiger A, Leibe B (2019) MOTS: multi-object tracking and segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7942–7951

  28. Sun S, Akhtar N, Song H, Mian AS, Shah M (2019) Deep affinity network for multiple object tracking. IEEE Trans Pattern Anal Mach Intell 43:104–119

    Google Scholar 

  29. Liu G, Liu S, Muhammad K, Sangaiah AK, Doctor F (2018) Object tracking in vary lighting conditions for fog based intelligent surveillance of public spaces. IEEE Access 6:29283–29296

    Article  Google Scholar 

  30. Xu S, Savvaris A, He S, Shin HS, Tsourdos A (2018) Real-time implementation of YOLO+ JPDA for small scale UAV multiple object tracking. In: 2018 international conference on unmanned aircraft systems (ICUAS). IEEE, pp 1336–1341

  31. Yoon YC, Boragule A, Song YM, Yoon K, Jeon M (2018) Online multi-object tracking with historical appearance matching and scene adaptive detection filtering. In: 2018 15th IEEE international conference on advanced video and signal based surveillance (AVSS). IEEE, pp 1–6

  32. Darknet: open source neural networks in c. https://github.com/AlexeyAB/darknet. Accessed 25 Apr 2020

  33. Hou X, Wang Y, Chau LP (2019) Vehicle tracking using deep SORT with low confidence track filtering. In: 2019 16th IEEE international conference on advanced video and signal based surveillance (AVSS). IEEE, pp 1–6

  34. Nguyen HQ, Nguyen TB, Nguyen TA, Le TL, Vu TH, Noe A. Comparative evaluation of human detection and tracking approaches for online tracking applications

  35. Lin JP, Sun MT (2018) A YOLO-based traffic counting system. In: 2018 conference on technologies and applications of artificial intelligence (TAAI). IEEE, pp 82–85

  36. Padilla R, Netto SL, da Silva EA (2020) A survey on performance metrics for object-detection algorithms. In: 2020 international conference on systems, signals and image processing (IWSSIP)

  37. Wang Z, Zheng L, Liu Y, Wang S (2019) Towards real-time multi-object tracking. arXiv preprint http://arxiv.org/abs/1909.12605

  38. Lu HC, Li PX, Wang D (2018) Visual object tracking: a survey. Pattern Recognit Artif Intell 31(1):61–76

    Google Scholar 

  39. Yao R, Lin G, Xia S, Zhao J, Zhou Y (2020) Video object segmentation and tracking: a survey. ACM Trans Intell Syst Technol (TIST) 11(4):1–47

    Article  Google Scholar 

  40. Llamazares Á, Molinos EJ, Ocaña M (2020) Detection and tracking of moving obstacles (DATMO): a review. Robotica 38(5):761–774

    Article  Google Scholar 

  41. Sam JR, Augasta G (2021) Review of recent advances in visual tracking techniques. Multimed Tools Appl 80:24185–24203

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Engineering and Informatics Department, University of Patras, Patras, Greece

    Dimitrios Meimetis, Ioannis Daramouskas, Isidoros Perikos & Ioannis Hatzilygeroudis

Authors
  1. Dimitrios Meimetis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ioannis Daramouskas
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Isidoros Perikos
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ioannis Hatzilygeroudis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Isidoros Perikos.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Meimetis, D., Daramouskas, I., Perikos, I. et al. Real-time multiple object tracking using deep learning methods. Neural Comput & Applic 35, 89–118 (2023). https://doi.org/10.1007/s00521-021-06391-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06391-y

Keywords

Search

Navigation