Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

RANGO: A Novel Deep Learning Approach to Detect Drones Disguising from Video Surveillance Systems

Published: 22 February 2024 Publication History

Abstract

Video surveillance systems provide means to detect the presence of potentially malicious drones in the surroundings of critical infrastructures. In particular, these systems collect images and feed them to a deep-learning classifier able to detect the presence of a drone in the input image. However, current classifiers are not efficient in identifying drones that disguise themselves with the image background, e.g., hiding in front of a tree. Furthermore, video-based detection systems heavily rely on the image’s brightness, where darkness imposes significant challenges in detecting drones. Both these phenomena increase the possibilities for attackers to get close to critical infrastructures without being spotted and hence be able to gather sensitive information or cause physical damages, possibly leading to safety threats.
In this article, we propose RANGO, a drone detection arithmetic able to detect drones in challenging images where the target is difficult to distinguish from the background. RANGO is based on a deep learning architecture that exploits a Preconditioning Operation (PREP) that highlights the target by the difference between the target gradient and the background gradient. The idea is to highlight features that will be useful for classification. After PREP, RANGO uses multiple convolution kernels to make the final decision on the presence of the drone. We test RANGO on a drone image dataset composed of multiple already-existing datasets to which we add samples of birds and planes. We then compare RANGO with multiple currently existing approaches to show its superiority. When tested on images with disguising drones, RANGO attains an increase of 6.6% mean Average Precision (mAP) compared to YOLOv5 solution. When tested on the conventional dataset, RANGO improves the mAP by approximately 2.2%, thus confirming its effectiveness also in the general scenario.

References

[1]
Ryo Abiko and Masaaki Ikehara. 2022. Channel attention GAN trained with enhanced dataset for single-image shadow removal. IEEE Access 10 (2022), 12322–12333. DOI:
[2]
K. S. Arikumar, A. Deepak Kumar, Thippa Reddy Gadekallu, Sahaya Beni Prathiba, and K. Tamilarasi. 2022. Real-time 3D object detection and classification in autonomous driving environment using 3D LiDAR and camera sensors. Electronics 11, 24 (2022), 4203.
[3]
arstechnica. 2013. German Chancellor’s Drone “attack” Shows the Threat of Weaponized UAVs. Retrieved from https://arstechnica.com/information-technology/2013/09/german-chancellors-drone-attack-shows-the-threat-of-weaponized-uavs/
[4]
Ali Awad. 2019. Denoising images corrupted with impulse, Gaussian, or a mixture of impulse and Gaussian noise. Eng. Sci. Technol. Int. J. 22, 3 (2019), 746–753. DOI:
[5]
Jimmy Ba and Rich Caruana. 2014. Do deep nets really need to be deep? In Annual Conference on Neural Information Processing Systems, Zoubin Ghahramani, Max Welling, Corinna Cortes, Neil D. Lawrence, and Kilian Q. Weinberger (Eds.). 2654–2662.
[6]
Dinesh Kumar Behera and Arockia Bazil Raj. 2020. Drone detection and classification using deep learning. In 4th International Conference on Intelligent Computing and Control Systems (ICICCS’20). IEEE, 1012–1016.
[7]
Alexey Bochkovskiy, Chien-Yao Wang, and Hong-Yuan Mark Liao. 2020. YOLOv4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934 (2020).
[8]
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. 2020. End-to-end object detection with transformers. In European Conference on Computer Vision (ECCV’20).
[9]
Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L. Yuille. 2018. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40, 4 (2018), 834–848. DOI:
[10]
Yueru Chen, Pranav Aggarwal, Jongmoo Choi, and C.-C. Jay Kuo. 2017. A deep learning approach to drone monitoring. In Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC’17). IEEE, 686–691. DOI:
[11]
Y. Chien. 1974. Pattern classification and scene analysis. IEEE Trans. Automat. Control 19, 4 (1974), 462–463. DOI:
[12]
Jinghui Chu, Xiaoqian Zhao, Dan Song, Wenhui Li, Shenyuan Zhang, Xuanya Li, and An-An Liu. 2022. Improved semantic representation learning by multiple clustering for image-based 3D model retrieval. Int. J. Semant. Web Inf. Syst. 18, 1 (2022), 1–20.
[13]
Navneet Dalal and Bill Triggs. 2005. Histograms of oriented gradients for human detection. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05). IEEE, 886–893.
[14]
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020).
[15]
Xianzhi Du, Tsung-Yi Lin, Pengchong Jin, Golnaz Ghiasi, Mingxing Tan, Yin Cui, Quoc V. Le, and Xiaodan Song. 2020. SpineNet: Learning scale-permuted backbone for recognition and localization. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11592–11601.
[16]
Kaiwen Duan, Song Bai, Lingxi Xie, Honggang Qi, Qingming Huang, and Qi Tian. 2019. CenterNet: Keypoint triplets for object detection. In IEEE/CVF International Conference on Computer Vision. 6569–6578.
[17]
Martins Ezuma, Fatih Erden, Chethan Kumar Anjinappa, Ozgur Ozdemir, and Ismail Guvenc. 2019. Detection and classification of UAVs using RF fingerprints in the presence of Wi-Fi and Bluetooth interference. IEEE Open J. Commun. Societ. 1 (2019), 60–76.
[18]
Yunlong Gao, Zhiyong Wu, Ming Ren, and Chuan Wu. 2022. Improved YOLOv4 based on attention mechanism for ship detection in SAR images. IEEE Access 10 (2022), 23785–23797. DOI:
[19]
Atik Garg, Xin-Wen Pan, and Lan-Rong Dung. 2022. LiCENt: Low-light image enhancement using the light channel of HSL. IEEE Access 10 (2022), 33547–33560. DOI:
[20]
Muhammad Shahzeb Khan Gul, M. Umair Mukati, Michel Bätz, Søren Forchhammer, and Joachim Keinert. 2022. Attention Mechanism-based Light-field View Synthesis. IEEE Access 10 (2022), 7895–7913. DOI:
[21]
Haibo He and Yunqian Ma. 2013. Imbalanced learning: Foundations, algorithms, and applications. Wiley-IEEE Press 1, 27 (2013), 12.
[22]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2014. Spatial pyramid pooling in deep convolutional networks for visual recognition. In 13th European Conference on Computer Vision (ECCV’14)(Lecture Notes in Computer Science, Vol. 8691), David J. Fleet, Tomás Pajdla, Bernt Schiele, and Tinne Tuytelaars (Eds.). Springer, 346–361. DOI:
[23]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition. 770–778.
[24]
Andrew Howard, Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang, Vijay Vasudevan, et al. 2019. Searching for MobileNetV3. In IEEE/CVF International Conference on Computer Vision. 1314–1324.
[25]
Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. 2017. MobileNets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017).
[26]
Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In 32nd International Conference on Machine Learning (ICML’15)(JMLR Workshop and Conference Proceedings, Vol. 37), Francis R. Bach and David M. Blei (Eds.). JMLR.org, 448–456.
[27]
Iza Sazanita Isa, Mohamed Syazwan Asyraf Bin Rosli, Umi Kalsom Yusof, Mohd Ikmal Fitri Maruzuki, and Siti Noraini Sulaiman. 2022. Optimizing the hyperparameter tuning of YOLOv5 for underwater detection. IEEE Access 10 (2022), 52818–52831. DOI:
[28]
Yong Jin, Huifang Gao, Xiaoliang Fan, Hassan Khan, and Youxing Chen. 2022. Defect identification of adhesive structure based on DCGAN and YOLOv5. IEEE Access 10 (2022), 79913–79924. DOI:
[29]
Glenn Jocher. 2020. YOLOv5 by Ultralytics. DOI:
[30]
Glenn Jocher, Ayush Chaurasia, and Jing Qiu. 2023. YOLO by Ultralytics. Retrieved from https://github.com/ultralytics/ultralytics
[31]
Ranpreet Kaur, Hamid GholamHosseini, and Roopak Sinha. 2022. Hairlines removal and low contrast enhancement of melanoma skin images using convolutional neural network with aggregation of contextual information. Biomed. Sig. Process. Contr. 76 (2022), 103653. DOI:
[32]
Gang Li, Chengpeng Zheng, Min Li, and Haosen Wang. 2022. Automatic requirements classification based on graph attention network. IEEE Access 10 (2022), 30080–30090. DOI:
[33]
Jeong-Seon Lim, Marcella Astrid, Hyun-Jin Yoon, and Seung-Ik Lee. 2021. Small object detection using context and attention. In International Conference on Artificial Intelligence in Information and Communication (ICAIIC’21). IEEE, 181–186. DOI:
[34]
Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott E. Reed, Cheng-Yang Fu, and Alexander C. Berg. 2016. SSD: Single shot multibox detector. In 14th European Conference on Computer Vision (ECCV’16)(Lecture Notes in Computer Science, Vol. 9905), Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling (Eds.). Springer, 21–37. DOI:
[35]
Yuqi Liu, Xianrong Wan, Hui Tang, Jianxin Yi, Yiyao Cheng, and Xun Zhang. 2017. Digital television based passive bistatic radar system for drone detection. In IEEE Radar Conference (RadarConf’17). IEEE, 1493–1497.
[36]
Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. 2022. A convnet for the 2020s. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11976–11986.
[37]
Xudong Long, Weiwei Zhang, and Bo Zhao. 2020. PSPNet-SLAM: A semantic SLAM detect dynamic object by pyramid scene parsing network. IEEE Access 8 (2020), 214685–214695. DOI:
[38]
Ben Nassi, Asaf Shabtai, Ryusuke Masuoka, and Yuval Elovici. 2019. SoK-security and privacy in the age of drones: Threats, challenges, solution mechanisms, and scientific gaps. arXiv preprint arXiv:1903.05155 (2019).
[39]
Seongjoon Park, Hyeong Tae Kim, Sangmin Lee, Hyeontae Joo, and Hwangnam Kim. 2021. Survey on anti-drone systems: Components, designs, and challenges. IEEE Access 9 (2021), 42635–42659.
[40]
Maciej L. Pawelczyk and Marek Wojtyra. 2020. Real world object detection dataset for quadcopter unmanned aerial vehicle detection. IEEE Access 8 (2020), 174394–174409. DOI:
[41]
Kim-Phuong Phung, Thai-Hoc Lu, Trung-Thanh Nguyen, Ngoc-Long Le, Huu-Hung Nguyen, and Van-Phuc Hoang. 2021. Multi-model deep learning drone detection and tracking in complex background conditions. In International Conference on Advanced Technologies for Communications (ATC’21). IEEE, 189–194.
[42]
Zhixin Piao, Yongyun Jin, Stephen Li, Chunhua Shen, and Anton van den Hengel. 2021. Vision transformer for large-scale aerial image segmentation. In Conference on Computer Vision and Pattern Recognition (CVPR’21).
[43]
Eric L. Piza, Brandon C. Welsh, David P. Farrington, and Amanda L. Thomas. 2019. CCTV surveillance for crime prevention: A 40-year systematic review with meta-analysis. Criminol. Pub. Polic. 18, 1 (2019), 135–159.
[44]
Wenguang Qian, Hua Li, and Haiping Mu. 2022. Circular LBP prior-based enhanced GAN for image style transfer. Int. J. Semant. Web Inf. Syst. 18, 2 (2022), 1–15.
[45]
Ilija Radosavovic, Raj Prateek Kosaraju, Ross Girshick, Kaiming He, and Piotr Dollár. 2020. Designing network design spaces. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10428–10436.
[46]
Shaoqing Ren, Kaiming He, Ross B. Girshick, and Jian Sun. 2017. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39, 6 (2017), 1137–1149. DOI:
[47]
Fredrik Svanström, Cristofer Englund, and Fernando Alonso-Fernandez. 2021. Real-time drone detection and tracking with visible, thermal and acoustic sensors. In 25th International Conference on Pattern Recognition (ICPR’20). IEEE, 7265–7272.
[48]
Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander A. Alemi. 2017. Inception-v4, Inception-ResNet and the impact of residual connections on learning. In 31st AAAI Conference on Artificial Intelligence, Satinder Singh and Shaul Markovitch (Eds.). AAAI Press, 4278–4284.
[49]
Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16). IEEE Computer Society, 2818–2826. DOI:
[50]
Mingxing Tan, Ruoming Pang, and Quoc V. Le. 2020. EfficientDet: Scalable and efficient object detection. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10781–10790.
[51]
Antonio Torralba. 2003. Contextual priming for object detection. Int. J. Comput. Vis. 53, 2 (2003), 169–191. DOI:
[52]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Annual Conference on Neural Information Processing Systems, Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 5998–6008.
[53]
Paul Viola and Michael Jones. 2001. Rapid object detection using a boosted cascade of simple features. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’01). IEEE, I–I.
[54]
Viktor Walter, Matouš Vrba, and Martin Saska. 2020. On training datasets for machine learning-based visual relative localization of micro-scale UAVs. In IEEE International Conference on Robotics and Automation (ICRA’20). 10674–10680. DOI:
[55]
Chien-Yao Wang, Alexey Bochkovskiy, and Hong-Yuan Mark Liao. 2023. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7464–7475.
[56]
Chien-Yao Wang, Hong-Yuan Mark Liao, Yueh-Hua Wu, Ping-Yang Chen, Jun-Wei Hsieh, and I-Hau Yeh. 2020. CSPNet: A new backbone that can enhance learning capability of CNN. In IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 390–391.
[57]
Haoxiang Wang, Zhihui Li, Yang Li, Brij B. Gupta, and Chang Choi. 2020. Visual saliency guided complex image retrieval. Pattern Recog. Lett. 130 (2020), 64–72.
[58]
Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon. 2018. CBAM: Convolutional block attention module. In 15th European Conference on Computer Vision (ECCV’18)(Lecture Notes in Computer Science, Vol. 11211), Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss (Eds.). Springer, 3–19. DOI:
[59]
Matthew D. Zeiler and Rob Fergus. 2014. Visualizing and understanding convolutional networks. In 13th European Conference on Computer Vision (ECCV’14). Springer, 818–833.
[60]
Ye Zheng, Zhang Chen, Dailin Lv, Zhixing Li, Zhenzhong Lan, and Shiyu Zhao. 2021. Air-to-air visual detection of micro-UAVs: An experimental evaluation of deep learning. IEEE Robot. Autom. Lett. 6, 2 (2021), 1020–1027. DOI:
[61]
Xingkui Zhu, Shuchang Lyu, Xu Wang, and Qi Zhao. 2021. TPH-YOLOv5: Improved YOLOv5 based on transformer prediction head for object detection on drone-captured scenarios. In IEEE/CVF International Conference on Computer Vision Workshops (ICCVW’21). IEEE, 2778–2788. DOI:

Cited By

View all
  • (2024)AI in the Sky: Developing Real-Time UAV Recognition Systems to Enhance Military SecurityData and Metadata10.56294/dm2024.4173Online publication date: 29-Sep-2024
  • (2024)Lightweight UAV Small Target Detection and Perception Based on Improved YOLOv8-EDrones10.3390/drones81106818:11(681)Online publication date: 19-Nov-2024
  • (2024)Radar-Based Autonomous Identification of Propellers Type for Malicious Drone Detection2024 IEEE Sensors Applications Symposium (SAS)10.1109/SAS60918.2024.10636396(1-6)Online publication date: 23-Jul-2024

Index Terms

  1. RANGO: A Novel Deep Learning Approach to Detect Drones Disguising from Video Surveillance Systems

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Transactions on Intelligent Systems and Technology
      ACM Transactions on Intelligent Systems and Technology  Volume 15, Issue 2
      April 2024
      481 pages
      EISSN:2157-6912
      DOI:10.1145/3613561
      • Editor:
      • Huan Liu
      Issue’s Table of Contents

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 22 February 2024
      Online AM: 23 January 2024
      Accepted: 02 January 2024
      Revised: 06 November 2023
      Received: 13 December 2022
      Published in TIST Volume 15, Issue 2

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Drone detection
      2. deep learning
      3. YOLO
      4. background fusion

      Qualifiers

      • Research-article

      Funding Sources

      • Natural Science Foundation of Shandong Province

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)280
      • Downloads (Last 6 weeks)13
      Reflects downloads up to 25 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)AI in the Sky: Developing Real-Time UAV Recognition Systems to Enhance Military SecurityData and Metadata10.56294/dm2024.4173Online publication date: 29-Sep-2024
      • (2024)Lightweight UAV Small Target Detection and Perception Based on Improved YOLOv8-EDrones10.3390/drones81106818:11(681)Online publication date: 19-Nov-2024
      • (2024)Radar-Based Autonomous Identification of Propellers Type for Malicious Drone Detection2024 IEEE Sensors Applications Symposium (SAS)10.1109/SAS60918.2024.10636396(1-6)Online publication date: 23-Jul-2024

      View Options

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Full Text

      View this article in Full Text.

      Full Text

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media