1

Air-to-Air UAV Detection Using Deep Learning

Models

Abstract—Objection detection with deep learning algo- commercial UAVs occupy low-altitude airspace for
rithms has seen a huge improvement but when it comes to the purpose of, for example, parcel delivery, how
UAV detection using deep learning algorithms, we require to ensure UAVs to detect other UAVs timely to
much better results. This paper is mainly focused on air-
navigate safely without colliding with each other is
to-air UAV detection for the detection of malicious UAVs in
airzone. We have used an existing dataset named DETFLY an important problem. Visual detection of malicious
and used 6,000 images to train the model and produced micro UAVs is a key technology for developing
better results than the existing ones. The images in this civilian UAV defense systems. Another application
dataset were captured by a micro UAV with a monocular is see-and-avoid among UAVs.
camera and thus it produced images in different views, in UAV detection can be classified into two ap-
different backgrounds, etc.After considering all these char-
plication scenarios. The first is ground-to-air sys-
acters in the images we have come with better results of
the four deep learning models than the previously existing tem.Cameras in the ground capture the images of
models. The dataset is available at:https://github.com/Jake- the flying UAVs. This is based on radar systems
WU/Det-Fly that are used with inbuilt cameras. This system is
very complex as they use an intricate combination
of sensors, radars and other complex devices to
I. INTRODUCTION detect by using UAV signals or by using the radars
Unmanned aerial vehicles (UAV) are a type of scan waves. Another major downside is setup and
aircraft that does not need a human pilot onboard maintenance are expensive and this requires a large
and is controlled by a remote using various sensors group of people. So for smaller zones, this system
and other components. Hence, it is also known as is unsuitable.
RPV (Remotely Piloted Vehicle). UAVs were first A cost-effective detection system is required to
used in defense systems, but nowadays in almost warn of the presence of UAVs. An efficient method
every field UAVs find their application.UAVs are to this is air-to-air detection. In this paper, we
extensively used in package delivery, surveillance, explore the feasibility of inexpensive detection of
etc. Thus, the consumer unmanned aerial vehicle UAVs based on the object detection method. The
market has seen significant growth in the last few camera in the UAVs acts as an eye and captures all
years. Insider Intelligence predicts total global ship- the images. These images are used in UAV detec-
ments to reach 2.4 million in 2023 – increasing at a tion. This paper is focused on UAVs detection for
66.8% compound annual growth rate (CAGR) [6]. the special zones based on image processing where
At the same time, there is a serious threat to UAVs are used for monitoring that is air-to-air. In
one’s privacy, and security, so UAVs are restricted this process, the original image obtained is cropped,
in some areas. So to detect the illegally flying and the further process takes place in the segmented
UAVs, the Federal Aviation Administration (FAA) image and the main aim is to cover a large num-
brought a scheme to register all the UAVs, and thus ber of possible results. Applying the convolutional
they can manage all the pieces of information of neural networks leads to precise results.So, here we
the registered UAVs.So there is major demand to have used an existing dataset DETFLY and used
detect or find the presence of the UAV intruder in 6,000 images to train the model which is obtained
the restricted areas. The idea is that if an illegal by micro UAVs to detect the target UAVs.Since
UAV enters a restricted area, the detection technique this is air-to-air, the images obtained to detect the
should warn people promptly. Hence, detecting target UAVs vary with different angles, different
UAVs in restricted areas or detecting the illegal backgrounds such as mountains, fields, lakes, clear
use of UAVs plays an important role in expanding skies, etc., and in different scenarios such as rain,
the UAVs market. In particular, as more and more lightning conditions, strong light, blur, etc. So this
 2

sets a big challenge for us to detect our target on object detection, in[16] they proposed a single
UAVs but with our structured deep learning models, shot-based detector named RefineDet++ with better
we have trained a model which produces better accuracies and comparable efficiencies. To detect
accuracy than the existing trained models. In this UAVs Haar wavelet-based AdaBoost is used in
paper, we have trained four models and produced [5].In [4] by using the geometric and other features
more accuracy than the previously existing models. of the segmented image it detects the moving target
YOLO V5, YOLOR, YOLOX, SCALED YOLO V4 by the discriminant function which is derived from
were the four models trained and produced better the Bayesian theorem.
results. So, from the obtained results we infer that
YOLO V5 AND SCALED YOLO V4 produced the B. Deep-learning approaches
most convincing accuracies. There are so many ways discovered so far for
the detection of Unmanned aerial vehicles(UAVs).
II. RELATED WORK Objection detection using a deep learning approach
In this section let’s see how the existing papers has seen a large progress but using this approach
worked on detecting UAVs using micro UAVs and for the detection of UAVs needs more improve-
monocular cameras. ment and progress. Few previously done works
were detecting the moving objects or targets using
A. Conventional Approaches motion compensation in [10] by convolutional neu-
ral networks[CNN]. This approach needed a high
The other techniques used in detecting the UAVs precision scale of moving camera.In [1] and [11]
were getting the target UAVs features by extracting detection of UAV s is obtained by using deep learn-
its characteristics for detecting target location by ing algorithms like YOLOV2,but the performance
using discriminant classifier and using generative of different representations of these algorithms is
classifier for detecting the target from the moving not evaluated.
objects in the images.
In [9] they proposed a new dataset for both C. Existing datasets for UAV detection
UAV and aircraft and used a regression-based ap-
proach which provided an efficient result on the There were so many datasets proposed till date
classification of images where they used a single to train the deep learning algorithms for detecting
moving camera for detecting Unmanned aerial ve- UAVs.In[8] the dataset consist of 20 videos and
hicles (UAVs) and aircraft.For detecting very small about 4000 gray frames of 752 x 480.The target
objects, a neural model called Recurrent Correla- UAV is detected by using a camera fixed on an-
tional Network was used in [15] where both the other UAV.The work in [7] the data size can be
detection and tracking were used in the multi- improved but more variation cannot be produced
frame representation. This work produced better by using augmentations.In the work [14] a dataset
classification between birds and Unmanned aerial named MIDGARD was proposed with different
vehicles (UAVs) in their dataset. In [2] implemen- background and different environmental conditions
tation of Cascade-R CNN was done and gave a and produced a new method for annotations done
better result on single-model object detectors in automatically.In [3] the dataset consist of two sub
the COCO dataset and produced consistent results. datasets- with 30 videos of different drones with one
To measure the relative motion between UAVs in as a Public Domain dataset and other one taken in
different conditions a vision-aided flocking system USC campus whose background is spotted mostly
for Unmanned aerial vehicles(UAVs) which even with clear sky or clouds also for image augmen-
works in the GPS-denied unknown environments tation they attached the obtained drone datasets in
for searching was presented in the paper[12]. A different backgrounds and obtained larger data size.
vision-based sensing system for UAVs to detect the
R EFERENCES
leader-follower formation flight is done in[13] with-
out the intervention of any third-party system. The [1] Cemal Aker and Sinan Kalkan. Using deep networks for drone
detection. In 2017 14th IEEE International Conference on
convolutional neural network is of one stage and Advanced Video and Signal Based Surveillance (AVSS), pages
two-stage approach. Since this has many drawbacks 1–6. IEEE, 2017.
 3

[2] Zhaowei Cai and Nuno Vasconcelos. Cascade r-cnn: Delving

into high quality object detection. In Proceedings of the IEEE
conference on computer vision and pattern recognition, pages
6154–6162, 2018.
[3] Yueru Chen, Pranav Aggarwal, Jongmoo Choi, and C-C Jay
Kuo. A deep learning approach to drone monitoring. In 2017
Asia-Pacific Signal and Information Processing Association
Annual Summit and Conference (APSIPA ASC), pages 686–691.
IEEE, 2017.
[4] Feng Lin, Xiangxu Dong, Ben M Chen, Kai-Yew Lum, and
Tong H Lee. A robust real-time embedded vision system on
an unmanned rotorcraft for ground target following. IEEE
Transactions on Industrial Electronics, 59(2):1038–1049, 2011.
[5] Feng Lin, Kemao Peng, Xiangxu Dong, Shiyu Zhao, and
Ben M Chen. Vision-based formation for uavs. In 11th IEEE
International Conference on Control & Automation (ICCA),
pages 1375–1380. IEEE, 2014.
[6] Andrew Meola. Drone industry analysis: Market trends &
growth forecasts. Business Insider, 2017.
[7] Xingchao Peng, Baochen Sun, Karim Ali, and Kate Saenko.
Learning deep object detectors from 3d models. In Proceedings
of the IEEE international conference on computer vision, pages
1278–1286, 2015.
[8] Artem Rozantsev, Vincent Lepetit, and Pascal Fua. Flying
objects detection from a single moving camera. In Proceedings
of the IEEE Conference on Computer Vision and Pattern
Recognition, pages 4128–4136, 2015.
[9] Artem Rozantsev, Vincent Lepetit, and Pascal Fua. Detecting
flying objects using a single moving camera. IEEE transactions
on pattern analysis and machine intelligence, 39(5):879–892,
2016.
[10] Artem Rozantsev, Vincent Lepetit, and Pascal Fua. Detecting
flying objects using a single moving camera. IEEE transactions
on pattern analysis and machine intelligence, 39(5):879–892,
2016.
[11] Arne Schumann, Lars Sommer, Johannes Klatte, Tobias
Schuchert, and Jürgen Beyerer. Deep cross-domain flying object
classification for robust uav detection. In 2017 14th IEEE
International Conference on Advanced Video and Signal Based
Surveillance (AVSS), pages 1–6. IEEE, 2017.
[12] Yazhe Tang, Yuchao Hu, Jinqiang Cui, Fang Liao, Mingjie
Lao, Feng Lin, and Rodney SH Teo. Vision-aided multi-
uav autonomous flocking in gps-denied environment. IEEE
Transactions on industrial electronics, 66(1):616–626, 2018.
[13] Yazhe Tang, Yuchao Hu, Jinqiang Cui, Fang Liao, Mingjie
Lao, Feng Lin, and Rodney SH Teo. Vision-aided multi-
uav autonomous flocking in gps-denied environment. IEEE
Transactions on industrial electronics, 66(1):616–626, 2018.
[14] Viktor Walter, Matouš Vrba, and Martin Saska. On training
datasets for machine learning-based visual relative localization
of micro-scale uavs. In 2020 IEEE International Conference on
Robotics and Automation (ICRA), pages 10674–10680. IEEE,
2020.
[15] Ryota Yoshihashi, Tu Tuan Trinh, Rei Kawakami, Shaodi You,
Makoto Iida, and Takeshi Naemura. Differentiating objects by
motion: Joint detection and tracking of small flying objects.
arXiv preprint arXiv:1709.04666, 2017.
[16] Shifeng Zhang, Longyin Wen, Zhen Lei, and Stan Z Li.
Refinedet++: Single-shot refinement neural network for object
detection. IEEE Transactions on Circuits and Systems for Video
Technology, 31(2):674–687, 2020.