ABSTRACT

Today Object Detection and classifying objects inside a single frame that contains
many objects is a time-consuming task. The accuracy rate has grown dramatically
as a result of the deep learning technique. Despite new flight control laws,
Unmanned Aerial Vehicles (UAVs) continue to grow in popularity for civilian and
military uses, as well as personal use. This growing interest has accelerated the
development of effective collision avoidance technologies. Such technologies are
crucial for UAV operation, particularly in congested skies. Due to the cost and
weight constraints of UAV payloads, camera-based solutions have become the de
facto standard for collision avoidance navigation systems. This requires multitarget
detection techniques from video that can be effectively run on board.
A drone is a quadcopter with on board sensors. This drone can be controlled using
wi-fi and laptop, using a programming language called Python and a Python library
called as drone kit. This paper will discuss a way for tracking a specific object
known as object detection tracking methods, which may be used to track any
arbitrary object chosen by the user, the camera of drone is used to take video
frames along with all the sensor's information such as ultrasonic sensors, GPS etc.
A train model will identify an object first and determines the direction at which the
drone should fly so that it keeps following person.

KEYWORDS - UAV, drone, Single shot detector, Mobile-Net, OpenCV, etc.

5
 TABLE OF CONTENTS
CHAPTER TITLE PAGE
NO NO

ABSTRACT 5
LIST OF FIGURES 8
1 INTRODUCTION
1.1 OUTLINE OF THE PROJECT 9
1.2 LITERATURE REVIEW 10
1.3 OBJECTIVE OF THE PROJECT 12
2 AIM AND SCOPE OF THE PRESENT
INVESTIGATION
2.1 AIM AND SCOPE OF THE PROJECT 13
2.2 HARDWARE REQUIREMENTS 13
2.3 SOFTWARE REQUIREMENTS 13
3 EXPERIMENTAL OR MATERIALS AND METHODS,
ALGORITHMS USED
3.1 SYSTEM DESIGN 15
3.1.1 EXISTING SYSTEM 16
3.1.2 PROBLEM STATEMENT 16
3.1.3 PROPOSED SYSTEM 16

3.2 METHODS USED 17

3.2.1 TAKE OFF 17
3.2.2 SEARCHING FOR THE OBJECT TO 18
TRACK
3.2.3 FIND THE OBJECT AND PRINT ITS 18
BOUNDING
3.2.4 ADJUST YAW, VELOCITY AND ANGLE
TO TRACK
3.2.5 LAND 18

3.3 ALGORITHMS
3.3.1 Mobilenet SSD
3.3.2 Result

6
4 RESULTS AND DISCUSSION
4.1 Results 26
5 CONCLUSIONS 28

6 REFERENCES 29

7 APPENDIX
38
7.1 SOURCE CODE 32
7.2 SCREENSHOTS 47
7.3 PUBLICATION WITH PLAGIARISM REPORT

7
 LIST OF FIGURES

FIG NO TITLE PAGE NO

1.1 INTRODUCTION
9

1.3
A typical UAV analysis model
12

The overall procedure diagram

3.1 of the object detection and 15
person tracking

Mobilenet SSD
3.3.1 18

Overview of models
3.3.1 21

Module Result
3.3.2 24

Screen shots
7.2 45

8
 1. INTRODUCTION

1.1 OUTLINE OF THE PROJECT:

Recently Drones are becoming affordable, practical, and versatile, which is

resulting in a reality where Drones are widely available in the sky for commercial
and individual purposes. The UAV’s has become more advantageous in the
compact structure and low maintains and increased processing ability. In order to
accomplish robust tracking on drones, UAVs outfitted with high-resolution cameras
and running computer vision algorithms in real-time have recently received
considerable study focus Annotations of high quality on large datasets are
essential for the development of algorithms, therefore specific difficulties related to
object detection are addressed. and person tracking algorithms applied to UAV
applications involve the following. The video feed is acquired from a moving
camera mounted on the UAV, where we detect the person in the drone camera
and capture his activities for surveillance purpose, it can be used in logistics in
remote location as well as cinematography etc. To achieve this task, we’ll use the
AR Drone's built-in sensors and actuators, but due to the built-in processor's
limited computing power, we'll do our reasoning on an external computer (a
regular laptop). The usage of Image Classification with the Mobile Net model to
accomplish detection and classification is the subject of this paper. The entire
system works by taking a frame from the camera and then using the on-board
computer to detect an object. computer processes the image to detect the object
using a Mobile-net classifier. Once the object is detected, the on-board goal is to
predict the object's position in relation to the quadrotor and transmits the
appropriate information to the flight controller to ensure proper tracking.

9
1.2 LITERATURE REVIEW

[1] Multi-Inertial Sensing Data for Real-Time Object Detection and tracking on a
Drone To extract features from an image, this study employs the Oriented Fast
and Rotated BRIEF[ORB] algorithm, as well as the Euclidean equation GPS and
IMU to compute the relative position between the drone and the target.

[2] Target Tracking and Recognition Systems Using Unmanned Aerial Vehicles.
This paper uses YOLO algorithm with custom dataset and train the algorithm for
motion blurred images and low resolution.

[3] Multi-Target Detection and Tracking in Unmanned Aerial Vehicles with a Single
Camera (UAVs). The Lucas-Kanade method is used in this research to recognize
and track other fast-moving UAVs.

[4] Object Detection and Classification for Autonomous Drones. This paper aims to
implement object detection and classification with high accuracy using SSD
architecture combined with Mobile Net

[5] Agent Sharing Network with Multi-Drone based Single Object Tracking.This
paper uses Agent Sharing Network [ASNet] for multiple drones to track and
identify single object.

[6] Path Following with Quad Rotorcraft Switching Control: An Application This
paper focus on estimation of track and road using UAV and here they used visual
sensors to identify lane.

[7] Any flying drone can track and follow any object. The Drones may track an
arbitrary target selected by the user in the video stream coming from the drone's
front camera. The proportional-integral-derivative method is then used to direct the
drone based on the location of the monitored object (PID controller). They
employed a tracking-learning-detection technique in computer vision.

10
[8] Haar-like Features for Object Recognition and Tracking Application of Cascade
classifiers on a quad-rotor UAV. In this research, to develop a functioning
Unmanned Aerial Vehicle (UAV) capable of tracking an object, we used a Machine
Learning-like vision system called Haar-similar features classifier. On-board image
processing is handled by a single-board computer with a powerful processor.

[9] Object Detection in Real-Time (YOLO) approach, which is based on UAV, has
been retrained to swiftly and accurately detect and distinguish objects in UAV
photos.

[10] Drone Identification and Tracking Using Phase-Interferometric Doppler Radar

This work focuses on UAV detection and tracking using phase interferometric
doppler radar, where data is collected using a double channel doppler and multi
radar called Doppler-azimuth, which will handle the processing.
[11] Deep Learning-Based Object Detection for Quadcopter Drones. The evolution
of object detection in AI and deep learning-based drone cameras is discussed in
this study. the main aim of this paper is delivering medical aids to patients. Here
they used single shot detector and Mobile Net.

[12] With an embedded UAV, real-time visual object detection and tracking is
possible. This is true even for embedded devices. A powerful neural network-
based object tracking system could be deployed in real time. A modular
implementation suited for on-the-fly execution is described and evaluated (based
on the well-known Robot Operating System) SYSTEM ARCHITECTURE.

[13]. Deep Learning-Based Object Detection for Quadcopter Drones Widodo

Budiharto1, Alexander A S Gunawan2, Jarot S. Suroso3, Andry Chowanda1,
Aurello Patrik1, and Gaudi Utama1 1Computer Science Department, Bina
Nusantara University's School of Computer Science.

[14]. Convolutional Neural Networks with Time Domain Motion Features for Drone
Video Object Detection Yugui Zhang1 Liuqing Shen1 Xiaoyan Wang1 Hai-Miao
Hu1, 2 1 Beijing Key Laboratory of Digital Media, School of Computer Science and
Engineering 2 State Key Laboratory of Virtual Reality Technology and Systems

11
Beihang University, Beijing, China

12
1.3 OBJECTIVE OF THE PROJECT

This study was conducted based on several objectives which are:

a) To record and detect human activities for surveillance purpose,
b) It can be used in logistics in remote locations, and used in cinematography
and photography etc.
c) Through using general unmanned aerial vehicle or drone and computer
vision.

Figure 1. A typical UAV analysis model

13
 2.AIM AND SCOPE OF THE PRESENT INVESTIGATION

2.1 AIM AND SCOPE OF THE PROJECT

The Aim of the project is based on object detection and person tracking. In this
process we get the real time input through OpenCV. video is converted into
images then the image is processed in open cv method. When the object
detection takes place then the model will be able to identify human from objects,
if human is in the video then UAV keeps on following for certain limit.
The scope of the project is working with a large number of features as it can
capture video frames from long range that’s the biggest advantage that can be
used in surveillance purpose. Even, it has also the risk of overfly or any failure for
that purpose we involved kill switch it going to terminate UAV, we used raspberry
pi and APM 2.8 flight controller to perform all the necessary tasks.

2.2 HARDWARE REQUIREMENTS

 UAV: The main elements of our quadcopter includes Battery, APM
2.8 flight controller, GPS module, Raspberry-pi on board computer,
USB camera, Ultra sonic sensor.
 APM 2.8 flight controller: it has ATMEGA 2560 processor and having
3 axis gyro meters on board, accelerometer and barometer by using
this sensor on circuit board we can use this to control any UAV’s.
 We use external GPS for pin point accuracy to find coordinates.
 Raspberry-pi is on board computer which run our python scripts, get
frames from USB camera and send signals to flight controller.
 Ultra-sonic sensor: This sensor uses ultra-sonic sound to get
distance.

2.3 SOFTWARE REQUIREMENTS

 Host computer: The host computer will reserve all the input frames
from camera on raspberry pi and process each frame using OpenCV
python for object detection and classification. Once the object is
detected the Mobile Net classifier will determine the bounding boxes
around the object and thus we can identify those objects along with
labels.
 Mobile Net: There are 2 forms of blocks in MobileNetV2. One could
be a one-stride residual block. Another choice for shrinking could be
a block with a two stride. Both forms of blocks have 3 levels. Eleven
convolution victimization is the first layer. These points should be
reLU6ed. The second layer has a depth-wise convolution. Another
eleven convolutions are employed inside the 3rd layer, However,
there is no non-linearity at this time. Deep networks can only have
the capability of a learning algorithm on the non-zero region as a

14
 component of the output region if RELU is applied again, according
to the statement.
 Drone Kit: Drone kit is a python library. Developers can use drone
Kit-Python to make programmed that operate using an onboard
companion computer, and use a low-latency link to communicate
with the Arduino control board. Onboard apps can assist the
autopilot in performing computationally hard or time-sensitive tasks,
as well as contributing knowledge to the vehicle's action. Drone Kit
python can also be used by base station apps that interface with
vehicles over a higher latency RF-link.
 MAV-Link is used by the API to communicate with automobiles. By
offering application support to a connected vehicle's data, status, and
parameter information, it offers either task management or total
control on vehicle movement and operations.

Figure .2 System Architecture

15