Atiq Ur Rehman, Naqeeb Ullah, Naeem Shehzad &Syed Agha Hassnain Mohsan

Image Processing Based Detecting and Tracking for Security

System

Atiq Ur Rehman atiqkhantareen@gmail.com

Department of Electronic Engineering
BUITEMS,
Quetta, 87300 Pakistan

Naqeeb Ullah naqeeb.ullah@buitms.edu.pk

Department of Electronic Engineering
BUITEMS,
Quetta, 87300 Pakistan

Naeem Shehzad naeemshehzad@ciitlahore.edu.com

Department of Electrical Engineering,
COMSATS (Lahore) Campus,
Lahore 53400, Pakistan

Syed Agha Hassnain Mohsan hassnainagha@zju.edu.cn

Optical Communications Laboratory
Zhejiang University,
Zhoushan, 316000, China

Abstract

Security systems are getting more attention and importance. Numeral security arrangements
based on sensors and wireless communication are available in the form of mechanical and
electronic applications. These systems are used widely in banks and government offices.
However, these traditional systems lack qualities like they cannot examine, track the suspect and
generate the alarm simultaneously. Thus, it is difficult to expand these systems with the help of
the sensors because of the complexity of the algorithms used. Hence, viewing these points, this
paper provides a security system based on Image Processing, which does not only inform the
host side at the runtime through alarm but also detects and tracks the present situation by
locating the target. The Optical Flow technique of image processing is used to detect using
motion analysis of two consecutive frames from the imaging source. The block analysis
determines the tracking capabilities of the system by the property of Good Features to track. The
wireless link between the imaging sources is established through XAMPP. While the functionality
of the alarm is performed at the commencement of the detection process. The results relating to
the process of detecting, tracking and generating alarm simultaneously are encouraging and will
help to increase the efficiency of the security systems.

Keywords: Detection, Embedded Systems, Image Processing, Optical Flow Technique,

Tracking.

1. INTRODUCTION
The security of public gathering spaces, banks, and large events are crucial for the smooth
functioning of public and government activities at all level. People have a profound trust in any
provided areas of well-being just by keeping the core of security in their minds. Henceforth,
keeping the former points into assessment, the security arrangements are done accordingly in
order to ensure the safety and order of that specific section. Thus, these schemes of security vary
from section to section in order of priority and threat. Moreover, these systems are divided into
monitored security systems: connected with a security agency and non-monitored security

International Journal of Engineering (IJE), Volume (13) : Issue (1) : 2021 1

Atiq Ur Rehman, Naqeeb Ullah, Naeem Shehzad, Syed Agha Hassnain Mohsan

systems: without any connection with the security agency. Similarly, these, schemes of security
can be in the form of machinery, human force, or electronic applications.

Many governmental and non-governmental agencies and institutions are working to find the
ultimate solution for this cause. Moreover, these agencies and institutions have succeeded by far
most. But still, there are areas, which are breached and damaged despite their high alert security
systems. Because of this, the higher profile regions and departments have concerns about their
security, which needs to be protected and secured from any type of intruder or loss such as the
hospitals, plant protection, schools, governmental and non-governmental places, department
store, residential areas and parking. Henceforward, these traditional security systems rely on
motion sensors, IR sensors, and CCTV cameras to observe a suspect, individual and ongoing
activity and require the human workforce to track continuously the images [1] . The technologies
used in these security systems are of diverse behaviors intended for different jobs according to
their credibility and nature [2]. Generally, the failure of the security systems can be attributed to
my reasons such as human error, system glitch, criminal attack, system overload, and the
shortage of the power supply.

Usually, the CCTV cameras for crime preventions in the factories’ warehouses, and banks merely
record the situation and do not notify the host side at runtime [3][4]. This paper provides a
security system based on Image Processing helping in removing the human error, informing the
host side at the runtime through an alarm, detecting and tracking the present situation by locating
the target. Moreover, it also provides the advantage of wireless communication between the
imaging sources in cases of a wide area coverages. The working procedure of the system is as
that any intruder activity within the restricted place will be captured by the imaging source, based
on which the alarm will be generated and further commands of detecting and tracking will be
initiated. Likewise, large area coverage is possible in this said project as a second imaging
source is wirelessly connected to the main imaging source. Apart from this, a sequence of a
password is issued to the authorities helping them to roam in the restricted place without
recognizing them as intruders.

2. LITERATURE REVIEW
These days the tracking of objects, for example, individuals or vehicles, coming all through
specific spaces is constantly a significant issue in security and operation management. For
example, in a shopping center, following the number of clients may help the market arranging,
staff organization, and tracking the number of staff individuals may help in the workforce. In
addition, object tracking for spots, for example, banks, buildings, and streets may improve traffic
control. In general, the object tracking is possible by utilizing entryway counters, or infrared
sensor to recognize objects coming all through spaces. Conventionally object tracking methods
require a camera above or the entrance/exit way of a place to capture the images. However,
these approaches are expensive and require human operator observers who detect any
suspicious individual and therefore, challenging to obtain a prior knowledge about all objects in
real-time [5][6]. Therefore, a real-time tracking and detecting system is required to detect an
unauthorized person in a prohibited area and alarming the security system at the same time

A GSM based home security system includes control panel, door [7] , and windows sensors to
monitor both coming in and out from the house. These systems use wireless communication to
receive the SMS from a cell phone and observe the safety and ensure the security of the home
by an SMS [8]. A Zigbee based parameter monitoring and controlling system using
microcontrollers are used to track and secure the environment of industries [9][10]. However,
these systems have no capability to analyzes the suspect and generate a real-time response.
Moreover, it is very difficult to expand the security system by adding several sensors because of
the complexity of the algorithm of the system. Similarly, the present bank security system focuses
only on the security of lockers and has a limited level of a security system. The digital cameras
are used only for monitoring and require the manual approach to inform the security [11].
Therefore, image processing-based tracking and detecting system is required for security
purpose that alerts the security through a certain alarm in case of any uncertainty [12][13][14].

International Journal of Computer Science and Security (IJCSS), Volume (1) : Issue (1) : 2021 2
Atiq Ur Rehman, Naqeeb Ullah, Naeem Shehzad, Syed Agha Hassnain Mohsan

Likewise, the security of the border and coastal area is of huge importance and cannot be
controlled and observed by the gaze of the human eye. A security system based on the
combination of image processing and Support Vector machine is used to overdue this process
and detect the intrusion of the unauthorized ships [15].

3. SYSTEM CONFIGURATION
The proposed system configuration makes the system cost-effective, reliable, and flexible in
performance. The block diagram of the proposed research work is illustrated in Fig.1.

FIGURE 1: Block Diagram of the system consisting of two same modules with a wireless link as a source of
communication between them.

A webcam is the main component, which detects a change in its field of view (FOV) connected
with the PC for its working and display. A microcontroller is used to control the motors both in
terms of pan and tilt. Further by using the technique of motion control, provides the capability to
track the subject of motion along with sending of an alarm signal.

4. METHODOLOGY
The proposed detecting and tracking system makes use of image processing and its subclasses
such as the Optical Flow method, Canny algorithm and Block analysis, which helps in the
achievement of the results as detecting, tracking, and motion control. A flow chart of the proposed
system is shown in Fig.2.

International Journal of Computer Science and Security (IJCSS), Volume (1) : Issue (1) : 2021 3
Atiq Ur Rehman, Naqeeb Ullah, Naeem Shehzad, Syed Agha Hassnain Mohsan

FIGURE 2: Flow chart of the proposed system.

4.1 Optical Flow

Among the parts of the segmentation, a technique to extract information and perform the action
according to the requirement is the optical flow method. In this step of the segmentation two or
more consecutive frames are taken. Moreover, the pixels are tracked from frame to frame in
single mode or can be tracked in the cluster form as well [16]. Firstly, the images are converted
into greyscale to identify the change and track the object easily. The process on both of them is
performed as such that the new image is compared with the background image. However, this
process is also known as the Thresholding process because of applying the values to the
produced change in the background as described through Eq. (1).

International Journal of Computer Science and Security (IJCSS), Volume (1) : Issue (1) : 2021 4
Atiq Ur Rehman, Naqeeb Ullah, Naeem Shehzad, Syed Agha Hassnain Mohsan

(1)

Where Di - Change between two connective frames and Ti – Threshold value.

A filter is used to make the process quick and flexible to detection of the change. Moreover, the
accuracy of the detection depends upon the size of the subject, its background, connection and
distance during the detecting process. A field of view (FOV) of the camera can be easily detected
by comparing the two consecutive frames of video streaming to detect the change between them
in terms of the change in single or cluster of pixels.

4.2 Motion Control

The computer or the processing unit capable enough to respond as fast as possible especially as
the speed is a critical point. the following procedures are applied [17].

4.2.1 Block Analysis: This block analysis is the method that tracks the object based on the
Good Features to track the subject extracted from the OpenCV. Thus, various blocks are being
created during the tracking process. Now to select the block a condition is applied to select the
block with the highest value.

4.2.2 Finding Intensity Gradient: After smoothing the Image, it is again filtered to get the
derivatives of the image both in the vertical and horizontal directions most probably first
derivatives. Thus, the direction, as well as the gradient of the pixel, can be easily determined by
using the Eq. (2) and Eq. (3) respectively.

(2)

(3)

Where is the Horizontal direction and is the Vertical direction. The direction of the gradient
is always perpendicular to the edges and is rounded to one of four angles to represent the
horizontal and vertical directions along with the two diagonal directions.

4.2.3 Threshold Values: The intensity gradient value of each edge is comparing with the
maximum and minimum values of the threshold. If the value of the intensity gradient is more than
the maximum threshold value, it will be considered as a sure edge. While the edges having
values of intensity gradient between the minimum and maximum values of the threshold will be
considered as edges only if they are connected to the sure edges. Further those edges are
neglected which are not connected to the sure edge or having a value less than the maximum
threshold value as determined for the process [17]. After the selection of the specific block, its
center point will be determined based on which the motion will be controlled as the cameras will
keep the target in their center. As illustrated in Fig. 3.

International Journal of Computer Science and Security (IJCSS), Volume (1) : Issue (1) : 2021 5
Atiq Ur Rehman, Naqeeb Ullah, Naeem Shehzad, Syed Agha Hassnain Mohsan

FIGURE 3: Center of the block is designed, as shown by a red spot.

The center of the block is identified by a red spot in the center. Hence based on the motion of the
object, the motion of the cameras will be controlled as when to pan and tilt.

4.3 Motor Delays

However, the delay in the motion of the motors does wholly depends upon the movement of the
subject, the processing speed of the microcontroller, its delays, and time associated with its
responsively. The control structure of the motors is shown in Fig. 4.

Process

Delay

Input Interface Input Interface

Control Module

Response Time

FIGURE 4: Motor delay control for the purpose of the precise motion of the motors w.r.t camera.

By motor delay control the process can be made more responsive and quicker by decreasing the
size of the image, as a smaller number of pixels will be involved, with less memory transportation
between the controllers and imaging source during the process of live surveillance.

4.4 Wireless Communication

In this modern era, more preference is given to the systems that are compact, reliable, and
flexible. Thus, looking at the advantages of wireless communication, we decided to establish a
wireless link between the two modules of the project rather than the traditional way of

International Journal of Computer Science and Security (IJCSS), Volume (1) : Issue (1) : 2021 6
Atiq Ur Rehman, Naqeeb Ullah, Naeem Shehzad, Syed Agha Hassnain Mohsan

communication. Hence, for this purpose, we are using Wi-Fi as a source of wireless
communication between the two modules by using the same IP address on two different
computers through the XAMPP software [18]. The need for wireless communication is needed,
when the threat goes out of the boundary of camera 1 of module 1, the information of the threat
will be transferred wirelessly to the module 2. Hence the position of the camera 2 of module 2 will
track the threat onwards. Wireless communication between the camera’s covering a vast area
and without the need for extra wiring. Thus, benefitting the cut in the cost. Password for the
authorized personals to detect and track them in the secured area without generating the alarm.

5. RESULTS
This section of the paper presents the results obtained during successful running of the system
as given below.

5.1 Detection
To detect the face using the property of the Good Features to Track. The proposed system can
detect any type of the object within the installed area up to 12 feet approximately by one module.
While the time interval for the system to detect any unauthorized entry is about 1.5 seconds as
illustrated in Fig. 5.

FIGURE 5: Detection property of the system.

5.2 Tracking
Motion of the subject within the prohibited region is accomplished with pan and tilt the camera
with the help of servo motors. This function of the project works as whenever there is any subject
of the motion, it will be tracked until the last range of distance value is achieved by the camera
which is approximately 12 feet after onward the subject of motion is tracked by the camera of the
second module after communicating through the established wireless link between them. The
following observations were concluded which are mentioned with the help of graphs.

5.2.1 Distance: The distance from the object is inversely proportional to the detection and
tracking properties of the system as shown in Fig. 6.

International Journal of Computer Science and Security (IJCSS), Volume (1) : Issue (1) : 2021 7
Atiq Ur Rehman, Naqeeb Ullah, Naeem Shehzad, Syed Agha Hassnain Mohsan

FIGURE 6: Detection and tracking by camera vs. distance.

5.2.2 Light Intensity: Light intensity is among the main factors which will greatly affect the
working of the system both in terms of detection and tracking of the object through the camera.
After running the project in different light intensities, the following observations are achieved
shown in Fig. 7.

FIGURE 7: Detection and tracking vs. light intensity.

5.2.3 Speed: The results achieved as illustrated in Fig. 8. with the help of the human motion and
a stopwatch representing detecting and tracking accuracy in percentage on the y-axis and speed
in meters per second on the x-axis respectively.

International Journal of Computer Science and Security (IJCSS), Volume (1) : Issue (1) : 2021 8
Atiq Ur Rehman, Naqeeb Ullah, Naeem Shehzad, Syed Agha Hassnain Mohsan

FIGURE 8: Detection and tracking by camera vs. speed,

Wireless communication between the camera’s covering a vast area and without the need for
extra wiring. Thus, making the system cost-effective. Password for the authorized personals to
detect and track them in the secured area without generating the alarm. The proposed method is
designed to track and detect a single object at a time. For multiple artifacts, however, a
complicated algorithm will be required. The proposed system can be further improved using
infrared lighting while utilizing the system at night time. In the case of blurry progression of the
image sources, the advanced algorithms of the image processing can be used i.e. Grasshopper
optimization [19].

6. CONCLUSION
In this paper, image processing-based detecting and tracking system is proposed for monitoring
public gathering venues like railway stations, banks, airports, and stadiums. The proposed
system uses the Optical flow method with the Canny algorithm along with the property of Good
Features with pan and tilt of the camera to track and detect any type of the moving object.
Moreover, an alarm will be generated automatically, whenever there is an unauthorized entry into
the entrance prohibited region. Moreover, in this paper, it has observed that distance between the
object and tacking system is inversely proportional to the detection and tracking properties of the
system. However light intensity of the light is directly proportional to detection and tracking
accuracy. Though, the speed of the moving object is also inversely proportional to the accuracy of
the detecting and tracking system. Similarly, the working of the system is limited in harsh
environmental conditions i.e. fog, rain and dust if used outside. Likewise, the properties of the
model will be restricted at night corresponding to the property of the light as discussed.

In the future, the accuracy and performance can be enhanced by implementing artificial
intelligence and advanced moving object detection methods. Apart from these, the system in the
future would enable detection and tracking of multiple subjects at once, which is limited to one for
the time being. Moreover, the system can be made into multiple modules consisting of several
imaging sources, which will help in vast area coverage. Similarly, the accuracy of the system
regarding detecting and tracking can be increased further with the installation of the white
background and higher luminosity. The artificial intelligence will further help in the future in a
comparison between human beings, animals, and materials based on their specific shapes,
colors, and sizes. The system would therefore be a significant advancement in the field of
security systems to protect areas of banks, airports, car parks, and factories. In addition, the
system's password-protected property would allow the authorized staff to roam freely without
creating an alarm. While in roaming, the properties such as detection and tracking will be intact of
the authorized staff within the area.

International Journal of Computer Science and Security (IJCSS), Volume (1) : Issue (1) : 2021 9
Atiq Ur Rehman, Naqeeb Ullah, Naeem Shehzad, Syed Agha Hassnain Mohsan

7. REFERENCES
[1] K. Salunkhe, P. Rajaram, S. Raut, and S. Kurle. “Real-time Activity Detection & Recognition
in Video.” International Journal of Future Generation Communication and Networking, Vol.
13, pp. 76- 83, May. 2020.

[2] J. Wolfendale. “Terrorism, security, and the threat of counterterrorism.” Studies in Conflict &
Terrorism, vol. 30, pp. 75–92, Dec 2006.

[3] E. L. Piza, B. C. Welsh, D. P. Farrington, and A. L. Thomas. “CCTV surveillance for crime
prevention: A 40-year systematic review with meta-analysis.” Criminol. Public Policy, vol. 18,
no. 1, pp. 135–159, 2019.

[4] G. Alexandrie. “Surveillance cameras and crime: a review of randomized and natural
experiments.”Journal of Scandinavian Studies in Criminology and Crime Prevention, vol. 18,
no. 2, pp. 210–222, 2017.

[5] N. N. Swathi, K. Alekhya, D. Pavithra and D. Kalyan Babu. "Implementation of Vision Based
Intelligent Home Security System Using ARM7."International Research Journal of
Engineering and Technology, vol. 4, no. 02, pp. 2395-0056, 2017.

[6] M. Dinesh and K. Sudhaman. “Real time intelligent image processing system with high
speed secured Internet of Things: Image processor with IOT.” International Conference On
Information Communciation And Embedded System (ICICE2016), Chennai, 2016, pp. 1-5.

[7] A. B. Dorothy, S. B. R. Kumar, and J. J. Sharmila. “IoT Based Home Security through Digital
Image Processing Algorithms.” 2017 World Congress on Computing and Communication
Technologies (WCCCT), Tiruchirappalli, 2017, pp. 20-23.

[8] Morbale, J. S., A. Kumari, H. Mishra, K. Sachan. "GSM Based Home Automation, Safety
and Security System Using Android Mobile Phone," Internation Journal of Current Research,
vol. 8, no. 03, pp. 28239-28243, 2016.

[9] N. Ullah, M. S. Akram, H. U. Buzdar, A. U. Rehman, M. A. Khan, and S. Khan. “ZigBee-

based Parameter Monitoring and Controlling Scheme for Multiple DC Motors,” Indian Journal
of Science and Technology, vol. 13, no. 6, pp. 725–734, 2020.

[10] H. Yetis and M. Karakose. "Image processing based anomaly detection approach for
synchronous movements in cyber-physical systems." 2018 23rd International Scientific-
Professional Conference on Information Technology (IT), Zabljak, 2018, pp. 1-4

[11] M. T. Awad, S. M. Aldaw, S. M. Aldaw and B. A. r. Osman. "Video Security System for
Intrusion Detection," 2019 International Conference on Computer, Control, Electrical, and
Electronics Engineering (ICCCEEE), Khartoum, Sudan, 2019, pp. 01-04.

[12] R. Socha and B. Kogut. "Urban Video Surveillance as a Tool to Improve Security in Public
Spaces," Sustainability, vol. 12, pp. 1-12, 2020.

[13] K. N. Pentaiah and P. J. Ker. "Development of a Microcontroller-based Portable Surveillance

System with User Alert Notification." Journal of Environmental Science and Technology, vol.
10, pp. 80-87, 2017.

[14] S. Tanwar, P. Patel, K. Patel, S. Tyagi, N. Kumar, and M. S. Obaidat. “An advanced Internet
of Thing based Security Alert System for Smart Home.” 2017 International Conference on
Computer, Information and Telecommunication Systems (CITS), Dalian, 2017, pp. 25-29.

[15] S. Dugad, V. Puliyadi, H. Palod, N. Johnson, S. Rajput and S. Johnny."Ship intrusion

detection security system using image processing & SVM." 2017 International Conference
on Nascent Technologies in Engineering (ICNTE), Navi Mumbai, 2017, pp. 1-7,

International Journal of Computer Science and Security (IJCSS), Volume (1) : Issue (1) : 2021 10
Atiq Ur Rehman, Naqeeb Ullah, Naeem Shehzad, Syed Agha Hassnain Mohsan

[16] S. Blakeman and D. Mareschal. "A complementary learning systems approach to temporal
difference Learning." Neural Networks, vol. 112, pp. 218-230, 2020.

[17] S. Kyle, "Human and Group Activity Recognition from Video Sequences." PhD thesis,
University of York, Heslington, UK, 2016.

[18] K. V. Kai “Oswald” Seidler, “XAMPP https://www.apachefriends.org/index.html.” [Online].

Available: https://www.apachefriends.org/index.html, [Oct, 15,2020]

[19] A. Sezavar, H. Farsi and S. Mohamadzadeh. "A Modified Grasshopper Optimization

Algorithm Combined with CNN for Content Based Image Retrieval," International Journal of
Engineering, vol. 32, no. 7, pp. 924-930, 2019.

International Journal of Computer Science and Security (IJCSS), Volume (1) : Issue (1) : 2021 11