ISSN (O) 2393-8021, ISSN (P) 2394-1588

IARJSET
International Advanced Research Journal in Science, Engineering and Technology
Impact Factor 8.066Peer-reviewed & Refereed journalVol. 11, Issue 7, July 2024
DOI: 10.17148/IARJSET.2024.11739

HUMAN MOTION PATTERN DETECTION

Raksha HV1, Swetha CS2
Student, Department of MCA, Bangalore Institute of Technology, Karnataka, India1
Asst Professor, Department of MCA, Bangalore Institute of Technology, Karnataka, India2

Abstract: There is a growing need to have effective surveillance systems that can identify and flag alarming behaviour
instantly in the current era. In this study, we propose a new method of recognizing suspicious activities of video data
using deep learning algorithms. We employed the DCSASS Dataset which contained videos from thirteen categories of
suspicious activity, like abuse, arson, assault, robbery and so on. A mixed architecture involving both ResNet50 and I3D
was used because it could handle the temporal and spatial complexities that come with video data. The model is trained
to recognize subtle cues concerning suspicious behaviour as it exhibits remarkable training accuracy. By subjecting our
model to rigorous evaluation on a separate validation set, it shows encouraging results at about 85% accuracy. To improve
the performance of the model further, we consider various strategies such as data augmentation, fine-tuning of hyper
parameters as well as ensemble methods. We also try to make our models interpretable by employing techniques such as
class activation mapping for better understanding of decision making process.

Keywords: DCSASS Dataset, Deep learning algorithms, ResNet50, I3D.

I. INTRODUCTION

There has been a widespread growth in the use of surveillance systems in the past few years, as cases of insecurity and
threats to safety have increased in both public and private places. The traditional surveillance methods depend largely on
human attention, which is expensive, error prone and not agile enough to offer immediate results. Thus, there has been a
rise in interest towards automated surveillance systems with capabilities of self- detecting unusual occurrences. The
subfield of artificial intelligence known as deep learning that is inspired by human brain structure and function has become
an important tool for examining intricate data varieties such as images, audio and video. Computer scientists have
exploited this approach extensively while working on computer vision studies leading to high precision and fast process
time. Well, our project fits into this framework because our objective is to create a sophisticated monitoring system that
will identify suspicious activities using deep learning algorithms. Our solution must also overcome several challenges
posed by video data such as temporal dynamics, spatial relationships as well as semantic understanding.
Keywords: Automated surveillance systems, Artificial intelligence, Computer vision studies leading, Deep learning
algorithms, Intricate data

II. LITERATURE SURVEY

HUMAN MOTION PATTERN DETECTION has recently been considered to be one of the most promising techniques
for its usage in surveillance, health care, smart homes, and many others. 15 papers related to HAR and application of deep
learning techniques are surveyed in this paper along with their methodologies, contributions, and future scope.

1. Machine Learning and Deep Learning Models for Human Activity Recognition (Liu et al., 2018)

Liu et al. (2018) explore the use of a Bayesian classifier combined with convolutional neural networks (CNNs) for
abnormal activity detection. Their approach demonstrated the potential of integrating statistical models with deep learning
to enhance recognition accuracy in real-time applications (SpringerLink).

2. Random Forests for Human Activity Recognition (Xu et al., 2017)

Xu et al. (2017) applied random forests for HAR, highlighting the method's robustness in handling diverse activity
datasets. Their work emphasizes the efficiency of ensemble methods in classifying complex human activities accurately
(SpringerLink)

3. Kalman Filter for ATM Surveillance (Nandyal & Angadi, 2021)

Nandyal and Angadi (2021) utilized the Kalman filter and Kanade-Lucas-Tomasi (KLT) tracker for recognizing
suspicious activities in ATM surveillance. Their framework successfully reduced false positives, showcasing the
effectiveness of combining traditional tracking algorithms with machine learning techniques (SpringerLink).

IARJSET This work is licensed under a Creative Commons Attribution 4.0 International License 255
 ISSN (O) 2393-8021, ISSN (P) 2394-1588
IARJSET
International Advanced Research Journal in Science, Engineering and Technology
Impact Factor 8.066Peer-reviewed & Refereed journalVol. 11, Issue 7, July 2024
DOI: 10.17148/IARJSET.2024.11739
4. Semi-supervised Recurrent Convolutional Attention Model (Chen et al., 2019)
Chen et al. (2019) proposed a semi-supervised model known as the Recurrent Convolutional Attention Network, which
enhanced the performance of HAR by capturing temporal patterns and salient features in the data. This approach showed
better performance in the situations where there is little amount of labeled data (SpringerLink).

5. Hybrid Network for Human Activity Recognition (Lv et al., 2020)

Lv et al. (2020) proposed a hybrid network combining dense connections and weighted feature aggregation, which
enhanced the model's ability to capture intricate patterns in human activities. Their method achieved high accuracy in
multi-class activity recognition tasks (SpringerLink).

Keywords: Convolutional neural networks(CNN),Human Activity Recognition, Recurrent Convolutional Attention

Network, Multi-class activity recognition.

III. EXISTING SYSTEM

The majority of espionage systems in use today rely on antiquated techniques that are unable to accurately recognize and
categorize certain human behaviors. Manual monitoring requires a lot of work, takes a long time, and is susceptible to
human error. While some computerized processes now in use may employ simple motion detection algorithms, they are
not sophisticated enough to reliably identify complicated actions. The shortcomings of the current methods highlight the
need for a more comprehensive and perceptive method of identifying human activities.

IV. PROPOSED SYSTEM

The drawbacks of conventional surveillance techniques are addressed by the suggested system, which makes use of a
CNN (convolution neural network) architecture. By concentrating on particular actions, such hiding one's face,
brandishing a weapon, fighting, stealing, beating, and violent incidents, the system seeks to offer a more precise and
effective way to recognize possible security risks.

The goal of the proposed system is to accomplish actual time detection and categorization of specified actions in video
streams by integrating decompression approaches, transfer learning, and exploring architectures that can capture both
spatial and temporal data. The initiative aims to improve overall security measures in various situations by advancing
intelligent surveillance systems.

V. IMPLEMENTATIONS AND METHODOLOGY

The implementation phase of the HUMAN MOTION PATTERN DETECTION System involves the translation of the
system design into functional code. This phase includes setting up the development environment, writing and testing the
code, integrating different modules, and ensuring that the system meets the specified requirements. This chapter details
the implementation steps, including setting up the environment, developing each component, integrating the modules,
and conducting testing.

Development Environment Setup

To begin the implementation, it's crucial to set up the appropriate development environment. This involves installing
necessary software, libraries, and configuring the hardware.

System Architecture
The human motion pattern detection system's architecture consists of a number of interconnected parts that process video
input and produce activity predictions collectively. The principal elements consist of:

1. Input Module: Accepts video files or live video streams.

2. Preprocessing Module: Extracts frames, resizes, normalizes, and augments them.
3. Feature Extraction Module: Extracts pertinent characteristics from frames using convolution neural networks
(CNNs).
4. Model Training Module: Utilizing labelled datasets, the deep learning model is trained.
5. Inference Module: Uses the trained model to process pre-processed frames and forecast actions.
6. Output Module: Displays predicted activities with confidence scores and integrates with external systems if
needed.

IARJSET This work is licensed under a Creative Commons Attribution 4.0 International License 256
 ISSN (O) 2393-8021, ISSN (P) 2394-1588
IARJSET
International Advanced Research Journal in Science, Engineering and Technology
Impact Factor 8.066Peer-reviewed & Refereed journalVol. 11, Issue 7, July 2024
DOI: 10.17148/IARJSET.2024.11739

VI. CONCLUSIONS

In this project, we developed a suspicious activity recognition system utilizing deep learning algorithms, specifically
ResNet50, to classify 13 different activities in video footage. Our training and evaluation were conducted using the
DCSASS dataset, which includes a diverse range of activities captured in surveillance videos.

The experimental results were promising, with the model achieving an accuracy of 98.39% on the training data and
85.03% on the validation data. These metrics highlight the effectiveness of our approach in accurately detecting
suspicious activities from video streams.

VII. FUTURE ENHANCEMENTS

For future enhancements, improving duplicate question pair classification models can be achieved by incorporating
advanced natural language processing techniques like word embeddings or contextualized representations (e.g., BERT)
to capture richer semantic information. Employing deep learning architectures like CNNs or RNNs can enhance the
model's ability to capture complex patterns.

Utilizing ensemble methods with Random Forest and XGBoost can boost overall accuracy and precision. Additionally,
considering domain- specific features and transfer learning approaches can further improve performance on niche
datasets. The various advanced natural language processing (NLP) strategies can be implemented. First, employing state-
of-the-art models such as BERT, GPT-4, or RoBERTa can greatly improve the understanding of context and nuances in
the text. These models, especially when fine-tuned on datasets specifically annotated for human activity detection,
become highly effective at identifying activities within paragraphs. Additionally, using contextual embeddings, such as
those generated by transformer-based models, ensures that the model considers the entire paragraph, leading to more
accurate interpretations of described activities.

Semantic Role Labeling (SRL), which identifies the functions of various things in sentences, is another important strategy.
SRL improves the extraction of actions and their actors from paragraphs by identifying who carried out the action, what
the action was, and to whom it was done. This process is further aided by the incorporation of Named Entity Recognition
(NER) and entity linking, which locate and associate entities with a knowledge base. This adds context to the process and
enhances comprehension of the operations.

IARJSET This work is licensed under a Creative Commons Attribution 4.0 International License 257
 ISSN (O) 2393-8021, ISSN (P) 2394-1588
IARJSET
International Advanced Research Journal in Science, Engineering and Technology
Impact Factor 8.066Peer-reviewed & Refereed journalVol. 11, Issue 7, July 2024
DOI: 10.17148/IARJSET.2024.11739
REFERENCES

[1]. Weiyao Lin, Ming-Ting Sun, R. Poovandran and Zhengyou Zhang, "Human activity recognition for video
surveillance," 2008 IEEE International Symposium on Circuits and Systems (ISCAS), Seattle, WA, USA, 2008, pp.
2737-2740, doi: 10.1109/ISCAS.2008.4542023.
[2]. K. K. Htike, O. O. Khalifa, H. A. Mohd Ramli and M. A. M. Abushariah, "Human activity recognition for video
surveillance using sequences of postures," The Third International Conference on e-Technologies and Networks for
Development (ICeND2014), Beirut, Lebanon, 2014, pp. 79-82, doi: 10.1109/ICeND.2014.6991357.
[3]. U. M. Kamthe and C. G. Patil, "Suspicious Activity Recognition in Video Surveillance System," 2018 Fourth
International Conference on Computing Communication Control and Automation (ICCUBEA), Pune, India, 2018,
pp. 1- 6, doi: 10.1109/ICCUBEA.2018.8697408.
[4]. S. Loganathan, G. Kariyawasam and P. Sumathipala, "Suspicious Activity Detection in Surveillance Footage," 2019
International Conference on Electrical and Computing Technologies and Applications (ICECTA), Ras Al Khaimah,
United Arab Emirates, 2019, pp. 1-4, doi: 10.1109/ICECTA48151.2019.8959600.
[5]. F. G. Ibrahim Salem, R. Hassanpour, A. A. Ahmed and A. Douma, "Detection of Suspicious Activities of Human
from Surveillance Videos," 2021 IEEE 1st International Maghreb Meeting of the Conference on Sciences and
Techniques of Automatic Control and Computer Engineering MI-STA, Tripoli, Libya, 2021, pp. 794-801,
doi: 10.1109/MI- STA52233.2021.9464477.
[6]. C. V. Amrutha, C. Jyotsna and J. Amudha, "Deep Learning Approach for Suspicious Activity Detection from
Surveillance Video," 2020 2nd International Conference on Innovative Mechanisms for Industry Applications
(ICIMIA), Bangalore, India, 2020, pp. 335-339, doi: 10.1109/ICIMIA48430.2020.9074920.
[7]. D. KAVITHASUSPICIOUS ACTIVITY RECOGNITIONINVIDEOSURVEILLANCE SYSTEM link:
https://zkginternational.com/archive/volume8/SUS PICIOUS-ACTIVITY-RECOGNITION-IN- VIDEO-
SURVEILLANCE-SYSTEM.pdf
[8]. Suspicious Human Activity Recognition for Video Surveillance System link:
https://www.researchgate.net/publication/26403701 0_Suspicious_Human_Activity_Recognition_for_V
ideo_Surveillance_System
[9]. U. M. Kamthe and C. G. Patil, "Suspicious Activity Recognition in Video Surveillance System," 2018 Fourth
International Conference on Computing Communication Control and Automation (ICCUBEA), Pune, India, 2018,
pp. 1- 6, doi: 10.1109/ICCUBEA.2018.8697408.
[10]. Ahmad Salihu Suspicious Human Activity Recognition for Video Surveillance System
[11]. Saleha Saudagar SUSPICIOUS ACTIVITY DETECTION IN VIDEO SURVEILLANCE USING
CONVOLUTIONAL NEURAL NETWORK

IARJSET This work is licensed under a Creative Commons Attribution 4.0 International License 258