Volume 9, Issue 4, April – 2024 International Journal of Innovative Science and Research Technology

ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/IJISRT24APR1530

Human Activity Recognition using

Machine Learning
Omkar Mandave1; Abhishek Phad2; Sameer Patekar3; Dr. Nandkishor Karlekar4
Mahatma Gandhi Mission’s College of Engineering and Technology, Navi Mumbai, Maharashtra

Abstract:- Working information and classification is one For instance, the MS-G3D network addresses some of
of the most important problems in computer science. these challenges by automatically learning features from
Recognizing and identifying actions or tasks performed skeleton sequences and incorporating 3D convolution and
by humans is the main goal of intelligent video systems. attention mechanisms [1]. To further enhance the
Human activity is used in many applications, from performance of action recognition systems, new methods like
human-machine interaction to surveillance, security and the 3D graph convolutional feature selection and dense pre-
healthcare. Despite continuous efforts, working estimation (3D-GSD) method have been proposed [1]. This
knowledge in a limitless field is still a difficult task and method leverages spatial and temporal attention mechanisms
faces many challenges. In this article, we focus on some of along with human prediction models to better capture local
the current research articles on various cognitive and global information of actions and analyse human poses
functions. This project includes three popular methods to comprehensively [1]. Meanwhile, in a separate line of
define projects: vision-based (using estimates), practical research, the use of Convolutional Neural Networks (CNNs)
devices, and smartphones. We will also discuss some for human behaviour recognition from different viewpoints
advantages and disadvantages of the above methods and has gained attention [2].
give a brief comparison of their accuracy. The results will
also show how the visualization method has become a CNNs are utilized for tasks such as object detection,
popular method for HAR research today. segmentation, and recognition in videos, aiming to identify
various classes of body movement [2]. In parallel, the
Keywords:- Artificial Intelligence (AI), Human Activity development of Human Activity Recognition (HAR) systems
Recognition (HAR), Computer Vision, Machine Learning. has been propelled by the demand for automation in various
fields including healthcare, security, entertainment, and
I. INTRODUCTION abnormal activity monitoring [3]. HAR systems rely on
computer vision-based technologies, particularly deep
Skeleton features are widely used in human action learning and machine learning, to recognize human actions or
recognition and human-computer interaction [1]. This abnormal behaviours [3]. These systems monitor human
technology involves detecting and tracking key points of a activities through visual monitoring or sensing technologies,
human skeleton from images or videos, typically utilizing categorizing actions into gestures, actions, interactions, and
depth cameras, sensors, and other equipment to capture group activities based on involved body parts and movements
movement trajectories and analyse them through computer [3]. The applications of HAR encompass diverse domains
vision and machine learning techniques. The applications of such as video processing, surveillance, education, and
skeleton behaviour recognition span across various domains healthcare [3].
such as games, virtual reality, healthcare, and security [1].
Similarly, action identification in videos remains a
However, recognizing actions from skeleton data poses crucial task in computer vision and artificial intelligence,
several challenges due to factors such as multiple characters with applications in intelligent environments, security
in videos, varying perspectives, and interactions between systems, and human-machine interfaces [4]. Deep learning
characters [1]. Early methods relied on hand-designed feature methods, including convolutional and recurrent neural
extraction and spatiotemporal modelling techniques, while networks, have been employed to address challenges such as
recent advancements have been made in deep learning-based focal point recognition, lighting conditions, motion
approaches, particularly focusing on skeleton key points or variations, and imbalanced datasets [4]. Proposed models
spatio-temporal feature analysis [1]. integrate object identification, skeleton articulation, and 3D
convolutional network approaches to enhance action
Deep learning methods have shown promise in recognition accuracy [4]. These models leverage deep
improving action recognition accuracy, but traditional learning architectures, including LSTM recurrent networks
approaches still face challenges in adapting to different and 3D CNNs, along with innovative techniques such as
scenarios and effectively handling pose changes and high feature extraction and object detection to classify activities in
motion complexity [1]. video sequences effectively [4].

IJISRT24APR1530 www.ijisrt.com 903

Volume 9, Issue 4, April – 2024 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/IJISRT24APR1530

 Human Activity Recognition System from Different Poses

with CNN:
Human Pose Detection: Utilizes HAAR Feature-based
Classifier for detecting human poses in videos or images.
Human Pose Segmentation: Segments human poses based on
the output of the HAAR feature-based classifier. Activity
Recognition: Uses Convolutional Neural Network (CNN) for
recognizing human activities from segmented human poses.
Human Pose Detection: Utilizes HAAR Feature-based
Classifier trained on grayscale images to detect human poses
accurately. Human Pose Segmentation: Segments human
poses based on the coordinates obtained from the HAAR
feature-based classifier and resizes the images to 64x64
resolution. Activity Recognition: Employs a CNN
architecture, specifically VGG19, for recognizing human
activities from the segmented human poses.

 Effectiveness of Pre-Trained CNN Networks for Detecting

Abnormal Activities in Online Exams:
Detection of online exam cheating using deep learning,
motion-based keyframe extraction, COVID-19 dataset
collection, pre-trained models (YOLOv5, Inception_
ResNet_v2, DenseNet121, Inception-V3), and CNN fine-
tuning for enhanced performance. Analysis of cheating
activities during online exams using deep learning models.

Collection and preprocessing of a dataset containing

videos of various cheating behaviors. Use of pre-trained and
Fig 1 The Overview of General HAR System this is Basic fine-tuned deep learning models for classification. Focus on
Building Blocks of Almost Every System model accuracy, loss, precision, recall, and F1-score for
evaluation.
 Problem Statement & Objectives
Human Activity Recognition (HAR) aims to develop  A Real-Time 3-Dimensional Object Detection Based
algorithms and systems capable of identifying and classifying Human Action Recognition Model:
various human activities based on data collected from sensors Data fusion technique to merge datasets. Utilization of
or video streams. In fields such as healthcare, sports 3D Convolutional Neural Network (3DCNN) with
performance analysis, and human-computer interaction, multiplicative Long Short-Term Memory (LSTM) for feature
accurate HAR systems can provide valuable insights into extraction. Object identification using finetuned YOLOv6.
individual behaviors, health monitoring, and enhancing user Skeleton articulation technique for human pose estimation.
experiences. However, challenges arise due to variations in Combination of four different modules into a single neural
sensor data quality, diverse activity patterns, and real-time network. Integration of data fusion, feature extraction, object
processing requirements. Thus, the need arises for robust and detection, and skeleton articulation techniques. Utilization of
efficient HAR techniques that can accurately classify human multiplicative LSTM to improve classification accuracy.
activities in real-time, considering the complexities of
dynamic environments and diverse user contexts.  Different Approaches to HAR:
A HAR system is necessary in order to accomplish the
II. LITERATURE REVIEW purpose of identifying human activity. For this reason,
sensor-based and vision-based activity recognition are the
 3D Graph Convolutional Feature Selection and Dense two most widely utilized methods. We are able to categorize
Pre-Estimation for Skeleton Action Recognition: them.
The proposed method utilizes a 3D graph convolutional
feature selection (3DSKNet) to adaptively learn important  Pose Based Approach:
features in skeleton sequences. Additionally, a DensePose Pose-based approaches focus on analysing the human
algorithm is introduced to detect complex key points of body's posture or pose to recognize activities. This approach
human body postures and optimize the accuracy and often involves using computer vision techniques to extract
interpretability of action recognition. 3DSKNet focuses on key joint positions or body configurations from images or
key skeletal parts, improving the accuracy and robustness of videos. One common technique in pose-based HAR is using
bone recognition by selecting important features adaptively. pose estimation algorithms, such as OpenPose or PoseNet, to
DensePose provides more detailed pose and shape detect and track key points on the human body. These
information, enhancing the accuracy of human motion algorithms can estimate the positions of joints like elbows,
analysis. shoulders, hips, and knees. Once the pose information is

IJISRT24APR1530 www.ijisrt.com 904

Volume 9, Issue 4, April – 2024 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/IJISRT24APR1530

obtained, machine learning or deep learning models can be The relationship between the growth of HAR and AI is
trained to classify different activities based on the detected symbiotic. As AI techniques evolve, they enable more
poses. Features derived from pose data, such as joint angles sophisticated and accurate methods for extracting meaningful
or body part velocities, can be used as input features for these information from raw sensor data, giving rise to improved
models. Pose-based approaches are particularly useful when HAR systems. In the past, HAR models relied on single
sensor data is not available or when activities can be reliably images or short image sequences. These advanced models,
inferred from visual cues, such as in surveillance or human- such as CNNs combined with LSTMs, have made HAR
computer interaction applications. designs more robust and adaptable.

 Sensor Based: Human activities are diverse and can encompass a wide
Sensor-based approaches involve using various types of range of motions and positions. HAR has diverse
sensors, such as accelerometers, gyroscopes, magnetometers, applications, including healthcare, surveillance, monitoring,
or inertial measurement units (IMUs), to capture motion or and assistance for the elderly and people with disabilities. It
physiological signals associated with human activities. These is imperative to note that while the field is making great
sensors can be embedded in wearable devices like strides, there are challenges that need to be addressed. These
smartwatches, fitness trackers, or smartphones, or they can challenges include the selection of appropriate sensors, data
be placed in the environment (e.g., on walls or floors) to collection, recognition performance, energy efficiency,
capture activity data. Data collected from sensors typically processing capacity, and system flexibility.
include time-series measurements of acceleration, rotation, or
other physical quantities. Machine learning algorithms, such Our goal in this project is to create an efficient and
as decision trees, support vector machines (SVMs), or deep effective human recognition system that can process video
neural networks, can then be trained on this data to recognize and image data to detect tasks performed. The system is
different activities. Sensor-based approaches are widely used versatile and can find applications in many areas, such as
in applications like activity tracking, fall detection, sports caring for and helping people with special needs.
performance analysis, and healthcare monitoring due to their
versatility and non-intrusiveness. III. METHODOLOGY

 Proposed System This application harnesses MediaPipe's human pose

Human Activity Recognition (HAR) is a burgeoning estimation pipeline, powered by machine learning models
field with myriad applications, driven by the proliferation of like convolutional neural networks (CNNs) or similar
devices equipped with sensors and cameras, and the rapid architectures. These models enable real-time detection and
advancements in Artificial Intelligence (AI). HAR involves estimation of key points on the human body, resulting in
the art of identifying and naming various human activities graphical representations of skeletal structures. These visuals
using AI algorithms on data collected from these devices. serve as a roadmap of a person's posture and movements,
The data from these sources, including smartphones, video offering valuable insights into their activities. At its core are
cameras, RFID, and Wi-Fi, provides a rich source of pose landmarks, individual coordinates corresponding to
information that can be harnessed for recognizing human specific joints in the body, pivotal for accurately depicting a
activities. person's pose and tracking their movements over time.
Furthermore, the connections between these key points,
known as pairs, contribute to a deeper understanding of
human motion dynamics.

Fig 2 Conventional Machine Learning Architecture for Human Activity Recognition

IJISRT24APR1530 www.ijisrt.com 905

Volume 9, Issue 4, April – 2024 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/IJISRT24APR1530

Fig 3 Basic Architecture of the CNN-LSTM Network

Another critical component of the HAR system is its

user interface (UI), acting as the primary point of interaction
between the user and the application. Leveraging the Tkinter
library, we have crafted an intuitive and visually appealing
UI that facilitates seamless navigation and interaction. The Fig 5 Selecting Video
interface prominently features labeled buttons for various
functionalities, neatly categorized under Camera and Video
options, empowering users to effortlessly select their
preferred mode for HAR.

 Working
The project boasts a Graphical User Interface (GUI)
crafted with Tkinter, facilitating human tracking while
leveraging Mediapipe's pose estimation model for exercise
supervision. Through Tkinter, the GUI provides users with a
variety of exercise tracking options, including fighting,
smoking, walking, reading, and playing. It offers the
flexibility of choosing between real-time camera recognition
or video input, with distinct buttons to toggle between the
two modes.

IV. RESULT

These are the results we achieved after developing the

application. The result shows the GUI, Human Activity Fig 6 Video Output
Recognition through real-time video feed and prerecorded
video.

Fig 4 Graphical user Interface Fig 7 Real-Time Camera Output

IJISRT24APR1530 www.ijisrt.com 906

Volume 9, Issue 4, April – 2024 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/IJISRT24APR1530

V. FUTURE SCOPE VI. CONCLUSION

The future holds a wealth of opportunities for further In an era dominated by the advancements in computer
enhancing and extending Human Activity Recognition vision, the development and implementation of Human
systems. Here are some promising areas of development and Activity Recognition (HAR) systems have emerged as a
exploration: compelling and indispensable technology. These systems has
proven a remarkably effective in addressing multitude of
 Multimodal Integration: real-world applications, from surveillance and monitoring to
Incorporating data from diverse sensor types, such as providing invaluable assistance to elderly and visually
audio, accelerometers, and more, can enable richer context impaired individuals. The project's CNN-LSTM-based HAR
and improved recognition accuracy. system represents a significant stride in this direction,
offering a versatile and accurate approach to recognize and
 Privacy and Ethics: categorize a wide array of human activities. This system's
Researching and implementing robust privacy measures adaptability to video streams and image data showcases its
and ethical guidelines is essential for deploying HAR potential in addressing the diverse needs of various industries
systems in sensitive or public environments. and domains.

 Real-World Deployment: REFERENCES

Scaling up the usage of HAR systems in practical
domains like smart homes, healthcare, and personalized [1]. J. Zhang, A. Yang, C. Miao, X. Li, R. Zhang and D.
assistance offers a wide array of benefits. N. H. Thanh, "3D Graph Convolutional Feature
Selection and Dense Pre-Estimation for Skeleton
 Edge Computing: Action Recognition," in IEEE Access, vol. 12, pp.
Investigating the feasibility of deploying HAR models 11733-11742, 2024, doi:
on edge devices can lead to more responsive real-time 10.1109/ACCESS.2024.3353622.
applications without heavy reliance on cloud resources. [2]. M. Atikuzzaman, T. R. Rahman, E. Wazed, M. P.
Hossain and M. Z. Islam, "Human Activity
 Incremental Learning: Recognition System from Different Poses with CNN,"
Developing techniques for incremental learning allows 2020 2nd International Conference on Sustainable
systems to adapt to new activities and environments over Technologies for Industry 4.0 (STI), Dhaka,
time without complete retraining. Bangladesh, 2020, pp. 1-5, doi:
10.1109/STI50764.2020.9350508.
 Environment Robustness: [3]. M. Ramzan, A. Abid, M. Bilal, K. M. Aamir, S. A.
Enhancing system robustness to challenging Memon and T. -S. Chung, "Effectiveness of Pre-
environments, including adverse weather and lighting Trained CNN Networks for Detecting Abnormal
conditions, is a crucial research direction. Activities in Online Exams," in IEEE Access, vol. 12,
pp. 21503-21519, 2024, doi:
 User Interaction: 10.1109/ACCESS.2024.3359689.
Exploring natural and intuitive user interaction [4]. S. Win and T. L. L. Thein, "Real-Time Human
methods, like gesture recognition or voice commands, can Motion Detection, Tracking and Activity Recognition
enhance the usability of HAR systems. with Skeletal Model," 2020 IEEE Conference on
Computer Applications(ICCA), Yangon, Myanmar,
In conclusion, the future of Human Activity 2020, pp. 1-5, doi:
Recognition is filled with exciting possibilities. The continual 10.1109/ICCA49400.2020.9022822.
development of these systems is set to unlock new levels of [5]. A. Gupta, K. Gupta, K. Gupta and K. Gupta, "A
accuracy, versatility, and applicability in various domains. Survey on Human Activity Recognition and
These advancements will contribute to the ongoing evolution Classification," 2020 International Conference on
of intelligent systems that can understand and respond to Communication and Signal Processing (ICCSP),
human actions across diverse scenarios, making a significant Chennai, India, 2020, pp. 0915-0919, doi:
impact on society and industries. 10.1109/ICCSP48568.2020.9182416.
[6]. https://www.mdpi.com/2078-2489/13/6/275

IJISRT24APR1530 www.ijisrt.com 907