Drowsiness Detection Using Haar and CNN Algorithm

Submitted for partial fulfillment of the requirements

for the award of

BACHELOR OF TECHNOLOGY
in
COMPUTER SCIENCE & ENGINEERING

by
M.SAKETH - 19BQ1A05B7
K. SAITEJA - 19BQ1A0577
K. SUMASRI - 19BQ1A0575
K. SHRAVANI -
19BQ1A0587

Under the guidance of

M. NAGA SRI HARSHA

Assistant Professor

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING

(B. Tech Program is Accredited by NBA)
VASIREDDY VENKATADRI INSTITUTE OF TECHNOLOGY
Permanently Affiliated to JNTU Kakinada, Approved by AICTE
Accredited by NAAC with ‘A’ Grade, ISO 9001:2008 Certified
NAMBUR (V), PEDAKAKANI (M), GUNTUR – 522 508
Tel no: 0863-2118036, url: www.vvitguntur.com
April 2023

Page | 1
 DECLARATION

We, Mr. M. Saketh, Mr. K. Sai Teja, Mrs. K. Suma Sri, Mrs. K. Shravani, hereby declare that the

Project Report entitled “Drowsiness Detection Using Haar and CNN Algorithm” done by us under

the guidance of Mr. N. Naga Sri Harsha, Assistant Professor, CSE at Vasireddy Venkatadri Institute

of Technology is submitted for partial fulfillment of the requirements for the award of Bachelor of

Technology in Computer Science & Engineering. The results embodied in this report have not been

submitted to any other University for the award of any degree.

DATE :

PLACE :

SIGNATURE OF THE CANDIDATE (S)

Page | 2
 ACKNOWLEDGEMENT

We take this opportunity to express my deepest gratitude and appreciation to all those people
who made this project work easier with words of encouragement, motivation, discipline, and faith by
offering different places to look to expand my ideas and helped me towards the successful completion
of this project work.

First and foremost, we express my deep gratitude to Mr. Vasireddy Vidya Sagar, Chairman,
Vasireddy Venkatadri Institute of Technology for providing necessary facilities throughout the
B.Tech programme.

We express my sincere thanks to Dr. Y. Mallikarjuna Reddy, Principal, Vasireddy

Venkatadri Institute of Technology for his constant support and cooperation throughout the B.Tech
programme.

We express my sincere gratitude to Dr. V. Ramachandran, Professor & HOD, Computer

Science & Engineering, Vasireddy Venkatadri Institute of Technology for his constant
encouragement, motivation and faith by offering different places to look to expand my ideas.

We would like to express my sincere gratefulness to our Guide Mr. M. Naga Sri Harsha,
Associate Professor, CSE for his insightful advice, motivating suggestions, invaluable guidance, help
and support in successful completion of this project.

We would like to express our sincere heartfelt thanks to our Project Coordinator Mr. P.
Krishna Prasad, Associate Professor, CSE for his valuable advices, motivating suggestions, moral
support, help and coordination among us in successful completion of this project.

We would like to take this opportunity to express my thanks to the Teaching and Non-
Teaching Staff in the Department of Computer Science & Engineering, VVIT for their invaluable
help and support.

Name (s) of Students

M.SAKETH(19BQ1A05B7)
K.SAI TEJA(19BQ1A0577)
K.SUMASRI(19BQ1A0575)

Page | 3
 K.SHRAVANI(19BQ1A0587)

Page | 4
 CERTIFICATE

This is to certify that this Socially Relevant Project Report is the bonafide work of M.
SAKETH, K.SAITEJA, K.SUMASRI, K.SHRAVANI bearing Reg.Nos 19BQ1A05B7,
19BQ1A0577, 19BQ1A0575, 19BQ1A0587 who had carried out the project entitled
“Drowsiness Detection Using Haar and CNN Algorithm” under our supervision.

Project Guide Head of the Department

M. Naga Sri Harsha Dr. V. Rama Chandran

Submitted for Viva voce Examination held on

Internal Examiner External Examiner

Page | 5
 TABLE OF CONTENTS

CH no. TITLE PAGE

NO

Contents i
List of Figures ii
Nomenclature iii
Abstract iv
1. INTRODUCTION
1.1. What is Drowsiness Detection?
1.2. What is Haar cascade classifier?
1.3. Haar Algorithm
1.4. CNN Algorithm
1.5. Machine Learning
1.6. Aim
1.7. Process
1.8. Features
1.9. Existing system
1.10.

1. Aim & Scope

2.1 Existing System
2.2 Proposed System
2.3 Feasibility Study
2.3.1 Technical Feasibility
2.3.2 Operational Feasibility
2.3.3 Economic Feasibility
2. Concepts & Methods
3.1 Problem Description
3.2 Proposed Solution
3.3 System Analysis Methods
3.3.1 Use case diagram

Page | 6
 3.3.2 Activity Diagram
3.4 System Requirements
3.5 System Design
3.5.1 Class Diagram
3.5.2 Sequence Diagram
3.6 E – R Diagrams
3. Implementation
4.1 Tools used
4.2 Pseudo Code/Algorithms
4.3 Deployment Diagram
4.4 Screenshots

4. Testing
5.1 Test cases
5. Summary (or) Conclusion
BIBLIOGRAPHY

ABSTRACT

The objective of this project is to create a drowsiness detection system that can recognize
when someone's eyes are closed for a brief period of time. When sleepiness is detected, this
system will give the user a warning. When someone is falling asleep, an alarm buzzes to
wake them up. Making the model platform independent, computationally efficient, and
affordable for the low-end spec platform is the main goal of this project. Furthermore, to
boost the detection's face-sensing accuracy, a mixture of two improved algorithms is applied.
The existing system occasionally generates false positive results, which results in erroneous
drowsiness detections. These systems might not function properly in various lighting
scenarios or with different facial expressions. The proposed system is made with the
intention of reducing accident rates and advancing technology in order to reduce the number
of deaths and injuries brought on by traffic accidents.

Page | 7
 1. INTRODUCTION

One of the most prevalent causes of accidents is driver sleepiness and weariness. Each year,
the deaths of individuals in such incidents rises across the world. The intent of this research is
to reduce the frequency of accidents caused by driver sleepiness and exhaustion. As a result,
transportation safety will improve. Driver drowsiness detection is a technology that can help
avoid accidents and save drivers' lives when they become drowsy. This research use
computer vision to identify driver drowsiness. This research focuses on creating an efficient
and cost-effective drowsiness detection system. The approach required in the current
circumstance identifies tiredness using geometric aspects of the eyes and lips. This research
aims to accomplish the same goal by constructing a sleepiness detection system to monitor
and avoid a negative consequence from tiredness neglect.

Page | 8
 2. AIM AND SCOPE

2.1 Existing System:

There are several existing methods for drowsiness detection, including:

 Electroencephalography (EEG) based methods: This method measures the electrical

activity of the brain through electrodes attached to the scalp. EEG-based methods can
detect changes in brain waves associated with drowsiness. However, this method is
expensive and requires specialized equipment and expertise.

 Eye-tracking methods: These methods use cameras to track eye movements and can
detect changes in eye behavior associated with drowsiness. However, this method is
limited to detecting only visual drowsiness, and may not detect other forms of
drowsiness.

 Facial recognition methods: These methods use cameras to detect changes in facial
expressions and movements associated with drowsiness. However, this method may
be limited by changes in lighting and facial occlusion.

Page | 9
  The existing system is a real time system. It uses image processing for eye and face
detection. HAAR based cascade classifier is used for face detection. An algorithm to
track objects is used to track the eyes continuously. In order to identify the drowsy
state of the driver, the PERCLOS algorithm issued.

Limitations:

1. False Positives: Haar-based classifiers can sometimes produce false positive results,
leading to false detections of drowsiness.
2. Limited Robustness: These systems may not work well under different lighting
conditions or with various face orientations.
3. Reliance on Haar Features: The system's accuracy is directly tied to the Haar features
used in the classifier, and their ability to accurately detect drowsiness may vary.
4. Inability to detect Cognitive Fatigue: Haar-based classifiers primarily focus on
detecting physical signs of drowsiness, such as eye closure, and may not be effective
in detecting cognitive fatigue.

2.2 Proposed System:

Drowsiness detection using Haar and CNN algorithm involves detecting whether a person's
eyes are closed or not, which is an indication of drowsiness. The Haar algorithm is used to
detect the face and eyes in an image or video stream, while the CNN algorithm is used to
classify the state of the eyes as open or closed. The Haar algorithm uses a series of
rectangular filters to detect features in an image. The algorithm detects the face in an image
by searching for rectangular patterns of pixel intensities that match a pre-defined set of Haar
features. Once the face is detected, the algorithm can then locate the eyes using a similar
approach.
Once the eyes are located, a CNN algorithm can be trained to classify the state of the eyes as
open or closed. The CNN algorithm takes the eye image as input and outputs a probability
score indicating the likelihood that the eyes are closed. The CNN is typically trained on a
large dataset of eye images labeled with their corresponding state (open or closed) and uses a
deep learning architecture to learn features that are relevant for drowsiness detection.

Page | 10
The combination of the Haar and CNN algorithms provides an effective approach for
drowsiness detection in real-time applications such as driver monitoring systems.

2.3. Feasibility Study:

The feasibility study of drowsiness detection using Haar and CNN algorithm involves
assessing whether the proposed solution is practical, technically feasible, and economically
viable.

2.3.1. Technical Feasibility:

The technical feasibility of drowsiness detection using Haar and CNN algorithm involves
evaluating whether the necessary hardware and software components are available and
whether the algorithm can be implemented effectively. The algorithm requires a camera,
computer or embedded system, and software for image processing, object detection, and deep
learning. The required components are generally available, and the algorithm has been
implemented effectively in many research studies.

2.3.2. Operational Feasibility:

The operational feasibility of drowsiness detection using Haar and CNN algorithm involves
evaluating whether the solution can be integrated with existing systems and whether it can be
operated effectively by the intended users. The algorithm can be integrated with existing
driver monitoring systems and can be operated by drivers or other users with minimal
training.

2.3.3. Economic Feasibility:

The economic feasibility of drowsiness detection using Haar and CNN algorithm involves
evaluating whether the benefits of the solution outweigh the costs. The cost of implementing
the algorithm depends on the hardware and software components required, the complexity of
the system, and the level of accuracy required. However, the benefits of the solution, such as

Page | 11
improved safety in transportation and reduced accidents, can outweigh the costs.

In summary, the feasibility study of drowsiness detection using Haar and CNN algorithm
suggests that the solution is technically feasible, economically viable, and operationally
feasible. However, further research and testing may be required to validate the performance
and effectiveness of the algorithm in different environments and conditions.

3. CONCEPTS AND METHODS

3.1. PROBLEM DESCRIPTION:

The operating of a motor vehicle when psychologically compromised owing to a lack of sleep
is known as sleep deprived driving. Driving when sleep deprived is a major cause of
vehicular accidents. When a person does not receive enough sleep, their ability to work
properly suffers. When their capacity to perform is impaired, they have a slower response
time and poor memory and judgement. Numerous studies have indicated that sleep
deprivation has the same effect on driving as alcohol inebriation.
According to research, 40% of highway accidents or near crashes in India are caused by
sleepy driving. Inebriation is a factor in more than 65% of all fatal single-car accidents.
Given these facts, it is critical that we create a driver safety system. To create such a system,
we must first assess the driver's state behind the wheel.

3.2. PROPOSED SOLUTION:

The proposed drowsiness detection system uses both Haar cascades and Convolutional
Neural Networks (CNNs) to identify indicators of drowsiness in a driver's facial
characteristics.

To begin, Haar cascades are used to recognize the face and eyes in a video stream acquired
by an in-car camera. CNNs are then utilized to categorize face characteristics such as the eyes
and mouth as awake or sleepy.

The Network is trained on a collection of photos of drivers with alert and sleepy facial
expressions, and it learns to spot drowsiness-indicating patterns in the features. When the

Page | 12
system senses tiredness, such as drooping eyelids or extended eye closure, it warns the driver
to discontinue driving.

3.3. System Analysis Methods:

3.3.1. Use Case diagram:

3.3.2. Activity Diagram:

Page | 13
3.4. SYSTEM REQUIREMENTS:
The basic system requirements for drowsiness detection are:

 Processor: A multi-core processor with a clock speed of at least 2.5 GHz or higher is
recommended for real-time processing of video data.

 Memory: At least 8 GB of RAM is recommended to handle the processing of large

video files.

 Graphics Processing Unit (GPU): A high-performance GPU with at least 2 GB of

dedicated memory is recommended to accelerate the processing of the CNN
algorithm.

 Operating System: A 64-bit operating system such as Windows 10, Linux, or macOS
is recommended.

 Software Libraries: The system will require software libraries for computer vision
such as OpenCV and TensorFlow to implement the Haar and CNN algorithms.

Page | 14
  Camera: A high-quality camera capable of capturing video at a frame rate of at least
30 fps is recommended.

 Storage: Sufficient storage capacity is required to store the recorded video data for
analysis.

3.5. System Design:

3.5.1. Class Diagram:

3.5.2. Sequence Diagram:

Page | 15
3.6.E-R Diagram:

4. IMPLEMENTATION

Page | 16
4.1. TOOLS USED:
Drowsiness detection is an important aspect of driver safety, and both Haar and CNN
algorithms can be used for this purpose. Here are some common tools that are used for
drowsiness detection using these algorithms:
 OpenCV: OpenCV is an open-source computer vision library that provides several
tools for image and video processing. It includes Haar Cascade Classifiers, which can
be used for object detection, including detecting eyes and faces in images and video
streams. OpenCV can be used with Python, C++, and Java programming languages.

 TensorFlow: TensorFlow is a popular open-source machine learning library

developed by Google. It provides tools for building and training deep learning
models, including CNNs. TensorFlow can be used with Python, C++, and Java
programming languages.

 Keras: Keras is a high-level deep learning library that provides a simple interface for
building and training neural networks. It can be used with TensorFlow and other
backend engines. Keras supports CNNs and can be used with Python programming
language.

 PyTorch: PyTorch is an open-source machine learning library developed by

Facebook. It provides tools for building and training deep learning models, including
CNNs. PyTorch can be used with Python programming language.

 Pygame: Pygame is a free and open-source Python library that is used for developing
2D video games and multimedia applications. It provides functionality for creating
graphical user interfaces, handling input events, playing sounds, and displaying
images and animations.

 Scikit-learn: Scikit-learn is a popular machine learning library that provides tools for
data analysis and modeling. It includes several classification algorithms, including
SVM and Random Forest, which can be used for drowsiness detection.

Page | 17
Prerequisites

1.Labeled Dataset: A labeled dataset of images or videos is required for training the CNN
algorithm. The dataset should contain a representative sample of images or videos of
individuals in different states of drowsiness.

2.Haar Cascade Classifiers: The Haar algorithm requires a set of pre-trained classifiers to
detect specific facial features such as eyes and mouth. These classifiers are typically available
in OpenCV library or can be trained from scratch.

3.Image Pre-processing: Pre-processing of images is necessary to reduce noise and improve

the accuracy of feature detection. The pre-processing steps can include image normalization,
resizing, and cropping.

4.Machine Learning Framework: A machine learning framework such as TensorFlow or

Keras is required for training the CNN algorithm. These frameworks provide a convenient
and efficient way to build, train and deploy machine learning models.

5.Hardware Requirements: Drowsiness detection using Haar algorithm and CNN algorithm
requires significant computational resources. This includes a powerful CPU or GPU, and
sufficient memory to process and store large datasets.

6.Real-Time Data Input: Drowsiness detection using Haar algorithm and CNN algorithm
requires real-time input data, such as from a camera or sensor. Therefore, a reliable data input
system should be in place.

Overall, drowsiness detection using Haar algorithm and CNN algorithm is a complex process
that requires significant expertise in computer vision, machine learning, and software
engineering. It is important to carefully consider the prerequisites before beginning to
develop a drowsiness detection system.

Page | 18
4.2. Pseudo Code/Algorithms:

Here is an algorithm for drowsiness detection using Haar and CNN :

 Import necessary libraries such as OpenCV, NumPy, and TensorFlow.
 Load the Haar Cascade classifier for detecting faces and eyes.
 Set the aspect ratio threshold and the frame counter threshold.
 Start the video capture.
 While the video capture is running, do the following:
a. Read the next frame from the video capture.
b. Convert the frame to grayscale.
c. Detect the face and eyes in the grayscale frame using the Haar classifier.
d. Calculate the aspect ratio of the eyes.
e. If the aspect ratio is below the threshold, increment the frame counter.
f. If the frame counter exceeds the threshold, sound an alarm or alert the driver.
Stop the video capture and release the resources.

Algorithm Study

A. Haar Algorithm

The Haar algorithm is a well-known image processing algorithm that is used for object
detection and facial recognition. The Haar algorithm can be used in the context of drowsiness
detection to identify characteristics on the face such as the eyes and mouth, which can
subsequently be used to assess whether the individual is drowsy.

The Haar algorithm works by analysing the contrast between different regions of an image. It
uses a set of pre-defined Haar features that are trained to recognize certain patterns in the
image. These features are typically rectangular and vary in size and orientation.
To detect sleepiness, the algorithm would first take a picture of the person's face with a
camera. Then it would use the Haar algorithm to identify the eyes and mouth in the image.
The algorithm would then analyse the position and movement of these features over time to
determine if the person is drowsy.

Page | 19
For example, if the algorithm detects that the person's eyes are closed or drooping, or that the
mouth is open for an extended period, this may indicate that the person is sleepy and needs
rest.
Overall, Haar edge and line detectors can be used to detect various facial features that can be
used to determine whether the driver is drowsy. By combining multiple features, it is possible
to create a more accurate drowsiness detection system.

Fig.1.Haar Feature detectors

The Haar method may be used to identify the edges of the driver's eyes. It is possible to tell if
the driver's eyes are closed or partially closed by measuring the aspect ratio of the observed
eyes, which is an indicative of drowsiness.

B. CNN Algorithm

Convolutional Neural Networks (CNNs) can be used in drowsiness detection systems to

automatically extract relevant features from images or videos of a person's face or eyes.

The CNN model extracts characteristics that are critical for discriminating between these two
states from a huge dataset of annotated images, where each image is classified as drowsy or
non-drowsy. These characteristics might include drooping eyelids, alterations in the structure
of the eyes, and the degree of eye closure.

Once trained, the CNN model may be used to categorize new images or video frames as
drowsy or non-drowsy. The CNN model analyses an image or video frame through a series of

Page | 20
convolutional layers that learn to detect patterns and characteristics in the input image. The
convolutional layer output is then input into fully connected layers, which learn to map these
characteristics to a final classification of drowsy or non-drowsy.
In summary, CNNs are used to detect drowsiness by automatically extracting key data from
images or video frames and using them to identify the state of drowsiness. This can aid in the
development of real-time systems that can inform drivers or operators if they are growing
drowsy and are at risk of an accident.

Role of Haar Algorithm:

1.Collect Data: Collect a dataset of images or videos of individuals in different states of

drowsiness.

2.Preprocess Data: Pre-process the data by converting the images or videos into a format that
can be processed by the Haar algorithm. This may involve normalization, resizing, and
cropping.

3.Face Detection: Apply the Haar algorithm to detect the faces in the images or videos.

4.Feature Detection: Apply a cascade of Haar classifiers to detect specific facial features such
as eyes and mouth.

5.Feature Tracking: Track the movement of the detected features over time to identify
changes that may indicate drowsiness.

6.Decision Making: Use the tracked features to determine if an individual is exhibiting signs
of drowsiness.

Haar-like features formula: The Haar-like features are used to detect specific features such as
the eyes and mouth of a driver. The formula for Haar-like features is:
f(x) = Σw(i) * p(i)
where f(x) is the value of the Haar-like feature at location x, w(i) is the weight of the ith
rectangle, and p(i) is the sum of pixel intensities within the ith rectangle.
Role of CNN Algorithm:

Page | 21
1.Collect Data: Collect a dataset of labeled images that have been preprocessed and feature
extracted using the Haar algorithm.

2.Split Data: Split the dataset into training and validation sets.

3.Build Model: Build a deep neural network using a machine learning framework such as
TensorFlow or Keras.

4.Train Model: Use the training dataset to train the model and the validation dataset to
validate it.

5.Model Evaluation: Evaluate the trained model's performance using measures like as
accuracy, precision, and recall.

6.Use Model: Use the trained model to classify new images as either drowsy or not drowsy in
real-time scenarios.

CNN is a deep learning algorithm that is commonly used for drowsiness detection. The
formula for CNN is:
y = f(W * x + b)
where y is the output of the CNN, W is the weight matrix, x is the input vector, b is the bias
vector, and f is the activation function such as ReLU, sigmoid, or softmax.

CNN Architecture:

Page | 22
We created the model using Keras and Convolutional Neural Networks (CNN). A
convolutional neural network is a sort of deep neural network that works very well for image
categorization. A CNN is made up of three layers: an input layer, an output layer, and a
hidden layer with numerous layers. Convolution is conducted on these layers with the help of
a filter that conducts 2D matrix multiplication on the layer and filter.

The CNN architecture is trained using a dataset of tagged video frames, with each frame
categorized as sleepy or not. The model is taught to recognize drowsiness-related patterns and
characteristics in video frames, such as drooping eyelids and sluggish head motions.

Once trained, the model may be used to identify further video frames in real-time, enabling
for the early identification of driver sleepiness and the provision of alerts or actions to avert
accidents.

Implementation of Drowsiness Detection:

Let us take a step-by-step look at how our algorithm works.

1. Using a camera, we will collect pictures as input. Thus, in order to access the webcam, we
created an infinite loop that captures each frame. Each frame is read and the picture is saved
in a frame variable.

2.Construct an Region of Interest based on the detection of a face in an image.

To find the face in the image, we must first convert it to grayscale because the method for
object identification only accepts grayscale images as input. Color information is not required

Page | 23
to detect the items. To detect faces, we will employ the Haar cascade classifier. The detection
is then carried out. It produces an array of detections with x,y coordinates and height, which
is the width of the object's border box. We can now iterate through the faces, drawing
boundary boxes for each one.

3.Using the ROI, detect the eyeballs and give them to the classifier.

The same method that is used to detect faces is utilized to detect eyes. We first configure the
cascade classifier for eyes in left eye and right eye. We must now extract only the eyes data
from the entire picture. This may be accomplished by extracting the eye's border box and
then using this code to extract the eye image from the frame.
Left eye just holds the eye's picture data. This information will be sent into our CNN
classifier, which will predict whether the eyes are open or closed. Similarly, the right eye will
be extracted into right eye.

4. The Classifier will determine if the eyes are open or closed. The CNN classifier is being
used to forecast the eye state. To input our image into the model, we must first execute
certain operations because the model need the proper dimensions to begin with.
To begin, we must transform the color image to grayscale. The image is then resized. We
then standardize our data to improve convergence. We prepared our model. Now we use our
model to forecast each eye. If the value of prediction variable is 1, the eyes are open; if the
value of prediction variable is 0, the eyes are closed.

5. Compute the score to see if the person is drowsy. We're drawing the result on the screen
with a function that displays the person's current state in real time.

If the score surpasses a certain limit, it shows that the person's eyelids have been closed for a
prolonged period of time. The alarm goes off at this point.

Overall, the Haar method is used to detect and track features, whilst the CNN algorithm is
employed to classify images. Drowsiness detection systems can identify indicators of
sleepiness in real-world circumstances by integrating the strengths of both methods.
The Pseudo code can be represented as shown below:

Page | 24
  The modules required are imported first as shown below:

import cv2
import os
from keras.models import load_model
import numpy as np
from pygame import mixer
import time

 We use the method provided by OpenCV, cv2.VideoCapture(0) to access the camera

and set the capture object (cap). cap.read() will read each frame and we store the
image in a frame variable.

 Detect Face in the Image and Create a Region of Interest (ROI).

face = cv2.CascadeClassifier('haar cascade files\

haarcascade_frontalface_alt.xml')
for (x,y,w,h) in faces:
cv2.rectangle(frame, (x,y), (x+w, y+h), (100,100,100), 1 )

 Detect the eyes from ROI and feed it to the classifier.

leye = cv2.CascadeClassifier('haar cascade files\

haarcascade_lefteye_2splits.xml') reye = cv2.CascadeClassifier('haar
cascade files\haarcascade_righteye_2splits.xml')
l_eye = frame[ y : y+h, x : x+w ]

l_eye only contains the image data of the eye. This will be fed into our CNN classifier
which will predict if eyes are open or closed. Similarly, we will be extracting the

Page | 25
 right eye into r_eye.

 Classifier will Categorize whether Eyes are Open or Closed.

model = load_model('models/cnncat2.h5')
path = os.getcwd()
cap = cv2.VideoCapture(0)
font = cv2.FONT_HERSHEY_COMPLEX_SMALL
count=0
score=0
thicc=2
rpred=[99]
lpred=[99]

for (x,y,w,h) in right_eye:

r_eye=frame[y:y+h,x:x+w]
count=count+1
r_eye = cv2.cvtColor(r_eye, cv2.COLOR_BGR2GRAY)
r_eye = cv2.resize(r_eye,(24,24))
r_eye= r_eye/255
r_eye= r_eye.reshape(24,24,-1)
r_eye = np.expand_dims(r_eye,axis=0)
rpred = model.predict_classes(r_eye)
 if(rpred[0]==1):
lbl='Open'
if(rpred[0]==0):
lbl='Closed'
break

 Calculate Score to Check whether Person is Drowsy.

Page | 26
 if(rpred[0]==0 and lpred[0]==0):
score=score+1
cv2.putText(frame, “Closed”, (10, height-20), font, 1,
(255,255,255),1,cv2.LINE_AA)
else:
score=score-1
cv2.putText(frame, “Open”, (10, height-20), font, 1,
(255,255,255),1,cv2.LINE_AA)

if(score<0):
score=0
cv2.putText(frame, ‘Score:'+str(score), (100,height-20), font, 1,
(255,255,255),1,cv2.LINE_AA)

if(score>5):
cv2.imwrite(os.path.Join(path,'image.jpg'),frame)
try:
sound.play()
except:
pass

if(thicc<16):
thicc= thicc+2
else:
thicc=thicc-2
if(thicc<2):
thicc=2
cv2.rectangle(frame,(0,0),(width,height),(0,0,255),thicc)
cv2.imshow('frame',frame)
 We

Page | 27
 are drawing the result on the screen using cv2.putText() function which will
display real time status of the person.

4.3. Deployment Diagram:

Process Flow:

Page | 28
4.5. Screenshots:

 Captured Image with Eyes Closed:

 Captured Image with Eyes Open:

Page | 29
  Captured Image with Eyes Open:

 Resultant Accuracy plot

Page | 30
5.Testing:

5.1. Test cases:

Eye position Mouth position Drowsiness Status

Open Open Partially detected

Closed Closed Detected
Open Closed Not detected
Closed Open Detected

6. SUMMARY/CONCLUSION:

We implemented a drowsiness detection system in this paper using the Haar algorithm and
the CNN algorithm, which is a combination of computer vision and machine learning
techniques. A dataset of images and videos of people in various states of drowsiness is
computed. The images or videos are transformed into a format that the algorithms can
understand. Normalization, scaling, and cropping are all involved. The Haar algorithm
detects certain face characteristics such as the mouth and eyes.This is performed through the
use of a cascade of Haar classifiers that have been trained to detect various patterns in image
data. The mobility of the identified features is monitored over time in order to detect
alterations that may suggest drowsiness. The labelled dataset is created by extracting the
characteristics and classifying them as sleepy or not drowsy. CNN Model Training is
accomplished by building a deep neural network with TensorFlow and Keras. The prepared
dataset is used to train the model, and a validation set is used to validate it. Metrics such as
accuracy, precision, and recall are used to assess the trained model's performance. In real-
time circumstances, such as when driving, the trained model is used to categorise new images
as drowsy or not drowsy. If the driver shows indications of drowsiness, audio warnings are
issued to warn them.

Page | 31
BIBLIOGRAPHY

References

[1] V B Navya Kiran, Raksha R , Anisoor Rahman , Varsha K N “Driver Drowsiness

Detection”, International Journal of Engineering Research & Technology (IJERT),
Special Issue - 2020.

[2] Mahek Jain, Bhavya Bhagerathi, Sowmyarani C N “Real-Time Driver Drowsiness

Detection using Computer Vision”, International Journal of Engineering and Advanced
Technology (IJEAT), Volume-11 Issue-1, October 2021.

[3] Erick Galarza, Fabricio D. Egas, Franklin M. Silva, Paola M. Velasco, Eddie D.
Galarza, “Real Time Driver Drowsiness Detection Based on Driver’s Face Image
Behavior Using a System of Human Computer Interaction”, Proceedings of the
International Conference on Information Technology & Systems (ICITS 2018) (pp.563-
572).

[4] Elena Magan, M. Paz Sesmero ,Juan Manuel Alonso-Weber and Araceli
Sanchis ,“Driver Drowsiness Detection by Applying Deep Learning Techniques to
Sequences of Images”, Appl. Sci. 2022.

[5] T. Vesselenyi, S. Moca , A. Rus , T. Mitran , B. Tataru, “Driver drowsiness detection

using ANN image processing”, CAR-2017 , IOP Conf. Series: Materials Science and
Engineering 252 (2017) 012097.

Page | 32