Project Report
Project Report
Project Report
Submitted by
We would like to thank our supervisor, Mr. Biswanath Dey, CSE, NIT Silchar, for
his invaluable direction, encouragement, and assistance during this project. His helpful
suggestions for this entire effort and cooperation are gratefully thanked, as they enabled
us to conduct extensive investigation and learn about many new subjects.
Our sincere gratitude to Mr. Biswanath Dey for his unwavering support and patience,
without which this project would not have been completed properly and on schedule.
This project would not have been feasible without him, from providing the data sets to
answering any doubts we had to helping us with ideas whenever we were stuck.
1 Introduction 3
2 Related Work 4
3 PROPOSED METHODOLOGY 5
3.0.1 Datasets: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.0.2 Model Architecture: . . . . . . . . . . . . . . . . . . . . . . . . . 6
4 Results 7
4.1 Training and validation performance . . . . . . . . . . . . . . . . . . . . 7
4.2 Multi-class Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
4.2.1 Classification Performance Metrics: . . . . . . . . . . . . . . . . . 8
4.2.2 Confusion Matrix: . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
4.2.3 Binary Evaluation: . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1
Internship Summary
Internship Details
Title of Internship: StressSense: A Deep Learning Approach to Facial Expression-Based
Stress Detection
Internship Program: SN Bose Internship Program, 2024
Institution: National Institute of Technology Silchar, Assam, Department of Computer
Science and Engineering
Internship Duration: June - July, 2024
• Data Collection: Utilized datasets like CK, CK+, and KDEF containing annotated
facial images.
2
Chapter 1
Introduction
Six universal facial expressions—happy, sad, anxious, disgusted, fearful, and sur-
prised—are linked to significant individual emotions [5]. Understanding these
expressions is essential for mental health and social functions, influencing empathy,
bonding, and conflict resolution. Nonverbal communication is also critical in profes-
sional settings, affecting interactions, leadership, and collaboration.
In summary, studying facial expressions and developing assessment tools are vital.
These studies enhance our understanding of human emotions and social interactions,
creating opportunities for innovative applications in daily life and professional work.
3
Chapter 2
Related Work
Adolescence is critical for mental health, with over half of adult mental illnesses
developing by age 14 [22]. Australian data shows that 1 in 7 young individuals have
anxiety disorders, and 1 in 16 experience depression. Symptoms of psychological distress
can appear in early adolescence before a formal diagnosis [23]. These trends are consistent
with data from other Western nations [24]. Youth with psychological issues may already
show abnormal identification of negative facial expressions even before a clinical diagnosis.
Studies show teenagers with depression often misinterpret sad facial expressions,
seeing other emotions as sadness, while those without depression tend to misread them
as happiness [10]. Adolescents with psychopathic tendencies also struggle with facial
recognition, particularly with fear and sadness [11].
4
Chapter 3
PROPOSED METHODOLOGY
(a) Webcam Integration: Captures video frames at intervals for continuous data
analysis.
(b) Facial Region Extraction: Uses algorithms to extract facial regions from video
frames.
(c) Emotion Classification: Classifies seven major emotions using a trained deep
learning model.
(d) Stress Detection Algorithm: Analyzes emotion likelihoods to detect dominant
emotions, prioritizing stress-related ones.
(e) User Notification and Feedback Loop: Alerts user of potential stress and solicits
feedback for accuracy.
(f) Continuous Monitoring: Processes images regularly for ongoing stress evaluation
and timely intervention.
3.0.1 Datasets:
Dataset Description:
The project utilized the Cohn-Kanade (CK), Cohn-Kanade Extended (CK+), and Karolin-
ska Directed Emotional Faces (KDEF) datasets, known for their extensive collections of
annotated facial images. The CK and CK+ datasets feature images with varied expression
5
intensities and sequences from neutral to peak emotions, while the KDEF dataset includes
standardized photos of actors portraying different emotions. All datasets were split 80:20
for training and testing to facilitate model training and validation.
Data Preprocessing:
The dataset, containing 32,900 grayscale images of faces in eight emotions, was preprocessed
by standardizing pixel values from 0 to 1 and resizing images from 48x48 to 128x128
pixels. Data augmentation using Keras’s ImageDataGenerator included random rotations,
shifts, shearing, zooming, and horizontal flipping. Finally, the dataset was split 80:20 into
training and test sets, with augmented images used for training and normalized images
for testing.
1. Traditional CNN:
2. MobileNet V2:
6
Chapter 4
Results
The traditional CNN shows effective learning with consistent training accuracy and
decreasing loss but exhibits significant validation fluctuations. It ultimately generalizes
well, with higher test accuracy compared to training. MobileNet V2 demonstrates more
stable learning and validation metrics, making it more reliable and better suited for varied
deployments. While both models perform well, MobileNet V2’s stability and generalization
make it the preferred choice.
7
4.2 Multi-class Evaluation
4.2.1 Classification Performance Metrics:
MobileNet V2 provides stable performance and better generalization, whereas the conven-
tional CNN shows greater variability and requires more fine-tuning.
8
anger contempt disgust fear happiness neutrality sadness surprise
The model captures photos of the user via their computer’s camera and processes
them to assess stress levels based on facial expressions, enabling real-time stress monitoring.
Users provide feedback on detected stress levels, helping to verify system accuracy
9
and refine the model. This feedback loop ensures the system becomes more precise and
personalized over time.
By continuously improving through user input, the system offers effective stress man-
agement support and builds a dataset to understand stress patterns, ultimately enhancing
users’ well-being through advanced emotion recognition technology.
10
Opportunity
Here are some potential future opportunities and areas for expansion for the ”StressSense”
project:
• Integration with Mental Health Platforms: The system could integrate with
existing mental health platforms, offering users seamless access to counseling or
therapy based on real-time stress data, enhancing mental health support accessibility.
• Hardware Limitations
– Challenge: Using webcam feeds for facial expression analysis raises privacy
issues, requiring informed user consent and secure data handling.
– Future Impact: Insufficient attention to data privacy may lead to user
resistance or regulatory issues, especially if sensitive data is mishandled.
– Challenge: The model may need to run on mobile devices or laptops with
constrained resources.
– Future Impact: Running complex models on low-resource devices could drain
battery life quickly or require cloud offloading, which can affect latency and
user experience.
Learning outcome
For the ”StressSense” project report, learning outcomes can emphasize various technical
and practical skills acquired. Here’s a list that could be relevant to include:
These learning outcomes will show the comprehensive skill set gained, covering both
technical expertise in AI and practical knowledge for real-world applications.
Conclusion
This study assessed the effectiveness of conventional CNN and MobileNetV2 models
in accurately recognising face emotions and detecting stress levels using the KDEF
dataset. The findings demonstrated that MobileNetV2 surpassed the conventional
CNN in terms of accuracy, stability, and generalisation. The study successfully
accomplished stress detection by classifying emotions such as anger, disgust, fear,
and melancholy as indications of stress. The enhanced architecture of MobileNetV2
enables more accurate stress detection, making it a reliable tool for real-time stress
monitoring and management. The model’s accuracy was further improved by
continuous feedback and adaptive learning, providing a promising approach for
enhancing users’ overall well-being by effectively managing stress.