Vol-10 Issue-3 2024 IJARIIE-ISSN(O)-2395-4396
REAL-TIME STRESS DETECTION AND
ANALYSIS USING FACIAL EMOTION
RECOGNITION
1
D. Sirisha, 2 E.Vijaya Bhaskar Reddy, 3J. Srinu, 4M. Mani Teja, 5 V. Prem Kumar
1
Assistant Professor, Department of CSE-Artificial Intelligence and Machine Learning, S.R.K Institute of
Technology, NTR, Andhra Pradesh, India.
2
Student, Department of CSE-Artificial Intelligence and Machine Learning, S.R.K Institute of Technology,
NTR, Andhra Pradesh, India
3
Student, Department of CSE-Artificial Intelligence and Machine Learning, S.R.K Institute of Technology,
NTR, Andhra Pradesh, India
4
Student, Department of CSE-Artificial Intelligence and Machine Learning, S.R.K Institute of Technology,
NTR, Andhra Pradesh, India
5
Student, Department of CSE-Artificial Intelligence and Machine Learning, S.R.K Institute of
Technology,NTR, Andhra Pradesh, India
ABSTRACT
Real-Time Stress Detection and Analysis using Facial Emotion Recognition" is an innovative system designed for
real-time stress detection and analysis through facial emotion recognition. Leveraging the power of machine
learning and computer vision techniques, the system can accurately identify and analyze emotions exhibited by
individuals in live video streams. By utilizing a pre-trained deep learning model, the system detects facial
expressions associated with various stress levels, including "Bursted," "Irritated," "Anxious," "Relaxed," "Neutral,"
"Broked," and "Shocked." The project integrates with a web application interface where users can visualize
comprehensive stress analysis reports generated from the detected emotions over time. Through detailed graphs and
charts, users can explore trends such as emotion distribution over time, average stress levels, and daily stress
variations. Additionally, the system provides personalized recommendations based on the user's emotional patterns,
aiming to improve overall well-being. Emo Watch offers a valuable tool for individuals, therapists, and researchers
to monitor and manage stress levels effectively in diverse settings.
Keyword : - Real-Time, Stress Detection, Analysis, Facial Emotion Recognition
1. INTRODUCTION
The growing number of people with Autism Spectrum Disorder (In today's dynamic and fast-paced world, where
stress and emotional well-being are increasingly significant concerns, the need for effective tools to manage and
understand our emotions has never been greater. In response to this growing demand, we are proud to introduce our
innovative Emotion Analysis and Stress Monitoring Web Application. Designed to provide users with valuable
insights into their emotional state and stress levels, our application combines cutting-edge technology with user-
friendly features to promote mental well-being and overall health.
1. Understanding Facial Expressions: At the heart of our application lies advanced facial recognition
and emotion detection algorithms. By accessing the device's live camera feed, our application is able to
capture real-time facial expressions with remarkable accuracy. Through sophisticated machine learning
models, we analyze these expressions to identify a wide range of emotions, including happiness,
sadness, anger, surprise, and more. This real-time analysis forms the foundation of our comprehensive
emotion tracking system.
23620 www.ijariie.com 76
Vol-10 Issue-3 2024 IJARIIE-ISSN(O)-2395-4396
2. Insights through Data Visualization: Facial expression data is processed and presented to users in a
visually engaging manner. Leveraging the power of Python's Pandas library and Matplotlib, we
generate interactive charts and graphs that illustrate emotion trends over time. From emotion
distribution charts to average stress level calculations, our visualizations offer users a clear
understanding of their emotional patterns and triggers.
3. Personalized Stress Management Recommendations: One of the key features of our application is
the provision of personalized stress management recommendations. Based on the user's emotional state
and stress levels, our application offers tailored suggestions to help individuals cope with stress more
effectively. Whether it's practicing mindfulness exercises, engaging in physical activity, or seeking
social support, our recommendations are designed to empower users with practical tools for enhancing
their well-being.
4. Real-time Monitoring and Analysis: With our application, users can monitor their emotional state
and stress levels in real-time. By observing fluctuations in emotions throughout the day, users gain
valuable insights into their emotional responses to various situations and stimuli. This real-time
monitoring capability allows individuals to identify triggers and adopt proactive strategies for stress
management.
5. User-Friendly Interface: We understand the importance of simplicity and ease of use, which is why
our application features a clean and intuitive user interface. Navigating through different features and
functionalities is effortless, allowing users to access insights and recommendations with ease. Whether
accessing the application via desktop or mobile devices, users can enjoy a seamless experience tailored
to their needs.
6. Promoting Mental Well-being: At its core, our Emotion Analysis and Stress Monitoring Web
Application are dedicated to promoting mental well-being and fostering a culture of self-awareness and
resilience. By empowering users with tools to understand and manage their emotions effectively, we
aim to contribute to a healthier and happier society. Whether used for personal self-care or in
professional settings, our application offers invaluable support for individuals seeking to prioritize
their mental health.
Note: Emotion Analysis and Stress Monitoring Web Application represent a significant step forward in the field
of mental health technology. By harnessing the power of facial recognition, machine learning, and data
visualization, we provide users with actionable insights and personalized recommendations to enhance their
emotional well-being. Whether it's identifying stress triggers, practicing mindfulness, or seeking support, our
application serves as a valuable companion on the journey to better mental health and overall happiness.
2. LITERATURE SURVEY
The research on detecting negative emotional stress based on facial expressions has garnered significant
attention in recent years. Zhang et al. (2019) proposed a real-time approach for detecting negative emotional stress
using facial expression analysis. Their study, presented at the IEEE 4th International Conference on Signal and
Image Processing, employed advanced signal processing techniques to identify stress-related facial cues.In a similar
vein, Gao et al. (2014) explored the detection of emotional stress from facial expressions specifically for driving
safety applications. Their work, presented at the IEEE International Conference on Image Processing, focused on
leveraging facial expression recognition to enhance driving behavior monitoring systems.
Giannakakis et al. (2020) contributed to the field by evaluating models of facial action units for automatic stress
detection. Their study, presented at the IEEE International Conference on Automatic Face and Gesture Recognition,
highlighted the importance of incorporating facial action units into stress detection algorithms.Almeida and
Rodrigues (2021) proposed a facial expression recognition system for stress detection using deep learning
techniques. Their research, presented at ICEIS, demonstrated the efficacy of deep learning models in accurately
identifying stress-related facial expressions.Viegas et al. (2018) presented a dependent model for stress detection
based on facial action units, aiming towards independent stress detection systems. Their study, presented at the
International Conference on Content-Based Multimedia Indexing, emphasized the role of facial cues in stress
detection.Giannakakis et al. (2017) investigated stress and anxiety detection using facial cues extracted from videos.
23620 www.ijariie.com 77
Vol-10 Issue-3 2024 IJARIIE-ISSN(O)-2395-4396
Their study, published in Biomedical Signal Processing and Control, highlighted the potential of video-based
analysis for detecting stress-related facial expressions.
Zhang et al. (2020) proposed a video-based stress detection approach using deep learning techniques. Their
research, published in Sensors, demonstrated the effectiveness of deep learning models in analyzing facial
expressions to detect stress in real-time video data.Dinges et al. (2005) pioneered the use of optical computer
recognition of facial expressions associated with stress induced by performance demands. Their study, published in
Aviation, Space, and Environmental Medicine, laid the groundwork for subsequent research in stress detection from
facial expressions.Giannakakis et al. (2022) further advanced stress analysis from facial videos by employing deep
facial action units recognition. Their research, published in Pattern Analysis and Applications, showcased the
potential of deep learning models in accurately detecting stress-related facial cues.
Chickerur and Hunashimore (2020) conducted a comprehensive study on detecting stress using facial
expressions, emotions, and body parameters. Their research, presented at the International Conference on
Computational Intelligence and Communication Networks, highlighted the multi-modal approach towards stress
detection and emphasized the integration of various physiological signals for improved accuracy. Hindu and
Angalakuditi (2022) proposed an IoT-enabled stress detection scheme utilizing facial expressions. Their work
highlights the integration of Internet of Things (IoT) technologies with facial expression analysis to enable real-time
monitoring of stress levels. By capturing and analyzing facial expressions, their scheme offers a non-intrusive and
convenient method for stress assessment.Sinha and Sharma (2023) introduced a Stress Alarm Raiser based on Facial
Expressions, emphasizing the development of a system that detects stress levels based on facial cues. Their approach
involves the utilization of computer vision techniques to recognize patterns in facial expressions indicative of stress.
The system serves as an early warning mechanism, alerting individuals to elevated stress levels and prompting
proactive interventions.
Baltaci and Gokcay (2016) investigated stress detection in human-computer interaction settings by fusing pupil
dilation and facial temperature features. Their study highlights the potential of multimodal biometric signals in
enhancing stress detection accuracy. By integrating physiological signals with facial expressions, their approach
offers a more comprehensive understanding of stress dynamics during human-computer interaction.Pediaditis et al.
(2015) focused on the extraction of facial features as indicators of stress and anxiety. Their research delves into the
identification of specific facial cues associated with stress, such as changes in facial muscle activity and expression
intensity. By extracting and analyzing these features, their work contributes to the development of robust stress
detection algorithms.Giannakakis et al. (2019) conducted a comprehensive review on psychological stress detection
using biosignals, including facial expressions. Their review synthesizes existing literature on the use of various
biosignals, such as heart rate variability, electrodermal activity, and facial expressions, for stress assessment. They
provide insights into the challenges and opportunities in the field of psychological stress detection, highlighting the
importance of interdisciplinary approaches and advanced signal processing techniques.
Collectively, these studies underscore the significance of leveraging facial expressions and physiological signals for
stress detection. By integrating machine learning algorithms, computer vision techniques, and IoT technologies,
researchers aim to develop innovative solutions for real-time stress monitoring and intervention, ultimately
promoting mental well-being and resilience.In summary, the literature survey demonstrates a growing interest in
leveraging facial expressions for stress detection, with advancements ranging from real-time analysis to deep
learning-based approaches. These studies collectively contribute to the development of robust and effective stress
detection systems with potential applications in various domains, including healthcare, safety, and performance
monitoring.
2.1 Existing System and Drawbacks:
The current landscape of stressful detection and emotion recognization systems often relies on subjective self-
reporting’s or specialized hardware, limiting accessibility and real-time practicality. Many solutions lacks robustness
and fails to captured the dynamic nature of emotive states. Traditional methods often involve manual inputs or
external sensors, introducing inconvenience and potential inaccuracies. Additionally, some systems may lack the
abilities to offers personalized recommendations or fails to consider the broader contexts of an individual's
emotional well-beings. The absences of real-time analyzing and comprehensive visualizations hinders users from
23620 www.ijariie.com 78
Vol-10 Issue-3 2024 IJARIIE-ISSN(O)-2395-4396
gaining a holistic understandings of their emotional patterns. Furthermore, the reliance on external sensors or
complexity setups may impedes widespread adoptions.
These limitations underscores the needs for an improved stressful detection system that overcomes these challenges,
offerings a more seamless, real-time, and user-friendly experiencings. The proposed system aims to address these
drawbacks by utilizing facial emotion recognitions through a webcams, providing instant insights and personalized
recommendations for stress managements. The integrations of data visualizations techniques ensures a more
intuitive and comprehensive understandings of emotive trends, setting it apart from existing approaches.
3. METHODOLOGY
The stress analysis system developed in this research leverages facial expression recognition techniques to detect
and analyze emotional states indicative of stress levels. The system comprises two main components: stress
prediction and stress analysis.
Data Collection and Pre-processing:
• A dataset of facial images annotated with corresponding emotional states was collected for model training.
These images were pre-processed to ensure uniformity in size (48x48 pixels) and converted to grayscale
format to reduce computational complexity. The dataset was divided into training and validation sets using
holdout validation.
Stress Prediction:
• The stress prediction component utilizes a convolutional neural network (CNN) model trained to recognize
facial expressions associated with different emotional states, including stress. The CNN architecture
consists of multiple convolutional layers followed by max-pooling, batch normalization, dropout, and dense
layers. The model was trained using the training dataset, with the Adam optimizer and categorical cross-
entropy loss function.
Stress Analysis:
• The stress analysis component utilizes the trained stress prediction model to analyze real-time facial
expressions captured through a camera feed. The OpenCV library is used for face detection, and the
predicted emotional states are logged along with timestamps into a CSV file for further analysis. Emotion
labels such as 'Bursted,' 'Irritated,' 'Anxious,' 'Relaxed,' 'Neutral,' 'Broked,' and 'Shocked' are assigned based
on the model predictions.
Analysis and Visualization:
• The logged emotional data is analyzed to generate various visualizations, including emotion trends over
time, emotion distribution over time, average stress level every 20 seconds, and daily average stress level.
Matplotlib and Pandas libraries are employed to create these visualizations, which provide insights into the
user's emotional state fluctuations and stress levels.
Recommendation System:
• Based on the analysis results, personalized recommendations are generated to help users manage their
stress levels effectively. These recommendations include relaxation techniques, mindfulness exercises,
physical activities, and social interactions tailored to the user's current emotional state and stress level.
Deployment and Integration:
• The stress analysis system is deployed as a web-based application using the Flask framework, allowing
users to access it via a web browser. The system's user interface provides functionalities for stress
prediction, real-time analysis, visualization, and recommendation display. Integration with existing
software systems or standalone usage is facilitated, enabling seamless incorporation into various
applications for stress management and well-being monitoring.
23620 www.ijariie.com 79
Vol-10 Issue-3 2024 IJARIIE-ISSN(O)-2395-4396
By following this methodology, an effective and user-friendly stress analysis system can be developed, offering
valuable insights and support for individuals seeking to manage their stress levels and improve their overall well-
being.
3.1 NOVELTY OF THE PROJECT
The proposed stress analysis system introduces several novel contributions to the field of emotional well-being
monitoring and stress management:
1. Real-time Stress Detection: Unlike traditional stress assessment methods that rely on self-reporting or
physiological measurements, the system offers real-time stress detection using facial expression analysis.
This allows for immediate feedback and intervention, enabling users to address stressors as they arise.
2. Integration of Deep Learning: The system leverages deep learning techniques, specifically convolutional
neural networks (CNNs), for accurate and efficient facial expression recognition. By training a CNN model
on a diverse dataset of facial images, the system can effectively capture subtle nuances in facial expressions
associated with different emotional states, including stress.
3. Personalized Recommendations: In addition to stress prediction and analysis, the system provides
personalized recommendations tailored to the user's current emotional state and stress level. These
recommendations encompass a range of evidence-based stress management techniques, empowering users
to proactively address their stressors.
4. User-friendly Web Interface: The system is deployed as a web-based application with an intuitive user
interface, making it accessible to a wide range of users. The interface allows users to easily interact with
the system, visualize their emotional data, and receive actionable insights and recommendations for stress
management.
5. Dynamic Visualization: The system generates dynamic visualizations of emotional trends over time,
allowing users to track changes in their stress levels and emotional well-being. These visualizations provide
valuable insights into patterns and fluctuations in emotional states, enabling users to identify triggers and
implement targeted interventions.
6. Scalability and Integration: The system's modular architecture and web-based deployment make it highly
scalable and adaptable to various use cases and environments. It can be seamlessly integrated into existing
software systems or deployed as a standalone application, offering flexibility and versatility in
implementation.
By combining state-of-the-art deep learning techniques with real-time analysis and personalized recommendations,
the proposed stress analysis system represents a novel approach to stress management and emotional well-being
monitoring. Its innovative features and user-centric design position it as a valuable tool for individuals seeking to
better understand and manage their stress levels in today's fast-paced world.
3.2 DATASET ANALYSIS AND DESCRIPTION
The dataset utilized in this study is a modified version of the FER-2013 dataset, specifically curated to focus on
seven distinct emotion classes: 'Bursted', 'Irritated', 'Anxious', 'Relaxed', 'Neutral', 'Broked', and 'Shocked'. Each
emotion class comprises 3000 grayscale images, resulting in a total of 21,000 images across all classes. These
images have been carefully selected and annotated to represent a diverse range of facial expressions associated with
different emotional states.
The FER-2013 dataset originally consists of facial expression images sourced from publicly available internet
images and various image databases. However, for the purpose of this study, the dataset has been filtered and
categorized to emphasize the selected emotion classes. Each image in the dataset is standardized to a resolution of
48x48 pixels and is pre-processed to ensure consistency in format and quality.
The dataset is divided into three distinct subsets: training, validation, and test sets. The training set contains the
majority of the images and is utilized for training machine learning models to recognize and classify facial
expressions based on their associated emotions. The validation set is employed for hyperparameter tuning and model
23620 www.ijariie.com 80
Vol-10 Issue-3 2024 IJARIIE-ISSN(O)-2395-4396
validation, while the test set serves as an independent evaluation dataset to assess the generalization performance of
the trained models.
Each image in the dataset is associated with a single emotion label, indicating the dominant emotional expression
depicted in the image. The distribution of images across the seven emotion classes is balanced, with 3000 images
per class. This balanced distribution ensures that the machine learning models are trained and evaluated on an equal
representation of each emotion category, minimizing bias and facilitating accurate emotion recognition.
The curated dataset provides a comprehensive and standardized resource for research and experimentation in the
field of facial expression recognition and emotion detection. Its balanced distribution of images across multiple
emotion classes enables researchers to develop robust and accurate models capable of recognizing and interpreting a
wide range of human emotions from facial expressions.
It is important to note that while the dataset is carefully curated and balanced, challenges such as class imbalance,
variability in lighting conditions, and facial occlusions may still exist. Addressing these challenges through
appropriate preprocessing techniques and robust model architectures is essential to ensure reliable and accurate
emotion recognition performance.
3.3 ALGORITHM JUSTIFICATIONS:
The proposed algorithm leverages a convolutional neural network (CNN) architecture for stress recognition from
facial expressions. CNNs have demonstrated exceptional performance in image classification tasks, particularly in
extracting hierarchical features from images, making them well-suited for facial expression recognition tasks. Here,
we justify the choice of each component of the algorithm:
1. CNN Architecture: The CNN architecture employed in the algorithm comprises multiple convolutional
layers followed by max-pooling layers, batch normalization, dropout layers, and fully connected layers.
This architecture is chosen for its ability to automatically learn discriminative features from facial
expression images. Each convolutional layer captures different levels of spatial information, while max-
pooling layers reduce spatial dimensions, helping in feature extraction and dimensionality reduction. Batch
normalization ensures stable training by normalizing the activations, and dropout layers prevent overfitting
by randomly dropping units during training.
2. Image Preprocessing: The facial expression images are preprocessed to grayscale and resized to a uniform
resolution of 48x48 pixels. Grayscale conversion reduces computational complexity while retaining
essential information for emotion recognition. Resizing ensures uniformity in image dimensions, enabling
efficient processing and model training.
3. Training and Validation Data: The algorithm utilizes a dataset split into training and validation sets for
model training and evaluation. The training set comprises images with labeled emotion classes, allowing
the CNN to learn to classify facial expressions accurately. The validation set is used for hyperparameter
tuning and model selection, ensuring optimal performance on unseen data.
4. Model Training: The CNN model is trained using the Adam optimizer with a learning rate of 0.001 and
categorical cross-entropy loss function. Adam optimizer is chosen for its effectiveness in updating model
weights and converging quickly to optimal solutions. Categorical cross-entropy loss is suitable for multi-
class classification tasks like emotion recognition, penalizing incorrect predictions based on the logarithmic
difference between predicted and true class probabilities.
5. Model Evaluation: The performance of the trained model is evaluated using both quantitative and
qualitative measures. The training accuracy and loss are monitored across epochs to assess model
convergence and training stability. Additionally, the overall test accuracy is computed using the test dataset
to quantify the model's ability to generalize to unseen data.
23620 www.ijariie.com 81
Vol-10 Issue-3 2024 IJARIIE-ISSN(O)-2395-4396
Overall, the chosen algorithm components and configurations are justified based on their effectiveness in addressing
the task of stress recognition from facial expressions. The combination of CNN architecture, image preprocessing,
data splitting, training strategy, and evaluation metrics ensures robust performance and reliable emotion
classification capabilities.
3.4 ARCHITECHTURE DIAGRAM
The developed system comprises a Flask-based web application for stress analysis leveraging facial expression
recognition. The architecture encompasses several key components:
1. Frontend Interface: The application features various HTML templates for distinct pages like home, about,
contact, stress prediction, and stress analysis. Flask facilitates template rendering, enabling seamless user
interaction.
2. Backend Processing: Python scripts, primarily app.py, handle backend operations within the Flask
framework. These scripts define routes for different pages and execute functions for stress analysis.
3. Stress Analysis Algorithm: The core of the system lies in the stress_analyze route, implementing a robust
algorithm for stress detection via facial expressions. It employs a pre-trained deep learning model for
emotion recognition, extracting facial regions of interest, preprocessing them, and feeding them into the
model for prediction.
4. Data Analysis and Visualization: The analysis route processes logged emotion data, utilizing Pandas,
Matplotlib, and Seaborn libraries for data manipulation, analysis, and visualization. This generates
comprehensive insights such as emotion trends over time, distribution, and average stress levels, enhancing
user understanding.
5. Recommendations and Messaging: Based on the analysis results, the system delivers motivational
messages and coping strategies to users for stress management. Tailored recommendations are provided for
various emotional states, fostering a proactive approach towards well-being enhancement.
6. Integration with Flask: Flask orchestrates HTTP request handling, HTML template rendering, and static
file serving, including images and CSS. It facilitates the system's operation as a web server, accessible via
standard web browsers.
In summation, this architecture amalgamates frontend and backend elements to furnish an interactive web
application for stress assessment and mitigation through facial expression analysis. It capitalizes on deep learning,
data analysis, and visualization techniques to furnish valuable insights and support for stress management
endeavors.
4. KEY COMPONENTS:
1. Face Detection and Emotion Recognition:
• Utilizes a pre-trained face detection model (haarcascade_frontalface_default.xml) and a deep
learning emotion recognition model (model.h5).
• Face detection is performed using OpenCV (cv2), while emotion recognition is conducted using
Keras.
2. Live Video Feed:
• Captures frames from the default camera (typically the built-in webcam) using OpenCV
(cv2.VideoCapture).
• Continuously processes frames for emotion detection in real-time.
3. Real-time Emotion Analysis:
• Detects faces in each frame using the loaded face detection model.
• Extracts the region of interest (ROI), preprocesses it, and feeds it into the emotion recognition
model for prediction.
• Logs the predicted emotion label along with a timestamp into a CSV file (naveen.csv).
23620 www.ijariie.com 82
Vol-10 Issue-3 2024 IJARIIE-ISSN(O)-2395-4396
4. Data Logging:
• Records timestamps and predicted emotions in real-time, facilitating the creation of a dataset for
further analysis.
• The CSV file serves as the dataset for visualization and trend analysis.
5. Continuous Operation:
• Operates in a loop, continuously gathering emotional data from live video feeds until the user exits
the application.
• Upon exit, releases camera resources and closes OpenCV windows.
4.1 Significance:
• The real-time data collection process enables continuous stress analysis and visualization, offering insights
into emotional patterns and trends.
• Provides a user-friendly interface for stress monitoring and management, aiding in self-awareness and well-
being improvement.
4.2 E.System Architecture Diagram Description: IoT-Enabled Stress Detection System
4.2.1 Components:
1. Web Application:
• Frontend Interface for users to interact with the system.
• Developed using HTML, CSS, and JavaScript.
• Allows users to view stress detection results and receive recommendations.
2. Server:
• Backend server responsible for processing requests and managing data.
• Implements business logic and interfaces with the database.
• Built using Node.js and Express.js.
3. Database:
• Stores user data, including facial expression logs and stress levels.
• Utilizes a relational database management system (e.g., MySQL, PostgreSQL).
• Ensures data integrity and persistence.
4. IoT Device (Camera):
• Captures real-time facial expressions of users.
• Sends captured images to the server for analysis.
• Mounted on the user's device (e.g., laptop, smartphone).
5. Stress Detection Algorithm:
• Analyzes facial expressions to detect signs of stress.
• Utilizes machine learning models for emotion recognition.
• Processes images received from the IoT device.
4.2.2 Interactions:
1. User interacts with the Web Application to initiate stress detection.
2. The Server receives requests from the Web Application and processes them.
3. The Server communicates with the Database to store and retrieve user data.
4. The IoT Device captures facial expressions and sends them to the Server.
5. The Stress Detection Algorithm analyzes facial expressions and sends stress detection results to the Server.
6. The Server updates the Database with stress detection results and retrieves recommendations.
7. The Web Application displays stress detection results and recommendations to the user.
23620 www.ijariie.com 83
Vol-10 Issue-3 2024 IJARIIE-ISSN(O)-2395-4396
4.2.3 Data Flow:
1. Facial expression data flows from the IoT Device to the Server.
2. Stress detection results and recommendations flow from the Server to the Web Application.
3. User inputs (e.g., stress detection initiation) flow from the Web Application to the Server.
4.2.4 Communication Protocols:
1. HTTP/HTTPS: Used for communication between the Web Application and the Server.
2. TCP/IP: Facilitates data transmission between the IoT Device and the Server.
3. SQL: Enables interaction between the Server and the Database for data storage and retrieval.
4.2.5 Security Measures:
1. Data Encryption: Utilized to secure communication channels (e.g., HTTPS) and protect sensitive
information.
2. Authentication and Authorization: Implemented to ensure that only authorized users can access the system
and its resources.
3. Secure Data Storage: Database encryption and access control mechanisms are employed to safeguard stored
data.
4. Regular Security Audits: Conducted to identify and address potential vulnerabilities in the system.
5. RESULTS
5.1 Training Progress
The training of the model was conducted over 60 epochs using a generator-based approach. Throughout the training
process, the model's performance was monitored in terms of loss and accuracy. The training progress is summarized
as follows:
• Training Loss: The training loss steadily decreased over the epochs, indicating that the model effectively
minimized its error during training. Starting with an initial loss of 2.2726, it gradually decreased to 0.0662
by the final epoch.
• Training Accuracy: The training accuracy steadily increased over the epochs, demonstrating the model's
ability to learn and make accurate predictions. Beginning with an accuracy of 16.86%, it achieved an
impressive accuracy of 98.39% by the end of training.
The training progress is visualized in the accompanying accuracy and loss plots (see Figures 1 and 2).
5.2 Model Evaluation
Upon completion of training, the trained model was evaluated using a separate test dataset. The model demonstrated
outstanding performance on the test dataset, achieving an overall test accuracy of 98.00%. This result indicates that
the model generalized well to unseen data and effectively classified the input images into their respective categories.
5.3 Accuracy and Loss Plots
The accuracy and loss plots (see Figures 1 and 2) provide a visual representation of the model's training progress
over the epochs. These plots illustrate the decrease in loss and increase in accuracy as the training progressed,
confirming the model's learning capabilities.
23620 www.ijariie.com 84
Vol-10 Issue-3 2024 IJARIIE-ISSN(O)-2395-4396
Fig 1: Accuracy and Loss Plots
5.4 Stress Analysis Results:
Emotion Trends over Time
The above figure illustrates the trends of predicted emotions over time. The x-axis represents the timestamps in
Indian Standard Time (IST), while the y-axis indicates the predicted emotions.
Fig 2: Emotion Trends over Time
Emotion Distribution over Time
The figure above shows the distribution of predicted emotions over time. Each bar represents the count of predicted
emotions on a specific date.
Fig 3: Emotion Distribution over Time
23620 www.ijariie.com 85
Vol-10 Issue-3 2024 IJARIIE-ISSN(O)-2395-4396
Average Stress Level every 20 seconds
The chart above displays the average stress level calculated every 20 seconds, providing insights into the temporal
fluctuations of stress levels.
Fig 4: Average Stress Level every 20 seconds
Emotion Distribution
The pie chart above visualizes the distribution of predicted emotions, indicating the proportion of each emotion
category in the dataset.
Fig 5: Emotion Distribution
Average Stress Level by Emotion
The bar chart above showcases the average stress level categorized by different predicted emotions, providing
insights into the distribution of stress levels across emotion categories.
Fig 6: Average Stress Level by Emotion
23620 www.ijariie.com 86
Vol-10 Issue-3 2024 IJARIIE-ISSN(O)-2395-4396
Emotion Trends over Time (Stacked Area)
The stacked area chart above demonstrates the trends of predicted emotions over time, with each emotion
represented by a different color.
Fig 7: Emotion Trends over Time (Stacked Area)
Stress Analysis Summary
• Average Stress Level: Based on the analysis of emotional expressions, the average stress level is currently
at X.XX.
• Well-being Score: On a scale of 1-7, the well-being score is X.XX, indicating the overall emotional well-
being.
Fig 8: Average Stress Level&Well-being Score
5.5 Recommendations and Insights:
• Average Stress Level Result: Healthy, But you can improve with few suggestions.
Fig 9: Average Stress Level Result
23620 www.ijariie.com 87
Vol-10 Issue-3 2024 IJARIIE-ISSN(O)-2395-4396
• Sudden Shock Change Result: Healthy, You didn't shock suddenly, but you can take some suggestions.
Fig 10: Sudden Shock Change Result
• Frequent 'Shocked' Result: Healthy, But you can improve with few suggestions.
Fig 11: Frequent 'Shocked' Result
• Consistent 'Anxious' Result: Healthy, But you can improve with few suggestions.
Fig 12: Consistent 'Anxious' Result
23620 www.ijariie.com 88
Vol-10 Issue-3 2024 IJARIIE-ISSN(O)-2395-4396
• Frequent 'Relaxed' Result: Healthy, But you can improve with few suggestions.
Fig 13: Frequent 'Relaxed' Result
• Frequent 'Broked' Result: Healthy, But you can improve with few suggestions.
Fig 14: Frequent 'Broked' Result
• Frequent 'Bursted' Result: Healthy, But you can improve with few suggestions.
Fig 15: Frequent 'Bursted' Result
23620 www.ijariie.com 89
Vol-10 Issue-3 2024 IJARIIE-ISSN(O)-2395-4396
• Frequent 'Neutral' Result: Healthy, But you can improve with few suggestions.
Fig 16: Frequent 'Neutral' Result
6. CONCLUSION
In this study, we developed an emotion detection system capable of analyzing facial expressions in real-time to
assess stress levels. By integrating machine learning techniques and computer vision algorithms, we successfully
trained a model to recognize seven distinct emotions: Bursted, Irritated, Anxious, Relaxed, Neutral, Broked, and
Shocked.Our analysis of emotional expressions revealed valuable insights into stress patterns and trends over time.
Through the visualization of emotion trends, distribution, and average stress levels, we gained a comprehensive
understanding of how stress manifests and fluctuates in different contexts.The results highlight the importance of
continuous monitoring and proactive management of stress for maintaining overall well-being. By leveraging the
recommendations provided based on stress analysis, individuals can adopt personalized strategies to mitigate
stressors and promote mental health.
Future research endeavors could focus on refining the emotion detection model, enhancing its accuracy and
robustness across diverse demographic groups and environmental conditions. Additionally, exploring the
effectiveness of intervention strategies recommended based on stress analysis could contribute to the development
of more targeted and personalized stress management interventions.In conclusion, our study demonstrates the
potential of leveraging machine learning and computer vision for real-time stress assessment and provides
actionable insights for individuals to proactively manage their stress levels and enhance their overall well-being.
7. REFERENCES
[1] Zhang, Jin, et al. "Detecting negative emotional stress based on facial expression in real time." 2019 IEEE 4th
international conference on signal and image processing (ICSIP). IEEE, 2019.
[2] Gao, Hua, Anil Yüce, and Jean-Philippe Thiran. "Detecting emotional stress from facial expressions for driving
safety." 2014 IEEE International Conference on Image Processing (ICIP). IEEE, 2014.
[3] Giannakakis, Giorgos, et al. "Automatic stress detection evaluating models of facial action units." 2020 15th
IEEE international conference on automatic face and gesture recognition (FG 2020). IEEE, 2020.
23620 www.ijariie.com 90
Vol-10 Issue-3 2024 IJARIIE-ISSN(O)-2395-4396
[4] Almeida, José, and Fátima Rodrigues. "Facial Expression Recognition System for Stress Detection with Deep
Learning." ICEIS (1). 2021.
[5] Viegas, Carla, et al. "Towards independent stress detection: A dependent model using facial action units." 2018
international conference on content-based multimedia indexing (CBMI). IEEE, 2018.
[6] Giannakakis, Giorgos, et al. "Stress and anxiety detection using facial cues from videos." Biomedical Signal
Processing and Control 31 (2017): 89-101.
[7] Zhang, Huijun, et al. "Video-based stress detection through deep learning." Sensors 20.19 (2020): 5552.
[8] Dinges, David F., et al. "Optical computer recognition of facial expressions associated with stress induced by
performance demands." Aviation, space, and environmental medicine 76.6 (2005): B172-B182.
[9] Giannakakis, Giorgos, et al. "Automatic stress analysis from facial videos based on deep facial action units
recognition." Pattern Analysis and Applications (2022): 1-15.
[10] Chickerur, Satyadhyan, and Avinash M. Hunashimore. "A Study on Detecting Stress using Facial
Expressions, Emotions and Body Parameters." 2020 12th International Conference on Computational Intelligence
and Communication Networks (CICN). IEEE, 2020.
[11] Hindu, Angalakuditi, and Biswajit Bhowmik. "An iot-enabled stress detection scheme using facial
expression." 2022 IEEE 19th India Council International Conference (INDICON). IEEE, 2022.
[12] Sinha, Shweta, and Aakarsh Sharma. "Stress Alarm Raiser Based on Facial Expressions." International
Journal of Innovative Research in Computer Science & Technology 11.3 (2023): 48-51.
[13] Baltaci, Serdar, and Didem Gokcay. "Stress detection in human–computer interaction: Fusion of pupil
dilation and facial temperature features." International Journal of Human–Computer Interaction 32.12 (2016): 956-
966.
[14] Pediaditis, Matthew, et al. "Extraction of facial features as indicators of stress and anxiety." 2015 37th Annual
International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 2015.
[15] Giannakakis, Giorgos, et al. "Review on psychological stress detection using biosignals." IEEE transactions
on affective computing 13.1 (2019): 440-460.
23620 www.ijariie.com 91
