PIP4001 Capstone Project

Review-1

Smart body posture recognition and Guiding system

Batch Number: CCS-G15
Roll Number Student Name
Under the Supervision of,
20211CCS0124 SANKETH S
20211CCS0133 MOHAN C V Ms.Sterlin Minish T N
20211CCS0140 MANU K Assistant Professor
20211CCS0158 SHAUN FRANKLYN School of Computer Science and Engineering
20211CCS0190 V VISHWA KIRAN REDDY Presidency University

Name of the Program: COMPUTER SCIENCE AND ENGINEERING - CYBER SECURITY

Name of the HoD: Dr. Ananda Raj S P
Name of the Program Project Coordinator: Dr. Sharmasth Vali Y
Name of the School Project Coordinators: Dr. Sampath A K / Dr. Abdul Khadar A / Mr. Md Ziaur Rahman
Introduction

• In today’s fast-paced world, self-service health monitoring systems, such as wellness KIOSKs,
are gaining popularity. These KIOSKs provide a convenient and accessible way for users to
measure vital health parameters like Body Mass Index (BMI), Bone Mass Composition
(BMC), Blood Pressure (BP), Electrocardiogram (ECG), pulse, and temperature without the
need for professional assistance.

• One key challenge with these systems is that accurate measurements depend heavily on the
user's body posture. Even slight misalignments, such as an arm positioned incorrectly during a
BP measurement, can lead to inaccurate results.

• Our project addresses this issue by integrating an advanced posture detection system into the
wellness KIOSK. Using a camera and image processing technology, the system captures and
analyzes the user’s posture in real time, ensuring the correct posture for each health test. By
providing immediate feedback and guidance, the system helps users achieve accurate and
reliable measurements.
Literature Review
Sl No Title Content

1 Damien Brulin, Yannick Benezeth, and Estelle The proposed system performs human detection prior
Courtial, “Posture Recognition Based on to the pos_x0002_ture analysis; posture recognition is
Fuzzy Logic for Home Monitoring of the performed only on a human silhouette.The posture
Elderly”, in IEEE TRANSACTIONS ON recognition method, based on fuzzy logic, identifies
INFORMATION TECHNOLOGY IN BIOMEDICINE, four static postures and is robust to variation in the
VOL. 16, NO. 5, SEPTEMBER 2012 distance between the camera and the person, andto
the person’s morphology.

2 Biljana Cvetkoska, Ninoslav Marina, Dijana This Model consists of a smart mirror which works on
Capeska Bogatinoska, Zhanko Mitreski, “Smart its own algorithm and behaves as smart assistant.
Mirror E-health Assistant – Posture This proposed model uses face recognition
Analyze Algorithm” , in IEEE EUROCON 2017, authentication, posture problem detection, and proper
6–8 JULY 2017, OHRID, R. MACEDONIA posture guidance, followed with suggestions for
preventive healthcare. The algorithm identifies the
person’s posture and carefully analyses the posture
and body changes over time.
Sl No Title Content

3 Jheanel Estrada, Larry Vea, “Sitting Recognize proper / improper sitting postures using accelerometer
Posture Recognition for readings from some human spinal points through small, thin, and
Computer Users using lightweight smartphones attached at those points, and by using a
Smartphones and a Web web camera which detects the upper body points’ location and
Camera”, in Proc. of the 2017 IEEE distances. It also established relationships of human body frames
Region 10 Conference (TENCON), and proper sitting posture
Malaysia, November 5-8, 2017

4 Chia-Feng Juang, Chung-Wei Liang, Vision-based human posture recognition method using a support
Chiung-Ling Lee,I-Fang Chung, vector machine (SVM) classifier. Recognition of four main body
“Vision-based Human Body postures two cameras are used to capture two sets of image
Posture Recognition Using sequences at the same time. After capturing the image sequences,
Support Vector Machines” ,in a RGB-based moving object segmentation algorithm is used to
978-1-4673-2112-9/12/$31.00 distinguish the human body from background. Two complete and
©2012 IEEE corresponding silhouettes of the human body are obtained. The
Discrete Fourier Transform (DFT) coefficients and length-width ratio
are calculated from horizontal and vertical projections of each
silhouette. Finally, these features are fed to a Gaussian-kernel-
based SVM to recognize postures. Experimental results show that
the proposed method achieves a high recognition rate.
Sl No Title Content

5 Ling Xie, Xiao Guo, “Object Human pose estimation algorithm called OpenPose has been more
Detection and Analysis of widely used. But its efficiency is very low. We used deep learning
Human Body Postures methods based on TensorFlow to recognize human body
Based on TensorFlow” , in postures.designed eight sets(MobileNet V2,Inception V2,Inception
2019 IEEE International V2, ResNet101,ResNet152, Inception_ResNet_v2,YOLO) of
Conference on Smart Internet experimental schemes through combining the classification model
of Things (SmartIoT) and the detection algorithm.
Existing method Drawback

• Environmental Dependency: Many models rely heavily on controlled environments for accurate
performance. Variations in lighting, background, and occlusions can significantly affect their
reliability.
• User Comfort and Compliance: Wearable devices, while effective for some applications, can be
intrusive or uncomfortable for users, particularly for the elderly or those with disabilities.
• Data Requirements: Deep learning models typically require large amounts of labeled data for
training, which can be difficult to obtain. This can result in models that are not well-tuned for
specific user needs.
• Accuracy and Generalization: While some models demonstrate high accuracy in controlled
settings, their performance may degrade when applied to diverse populations or environments.
This lack of generalization can limit their effectiveness in real-world applications.
• Feature Extraction Complexity: Traditional machine learning approaches often require
significant manual feature extraction, which can be both time-consuming and dependent on
domain-specific knowledge. This can make it challenging to adapt models to different scenarios or
user populations.
Proposed Method

Body Posture Detection and Correction Using Mediapipe - Frontend Integration

 Overview:
This document outlines the frontend integration of the body posture detection and correction system using JavaScript, React, HTML, and CSS. The
frontend interacts with the backend (already built) to provide a user-friendly interface for posture correction during health tests at the self-service wellness
KIOSK. The goal is to guide users into the correct posture for accurate readings during BMI, BP, ECG, and other health assessments.

 Key Features:

• Real-time Video Stream Integration: Captures the user's video stream using the browser's camera API and displays it within the KIOSK interface.
• Mediapipe Landmark Overlay: Overlays the detected body landmarks (from the backend) onto the video stream in real-time, visualizing the posture
analysis.
• Posture Guidance Display: Presents clear visual cues (e.g., arrows, highlights) and textual instructions to guide the user towards the correct posture.
These cues are triggered by the backend's posture deviation analysis.
• Test-Specific Posture Prompts: Displays instructions and visual examples of the ideal posture for each selected health test (BMI, BP, ECG, etc.).
• User Feedback Mechanisms: Provides real-time feedback (visual and potentially audio) based on posture analysis. This feedback helps users
understand and correct their posture.
• Interactive UI: A user-friendly interface that allows users to select tests, initiate posture checks, and receive feedback.
• Communication with Backend: Establishes communication with the backend to receive posture analysis data and send user-selected test information.
Proposed Method

 Frontend Implementation Details:

• Video Capture: Uses the navigator.mediaDevices.getUserMedia API to access the user's camera and stream the
video feed.
• Canvas Integration: A <canvas> element is used to overlay the Mediapipe landmarks and guidance cues onto
the video stream.
• React Components: React components are used to manage the UI elements, video stream, canvas rendering, and
communication with the backend.
• Backend Communication: Fetches posture analysis data from the backend via API calls (e.g., using fetch or
axios). Sends user test selection information to the backend.
• Posture Guidance Logic: Implements logic to translate the backend's posture deviation data into visual and
textual guidance for the user. This includes displaying arrows, highlighting body parts, and providing
descriptive instructions.
• UI Design: Uses HTML and CSS to create a clean and intuitive user interface. Considers accessibility
guidelines for optimal user experience.
• State Management: Utilizes React's state management capabilities (e.g., useState, useContext) to manage the
application's data and UI updates.
 Example Workflow:

• User selects a health test (e.g., BMI) on the KIOSK interface.

• The frontend sends the test selection to the backend.
• The backend provides the frontend with the reference posture for the selected test.
• The frontend starts capturing the video stream and sends frames to the backend for posture analysis.
• The backend processes the frames using Mediapipe and sends back landmark data and posture deviation information.
• The frontend overlays the landmarks on the video stream and displays posture correction guidance based on the backend's analysis.
• The user adjusts their posture based on the feedback.
• Once the posture is deemed correct by the backend, the frontend signals the user and the health test can proceed.

 Technologies Used:

• HTML
• CSS
• JavaScript
• React
• Mediapipe (for landmark detection - primarily handled by backend but visualized on the frontend)
• Backend API (for communication)
This frontend implementation ensures a seamless and interactive user experience, guiding users towards correct postures for accurate health test
results at the self-service wellness KIOSK.
Objectives

• Develop a Non-Assisted Wellness Kiosk: Design and implement a self-service wellness

kiosk enabling users to independently measure vital health parameters (BMI, BMC, BP,
ECG, Pulse, Temperature).
• Ensure Accurate Health Measurements: Improve the accuracy of vital health
measurements by guiding users to adopt correct body postures using real-time image
processing and posture analysis.
• Utilize Mediapipe for Posture Detection: Leverage the Mediapipe framework to detect
key body landmarks and ensure users maintain correct posture during health tests.
• Provide Real-Time Feedback: Develop a system analyzing user posture in real-time and
providing immediate visual or auditory feedback to correct deviations from required
posture for each health test.
• Enhance User Experience through Automation: Create a user-friendly, fully automated
system for seamless operation, accurately guiding users without medical personnel
assistance.
• Implement a Flexible Posture Correction System: Develop a posture correction
mechanism adaptable to various body types and health tests, providing tailored corrections
based on individual deviations.
• Improve Health Awareness: Promote better health awareness by educating users on the
importance of correct posture during health tests, contributing to more accurate
monitoring and healthier lifestyles.
Methodology/Modules
 Image Capture and Processing Module
Overview:
This module captures images of the user's posture using an integrated camera system and processes them using computer vision
techniques.
Features:
• Utilizes the Mediapipe framework for landmark detection of key body parts (e.g., hands, legs, torso).
• Extracts posture-related features such as the angle of joints and alignment of body parts.

 Posture Analysis and Guidance Module

Overview:
This module is responsible for analyzing the detected body landmarks to ensure that the user's posture is correct during the measurement
process.
Features:
• Reference Axis Calculation: A reference axis is established using the detected landmarks, and the system calculates the angle of
deviation from the ideal posture.
• Deviation Detection: If the posture deviates beyond a predefined threshold angle, the system provides visual and audio guidance to
help the user correct it.
• Feedback System: Real-time feedback loop that helps users adjust their posture before taking the measurement, ensuring accurate
results.
Architecture

Draw
Acquire
Capture Landmarks
Pre-Process Required
Frame and
Landmarks
Connections

Check
Posture Yes Is alignment
Matched Aligned for required
test

No

Display Re-Check
Warning
Hardware/software components

Software Components

• Operating System : Windows 10 or Higher

• Development Environment : Visual Studio Code,Jupyter Notebooks
• Image Processing and Machine Learning : OpenCV, MediaPipe .
• External Libraries and Dependencies : NumPy, Pandas, Keras.

Hardware Components

• Camera: 720p HD Webcam.

• Processor: Intel Core i3 or equivalent.
• RAM: 4GB.
Timeline of the Project (Gantt Chart)

Project Timeline

Final Viva-voce

Review-3

Review-2

Review-1

Review-0

28/Jan 17/Feb 09/Mar 29/Mar 18/Apr 08/May

Review-0 Review-1 Review-2 Review-3 Final Viva-voce
Start-Dates 45686 45706 45733 45763 45787
Duration 3 4 5 4 8
Expected Outcomes

• Accurate Vital Health Measurements: The system will ensure reliable and
precise data collection for vital health parameters (BMI, BMC, BP, ECG, pulse,
temperature).
• Posture Correction Guidance: The system will detect and correct improper user
posture during measurements, providing real-time feedback and guidance for
accurate results.
• Real-Time Feedback and Notifications: Users will receive immediate visual
and/or auditory feedback on posture and vital measurements, including alerts for
corrections or successful test completion.
• Increased Measurement Accuracy: By ensuring correct posture, the system will
minimize measurement errors, providing more reliable health data for analysis and
decision-making.
Conclusion

• The development of a self-service wellness KIOSK, integrating posture correction guidance

with vital health parameter measurement, represents a significant advancement in non-
assisted health monitoring. This KIOSK leverages the Mediapipe framework, primarily
within its backend processing, for robust posture detection. The frontend then visualizes the
results of this analysis, ensuring users maintain the correct posture during tests, leading to
more accurate health data.
• Specifically, the frontend incorporates a real-time video stream captured via the user's
device camera. Mediapipe landmarks, calculated by the backend, are overlaid onto this
stream, providing a visual representation of the user's posture. The frontend also displays
real-time feedback and correction prompts, driven by the backend's posture analysis.
• This feedback can take the form of visual cues (e.g., arrows, highlights) and textual
instructions, guiding users toward the correct positioning for each specific health test. This
combined backend-frontend approach enhances the user experience, enabling individuals to
perform measurements independently, increasing accessibility and convenience while
ensuring data accuracy.
Github Link

https://github.com/Mohancv2003/CCS-G15-Final_project
References

• Reference Papers, https://ieeexplore.ieee.org/Xplore/home.jsp

• Damien Brulin, Yannick Benezeth, and Estelle Courtial, “Posture Recognition Based on Fuzzy Logic for Home
Monitoring of the Elderly”, in IEEE TRANSACTIONS ON INFORMATION TECHNOLOGY IN BIOMEDICINE, VOL.
16, NO. 5, SEPTEMBER 2012

• Biljana Cvetkoska, Ninoslav Marina, Dijana Capeska Bogatinoska, Zhanko Mitreski, “Smart Mirror E-health Assistant –
Posture Analyze Algorithm” , in IEEE EUROCON 2017, 6–8 JULY 2017, OHRID, R. MACEDONIA

• Jheanel Estrada, Larry Vea, “Sitting Posture Recognition for Computer Users using Smartphones and a Web
Camera”, in Proc. of the 2017 IEEE Region 10 Conference (TENCON), Malaysia, November 5-8, 2017

• Chia-Feng Juang, Chung-Wei Liang, Chiung-Ling Lee,I-Fang Chung, “Vision-based Human Body Posture Recognition
Using Support Vector Machines” ,in 978-1-4673-2112-9/12/$31.00 ©2012 IEEE

• Ling Xie, Xiao Guo, “Object Detection and Analysis of Human Body Postures Based on TensorFlow” , in 2019 IEEE
International Conference on Smart Internet of Things (SmartIoT)

• Mediapipe Documentation, https://pypi.org/project/mediapipe/

• Sustainable Development Goals, https://sdgs.un.org/goals

Project work mapping with SDG

SDG 3: Good Health and Well-being

• Reduce premature mortality from non-communicable diseases through prevention and treatment and
promote mental health and well-being.
• Achieve universal health coverage, including access to quality essential health-care services.

SDG 9: Industry, Innovation, and Infrastructure

• Enhance scientific research, upgrade technological capabilities of industrial sectors, and encourage
innovation.

SDG 10: Reduced Inequalities

• Empower and promote the social, economic, and political inclusion of all, irrespective of age, sex,
disability, race, ethnicity, origin, religion or economic or other status.

SDG 11: Sustainable Cities and Communities

• Reduce the adverse environmental impact of cities, particularly air quality and waste management.
Thank You