Project Report on
AI-POWERED VIRTUAL OUTFIT TRY-ON
SYSTEM FOR SUSTAINABLE FASHION
AND SIZING ACCURACY
Submitted by
Pinank Trivedi (Roll No.: 53)
Yashraj Verma (Roll No.: 55)
Nishchay Sinha (Roll No.: 45)
Under the guidance of
Dr. Deepak S. Shete
Professor, E&TC
ELECTRONICS AND TELECOMMUNICATION
Thakur College of Engineering & Technology
Thakur Village, Kandivali (East), Mumbai-400101
(Academic Year 2024-25)
ABSTRACT
Write briefly about the project and expected outcome of the project. Whom will the project
be useful?
The AI-Powered Virtual Outfit Try-On System is designed to transform online shopping by
allowing users to visualize how clothes would look and fit on them in real time. It uses
advanced technologies such as computer vision (pose estimation, image segmentation), deep
learning (GANs, style transfer), and 3D rendering to simulate realistic clothing draping and
fitting. The system aims to provide a highly immersive and personalized virtual dressing room
experience.
Expected Outcome:
A web-based platform that allows users to upload their image or use a live feed to try
on different clothes virtually.
Accurate simulation of clothing fit, texture, and movement on the user’s body.
Reduction in product return rates due to better size accuracy and visualization.
Enhanced customer satisfaction and trust in online fashion purchases.
Contribution to sustainable fashion by reducing unnecessary production and textile
waste.
Who Will Benefit:
1. E-commerce Customers – Can try clothes virtually before buying, improving
confidence in purchases.
2. Online Retailers & Fashion Brands – Benefit from reduced return rates, improved user
engagement, and brand loyalty.
3. Fashion Designers – Can preview designs on different body types quickly.
4. Sustainable Fashion Advocates – As it promotes conscious consumerism and reduces
waste.
5. Tailors & Fit Experts – May use it for virtual fittings and consultations.
CONTENTS
List of Figures i
List of Tables ii
Abbreviations and symbols iii
Chapter No. Topic Pg. No.
Chapter 1 Overview 01
1.1 Introduction
1.2 Background
1.3 Importance of the Project
1.4 Perspective of stakeholders and customers
1.5 Objectives and Scope of the project
1.6 Summary
Chapter 2 Literature Survey 03
2.1 Introduction
2.2 Literature Survey (Limitation Existing system or research gap)
2.3 Problem Statement
2.4 Summary
Chapter 3 Planning & Design 06
3.1 Introduction
3.2 Resources, Tools required, etc.)
3.3 Proposed System
3.4.1 Block Diagram/Flowchart
3.4.3 Methodology (approach to solve the problem)
3.5 Summary
Chapter 4 Paper based on 5 literatures 11
References
Python Based Smart Trial Room
Smart Virtual Trial Room for Apparel Industry
Style-controlled Synthesis of Clothing Segments for Fashion Image Manipulation
A virtual try-on system based on deep learning
Research and Development of Virtual Try-On System Based on Mobile Platform
Chapter 1 – Overview
Introduction
1.1 Introduction
The digital transformation of the fashion and retail industry has led to significant advances in
how consumers interact with products online. Despite the convenience of e-commerce, one of
its enduring limitations is the inability of customers to try on clothing before purchasing, often
resulting in dissatisfaction and high return rates. This research project presents an AI-Powered
Virtual Outfit Try-On System, designed to replicate the in-store experience of trying on
garments through a web-based interface. Leveraging state-of-the-art techniques in computer
vision, deep learning, and 3D rendering, the system enables real-time visualization of garments
on user-provided images or live body feeds. It aims to address critical issues in online retail,
including size uncertainty, customer indecision, and unsustainable return practices.
1.2 Background
Recent advancements in artificial intelligence and visual computing have paved the way for
intelligent systems that can simulate human body poses, segment clothing items from complex
backgrounds, and render garments in three dimensions. Techniques such as OpenPose,
MediaPipe, U-Net, and GAN-based architectures like TryOnGAN have demonstrated
promising results in controlled environments. However, the integration of these technologies
into a seamless, real-time virtual try-on solution remains a challenge due to variations in body
shapes, clothing textures, lighting conditions, and computational efficiency. This project builds
upon previous academic and industrial efforts by combining pose estimation, clothing
segmentation, and mesh-based 3D simulation into a single interactive platform. The goal is not
only to improve the technological robustness of such systems but also to democratize access to
personalized fashion experiences across digital channels.
1.3 Importance of the Project
The proposed virtual try-on system addresses a pressing need in the e-commerce fashion space
by offering a technologically advanced yet user-centric solution. Its importance spans multiple
dimensions:
1) Consumer Empowerment: The system allows users to try on clothes virtually, which
empowers them to make more confident purchasing decisions. This is especially
beneficial for users who cannot access physical stores or prefer online shopping.
2) Business Optimization: For retailers, high return rates due to sizing issues represent a
major logistical and financial burden. By offering an accurate preview of clothing fit,
the system can significantly reduce product returns, leading to cost savings and
improved profit margins.
3) Sustainable Impact: A key objective of the project is to contribute to sustainable fashion.
The fashion industry is one of the largest contributors to global waste and carbon
emissions. Reducing unnecessary purchases and returns directly translates to lower
resource consumption and reduced environmental degradation.
4) Technological Advancement: From a research perspective, the integration of GANs,
real-time pose estimation, and 3D mesh rendering demonstrates a novel application of
these technologies. This not only adds value to the academic community but also creates
opportunities for further exploration in virtual human-computer interaction.
1.4 Perspective of Stakeholders and Customers
This project considers the varying perspectives of its key stakeholders, each of whom derives
distinct value from the system:
1) Customers: End-users are at the heart of the system. With features such as personalized
size recommendations and realistic try-on visuals, customers are less likely to feel
uncertain about product choices. This improves overall satisfaction and loyalty to online
platforms.
2) Retailers and E-Commerce Companies: These stakeholders stand to benefit
significantly from reduced return rates, better inventory management, and higher
conversion rates. The system also provides data analytics on user preferences, which
can be used to fine-tune marketing and production strategies.
3) Fashion Designers and Brands: Virtual visualization tools open new avenues for
designers to test and showcase clothing across different demographics without the need
for physical samples. This supports agile development and inclusive design practices.
4) Technology Providers and Developers: For cloud service providers, AI framework
developers, and UI/UX designers, this system represents a use-case that combines
performance with scalability, pushing the boundaries of what modern computing can
deliver in consumer-facing applications.
5) Environmental Advocates and Policymakers: The system offers a tangible solution to
overconsumption and textile waste, two critical issues in fashion sustainability. It aligns
with circular economy principles and can be a strong candidate for government or NGO-
backed sustainability initiatives.
1.5 Objectives and Scope of the Project
Objectives:
1) To develop a real-time virtual try-on system that utilizes body pose estimation, image
segmentation, and garment simulation to visualize how clothes fit on different users.
2) To provide a dynamic and personalized experience that accurately reflects clothing
drape, texture, and fit.
3) To reduce the return rate in e-commerce fashion by improving the precision of size and
fit recommendations.
4) To support sustainable consumption practices by minimizing over-purchasing and
reducing waste caused by returned or discarded items.
5) To create a scalable and responsive platform suitable for integration with existing e-
commerce systems.
Scope:
1) Technical Stack: Backend built using FastAPI and Flask, with Firebase for user and data
management. Frontend developed using React.js and Three.js for 3D rendering.
2) AI/ML Integration: Incorporates MediaPipe for body pose detection, U-Net or Mask R-
CNN for clothing segmentation, and TryOnGAN for generating realistic visual outputs.
3) 3D Simulation: Utilizes mesh generation and cloth physics (through Blender or WebGL
frameworks) for lifelike garment behavior on virtual models.
4) Cloud Infrastructure: Hosted on AWS S3 and Microsoft Azure for scalability, storage,
and deployment flexibility.
5) Target Users: Includes individual consumers, online retailers, fashion designers, and
sustainability advocates.
6) Platform Reach: Designed as a web-based tool, with extensibility toward mobile and
AR-based solutions in future iterations.
1.6 Summary
The AI-powered virtual try-on system outlined in this chapter is a technologically rich, user-
centric response to the evolving demands of digital fashion retail. By blending deep learning,
computer vision, and interactive 3D modeling, it aims to recreate the tactile experience of in-
store shopping in a digital environment. Beyond commercial utility, the project aligns with the
larger goals of environmental sustainability and inclusive design. The coming chapters will
explore the system architecture, algorithmic design, implementation challenges, and the
measurable impact of the system in real-world use cases.
Chapter 2 – Literature Survey
2.1 Introduction
Virtual try-on systems have emerged as a pivotal technology in the e-commerce sector,
particularly within the apparel industry, enabling customers to visualize how clothing items
would appear on them without physical trials. This technology has gained significant traction,
especially during the COVID-19 pandemic, as consumers sought safe and convenient
alternatives to traditional in-store shopping. According to recent market research, the global
virtual try-on software market is projected to grow at a compound annual growth rate of 25.5%
from 2021 to 2026, underscoring the rising demand for these technologies MobileAppDaily.
Studies indicate that 70% of online shoppers struggle to find apparel that fits correctly, leading
to high return rates that cost businesses over $550 billion annually, highlighting the potential
of virtual try-on to mitigate these issues MobileAppDaily. This chapter reviews three key
studies on virtual try-on technologies, identifies their contributions and limitations, and
discusses broader challenges to propose a problem statement for future research.
2.2 Literature Survey
This section reviews three studies on virtual try-on technologies, summarizing their
contributions and identifying limitations, supplemented by broader challenges from recent
literature.
Study Summaries and Limitations
Study 1: The first study presents a "Smart Trial Room" developed using Python, HTML, CSS,
and OpenCV. This system allows users to virtually try on garments through a webcam, enabling
multiple users to try products simultaneously, with the primary goal of demonstrating that
enhanced interaction features can boost online sales.
Limitations: The system likely relies on 2D image overlays, which may not accurately depict
the fit and drape of garments on different body types. Accuracy is dependent on webcam quality
and lighting conditions, potentially leading to inconsistent user experiences.
Study 2: The second study introduces an approach for digitally altering outfits in images using
neural networks, generating new clothing items that match the geometry of the original item
but with a different style, achieved through segment synthesis. It is shown to be superior to
prior synthesis methods in virtual try-on benchmarks.
Limitations: Despite using advanced neural networks, the approach is still based on 2D images
and may not capture the three-dimensional aspects of clothing fit. The computational intensity
of neural networks might limit real-time applications, affecting user experience.
Study 3: The third study describes a virtual trial room system that detects the user's posture and
adjusts 2D virtual garments accordingly, predicting the right fit by measuring the height and
width of the user's body, aiming to improve decision-making in online apparel purchasing.
Limitations: By relying on only height and width measurements, the system may oversimplify
body shapes, leading to inaccurate fit predictions. The use of 2D virtual garments also limits
the realism of the try-on experience.
Broader Challenges in Virtual Try-On Technology
Beyond these specific studies, recent literature identifies several challenges that impact the
effectiveness of virtual try-on systems:
Accuracy and Realism: Ensuring that the virtual representation accurately reflects how the
product will look and fit on the customer is crucial. Poorly executed experiences can lead to
customer dissatisfaction and potentially harm sales, as noted in discussions on luxury retail
applications Wanna.fashion.
Technical Requirements: Virtual try-on systems often require advanced hardware and software,
which may not be accessible to all users, particularly those with older devices. This limitation
can restrict the technology's potential reach, as highlighted in analyses of virtual try-on adoption
FutureSoftIndia.
Product Limitations: Certain products, such as jewelry or hats, pose challenges for virtual try-
on due to their complexity or the way they interact with the user's body. This can limit the
applicability of the technology across different product categories Wanna.fashion.
User Experience: Balancing an engaging and fun experience with providing accurate and useful
information for purchase decisions is essential. This balance is critical for maintaining customer
satisfaction and encouraging conversions, as discussed in implementation challenges Netguru.
Computational Demands: Advanced systems using AI and AR can be computationally
intensive, potentially leading to performance issues on some devices, particularly mobile
devices with limited computing power. This is a noted concern in discussions on virtual try-on
performance Codete.
Lack of Tactile Feedback: The inability to feel the texture or weight of the product remains a
significant limitation compared to physical try-ons, which can affect customer confidence in
their purchase decisions FutureSoftIndia.
These challenges underscore the need for continued research and development to enhance the
accuracy, accessibility, and overall user experience of virtual try-on technologies.
2.3 Problem Statement
Current virtual try-on systems face significant limitations in accurately simulating the fit and
appearance of garments due to their reliance on 2D representations and simplified body
measurements. These limitations can lead to inaccurate depictions, reducing customer
confidence and potentially increasing return rates. Furthermore, technical barriers, such as high
computational demands and hardware requirements, and product-specific challenges, such as
difficulties with certain item types, hinder the widespread adoption and effectiveness of these
technologies. There is a critical need to develop advanced virtual try-on solutions that leverage
3D modeling, AI-driven personalization, and improved user interfaces to provide realistic,
accurate, and accessible fitting experiences for a diverse range of products and users, addressing
the identified gaps in current systems.
2.4 Summary
Virtual try-on technologies are transforming the e-commerce landscape by offering customers
a way to visualize products without physical trials, particularly important during the COVID-
19 pandemic. The reviewed studies demonstrate various approaches, from basic 2D overlays to
more sophisticated neural network-based synthesis, each contributing to enhancing online
shopping experiences. However, common limitations such as inaccurate fit representation,
technical requirements, and lack of tactile feedback persist, as do broader challenges like
product-specific applicability and computational demands. Addressing these challenges
through innovative research and development, such as integrating 3D modeling and AI, is
essential to unlock the full potential of virtual try-on systems, ultimately enhancing customer
satisfaction and reducing return rates in online shopping.
Chapter 3 – Planning & Design
3.1 Introduction
This chapter outlines the planning and design aspects of developing a virtual try-on platform
integrated with an e-commerce site for fashion retail. The proposed system leverages computer
vision (CV) technologies to allow users to virtually try on clothing items before purchase,
addressing a critical limitation in online apparel shopping. By implementing this solution as a
Vite React application, we aim to create a fast, responsive user experience that combines
traditional e-commerce functionality with innovative virtual fitting room capabilities. The
design approach focuses on creating a seamless integration between shopping and virtual try-
on features while ensuring optimal performance and user experience.
3.2 Resources and Tools Required
The development of our virtual try-on e-commerce platform requires a comprehensive set of
tools and resources. For frontend development, we will utilize Vite as our build tool due to its
exceptional speed and optimization capabilities, paired with React for creating a dynamic and
responsive user interface. Node.js will serve as the JavaScript runtime for our backend
operations. Visual Studio Code will be our primary integrated development environment,
providing powerful features for efficient coding and debugging.
Our implementation will leverage several key libraries and frameworks. OpenCV.js will form
the backbone of our computer vision functionality, enabling sophisticated image processing
directly in the browser. TensorFlow.js will power our machine learning models for accurate
body detection and measurements. MediaPipe will facilitate pose estimation and body tracking,
essential for proper garment overlay. For application architecture, React Router will handle
navigation, while Redux will manage state across the application. Tailwind CSS will be
employed for creating a responsive and visually appealing interface with minimal custom CSS.
External services will enhance our platform's capabilities. We will integrate a payment gateway
API for secure transaction processing, cloud storage services for managing product images and
user data, and an authentication service for secure user account management. On the hardware
side, camera access is essential for capturing user images for the virtual try-on functionality.
Development systems should have a minimum of 4GB RAM, and optionally, a Raspberry Pi
camera module could be utilized for enhanced image capture in certain deployment scenarios.
3.3 Proposed System
The proposed system is an integrated e-commerce and virtual try-on platform built with Vite
and React. The application will function as a standard e-commerce platform with additional
virtual try-on capabilities powered by computer vision technology. The e-commerce
functionality will include comprehensive product browsing and search features, robust user
account management, a seamless shopping cart and checkout experience, and detailed order
history and tracking. These fundamental features ensure the platform meets standard e-
commerce expectations.
What distinguishes our system is its virtual try-on capabilities. The platform incorporates real-
time body detection and pose estimation to accurately map the user's physique. This mapping
enables precise garment overlay and fitting, creating a realistic representation of how clothing
items would look on the user. Furthermore, the system will provide size recommendations
based on automatically determined body measurements, eliminating sizing guesswork. Multiple
view options, including front and side perspectives, will give users a comprehensive
understanding of fit and appearance.
To enhance the user experience further, our system includes a well-organized garment catalog
with integrated try-on options for seamless browsing and testing. Users can share their virtual
try-on results on social media platforms, extending the shopping experience into social
networks. The platform will offer fit feedback and personalized recommendations based on user
preferences and previous interactions. Additionally, users can easily compare different sizes
and styles of the same garment to make informed purchase decisions. Throughout the
implementation, we prioritize performance optimization and user privacy, with image
processing conducted primarily on the client side to minimize sensitive data transmission.
3.4 System Architecture
3.4.1 Block Diagram/Flowchart
The system architecture diagram illustrates the comprehensive structure of our virtual try-on e-
commerce platform, clearly depicting the relationship between client-side and server-side
components. On the client side, the architecture shows the Vite React application at the core,
encompassing UI components, state management, routing, and API integration capabilities.
Below this, the computer vision module handles body detection and tracking, pose estimation,
garment overlay, and size recommendation algorithms. The user interface section details the
product catalog, virtual mirror, shopping cart, user profile, and checkout systems that provide
the complete shopping experience.
The server side includes API services for authentication, product management, order
processing, and payment gateway integration. The database layer stores user data, product
information, and order history, while the asset storage section manages product images and
garment templates. Connection arrows between the client and server components illustrate the
data flow, including API requests, data exchange, and asset retrieval paths. This architecture
balances processing responsibilities between client and server, optimizing performance while
maintaining security and functionality.
Fig 3.1 - Workflow of the project
3.4.2 User Flow
The user experience begins with registration or login, where the system securely stores profile
information for personalization. Users then browse the product catalog, applying filters by
categories, sizes, styles, and other attributes to find items of interest. When a user discovers a
desirable product, they can activate the virtual try-on process by selecting the "Try On" option.
This prompts the system to request camera access, which the user must grant. Once authorized,
the computer vision module detects the user's body and posture in real-time, enabling the system
to superimpose the selected garment on the user's image with appropriate scaling and
positioning. During this interactive session, users can adjust fit parameters, explore color
variations, or try different sizes to achieve their desired look. The system provides intelligent
size recommendations based on the measurements it detects.
When satisfied with their selection, users add items to their cart and proceed through the
checkout flow, completing payment securely and receiving order confirmation. The post-
purchase experience includes order tracking capabilities, options to submit product reviews,
and social sharing features to engage with their network. This comprehensive flow creates a
seamless transition from browsing to virtual try-on to purchase, addressing the primary
limitations of traditional online apparel shopping.
3.4.3 Methodology
Our methodology for implementing the virtual try-on e-commerce platform focuses on
integrating advanced computer vision capabilities with a modern web application framework.
The computer vision implementation begins with body detection and segmentation using pre-
trained models available through MediaPipe or TensorFlow.js. This process extracts key
reference points on the user's body, particularly shoulders and hips, creating a silhouette that
serves as the foundation for garment overlay. Pose estimation tracks the user's posture through
skeletal tracking, identifying joint positions to adapt garment positioning dynamically. This
capability ensures the system can handle different poses and movements, resulting in realistic
garment draping across various body positions.
The garment overlay process involves processing product images to create transparent
backgrounds, then scaling and positioning these images based on the detected body
measurements. The system applies appropriate transformations to match the user's pose and
implements edge detection algorithms for precise alignment of garments with the body
silhouette. Size recommendation functionality calculates the user's measurements from camera
input, comparing these values with product size charts to suggest the most appropriate size
based on individual fit preferences.
Front-end development begins with configuring a Vite React project, establishing routing for
the single-page application, and implementing responsive design principles to ensure
compatibility across all device types. The user interface includes a comprehensive product
catalog with robust filtering capabilities, an intuitive virtual mirror interface with adjustment
controls, a streamlined shopping cart and checkout flow, and comprehensive user profile
management features. State management utilizes Redux for application-wide data handling,
complemented by React's Context API for component-specific states and mechanisms to persist
user preferences and cart items between sessions.
Back-end integration involves creating RESTful endpoints for product management,
implementing secure user authentication and authorization protocols, developing order
processing systems with payment gateway integration, and establishing efficient asset
management for product images. The database design includes well-structured schemas for
users, products, and orders, with appropriate data validation and security measures, and query
optimization for performance.
The methodology concludes with rigorous testing and optimization protocols. Performance
testing focuses on optimizing computer vision algorithms for browser environments,
implementing lazy loading and code splitting techniques, and benchmarking rendering
performance. User testing involves sessions with representative potential customers, gathering
feedback on the accuracy and experience of the virtual try-on feature, and iterating on the design
based on this valuable user input.
3.5 Summary
The planning and design phase establishes a comprehensive framework for developing a Vite
React-based e-commerce platform with integrated virtual try-on capabilities. By leveraging
computer vision technologies, the system aims to enhance the online shopping experience by
allowing users to visualize how clothing items would look on them before purchase. This
approach addresses one of the fundamental limitations of online apparel shopping—the
inability to try items before buying—potentially reducing return rates and increasing customer
satisfaction.
The proposed architecture balances client-side processing for the virtual try-on functionality
with server-side support for e-commerce operations. This design ensures that sensitive
operations like payment processing remain secure while computationally intensive tasks like
image processing occur locally for better performance and privacy. The detailed methodology
outlines a systematic approach to implementing the system, covering everything from computer
vision algorithms for accurate body detection to the development of an intuitive user interface.
During implementation, several key challenges will be addressed, including optimizing
computer vision algorithms for various browser environments, ensuring accurate garment
overlay and fitting across diverse body types, and creating a seamless integration between the
virtual try-on experience and the traditional e-commerce workflow. The success of the system
will ultimately be measured by its ability to provide accurate visual representations of garments
on users, leading to increased customer confidence in online purchases and a corresponding
reduction in return rates due to size or fit issues. This innovative approach to e-commerce
represents a significant advancement in bridging the gap between physical and digital shopping
experiences.
REFERENCES
IEEE standard reference format
[1] J.F.Curtis,(Ed.),ProcessesandDisordersofHumanComm-unication.NewYork: Harper
and Row, 1978.
[2] J.Schroeterand,M.M.Sondhi,“Techniquesforestimatingvocal-tractshapesfromthe
speechsignal,”IEEE Trans. Speech Audio Process., vol. 2, no. 1, pp. 133–150,
1994.
[3] J. M.Pardo,“Vocal tract shape analysis for children,”in Proc. IEEE Int. Conf.
Acoust., Speech, SignalP
