TRIBHUVAN UNIVERSITY

INSTITUTE OF ENGINEERING

Kathmandu Engineering College

Department of Computer Engineering

Major Project Report

On
MEDICINAL PLANT RECOGNITION AND
CLASSIFICATION
[Code No: CT 755] By

Aaditya Raj Baidya - BCT76001

Aniket Chand - BCT76013
Arbin Sigdel - BCT76018
Diwash Ghimire - BCT76024

Kathmandu, Nepal
2080
 TRIBHUVAN UNIVERSITY
INSTITUTE OF ENGINEERING

Kathmandu Engineering College

Department of Computer Engineering

MEDICINAL PLANT RECOGNITION AND

CLASSIFICATION

[Code No: CT 755]

PROJECT REPORT SUBMITTED TO THE DEPARTMENT OF
COMPUTER ENGINEERING IN PARTIAL FULFILLMENT OF THE
REQUIREMENT FOR THE BACHELOR OF ENGINEERING

By:
Aaditya Raj Baidya - BCT76001
Aniket Chand - BCT76013
Arbin Sigdel - BCT76018
Diwash Ghimire - BCT76024

Kathmandu, Nepal
2080
 ACKNOWLEDGEMENT

First of all, we would like to acknowledge our deep sense of gratitude to the Institute
of Engineering for including this major project in our course as this has been very
helpful for us to obtain proper insights into the development of software. Similarly, we
are very thankful to the Department of Computer Engineering, KEC for guiding us
throughout this project.
We are highly indebted to our respected supervisor Prof. Dr. Subarna Shakya who
has been overseeing our project and documentation and sharing his insights on how a
project must be developed. We are also grateful to Er. Sharad Chandra Joshi, our
project coordinator and year coordinator for his recommendations regarding the
development of our project. We also extend our special thanks to Er. Sudeep Shakya,
Head of the Computer Department for all the help support, and supervision provided to
us so far. We cannot thank the teachers of KEC enough for providing us with invaluable
supervision and necessary information for the successful completion of the project. We
are highly obliged to express our thanks to our parents and friends for their kind support
cooperation and encouragement which helped a lot throughout our project.

iii
 ABSTRACT

Plants hold immense importance in Nepal, playing a multifaceted role in various aspects
of the country's ecosystem, culture, economy, and daily life. Nepal's diverse
topography, ranging from the lowlands of the Terai to the towering Himalayas,
accommodates a wide array of plant species that contribute significantly to the nation's
ecological balance. Medicinal plant detection aims to create a mobile application that
takes an image of a plant leaf and identifies the medicinal plant along with its
description, and scientific name. This system identifies plant through its leaf attributes,
such as color, shape, and pattern by introducing image processing techniques and deep
learning algorithms using CNN-based transfer learning methods that use various
architectures like VGG16, SVM. Because of the wide variety of plants, we have limited
the system to 12 plants namely: Jatropha, Neem, Jamun, Lemon, Lemongrass,
Pomegranate, Guava, Arjun, Basil. We have collected 200 images for each plant. The
data undergoes augmentation and preprocessing such as shearing and flip. The model
is trained, validated and then tested. The superior model VGG16 was able to correctly
identify medicinal leaves with an accuracy of 97.06%. Images of the leaves are
uploaded through the UI which is integrated with the CNN model.

Keywords: CNN, Deep learning, transfer learning

iv
 TABLE OF CONTENTS
ACKNOWLEDGEMENT ........................................................................................................ iii

ABSTRACT.............................................................................................................................. iv

LIST OF FIGURES ................................................................................................................. vii

LIST OF TABLES .................................................................................................................. viii

ABBREVIATIONS .................................................................................................................. ix

CHAPTER 1: INTRODUCTION .............................................................................................. 1

1.1 BACKGROUND ............................................................................................................. 1

1.2 PROBLEM STATEMENT .............................................................................................. 2

1.3 OBJECTIVES .................................................................................................................. 3

1.4 SCOPE AND APPLICATION ........................................................................................ 3

CHAPTER 2: LITERATURE REVIEW ................................................................................... 4

2.1 EXISTING SYSTEMS IN NEPAL AND WORLDWIDE ............................................. 6

2.1.1 LeafSnap ................................................................................................................... 6

2.1.2 Garden Answers ........................................................................................................ 6

2.1.3 Infoplant .................................................................................................................... 6

2.2 LIMITATIONS OF PREVIOUS SYSTEM .................................................................... 7

2.3 SOLUTIONS PROVIDED BY OUR SYSTEM ............................................................. 7

CHAPTER 3: METHODOLOGY ............................................................................................. 8

3.1 PROCESS MODEL ......................................................................................................... 8

3.2 DEVELOPMENT METHODOLOGY .......................................................................... 10

3.3 DATA COLLECTION .................................................................................................. 11

3.4 SYSTEM BLOCK DIAGRAM ..................................................................................... 12

3.5 ALGORITHM ............................................................................................................... 13

3.5.1 COVOLUTIONAL NEURAL NETWORK ........................................................... 13

3.5.2 TRANSFER LEARNING ....................................................................................... 15

3.5.3 VGGNet ARCHITECTURE (VGG16) .................................................................. 17

3.5.4 SUPPORT VECTOR MACHINES ........................................................................ 18

3.6 FLOWCHART............................................................................................................... 20

v
 3.7 UML DIAGRAMS ........................................................................................................ 21

3.7.1 Use Case Diagram................................................................................................... 21

3.7.2 Activity Diagram .................................................................................................... 21

3.8 DATA FLOW DIAGRAMS .......................................................................................... 22

3.8.1 DFD Level 0 ........................................................................................................... 22

3.8.2 DFD Level 1 ........................................................................................................... 23

3.8.3 DFD Level 2 ........................................................................................................... 23

3.9 TOOLS USED ............................................................................................................... 24

3.9.1 Python ..................................................................................................................... 24

3.9.2 Scikit-Learn............................................................................................................. 24

3.9.3 TensorFlow ............................................................................................................. 24

3.9.4 NumPy .................................................................................................................... 24

3.9.5 Visual Studio........................................................................................................... 25

3.9.6 Google Collaboratory.............................................................................................. 25

3.9.7 Flutter ...................................................................................................................... 25

3.10 VERIFICATION AND VALIDATION ...................................................................... 26

3.10.1 Model Performance for Testing Data- VGGNet ................................................... 26

3.10.2 Model Performance for Testing Data- CNN-SVM ............................................... 27

CHAPTER 4: RESULTS, ANALYSIS AND COMPARISON .............................................. 28

4.1 ANALYSIS AND COMPARISON ............................................................................... 28

4.1.2 Deployment ............................................................................................................. 30

CHAPTER 5: CONCLUSION AND FUTURE WORK ......................................................... 31

5.1 CONCLUSION .............................................................................................................. 31

5.2 FUTURE ENHANCEMENT......................................................................................... 31

REFERENCES ........................................................................................................................ 32

BIBLIOGRAPHY .................................................................................................................... 33

SCREENSHOTS ..................................................................................................................... 34

vi
 LIST OF FIGURES
Figure 3.1 Incremental Process Model
Figure 3.2 (a) Feature Map for Block 1 Conv 1
Figure 3.2 (b) Feature Map for Block 5 Conv 3
Figure 3.3 Sample Medicinal Leaf Images
Figure 3.4 System Block Diagram
Figure 3.5.1 Convolution Neural Network Layers
Figure 3.5.2 Transfer Learning
Figure 3.5.3 VGG16 Architecture
Figure 3.5.4 Architecture of SVM
Figure 3.6 Flowchart
Figure 3.7.1 Use Case Diagram
Figure 3.7.2 Activity Diagram
Figure 3.8.1 DFD Level 0
Figure 3.8.2 DFD Level 1
Figure 3.8.3 DFD Level 2
Figure 3.10.1 Training and Validation Accuracy and Loss (VGG16)
Figure 3.10.2 Training and Validation Accuracy and Loss (CNN-SVM)

vii
 LIST OF TABLES
Table 3.2 Data Augmentation on Guava leaf
Table 4.1 (a) SVM Model Outcome
Table 4.2 (b) VGG16 Model Outcome

viii
 ABBREVIATIONS

AI Artificial Intelligence

ANN Artificial Neural Network

CNN Convolution Neural Network

GUI Graphical User Interface

IDE Integrated Development Environment

KNN K-Nearest Neighbour

ReLU Rectified Linear Unit

RGB Red, Green and Blue

SVM Support Vector Machine

TF TensorFlow

VGG Visual Geometry Group

ix
 CHAPTER 1: INTRODUCTION

1.1 BACKGROUND

According to various sources, there are approximately 380,000 to 450,000 plant species
worldwide, with a significant number possessing medicinal properties [1]. Medicinal
plants play a crucial role in traditional healing practices and modern medicine, offering
remedies for various ailments and health conditions. These plants are rich sources of
bioactive compounds, including alkaloids, flavonoids, and terpenoids, which exhibit
pharmacological effects and therapeutic benefits. In many cultures, including Nepal,
medicinal plants hold cultural significance and are integral to traditional healing
systems. They are utilized in herbal remedies, preparations, and formulations,
contributing to the health and well-being of communities. Moreover, medicinal plants
have economic importance, with the herbal medicine industry contributing to local
economies and livelihoods. Sustainable management and conservation efforts are
essential to ensure the preservation of medicinal plant species and the continuation of
their valuable contributions to healthcare and society.

In Nepal, utilizing Convolutional Neural Networks (CNNs) for the detection of

medicinal plants entails the application of advanced deep learning and computer vision
techniques tailored to the country's distinct environment. Recent advancements have
seen the adoption of state-of-the-art methodologies leveraging deep learning for the
identification of medicinal plant species. These methodologies rely on deep learning
algorithms such as convolutional neural networks (CNNs) to extract intricate features
from leaf images and accurately classify them. Notably, CNN techniques have been
employed for the classification of medicinal plant types by analyzing leaf vein patterns
[2], showcasing the potential of deep learning in enhancing plant detection and
classification efforts within Nepal's rich botanical landscape.

This project focuses on employing various Convolutional Neural Network (CNN)-

based architectures to analyze images of medicinal plants found in Nepal. The dataset
comprises images collected from diverse sources, including manual acquisition and
online repositories, to construct a comprehensive training dataset. Subsequently, the

1
dataset undergoes partitioning into training and validation sets. The training sets
undergo training via transfer learning, leveraging architectures like VGG16, and SVM.
The trained models are then evaluated using the validation set to assess their
performance. Mobile applications feature templates allowing users to upload images of
medicinal plants. The system identifies the plant with the highest accuracy and provides
detailed descriptions, including its scientific and common names, their favorable
climatic conditions, and usage in medicine. While above mentioned architectures like
VGG16 and SVM were used to train the data, the superior accuracy of VGG16 led to
its implementation in creating the required system.

1.2 PROBLEM STATEMENT

Identifying medicinal plants is essential due to their widespread usage in various

aspects of our daily lives. They hold significant importance in fields such as traditional
medicine, botanical research, and Ayurvedic treatments. However, identifying
medicinal plants can be challenging due to their vast diversity, intricate structures, and
unpredictable variations in nature. Complications arise when different plant species
exhibit similar features, or when the same species manifest different characteristics.
This complexity can confound individuals, particularly farmers or those with limited
expertise in the field. In Nepal, the task of medicinal plant detection poses a
considerable challenge owing to several factors. The country's diverse and rugged
geographical terrain, coupled with limited access to remote regions, adds to the
complexity. Moreover, Nepal's landscape spans from lowland subtropical forests to
alpine meadows, hosting a diverse array of flora, including many endangered and
endemic medicinal plant species.
Efforts to accurately detect and monitor these plant species encounter obstacles such as
inadequate infrastructure, sparse data collection, and a lack of expertise in advanced
technologies like remote sensing and artificial intelligence. The traditional methods of
plant identification heavily rely on manual fieldwork, which can be time-consuming
and resource-intensive. Additionally, the region's rugged topography often hinders the
collection of comprehensive plant data.

2
1.3 OBJECTIVES

i. To create a system for users to identify medicinal plants.

ii. To provide descriptions of recognized plants without the necessity of an expert.

1.4 SCOPE AND APPLICATION

From ancient traditions to modern medicine, medicinal plants have played a vital role
in healthcare throughout human history. With their vast array of bioactive compounds,
these natural treasures offer a wealth of potential benefits for human health and well-
being. Medicinal plant recognition and identification plays a crucial role in various
fields and applications, ranging from agricultural exports and environmental
monitoring for research and conservation efforts. Similarly, many plants can be
consumed directly as teas, infusions, or tinctures to treat various conditions like coughs,
digestive issues, and skin problems. Plant extracts can also be used in cosmetics and
personal care products for their antioxidant, anti-inflammatory, and other beneficial
properties.
CNNs, inspired by the visual processing of the human brain, excel at recognizing
patterns and extracting subtle features from images. This allows them to identify
medicinal plants with greater accuracy compared to traditional methods, especially in
challenging scenarios like blurry images or diverse lighting conditions.
In general, identifying plants for personal benefit it is much more accessible and
beneficial. It can also help with educating people on the benefits provided by certain
plants in different terms such as health benefits, environmental benefits, etc.

Applications
i. Useful for automating Quality control in Herbal Products ensuring correct
identification.
ii. Medicinal plants can also be used to treat animals, offering natural alternatives
to conventional veterinary drugs.
iii. Educate people on the medicinal values of various plants for home remedies.
iv. Proper Wild Harvesting and Collection where misidentifications and confusions
are common.

3
 CHAPTER 2: LITERATURE REVIEW
One of the earliest forms of plant identification was proposed by Kevin J. Gaston and
Mark A. O'Neill in 2004 highlighting why automated species identification has not
become widely employed, and whether it is a realistic option, addressing the notions
that it is too difficult, too threatening, too different or too costly [3]. Gaston and O'Neill
(2004), proposed to use a computer vision-based search engine to partially assist with
plant identification. Since then, there has been a lot of research in this area, and a
number of different methods have been proposed.
One common approach is to use features such as shape, texture, and color to represent
a plant image. These features can then be used to train a classifier to distinguish between
different plant species. Another approach is to use deep learning methods, which have
been shown to be very effective for plant identification. The specific method that is
used will depend on the specific application. For example, if the goal is to identify
plants in the wild, then a method that is robust to changes in lighting and viewpoint is
needed. If the goal is to identify plants in a greenhouse, then a method that is fast and
efficient is needed.
Li et al. (2018) introduced a CNN-based approach for the accurate classification of
plant species using leaf images. The key innovation was leveraging hierarchical feature
learning within the CNN to accurately classify diverse plant species. The model
demonstrated high accuracy, showcasing its effectiveness in discerning subtle
differences between species. The research laid the foundation for using CNNs in
automated plant species identification, offering potential applications in agriculture and
biodiversity monitoring. Building on Li et al.'s work, Cruz et al. (2019) further explored
CNNs for plant species identification. Their research emphasized the adaptability of the
model to diverse plant characteristics and environmental conditions. By refining the
CNN architecture and training methodology, the study contributed insights into
improving the robustness and generalization capabilities of automated plant species
identification systems.
Zhang et al. (2020) proposed a lightweight CNN architecture designed for resource-
constrained environments, ensuring real-time processing capabilities without
compromising accuracy. The study demonstrated the feasibility of achieving real-time
capabilities without compromising accuracy. This work addressed the practical

4
constraints of real-world agricultural settings, showcasing the adaptability of CNNs to
diverse scenarios.
Chen et al. (2022) explored advancements in parallel computing and distributed
systems, contributing to the scalability of CNN-based plant detection systems,
particularly in handling extensive datasets. The study focused on handling extensive
datasets, making CNNs more applicable to large-scale agricultural scenarios. The
research provided valuable insights into the infrastructure and computing
considerations necessary for deploying CNNs in scalable agricultural applications.
Despite the successes, challenges persist in achieving robust plant detection using
CNNs. Variability in environmental conditions and image quality continues to be a
focus of ongoing research. Transfer learning, as suggested by Wang et al. (2021),
represents a promising avenue to address these challenges, enhancing model
generalization across diverse agricultural landscapes.
In 2019, N. Manasa, P. Herur, P. Shetty, N. Prarthana, and P. Venkatrao proposed a
plant identification method using watershed algorithm and convolutional neural
network. The input image undergoes various preprocessing stages when the leaf is
surrounded by multiple leaves, watershed algorithm is used to separate each leaf. In the
training phase the images were pre-processed, features were extracted from the pre-
processed image and classified. In the testing phase the images not used for training
were processed and tested by feeding them into the neural network. In this model a pre-
trained convolutional neural network was used to solve classification problem.
Stochastic Gradient Descent algorithm with momentum was used to train the network
and was successfully trained with a validation accuracy of 100%. Confusion matrix was
used to evaluate the performance of the network [4].
In 2020, B. K. Varghese, A. Augustine, J. M. Babu, D. Sunny, and E. S. Cherian
proposed an android application called INFOPLANT to identify plants using the CNN
(Convolutional Neural Network). Transfer learned MobileNet model was used in this
system. It gave details like biological name, common name, location, nutrient
requirement needed by the plant and the medicinal value of the plant [5].
Beikmohammadi, and Faez, proposed a method that uses transfer learning to identify
plant leaves for classification. This method uses the MobileNet. With the help of the
image label and the feature vector logistic regression classifier was trained to predict
the target class. The presented method worked directly with RGB images which
eliminated the use of pre-processing stage and hand-crafted feature extraction. The

5
proposed method was evaluated on the Flavia dataset and LeafSnap dataset and
achieved an accuracy of 99.6% and 90.54% respectively [6].
Currently, PlantSnap has partnered with Snapchat to allow users to identify 600,000+
plants in 37 languages. Leafsnap operates as a field guide to all thing’s foliage.
Developed with experts from Columbia University, the University of Maryland, and
the Smithsonian Institution, the app uses high-tech AI to identify plants based on only
photos of their leaves.

2.1 EXISTING SYSTEMS IN NEPAL AND WORLDWIDE

The existence of systems for recognizing plants have been far and few in between.
Some systems though have emerged in recent times especially due to growing interest
of people in keeping plants indoor.

2.1.1 LeafSnap

Leafsnap is a free app with added subscription features. It operates as a field guide to
all thing’s foliage. Developed with experts from Columbia University, the University
of Maryland, and the Smithsonian Institution, the app uses high-tech AI to identify
plants based on only photos of their leaves. It gives a detailed plant care guide and also
allows its users to keep track of upcoming tasks.

2.1.2 Garden Answers

Garden Answers is an app available for both Android and iOS. Its main purpose it to
classify plants. Aside from classifying plants, it also provides detection of diseases and
pests in plants and also tips on how to deal with common plant pests and diseases.

2.1.3 Infoplant

Infoplant is an educational plant catalog application, It has the technology of quick

response code (QR Code), which can be easily read by most modern cell phones
through your camera. The use of the QR Code in the application is done through the
digitization of the codes present in each plant of the studied spaces. After the user opens
the code, it will be directed to the information screen of the scanned plan.

6
2.2 LIMITATIONS OF PREVIOUS SYSTEM

Most of the plant recognition systems are either not accurate enough or just specialize
on a specific domain of interest. Most of them are only limited to just recognizing the
species of plants.

Only a handful of them seem to provide an overall package capable of not only just
recognizing the plant but also giving necessary and sufficient information on the
different aspects such as the uses of the plant and its medicinal significance, favorable
conditions and more.

2.3 SOLUTIONS PROVIDED BY OUR SYSTEM

Our proposed system is based upon the implementation of convolutional neural network
(CNN). CNNs can automatically and adaptively learn spatial hierarchies of features
from images, which makes them highly effective for image recognition tasks. The
architecture of CNNs is especially suited to recognize patterns in a hierarchical manner
which suits our application.

Our proposed system is designed to help people identify a medicinal plant by clicking
the photo of its leaf through the application. It not only gives the common and scientific
name of the species but also gives description of the plant along with its favorable
climatic conditions.

7
 CHAPTER 3: METHODOLOGY
The process followed for the implementation of this project are shown through the
below methods:

3.1 PROCESS MODEL

Figure 3.1 Incremental Model

For this project, we have used incremental model. Following are the processes through
which we will be using incremental model in our project.

• Requirement analysis:
The first step in any machine learning task is to collect a dataset of data that is
relevant to the task. In the case of plant identification, this would be a dataset of
images of leaves.
The model is then trained on this dataset, and it learns to identify plants based on
the features of the images. Once the model is trained, it can be used to identify
plants in new images.

8
• Design and development:
In this phase, the system is developed based on the design of its functionality. The
data is preprocessed and trained in this stage. The collected data is not ready to be
used for training at first. The image dataset is split into a training set and a testing
set. Image processing is performed, and the algorithm is applied to the training set
to train the images. Each of the training images is passed through a stack of layers,
including convolutional layers, ReLU layers, pooling layers, and fully connected
layers.
• Testing:
The testing data is used to evaluate how well the model performs. The model should
be able to classify plants based on the knowledge it acquired during the training
phase. The testing phase also checks the performance of new functions that were
not part of the training data. In this phase, different testing methods are used to test
whether the system can produce output using the knowledge and self-organization
it acquired during the training phase.
• Implementation and review:
In this phase, a working model is developed that is able to take an unknown image
of a plant and predict its name based on the knowledge it gained during the training
and validation phases. Once the model classifies the plant, it displays the common
name and family name of the plant. The performance of the system is then analyzed,
and the functionalities are thoroughly reviewed. The process is repeated until a
satisfactory and complete system is obtained.
• Deployment and maintenance:
The phase of model deployment involves the process of transitioning a model into
practical usage, enabling it to generate predictions for novel data. This entails the
establishment, evaluation, and upkeep of the necessary infrastructure to ensure
effective data gathering, storage, and retrieval. The supervised model evolves
iteratively through the assimilation of collective data, and its performance should
be optimized for rapid and accurate plant species identification, collection, and
ongoing monitoring.

9
3.2 DEVELOPMENT METHODOLOGY

The Software Development Life Cycle (SDLC) is a systematic process for planning,
creating, testing, deploying, and maintaining software applications. When applied to
projects involving Convolutional Neural Networks (CNNs), such as plant detection
systems, the SDLC ensures a structured and efficient development process.
1. Planning and Requirements Analysis:
The objectives of the plant detection and classification project were clearly
defined. The specific requirements, including the types of plants to be detected,
were defined.
2. Data Collection and Preprocessing:
A diverse dataset of plant images, which is crucial for training CNNs effectively
was collected. Preprocess the data by resizing, normalizing, and augmenting
images to enhance the network's ability to learn relevant features.

Table 3.2 Data Augmentation on Guava leaf

3. Model Design and Architecture:

An appropriate CNN architecture (VGGNet) was chosen based on the project
requirements. The model architecture was designed, specifying the number of
layers, filter sizes, and activation functions to optimize feature extraction from
images.

Figure 3.2 (a) Feature Map for Block 1 Conv 1 and (b) Block 5 Conv 3

10
 4. Implementation (Coding):
Code was written to implement the chosen VGGNet architecture. Deep learning
frameworks like TensorFlow or PyTorch were used for efficient
implementation. The training and validation processes were also coded,
including handling data input and output.
5. Training and Validation:
Rigorous testing is conducted to evaluate the model's performance on unseen
data. Assess metrics like accuracy, precision, recall, and F1 score. Address
issues related to overfitting or underfitting.
6. Deployment and Maintenance:
Once satisfied with the performance of our model, we can deploy it. This could
involve deploying it as a mobile app. Monitor the performance and make
adjustments as needed. This could involve retraining the model on new data or
fine-tuning its hyperparameters.

3.3 DATA COLLECTION

1. Dataset Definition:
Plant Types: For the initial phase of the project, we collected images of leaves
of few common medicinal plants like Neem, Basil, Lemon, Jatropha, Arjun, etc.

2. Data Sources:
For data collection purpose, data repositories such as 'kaggle.com' and
'data.mendeley.com' were used as sources of data. These repositories were
chosen as the datasets in these platforms are provided by data practitioners.

3. Data Quality:
All the images used in the dataset have a size greater than 10KB and the images
that were found to be blurry have been removed through manual inspection.

11
 Figure 3.3 Sample Medicinal Leaf Images

3.4 SYSTEM BLOCK DIAGRAM

The images are required from the gallery or camera. The acquired image is preprocessed
by resizing the image into 224x224 and then normalized. The image is then fed into the
transfer learning-VGGNet architecture model and features extraction of the input
images done. On the basis of the features, the image is identified as one of the plants.
Since the model has already been learned, the information about the feature is stored in
the knowledge domain from which the details of plant datasets are obtained, once the
image is recognized.

12
 Figure 3.4 System Block Diagram

3.5 ALGORITHM

3.5.1 COVOLUTIONAL NEURAL NETWORK

The proposed system will be implemented using convolutional neural network (CNN).
The convolutional neural network (ConvNet/CNN) is a deep learning algorithm which
takes an input image, assign importance (learnable weight and biases) to various aspects
in the image and be able to differentiate one from the other. The preprocessing required
in a ConvNet is much lower as compared to other classification algorithms. While in
primitive methods filters are hand engineered, with enough training, ConvNets have the
ability to learn these filters or characteristics on its own.
The architecture of a ConvNet is analogous to that of the connectivity pattern of neurons
in the brain and was inspired by the organization of Visual Cortex. Individual neurons
respond to stimuli only in a restricted region of the visual field known as receptive field.
A collection of such fields overlaps to cover the entire visual area. The role of the
ConvNet is to reduce the images into a form which is easier to process, without losing
features which are critical for getting a good prediction.

13
 Figure 3.5.1 Convolutional Neural Network Layers

The fundamental layers of CNN are explained below:

i. Convolutional layer
Each convolutional layer contains a series of filters known as convolutional
kernels. The filter is a matrix of integers that are used on a subset of the input
pixel values, the same size as the kernel. Each pixel is multiplied by the
corresponding value in the kernel, then the result is summed up for a single
value for simplicity representing a grid cell, like a pixel in the output
channel/feature map.
Most image processing needs to operate on RGB images with three channels.
When processing a 3 channel RGB image, a convolutional kernel that is a 3-
dimensional array/ Rank 3 tensor of numbers would normally be used. It is very
common for the convolutional kernel to be of size 3 x 3 x 3, the convolutional
kernel being like a cube.
ii. Pooling layer
The pooling layer is responsible for reducing the spatial size of the convolved
feature. This is to decrease the computational power required to process the data
through dimensionality reduction. Furthermore, it is useful for extracting
dominant features which are rotational and positional invariant. Thus,
maintaining the process of effectively training of the model.
There are two types of pooling: max pooling and average pooling. Max pooling
returns the maximum value from the portion of the image covered by the kernel.

14
 On the other hand, average pooling returns the average of all the values from
the portion of the image covered by the kernel.
iii. Fully connected layer
Adding a fully connected layer is a cheap way of learning nonlinear
combinations of the high-level features as represented by the output of the
convolutional layer. The fully connected layer is learning a possibly non-linear
function in that space.
After multiple convolutional layers and down sampling operations the 3D
representation of the image is converted into a feature vector. The flattened
output is fed to a feed forward neural network and back propagation is applied
to every iteration of training. Over a series of a box, the model is able to
distinguish between dominating and certain low-level features in images and
classify them using SoftMax classification technique.

3.5.2 TRANSFER LEARNING

Generally, transfer learning is the process of transferring the knowledge gain from one
to another. It is a search problem in machine learning that focuses on storing knowledge
gained while solving one problem in applying it to a different but related problem. For
example, knowledge gained while learning to recognize cars could apply when trying
to recognize trucks. Transfer learning has the advantage of decreasing the training time
for a learning model and can result in lower generalization error. Transfer learning is
not a distinct type of machine learning algorithm, instead it's a technique or method
used while training models. The knowledge developed from previous training is
recycled to help perform a new task. The new task will be related in some way to the
previously trained task, which could be used to categorized objects in a specific file
type. The original trained model usually requires a high level of generalization to adapt
to the new unseen data. Transfer learning is generally used:
● to save time and resources from having to train multiple machine learning
models from scratch to complete similar tasks.
● as an efficiency saving in areas of machine learning that require high amounts
of resources such as image categorization or natural language processing.
● to negate a lack of labeling data held by an organization by using pre-trained
models.

15
Transfer learning can be used in two methods:
i. Develop model approach
• Select source task: You must select a related predictive modeling problem with
an abundance of data where there is some relationship in the input data, output
data, and/or concepts learned during the mapping from input to output data.
• Develop source model: Next, you must develop a skillful model for this first
task. The model must be better than a naive model to ensure that some feature
learning has been performed.

Figure 3.5.2 Transfer Learning

• Reuse model: The model fit on the source task can then be used as the
starting point for a model on the second task of interest. This may involve
using all or part of the model, depending on the modeling technique used.
• Tune model: Optionally, the model may need to be adapted or refined on
the input-output pair data available for the task of interest.
ii. Pre-trained model approach
• Select source model: A pre-trained source model is chosen from available
models. Many research institutions release models on large and challenging
data sets that may be included in the pool of candidate models from which
to choose from.

16
 • Reuse model: The pretrained model can then be used as the starting point
for a model on the second task of interest. This may involve using all or
parts of the model, depending on the modeling technique used.
• Tune model: Optionally, the model may need to be adapted or refined on
the input output pair data available for the task of interest.

3.5.3 VGGNet ARCHITECTURE (VGG16)

VGGNet, or Visual Geometry Group Network, is a convolutional neural network
(CNN) architecture that gained prominence for its simplicity and effectiveness in image
classification tasks. VGGNet is known for its simplicity and uniformity in architecture.
The consistent use of 3x3 filters and max-pooling layers simplifies the network
structure. VGG16, with its 16 weight layers, demonstrates the importance of depth in
neural networks. Deeper networks can capture hierarchical features in images,
enhancing their representational power.
The number 16 in the name VGG refers to the fact that it is 16 layers deep neural
network (VGGNet). This means that VGG16 is a pretty extensive network and has a
total of around 138 million parameters. Even according to modern standards, it is a huge
network. However, VGGNet16 architecture’s simplicity is what makes the network
more appealing. Just by looking at its architecture, it can be said that it is quite uniform.
There are a few convolution layers followed by a pooling layer that reduces the height
and the width. If we look at the number of filters that we can use, around 64 filters are
available that we can double to about 128 and then to 256 filters. In the last layers, we
can use 512 filters.
Although VGGNet is computationally intensive compared to some later architectures,
remains influential in the design of convolutional neural networks. Its architectural
principles, simplicity, and success in image classification tasks have contributed to its
lasting impact on the field of deep learning. Researchers and practitioners often use
VGGNet as a benchmark and a starting point for understanding more complex
architectures.
Architecture of VGG:
• Input: The VGGNet takes in an image input size of 224×224. For the ImageNet
competition, the creators of the model cropped out the center 224×224 patch in
each image to keep the input size of the image consistent.

17
 • Convolutional Layers: VGG’s convolutional layers leverage a minimal
receptive field, i.e., 3×3, the smallest possible size that still captures up/down
and left/right. Moreover, there are also 1×1 convolution filters acting as a linear
transformation of the input. This is followed by a ReLU unit, which is a huge
innovation from AlexNet that reduces training time. ReLU stands for rectified
linear unit activation function; it is a piecewise linear function that will output
the input if positive; otherwise, the output is zero. The convolution stride is fixed
at 1 pixel to keep the spatial resolution preserved after convolution (stride is the
number of pixels shifts over the input matrix).
• Hidden Layers: All the hidden layers in the VGG network use ReLU. VGG does
not usually leverage Local Response Normalization (LRN) as it increases
memory consumption and training time. Moreover, it makes no improvements
to overall accuracy.
• Fully-Connected Layers: The VGGNet has three fully connected layers. Out of
the three layers, the first two have 4096 channels each, and the third has 1000
channels, 1 for each class.

Figure 3.5.3 VGG16 Architecture

3.5.4 SUPPORT VECTOR MACHINES

Support Vector Machines (SVMs) are renowned for their versatility and effectiveness
in classification tasks. Imagine data points scattered across a landscape, representing
different categories. An SVM seeks the widest possible gap between these categories,
creating a decision boundary – a line or hyperplane – that optimally separates them.
This "margin maximization" approach ensures robust classification, even with noisy or
overlapping data. The world isn't always linearly separable and your data may resemble

18
swirling clouds instead of neat clusters. Kernels are ingenious tools that map your data
into a higher-dimensional space, where an SVM can then find a clear separation. This
empowers SVMs to handle complex, non-linear relationships within your data. SVMs
often excel in achieving high classification accuracy, particularly with smaller datasets.

Figure 3.5.4 Architecture of SVM

19
3.6 FLOWCHART

Figure 3.6 Flowchart

20
3.7 UML DIAGRAMS

3.7.1 Use Case Diagram

Figure 3.7.1 Use Case Diagram

3.7.2 Activity Diagram

This is a UML diagram that shows the behavior of the system. At first, the user starts
the system. Then, the user inputs the image of the plant which can be either from camera
or gallery. This image is then preprocessed by the system and then fed into the model.
The model then detects the defined classes and then saves the output with predicted
classes. The output result can be viewed by the user. This activity diagram shows the
various activities involved in this system.

21
 Figure 3.7.2 Activity Diagram

3.8 DATA FLOW DIAGRAMS

3.8.1 DFD Level 0

This is the Data Flow Diagram level 0 of the system. This level shows the flow of data
between all the entities at the lowest level of implementation. Here, the user responsible
to input data as image, to the system. The system is responsible to process the image
and finally detect the plant using CNN model.

Figure 3.8.1 DFD Level 0

22
3.8.2 DFD Level 1
This data flow diagram shows the flow of data between entities at a level higher than
that of DFD Level 0. Here, the user decides the image source to the system: camera or
gallery. The system then processes the image provided by the user and then feeds the
image to the model. The trained model then detects the classes mentioned and provides
the output to the user.

Figure 3.8.2 DFD Level 1

3.8.3 DFD Level 2

Here, the source can be chosen from the user interface: camera or gallery. The gallery
asks the user to upload an image file and the source opens the camera of the user’s
device. The result of both source is image and this image is provided to the image
processing module. This module processes the image and feeds it to the model. This
trained model detects the specified classes and classifies it accordingly. This
classification result is provided to the user.

Figure 3.8.3 DFD Level 2

23
3.9 TOOLS USED

3.9.1 Python
Python is a widely used general-purpose, high-level programming language. It was
created by Guido van Rossum in 1991 and further developed by Python Software
Foundation. It was designed with an emphasis on good reliability, and its syntax
allowed programmers to express their concepts in fewer lines of code. Python is a
programming language that lets you work quickly and integrate systems more
efficiently. It is popular amongst the users because it is easy to code, free and open-
source code.

3.9.2 Scikit-Learn
Scikit-learn is an open-source machine learning framework for python that is useful for
data mining data analysis and data visualization.it is good for classification, regression,
clustering, dimensionality reduction, model selection, and preprocessing.

3.9.3 TensorFlow
TensorFlow is an open python library for fast numerical computing created and released
by Google. It is a foundational library that can be used to create deep learning models
directly or by using rapid libraries. Tensor flow allows developers to create data flow
graphs-structures that describe how data moves through a graph or a series of
preprocessing nodes. Each node in the graph represents a mathematical operation, and
each connection or edge between nodes is a multidimensional data array, or tensor. It
allows developers to create large scale neural networks with many layers. tensor flow
is mainly used for: classification, perception, understanding, discovering, prediction
and creation. TensorFlow is used for voice recognition and taste-based application
image recognition time series and video detection.

3.9.4 NumPy
NumPy is a general-purpose area processing package. It provides a high-performance
multidimensional array object, and tools for working with these arrays. It is the
fundamental package for scientific computing with python. It includes Important
features like a powerful N dimensional array object, sophisticated functions, Tools for

24
integrating C/C++, useful linear algebra, Fourier transform, and random number
capabilities.

3.9.5 Visual Studio

Visual Studio is an integrated development environment from Microsoft. It is a code
editor supporting IntelliSense as well as code refactoring. The integrated debugger
works as both a source-level debugger and as a machine level debugger.

3.9.6 Google Collaboratory

Google Collab, short for Google Collaboratory, is a free cloud-based platform for
collaborative Python coding. Integrated with Jupyter Notebooks, it provides a seamless
environment with free GPU access for tasks like machine learning. With real-time
collaboration features, pre-installed libraries, and easy integration with Google Drive,
Collab simplifies data science workflows, making it a popular choice for collaborative
coding and machine learning projects.

3.9.7 Flutter
Flutter is an open-source UI software development kit created by Google. It is used to
develop cross platform applications from a single codebase for any web browser.
Flutter simplifies the process of creating consistent, appealing UIs for an application
across the six platforms it supports. It uses the open-source programming language
Dart, which was also developed by Google.

25
3.10 VERIFICATION AND VALIDATION

Metrics like loss and accuracy are used to assess the performance of the models during
the training and validation stages. The loss value gradually lowers and reaches values
of 0.0914 and 0.9996 for VGG16 and SVM respectively as the epoch count increases.
The second metric considered is accuracy. The accuracy scores for VGG16 and SVM
models are 0.9706 and 0.780 respectively at the final 11th epoch.

3.10.1 Model Performance for Testing Data- VGGNet

Total dataset used = 2906
Out of 2906, 200 were used for training purpose, 25 were used for validation, and 10-
15 were used for test purpose.
Training accuracy = 97.06%
Validation accuracy = 87.14%

Figure 3.10.1 (a) Training and Validation accuracy

Figure 3.10.1 (b) Training and Validation loss

26
3.10.2 Model Performance for Testing Data- CNN-SVM
Total dataset used = 2906
Out of 2906, 200 were used for training purpose, 25 were used for validation, and 10-
15 were used for test purpose.
Training accuracy = 78.03%
Validation accuracy = 75.14%

Figure 3.10.2 (a) Training and Validation accuracy

Figure 3.10.2 (b) Training and Validation loss

27
CHAPTER 4: RESULTS, ANALYSIS AND COMPARISON
The purpose of this study is to identify medicinal plants in real-time. The medicinal leaf
images necessary were obtained from Kaggle, dataMendeley, and some from google
with careful curation, to complete this challenge. The collected images are of various
sizes, and therefore have been scaled to a specific dimension. Geometrical
augmentation rescaling, shearing, and flipping were performed on the leaf images
because the collected data is insufficient to train our models. The data is fed into both
VGG16 and SVM model, which is trained with an epoch of 11.

4.1 ANALYSIS AND COMPARISON

CNN-SVM combines the feature extraction power of CNNs with the classification
strengths of SVMs. This hybrid approach leverages the best of both worlds: the
accuracy of CNNs and the explainability of SVMs. VGGNet's deep learning
architecture allows it to automatically learn these features directly from the training
data, eliminating the need for manual feature engineering. This inherent advantage
gives VGGNet a significant edge in capturing the complex visual nuances of plant
species.

28
Epoch Loss Accuracy
Train Validation Train Validation
1 1.2548 1.1934 0.2396 0.2857
2 1.1240 1.1047 0.4769 0.4600
3 1.0777 1.0853 0.5280 0.4857
4 1.0571 1.0700 0.5922 0.5486
5 1.0376 1.0373 0.6700 0.6800
6 1.0259 1.0359 0.7093 0.6686
7 1.0195 1.0403 0.7319 0.6514
8 1.0158 1.0310 0.7423 0.6886
9 1.0100 1.0192 0.7532 0.7286
10 1.0059 1.0560 0.7726 0.5971
11 0.9996 1.0125 0.7803 0.7514

Table 4.1 (a) SVM Model Outcome

Epoch Loss Accuracy

Train Validation Train Validation
1 1.0257 0.7078 0.6804 0.7857
2 0.4223 0.6398 0.8599 0.7971
3 0.2947 0.6515 0.9046 0.8171
4 0.2325 0.4649 0.9254 0.8686
5 0.1781 0.4953 0.9444 0.8743
6 0.1358 0.4570 0.9625 0.8571
7 0.1353 0.4383 0.9620 0.8657
8 0.1377 0.5179 0.9534 0.8629
9 0.0994 0.4731 0.9661 0.8629
10 0.0906 0.5258 0.9729 0.8714
11 0.0914 0.4982 0.9706 0.8714

Table 4.1 (b) VGG16 Model Outcome

29
Table 4.1 displays the obtained metrics values at the train and validate stages. The loss
outcome of CNN-SVM model is 1.25 and 1.19 at the 1st epoch whereas the
corresponding loss of VGG16 model is 1.02 and 0.70. The loss value gradually lowers
and reaches values of 0.99 and 1.01 for SVM and 0.09 and 0.49 for VGG16 as the epoch
count increases to 11.
Similarly, accuracy is another measure to assess the models. The training and validation
accuracy values are 0.23 and 0.28 for SVM and 0.68 and 0.78 for VGG16 at the 1st
epoch. The accuracy values for both phases are 0.78 and 0.75 for SVM and 0.97 and
0.87 for VGG16 at the 11th epoch.
From the outcome of the two models, we can conclude that VGGNet is more accurate
than CNN-SVM.

4.1.2 Deployment
We finalized the VGG16 model and deployed it using flask API. Then we designed the
mobile app with minimum features to make it easy to use. The login screen
automatically when the user launches the app. A user registers for the app by entering
their required credentials. The homepage hosts the option to scan the leaf or upload it
from gallery. The pictures are required to be clear and of a healthy plant. The uploaded
image is forwarded to the VGG16 model for analysis. The results are sent back to the
UI displaying the common name, scientific name, medicinal values, and favorable
climatic conditions.

30
CHAPTER 5: CONCLUSION AND FUTURE WORK

5.1 CONCLUSION

Plants have a crucial role in human life. Mankind has been using traditional herbal
remedies for thousands of years, each in unique ways. Identification of such medicinal
plants have been greatly aided by recent developments in analytical technologies. In
today’s world, where ignorance about the nature is every so growing, proficiency in
identification of plants is crucial. In this study, we two algorithms and used the better
one to develop an automated system for identifying medicinal plants. We have used 12
plants for this study. 97.06% accuracy is achieved by the VGG16 model which
outperformed CNN-SVM. This model will be deployed to create a smart mobile app
that can instantly identify medicinal plants. People who do not have access to expensive
instruments, this VGG16 model and mobile-based technique will be the method of
choice for the rapid detection of medicinal plants.

5.2 FUTURE ENHANCEMENT

Further development can lead to following features being added:

i. Improving the model’s classification performance by introducing a larger
dataset.
ii. Accurate recognition of leaves under various conditions.

31
REFERENCES
[1] https://a-z-animals.com/blog/how-many-plants-are-in-the-world
[2] Neethu Mariam Sunny, 2020, A Review on Deep Learning for Plant Species
Classification using Leaf Vein, INTERNATIONAL JOURNAL OF ENGINEERING
RESEARCH & TECHNOLOGY (IJERT) Volume 09, Issue 06 (June 2020)
[3] Gaston KJ, O'Neill MA. Automated species identification: why not? Philos Trans
R Soc Lond B Biol Sci. 2004 Apr 29;359(1444):655-67
[4] N. Manasa, P. Herur, P. Shetty, N. Prarthana, and P. Venkat Rao. Plant recognition
using watershed and convolutional neural network. In 2019 3rd International
conference on Electronics, Communication and Aerospace Technology (ICECA),
pages 969 973. IEEE, 2019.
[5] B. K. Varghese, A. Augustine, J. M. Babu, D. Sunny, and E. S. Cherian. Infoplant:
Plant recognition using convolutional neural networks. In 2020 Fourth International
Conference on Computing Methodologies and Communication (ICCMC), pages
800807. IEEE, 2020.
[6] A. Beikmohammadi and K. Faez. Leaf classification for plant recognition with deep
transfer learning. In 2018 4th Iranian Conference on Signal Processing and Intelligent
Systems (ICSPIS), pages 2126. IEEE, 2018.

32
BIBLIOGRAPHY
Li et al. (2018): Li, J., Xin, J., Ma, J., & Li, X. (2018). "Plant identification using deep
neural networks via optimization of transfer learning parameters." Computers,
Materials & Continua, 57(2), 183-196.
Cruz et al. (2019): Cruz, A., Silva, D. F., Pereira, C., & Silva, D. F. (2019). "Deep
neural networks transfer learning for plant identification." Computers, Materials &
Continua, 61(2), 761-774.
Zhang et al. (2020): Zhang, M., & Zhang, H. (2020). "Real-time plant disease detection
and classification using deep learning." Computers, Materials & Continua, 66(3), 3211-
3226.
Chen et al. (2022): Chen, J., Chen, J., Huang, L., & Tao, C. (2022). "Scalable and
distributed deep learning for plant disease classification." Computers, Materials &
Continua, 69(1), 585-598.
Wang et al. (2021): Wang, Y., Zhang, Y., & Liu, Y. (2021). "Transfer learning in plant
disease classification: A systematic review." Computers, Materials & Continua, 69(3),
2897-2915.

33
SCREENSHOTS

34