How to use Classification Metrics in Python?

This recipe helps you use Classification Metrics in Python

Recipe Objective

In a dataset after applying a Classification model how to evaluate it. There are many metrics that we can use. We will be using accuracy , logarithmic loss and Area under ROC.

So this is the recipe on how we we can use Classification Metrics in Python.

Get Access to Plant Species Identification Project using Machine Learning

Step 1 - Import the library

from sklearn import datasets from sklearn import tree, model_selection, metrics from sklearn.model_selection import train_test_split

We have imported datasets, tree, model_selection and test_train_split which will be needed for the dataset.

Step 2 - Setting up the Data

We have imported inbuilt wine dataset and stored data in x and target in y. We have used to split the data by test train split. Then we have used model_selection.KFold. seed = 42 dataset = datasets.load_breast_cancer() X = dataset.data; y = dataset.target X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25) kfold = model_selection.KFold(n_splits=10, random_state=seed) kfold = model_selection.KFold(n_splits=10, random_state=seed)

Step 3 - Training model and calculating Metrics

Here we will be using DecisionTreeClassifier as a model model = tree.DecisionTreeClassifier() Now we will be calculating different metrics. We will be using cross validation score to calculate the metrices. So we will be printing the mean and standard deviation of all the scores.

    • Calculating Accuracy

scoring = "accuracy" results = model_selection.cross_val_score(model, X_train, y_train, cv=kfold, scoring=scoring) print("Accuracy: ", results.mean()); print("Standard Deviation: ", results.std())

    • Calculating Logarithmic Loss

scoring = "neg_log_loss" results = model_selection.cross_val_score(model, X_train, y_train, cv=kfold, scoring=scoring) print("Logloss: ", results.mean()); print("Standard Deviation: ", results.std())

    • Calculating Area under ROC curve

scoring = "roc_auc" results = model_selection.cross_val_score(model, X_train, y_train, cv=kfold, scoring=scoring) print(); print("AUC: ", results.mean()); print("Standard Deviation: ", results.std())

So the output comes as:

Accuracy:  0.9248615725359912
Standard Deviation:  0.03454639234547574

Logloss:  -2.675538335423929
Standard Deviation:  1.2623224750420183

AUC:  0.9168731849436718
Standard Deviation:  0.027925303925433888
​

Download Materials


What Users are saying..

profile image

Ameeruddin Mohammed

ETL (Abintio) developer at IBM
linkedin profile url

I come from a background in Marketing and Analytics and when I developed an interest in Machine Learning algorithms, I did multiple in-class courses from reputed institutions though I got good... Read More

Relevant Projects

Build a Review Classification Model using Gated Recurrent Unit
In this Machine Learning project, you will build a classification model in python to classify the reviews of an app on a scale of 1 to 5 using Gated Recurrent Unit.

Customer Churn Prediction Analysis using Ensemble Techniques
In this machine learning churn project, we implement a churn prediction model in python using ensemble techniques.

AWS MLOps Project to Deploy Multiple Linear Regression Model
Build and Deploy a Multiple Linear Regression Model in Python on AWS

NLP Project on LDA Topic Modelling Python using RACE Dataset
Use the RACE dataset to extract a dominant topic from each document and perform LDA topic modeling in python.

Expedia Hotel Recommendations Data Science Project
In this data science project, you will contextualize customer data and predict the likelihood a customer will stay at 100 different hotel groups.

Build a Text Classification Model with Attention Mechanism NLP
In this NLP Project, you will learn to build a multi class text classification model with attention mechanism.

Deploy Transformer-BART Model on Paperspace Cloud
In this MLOps Project you will learn how to deploy a Tranaformer BART Model for Abstractive Text Summarization on Paperspace Private Cloud

Learn How to Build a Logistic Regression Model in PyTorch
In this Machine Learning Project, you will learn how to build a simple logistic regression model in PyTorch for customer churn prediction.

Build a Multi Touch Attribution Machine Learning Model in Python
Identifying the ROI on marketing campaigns is an essential KPI for any business. In this ML project, you will learn to build a Multi Touch Attribution Model in Python to identify the ROI of various marketing efforts and their impact on conversions or sales..

Credit Card Default Prediction using Machine learning techniques
In this data science project, you will predict borrowers chance of defaulting on credit loans by building a credit score prediction model.