Difference Between StandardScaler and Normalizer in sklearn.preprocessing

Preprocessing step in machine learning task that helps improve the performance of models. Two commonly used techniques in the sklearn.preprocessing module are StandardScaler and Normalizer. Although both are used to transform features, they serve different purposes and apply different methods.

In this article, we will explore the differences between StandardScaler and Normalizer, and provide implementations to illustrate their usage.

StandardScaler

StandardScaler standardizes features by removing the mean and scaling to unit variance. It transforms the data to have a mean of 0 and a standard deviation of 1. This process is also known as z-score normalization.

The transformation applied by StandardScaler can be represented as: ​X_{scaled} = \frac{X - \mu}{\sigma}

Where:

• X is the original feature.
• \mu is the mean of the feature.
• \sigma is the standard deviation of the feature.

When to Use StandardScaler?

• Normally Distributed Data: Best suited for data that follows a Gaussian (normal) distribution.
• Algorithms Sensitive to Feature Scales: Useful for algorithms like Support Vector Machines (SVM), k-Nearest Neighbors (k-NN), and logistic regression that are sensitive to the scale of the data.

Normalizer

Normalizer scales individual samples to have unit norm. It transforms each sample (row) to a unit vector, which helps maintain the direction of the data while scaling.

For each sample x:

x' = \frac{x}{||x||}

Where ∣∣x∣∣ is the norm of the vector x. The default norm used is L2 norm (Euclidean distance), but you can specify L1 or max norms as well.

When to Use Normalizer?

• High-Dimensional Sparse Data: Useful for text classification, image processing, or any situation where the focus is on the direction of the data points.
• Data with Varying Magnitudes: When the magnitude of the feature vectors matters less than their direction.

Key Differences Between StandardScaler and Normalizer

Implementation: StandardScaler and Normalizer

Let’s illustrate the differences between StandardScaler and Normalizer using a sample dataset. We will create a synthetic dataset and apply both transformations.

import numpy as np
import pandas as pd
from sklearn.preprocessing import StandardScaler, Normalizer

# Creating a synthetic dataset
data = np.array([[1, 2, 3],
                 [4, 5, 6],
                 [7, 8, 9]])

# Convert to DataFrame for better visualization
df = pd.DataFrame(data, columns=['Feature1', 'Feature2', 'Feature3'])

# Applying StandardScaler
scaler = StandardScaler()
standardized_data = scaler.fit_transform(df)

# Applying Normalizer
normalizer = Normalizer()
normalized_data = normalizer.fit_transform(df)

# Displaying the results
print("Original Data:\n", df)
print("\nStandardized Data (StandardScaler):\n", standardized_data)
print("\nNormalized Data (Normalizer):\n", normalized_data)

Output:

Original Data:
    Feature1  Feature2  Feature3
0         1         2         3
1         4         5         6
2         7         8         9

Standardized Data (StandardScaler):
 [[-1.22474487 -1.22474487 -1.22474487]
 [ 0.          0.          0.        ]
 [ 1.22474487  1.22474487  1.22474487]]

Normalized Data (Normalizer):
 [[0.26726124 0.53452248 0.80178373]
 [0.45584231 0.56980288 0.68376346]
 [0.50257071 0.57436653 0.64616234]]

Conclusion

Both StandardScaler and Normalizer are essential tools in the preprocessing step of machine learning workflows, but they serve distinct purposes. StandardScaler is ideal for standardizing features to have a mean of 0 and a standard deviation of 1, making it suitable for algorithms sensitive to feature scales. In contrast, Normalizer scales individual samples to unit norms, focusing on the direction of the data points rather than their magnitude.