Algorithms for Image Comparison

In an era where images play a crucial role in digital experiences, image comparison algorithms are key to identifying similarities and differences between images for various use cases. Whether in object detection, facial recognition, image retrieval, or image deduplication, comparing images accurately has become essential.

The article dives into the different algorithms used for image comparison, their working mechanisms, and their application in real-world scenarios.

What is Image Comparison?

Image comparison refers to the process of analyzing two or more images to find similarities or differences. This process can be applied across a wide range of domains, including digital image processing, computer vision, and artificial intelligence.

There are several real-world applications where image comparison is necessary:

• Duplicate Image Detection: Social media platforms and digital storage systems need to detect duplicate images to optimize storage.
• Facial Recognition: Security systems use image comparison algorithms to verify identities.
• Content-Based Image Retrieval (CBIR): E-commerce websites use image comparison algorithms for reverse image searches.
• Medical Imaging: Doctors compare medical scans over time to track disease progression.

Popular Algorithms for Image Comparison

Let’s explore the most widely used algorithms that help us compare images efficiently.

1. Mean Squared Error (MSE)

The Mean Squared Error (MSE) is one of the simplest image comparison metrics, which calculates the average of the squared differences between corresponding pixels of two images. The formula is:

MSE = \frac{1}{mn} \sum_{i=1}^{m}\sum_{j=1}^{n}(I1(i,j) - I2(i,j))^2

Where I1 and I2 are the two images being compared, and m and n are the dimensions of the images.

• Pros: Easy to implement and computationally efficient.
• Cons: Does not account for perceptual differences and can be insensitive to small variations.

For this implementation you can download the image from here: image 1 and image 2

import cv2
import numpy as np

def mse(imageA, imageB):
    # Ensure the images have the same size
    assert imageA.shape == imageB.shape, "Images must be the same size."
    
    # Calculate the MSE between the images
    err = np.sum((imageA.astype("float") - imageB.astype("float")) ** 2)
    err /= float(imageA.shape[0] * imageA.shape[1])
    
    return err

# Load images
imageA = cv2.imread('image1.webp', cv2.IMREAD_GRAYSCALE)
imageB = cv2.imread('image2.webp', cv2.IMREAD_GRAYSCALE)

# Compute MSE
error = mse(imageA, imageB)
print(f"Mean Squared Error: {error}")

Output:

Mean Squared Error: 3294.985176086426

2. Structural Similarity Index (SSIM)

SSIM is a more advanced metric than MSE, as it compares the structure, luminance, and contrast between two images. SSIM ranges from -1 to 1, where 1 indicates that two images are identical.

SSIM is computed using:

SSIM(x, y) = \frac{(2\mu_x \mu_y + C_1)(2\sigma_{xy} + C_2)}{(\mu_x^2 + \mu_y^2 + C_1)(\sigma_x^2 + \sigma_y^2 + C_2)}

Where:

• \mu_x and \mu_y are the average pixel intensities.
• \sigma_x^2 and \sigma_y^2 are the variances.
• \sigma_{xy} is the covariance of the two images.

• Pros: Accounts for visual differences, making it more perceptually accurate.
• Cons: Slightly slower than MSE.

from skimage.metrics import structural_similarity as ssim
import cv2

def compare_ssim(imageA, imageB):
    # Ensure the images have the same size
    assert imageA.shape == imageB.shape, "Images must be the same size."
    
    # Compute SSIM between two images
    score, diff = ssim(imageA, imageB, full=True)
    return score

# Load images
imageA = cv2.imread('image1.webp', cv2.IMREAD_GRAYSCALE)
imageB = cv2.imread('image2.webp', cv2.IMREAD_GRAYSCALE)

# Compute SSIM
ssim_score = compare_ssim(imageA, imageB)
print(f"SSIM Score: {ssim_score}")

Output:

SSIM Score: 0.3995825443924526

3. Histogram Comparison

In this method, the images are represented as histograms of pixel intensities. The comparison is then done by calculating the difference between their histograms. Several techniques like correlation, Chi-Square, and Bhattacharyya distance can be used to measure the similarity.

• Pros: Effective in comparing images with slight lighting variations.
• Cons: Not robust to spatial changes in the image.

import cv2

def compare_histograms(imageA, imageB, method='correlation'):
    # Convert images to HSV color space for better comparison
    imageA = cv2.cvtColor(imageA, cv2.COLOR_BGR2HSV)
    imageB = cv2.cvtColor(imageB, cv2.COLOR_BGR2HSV)

    # Calculate histograms
    histA = cv2.calcHist([imageA], [0, 1], None, [50, 60], [0, 180, 0, 256])
    histB = cv2.calcHist([imageB], [0, 1], None, [50, 60], [0, 180, 0, 256])

    # Normalize histograms
    cv2.normalize(histA, histA, alpha=0, beta=1, norm_type=cv2.NORM_MINMAX)
    cv2.normalize(histB, histB, alpha=0, beta=1, norm_type=cv2.NORM_MINMAX)

    # Use correlation or other methods
    methods = {
        'correlation': cv2.HISTCMP_CORREL,
        'chi-square': cv2.HISTCMP_CHISQR,
        'bhattacharyya': cv2.HISTCMP_BHATTACHARYYA
    }

    comparison = cv2.compareHist(histA, histB, methods[method])
    return comparison

# Load images
imageA = cv2.imread('image1.webp')
imageB = cv2.imread('image2.webp')

# Compare histograms using correlation
hist_score = compare_histograms(imageA, imageB, method='correlation')
print(f"Histogram Comparison Score (Correlation): {hist_score}")

Output:

Histogram Comparison Score (Correlation): 0.20331759866276455

4. Feature-Based Methods (SIFT, SURF, ORB)

Feature-based methods rely on detecting key points and descriptors in the images. The most popular algorithms in this category include:

• SIFT (Scale-Invariant Feature Transform): Detects local features that remain consistent across scale and rotation changes.
• SURF (Speeded Up Robust Features): Faster than SIFT, but still detects features invariant to scale and rotation.
• ORB (Oriented FAST and Rotated BRIEF): An open-source, faster alternative to SIFT and SURF.

These algorithms detect unique features and use them to compare images, which works well even when images undergo transformations such as rotation, scaling, or partial occlusion.

• Pros: Robust to various transformations like rotation, scaling, and partial occlusion.
• Cons: Computationally more expensive than pixel-based methods.

import cv2

def orb_feature_compare(imageA, imageB):
    # Convert to grayscale
    imageA = cv2.cvtColor(imageA, cv2.COLOR_BGR2GRAY)
    imageB = cv2.cvtColor(imageB, cv2.COLOR_BGR2GRAY)

    # Initialize ORB detector
    orb = cv2.ORB_create()

    # Find the keypoints and descriptors with ORB
    kpA, desA = orb.detectAndCompute(imageA, None)
    kpB, desB = orb.detectAndCompute(imageB, None)

    # Match descriptors
    bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)
    matches = bf.match(desA, desB)
    
    # Sort matches by distance
    matches = sorted(matches, key=lambda x: x.distance)
    
    return len(matches), matches

# Load images
imageA = cv2.imread('image1.webp')
imageB = cv2.imread('image2.webp')

# Compare features using ORB
matches_count, matches = orb_feature_compare(imageA, imageB)
print(f"Number of Matches: {matches_count}")

Output:

Number of Matches: 111

5. Deep Learning-Based Approaches

Deep learning models, especially Convolutional Neural Networks (CNNs), have proven to be highly effective in image comparison tasks. In deep learning-based image comparison:

• Pre-trained CNN models extract high-level features from the images.
• These features are then compared using a similarity metric like cosine similarity.

Popular architectures such as ResNet or VGGNet can be fine-tuned for tasks like image similarity, and techniques such as Siamese networks are often used to train models specifically for comparing images.

• Pros: Extremely powerful for complex image comparison tasks and can handle a variety of transformations.
• Cons: Requires large datasets and computational resources for training.

import torch
import torch.nn as nn
from torchvision import models, transforms
from PIL import Image

# Load pre-trained ResNet model
model = models.resnet50(pretrained=True)
model = nn.Sequential(*list(model.children())[:-1])  # Remove the last classification layer

def preprocess_image(image_path):
    preprocess = transforms.Compose([
        transforms.Resize(256),
        transforms.CenterCrop(224),
        transforms.ToTensor(),
        transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
    ])
    img = Image.open(image_path)
    img = preprocess(img).unsqueeze(0)  # Add batch dimension
    return img

def cosine_similarity(featureA, featureB):
    cos = nn.CosineSimilarity(dim=1, eps=1e-6)
    return cos(featureA, featureB)

# Load and preprocess images
imageA = preprocess_image('image1.webp')
imageB = preprocess_image('image2.webp')

# Extract features
with torch.no_grad():
    featuresA = model(imageA)
    featuresB = model(imageB)

# Compare using cosine similarity
similarity = cosine_similarity(featuresA, featuresB)
print(f"Cosine Similarity: {similarity.item()}")

Output:

Cosine Similarity: 0.9958846569061279

Applications of Image Comparison Algorithms

Image comparison algorithms have widespread applications across industries:

• Medical Imaging: Doctors use these algorithms to compare X-rays or MRI scans for diagnosis.
• E-commerce: Reverse image search allows customers to find products based on images.
• Security: In facial recognition systems, algorithms compare a live image with a database of known faces.
• Social Media: Platforms use image comparison to detect and remove duplicate content.

Challenges in Image Comparison

Despite the advancements in image comparison algorithms, there are still challenges:

• Lighting and Background Changes: Slight changes in lighting or background can drastically affect comparison results in basic algorithms like MSE.
• Image Transformation: Images rotated or scaled differently may still be the same, but simpler algorithms like histogram comparison may fail.
• High Computational Costs: Feature-based and deep learning methods can be resource-intensive and slow.

Conclusion

Image comparison is essential for various applications like image retrieval, face recognition, and duplicate detection. From simple pixel-based methods like MSE to sophisticated deep learning-based approaches, the choice of algorithm depends on the specific use case, the complexity of the images, and the required computational efficiency.