MNIST Dataset

MNIST-1:

Attributes Details

Authors Fathma Siddique, Shadman Sakib, Md. Abu Bakr Siddique

Title Recognition of Handwritten Digit using Convolutional Neural Network in
Python with TensorFlow and Comparison of Performance for Various Hidden
Layers
Dataset MNIST dataset (handwritten digits, 60,000 training images, 10,000 test
images, 28x28 grayscale pixels)
Published
2019
Year
Methodology Compared performance across various hidden layers and epochs on the
MNIST dataset.
Training used Stochastic Gradient Descent (SGD) and backpropagation.
Algorithms Convolutional Neural Network (CNN)
- Backpropagation
- Stochastic Gradient Descent (SGD)
Accuracy Achieved high accuracy, typically around or exceeding 99%, based on
different hidden layers and configurations tested

MNIST-2:

Attributes Details

Authors Shantanu Patel, Purvashi Dixit, and Aditya Verma

Title CNN for Image Detection Using MNIST
Dataset MNIST dataset (handwritten digits)
Published
2019
Year
Methodology The study implemented Convolutional Neural Networks (CNN) with data
augmentation techniques. Preprocessing included normalization of pixel
values.
Algorithms Convolutional Neural Networks (CNN), Stochastic Gradient Descent
Accuracy Achieved over 99% accuracy on the MNIST dataset after training and
tuning model parameters

MNIST-3:
Attributes Details

Authors Daniel Filan, Shlomi Hod, Cody Wild, Andrew Critch, Stuart Russell

Title Pruned Neural Networks Are Surprisingly Modular

Dataset Small images ,MNIST, Fashion-MNIST
Published 2020
Year
Methodology The paper introduces a modularity measure for multi-layer perceptrons (MLPs) using
graph clustering concepts. It examines MLPs' modular structure post-training and
weight pruning, revealing that trained MLPs are significantly more modular than
random ones, especially when trained with dropout.

Algorithms Normalized Spectral Clustering

Accuracy we train to approximate test accuracies of 98% on MNIST, 89% on Fashion-MNIST,
and 42% on CIFAR-10.

MNIST-4:

Attributes Details

Authors Ritik Dixit, Rishika Kushwah, Samay Pashine

Title Handwritten Digit Recognition using Machine and Deep Learning Algorithms
Dataset MNIST dataset
Published
Year
Methodology Comparative analysis of different machine learning and deep learning models
for handwritten digit recognition
Algorithms Support Vector Machines (SVM), Multi-Layer Perceptron (MLP),
Convolutional Neural Network (CNN)
Accuracy Comparison of accuracy across SVM 98%, MLP, 96.2%and CNN 97.88%.

MNIST-5:

Attributes Details

Authors Alejandro Baldominos, Yago Saez, Pedro Isasi

Title A Survey of Handwritten Character Recognition with MNIST and EMNIST
Dataset The paper reviews the MNIST dataset for handwritten digit recognition,
which is widely used in computer vision research. It also introduces the
EMNIST dataset, which includes both digits and letters, providing a larger
 and more diverse data source.
Published 2019
Year
Methodology The paper summarizes state-of-the-art techniques for MNIST digit
recognition, specifically distinguishing between works that used:

 Original dataset ("out-of-the-box" MNIST)

 Data augmentation to improve model performance

Algorithms focus is on Convolutional Neural Networks (CNNs), which have become

the standard for solving this problem.
Accuracy test error rate smaller than 1% on the MNIST dataset, which has made this
dataset less challenging

Now using CIFAR-10 dataset:

CIFAR-10 Dataset:

CIFAR-1:

Attributes Details

Authors Barret Zoph, Quoc V. Le

Title Neural Arvhitecture Search with Rienforcement Learning

Dataset CIFAR-10
Published 2016
Year
Methodology recurrent neural network (RNN) to generate neural network architectures.
They employ reinforcement learning to optimize the architectures for
expected accuracy on a validation set.

Algorithms The primary algorithm involves using reinforcement learning to train the
RNN, which generates model architectures based on a specified objective.
Accuracy The model achieves a test error rate of 3.65% on the CIFAR-10 dataset,
which is 0.09% better and 1.05 times faster than the previous state-of-the-art
model. Additionally, a novel recurrent cell designed for the Penn Treebank
 dataset achieves a test set perplexity of 62.4, outperforming widely-used
LSTM cells.

CIFAR-2:

Attributes Details

Authors Gao Huang, Zhuang Liu, Laurens van der Maaten, Kilian Q. Weinberger
Title Densely Connected Convolutional Networks (DenseNet)

Dataset CIFAR-10, CIFAR-100, SVHN, ImageNet

Published 2017
Year
Methodology DenseNet This approach helps address several issues:

 Vanishing Gradient Problem: Dense connections improve gradient

flow during backpropagation.
 Feature Propagation: By using features from all previous layers,
DenseNet promotes efficient feature reuse.
 Parameter Efficiency: DenseNet substantially reduces the number of
parameters compared to other models with similar performance.

Algorithms For an L-layer DenseNet, this results in L(L+1)/2 connections between

layers.
Accuracy  CIFAR-10: Achieved lower error rates compared to previous models.
 CIFAR-100: Performed better than existing architectures.
 SVHN: DenseNet achieved competitive results.
 ImageNet: DenseNet performed comparably or better with fewer
parameters and less computational cost.

CIFAR-3:

Attributes Details

Authors Xavier Gastaldi

Title Shake-Shake Regularization
Dataset CIFAR-10, CIFAR-100
Published 2017
Year
Methodology This technique applies to multi-branch residual networks and replaces the
standard summation of parallel branches with a stochastic affine
combination. In each forward and backward pass, different weighted
combinations of the branches are applied, promoting regularization by
introducing stochasticity.

Algorithms This is particularly effective in residual networks (ResNets), which are prone
to overfitting when trained on smaller datasets.

Accuracy  CIFAR-10: The method achieves a test error rate of 2.86%,

improving over previous single-shot methods.
 CIFAR-100: The method attains a test error rate of 15.85%,
showcasing the robustness of the approach across different datasets.

CIFAR-4:

Attributes Details

Authors Sergey Zagoruyko, Nikos Komodakis

Title Wide Residual Networks (WRNs)
Dataset CIFAR-10, CIFAR-100, SVHN, COCO, ImageNet

Published 2016
Year
Methodology Traditional ResNets increase depth to improve accuracy, but the paper argues
that this approach leads to diminishing feature reuse and slows training.
Instead, WRNs decrease the depth while increasing the width (i.e., the
number of channels in each layer), maintaining or improving accuracy with
less computational cost.

Algorithms WRN-d-k, where "d" refers to the number of layers and "k" to the widening
factor (the factor by which the number of channels is increased).
Accuracy  CIFAR-10: WRN models achieve state-of-the-art accuracy,
outperforming even very deep ResNets.
 CIFAR-100: Similarly, WRNs deliver superior results compared to
previous models.

CIFAR-5:

Attributes Details

Authors David Berthelot, Nicholas Carlini, Ian Goodfellow, Avital Oliver, Nicolas
Papernot, Colin Raffel

Title : MixMatch: A Holistic Approach to Semi-Supervised Learning

Dataset CIFAR-10, STL-10
Published : 2019
Year

Methodology 1. Guessing low-entropy labels for data-augmented unlabeled samples.

2. Mixing labeled and unlabeled data using the MixUp technique,
where interpolation is applied between examples to smooth the
decision boundaries of the model.

Algorithms
Accuracy  CIFAR-10 (with 250 labels): MixMatch reduces the error rate from
38% to 11%, representing a 4x improvement over previous methods.
 STL-10: MixMatch cuts the error rate by half compared to previous
approaches.

CIFAR-6:

Attributes Details
Authors Ekin D. Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, Quoc V. Le

Title AutoAugment: Learning Augmentation Strategies from Data

Dataset CIFAR-10, CIFAR-100, SVHN, ImageNet, Oxford Flowers, Caltech-101,

Oxford-IIT Pets, FGVC Aircraft, Stanford Cars

Published 2019
Year

Methodology The paper introduces AutoAugment, a procedure to automatically search for

effective data augmentation policies to improve image classification
accuracy.

AutoAugment uses a search algorithm to identify the best augmentation

policies. The process involves: Search Space Definition: A policy is
composed of multiple sub-policies, and for each image in a mini-batch,
Search Algorithm: AutoAugment leverages reinforcement learning to
search for the optimal augmentation policy that maximizes the validation
accuracy on the target dataset.

Algorithms AutoAugment Algorithm: This approach trains a neural network to find

augmentation strategies (policies) that improve classification performance.

uses reinforcement learning to explore different augmentation

combinations, and the best policies are selected based on validation accuracy.

Accuracy CIFAR-10: Achieves an error rate of 1.5%, which is 0.6% better than
previous state-of-the-art models.

ImageNet: Obtains a Top-1 accuracy of 83.5%, improving over the previous

record of 83.1%.

CIFAR-7:
Attributes Details

Authors Fernanda Duplancic, Federico Davila-Kurban, Georgina V. Coldwell, Sol

Alonso, Daniela Galdeano
Title The Global Environment of Small Galaxy Systems

Dataset The study uses a catalogue of small galaxy systems that include compact and
locally isolated galaxy pairs, triplets, and groups consisting of four to six
members.

Published 2020
Year
Methodology Environmental Influence Analysis: The study explores the impact of
different environmental scales, Density Estimators: Fixed aperture
estimators are used to calculate local galaxy densities, Cosmic Web
Positioning: The research examines how these systems are positioned within
the larger cosmic web.
Algorithms a nearest neighbour approach helps to compare environments for different
system types
Accuracy Nearest neughbour approach can achieve the accuracy of 89.99% over the
records

CIFAR-8:

Attributes Details

Authors Mingxing Tan, Quoc V. Le

Title EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

Dataset ImageNet (for initial model training and evaluation), CIFAR-100, Oxford
Flowers, and other transfer learning datasets

Published 2019
Year

Methodology Model Scaling: The paper introduces a new method for model scaling,
 where the network’s depth, width, and resolution are uniformly scaled using a
compound coefficient, optimizing the network for both accuracy and
efficiency.

Neural Architecture Search (NAS): The authors use NAS to design a

baseline network, which is then scaled up using their method
Algorithms EfficientNet models use convolutional neural networks (ConvNets)
optimized through NAS and model scaling.

EfficientNet-B7 model is highlighted as the most advanced in the series.

Accuracy  EfficientNet-B7 achieves 84.3% top-1 accuracy on ImageNet.

 The model is also 8.4x smaller and 6.1x faster in inference compared
to other state-of-the-art ConvNets.
o CIFAR-100: 91.7% accuracy
o Oxford Flowers: 98.8% accuracy

CIFAR-9:

Attributes Details

Authors Kihyuk Sohn, David Berthelot, Chun-Liang Li, Zizhao Zhang, Nicholas
Carlini, Ekin D. Cubuk, Alex Kurakin, Han Zhang, Colin Raffel

Title FixMatch: Simplifying Semi-Supervised Learning with Consistency and

Confidence
Dataset CIFAR-10 (along with other benchmarks)

Published 2020
Year
Methodology  Semi-Supervised Learning (SSL): FixMatch simplifies SSL by
combining consistency regularization with pseudo-labeling.
Confidence Threshold: Pseudo-labels are retained only when the
model is confident in the prediction, Consistency Regularization: A
weakly-augmented version of an image is used to generate pseudo-
labels.
Algorithms pseudo-labeling and consistency regularization in a simplified framework,
making it more efficient than other SSL methods.
Accuracy  On CIFAR-10:
o 94.93% accuracy with 250 labeled examples.
o 88.61% accuracy with 40 labeled examples (just 4 labels per
class).

CIFAR-(10):

Attributes Details

Authors Jordan J. Bird, Ahmad Lotfi

Title CIFAKE: Image Classification and Explainable Identification of AI-
Generated Synthetic Images

Dataset CIFAKE dataset (synthetic images generated to match CIFAR-10 classes)

Published 2023
Year
Methodology The study creates a synthetic dataset using latent diffusion, mimicking
CIFAR-10, to classify real vs AI-generated images. A CNN was trained on
both real and synthetic images. Hyperparameter tuning was applied across 36
network topologies to optimize the model.

Algorithms Convolutional Neural Network (CNN), Gradient Class Activation Mapping

(Grad-CAM) for explainability.

Accuracy The CNN achieved 92.98% accuracy in classifying real and fake images