K-means

K-Means Clustering is an Unsupervised Learning algorithm, which groups the unlabeled dataset into
different clusters. Here K defines the number of pre-defined clusters that need to be created in the
process, as if K=2, there will be two clusters, and for K=3, there will be three clusters, and so on.

It is an iterative algorithm that divides the unlabeled dataset into k different clusters in such a way
that each dataset belongs only one group that has similar properties.

It allows us to cluster the data into different groups and a convenient way to discover the categories
of groups in the unlabeled dataset on its own without the need for any training.

It is a centroid-based algorithm, where each cluster is associated with a centroid. The main aim of
this algorithm is to minimize the sum of distances between the data point and their corresponding
clusters.

The algorithm takes the unlabeled dataset as input, divides the dataset into k-number of clusters,
and repeats the process until it does not find the best clusters. The value of k should be
predetermined in this algorithm.

The k-means clustering algorithm mainly performs two tasks:

o Determines the best value for K center points or centroids by an iterative process.

o Assigns each data point to its closest k-center. Those data points which are near to the
particular k-center, create a cluster.

Hence each cluster has datapoints with some commonalities, and it is away from other clusters.

The below diagram explains the working of the K-means Clustering Algorithm:

It’s a pretty fast and efficient method, but it works best when the clusters are distinct and not too
mixed up. One challenge, though, is figuring out the right number of clusters (K) beforehand. Plus, if
there’s a lot of noise or overlap in the data, K Means might not perform as well.
Optimization plays a crucial role in the k-means clustering algorithm. The goal of the optimization
process is to find the best set of centroids that minimizes the sum of squared distances between
each data point and its closest centroid.

How does the K-Means Algorithm Work?

The working of the K-Means algorithm is explained in the below steps:

Step-1: Select the number K to decide the number of clusters.

Step-2: Select random K points or centroids. (It can be other from the input dataset).

Step-3: Assign each data point to their closest centroid, which will form the predefined K clusters.

Step-4: Calculate the variance and place a new centroid of each cluster.

Step-5: Repeat the third steps, which means reassign each datapoint to the new closest centroid of
each cluster.

Step-6: If any reassignment occurs, then go to step-4 else go to FINISH.

Step-7: The model is ready.

Example from youtube

Advantages of K-means

1. Simple and easy to implement: The k-means algorithm is easy to understand and implement,
making it a popular choice for clustering tasks.

2. Fast and efficient: K-means is computationally efficient and can handle large datasets with
high dimensionality.

3. Scalability: K-means can handle large datasets with many data points and can be easily scaled
to handle even larger datasets.

4. Flexibility: K-means can be easily adapted to different applications and can be used with
varying metrics of distance and initialization methods.

Disadvantages of K-Means

1. Sensitivity to initial centroids: K-means is sensitive to the initial selection of centroids and
can converge to a suboptimal solution.

2. Requires specifying the number of clusters: The number of clusters k needs to be specified
before running the algorithm, which can be challenging in some applications.

3. Sensitive to outliers: K-means is sensitive to outliers, which can have a significant impact on
the resulting clusters.
Applications of K-Means Clustering

Here are some interesting ways K-means clustering is put to work across different fields:

 Distance Measures

At the heart of K-Means clustering is the concept of distance. Euclidean distance, for example, is a
simple straight-line measurement between points and is commonly used in many applications.
Manhattan distance, however, follows a grid-like path, much like how you'd navigate city streets.
Squared Euclidean distance makes calculations easier by squaring the values, while cosine distance is
handy when working with text data because it measures the angle between data vectors. Picking the
right distance measure really depends on what kind of problem you’re solving and the nature of your
data.

 K-Means for Geyser Eruptions

K-Means clustering has even been applied to studying the eruptions of the Old Faithful geyser in
Yellowstone. The data collected includes eruption duration and the waiting time between eruptions.
By clustering this information, researchers can uncover patterns that help predict the geyser’s
behavior. For instance, you might find clusters of similar eruption durations and intervals, which
could improve predictions for future eruptions.

 Customer Segmentation

One of the most popular uses of K-means clustering is for customer segmentation. From banks to e-
commerce, businesses use K-means clustering customer segmentation to group customers based on
their behaviors. For example, in telecom or sports industries, companies can create targeted
marketing campaigns by understanding different customer segments better. This allows for
personalized offers and communications, boosting customer engagement and satisfaction.

 Document Clustering

When dealing with a vast collection of documents, K-Means can be a lifesaver. It groups similar
documents together based on their content, which makes it easier to manage and retrieve relevant
information. For instance, if you have thousands of research papers, clustering can quickly help you
find related studies, improving both organization and efficiency in accessing valuable information.

 Image Segmentation

In image processing, K-Means clustering is commonly used to group pixels with similar colors, which
divides the image into distinct regions. This is incredibly helpful for tasks like object detection and
image enhancement. For instance, clustering can help separate objects within an image, making
analysis and processing more accurate. It’s also widely used to extract meaningful features from
images in various visual tasks.

 Recommendation Engines

K-Means clustering also plays a vital role in recommendation systems. Say you want to suggest new
songs to a listener based on their past preferences; clustering can group similar songs together,
helping the system provide personalized suggestions. By clustering content that shares similar
features, recommendation engines can deliver a more tailored experience, helping users discover
new songs that match their taste.
 K-Means for Image Compression

K-Means can even help with image compression by reducing the number of colors in an image while
keeping the visual quality intact. K-Means reduces the image size without losing much detail by
clustering similar colors and replacing the pixels with the average of their cluster. It’s a practical
method for compressing images for more accessible storage and transmission, all while maintaining
visual clarity.