K Means is one of the most popular Unsupervised Machine Learning Algorithms used for solving classification problems in data science, making it a crucial skill for those aspiring to excel in a data scientist role. K Means segregates unlabeled data into various groups, known as clusters, by identifying similar features and common patterns within the dataset. This tutorial aims to provide a comprehensive understanding of clustering, with a specific focus on the K Means clustering algorithm and its implementation in Python. By delving into the nuances of K means clustering in Python, you will gain valuable insights into how to effectively organize and analyze data. Additionally, the tutorial will guide you on determining the optimum number of clusters for a dataset, enhancing your ability to apply K means clustering in practical scenarios.
Learning Objectives
This article was published as a part of the Data Science Blogathon.
Suppose we have N number of unlabeled multivariate datasets of various animals like dogs, cats, birds, etc. The technique of segregating these datasets into various groups on the basis of having similar features and characteristics is called clustering.
The groups being formed are known as clusters. Clustering techniques are used in various fields, such as image recognition, spam filtering, etc. They are also used in unsupervised learning algorithms in machine learning, as they can segregate multivariate data into various groups, without any supervisor, on the basis of common patterns hidden inside the datasets.
The k-means clustering algorithm is an Iterative algorithm that divides a group of n datasets into k different clusters based on the similarity and their mean distance from the centroid of that particular subgroup/ formed.
K, here is the pre-defined number of clusters to be formed by the algorithm. If K=3, It means the number of clusters to be formed from the dataset is 3.
The implementation and working of the K-Means algorithm are explained in the steps below:
Step 1: Select the value of K to decide the number of clusters (n_clusters) to be formed.
Step 2: Select random K points that will act as cluster centroids (cluster_centers).
Step 3: Assign each data point, based on their distance from the randomly selected points (Centroid), to the nearest/closest centroid, which will form the predefined clusters.
Step 4: Place a new centroid of each cluster.
Step 5: Repeat step no.3, which reassigns each datapoint to the new closest centroid of each cluster.
Step 6: If any reassignment occurs, then go to step 4; else, go to step 7.
Step 7: Finish
K-Means clustering is a method in Python for grouping a set of data points into distinct clusters. The goal is to partition the data in such a way that points in the same cluster are more similar to each other than to points in other clusters. Here’s a breakdown of how to use K Means clustering in Python:
Import Libraries:
from sklearn.cluster import KMeansPrepare Your Data:
import numpy as np
data = np.array([[1, 2], [5, 8], [1.5, 1.8], [8, 8], [1, 0.6], [9, 11]])
Choose the Number of Clusters (K):
kmeans = KMeans(n_clusters=2)
Fit the Model:
kmeans.fit(data)
Get Results:
# Get the cluster centers
centroids = kmeans.cluster_centers_
# Get the labels (cluster assignments for each data point)
labels = kmeans.labels_
In this example, n_clusters=2 indicates that we want the algorithm to find two clusters. The fit method trains the model, and then you can access information about the clusters, such as the cluster centers and labels. Visualizing the results can be helpful to see how well the algorithm grouped your data points.
Here is Step-by-Step Explanation that How K-means Clustering in Python works:
Initialize Centroids:
Assign Data Points to Nearest Centroid:
Update Centroids:
Repeat:
Final Result:
Here’s a simple example using Python with the popular machine learning library, scikit-learn:
from sklearn.cluster import KMeans
import numpy as np
# Sample data
data = np.array([[1, 2], [5, 8], [1.5, 1.8], [8, 8], [1, 0.6], [9, 11]])
# Specify the number of clusters (K)
kmeans = KMeans(n_clusters=2)
# Fit the data to the algorithm
kmeans.fit(data)
# Get the cluster centroids and labels
centroids = kmeans.cluster_centers_
labels = kmeans.labels_
print("Centroids:")
print(centroids)
print("Labels:")
print(labels)
Step 1: Let’s choose the number k of clusters, i.e., K=2, to segregate the dataset and put them into different respective clusters. We will choose some random 2 points which will act as centroids to form the cluster.
Step 2: Now, we will assign each data point to a scatter plot based on its distance from the closest K-point or centroid. It will be done by drawing a median between both the centroids.
Step 3: points on the left side of the line are near the blue centroid, and points to the right of the line are close to the yellow centroid. The left forms a cluster with the blue centroid, and the right one with the yellow centroid.
Step 4: Repeat the process by choosing a new centroid. To choose the new centroids, we will find the new center of gravity of these centroids, as depicted below.
Step 5: Next, we will reassign each data point to the new centroid. We will repeat the same process as above (using a median line). The yellow data point on the blue side of the median line will be included in the blue cluster.
Step 6: As reassignment has occurred, we will repeat the above step of finding new k centroids.
Step 7: We will repeat the above process of finding the center of gravity of k centroids, as depicted below.
Step 8: After finding the new k centroids, we will again draw the median line and reassign the data points, like the above steps.
Step 9: We will finally segregate points based on the median line, such that two groups are being formed and no dissimilar point is to be included in a single group.
The final cluster formed is like this:
The number of clusters that we choose for the algorithm shouldn’t be random. Each and every cluster is formed by calculating and comparing the mean distances of each data point within a cluster from its centroid.
We can choose the right number of clusters with the help of the Within-Cluster-Sum-of-Squares (WCSS) method. WCSS stands for the sum of the squares of distances of the data points in each and every cluster from its centroid.
The main idea is to minimize the distance (e.g., euclidean distance) between the data points and the centroid of the clusters. The process is iterated until we reach a minimum value for the sum of distances.
Here are the steps to follow in order to find the optimal number of clusters using the elbow method:
Step 1: Execute the K-means clustering on a given dataset for different K values (ranging from 1-10).
Step 2: For each value of K, calculate the WCSS value.
Step 3: Plot a graph/curve between WCSS values and the respective number of clusters K.
Step 4: The sharp point of bend or a point (looking like an elbow joint) of the plot, like an arm, will be considered as the best/optimal value of K.
Python Implementation:
Importing relevant libraries
import numpy as np
import pandas as pd
import statsmodels.api as sm
import matplotlib.pyplot as plt
import seaborn as sns
sns.set()
from sklearn.cluster import KMeansLoading the data
data = pd.read_csv('Countryclusters.csv')
data
Plotting the data
.png)
Selecting the feature
x = data.iloc[:,1:3] # 1t for rows and second for columns
x
Clustering
kmeans = KMeans(3)
means.fit(x)Clustering results
identified_clusters = kmeans.fit_predict(x)
identified_clustersarray([1, 1, 0, 0, 0, 2])data_with_clusters = data.copy()
data_with_clusters['Clusters'] = identified_clusters
plt.scatter(data_with_clusters['Longitude'],data_with_clusters['Latitude'],c=data_with_clusters['Clusters'],cmap='rainbow').png)
wcss=[]
for i in range(1,7):
kmeans = KMeans(i)
kmeans.fit(x)
wcss_iter = kmeans.inertia_
wcss.append(wcss_iter)
number_clusters = range(1,7)
plt.plot(number_clusters,wcss)
plt.title('The Elbow title')
plt.xlabel('Number of clusters')
plt.ylabel('WCSS').png)
This method shows that 3 is a good number of clusters.
To summarize everything that has been stated so far, k means clustering in python is a widely used unsupervised machine learning technique that enables the grouping of data into clusters based on similarity. It is a simple algorithm that can be applied to various domains and data types, including image and text data. k-means can be used for a variety of purposes. We can use it to perform dimensionality reduction also, where each transformed feature is the distance of the point from a cluster center.
Key Takeaways
A. n_init is an integer and represents the number of times or the number of iterations the k-means algorithm will be run independently.
A. Advantages of K-means Clustering include its simplicity, scalability, and versatility, as it can be applied to a wide range of data types. Disadvantages include its sensitivity to the initial placement of centroids and its limitations in handling complex, non-linear data. k-means is also sensitive to outliers.
A. In K-Means, random_state represents random number generation for centroid initialization. We can use an Integer value to make the randomness fixed or constant. Also, it helps when we want to produce the same clusters every time.
The media shown in this article are not owned by Analytics Vidhya and is used at the Author’s discretion.
Lorem ipsum dolor sit amet, consectetur adipiscing elit,
In part 2 - "What is K Means Algorithm" - you forgot a very important step, which is to determine the new cluster center by computing the average of the assigned points
hi it is a very good article. Please can you also write an article with coding how to choose cluster head based on battery power in each cluster?
thanks for your helpful article I just wanted to say there are some parts that needs to be corrected : first one : 5. Python Implementation >> Clustering >> means -> kmeans second one : in section "Trying different method ( to find no .of clusters to be selected) WCSS and Elbow Method" after "for" loop indent is forgotten