Loading Curriculum...

Artificial Intelligence (AI) Machine Learning

Clustering

Lesson 13/19

Machine Learning • Clustering

Discover structure without labels.

K-means repeatedly assigns points to the closest centroid, then moves centroids to the mean of their assigned points. Watch inertia decrease as clusters stabilize.

Click the plot to see nearest cluster

Beginner friendly

What is Clustering?

Group similar things.

Clustering finds hidden groups in data when you don’t have labels. Instead of predicting “spam/not spam”, you discover “these points behave similarly”.

Real-life analogy

Think of sorting mixed-colored balls into boxes. You don’t know the correct labels—so you keep moving the “center of each box” until balls settle into the nearest one.

Visual diagram

The K-means loop

Assignment ↔ Update.

1) Assign

Each point joins the closest centroid.

2) Update

Each centroid moves to the mean of its group.

Repeat

Inertia acts like an energy score you’re lowering.

Core intuition

Why the algorithm exists

Because “similarity” needs a distance rule.

Clustering starts by assuming you can measure how “close” two points are. In K-means, that’s usually Euclidean distance.

What “centroid” means

A centroid is the center of a cluster (mean location). K-means moves centers until they match the points assigned to them.

Intuition hint

Inertia = “tightness” score

It grows when points are far from their cluster centers. The algorithm tries to push points closer.

Interactive simulation

K-means Clustering Lab

Step through assign → update and watch centroids converge.

step: 0/18

Loading Lab...

Controls

Number of clusters (k)3

Iterations (steps)18

Animation speed550ms

Initialization

Cohesion

0.00

Stability

1.00

Learning Playground

PatternBlobs

Points110

True clusters3

Separation66%

Noise24%

Upload CSV

Chart

Cluster quality snapshot

Cohesion

0.00

Separation

0.00

Silhouette proxy

0.00

Approximate “good clusters” score: high when points are closer to their own centroid than others.

Mathematical explanation

K-means objective (the intuition behind inertia)

Math that matches what you see moving.

What we minimize

\min_{\{\mu_j\}} \; J=\sum_{i=1}^{N} \sum_{j=1}^{K} r_{ij} \lVert x_i-\mu_j \rVert^2

Here, $r_{ij}$ is 1 if point $x_i$ is assigned to centroid $\mu_j$ . Inertia chart is this energy—lower means tighter groups.

The two alternating steps

\text{(Assignment)}\quad r_{ij}=1\;\text{if } j=\arg\min_k \lVert x_i-\mu_k\rVert^2

\text{(Update)}\quad \mu_j=\frac{1}{|C_j|}\sum_{x_i\in C_j} x_i

Now in the lab

Current k: 3

Current inertia: 0.000

What improves: points get closer to their centroid.

Step-by-step workflow

Input → Processing → Learning → Prediction → Output

Input

Unlabeled points

Processing

Measure distances to centroids

Learning

Repeat: assign → update centroids

Prediction

Assign each point to a cluster

Output

Groups + centroids

Real-world applications

Clustering in the real world

Why teams use unlabeled grouping.

Customer segmentation

Group users by behavior patterns to personalize offers.

Image compression & grouping

Cluster similar pixels/features to reduce redundancy.

Anomaly triage

Separate “weird” points into small clusters for investigation.

Why it’s popular

• Simple and fast.
• Great baseline when you don’t have labels.
• Produces interpretable “centers” for downstream logic.

Visualization

Heat regions + cluster transitions

Where each centroid wins.

What “regions” mean

Each cell in the heatmap belongs to the nearest centroid. When centroids move, the regions reshape.

Objective dynamics

Inertia trend: —

Stability: 1.00 (higher = fewer assignment changes vs previous step)

Code implementation

Beginner-friendly K-means

Copy-friendly Python example with the same logic as the lab.

Python (from scratch)

python

# K-Means clustering (from scratch)
import random
import math

def dist2(a, b):
    return (a[0]-b[0])**2 + (a[1]-b[1])**2

def kmeans(points, k, steps, seed=0):
    random.seed(seed)
    # init: pick random points as centroids
    centroids = random.sample(points, k)

    for _ in range(steps):
        # 1) assign each point to nearest centroid
        assign = []
        for p in points:
            best = 0
            bestD = float("inf")
            for j in range(k):
                d = dist2(p, centroids[j])
                if d < bestD:
                    bestD = d
                    best = j
            assign.append(best)

        # 2) update centroids (mean of assigned points)
        nextC = [[0.0, 0.0] for _ in range(k)]
        counts = [0 for _ in range(k)]
        for (p, a) in zip(points, assign):
            nextC[a][0] += p[0]
            nextC[a][1] += p[1]
            counts[a] += 1

        for j in range(k):
            if counts[j] > 0:
                centroids[j] = [nextC[j][0]/counts[j], nextC[j][1]/counts[j]]

    return centroids, assign

Match the lab to the code

• Lab step = one assign→update loop.
• Inertia chart corresponds to sum of squared distances.
• Clicking the plot chooses the nearest centroid at that step.

Common mistakes & limitations

Why clustering can fail

Spot the symptoms in the visualization.

Choosing wrong k

Too small merges groups; too large fragments them. Try changing k and watch inertia/stability.

Non-spherical clusters

Rings or moons don’t fit “mean = best center”. Switch pattern to Rings to see it.

Feature scaling issues

If features have different scales, distance becomes misleading (units matter!).

Failure case demo

K-means optimizes squared distance to centroids—so it struggles when the “true groups” are curved or density-based.

Final summary

Takeaways + cheat sheet

Everything important, quickly.

Key takeaways

• K-means alternates assign and update steps.
• Inertia is the energy it tries to reduce.
• k and initialization strongly affect outcomes.

Important formula

J=\sum_{i=1}^N\min_j\lVert x_i-\mu_j\rVert^2

That “min” is the “nearest centroid” assignment step you see in the regions.

Interview questions

• What’s the difference between inertia and silhouette?
• Why does initialization matter in K-means?
• When should you avoid K-means?

Loading Lab...

Machine Learning • Clustering

Discover structure without labels.

K-means repeatedly assigns points to the closest centroid, then moves centroids to the mean of their assigned points. Watch inertia decrease as clusters stabilize.

Click the plot to see nearest cluster

Beginner friendly

What is Clustering?

Group similar things.

Clustering finds hidden groups in data when you don’t have labels. Instead of predicting “spam/not spam”, you discover “these points behave similarly”.

Real-life analogy

Think of sorting mixed-colored balls into boxes. You don’t know the correct labels—so you keep moving the “center of each box” until balls settle into the nearest one.

Visual diagram

The K-means loop

Assignment ↔ Update.

1) Assign

Each point joins the closest centroid.

2) Update

Each centroid moves to the mean of its group.

Repeat

Inertia acts like an energy score you’re lowering.

Core intuition

Why the algorithm exists

Because “similarity” needs a distance rule.

Clustering starts by assuming you can measure how “close” two points are. In K-means, that’s usually Euclidean distance.

What “centroid” means

A centroid is the center of a cluster (mean location). K-means moves centers until they match the points assigned to them.

Intuition hint

Inertia = “tightness” score

It grows when points are far from their cluster centers. The algorithm tries to push points closer.

Interactive simulation

K-means Clustering Lab

Step through assign → update and watch centroids converge.

step: 0/18

Loading Lab...

Controls

Number of clusters (k)3

Iterations (steps)18

Animation speed550ms

Initialization

Cohesion

0.00

Stability

1.00

Learning Playground

PatternBlobs

Points110

True clusters3

Separation66%

Noise24%

Upload CSV

Chart

Cluster quality snapshot

Cohesion

0.00

Separation

0.00

Silhouette proxy

0.00

Approximate “good clusters” score: high when points are closer to their own centroid than others.

Mathematical explanation

K-means objective (the intuition behind inertia)

Math that matches what you see moving.

What we minimize

\min_{\{\mu_j\}} \; J=\sum_{i=1}^{N} \sum_{j=1}^{K} r_{ij} \lVert x_i-\mu_j \rVert^2

Here, $r_{ij}$ is 1 if point $x_i$ is assigned to centroid $\mu_j$ . Inertia chart is this energy—lower means tighter groups.

The two alternating steps

\text{(Assignment)}\quad r_{ij}=1\;\text{if } j=\arg\min_k \lVert x_i-\mu_k\rVert^2

\text{(Update)}\quad \mu_j=\frac{1}{|C_j|}\sum_{x_i\in C_j} x_i

Now in the lab

Current k: 3

Current inertia: 0.000

What improves: points get closer to their centroid.

Step-by-step workflow

Input → Processing → Learning → Prediction → Output

Input

Unlabeled points

Processing

Measure distances to centroids

Learning

Repeat: assign → update centroids

Prediction

Assign each point to a cluster

Output

Groups + centroids

Real-world applications

Clustering in the real world

Why teams use unlabeled grouping.

Customer segmentation

Group users by behavior patterns to personalize offers.

Image compression & grouping

Cluster similar pixels/features to reduce redundancy.

Anomaly triage

Separate “weird” points into small clusters for investigation.

Why it’s popular

• Simple and fast.
• Great baseline when you don’t have labels.
• Produces interpretable “centers” for downstream logic.

Visualization

Heat regions + cluster transitions

Where each centroid wins.

What “regions” mean

Each cell in the heatmap belongs to the nearest centroid. When centroids move, the regions reshape.

Objective dynamics

Inertia trend: —

Stability: 1.00 (higher = fewer assignment changes vs previous step)

Code implementation

Beginner-friendly K-means

Copy-friendly Python example with the same logic as the lab.

Python (from scratch)

python

# K-Means clustering (from scratch)
import random
import math

def dist2(a, b):
    return (a[0]-b[0])**2 + (a[1]-b[1])**2

def kmeans(points, k, steps, seed=0):
    random.seed(seed)
    # init: pick random points as centroids
    centroids = random.sample(points, k)

    for _ in range(steps):
        # 1) assign each point to nearest centroid
        assign = []
        for p in points:
            best = 0
            bestD = float("inf")
            for j in range(k):
                d = dist2(p, centroids[j])
                if d < bestD:
                    bestD = d
                    best = j
            assign.append(best)

        # 2) update centroids (mean of assigned points)
        nextC = [[0.0, 0.0] for _ in range(k)]
        counts = [0 for _ in range(k)]
        for (p, a) in zip(points, assign):
            nextC[a][0] += p[0]
            nextC[a][1] += p[1]
            counts[a] += 1

        for j in range(k):
            if counts[j] > 0:
                centroids[j] = [nextC[j][0]/counts[j], nextC[j][1]/counts[j]]

    return centroids, assign

Match the lab to the code

• Lab step = one assign→update loop.
• Inertia chart corresponds to sum of squared distances.
• Clicking the plot chooses the nearest centroid at that step.

Common mistakes & limitations

Why clustering can fail

Spot the symptoms in the visualization.

Choosing wrong k

Too small merges groups; too large fragments them. Try changing k and watch inertia/stability.

Non-spherical clusters

Rings or moons don’t fit “mean = best center”. Switch pattern to Rings to see it.

Feature scaling issues

If features have different scales, distance becomes misleading (units matter!).

Failure case demo

K-means optimizes squared distance to centroids—so it struggles when the “true groups” are curved or density-based.

Final summary

Takeaways + cheat sheet

Everything important, quickly.

Key takeaways

• K-means alternates assign and update steps.
• Inertia is the energy it tries to reduce.
• k and initialization strongly affect outcomes.

Important formula

J=\sum_{i=1}^N\min_j\lVert x_i-\mu_j\rVert^2

That “min” is the “nearest centroid” assignment step you see in the regions.

Interview questions

• What’s the difference between inertia and silhouette?
• Why does initialization matter in K-means?
• When should you avoid K-means?