Loading Curriculum...

Artificial Intelligence (AI) Machine Learning

Classification

Lesson 6/19

Machine Learning • Classification

Classify inputs with a moving decision boundary.

Logistic regression learns a line that separates classes by turning raw features into probabilities. Play the simulation to watch weights update and the boundary refine.

Click the plot to test inference

Beginner friendly

What is Classification?

Turn numbers into decisions.

Classification is the task of taking an input (like an email’s text features or a patient’s test results) and outputting a category (spam/not spam, disease/not disease).

Real-life analogy

Imagine a bouncer with two doors. A simple model watches your features and decides which door you should walk through—then it gives a confidence based on how close you are to the boundary.

Visual diagram

From input to class

A mental model you can reuse.

Input

Features

→

Model

Probabilities

→

Output

Class

Key idea

A classifier doesn’t just pick a label. It learns a boundary where the meaning shifts from class 0 to class 1.

Core intuition

Why do we need a model?

Because we want decisions that generalize—not guesses.

You’re given examples: some points are class 1, others are class 0. The job of the model is to learn a rule that predicts the label for new, unseen points.

What logistic regression does

• Converts a weighted sum into a probability.
• Uses gradient descent to reduce classification error.
• Produces a decision boundary you can visualize.

Intuition animation

Weights move → boundary rotates

Each training step nudges weights to make misclassified points less likely.

A quick check

In the simulation below, if the loss is going down, the boundary is becoming more helpful.

Interactive simulation

Logistic Regression Lab

Decision boundary learns a line

step: 0/70

Loading Lab...

Controls

Learning rate0.14

Training steps70

Inference threshold50%

Animation speed500ms

Precision

0.64

Recall

1.00

Learning Playground

PatternLinear

Points96

Separation68%

Feature noise26%

Label noise12%

Class balance50% class 1

Upload CSV

Chart

Loss drops when predictions match labels more often. If it flattens too early, the model may underfit.

Weight trajectory (bypass intuition → see movement)

The point moves as gradient descent updates weights. A better boundary often correlates with decreasing loss.

Mathematical explanation

Math intuition (without the intimidation)

Each formula maps to a visual part of the simulation.

1) Sigmoid = “how likely class 1 is”

p(y=1\mid x)=\sigma(z)

where

z=b+w_1x_1+w_2x_2

\sigma(z)=\frac{1}{1+e^{-z}}

2) Cross-entropy = “penalize wrong probabilities”

\mathcal{L}=-\frac{1}{N}\sum_i \left[y_i\log(p_i)+(1-y_i)\log(1-p_i)\right]

If a point is class 1 but the model says probability is tiny, the log penalty is large—so loss shoots up.

3) Gradient descent = “nudge weights”

\theta \leftarrow \theta - \eta\,\nabla_\theta \mathcal{L}

Think of $\eta$ as your step size (learning rate). In the lab, moving the learning-rate slider changes how fast the boundary tries to improve.

What you’re seeing right now

Weightsb=0.00 • w1=0.00 • w2=0.00

Current loss0.0000

Boundary meaningp ≈ 0.50

Step-by-step workflow

Input → Processing → Learning → Prediction → Output

Input

Features (x1, x2) + labels (y)

Processing

Compute probabilities with sigmoid

Learning

Update weights using gradient descent

Prediction

Compare prob vs decision threshold

Output

Class + confidence

Real-world applications

Where classification shows up

Industry use-cases + why teams choose it.

Spam & phishing detection

Email features → probability spam → threshold by risk tolerance.

Medical decision support

Lab values → disease likelihood for triage.

Credit scoring

Customer features → approve/decline with confidence.

Why companies use models like this

• Fast to train and easy to monitor.
• Probabilities help you handle different business costs.
• Works well when class boundaries are close to linear.

Visualization

Feature space + state transitions

A few extra visuals to deepen intuition.

Animated decision boundary checkpoints

Tap a checkpoint to jump the simulation to that training moment.

Confusion view (at threshold)

Changing the inference threshold shifts how many positives you predict—use this to reason about false positives vs false negatives.

Code implementation

Beginner-friendly logistic regression

Copy-friendly Python example.

Python (from scratch)

python

# Logistic Regression (binary classification) - tiny gradient descent demo
import math
import random

def sigmoid(z):
    return 1 / (1 + math.exp(-z))

def cross_entropy(points, w):
    # w = [b, w1, w2]
    eps = 1e-8
    total = 0.0
    for x1, x2, y in points:
        z = w[0] + w[1]*x1 + w[2]*x2
        p = sigmoid(z)
        total += -(y*math.log(p+eps) + (1-y)*math.log(1-p+eps))
    return total / len(points)

def grad(points, w):
    gb = gw1 = gw2 = 0.0
    n = len(points)
    for x1, x2, y in points:
        z = w[0] + w[1]*x1 + w[2]*x2
        p = sigmoid(z)
        err = p - y
        gb += err
        gw1 += err * x1
        gw2 += err * x2
    return [gb/n, gw1/n, gw2/n]

# Synthetic data: (x1, x2, y) where y in {0,1}
points = [(0.1,0.2,0), (0.9,0.8,1)]  # replace with your dataset

lr = 0.2
steps = 100
w = [0.0, 0.0, 0.0]

for t in range(steps):
    loss = cross_entropy(points, w)
    g = grad(points, w)
    w = [w[0]-lr*g[0], w[1]-lr*g[1], w[2]-lr*g[2]]
    if t % 10 == 0:
        print(t, "loss=", loss, "w=", w)

How to read this

• `sigmoid(z)` turns a score into probability.
• `cross_entropy` is the loss you see in the chart.
• `grad()` computes “which direction makes things better”.
• `w ← w - lr * grad` is the update step.

Common mistakes & limitations

When logistic regression doesn’t shine

Visual failure modes you can recognize.

Non-linear boundaries (XOR / circles)

A linear model can’t represent curved boundaries—switch Pattern to “Noisy XOR” to see it.

Bad learning rate

Too high → unstable updates; too low → slow progress (loss chart helps debug).

Class imbalance

With imbalanced classes, a 0.5 threshold may hide false negatives or false positives.

Quick diagnostic

If the loss is still high after many steps, either the data pattern isn’t linearly separable, or the learning rate is making progress inefficient.

Final summary

Takeaways you can reuse anywhere

Cheat sheet + interview-friendly questions.

Key takeaways

• Probabilities come from sigmoid of a weighted sum.
• Loss tells you if the boundary is getting better.
• Gradient descent updates weights step-by-step.

Important formulas

Sigmoid

\sigma(z)=\frac{1}{1+e^{-z}}

Loss

\mathcal{L}=-\frac{1}{N}\sum_i\left[y_i\log p_i+(1-y_i)\log(1-p_i)\right]

Interview questions

• Why do we use a probability threshold?
• What happens when learning rate is too high?
• When does logistic regression underfit?

Loading Lab...

Machine Learning • Classification

Classify inputs with a moving decision boundary.

Logistic regression learns a line that separates classes by turning raw features into probabilities. Play the simulation to watch weights update and the boundary refine.

Click the plot to test inference

Beginner friendly

What is Classification?

Turn numbers into decisions.

Classification is the task of taking an input (like an email’s text features or a patient’s test results) and outputting a category (spam/not spam, disease/not disease).

Real-life analogy

Imagine a bouncer with two doors. A simple model watches your features and decides which door you should walk through—then it gives a confidence based on how close you are to the boundary.

Visual diagram

From input to class

A mental model you can reuse.

Input

Features

→

Model

Probabilities

→

Output

Class

Key idea

A classifier doesn’t just pick a label. It learns a boundary where the meaning shifts from class 0 to class 1.

Core intuition

Why do we need a model?

Because we want decisions that generalize—not guesses.

You’re given examples: some points are class 1, others are class 0. The job of the model is to learn a rule that predicts the label for new, unseen points.

What logistic regression does

• Converts a weighted sum into a probability.
• Uses gradient descent to reduce classification error.
• Produces a decision boundary you can visualize.

Intuition animation

Weights move → boundary rotates

Each training step nudges weights to make misclassified points less likely.

A quick check

In the simulation below, if the loss is going down, the boundary is becoming more helpful.

Interactive simulation

Logistic Regression Lab

Decision boundary learns a line

step: 0/70

Loading Lab...

Controls

Learning rate0.14

Training steps70

Inference threshold50%

Animation speed500ms

Precision

0.64

Recall

1.00

Learning Playground

PatternLinear

Points96

Separation68%

Feature noise26%

Label noise12%

Class balance50% class 1

Upload CSV

Chart

Loss drops when predictions match labels more often. If it flattens too early, the model may underfit.

Weight trajectory (bypass intuition → see movement)

The point moves as gradient descent updates weights. A better boundary often correlates with decreasing loss.

Mathematical explanation

Math intuition (without the intimidation)

Each formula maps to a visual part of the simulation.

1) Sigmoid = “how likely class 1 is”

p(y=1\mid x)=\sigma(z)

where

z=b+w_1x_1+w_2x_2

\sigma(z)=\frac{1}{1+e^{-z}}

2) Cross-entropy = “penalize wrong probabilities”

\mathcal{L}=-\frac{1}{N}\sum_i \left[y_i\log(p_i)+(1-y_i)\log(1-p_i)\right]

If a point is class 1 but the model says probability is tiny, the log penalty is large—so loss shoots up.

3) Gradient descent = “nudge weights”

\theta \leftarrow \theta - \eta\,\nabla_\theta \mathcal{L}

Think of $\eta$ as your step size (learning rate). In the lab, moving the learning-rate slider changes how fast the boundary tries to improve.

What you’re seeing right now

Weightsb=0.00 • w1=0.00 • w2=0.00

Current loss0.0000

Boundary meaningp ≈ 0.50

Step-by-step workflow

Input → Processing → Learning → Prediction → Output

Input

Features (x1, x2) + labels (y)

Processing

Compute probabilities with sigmoid

Learning

Update weights using gradient descent

Prediction

Compare prob vs decision threshold

Output

Class + confidence

Real-world applications

Where classification shows up

Industry use-cases + why teams choose it.

Spam & phishing detection

Email features → probability spam → threshold by risk tolerance.

Medical decision support

Lab values → disease likelihood for triage.

Credit scoring

Customer features → approve/decline with confidence.

Why companies use models like this

• Fast to train and easy to monitor.
• Probabilities help you handle different business costs.
• Works well when class boundaries are close to linear.

Visualization

Feature space + state transitions

A few extra visuals to deepen intuition.

Animated decision boundary checkpoints

Tap a checkpoint to jump the simulation to that training moment.

Confusion view (at threshold)

Changing the inference threshold shifts how many positives you predict—use this to reason about false positives vs false negatives.

Code implementation

Beginner-friendly logistic regression

Copy-friendly Python example.

Python (from scratch)

python

# Logistic Regression (binary classification) - tiny gradient descent demo
import math
import random

def sigmoid(z):
    return 1 / (1 + math.exp(-z))

def cross_entropy(points, w):
    # w = [b, w1, w2]
    eps = 1e-8
    total = 0.0
    for x1, x2, y in points:
        z = w[0] + w[1]*x1 + w[2]*x2
        p = sigmoid(z)
        total += -(y*math.log(p+eps) + (1-y)*math.log(1-p+eps))
    return total / len(points)

def grad(points, w):
    gb = gw1 = gw2 = 0.0
    n = len(points)
    for x1, x2, y in points:
        z = w[0] + w[1]*x1 + w[2]*x2
        p = sigmoid(z)
        err = p - y
        gb += err
        gw1 += err * x1
        gw2 += err * x2
    return [gb/n, gw1/n, gw2/n]

# Synthetic data: (x1, x2, y) where y in {0,1}
points = [(0.1,0.2,0), (0.9,0.8,1)]  # replace with your dataset

lr = 0.2
steps = 100
w = [0.0, 0.0, 0.0]

for t in range(steps):
    loss = cross_entropy(points, w)
    g = grad(points, w)
    w = [w[0]-lr*g[0], w[1]-lr*g[1], w[2]-lr*g[2]]
    if t % 10 == 0:
        print(t, "loss=", loss, "w=", w)

How to read this

• `sigmoid(z)` turns a score into probability.
• `cross_entropy` is the loss you see in the chart.
• `grad()` computes “which direction makes things better”.
• `w ← w - lr * grad` is the update step.

Common mistakes & limitations

When logistic regression doesn’t shine

Visual failure modes you can recognize.

Non-linear boundaries (XOR / circles)

A linear model can’t represent curved boundaries—switch Pattern to “Noisy XOR” to see it.

Bad learning rate

Too high → unstable updates; too low → slow progress (loss chart helps debug).

Class imbalance

With imbalanced classes, a 0.5 threshold may hide false negatives or false positives.

Quick diagnostic

If the loss is still high after many steps, either the data pattern isn’t linearly separable, or the learning rate is making progress inefficient.

Final summary

Takeaways you can reuse anywhere

Cheat sheet + interview-friendly questions.

Key takeaways

• Probabilities come from sigmoid of a weighted sum.
• Loss tells you if the boundary is getting better.
• Gradient descent updates weights step-by-step.

Important formulas

Sigmoid

\sigma(z)=\frac{1}{1+e^{-z}}

Loss

\mathcal{L}=-\frac{1}{N}\sum_i\left[y_i\log p_i+(1-y_i)\log(1-p_i)\right]

Interview questions

• Why do we use a probability threshold?
• What happens when learning rate is too high?
• When does logistic regression underfit?