Loading Curriculum...

Artificial Intelligence (AI) Machine Learning

EvaluationMetrics

Lesson 19/19

Machine Learning Lesson

Evaluation Metrics

Evaluation metrics are used to measure how well a machine learning model performs. They help assess whether the model is making accurate predictions and meeting the desired goals.

Interactive Simulation Lab

Real-time Learning Signal

Metric Alignment70%

Threshold Policy55%

Class Imbalance Awareness40%

Decision Quality

93%

Monitoring Coverage

97%

Error Visibility

50%

Business Alignment

86%

Professional insight: Choose metrics that reflect business impact, not just model convenience.

Key Ideas

- Model performance measurement
- Metric choice depends on task (classification, regression, clustering)
- Proper metrics guide better decisions
- Avoid misleading conclusions from single metrics

Learning Workflow

1.Identify the problem type (classification, regression, clustering)
2.Select appropriate evaluation metrics
3.Compute metrics on validation/test data
4.Interpret results and iterate on model improvements

Where It Is Used

Assessing classification modelsEvaluating regression accuracyMeasuring clustering quality

Professional Explanation

In production machine learning systems, this topic is not used in isolation. Teams combine data quality checks, controlled model complexity, strong validation discipline, and continuous monitoring to keep performance stable over time. The most reliable outcomes come from iterative experimentation, reproducible pipelines, and clear alignment between model metrics and real business impact.

Last Updated: 30 Mar, 2026

Why Evaluation Metrics Matter

Model Performance – Measures how well the model works
Different Tasks – Used for classification, regression and clustering
Right Metric Choice – Helps select the best way to evaluate a model
Better Decisions – Ensures the model meets its objectives

Classification Metrics

Classification problems aim to predict discrete categories.

Accuracy
Proportion of correct predictions out of all predictions.
```
Accuracy = Number_of_Correct_Predictions / Total_Number_of_Predictions
```
Note: Accuracy can be misleading on imbalanced datasets.
Precision
How many of the positive predictions are actually correct.
```
Precision = TP / (TP + FP)
```
Recall (Sensitivity)
How many actual positive cases were correctly identified.
```
Recall = TP / (TP + FN)
```

F1 Score

Harmonic mean of precision and recall.

F1 = 2 * (Precision * Recall) / (Precision + Recall)

Logarithmic Loss (Log Loss)
Penalises low probability assigned to the correct class.
```
LogLoss = - (1/N) * Σ_i Σ_j y_ij * log(p_ij)
```
ROC Curve & AUC
Measures the ability of the model to discriminate between classes.

ROC‑Related Rates

True Positive Rate (TPR) = TP / (TP + FN)
True Negative Rate (TNR) = TN / (TN + FP)
False Positive Rate (FPR) = FP / (FP + TN)
False Negative Rate (FNR) = FN / (FN + TP)

Confusion Matrix

A 2×2 matrix for binary classification:

               Predicted No   Predicted Yes
Actual No          TN               FP
Actual Yes         FN               TP

Regression Metrics

Mean Absolute Error (MAE)
```
MAE = (1/N) * Σ_j |y_j - ŷ_j|
```
Mean Squared Error (MSE)
```
MSE = (1/N) * Σ_j (y_j - ŷ_j)^2
```
Root Mean Squared Error (RMSE)
```
RMSE = sqrt(MSE)
```

Root Mean Squared Logarithmic Error (RMSLE)

RMSLE = sqrt( (1/N) * Σ_j (log(y_j + 1) - log(Ŷ_j + 1))^2 )

R² (Coefficient of Determination)

R² = 1 - Σ_j (y_j - ŷ_j)^2 / Σ_j (y_j - ȳ)^2

Clustering Metrics

Silhouette Score
```
Silhouette = (b - a) / max(a, b)
```

Davies‑Bouldin Index

DB = (1/N) * Σ_i max_{j≠i} (σ_i + σ_j) / d(c_i, c_j)

Loading Lab...