Probability

Probability gives a language for uncertainty. ML models often output probabilities (classification), assume probabilistic data-generating processes (Bayesian methods), or use randomness for sampling and generalization.

Random Variables & Distributions

• Discrete: probabilities P(X = x) sum to 1.
• Continuous: densities p(x) integrate to 1.

Expectation & Variance

Expectation is the average value under a distribution; variance measures spread.

E[X] = Σₓ x P(X=x)   or   E[X] = ∫ x p(x) dx

Var(X) = E[(X - E[X])²]

Conditional Probability & Bayes’ Rule

Conditioning updates beliefs given evidence. Bayes’ rule is the foundation of Bayesian inference.

P(A|B) = P(B|A) P(A) / P(B)

Independence & Correlation

• Independence: P(A,B) = P(A)P(B) (stronger than “uncorrelated”).
• Correlation measures linear relationship; zero correlation does not imply independence in general.

Common ML Distributions

• Bernoulli / Categorical (classification targets)
• Gaussian (noise models, regression, latent variables)
• Multinomial (counts, bag-of-words)
• Exponential family (unifies many models; links to GLMs)