Differential Equations

Differential equations describe systems that change over time using relationships between a function and its derivatives. They appear in ML when modeling dynamics (physics, robotics, time series) and in modern approaches like neural ODEs.

Why They Matter In ML

• Time-evolving systems: motion, fluids, population dynamics, chemical reactions.
• Continuous-time modeling: irregularly sampled sequences and latent dynamics.
• Stability intuition: how small perturbations grow or decay over time.

Simple Example

Exponential growth/decay is modeled by:

dy/dt = k y

Its solution is y(t) = y(0) e^{kt}. This pattern shows up in decay processes and continuous-time hidden states.

Neural ODE Intuition

A neural ODE defines dynamics with a neural network:

dz/dt = f(z, t; θ)

Given an initial state z(t₀), we integrate forward to get z(t₁). Training adjusts θ so the resulting trajectories fit data.

What To Learn First

• First-order ODEs and separation of variables.
• Linear ODEs and stability (equilibria, eigenvalues).
• Numerical solvers (Euler, Runge–Kutta) and step-size tradeoffs.