Loading...
Loading...
Loading Curriculum...
Loading Subject...
Loading Topic...
Loading Lesson...
Loading Lab...
A 3D interactive visualizer of a GPT-style large language model running inference — walking through every layer, attention head, and token embedding step by step.
Initializing 3D transformer… , if it takes a while refresh the page
Each input token is converted into a high-dimensional vector. The model learns these representations during training.
Queries, Keys, and Values let each token attend to every other token. Multi-head attention learns different relationship patterns in parallel.
Stacks of attention + MLP layers with residual connections and layer norms. GPT-2 small has 12 such blocks.
The final layer projects hidden states to vocabulary logits. Softmax turns them into a probability distribution over the next token.
Normalizes activations across the hidden dimension. Stabilizes training and allows much deeper networks.
Information flows through a residual stream across all layers. Each block adds its contribution to this stream.
// Transformer blocks are stacks of self-attention + MLP layers with residual connections. // (ASCII preview placeholder — replace with a full diagram if needed)
Visualization by Brendan Bycroft · MIT License