Part 04 · LLM Internals
Self-Attention
Query, Key, ValueScaled dot-productCausal maskMulti-head attention
The heart of the transformer, worked out with real numbers: queries, keys and values, the causal mask that stops a word peeking ahead, and why many heads beat one.
⏱ 2-3 hours · Station 26 of 92