The Models
The math, made simple.
The few ideas that actually make these tools work, each one shown as something you can poke, with no equations to solve.
Four short scenes. Poke everything.
Scene 1 of 4 · about 40 seconds
Scores become odds.
The sentence so far: "The cat sat on the ___"
Those percentages in the very first demo of The Basics? This is the math that makes them.
Softmax takes a pile of raw scores and turns them into percentages that add up to 100. Slide one score up and that word takes a bigger slice of the pie, and the others give some back.
Temperature decides how boldly it bets. Turn it down and it almost always picks the top word. Turn it up and the odds flatten, so the long shots get a real chance. That dial is the difference between a safe answer and a surprising one.
The technical name for thissoftmax is the step that converts raw scores into percentages that sum to one. Temperature is the number those scores are divided by first, which stretches or squashes the gaps between them.
Try this tonight
Run these in your own AI, like ChatGPT, Claude, or Copilot. They are safe.
✓You just saw how raw scores become odds.
Scene 2 of 4 · about 40 seconds
Words become places.
The AI doesn't store the word "cat". It stores a location. Things that mean similar things sit close together.
Every word gets turned into a point on a map. Meaning becomes distance, so words about the same thing land near each other, even when they share no letters.
How alike are two words? It comes down to the angle between the two arrows drawn from the center of the map, and that angle is what "cosine similarity" measures. This is the same idea behind the semantic search from The Decoder.
The technical name for thisembeddings are the coordinates each word gets turned into. Cosine similarity is the score for how closely two of those arrows point the same way.
Try this tonight
Run these in your own AI and see how it reasons about closeness.
✓You just saw meaning turned into a map.
Scene 3 of 4 · about 40 seconds
It decides what to look at.
Illustrative weights, pre-baked on your device, to show the idea.
Attention is how the model weighs which earlier words matter for the word it is working on. Here the word in question is "it", and the bars show how much each earlier word pulls on it.
Change one word at the end and the same "it" points somewhere else. With "big" it leans on the trophy. With "small" it leans on the suitcase. The word never moved. The context around it did.
The technical name for thisattention is the weighing step that scores how much each word matters to another. A transformer is the kind of model built out of many of those attention steps stacked together.
Try this tonight
Give your own AI the classic sentence and flip the last word.
✓You just saw attention pick what matters.
Scene 4 of 4 · about 45 seconds
Learning is rolling downhill.
No data leaves your device. This is just the shape of the idea.
Training is a long walk downhill. The model checks which way lowers the error, takes a small step that way, and repeats. Billions of tiny nudges to little dials, one step at a time.
The learning rate is the size of each step. Too small and it crawls for ages. Too big and it overshoots and bounces across the valley without ever settling. Too big is as bad as too small.
The technical name for thisgradient descent is the walk downhill along the steepest direction. The learning rate is the number that sets how far each step goes.
Try this tonight
Ask your own AI to teach the idea back to you.
✓You just saw how a model learns.