Text size
Font

Saved on this device, like the light and dark setting.

The Models

The math, made simple.

The few ideas that actually make these tools work, each one shown as something you can poke, with no equations to solve.

Four short scenes. Poke everything.

Scene 1 of 4 · about 40 seconds

Scores become odds.

Give each next word a raw score. Watch the odds move.

The sentence so far: "The cat sat on the ___"

Raw scores
temp 1.00
The odds it prints

Those percentages in the very first demo of The Basics? This is the math that makes them.

Softmax takes a pile of raw scores and turns them into percentages that add up to 100. Slide one score up and that word takes a bigger slice of the pie, and the others give some back.

Temperature decides how boldly it bets. Turn it down and it almost always picks the top word. Turn it up and the odds flatten, so the long shots get a real chance. That dial is the difference between a safe answer and a surprising one.

The technical name for thissoftmax is the step that converts raw scores into percentages that sum to one. Temperature is the number those scores are divided by first, which stretches or squashes the gaps between them.

Try this tonight

Run these in your own AI, like ChatGPT, Claude, or Copilot. They are safe.

Explain what "temperature" changes about your answers, in one short paragraph.
Give me five names for a cafe. Now give me five more, but much more adventurous.

✓You just saw how raw scores become odds.

Scene 2 of 4 · about 40 seconds

Words become places.

Click a word. See what sits closest to it in meaning.

The AI doesn't store the word "cat". It stores a location. Things that mean similar things sit close together.

Every word gets turned into a point on a map. Meaning becomes distance, so words about the same thing land near each other, even when they share no letters.

How alike are two words? It comes down to the angle between the two arrows drawn from the center of the map, and that angle is what "cosine similarity" measures. This is the same idea behind the semantic search from The Decoder.

The technical name for thisembeddings are the coordinates each word gets turned into. Cosine similarity is the score for how closely two of those arrows point the same way.

Try this tonight

Run these in your own AI and see how it reasons about closeness.

Which pair is closer in meaning: kitten and puppy, or kitten and truck? Explain why.
List ten words that live near "refund" in meaning, closest first.

✓You just saw meaning turned into a map.

Scene 3 of 4 · about 40 seconds

It decides what to look at.

Pick the last word. Watch which earlier words light up for "it".

Illustrative weights, pre-baked on your device, to show the idea.

Attention is how the model weighs which earlier words matter for the word it is working on. Here the word in question is "it", and the bars show how much each earlier word pulls on it.

Change one word at the end and the same "it" points somewhere else. With "big" it leans on the trophy. With "small" it leans on the suitcase. The word never moved. The context around it did.

The technical name for thisattention is the weighing step that scores how much each word matters to another. A transformer is the kind of model built out of many of those attention steps stacked together.

Try this tonight

Give your own AI the classic sentence and flip the last word.

In "The trophy didn't fit in the suitcase because it was too small," what does "it" refer to? Now change "small" to "big" and answer again.

✓You just saw attention pick what matters.

Scene 4 of 4 · about 45 seconds

Learning is rolling downhill.

Roll the ball to the bottom. Change the step size and watch what happens.
low error at the bottom, high error up the sides
step 0.20
steps: 0 error: 0.00 position: 0.00

No data leaves your device. This is just the shape of the idea.

Training is a long walk downhill. The model checks which way lowers the error, takes a small step that way, and repeats. Billions of tiny nudges to little dials, one step at a time.

The learning rate is the size of each step. Too small and it crawls for ages. Too big and it overshoots and bounces across the valley without ever settling. Too big is as bad as too small.

The technical name for thisgradient descent is the walk downhill along the steepest direction. The learning rate is the number that sets how far each step goes.

Try this tonight

Ask your own AI to teach the idea back to you.

Explain gradient descent to me like I'm choosing how big a step to take while walking downhill in fog.
What goes wrong when the learning rate is set too high?

✓You just saw how a model learns.

Link copied. Send it to someone.