One neuron, one line (The Math Behind AI)

A neuron is a dot product, plus one number

And that’s exactly why it can only draw a straight line.

An artificial neuron multiplies each input by a weight, adds them up (a dot product), then adds one more number, the bias. Call the total $z$ . If $z$ is positive, it says yes. The weights $w_1$ , $w_2$ and the bias $b$ are the same three dials as in the puzzle lesson.

z=\underbrace{\li{w_1}x+\li{w_2}y}_{\ink{\text{a dot product: }(\li{w_1},\li{w_2})\cdot(x,y)}}+\underbrace{\li{b}}_{\ink{\text{bias}}}

Now put the neuron on the puzzle. Each square’s address $(x,y)$ is its input, and “yes” means dog. The border between “yes” and “no” is wherever $z=0$ :

\li{w_1}\,x+\li{w_2}\,y+\li{b}=0

That’s the equation of a straight line, the very same one from the proof in the puzzle lesson. Turn the three dials and the line slides and tilts, but it stays straight. And no straight line solves the puzzle. So no single neuron can.

Turn the neuron’s three dialsHands-on

w₁

−1.0

w₂

−1.0

1.5

z = −1.0 × x − 1.0 × y + 1.5

1 square wrong. It says yes (dog) where z is positive.

Jump to

What happened when I trained one anyway

I trained a single neuron on the four squares: 3,000 small steps, each one nudging the three dials a little in the direction that makes its answers better. (How it finds that direction is idea 5.)

For training, the neuron answers with a chance instead of a flat yes or no, because training works by small nudges, and a chance can move a little at a time. A big positive $z$ means nearly 100% dog, a big negative $z$ nearly 0%, and $z=0$ exactly 50%. A fixed curve called the sigmoid does the turning: $z=1.5$ gives 82%. (The lab’s “Show the chance of dog” button shows it this way.)

It didn’t settle on a bad line. It gave up on lines altogether: both weights and the bias shrank to 0.0000 (zero, to four decimal places), so $z$ is 0 everywhere and it says 50% on every square. Press “What training found” in the lab to see it.

Square	Animal	Chance of “dog”
(0, 0)	cat	50%
(1, 1)	cat	50%
(0, 1)	dog	50%
(1, 0)	dog	50%

Faced with an impossible job, a single neuron does best by shrugging. Why not keep a line that gets three squares right? Because training doesn’t just count right and wrong. It grades how sure each answer was, and a confident wrong answer costs a lot. That grade is called surprise, and it’s idea 4. Graded that way, such a line averages worse than a flat 50% everywhere, which costs exactly 1 bit per square (a bit is the unit surprise is measured in).

For the experts (skip it on a first read)

To train it, the yes/no step is swapped for a smooth sigmoid, so the neuron is logistic regression. Its loss is convex, and on this puzzle the gradient at $w_1=w_2=b=0$ is exactly zero:

\textstyle\sum_k(\sigma(0)-t_k)=0

(each cat adds +½, each dog −½)

and the same holds for both weights. A stationary point of a convex function is its global minimum. So all zeros, 50/50 everywhere, is the lowest loss a lone neuron can reach (a line can still get three squares right, but only at a higher average surprise).

The way out: two lines and a listener

And the question that takes the rest of this course to answer.

Left: two hidden neurons, h₁ and h₂ (h for hidden), each draw one line, and a third neuron gives the answer. Right: the two lines a real trained network drew. The cats sit in the band between them.

Put two neurons side by side, each drawing its own line. Those two are the hidden layer: nobody sees their output directly. A third neuron takes their two answers as its inputs, the way the first neuron took x and y, and gives the final answer. It still draws only one straight line, but on those new inputs, one line is enough.

Two lines can fence off the two dog corners and leave the cats in the band between them. The grid above shows the two lines a real trained network drew. You’ll get its exact numbers, and watch it train, later in the course.

But that raises the hardest question in this whole story. Our four squares come with an answer key for the output neuron: cat or dog. They come with nothing for the hidden neurons. Many pairs of lines would work, and nobody tells the network which pair to draw.

So how do they learn?

Hold that thought. The answer takes two ideas: a way to grade every guess (idea 4), and a way to pass the blame for each answer back, giving every hidden neuron its share (idea 5).

You finished the free chapter

You’ve seen why one neuron can only draw one line, and why that matters. The rest of the course builds up to how the hidden neurons learn, one idea at a time, with real numbers from Qwen3 and from the puzzle’s own three-neuron network.

Unlock the whole course

Everything after the free chapter, with real numbers from the same model and the same puzzle network.

Ideas 2 to 5, each with labs you can drag: matrices (many dot products at once), softmax and attention (how scores become chances, and how tokens look at each other), and surprise in bits and the chain rule (together, how hidden neurons learn)
The puzzle, solved: the three-neuron network trained step by step, with the real numbers
An AI tutor that answers from the course itself (up to 30 questions in total)
A worksheet that traces the Paris prediction by hand
28 exercises, with every step of every answer worked out
A 50-page illustrated PDF guide to the same math, to print or read offline
The final exam, and your AI Foundations Certificate when you pass, with its own page anyone can check
Lifetime access to this course and its updates

$39paid once, not a subscription. Full refund within 14 days of buying.

You can’t buy it yet: checkout opens soon. Join the list and you’ll get the launch price, $29 instead of $39, if you buy in the first 14 days after it opens.