US AI Academy

Idea 4

Surprise, in bits

Grading a guess, Paris versus Tuesday, and the loss every lab trains down.

This lesson is part of the full course. If you haven’t yet, start with the free chapter: it ends on the question the rest of the course answers.

Unlock the whole course

Everything after the free chapter, with real numbers from the same model and the same puzzle network.

  • Ideas 2 to 5, each with labs you can drag: matrices (many dot products at once), softmax and attention (how scores become chances, and how tokens look at each other), and surprise in bits and the chain rule (together, how hidden neurons learn)
  • The puzzle, solved: the three-neuron network trained step by step, with the real numbers
  • An AI tutor that answers from the course itself (up to 30 questions in total)
  • A worksheet that traces the Paris prediction by hand
  • 28 exercises, with every step of every answer worked out
  • A 50-page illustrated PDF guide to the same math, to print or read offline
  • The final exam, and your AI Foundations Certificate when you pass, with its own page anyone can check
  • Lifetime access to this course and its updates
$39paid once, not a subscription. Full refund within 14 days of buying.

You can’t buy it yet: checkout opens soon. Join the list and you’ll get the launch price, $29 instead of $39, if you buy in the first 14 days after it opens.

I’ll email you when the full course opens, and when new lessons and courses arrive. You can unsubscribe any time.