Neural network

Each layer takes numbers in, multiplies them by its weights, adds a bias, applies a simple non-linear function and passes the result on. One layer alone can only draw straight lines. Stack many, and the network can represent extraordinarily complicated relationships without anyone specifying what those relationships are. Networks in use today span roughly six orders of magnitude in size, from a few million weights for a small classifier to well over a trillion for the largest language models.

Training runs an example through, measures how wrong the output was, and works backwards through the layers calculating how much each weight contributed to the error. Every weight moves a little in the direction that would have reduced it. Repeat across billions of examples and the network gradually stops being wrong.

Traditional software requires somebody to work out the rules and write them down. Nobody can write the rules for recognising a cat, or for what makes a sentence sound natural. A neural network learns the rules from examples, which moves the hard part from specifying behaviour to gathering good data.

Every system called AI today — language models, image generators, recommendation engines, speech recognition — is a neural network with a particular shape trained on particular data. The idea is genuinely one idea.

The brain comparison. The name is a historical accident from a rough 1940s analogy, and a neural network resembles a brain roughly as much as a paper aeroplane resembles a bird. Taking the metaphor seriously leads people to expect understanding, intent or memory where none of those exist.

The other is expecting to be able to inspect it. A trained network is billions of numbers with no labels and no comments. You cannot open it and find where it stored a fact, which is why debugging AI behaviour is done by testing outputs rather than by reading the model.