What is a neural network?

What a neuron actually is

In AI, a neuron is a tiny calculator inside a neural network. It receives some numbers, decides how much each one matters, combines them, and produces one new number. No biology is involved, and nothing about it is clever on its own.

Picture a neuron deciding whether the weather suits going outside. Three numbers reach it: clouds 0.9, humidity 0.6, wind 0.2. But they do not matter equally, and the neuron has learned weights that say how important each one is.

The idea is far simpler than the arithmetic: inputs are numbers, weights say how much each input matters, and the neuron multiplies, adds, adds its bias, and produces a number.

Those weights are where the importance lives, and a weight's value is its whole message:

• 1.5 - this input matters a lot. Whatever arrives gets amplified.
• 0.1 - it barely counts. Heard, faintly.
• -0.4 - it argues the other way: more of this input, less of the answer.
• near zero - the wire may as well not be there. Training decides that too.

One neuron is not intelligent. A network is huge numbers of these simple calculations wired together, where the numbers coming out of one layer are the numbers going into the next.

The neuron does the calculation. The weights contain what was learned.- the distinction worth keeping

What one neuron does

Every neuron takes a few numbers in and gives one number out. It does that in three steps - multiply, add, clamp - and it does all three itself. The only thing that belongs to the wire is the weight; the neuron multiplies each arriving number by the weight of the wire it came in on. Here is a whole one, with real numbers, deciding whether to take an umbrella.

Those are the only two kinds of number involved. The ones the inputs get multiplied by are the weights, and they are what the network learned: a big weight means "this input matters", a negative one means "this input argues the other way". The number added at the end is the bias, and here it is negative: -1.00. That makes it a bar the inputs have to clear before the neuron says anything.

Why add them up? Because the neuron arrived with three numbers and has to leave with one. Adding is how several inputs become a single score, and the weights are what make it a vote rather than a tally: an input weighted 1.5 gets half again the say of one weighted 1.0, and a negative weight votes the other way. The bias is one more vote, always cast, by the neuron itself.

And the clamp? A rule applied to the total on the way out: if it is negative, report 0 instead. That is the whole rule, and this example never triggers it, because 0.75 is already positive. It earns its place twice over. It gives a neuron a way to say nothing at all, which is how one learns to sit out the inputs that are irrelevant to it. And without some bend like it, a stack of layers is mathematically identical to a single layer, so nothing whatsoever is gained by going deeper. Its formal name is the activation function, and clamping negatives to zero is the most common one in use.

That is a neuron, complete. A frontier model runs this same multiply-add-clamp billions of times to produce a single word.

Why it is called a network

Neurons are arranged in layers, and each one takes its inputs from everything in the layer before it: the boxes of raw numbers first, then the neurons that follow them. The wiring is the point - not the dots, but what connects them.

Count what is actually stored in that picture. Twenty weights on the wires, plus one bias for each of the six neurons that have inputs: twenty-six numbers. Feed the same three values in twice and the same two come out, because nothing else is kept anywhere.

Depth is what the wiring buys. The first layer sees only the raw inputs; the second sees what the first made of them; a third would see what the second made of that. Nobody assigns a layer its job - training arrives at a division of labour on its own, and in a large model the middle layers work in terms no one has names for.

It also explains why model sizes sound absurd. Add a layer and you add a wire for every pair of neurons across the join, so the count grows with the product of the layer sizes rather than their sum. "Seven billion parameters" is not seven billion ideas; it is mostly seven billion wires.

The dots are empty until you press go. The learned numbers - a weight per wire, a bias per neuron - are everything the model kept.- why it is a network

Why it is not a brain

The name comes from a 1940s attempt to describe a nerve cell with arithmetic, and it stuck for eighty years. It is worth knowing exactly where the analogy gives out, because most overclaiming about AI leans on it.

• There is no cell, no chemistry, and no sense of time. A biological neuron fires in spikes, and when it fires carries information. An artificial one is a single number recomputed on demand, with no notion of before or after.
• It does not change while you use it. Every weight and bias is frozen when training ends. The network answers your question with exactly the numbers it had before you asked, which is why nothing you type in a chat teaches it anything - see How a model learns.
• Nothing is stored where you would expect. There is no address holding "cat". Any concept you can name is smeared across a great many of those learned numbers, each of which is also busy doing other things.
• The size comparison is a party trick. Weights against synapses looks like a scoreboard, but they are not the same unit and they do not do the same job, so the number means nothing in either direction.

None of which makes the term wrong. "Neural network" names a technique, not a claim about biology - it is simply a name carrying a much larger promise than the arithmetic underneath it makes.

Network, model, LLM: where the word sits

Three words get used as though they were one thing, and they sit at three different levels.

So the nesting runs: a neural network is a shape, a model is a trained instance of one, and an LLM is a model whose training material was text. Every LLM is a neural network. Not every model is - and not every neural network is doing language: the same idea, wired differently, is what produces pictures in How AI actually generates images.

What training is, and what it changes

Before training, every weight in a network is a random number and the answers are nonsense. Training is the process that turns those random numbers into useful ones. It works by correction, not instruction: nobody tells the network what any weight should be.

Take the weather neuron from section 01, the one worked through in section 02. It answered 0.75, which section 02 reads as: take the umbrella. Suppose that day the right answer was: leave it at home. The answer was too high, which means the numbers that pushed it up were too big, so training nudges them down a fraction. The 1.5 on clouds might become 1.48. The bias might drop a little too. Nothing else about the neuron changes.

Then it does the same with the next example, and the next. The nudges are tiny and most of them half undo each other, but across an enormous number of examples they settle into numbers that get most answers right. That is all training is.

Step 3 is the one with a famous name - backpropagation. It is what makes the rest possible: it shares the blame for a single wrong answer across billions of numbers in one sweep backwards through the layers, instead of testing them one at a time, which nobody could afford. How big each nudge in step 4 is has a name too: the learning rate.

And what does training change? The weights and the biases, and nothing else. The wiring is chosen by the people who design the network, the arithmetic each neuron does never varies, and the inputs come from you. That is why a network cannot learn a new fact while you are using it: your prompt reaches the inputs, and the inputs are not where anything is stored.

Two consequences follow, and they explain most of what you read about AI costs. Training is expensive and answering is cheap, because answering runs the network once while training ran it an enormous number of times. And the numbers stop moving the moment training stops, which is where a model's cutoff date comes from. How a model learns takes it further, including what fine-tuning changes and what it does not.

Training does not teach a network facts. It corrects numbers until the answers stop being wrong.- training in one sentence