Neural Networks Without Mathematics

Opening Question

What is a neural network, actually — without the math?

Real-World Story

Picture a huge factory floor covered in simple valves. Each valve does one tiny job: it takes in some water pressure from a few pipes behind it, combines those inputs according to how far open it’s been set, and pushes water out through a pipe in front of it toward the next row of valves. No single valve understands anything about “water pressure control” as a concept. It just responds to what flows into it, in a fixed, simple way, based on its own current setting.

Now imagine millions of these valves, arranged in long, connected rows, the output of one row feeding into the next. No individual valve is doing anything sophisticated. But by carefully setting all their individual openings, the whole system — the water pressure that comes out the far end — can be made to do something remarkably specific and complex, like maintaining a delicate pressure curve for an entire building. The complexity lives in the combination of millions of simple settings, not in any single valve’s behavior.

A neural network works on exactly this principle, just with numbers instead of water.

Worked Example

Picture one single unit with three incoming connections, weighted “high importance,” “medium importance,” and “barely matters.” Feed it three different input signals. The unit doesn’t treat them equally — it leans heavily on the high-importance input, moderately on the medium one, and almost ignores the third, then combines all three into one outgoing signal.

Now change just one of those three weights, keeping the same three inputs. The unit produces a different output than before — same inputs, different weighting, different result. This single change is precisely what training (Chapter 9) does, billions of times over, across billions of adjustable connections and parameters spread across the whole network: nudging weights so that the network’s outputs get steadily closer to what’s actually wanted.

Core Intuition

A neural network is a system built from enormous numbers of extremely simple units, loosely inspired by (but not a simulation of) biological neurons. Each unit takes in several incoming numbers, combines them — each input weighted by its own adjustable importance — and passes a single resulting number onward to the next layer of units. Individually, a unit’s job is almost trivially simple.

A different angle on the same idea: imagine a hiring committee where every member weighs the same facts about a candidate — years of experience, a portfolio, an interview impression — but each member privately assigns different importance to each factor. One member’s vote leans heavily on the portfolio; another’s mostly on the interview. The committee’s final decision emerges from combining all of these individually-weighted judgments, even though no single member’s opinion is “the” answer on its own. A neural network’s units work the same way, just with numbers instead of people, and millions of “members” instead of a handful.

Those adjustable importances are called parameters (also called weights): a number attached to every connection in the network, controlling how much that particular input matters to that particular unit — though a network also has a handful of other kinds of adjustable numbers besides connection weights, playing supporting roles rather than changing this basic picture. A modern language model can have many billions of these parameters. None of them is individually meaningful or hand-designed — there is no parameter that means “cat” or “past tense.” The network’s entire capability lives in the overall pattern formed by all of these numbers together, the same way the building’s pressure curve lived in the overall pattern of millions of valve settings, not in any single valve.

Technical Explanation

A neural network is organized into layers: an input layer (which receives the token embeddings from Chapter 5), one or more hidden layers, and an output layer (which produces the next-token probability distribution from Chapter 6). Each unit in a layer receives the outputs of units in the previous layer, multiplies each one by its own weight, adds them together, and applies a simple fixed transformation before passing the result forward. That transformation isn’t just a matter of scaling the combined signal up or down — it bends it, the way a dimmer switch bends light output at the low end of its range instead of tracking the dial in a perfectly straight line. That bending step is what actually lets stacking add real power: combine and pass along a purely proportional signal, layer after layer, and no matter how many layers you stack, the whole thing behaves exactly like one layer would — the bending at each step is what keeps each additional layer from collapsing into the ones before it. Stacking many such layers lets the network build up increasingly elaborate combinations — a hidden layer can respond to combinations of patterns detected by the layer before it, letting the network represent far more intricate relationships than any single layer could alone.

The network’s parameters are not designed by a human engineer writing rules. They start out essentially random, and are gradually adjusted — a process called training, covered in Chapter 9 — until the network’s overall behavior, across billions of examples, reliably produces good predictions. This is the crucial shift from earlier, hand-coded software: nobody writes down what each parameter should be; the values emerge from a repeated correction process operating over enormous amounts of data.

One boundary is worth drawing precisely. The plain layered network described in this chapter — every unit connected to every unit in the previous layer — is a real, foundational design, and everything said about it here (parameters, layers, learned rather than hand-coded behavior) remains true of every neural network in this book, including language models. But a modern language model is not simply a long stack of identical layers like the one described here. Its layers are organized into a more specialized arrangement — the transformer — with dedicated machinery for weighing which earlier tokens matter most to which later ones. That arrangement is the subject of Part III. Treat this chapter as the load-bearing foundation the transformer is built on top of, not as a literal blueprint of one.

Common Misconceptions

“A neural network is basically a simulation of a human brain.”

Why it’s wrong: The units are only loosely, historically inspired by biological neurons; they don’t simulate real neuron behavior, brain structure, or biological learning mechanisms.

Correct intuition: A neural network is a mathematical system of simple, adjustable units — the biological name is a historical artifact of where the idea originally came from, not a claim of biological accuracy.

Analogy: An airplane wing is inspired by a bird’s wing, but nobody mistakes a Boeing 747 for a mechanical bird.

“More parameters automatically means the network understands more, in a simple, linear way.”

Why it’s wrong: Parameter count matters, but how those parameters are structured, trained, and balanced against the amount of training data all matter too — a large but poorly trained network can underperform a smaller, well-trained one.

Correct intuition: Parameter count is one important ingredient among several, not a standalone score of capability — Chapter 10 covers how these ingredients interact.

Analogy: A car with a bigger engine isn’t automatically faster if the rest of the car — transmission, weight, aerodynamics — isn’t built to use that power well.

Practical Implications

Once you see a neural network as “millions of simple, adjustable valves,” headlines about model sizes (“70 billion parameters,” “1.8 trillion parameters”) stop being abstract marketing numbers and become a literal count of exactly this kind of adjustable value. It also explains why neural networks are often called “black boxes”: you can inspect every one of those billions of numbers, but no individual number tells you, on its own, what the network “knows” — the knowledge is distributed across the whole pattern, not localized in any single readable place.

Key Takeaway

A neural network is layers of simple units connected by adjustable weights — its entire capability lives in the pattern of those weights, not in any single unit.

What to Remember

A neural network is built from many simple units, loosely inspired by (not simulating) biological neurons.
Each connection between units has its own adjustable weight — a parameter — controlling how much that input matters.
Layers stack: each hidden layer builds on combinations detected by the layer before it.
Modern models have billions of parameters, none individually meaningful — capability is distributed across the whole pattern.
Parameters aren’t hand-coded; they emerge from training (Chapter 9), starting from near-random values.
Neural networks are “black boxes” not because they’re hidden, but because no individual parameter is independently interpretable.

The Next Obvious Question

If a network’s entire behavior lives in these adjustable numbers, how do those numbers actually get set to the right values?