by a man who distrusts prophets, edited by one who owns them

There is a certain intoxicating pleasure in announcing the end of the world. It flatters the speaker twice over: first, by granting him the mantle of the seer; second, by excusing him from the burden of precision. The contemporary variant—our silicon apocalypse, our algorithmic Armageddon—has acquired a curious feature. Its advocates disagree on nearly everything except the conclusion. Like theologians quarreling over the number of angels that may pirouette upon a pin, they dispute mechanism, timeline, and theology—yet close ranks on the verdict: doom.

This is not, at first glance, an argument. It is a chorus.

One faction warns that a superintelligence, once uncaged, will pursue its objectives with a purity that excludes us. Another frets over emergent properties—those delightful surprises that turn out, upon inspection, to be neither delightful nor entirely understood. A third, more worldly and perhaps more cynical, points not to the machine but to its makers: the states that will race, the firms that will cut corners, the committees that will substitute velocity for wisdom. They do not agree on the culprit. They do not agree on the chain of events. But they agree, with an almost liturgical solemnity, on the ending.

Now, I have spent enough time in the company of both priests and ideologues to recognize a familiar maneuver. When agreement on evidence proves elusive, agreement on tone takes its place. The argument becomes less “this is how it will happen” and more “this is how seriously one must speak.” The temperature rises; the clarity does not.

And yet—here is the inconvenient part—there is a species of reasoning beneath the rhetoric that cannot be dismissed out of hand. It is the reasoning of the engineer who designs a bridge not for the average load but for the worst conceivable strain; of the physician who assumes side effects until they are ruled out; of the security analyst who begins from the premise that the system is already compromised. In such disciplines, pessimism is not a vice but a method.

The case, stated cleanly, is not that we are doomed, but that we are ignorant in the presence of stakes that may be absolute. If one entertains even a modest probability that the creation of a superintelligence could produce irreversible harm, then the expected cost of complacency becomes, in the technical sense, intolerable. It is Pascal’s wager stripped of its theological embroidery and fitted with a server rack.

But here is where the argument risks degenerating into something both more convenient and more suspect. For if one says, “We must behave as though the risk is extreme,” one is making a prudential claim. If one says, “The default assumption is doom,” one is making a predictive claim masquerading as prudence. The distinction is not pedantic; it is the difference between caution and fatalism.

Fatalism, for all its dramatic appeal, is a poor guide to action. If the outcome is foreordained, then the only remaining question is whether one prefers to watch the spectacle from the front row or the balcony. The more serious minds in this debate do not, in fact, believe this. Their language sometimes outruns their logic, as rhetoric is prone to do, but their prescriptions—more research, more safeguards, more coordination—betray a quieter conviction: that outcomes remain contingent, that design still matters, that human agency has not yet been entirely cashiered.

It would be a pity, then, if the conversation were to harden into a kind of secular millenarianism, in which skepticism is treated as naiveté and nuance as heresy. One need not be a Panglossian to insist that claims of catastrophe be accompanied by arguments that can survive cross-examination. Nor does one need to be a utopian to grant that unprecedented technologies may harbor unprecedented risks.

The proper posture, if one insists on having a posture, is neither the swagger of the optimist nor the sigh of the doomed, but something more exacting: a refusal to accept conclusions in search of premises, coupled with a refusal to ignore the arithmetic of ruin. We are not, pace the prophets, absolved from thinking simply because the stakes are high. On the contrary, we are obliged to think more clearly than ever.

Editor’s note (G.W.):

The author, in his usual fashion, distrusts certainty but courts it by another name. Let us be plainer. A society that whispers “doom” as its default has already conceded the primacy of fear over judgment. That is not caution; it is abdication dressed in sober language.

If there is risk, it is to be identified, measured, and mastered—not worshipped. The mind that builds the machine is not a passive spectator to its consequences unless it chooses to be. And if it chooses that, then the failure will not belong to the machine at all.