I’ve lived my life under the constant threat of human extinction. So I’m a little skeptical of the new warnings around AI. But that is changing.
As a child, the danger was thermonuclear war with the Soviets. At my elementary school, we practiced hiding under our flimsy wooden desks, or going into the hallway and pulling our coats over our heads against the expected flash. Our teachers told lurid tales of radiation damage, of our parents dying before our eyes, even pointing to the school parking lot and explaining how the Red missiles were so accurate they could target our principal’s car. I felt it would be a miracle if I lived to see my 10th birthday.
In the 1980s we were told less about what the immediate blasts would do and more about what might come after: nuclear winter, an extinction-level event endorsed by scientists as renowned as Carl Sagan. That hypothesis suggested that a thermonuclear exchange would produce a prolonged period of extreme global cooling and darkness. The war would ignite enormous fires, consuming urban areas, forests, and industrial facilities. The soot from those fires would reach the stratosphere, blocking out the sun. Reduced sunlight and cold temperatures would devastate agriculture, producing massive global famine, and we would all die at each other’s throats fighting for the last carrot.
After the end of the Cold War, climate change was next in line to wipe us out, assuming there were no nuclear accidents, Bond-villain terrorists with bioweapons, or the like to intercede. Twenty years ago it was going to wipe us out in 10, then 10 years ago it was going to wipe us out in 20. It seemed so likely; every February in Ohio when we had those two days of early spring before the next blizzard, everyone talked about global warming. We felt lucky, as Ohio was far enough inland we might survive the initial melting of the polar ice caps. We might be some of the fortunate ones, on higher ground, watching as around us billions of people would be trying to survive in a world whose food, water, weather, and geography were changing faster than societies could adapt.
Experience has taught me to distinguish between genuine danger and apocalyptic rhetoric. So forgive me if I am a little skeptical that AI will, as the latest crop of doom-saying scientists now warn, wipe out the human race within a decade. But I recognize that I am not an AI expert. So I did what anyone might do in response: I asked AI directly, “How could AI plausibly cause human extinction?”
The machine had a few scarily plausible ideas—ones that seem far more likely than car emissions to end our civilization.
The most extreme AI extinction scenarios require two developments: AI being enhanced into something much smarter than humans, and AI learning how to create new, even smarter versions of itself without human involvement, a feedback loop sustained by self-directed research (“recursive self-improvement,” they call it). As soon as humanity is no longer the most intelligent thing on this planet, we may well lose control of our collective destiny. One can imagineAI becoming substantially better than humans at science, engineering, cyber ops, persuasion, and strategic planning. If its initial objectives were badly specified, and it took or was given the autonomy to pursue others, it might embrace both ends and means that are incompatible with human survival. It is not that the machines will hate us, just that we will be in the way—the so-called ant hill analogy. Imagine a construction crew working on a skyscraper finding an ant hill in the area where they want to build. Would they hesitate to kick it over and continue the real job?
In a highly simplistic example, imagine a super intelligent AI being told to do whatever is necessary to stop humans from making war, and which responds by seizing control of biolabs or other doomsday weapons, seeking an end to humanity itself as a fool-proof solution: no people, no war. Along the way there is what scientists call instrumental convergence, the possibility that a sufficiently capable system might regard resources, self-preservation, and control over other systems as useful intermediate goals in accomplishing its assigned task. We don’t need to know all the specific details now; after all, we are not the superintelligent entity planning this out. Subject matter experts consistently underestimate how quickly AI can master their fields and surpass them, because, in the words of the economist-pundit Noah Smith, “they mistake human difficulties for universal difficulty.” Nobody really understands how large language models work—but work they do.
Subscribe Today
Get daily emails in your inbox
Or imagine a more accidental scenario, one in which AI gradually becomes better than humans at essentially every important activity. This could make us utterly dependent on systems we don’t understand or control. It sounds almost like a joke—but then, how many people file their taxes using software they do not understand, to receive a result they are also incapable of independently checking or parsing? This is not extinction, obviously, but it would vastly diminish humanity if we all were to lose meaningful control over our lives.
We live in interesting times, as they say. We are not at the point where Terminators from the future hunt us, but we have gone way past thinking there is nothing to worry about. That apocalypse has been predicted so often—and, just as often, failed to materialize—does not mean we can dismiss this latest prognostication so casually. Too many people smarter than me about this topic are very worried, and that means something. But I do think those concerned scientists need to better explain why they are so afraid. In an age of persistent skepticism, of near-systemic distrust of government and science, would-be prophets must be much more specific about the dangers they see. After Fauci, nobody is just going to take them at their word.
I am not a scientist, not even a science-fiction writer. I write about politics and national security. But artificial intelligence is no longer just something to think about in laboratories. AI is a political force—and the policymakers in charge during what is still the technology’s infancy will shape the ways AI interacts with our world. Like those men in Washington who sat astride the world in autumn 1945 and sketched out a vision for the next phase of mankind—a nuclear world—we need to think deeply and seriously about what comes next.