An Open Letter from Academics about AI Risk / Life Without the Internet & the Case for Non-Network Redundancy
There has been an increase in concern over AI destroying humanity, most recently spurred by tweets from former Anthropic employee Jacob Coxon about how dangerous AI is, and a follow-up tweet from current employee Evan Hubinger, who said “Jacob is correct here — we really do earnestly believe A.I. could kill all humans! I personally think it is >10% within the next decade.”
Some in the AI world are calling for “the U.S. government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development” and there has been much discussion of trying to slow down or stop all AI research.
Now, a new open letter is circulating, this time from academics worried about the possibility of a “global catastrophe” resulting from AI, calling for political leaders to “create an international treaty, much like the Treaty on the Non-Proliferation of Nuclear Weapons.” It was written by philosopher Molly Gardner (University of Florida) and made public yesterday. If you’re interested in reading or signing it, it’s here: “Open Letter from Academics Concerning Recent Developments in Artificial Intelligence“.
What follows are some thoughts of mine about one approach to mitigating AI risk. I don’t intend to discourage people from signing the “Open Letter from Academics Concerning Recent Developments in Artificial Intelligence“—it couldn’t hurt to do so.
But for various reasons, I think efforts to slow or pause or stop AI research face an insurmountable collective action problem. Even if the well-known corporations working on AI cease their efforts (itself highly unlikely), research and development will continue. AI tools are relevantly not like nuclear weapons, insofar as work to create and improve them is much less detectable, so enforcement will be extremely difficult or invasive. And since the claims about AI destroying humanity are speculative, the scenarios they describe are unprecedented, and AI is not seen unambiguously as a weapon, there will be insufficient will to implement a regime of punishment (on individuals, corporations, nations) severe enough to effectively enforce cooperation.
I don’t know more about AI than people like Coxon and Hubinger, and while I’m not fully convinced of their pessimism, I haven’t read anything (yet) by other knowledgeable people to show me they’re wrong. So let’s suppose they’re correct to at least this degree: AI poses a serious and pressing risk to the survival of humanity.
What should we do?
Both individually and collectively, we need to develop the means to survive without the internet (and other device and communications networks). Without networked computers, the ability of an AI to do much harm to us decreases dramatically. If we find ourselves imminently facing an AI threat, what we may have to do to save our lives is what many have argued we should do anyway to save our souls: disconnect.
It won’t be easy. This isn’t just about giving up social media, or the hassle of not being able to order things online. And the fact that we all lived without the internet just a few decades ago isn’t all that helpful to us today. Now, every company we rely on for everyday items like food, clothing, shelter, and transportation, every public utility we depend on for things like electricity and clean water, every doctor’s office and hospital, every government agency that’s supposed to provide us with public safety and maintain a good society—they all rely on computer networks. Were we to shut them down, most of these organizations would be barely functional, and the downstream effects of that would be destructive chaos. At that point, AI would be superfluous: we might destroy ourselves.
What we need is non-network redundancy. That is, we need to have in place the means by which to function in a situation in which all of the computer networks have been shut down. By “we” I mean every government agency, every utility, every firm above a certain size, every hospital or medical office, and so on. By “the means” I mean not just plans, schemes, ideas (which some agencies and institutions have), but also the physical infrastructure, the equipment, and the employee know-how to adequately function in the absence of computer networks. (Some government agencies have something like this in place, but not all.) There also needs to be in place the means by which to communicate to the public how to navigate a non-networked world, should that be necessary. Additionally, governments may need to improve their ability to keep track of or monitor potential AI threats, so as to better know if or when it is advisable to pull the plugs.
A demand for action on the AI threat that calls for something highly unlikely to successfully happen may have value as a message to others, including politicians, to pay attention to the problem. But it would be even better if it asked for something useful that can be done. People around the world should be calling on their governments for regulation that requires non-network redundancy.
Will it be easy? No. Will it be cheap? No. But such regulation is of a type that is familiar to people and governments around the world. We have regulatory schemes for workplace safety, for non-discrimination, for accounting, for engineering, and so on. What I’m proposing is a kind of thing we have done before.
This is not a new idea. In some ways it is just an expansion of initiatives like CI Fortify by the Cybersecurity and Infrastructure Security Agency (CISA), a branch of the Department of Homeland Security.
And it’s not a radical idea. It isn’t a call to ditch computers or to necessarily forgo the benefits that AI may bring. (It’s not a call for a Dune‘s “Butlerian Jihad“.) Rather, it is a form of disaster preparedness.
Does non-network redundancy eliminate all AI risk? Of course not. Eliminating all AI-risk is not possible. Even if there is no further development of the technology, some humans may use existing AI to develop a deadly mass weapon (or something else dangerous, a la “Ice-nine“). (One might note that “eliminate all risk” is not generally a rational regulatory aim for a technology, but we can leave that debate aside.) But, perhaps alongside other measures, widespread non-network redundancy could go a long way toward reducing the risk of catastrophe.
(The foregoing is about non-network redundancy for government agencies, utilities, businesses, and other organizations. What about individuals? Should each of us be ready—really ready—just in case we have to live without the internet? To be honest, I don’t know. Everyone, it seems, should be prepared to weather some kind of emergency. But beyond that, I don’t know what to recommend, or whether it would be a good idea to do much more. For a few reasons, I don’t think a nation of “doomsday preppers” will make for a good society. But it does seem valuable for individuals and communities to be at least discussing these issues.)
I eagerly signed this open letter and encourage others to do so. But I don’t think regulating and even perhaps banning AI research is anywhere near all that needs to be done. This is a HUGE problem in my view and one that requires a tremendous amount of collective effort to navigate. I’m pessimistic we’ll be able to navigate it.
I’m unconvinced that there is a radical difference between detectability / enforcement of an AI limitation treaty, and the equivalent situation in nuclear nonproliferation. In both cases, you need a somewhat scarce and somewhat monitorable material (cutting edge GPUs vs. uranium ore); in both cases, you need an industrial facility with a fairly large footprint (data farms vs. centrifuges); in both cases, you need a reasonably high level of expertise. If you’ve a source that says otherwise, I’d be interested in seeing it. (Nuclear nonproliferation is actually quite difficult, but we’ve still been somewhat successful at it.)
I hope you’re right. Even if you are, though, things might change. This article in Nature suggests that a decentralized, smaller-footprint alternative to massive data farms is possible (they argue in favor of this alternative), and also note improvements in chips that may facilitate this.
I hope that this is the case. Of course, one key difference is that nuclear capabilities are still very much controlled by governments while AI companies are motivated by private interests that don’t have to include the safety of their fellow citizens. All it takes is one bad actor willing to take risks for a competitive advantage.
I’m not sure I understand your non-network redundancy proposal. I don’t think you’re saying we should shut down the Internet right now. But if you’re saying that we should wait until AI gets dangerous and then shut down the Internet, it seems to me like that’s too late. (At that point, AI will already have engineered a deadly virus or gotten control of nuclear weapons or whatever, so turning off the Internet will not save us.)
I am not going to pretend to know enough about the capabilities of AI or the operation of nuclear weapons (etc.) to state when it might be advisable to shut down computer networks. As I say in the post, “governments may need to improve their ability to keep track of or monitor potential AI threats, so as to better know if or when it is advisable to pull the plugs.” If it is in fact highly unlikely that monitoring of potential risks will be sufficiently effective (however we want to specify “sufficiently” or “effective”), then sure, that’s a problem with my proposal. That seems overly skeptical to me, but I’m no expert. (I’d be happy to hear from experts on this.)