Project Maven
Katrina Manson, W.W. Norton
Israel’s military is using artificial intelligence to identify targets in Gaza, the US is doing the same in Iran and Ukraine pushes on with smart drones. AI war is not the future of conflict, it is the present.
Unpacking global policies on the use of AI by militaries – the potential benefits, pitfalls and murky ethics – will fill books for decades to come. But that’s not what Katrina Manson is setting out to do in Project Maven. Instead, she uses interviews with over 200 people to tell the story of the US military’s journey into AI warfare – or one of them, since there are 800 AI projects hidden in the Pentagon.
In 2017, Project Maven was launched to build a tool to scour footage from drones and extract useful intelligence – the drones had collected more data than any human could interpret. Maven had a rocky start, says Manson. The military deployed it with soldiers in Somalia eight months after the launch of the project, and algorithms told analysts there were schoolbuses in the clouds and trees were people.
We follow one project leader back to his days as an intelligence officer in Afghanistan trying to plan missions and direct troops with nothing more than a dusty laptop loaded with Microsoft Office: where is the enemy, where is safe, what does success look like?
Humans in war are inefficient, get tired, make mistakes. The fog of war could be cleared by AI, believed the usually secretive Project Maven builders who spoke to Manson. But they intended it to go much further: choose targets, hunt them and then kill them. Without slow, deliberate human decision-making, killer robots could overwhelm enemies, fast.
“We kill the wrong people all the time. A machine can’t be worse than a human,” one insider says. The team developed Maven into a host of tools and tried to convince people on the front line to adopt them. Results improved, but mistakes still happened.
Since then, the US and other NATO members have rolled out Maven in conflicts. Some 32 companies are working on it, writes Manson, and 25,000 US military users log in regularly. But she also tells of it being used at border crossings and in hunting drug runners in the Caribbean. Can a state with such tools resist using them on its citizens?
Most worryingly, work is under way to cut humans out of the loop entirely, says Manson. So-called Goalkeeper flying drones and Whiplash naval drones can find their own targets and take them out. And humans have never invented a weapon and not used it.
It’s hard not to think of Stanislav Petrov, the Soviet lieutenant colonel who, in 1983, used his own judgement and decided reports of a US missile launch were a false alarm and avoided all-out nuclear war. Would AI make that call?
For all the fascinating insight into Maven, the book tells us more about Pentagon bureaucracy and Silicon Valley’s willingness to take on any project – no matter how distasteful – if the money is right, than it does about AI. Manson’s access is phenomenal, yet the nature of military secrecy means we likely won’t know exactly what technology the US government has produced, and how and when it is being used, for years to come.
War has always been deeply unpleasant, but modern conflicts where humans watch someone thousands of miles away via drone and decide if they warrant a fatal strike have made it impersonal. Handing this to AI risks making war too easy to wage, and the repercussions too easy to ignore.
We need to ensure the power granted by AI weapons is treated with the gravity it deserves, but Manson tells us a chilling story that suggests the reality is otherwise. One interviewee hoping to join Project Maven reportedly told the panel their motivation was to “reduce the non-American population” – and then got the job.
Two more great reads on AI and warfare
The Making of the Atomic Bomb by Richard Rhodes
There are a lot of lessons here about where military AI might go. Like the Manhattan Project, it threatens to permanently ratchet up global tensions and increase the stakes of war, just for starters.
Should We Ban Killer Robots? by Deane Baker
This is a dive into the debate from an ethics professor, looking at the thorny problems of reliability, control and accountability when governments hand over the work of soldiers to computers.
Topics:
Facts Only
* The US military launched Project Maven in 2017.
* The project aimed to analyze data from drones.
* The project deployed soldiers in Somalia eight months after launch.
* Algorithms misidentified school buses and trees as targets.
* Over 200 people were interviewed for the project.
* 32 companies are working on Project Maven.
* 25,000 US military users regularly log in.
* The project is being used at border crossings and in drug interdiction.
* Goalkeeper and Whiplash drones can find and engage targets autonomously.
* Stanislav Petrov’s 1983 incident was referenced.
* The Pentagon bureaucracy and Silicon Valley investment were significant factors.
* One interviewee expressed a desire to “reduce the non-American population.”
Executive Summary
Full Take
The narrative presented by Katrina Manson doesn’t offer a straightforward account of AI’s integration into military strategy; instead, it’s a deeply unsettling portrait of a system’s self-replication and potential for abuse. The story of Project Maven reveals not a technological marvel but a bureaucratic swamp—a “hundred-dollar bills in the sand” scenario, as one insider put it—where ambitious ventures are pursued with little oversight and significant ethical compromises. The initial failures, misidentifying innocuous objects as threats, are unsettling in themselves, but Manson’s framing elevates this beyond simple technical glitches. The move toward autonomous targeting, championed by figures advocating for “killer robots,” reveals a fundamental miscalculation about the nature of human judgment in high-stakes environments. This isn’t merely about increased efficiency; it’s about delegating the most morally fraught decisions to algorithms, a process that strips away accountability and potentially amplifies existing biases.
Pattern detected: ARC-0043 Motte-and-Bailey – Manson utilizes a seemingly straightforward account of Project Maven’s development, initially focusing on the technical challenges (the misidentification of school buses) while simultaneously hinting at its broader, more troubling ambitions (autonomous targeting). This strategy obscures the deeper ethical concerns by presenting them as a natural progression of the project’s goals, effectively denying responsibility for the potentially dystopian outcomes. The story effectively functions as a cautionary tale about the illusion of control inherent in complex systems. The project’s expansion beyond simple intelligence gathering suggests a fundamental shift in the role of human judgment within warfare, a shift presented as a pragmatic response to human limitations – but one that fundamentally alters the moral landscape of conflict.
Pattern detected: ARC-0024 Ambiguity – The article deliberately avoids definitively stating whether or not Project Maven is "successful," instead focusing on the inherent challenges and continued mistakes, creating uncertainty about the overall impact and trajectory of the program. This ambiguity serves to obscure the full extent of the project's potential dangers and obfuscate a clear assessment of its effectiveness. The article’s reliance on anecdotes—the soldier’s laptop in Afghanistan, the insider’s unsettling motivation—adds to this ambiguity.
The narrative resonates with the classic “Manhattan Project” comparison, highlighting the potential for technological advancements to destabilize global security and exacerbate existing tensions. It raises fundamental questions about the long-term implications of relinquishing control over lethal force to artificial intelligence. The inclusion of Stanislav Petrov’s story serves as a stark reminder that human judgment, however imperfect, can sometimes be the most effective safeguard against catastrophic error. The closing anecdote—the interviewee's disturbing “reduce the non-American population” declaration—is a chilling illustration of the potential for ideological contamination within a technologically driven military initiative.
Questions detected: If Project Maven continues to evolve towards fully autonomous targeting, how can safeguards be implemented to prevent unintended consequences and ensure compliance with international humanitarian law? What regulatory frameworks are necessary to govern the development and deployment of AI weapons, and who should be responsible for enforcing them?