OpenAI says it has agreed changes to the “opportunistic and sloppy” deal it struck with the US government over the use of its technology in classified military operations.
On Monday OpenAI chief executive Sam Altman said the company would add the language to its agreement, including explicitly prohibiting the use of its systems to spy on Americans.
The deal had emerged on Friday following a fallout between OpenAI’s rival Anthropic and the Department of Defense, over concerns around the use of its AI model Claude for mass surveillance and in fully autonomous weapons.
But it has raised questions over how AI is used in war and how much power rests with government and private companies.
A statement made on Saturday by OpenAI claimed its agreement with the Pentagon had “more guardrails than any previous agreement for classified AI deployments, including Anthropic’s”.
But on Monday, Altman posted on X to say further changes were being made, including making sure its system would not be “intentionally used for domestic surveillance of U.S. persons and nationals”.
As part of the new amendments, intelligence agencies such as the National Security Agency would also not be able to use OpenAI’s system without a “follow-on modification” to the contract.
Altman added the company had made a mistake by rushing “to get this out on Friday”.
“The issues are super complex, and demand clear communication,” he said.
“We were genuinely trying to de-escalate things and avoid a much worse outcome, but I think it just looked opportunistic and sloppy.”
OpenAI has faced backlash from users following its announcement it was working with the Pentagon.
According to data from Sensor Tower, the number of people uninstalling ChatGPT has surged since the news of OpenAI’s partnership with the Department of Defense was announced on Friday.
The market intelligence firm said the daily average uninstall rate was up by 200% compared to normal rates.
Meanwhile, Anthropic’s Claude rose to the top of Apple’s App Store ranking, where it still remains on Tuesday.
The AI model was blacklisted by the Trump adminstration following Anthropic’s refusal to drop a corporate “red-line” principle that its technology should not be used to create fully autonomous weapons.
Despite this, the use of Claude in the US-Israel war with Iran has since emerged, with BBC News’ US partner CBS News reporting it was still in use on Tuesday.
The Pentagon has declined to comment on its dealings with Anthropic.
How AI is used by the military
AI is used in a number of ways in the military, for example streamlining logistics or quickly processing large amounts of information.
The US, Ukraine, and Nato all use tech from Palantir, an American company which provides data analytics tools to government customers for intelligence gathering, surveillance, counterterrorism and military purposes.
The UK Ministry of Defence recently signed a £240m contract with the firm.
At the end of last year, the BBC spoke to some of those involved in integrating Palantir’s AI-powered defence platform Maven into Nato.
The software brings together a huge range of military information, from satellite data to intelligence reports, which can then be analysed by commercial AI systems such as Claude to help make “faster, more efficient, and ultimately more lethal decisions where that’s appropriate”said Louis Mosley, head of Palantir’s UK operations.
But AI large language models can make mistakes, or even make things up – known as “hallucinating”.
Lieutenant Colonel Amanda Gustave, chief data officer for Nato’s Task Force Maven, stressed there was human oversight, adding that they were “always introducing a human in the loop” and that it “would never be the case” that an AI would “make a decision for us”.
Palantir, unlike Anthropic, does not support a blanket ban on autonomous weapons, but says there should be a “human in the loop”.
But Professor Mariarosaria Taddeo of Oxford University told the BBC that with Anthropic out of the Pentagon, “the most safety-conscious actor” was now “out from the room”.
“That is a real problem,” she added.
Read Also…
Explore the world of impactful news with CitiNewsroom on WhatsApp!
Click on the link to join the Citi Newsroom channel for curated, meaningful stories tailored just for YOU:
https://whatsapp.com/channel/0029VaCYzPRAYlUPudDDe53x
No spam, just the stories that truly matter! #StayInformed #CitiNewsroom #CNRDigital
