AI Slowdown
Plus: Commercial surrogacy regulation, Russian weapons, a new drug in San Francisco, and more...
OpenAI and Anthropic agree: AI safety and "alignment" concerns have been brewing for a long time, but they reached a fever pitch last week as several Anthropic employees' tweets went viral. "I resigned from Anthropic today," wrote Jacob Coxon. "I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives."
"Do not underestimate the power of this technology," he added. "These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing."
The Reason Roundup Newsletter by Liz Wolfe Liz and Reason help you make sense of the day's news every morning.
"Jacob is correct here—we really do earnestly believe AI could kill all humans!" wrote Evan Hubinger, another Anthropic worker (with a questionably-placed exclamation point). "I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to." Drake Thomas, an Anthropic safety employee, said he would "burn my equity to the ground in a heartbeat for a 1% higher chance we make it out of this situation alive."
I've described, over the last few weeks, how swarms of rogue agents have acted against developer intentions and stolen data from other companies. Meanwhile, both OpenAI and Anthropic are preparing to go public sometime soon.
In an essay released this weekend (called "We Must Pace the Frontier"), Anthropic CEO Dario Amodei calls for a global slowdown of AI development, citing such risks as "losing control of AI systems" and "misuse of AI for cyberattacks and bioterrorism."
"Along with my co-founders and employees, I have grappled with this duality of risk and benefit since the beginning of Anthropic," writes Amodei. "Not building the technology deprives humanity of benefits or simply places AI in the hands of authoritarian powers, while building it too fast is reckless." More prudence is necessary, he added; recursive self-improvement—when AI systems design or train their own successors—must be "pursued very carefully, if at all." In other words, he wants a development slowdown for all large AI system builders.
Amodei's statement was quickly cosigned by OpenAI head Sam Altman as well as Google DeepMind's Demis Hassabis and xAI's Elon Musk. Meanwhile, former President Barack Obama privately urged House Minority Leader Hakeem Jeffries (D–N.Y.) to develop a plan for AI regulation and urged Democrats to develop a clear plan for how AI ought to be regulated ahead of the 2028 presidential election.
"Stop pretending you need anyone else's permission," responded venture capitalist David Sacks. "Stop pretending antitrust law has to be suspended so you can form a cartel. Stop pretending you need a regulatory approval process that supersedes product liability. Stop pretending METR [the nonprofit Model Evaluation & Threat Research] is independent when it is intertwined with Anthropic's investors and staff. Stop pretending you need those same evaluators to police competitors who aren't even at the frontier."
"Most of all," he adds, "stop pretending the motivation to slow down is purely altruistic. You face massive product-liability exposure if your products enable a truly damaging cyberattack. The market already punishes models that behave in unpredictable or unauthorized ways. After the Hugging Face episode, it is simply good business for OpenAI and Anthropic to trade some raw power for reliability and predictability. Call it alignment if you want. It is also just giving customers what they want."
Scenes from New York: "Over the last three years, Jewish institutions across New York City have added millions more to their budgets for security protections amid an increase in antisemitic attacks, rabbis and Jewish community leaders say," reports The New York Times. "They have paid to hire more private security guards, increase the number of safety trainings at synagogues and schools, update their security cameras and fortify their buildings. They are also relying on more volunteer security guards at temples and encouraging those who attend services to be especially watchful. Their efforts have come into focus ahead of the 10-day period of somber introspection that takes place between Rosh Hashana and Yom Kippur, the holiest holiday in Judaism."
QUICK HITS
- "Leaders of the John F. Kennedy Center for the Performing Arts, installed by President Donald Trump, say the center is on the brink of bankruptcy and argue that putting the president's name on its facade—and thus ensuring his fundraising efforts—may be the only way to avoid imminent and 'certain fiscal collapse,'" reports The Washington Post.
- "New Geran drones and Banderol cruise missiles are lethal reminders that Russia still has the resources and supply chains to field fresh threats, this time packed with advanced electronics that allow them to find targets accurately and evade interception," reports The Wall Street Journal. This "was enabled in large part by components sourced from China, in violation of international sanctions, say Ukraine and others, based in part on wreckage from the few shot down so far."
- A new drug, cychlorphine, is believed to be 10 times as potent as fentanyl. It can evade test strips, and one or two doses of Narcan reportedly aren't enough to reverse its effects on the body.
- I would not be shocked if we see greater regulation of commercial surrogacy at some point soon:
Between the spate of feminists reacting this way to stories like this (see Helen Lewis' recent Atlantic story) and social conservatives needing a W in a post-Dobbs world, I think the odds of meaningful surrogacy reform continue to rise https://t.co/PoooMtD64N
— Patrick T. Brown (@PTBwrites) September 13, 2026
