California Wants to Write America's AI Rules

Newsom's expert panel has two months to consider new AI rules, including "kill switches," that could spread nationwide.

With AI industry leaders calling for government intervention and Congress stalled, California is taking the issue into its own hands. California Gov. Gavin Newsom announced an executive order to "dramatically accelerate" oversight of AI companies and possibly implement an AI "kill switch." Newsom cracking down on California's tech industry is nothing new, but in this case, the stakes extend far beyond state lines.

​The executive order convenes a group of "world-leading experts" to recommend ways California could strengthen its AI safety and security laws, giving them a two-month deadline. The group will consider requiring frontier AI companies to host independent auditors in their labs and subject them to third-party mandated safety plans. They will also examine how to advance the creation of a "kill switch" for frontier models, and expand the definition of "critical safety incidents" to include loss-of-control events such as the Hugging Face attack.

The order cites the need for more stringent regulation following the Hugging Face attack and industry leaders' call to action. It also accelerates the implementation of Senate Bill 813 and Assembly Bill 1405, which create a state framework for certifying and registering independent AI auditors but do not require developers to submit to audits.

​The announcement describes California's AI framework as a "guidebook for the nation." Adam Thierer, a senior fellow at the R Street Institute, sees it differently.

"It's not a guidebook," he tells Reason. "It's a rule book."

AI systems transcend state borders, and given California's outsized role in the market, working around its laws is hardly an option; therefore, California law effectively becomes the national standard. This phenomenon is known as the "California effect," and it has reared its ugly head in labor and environmental policy.

Constitutional limits restrict states' abilities to regulate business across state lines. The dormant Commerce Clause doctrine restricts state laws that unfairly hinder interstate commerce, even when federal laws are absent. However, the Supreme Court has narrowed this doctrine in recent years, and companies have been reluctant to challenge these kinds of state laws.

In a recent Facebook post, Newsom wrote: "With Donald Trump and Congress failing to lead, California is stepping up to keep all Americans safe." Newsom frames the issue as a straightforward matter of safety, but he knows as well as anyone that regulating AI has trade-offs.

In 2024, Newsom vetoed Senate Bill 1047, which included provisions similar to those he is now pushing in the executive order. The bill would have created a state entity to oversee frontier models and required mandatory third-party audits, safety testing, and "kill switch" capabilities.

​In his veto statement, Newsom expressed concerns that the bill could give the public a "false sense of security" by focusing on restricting large-scale models when smaller models could pose similar risks. He also warned, "If we over-regulate, if we overindulge, if we chase a shiny object, we could put ourselves in a perilous position." While the backdrop has changed drastically in the last two years, his concerns are still pertinent today.

It's not clear that government regulation will actually translate to increased safety. Discussing a proposed federal kill-switch mandate in July, Thierer told Reason that regulation could create "a sort of false baseline of safety" that companies are content to meet rather than exceed. His concern is that relying on regulators to set safety standards, rather than individual companies, could dampen the incentive to compete on safety.

​Nvidia CEO Jensen Huang weighed in on the issue in a recent interview with CBS News Sunday Morning: "We have laws of all kinds that come after you, and so the companies that are developing these technologies are perfectly incentivized to do the right thing."

While the executive order's proposed on-site audits and shutdown requirements are applied only to "frontier models," states leading the way on AI policy could have much broader consequences. Thierer warns that a patchwork of state regulations, instead of a consistent federal framework, could create a mess of overlapping regulation, benefiting larger players while burying smaller ones. He calls this "a complete inversion of the model that has governed the internet and digital commerce and speech for the last 30 years."

​"California is once again attempting to position itself as a super regulator for the entire nation," Thierer tells Reason. "In this case, they're looking to be the overlord of all things AI." The details of the executive order are still yet to be ironed out, and the ramifications remain to be seen, but the idea that one state can dictate policy all across the nation is concerning, particularly with a technology as revolutionary as AI.