California governor approves AI law requiring safeguards from developers

SACRAMENTO, Calif. (CN) — A California bill imposing guardrails on the development of artificial intelligence will become law effective Jan. 1.

Governor Gavin Newsom signed Senate Bill 53, known as the Transparency in Frontier Artificial Intelligence Act, on Monday. It will require certain AI developers to produce safety frameworks, reveal transparency reports and report critical safety incidents to the state Office of Emergency Services.

It also makes whistleblower protections for employees who report AI safety violations. Additionally, the bill will establish a group to design the foundation for CalCompute, a public cloud platform designed to foster safe and equitable research in AI.

“California has proven that we can establish regulations to protect our communities while also ensuring that the growing AI industry continues to thrive,” Newsom said in a statement. “This legislation strikes that balance.”

State Senator Scott Wiener, a San Francisco Democrat and the bill’s author, highlighted the potential dangers AI poses as it grows in power. The technology has brought benefits in areas ranging from improved medical diagnostics and climate modeling to enhanced wildfire prediction and agriculture.

But a state panel of AI experts, known as The Joint California Policy Working Group on AI Frontier Models, convened by Newsom, warned last year that the technology also poses serious risks, including potential use in chemical, biological, radiological and nuclear weapons.

“The California report on frontier AI policy acknowledged that ‘if those whose analysis points to the most extreme risks are right … then the stakes and costs for inaction on frontier AI at this current moment are extremely high,’" said Thomas Woodside, co-founder and senior policy advisor at the Secure AI Project, in a statement.

Some AI firms, including Google and OpenAI, have voluntarily agreed to adopt safety testing and security protocols. Wiener said his bill would codify those commitments, creating a level playing field and boosting accountability.

“With a technology as transformative as AI, we have a responsibility to support that innovation while putting in place commonsense guardrails to understand and reduce risk,” Wiener said. “With this law, California is stepping up, once again, as a global leader on both technology innovation and safety.”

Newsom regularly touts the state as a leader in AI, saying it dominates the industry. His office on Monday said that California has 32 of the top 50 AI companies in the world. Last year, almost 16% of all national AI job postings were for California, making it the top destination for those jobs.

Additionally, the Golden State has three of the four companies with a valuation over $3 trillion — Apple, Google and Nvidia.

Senate Bill 53 isn’t Wiener’s first attempt at imposing rules on AI development in the state.

Last year, he introduced Senate Bill 1047 to require AI developers and computing providers to adopt safeguards preventing catastrophic harm. Newsom vetoed it, saying the measure lacked flexibility, though he agreed on the need to act before a major incident.

The governor instead created the Joint California Policy Working Group on AI Frontier Models to study AI’s growth, abilities and risks, and to recommend workable guardrails.

The group issued its final report in June, advocating a “trust but verify” framework to establish guardrails that lower risks while promoting innovation.

“SB 53 shows what’s possible when policymakers act with foresight,” said Nick Beckstead, CEO of the Secure AI Project, in a statement.