CA SB1047: Safe and Secure Innovation for Frontier Artificial Intelligence Models Act.

In one paragraph

Vetoed. California's SB 1047 (2024) would have required developers of covered AI models (trained with more than 10^26 operations at a compute cost over $100 million) to adopt safety and security protocols and full-shutdown capability before training, assess and guard against critical harms before release, hire annual third-party auditors from 2026, and report AI safety incidents to the Attorney General within 72 hours. The Attorney General could have sued for penalties. Governor Newsom vetoed it September 29, 2024, and the Legislature did not override the veto.

Plain-English summary, written with AI assistance. The official text controls.

What it means for business

None: the bill was vetoed and never became law. It would have required developers of the largest AI models to adopt safety protocols and shutdown capability, commission annual third-party audits and report safety incidents to the Attorney General.

Key provisions

1. 1Covered model (before January 1, 2027): an AI model trained with more than 10^26 integer or floating-point operations costing over $100,000,000 at average cloud-compute market prices, or a covered model fine-tuned with at least 3 x 10^25 operations costing over $10,000,000; later, compute thresholds set yearly by the Government Operations Agency
2. 2Before initial training, developers would have had to implement cybersecurity protections, the capability to promptly enact a full shutdown, and a written safety and security protocol, publish a redacted copy, send it to the Attorney General and review it annually
3. 3Before commercial or public use, developers would have had to assess whether the model could cause or materially enable a critical harm (such as mass casualties from a chemical, biological, radiological or nuclear weapon, or at least $500 million in damage from cyberattacks on critical infrastructure), and could not use or release it if there was an unreasonable risk of that
4. 4Annual independent third-party audits beginning January 1, 2026; annual statements of compliance to the Attorney General signed by the chief technology officer or a more senior officer; and reports of each AI safety incident to the Attorney General within 72 hours of learning of it
5. 5Operators of computing clusters would have had to collect the identity, payment details and business purpose of customers using enough compute to train a covered model, assess whether they intended to do so, keep records for seven years and be able to shut down customer resources
6. 6Attorney General civil actions, with penalties up to 10% of the model's training-compute cost for a first violation occurring on or after January 1, 2026 that caused death, bodily harm, property harm or an imminent public-safety threat (30% for later ones); whistleblower protections for employees reporting to the Attorney General or Labor Commissioner
7. 7A nine-member Board of Frontier Models in the Government Operations Agency would have had to approve that agency's regulations and guidance (updated covered-model thresholds, audit rules) before they took effect; a CalCompute consortium would have reported a public cloud computing framework if funded
8. 8Passed the Senate August 29, 2024 (concurrence 30-9); vetoed by the Governor September 29, 2024, who said its focus on only the most expensive, large-scale models could give the public a false sense of security and that it applied stringent standards regardless of whether a system is deployed in high-risk settings; the Legislature did not override it by the November 30, 2024 deadline

Latest legislative action

Sep 29, 2024. Vetoed by the Governor.

Sponsors (showing 5 of 29)

Roll call votes (showing 3 of 10)