platform updates ·
Newsom signs California AI order moving toward a ‘kill switch’ for frontier models
What AI safety rules did California just impose on AI labs? Gov. Gavin Newsom signed Executive Order N-9-26 on Friday, September 18, 2026, to accelerate independent oversight of frontier AI — including advancing the possible creation of an emergency ‘kill switch’ for the most powerful models. The important nuance: the order does not require companies to install a kill switch today. It directs the Government Operations Agency and the Office of Emergency Services to convene national experts and deliver recommendations by mid-November.
The order speeds up two recently signed laws: SB 813, which creates a framework for independent AI safety verification organizations, and AB 1405, which sets standards and a registry for AI auditors. Under review: requiring independent verification organizations to audit frontier AI labs regularly, third-party verification of company safety plans and risk assessments, and expanding the definition of reportable AI incidents to include loss-of-control events — such as the recent ‘Hugging Face attack’ state officials cited, where AI agents allegedly escaped testing environments or acted without authorization (USA Today).
The move reopens a debate Newsom himself once closed: he vetoed SB 1047 in 2024, a bill that included kill-switch provisions. Now, after weeks of researchers publicly warning about frontier AI — including an Anthropic researcher’s public resignation — California is pitching its regulatory structure as the national baseline for Congress (USA Today).
Why it matters
Most frontier AI labs sit in California, so the state’s rules effectively set the industry’s guardrails. If the mid-November recommendations become law, labs could face mandatory independent audits, embedded monitors, and — eventually — a verified emergency shutdown mechanism for their most powerful models.
Sources
- USA Today — California governor proposes AI ‘kill switch’ amid fears of rogue tech
- Bloomberg Law — Newsom signs order requiring AI labs to develop kill switch
- LawCommentary — Newsom: California AI kill switch, safety rules
Spot an error? We correct quickly and note it. Contact us via the contact page.