Khanna's "Human Control Over AI Act": A Ban on Recursive Self-Improvement Until the Government Says Otherwise
Rep. Ro Khanna (D-Calif., Silicon Valley) will introduce the Human Control Over AI Act, with a summary shared exclusively with CNBC (Sept 28, 2026). It is the most aggressive legislative translation yet of the pacing debate — and the first proposal to ban the recursive-self-improvement dynamic itself, not just regulate model releases.
Core mechanism: the bill "would ban models that recursively self-improve or autonomously modify their own core objectives, containment or shutdown controls until federal guardrails exist and an agency approves the activities."
What it would build (CNBC, with additional detail from Chosun English):
- A new federal AI agency covering OpenAI, Anthropic, Google DeepMind and xAI: safety regulations, a licensing system for model training and deployment, frontier-model audits, and security standards for continuous testing and "effective human control."
- Independent auditors embedded at every frontier lab, reporting directly to the agency — the same mechanism Amodei proposed (Anthropic Embeds Its First Independent Evaluators: Accenture's Faculty, $1B Each, Employee-Level Access), but reporting to government rather than being contracted by the lab.
- Standards for sandbox testing environments, air gaps, kill switches, and controls preventing models from escaping lab settings; the agency would also regulate the advanced chips frontier labs use, including monitoring and control of their use.
- Criminal penalties: "crimes against humanity" for deploying AI models that result in destruction of civilian populations; criminal liability for employees who disable safeguards, kill switches, logging or containment, or knowingly deploy an unauthorized system; mandatory large-scale liability insurance for model releases.
- Enforceable international agreements and targeted export controls to deter China.
Khanna's framing is explicitly anti-lab-led-governance: "There's actually a civilizational extinction risk... there's a safety risk of loss of control, and then there's a misuse risk, and we need to take both seriously." And: "I want this to be a model to give voice to the AI safety community... they need to be driving this debate, the AI safety community, not the executives of frontier labs." He says the bill was modeled on conversations with METR, MIRI and Palisade Research — "rather than the asks of executives from large AI companies."
It joins the FRONTIER Act (Obernolte-Trahan, auditors + shutdown authority) and the Thune-Cruz-Klobuchar Senate effort (Commerce Secretary pre-release audits). Reality check from CNBC: "No House bills are expected to receive a vote until after the midterm election, and the Senate is expected to leave Washington and not return until after the election after this week."
Why it matters: the bill's premise — that RSI should be legally gated, not self-governed — directly challenges the labs' framing that they should define and measure the pace (The Pacing Debate Ships Product: OpenAI Kills GPT-6.1, Nvidia Ships Agent Containment, the Market Starts Pricing the Slowdown). Even if it stalls until after November, it gives the post-midterm Congress a ready-made maximalist template.