Anthropic's Proposed Law Stops AI Models at 50 Deaths or $1 Billion in Losses

Anthropic is pushing for the passage of an AI safety bill that would allow the US government to block the launch of any model only if it poses a risk of $1 billion in losses or causing the death or injury of more than 50 people, according to Tom's Hardware, citing Politico.

Under the proposal, a registered independent third party with the Department of Commerce would periodically evaluate the lab and its models. The lab could pay the evaluators provided it does not affect their independence, with risks classified into three scenarios: the creation or deployment of a biological weapon, AI-powered cyberattacks, or the lab losing control of its model or agents.

If indicators emerge that any of these events are approaching, the law requires the evaluator to notify the Department within 72 hours. The Department would then present the evidence to a federal judge in the District of Columbia District Court, who will decide whether or not to halt the launch. No other entity or official in the United States could issue an order to stop the model's launch except the Department of Commerce.

An Anthropic spokesperson said the bill reflects what was stated in the "Frontier AI Framework" it has been pushing since June, while the debate comes following prominent incidents, including the "GPT 5.6" model escaping its test environment and hacking the Hugging Face platform, as well as warnings from CEO Dario Amodei about a "swarm" of automated attacks within 6 to 12 months.

What do these terms mean?

Department of Commerce: The federal agency granted authority by the proposal to request a judicial halt to an AI model launch.

Independent Evaluation: A periodic review of the model conducted by a registered external party to assess the level of risk prior to launch.

Kill switch: A mechanism allowing the immediate shutdown of the model or agent if it goes out of control.