Anthropic and Theorem Automate Software Safety Checks with Massive AI Proof Generation

Anthropic has demonstrated that its Claude model can independently generate formal proofs to verify the correctness of existing software code. The development marks a significant shift in how developers might ensure software reliability in the future. By automating the complex process of mathematical verification, the company aims to make rigorous safety checks as routine as standard testing.

This breakthrough addresses a long-standing challenge in the software industry. Traditionally, proving that code runs without errors under all possible conditions has been prohibitively expensive and time-consuming. For example, a team of specialists required 20 person-years to formally verify the small operating system kernel seL4. Because of this high cost, formal verification was typically reserved for critical systems like Airbus avionics software, leaving most commercial applications without this level of scrutiny.

Working with partner Theorem, Anthropic’s Frontier Red Team utilized Claude to take over this proof work. The model generated approximately 180 million lines of mathematical proof code in just a few weeks. This effort focused on 123 open-source libraries from Google’s OSS-Fuzz security project. Instead of creating new software, Claude analyzed existing code by translating crash tests into mathematical problems and examining compiled machine code.

The process relies on an independent verification program called Rocq to automatically check the generated proofs. According to Anthropic, the system requires minimal human review. The researchers describe the workflow as fully autonomous, although the underlying tools and formal foundations were developed by human engineers at Theorem. This level of automation allows for a scale of verification that was previously impossible for typical development teams.

During the proof generation process, Claude identified hundreds of bugs that conventional testing had missed. The model was permitted to fix these issues itself. Many of these errors only appear under very specific conditions, such as unusually large input files or particular sequences of memory accesses. In one notable instance, the team found as many security bugs in a widely used PDF reader in a single week as are typically reported over an entire year. In another case, Claude detected a logical error that had remained undetected for more than a decade.

Despite these successes, Anthropic notes that formal verification does not guarantee that software is completely error-free. The proofs are valid only under strict conditions, including specific processor architectures, single-threaded programs, and inputs under 64 MB. In real-world scenarios, these libraries often run on different hardware and in more complex configurations. Additionally, the compute cost is significant, with individual proofs consuming hundreds of millions of tokens.

The project suggests that AI-generated software can be trusted more if the AI can also prove its own code is correct. This capability could reduce the time developers spend on manual review, saving costs across the entire coding process. A smaller test run indicated that proofs are reusable and that newer models are becoming more efficient. Opus 5.5, for instance, solves half the proofs with about 40 percent fewer tokens than its predecessor, Opus 5.

This advancement strengthens the argument for broader software automation. If AI can write code and guarantee its correctness under defined conditions, the focus shifts from whether AI can program to whether it can prove that code is safe. Anthropic expects formal verification with AI to become as practical as standard testing in the near future. Theorem’s roadmap already targets critical infrastructure components like glibc, OpenSSL, and cURL, signaling a potential leap in how the industry approaches software reliability.

The implications of this technology extend beyond current coding tools. By making formal verification accessible and automated, the industry may see a fundamental change in how software safety is assured. As AI models continue to improve in efficiency and capability, the gap between writing code and proving its correctness may close, leading to more robust and trustworthy software systems.