Anthropic Uses Claude to Formally Verify 123 Open-Source Libraries
Anthropic's Frontier Red Team, in collaboration with Theorem, used Claude to prove security properties at the RISC-V assembly level across 123 real-world open-source libraries over a few weeks. Claude also discovered bugs that had eluded detection despite trillions of test runs, marking the first large-scale effort to apply AI-driven formal verification to production software at the assembly level.
Event version 1.
Key insights
Large-scale assembly-level verification
Claude proved security properties at the RISC-V assembly code level in over 100 real-world open-source software libraries. Anthropic describes this as the first large-scale effort to use AI to apply formal verification to production software at the assembly code level.
Bugs found despite extensive testing
While writing the proofs, Claude discovered bugs that had eluded detection for years despite trillions of test runs. The proofs showed that some executable artifacts can never crash under certain conditions.
Collaboration with Theorem
Anthropic conducted the work in collaboration with Theorem. The proofs were produced over the span of a few weeks.
Shift from tests to proofs
Formal verification historically required years of expert effort and was practical only for mission-critical applications such as Airbus avionics code. Anthropic states that LLMs can now produce formal proofs at large scale, enabling software development workflows with proven security guarantees.