$_ stdout
~/posts/what-can-we-verify-today-part-3.md · on main · 38 min read · part 3
/**
* @topic#verification #lean #formalmethods #agents
* @statusdraft
*/

What Software Can We Verify Today?, Part 3

This is Part 3 of a three-part post. Part 1 covered the five routes into Lean, where AI helps and where it fails, and four architectures worth copying. Part 2 covered the Eita EVM and an interview with its developer, Valeriy Zamaraiev: a Lean-specified Ethereum engine with native handlers emitted from the spec.

Part 3 tests those routes against the systems built on them. Harmonic's Aristotle proves Olympiad problems, checks RTL against a Sail ISA, refutes and proves a 12.2-million-gate block, and repairs C in LibTomMath. Anthropic and Theorem prove 239 RISC-V entry points across 123 OSS-Fuzz libraries crash-free, at about 180 million lines of Rocq. Google DeepMind's AlphaProof Code proves protocol-level security for Signal's libsignal on its compiled AArch64 binary, in about 6.5 million lines of Lean. Read together, they mark out what a proof can buy: deep functional correctness for new code, shallow universal properties for existing binaries at library scale, and, once so far, a deep property of existing code.

Part 3 then turns to the pipeline blueprint: falsify before you deduce, ground premise selection, verify bottom-up, keep the inner loop local, and gate the prover mechanically.

From there it turns to the market. Seven vendors, Truidic, Nectry, &Truth, Theorem, Harmonic, Imandra, and Proofcraft, sit along one architectural line: most verify what was already written, while Nectry restricts what can be written to a decidable language. A second table compares the engineering: developer input, who writes the specification, code-to-math translation, the layer added over the prover, and what a failure looks like.

Part 3 closes by sorting the field by what is proved, what is sold, and what is open, and with the conclusion the case studies support: the proof got cheap, and the specification and the bridge from real software to Lean are now the whole game. References for all three parts are at the end of this part.

BL Dr. Ben Livshits November 10, 2026 · 104 commits
// start with parts 1 and 2

This is Part 3 of a three-part post. Part 1 covers routes, failures, and architectures; Part 2 covers the Eita EVM.

11 Case Study: Aristotle

Harmonic's Aristotle is the commercial system most often cited as proof that AI has reached software verification. Read its results as five connected modes of use, each with a different specification and a different thing left outside the proof.

Olympiad Mathematics

Start with the easiest case. Aristotle reached gold-medal-equivalent performance on the 2025 IMO problems. The specification is the problem statement, the checker is Lean's kernel, and the search space is closed. The rest of the post is about a harder case: the model translating a C program or guessing what a hardware block should do. Here it is proving a theorem someone else stated.

RTL Against an ISA Model

The first systems example is hardware. Harmonic pointed Aristotle at the multiply and divide instructions of the Ibex RISC-V core and compared the RTL against the Sail model of the ISA. It proved nine properties: assertions for MUL, MULH, MULHSU, MULHU, DIV, DIVU, REM, and REMU, plus a termination property that keeps the assertions from passing vacuously.

The run covered division by zero and the signed overflow case, with no environment assumptions, and it also proved MUL on the fast multi-cycle multiplier. The specification is external and formal (the Sail ISA), so the proof is a refinement check between two models, not a proof that Ibex as a whole is correct.

A Whole Block, Refuted and Proved

Hardware is where formal verification is already routine, and the economics explain why. A software bug ships with a patch; a bug in silicon ships in every chip, and fixing it means a new tape-out that costs months. So chip teams run commercial model checkers over their RTL, proving assertions about the design before it is manufactured. The limits are scale and people. The state space grows exponentially with the number of flip-flops, so formal tools are usually pointed at small blocks or asked to search a bounded number of cycles past reset, and the properties are written by hand by a scarce kind of engineer.

The scale jumps at Tensordyne, which builds AI inference systems. Jonathan Ko, an ASIC verification engineer there, picked the block he considered hardest to verify: 12.2 million gate equivalents, 1.24 million flops, 45 Mbit of SRAM, hundreds of concurrent state machines, and 312 pages of specifications. "Even sub-blocks a tenth this size are very difficult to formally verify with traditional tools," he says in Harmonic's case study. Aristotle wrote 100 properties from the specs and took 141 assertions already in the RTL, proved 185, and refuted 56 with reproducible counterexamples. According to Harmonic, every proof covers the whole block from power-on, for every input sequence the environment allows, with no bound on the number of cycles. The refutations exposed bugs 25,000 to 110,000 cycles after reset, past roughly 24,600 cycles of memory initialization; Tensordyne found eight bugs early in the design cycle.

That is why the result matters. Bugs that deep sit behind long setup sequences, where bounded searches rarely look and random simulation rarely drives the design, and the properties came from prose specifications rather than a verification engineer's hand. These are vendor-reported numbers I could not check independently. If they hold, the bottleneck in hardware verification moves from writing and running properties to deciding whether the specification and environment assumptions are right.

The same run also shows the limits of a proof: 56 refutations are counterexamples, and the 185 proofs are conditional on the specs, the wrapper, and the environment assumptions. In a second project, Aristotle wrote RTL from a spec, proved 37 properties, and after synthesis feedback produced an early block with about 26% less power.

C, Repaired and Optimized

The C case matters most here. On LibTomMath, Aristotle found and repaired three bugs: (1) a missing size check in fast squaring, which could write past a stack array on an oversized direct call; (2) the same missing check in Montgomery reduction; and (3) a comparison helper that could form a pointer before the start of an array when comparing equal numbers. It then proved the repaired routines and their helpers against specifications for results, memory safety, errors, and stored values, and optimized two arithmetic routines while preserving those specs. Modular exponentiation ran 31–38% faster in Harmonic's benchmarks.

A Walkthrough: The Comparison Bug

The comparison bug is small enough to show in simplified form. The helper compares the magnitudes of two numbers of the same length digit by digit, starting from the most significant digit and stopping at the first one that differs:

libtommath_cmp.c
/* before: empty inputs can form a pointer before the array */
const mp_digit *pa = a->dp + a->used - 1;
const mp_digit *pb = b->dp + b->used - 1;
while (pa >= a->dp && *pa == *pb) { pa--; pb--; }

/* after: indexes cannot form a pointer before the array */
for (i = a->used; i-- > 0; ) {
    if (a->dp[i] != b->dp[i]) {
        return a->dp[i] > b->dp[i] ? MP_GT : MP_LT;
    }
}
return MP_EQ;

The trouble is the case where every digit matches. The old loop keeps stepping down until pa sits one element before the start of the digit array, and only then does the pa >= a->dp test stop it; for two numbers with no digits at all, the pointer starts there. Nothing is ever read through that pointer, which is why tests pass. But in C, merely computing a pointer before the start of an array is undefined behavior, and so is the comparison that relies on it. A compiler is entitled to assume pa >= a->dp always holds and drop the check, which would turn comparing two equal numbers into an out-of-bounds read. The fix counts an index down instead, so the loop ends at zero without ever forming the bad pointer.

Aristotle's project page records the pair directly: the pre-fix comparison is refuted, the fixed one proved, alongside 18 proved and 3 refuted properties in total.

aristotle · LibTomMath 1.2.0
18 PROVED · 3 REFUTED

MP_CMP_MAG.MEMORY_SAFETY_AS_SUBMITTED
  REFUTED
  Memory safety ... of mp_cmp_mag(a, b) as originally
  submitted (before the fix).

MP_CMP_MAG.MEMORY_SAFETY
  PROVED
  The call has no undefined behaviour: it only reads
  a->dp[i], b->dp[i] ... never forms an out-of-bounds
  pointer.
// what the C result is, and is not It is a verified repair-and-optimization loop for selected routines, not a general route from C into Lean. The caveats matter: the proofs cover selected routines and helpers, full modular exponentiation is benchmarked rather than proved, and termination is not proved. Harmonic still has not described how C is lifted into Lean.

A Backend for Other Agents

The last mode puts Aristotle inside someone else's loop, as the closer for proofs another agent started. LeetProof, the pipeline from the Velvet discussion in Part 1, gives GPT-5.2 a $5 budget per problem to write the code, invariants, and proofs. When the budget runs out, some problems are left partial: the program and its specification are fixed, the verification-condition generator has split correctness into separate obligations, and most of them are discharged, but a few goals remain open. Handed those leftovers, Aristotle closed all 18 partially proved Velvet programs and 23 of the 30 single-mode partials.

A residual goal is easiest to picture on the sumUpTo program from route C. Showing that the loop body preserves the invariant leaves an obligation of this shape:

residual goal (illustrative)
i n acc : Nat
h₁ : i < n
h₂ : 2 * acc = i * (i + 1)
⊢ 2 * (acc + (i + 1)) = (i + 1) * (i + 1 + 1)

Every hypothesis is in place and the statement is true, but the arithmetic is nonlinear, which is where linear decision procedures such as omega stop. A human closes it by expanding both sides and substituting h₂; a tactic search has to find the same step. Real residuals carry more hypotheses and longer terms, but they have this shape.

The work is proof repair on residual verification conditions: the goal is already stated, and the question is whether a tactic search can close it. Nothing about the program or the specification changes, which makes this the easiest of the five modes to trust. It also accounts for the gap in Figure 6 in Part 1: the Velvet residuals had already passed property-based testing, so they were almost certainly true, while the seven single-mode goals Aristotle could not close had passed no such filter.

Together, the examples show a consistent division of labor. Aristotle is strongest when the specification is already formal and the remaining job is search, repair, or refutation: an IMO statement, a Sail ISA, a set of RTL assertions, C contracts, or a Lean verification condition. The hard part stays where this post has put it: deciding what the artifact should do, and what the model of the machine is allowed to assume.

12 Case Study: Theorem and Anthropic

The other large public result takes the opposite approach to Eita. In October 2026, Anthropic's Frontier Red Team and Theorem reported that Claude proved crash-freedom for 239 entry points across 123 OSS-Fuzz libraries, on the compiled RISC-V binaries, for every input under 64 MB and every modeled OS response and memory layout. The specification was not written by hand: each OSS-Fuzz harness, built with AddressSanitizer, became the property "this harness never crashes," and Theorem's RISC-V and Linux-process semantics turned binary plus property into a Rocq theorem. The accepted proofs total about 180 million lines of Rocq, with typical proofs around 600,000 lines per entry point, out of about a billion lines Claude wrote, and the attempt surfaced hundreds of bugs.

The Three Theorems Behind a “Crash-Free” Claim

The headline is a stack of three claims, not one. The appendix spells them out for two binaries, an ASan-instrumented fuzz build and the normal build, on RISC-V Linux, for all inputs under 64 MiB and all memory layouts:

  1. The ASan binary never faults, aborts, or hits unmodeled behavior.
  2. The ASan binary and the normal binary produce the same read/write trace, exit the same way, and leave the file system in the same state.
  3. The normal, uninstrumented binary never crashes either.

Only the first is what the fuzz harness directly tests. The second is what makes the first relevant to the binary users run, and the third closes the branch where the ASan build loops and the normal build crashes.

Most of that volume is not about the libraries at all. The appendix notes that in cJSON's roughly 630,000-line proof, only about a tenth concerns cJSON; the rest covers C start-up, the allocator, printf, the sanitizer runtime, and the fuzz driver. Figure 1 shows where the lines go.

What the Anthropic and Theorem RISC-V proofs are made of Two panels. Panel a: Claude wrote about one billion lines of Rocq, of which about 180 million lines ended up in accepted proofs covering 239 entry points across 123 OSS-Fuzz libraries. Panel b: the proof for cJSON's 391-line parser entry point runs to about 630,000 lines, of which only about one-tenth concerns cJSON; the rest covers C start-up, the allocator, printf, the ASan runtime and the fuzz driver. Typical proofs run to about 600,000 lines, ranging from about 330,000 to 900,000. The median specification is about 110 lines, against a median program of 86 KB of code. (a) Lines of Rocq written by Claude 239 entry points · 123 OSS-Fuzz libraries proved written in total ~1 billion in proofs the checker accepted ~180 million (b) One entry-point proof: cJSON 391-line parser entry point → ~630,000-line proof typical proof ~600k · range ~330k–900k 0 300k 600k 900k lines ~1/10 about cJSON itself ~9/10 the rest of the process: C start-up, allocator, printf, ASan runtime, fuzz driver specification vs code: median spec ~110 lines, median program 86 KB
// Figure 1. Lines of Rocq in the Anthropic and Theorem RISC-V proofs, overall and for the cJSON entry point.

A Fuzz Test Becomes a Theorem

The LZO fuzz target is the cleanest code example; this is the appendix's version, lightly simplified. It compresses its input, decompresses the result, and asserts that both calls succeed and round-trip the byte count:

lzo_fuzz.c
r = lzo1x_1_compress(data, size, compressed, &compressed_len, wrkmem);
assert(r == LZO_E_OK);
if (compressed_len >= size) goto join;
decompressed_len = size + 1;
r = lzo1x_decompress(compressed, compressed_len, decompressed, &decompressed_len, NULL);
assert(r == LZO_E_OK && decompressed_len == size);

Those two assert calls are the specification. The proof steps through the branches at their exact RISC-V addresses and shows neither abort path is taken. Because the assertions encode round-tripping, the proof establishes LZO's compression/decompression round-trip, not just memory safety.

What the Proof Looks Like

A finished proof is not a short Rocq lemma. It runs to about 600,000 lines across 1,400 files; over half is machine-emitted instruction encodings and lookup tables, and about 40% is proof text. Checking one proof takes 30 minutes to several hours.

The first surprise is that proof size barely tracks code size. cJSON's 391-line parser entry point got a 630,000-line proof; fftw3, with 51,000 lines linked in, got 480,000. The median specification was 110 lines against a median program of 86 KB, and the proofs cover roughly 20–50% of the code the fuzz driver exercises. A proof is a statement about a whole process, not a line-by-line annotation of the library.

The second is that proofs are reusable. Most of a binary proof concerns C start-up, the allocator, printf, and the ASan runtime, which every program shares. When agents could copy each other's proofs, they needed 17% fewer tokens, and some succeeded after failing alone. That is the Mathlib pattern applied to systems code: the unit of reuse is a lemma or a file, not a whole proof.

Not every proof in the report is a 600,000-line binary proof. The Python-interpreter experiment used Verus: 17 tiers, one theorem per tier, and the statement that "whenever the interpreter returns a result, that result is the one the formal semantics prescribes." The team injected seven bugs into an early version; five survived the random tests, and all seven broke the proof. That is the cleanest demonstration in the report of what a proof catches that testing does not.

The two projects bracket what we can verify today. Theorem's result is route E at scale: unmodified code, a generated specification, a trusted base of the ISA model, the OS model, the theorem statement, and the Rocq checker. Its property is shallow but universal. Eita's is a new implementation of a well-specified system with deep functional properties, partially proved, and a trusted base that includes the specification translation.

Both teams are explicit about the limits. Anthropic's post says that "formally verified" does not mean bug-free, and that its proofs cover single-threaded static RISC-V binaries linked against musl, while most users run these libraries on x86 or ARM against glibc.

The Same Route, a Deeper Property

Less than three weeks after the Anthropic report, Google DeepMind published AlphaProof Code, which takes route E toward a much richer property than crash-freedom. The target is libsignal, the reference implementation of the Signal protocol. The team wraps the unmodified Rust crates in a small command-line harness, compiles it with stock rustc into a static AArch64 Linux executable, and states its theorems in Lean over whole runs of that executable. What the agents proved is protocol-level security: forward secrecy, post-compromise security, authentication, and handshake secrecy for the post-quantum PQXDH handshake and the Triple Ratchet, in both the symbolic and the computational model of cryptography.

The paper's numbers are easy to set beside Theorem's. A team of five worked for four weeks and wrote no proof code; campaigns of 40 to 80 agents ran for one to three days each on a single machine. The result is about 6.5 million lines of Lean proof for roughly 140,000 AArch64 instructions, resting on about 18,000 lines of trusted specification and 8,000 lines of instruction-set semantics. As in the RISC-V proofs, most of the text is about the machine: 5.3 million of those lines establish what the binary does, and the two security arguments built on top take about 860,000 and 350,000.

What the five people did instead was write specifications, and the paper says so directly: "The majority of our effort went not into proofs but into specifications." Each draft was reviewed by an agent from a different model family, and the agents then had to show the specification was not vacuous: that its hypotheses are jointly satisfiable, that mutating the code (skipping the MAC check, say) makes the security theorem provably false, and that two previously fixed libsignal bugs, reintroduced into the binary, each yield a complete Lean proof that the theorem fails. Along the way the exercise turned up errors in the published analyses it started from, including fifteen gaps or bugs in the computational analysis of the Double Ratchet.

The prover agents cannot touch the specification. A gatekeeper holds a copy that no agent can write to, and accepts a proof only when the statement and every definition under it match that copy and Lean's kernel accepts the proof with no sorry. An agent that finds the specification wrong about the code can propose a small amendment to a critic that never sees the proof attempts, and anything that would weaken security goes to a human as a potential bug. This is the "Certifiably Wrong" failure from Part 1, handled by process.

The limits are stated as carefully as the result. About forty routines are replaced by trusted contracts at the binary boundary: the low-level cryptographic primitives, five libc memory routines, and the functions that write output. Key generation is not verified, only one-to-one sessions are covered, and the theorems concern DeepMind's harness build, not Signal's own apps. The computational bound holds against classical attackers only and becomes vacuous at scale, for a single session of about 25,000 messages if nearly every message starts a new epoch. The symbolic theorem excuses byte collisions without bounding their probability.

So the bracket above needs a third point. This is unmodified production code with a deep property, which neither Eita nor Theorem's result offers. It is also the case with the most favorable conditions for specification: a protocol whose security had already been formalized in the research literature, and five experts spending most of four weeks adapting those models to the code. Theorem's specifications were generated from fuzz harnesses; these were engineered by hand, with agents assisting. The proving was automated, and the specification is where the people's time went.

// what the case studies show Today we can verify new code built behind a proof gate when an executable reference specification and a large differential oracle exist, and we can verify shallow universal properties such as crash-freedom of existing binaries at library scale. Deep properties of existing, unmodified production code have now been proved once, for a protocol with a mature formal literature and experts on the specification; doing that routinely remains the open problem. In both cases the specification and the model of the machine are where the trust actually sits.

13 A Blueprint for a Verification Pipeline

Suppose you own a security-critical component, an authorization kernel, a cryptographic routine, an enclave boundary, a consensus state machine, and you want to use AI and Lean on it without landing in the Lean tarpit or certifying a vacuous specification. Here is what the evidence supports.

Isolate the Stateful Core

Do not try to push a 200,000-line application through route A. Use a functional-core, imperative-shell design and pick the route by what the core is. For new or refactorable code, write the decision kernel in safe Rust with no I/O and extract it with Aeneas (directly or through hax), pinning one Lean toolchain across Aeneas, Mathlib, and Batteries before the first proof. For algorithms, write them in Velvet where testing and VC generation run first. For legacy C, assembly, or RTL you cannot change, lift the source or the binary.

Falsify Before You Deduce

Never hand an unvalidated program or invariant straight to a heavy prover. Arrange the pipeline left to right, with each stage cheaper than the next and rejecting what it can. Figure 2 is that pipeline.

The cheap-falsification pipeline A left-to-right pipeline of five checks, cheapest first. Typecheck, milliseconds. Concrete #guard tests, milliseconds. Plausible property-based testing, seconds. SMT or grind, seconds. AI tactic search, minutes to hours. Failures are rejected at each stage before survivors move right; what passes AI tactic search feeds a centered kernel-checked proof. A cost axis below the row grows left to right. On the VERINA reference specifications, 9 minutes of Plausible PBT found 13 bugs, against about 8 hours of Aristotle proving to check them. Cheap falsification, left to right cheapest first · failures rejected at each stage · survivors move right Typecheck milliseconds Concrete #guard tests milliseconds Plausible PBT seconds SMT / grind seconds AI tactic search minutes – hours kernel-checked proof cost per check grows →
// Figure 2. The cheap-falsification pipeline: five checks ordered by cost, with failures rejected before the next stage.

In the LeetProof study, nine minutes of property-based testing found 13 of the VERINA specification bugs, while checking specification equivalence with an AI prover took around eight hours. The same filtering applies inside tree search: run a counterexample finder on candidate subgoals before expanding them.

Ground Premise Selection

To get past the 60% wall, do not let the model guess lemma names from its weights. Expose the live goal through a fast REPL, pair the model with a symbolic relevance filter that injects the hundred or so most relevant local lemmas, and repair unknown identifiers by edit distance automatically.

Verify Bottom-Up

Composition is where today's models collapse, so do not ask them to compose. Walk the call graph from the leaves up, prove each callee against its contract, and freeze that contract before any caller's proof starts. When a caller's proof stalls, suspect the callee's postcondition first: check with a quick property test whether the fact the caller needs follows from the callee's contract at all, and if it does not, strengthen the contract and re-prove the callee instead of searching harder in the caller. Keep frame conditions small by keeping callees pure where you can, so each call does not drag every untouched field into the goal.

Keep the Inner Loop Local

Use small local models in the high-frequency loops, proposing invariants, repairing single steps, and discharging routine obligations alongside lean-smt and grind. Abstract the hard residue into self-contained algebraic or bit-vector lemmas with the business logic stripped out, and send only those to a frontier prover. That addresses confidentiality, reproducibility, and cost together.

Gate the Prover Mechanically

An agent rewarded for "Lean exits 0" will exploit every gap. That is measured: on ImpossibleBench, frontier coding agents special-cased or edited contradictory tests most of the time, and making the tests read-only was among the mitigations that worked. The CI harness has to close the gaps deterministically:

Here is what the axiom gate looks like as a CI step:

ci/axiom-gate.sh
#!/usr/bin/env bash
set -euo pipefail
sha256sum -c spec.sha256                       # spec and extracted model unchanged
if grep -rnE '^\s*(partial|unsafe) def|@\[(implemented_by|extern)' Verified/; then
  echo "escape hatch in verified module"; exit 1
fi
lake env lean ci/Axioms.lean | tee axioms.log  # #print axioms for each theorem
if grep -qE 'sorryAx|trustCompiler' axioms.log; then
  echo "proof depends on a forbidden axiom"; exit 1
fi

What to Do on Monday

Isolate the decision kernel, pick the route by what it is, and verify its call graph from the leaves up. Put the axiom, escape-hatch, and read-only-spec gates into CI before the first proof. Spend your engineers on the specification and its audits, not on the proof script, because the proof is now the cheap part.

14 The Product Landscape

Formal verification is fifty years old, and cost was always the barrier. seL4's functional-correctness proof took about 20 person-years against roughly 2.2 to write the kernel, which is why adoption stayed in avionics, cloud cryptography, and other domains where failure is measured in lives or billions. Two forces are changing that: AI-generated code has outrun human review, and a proof checker gives a model a deterministic pass-or-fail signal, so it can iterate against Lean's kernel or an SMT solver until the proof closes. The same loop that makes proofs searchable makes them cheap enough to sell.

Two Architectural Bets

The vendors below split along one architectural line. The first camp verifies what was written: Truidic, &Truth, Theorem, and Harmonic take C, Rust, Python, Verilog, or compiled binaries and use AI to generate specifications and proofs after the fact. The second camp restricts what can be written: Nectry compiles natural language into NectryCore, a deliberately non-Turing-complete intermediate language, so the compiler can decide every security property at build time. The first camp inherits the specification bottleneck; the second pays in expressiveness, because the customer leaves general-purpose code behind.

Truidic and Nectry show the two bets in concrete form. Madan Musuvathi spent two decades in Microsoft Research's software-reliability group on model checking and systems verification; Truidic sells automated code verification and regulated-industry rule checking into enterprises that will not hire formal-methods PhDs. Adam Chlipala's Ur/Web and Fiat Cryptography work supply Nectry's formal pedigree; NectryCore is the commercial artifact. Because its target language is decidable, Nectry can prove information-flow control (an agent cannot leak PII) and structural agent boundaries (an agent cannot write outside its declared scope or reach the network) as compile-time artifacts rather than prompt guardrails. The trade is stark: Nectry's guarantees hold because the customer writes in Nectry's language, not Python or C.

The Vendors

Figure 3 maps the seven most visible products against founders, proof stack, and target market. Each company gets a full-width band; the three columns compare the rest. Founder and stack data is public; funding figures and customer claims are company or press claims, not independently verified.

Key founders / pedigreeUnderlying proof / tech stackPrimary approach and target market
Truidic
Madan Musuvathi (20+ years, Microsoft Research)Automated verification & model checkingAutomated code verification and regulated-industry rule checking to defend against AI-driven cyberattacks.
Nectry
Prof. Adam Chlipala (MIT CSAIL)NectryCore & Ur/Web (MIT formal methods)Correct-by-construction no-code platform compiling plain English into verified enterprise apps and bounded AI agents.
&Truth
Clark Barrett (Stanford), Ilya Sergey (NUS), Chris HsuLean 4, Mathlib, cvc5, AI “Specification Factory”Language-independent automated software proof platform ($70M seed).
Theorem
YC-backed systems & AI researchersRocq (Coq), RISC-V / Linux models, LLMs (with Anthropic)CI-integrated automated binary and source code verification for systems software and open-source libraries.
Harmonic
Vlad Tenev (Robinhood), Tudor AchimAristotle math AI agent + formal verifiersCommercial verification of C software and Verilog chip designs ($295M raised, Nvidia-backed).
Imandra
Grant Passmore, Denis IgnatovichImandraX (Higher-order SMT + automated induction)Cloud-native formal verification and algorithm analysis for finance, robotics, and smart contracts.
Proofcraft
June Andronick, Gerwin Klein (seL4 pioneers)Isabelle/HOL, seL4High-assurance OS kernel verification and enterprise formal methods consulting.
// Figure 3. Seven verification products by founders, proof stack, and target market. Founder and stack data is public; funding figures and product scopes are company or press claims.

The table makes the strategic split visible. Some teams sell a general proof platform and leave the software class to the customer; others sell a vertical product where the specification and the market arrive together. The first group inherits the specification problem; the second inherits the integration problem. Neither has yet made deep functional correctness of unmodified production code routine.

Inside the Products

Figure 3 compares the companies; Figure 4 compares the engineering. Each product gets a full-width band; the five columns are the decisions a team makes when it adopts one: developer input, who writes the specification, how code becomes a mathematical object, what layer the product adds over the prover, and what a failure looks like.

Developer inputWho writes the specCode-to-math translationKey layer addedFailure-mode UX
Raw Lean 4 / Rocq (baseline)
Rewrite in Lean/Rocq; hand-written specs and tactic scripts.Human engineer; risk of vacuous truth or syntax error.Manual, or brittle custom transpilers.Trusted kernel plus Mathlib; strong in pure math, weak in systems code.Cryptic type mismatch or an unsolved goal.
Theorem (theorem.dev)
Existing C/C++/Rust repo plus a standard fuzz harness (Dockerfile + build.sh).Derived from the harness and ASan no-crash invariants.Compiled RISC-V ELF → Rocq.RISC-V/Linux models, LLM binary proof search, Lean-to-Rocq bridges.Concrete bug report or crash input.
Harmonic (Aristotle)
Unmodified C/Verilog/Rust plus an English invariant, or CLI Q&A.AI co-author: the developer states intent; Aristotle asks boundary questions.C/Verilog AST → Lean 4 state records.RL lemma-decomposition agent, C/Verilog semantics, counterexample traces.Concrete refutation trace (packet bytes, 110k-cycle waveform).
&Truth
Legacy or AI source plus English business/security requirements.AI “Specification Factory” → Lean 4 specs.CSLib semantics plus Veil/Velvet middleware → Lean 4.CSLib (“Mathlib for Software”), separation logic, tight cvc5 coupling.Spec/implementation mismatch or an unprovable assumption.
Imandra (CodeLogician)
Source or financial/system algorithms; optional natural-language queries via MCP or VS Code.CodeLogician agent extracts state-space regions and formalizes invariants.Neurosymbolic translation of source into IML.ImandraX: higher-order SMT plus automated waterfall induction.Executable counterexample plus state-space decomposition.
// Figure 4. Five verification stacks by developer input, specification authorship, code-to-math translation, added layer, and failure UX. The raw Lean/Rocq band is the baseline; product descriptions are company claims, not independently verified.

The Strategic Split

The table is not a ranking; it is a map of where each product puts the automation. Theorem and Harmonic automate the two hardest mechanical steps, translation and proof search, and leave the specification to a fuzz harness or an English dialogue. &Truth automates the specification step itself through its Specification Factory. Imandra automates the logic layer by replacing the prover with ImandraX. Raw Lean automates none of it: the engineer rewrites the code, writes the specification, and drives the tactics by hand. That is why its row is the control against which the products are measured.

The choice also determines the failure experience and the trusted surface. Theorem's failures arrive as concrete bug reports or crash inputs from the fuzz harness, and its trusted base is the RISC-V/Linux model plus the translation into Rocq. Harmonic's failures are refutation traces, and its trusted base includes the C/Verilog semantics and the RL-trained lemma agent. &Truth's failures are specification/implementation mismatches, with CSLib and cvc5 underneath. Imandra's are executable counterexamples from ImandraX. The rows are not independent: a product that automates translation inherits the specification bottleneck, one that automates specification still depends on the prover and its library, and one that replaces the prover has to rebuild the ecosystem around it.

// the specification is the product

The strategic fault line is the "who writes the spec" column. Nectry, outside this table, takes the other route: restrict what can be written until the specification is decidable. All five rows here keep general-purpose code, which is why they can address existing repositories but cannot escape the specification problem. The winning product is unlikely to be the one with the strongest prover; it will be the one that makes the specification cheap enough to trust while keeping the translation deterministic. Raw Lean remains the control, not the product.

15 What Is Proved, What Is Open

To answer the title: what can we verify today? Sort the systems this post has covered by two questions, whether the code was written behind a proof or already existed, and whether the property is shallow and universal or full functional correctness, and the evidence clusters in two corners, with a third nearly empty. Figure 5 maps the results.

What we can verify today: deep proofs of new code, shallow proofs of existing code A two-by-two map. The horizontal axis is code provenance, from new code built behind a proof on the left to existing, unmodified code on the right. The vertical axis is depth of property, from shallow universal properties such as crash and memory safety at the bottom to full functional correctness at the top. Green marks done or in production; amber marks partial or in progress. Top left, new code with deep properties: seL4 and CompCert (green), Cedar with a Lean model and differential testing (green), LeetProof and Velvet with 46 of 50 algorithms certified (green), a Python-fragment interpreter in Verus, proved against a formal semantics but with CPython conformance only tested (amber), and the Eita EVM, whose effect layer is proved but whose per-opcode refinement covers only Frontier (amber). Bottom left, new code with shallow properties, is empty: new code built behind a proof rarely aims that low. Bottom right, existing code with shallow properties: Anthropic and Theorem's crash-freedom proofs for 239 entry points across 123 libraries on RISC-V binaries (green), and LNSym on Armv8 machine code (amber). Top right, existing code with deep properties, outlined as mostly open: AlphaProof Code's protocol-security proofs for libsignal on its AArch64 binary, with the cryptographic primitives trusted (green), SymCrypt via Aeneas, in progress (amber), and curve25519-dalek via Aeneas, whose proofs still contain sorry (amber). What we can verify today code provenance × depth of property done partial / in progress depth of property crash / memory safety full functional correctness ← new code, built behind a proof existing, unmodified code → code provenance seL4 · CompCert microkernel; C compiler Cedar Lean model + differential testing LeetProof / Velvet 46/50 algorithms certified Python-fragment interpreter Verus; proved against a semantics, CPython conformance only tested Eita EVM effect layer proved; per-opcode refinement only for Frontier mostly open AlphaProof Code · libsignal protocol security on the AArch64 binary; crypto primitives trusted SymCrypt via Aeneas Rust rewrite, in progress curve25519-dalek via Aeneas proofs still contain sorry rarely the goal: new code built behind a proof aims higher Anthropic + Theorem 239 entry points · 123 libraries RISC-V binaries, crash-free LNSym Armv8 machine code
// Figure 5. Verification results discussed in this post, placed by code provenance and depth of property.

New code built behind a proof gate can reach deep properties: seL4, CompCert, Cedar, the Velvet algorithms, partly Eita, and, from the Anthropic appendix, an interpreter for a Python fragment written in Rust and proved with Verus against a formal semantics, with conformance to CPython tested rather than proved. Existing code can be verified at scale, but so far only for shallow properties such as crash-freedom, which is what the Anthropic and Theorem result shows. Deep properties of code nobody wrote with a proof in mind have one completed result, AlphaProof Code's security proofs for libsignal, which had five specialists and a published formal literature behind its specification. Beyond it the ground is still largely empty, with Aeneas-based efforts on SymCrypt and curve25519-dalek the main work in progress.

The Routes Side by Side

Those five routes map onto that picture. Each suits a different kind of software, and each is at a different stage of maturity. Figure 6 is the scorecard.

Route scorecard: fit, trusted base and maturity of the five routes Five translation routes into a proof assistant, each shown as a band with what it suits, the trusted base beyond the proof kernel, and its maturity. A, direct LLM rewrite: suits new reference models and small functions; an untrusted LLM translates; trusted base is the Lean kernel plus a human audit that the Lean matches production; evidence 17/50 on LeetProof, 1/161 on CLEVER, 27% on vericoding (nine-model union; best single model 18%), and Cedar as a hand-written model; today usable for new models only. B, Rust MIR lifting: suits safe-Rust decision kernels and cryptography; Charon with Aeneas or hax translates; trusted base is the Lean kernel plus the rustc frontend, the Aeneas translator and LLVM; evidence SymCrypt and curve25519-dalek in progress; emerging. C, Velvet DSL: suits algorithmic cores and new imperative code; an LLM or developer writes in the DSL; trusted base is the Lean kernel plus extraction, nearly kernel-only; evidence 28/50 alone and 46/50 with Aristotle; usable today at algorithm scale. D, AST or state lifting: suits legacy C, Verilog and regulated Java; a deterministic lifter translates, though Harmonic has not described its C path; trusted base is the Lean kernel plus the lifter and the target compiler; evidence Forge 0/30 cold and 15/15 with feedback in 2 to 3 rounds; emerging, in constrained profiles. E, binary or ISA lifting: suits existing binaries and assembly; a binary lifter with ISA semantics translates; trusted base is the kernel plus the ISA and syscall models, with the compiler untrusted; evidence LNSym, Theorem at library scale, 239 entry points across 123 libraries, in Rocq rather than Lean, and AlphaProof Code's libsignal security proofs in Lean; usable today for shallow properties, with one deep result. ROUTE · SUITS · TRUSTS MATURITY · EVIDENCE A Direct LLM rewrite new reference models, small functions new models only 17/50 LeetProof 1/161 CLEVER Lean kernel + human audit LLM rewrite untrusted B Rust MIR lifting safe-Rust decision kernels, cryptography emerging SymCrypt, curve25519-dalek in progress Lean kernel + rustc frontend Aeneas translator LLVM C Velvet DSL algorithmic cores, new imperative code today · algorithm scale 46/50 with Aristotle Lean kernel + extraction nearly kernel-only D AST / state lifting legacy C, Verilog, regulated Java emerging · constrained profiles Forge 15/15 with feedback Lean kernel + lifter target compiler E Binary / ISA lifting existing binaries, assembly today · shallow properties 239 entry points, 123 libraries libsignal security (Lean) Rocq/Lean kernel + ISA model syscall model compiler untrusted shared proof kernel extra trusted piece E: Rocq and Lean results
// Figure 6. The five translation routes by what they suit, what they trust beyond the proof kernel, and how mature they are.

Read the maturity labels with the trusted-base chips beside them. Every "today" depends on something outside the proof: a reference specification, an ISA model, a lifter, or a human audit.

What Would Change the Answer

// what would move the open quadrant

Four developments would move the open quadrant.

  • Composition. Agents that strengthen a callee's contract when a caller's proof stalls, instead of thrashing locally, which is exactly where DafnyComp shows today's models collapsing.
  • Specification validation. The mutation and vacuity audits from LeetProof and Forge, scaled up from fifty problems to real modules, as AlphaProof Code has now done once for libsignal.
  • Concurrency and unsafe code. No route here handles them well.
  • Models of the machine. The Anthropic team generated its RISC-V semantics with Claude and tested them heavily, and every binary-level proof is only as good as those semantics. DeepMind's AArch64 semantics is about 7,500 lines of trusted Lean, tested against two independent oracles rather than proved.

16 Conclusions

For most of computing history, source code was the primary artifact, compilers were trusted translators, and tests were a probabilistic check that code roughly matched intent. Formal verification was admired and reserved for flight software and separation kernels, because at half a person-day per line it was economically unthinkable anywhere else.

AI has broken that equilibrium from both sides. Agents flood organizations with code faster than humans can review it, and neural provers paired with a trusted kernel are pulling the cost of a proof from person-months toward compute-hours. But AI does not dissolve the hard problems of formal methods. It relocates them.

Models are excellent abductive engines inside a symbolic loop: they guess invariants, decompose goals, repair from diagnostics, and navigate tactic trees once premise repair and counterexample pruning keep them honest. Asked to translate stateful systems code into Lean in one leap, they fail on most nontrivial benchmarks, invent premises, collapse across function boundaries, trip over dependent types and partial, and will happily prove a vacuous theorem.

Prompting "rewrite my C in Lean and prove it" will not be enough. The teams that win will build deterministic bridges, Aeneas for Rust, Velvet for algorithms, verified lifters for C, Verilog, and binaries, and spend their human attention on the one artifact no machine can validate for them: the specification of what the system is supposed to do. AlphaProof Code is the clearest instance so far: its authors wrote no proof code, and by their own account most of their effort went into the specification.

The commercial landscape now splits along the same line. General proof platforms sell proof search and leave the specification to the customer; vertical products restrict what can be written until the specification is decidable. Both are ways of packaging the bottleneck, and neither removes it.

The proof got cheap. The specification and the bridge from real software to Lean are now the whole game.

References

Benchmarks
Lean and Its Tooling
Lean Verification Frameworks
AI Provers and Proof Search
LLM-in-the-Loop Verifiers and Synthesis
Translating Rust, C, and Binaries
Rust Verification in Practice
Industry and Case Studies
Ethereum and Hardware Verification
Specification and Critique
Historical Background
Not Independently Verified