In brief
- EVMbench tests AI agents on 120 real-world Ethereum smart contract vulnerabilities.
- Tool evaluates detection, patching, and exploitation across three distinct modes.
- GPT-5.3-Codex achieved 72.2% success rate in exploit mode testing.
ChatGPT maker OpenAI and crypto-focused investment firm Paradigm have introduced EVMbench, a tool to help improve Ethereum Virtual Machine smart contract security.
EVMbench is designed to evaluate AI agents’ ability to detect, patch, and exploit high-severity vulnerabilities in Ethereum Virtual Machine (EVM) smart contracts.
Smart contracts are the heart of the Ethereum network, holding the code that powers everything from decentralized finance protocols to token launches. The weekly number of smart contracts deployed on Ethereum reached an all-time high of 1.7 million in November 2025, with 669,500 deployed last week alone, according to Token Terminal.
EVMbench draws on 120 curated vulnerabilities from 40 audits, most sourced from open audit competitions such as Code4rena, according to an OpenAI blog post. It also includes scenarios from the security auditing process for Tempo, Stripe’s purpose-built layer-1 blockchain focused on high-throughput, low-cost stablecoin payments.
Payments giant Stripe launched the public testnet for Tempo in December, saying at the time that it was being built with input from Visa, Shopify, and OpenAI, among others.
The goal is to ground testing in economically meaningful, real-world code—particularly as AI-driven stablecoin payments expand, the firm added.
EVMbench is meant to evaluate AI models across three modes: Detect, patch, and exploit. In “detect,” agents audit repositories and are scored on their recall of ground-truth vulnerabilities. In “patch,” agents must eliminate vulnerabilities without breaking intended functionality. Finally, in the “exploit” phase, agents attempt end-to-end fund-draining attacks in a sandboxed blockchain environment, with grading performed via deterministic transaction replay.
In exploit mode, GPT-5.3-Codex running via OpenAI’s Codex CLI achieved a score of 72.2%, compared to 31.9% for GPT-5, which was released six months earlier. Performance was weaker in the detect and patch tasks, where agents sometimes failed to audit exhaustively or struggled to preserve full contract functionality.
The ChatGPT makers’ researchers cautioned that EVMbench does not fully capture real-world security complexity. Still, they added that measuring AI performance in economically relevant environments is critical as models become powerful tools for both attackers and defenders.
Sam Altman’s OpenAI and Ethereum co-founder Vitalik Buterin have previously been at odds over the pace of AI development.
In January 2025, Altman said that his firm was “confident we know how to build AGI as we have traditionally understood it.” But Buterin advocated that AI systems should include a “soft pause” capability that could temporarily restrict industrial-scale AI operations if warning signs emerge.
Daily Debrief Newsletter
Start every day with the top news stories right now, plus original features, a podcast, videos and more.
