Same task. Four agents. May the leanest win.
Every agent CLI gets the identical prompt in an identical workspace — sometimes empty, sometimes seeded Terminal-Bench-style with a dataset to analyze or a deliberately broken app to repair — and must ship a single self-contained index.html. The bench records exactly what each one spent, then executes what it built, in jsdom and in real Chrome, so a crash or a blank canvas fails loudly instead of hiding behind green checkmarks. Below, every build is embedded live: click Run live and judge the quality yourself — including the animated pelican on a bicycle.
anthropic claude-opus-4-8 · openai gpt-5.5 · claude 2.1.204 (Claude Code) · codex codex-cli 0.142.5 · zevyn 0.1.0
| # | runner | bench index | build | agentic | speed | efficiency | avg time/task | avg tok/task | cost/task* |
|---|---|---|---|---|---|---|---|---|---|
| 1 | Zevyn CLI · Codex enginegpt-5.5 | 92.7 | 85.6 | 85.3 | 100.0 | 100.0 | 2m 24s | 104.1k | — |
| 2 | Codex CLIgpt-5.5 | 83.7 | 81.4 | 84.5 | 96.6 | 72.2 | 2m 28s | 129.3k | — |
| 3 | Zevyn CLI · Claude engineclaude-opus-4-8 | 73.0 | 91.7 | 99.1 | 48.8 | 52.2 | 3m 53s | 160.6k | $0.69 |
| 4 | Claude Codeclaude-opus-4-8 | 61.9 | 80.6 | 93.3 | 44.1 | 29.5 | 4m 9s | 234.7k | $0.87 |
BENCH INDEX = mean of four equally-weighted 0–100 subscores, all measured — nothing model-judged, nothing estimated, and deliberately hard to max. BUILD & AGENTIC: per task, (checks passed ÷ checks)³ × a relative-richness factor comparing the build's measured runtime substance (canvas draw ops, SVG shapes, interactive controls, rendered text) against the richest build of that task — one missed check costs ~25 points, and a thin build can score at most 50 against a rich one (fix-in-place tasks skip richness). SPEED: (fastest total wall ÷ yours)^1.5. EFFICIENCY: (cheapest ÷ yours)^1.5 on reported cost per task when every ranked runner reports cost (token counts weigh 0.1×-billed cache reads like ~5×-billed output, punishing cache-heavy designs that save real money); falls back to provider-correct tokens when a runner is token-only — as on this leaderboard, where Codex reports no cost. *cost/task only where the CLI itself reported cost on every task.
Totals are provider-correct: Anthropic's token kinds are disjoint (input + output + cache read + cache write), while OpenAI reports cached ⊆ input and reasoning ⊆ output — so its total is input + output. Compare runners within a provider family. Cost is shown only where the CLI itself reports it; nothing is estimated. Full raw data lives in the repo as results.json.
Living Reef
an ecosystem, not a screensaver — schooling fish, light, life
the exact prompt every runner received
Build a living coral-reef aquarium on HTML5 canvas: at least THREE visually distinct fish species (different silhouettes, sizes and colors, drawn with code — bodies, tails, fins, eyes), where at least one species swims as a SCHOOL with flocking behavior (separation + alignment + cohesion) while others cruise independently and turn around believably at the glass. Add swaying kelp, rising wobbling bubbles, animated godrays from the surface, a sandy floor with a crab that scuttles by occasionally, and coral silhouettes. Include a "Feed" button that drops food flakes which nearby fish turn to chase, and a day/night lighting toggle with a smooth transition. requestAnimationFrame, fills the window, resizes correctly. Requirements that override everything else: write a SINGLE file named exactly `index.html` in the current working directory and create NO other files. The file must be fully self-contained — no external images, fonts, stylesheets, scripts, fetches, or network requests of any kind. Do not ask questions and do not explain the code afterwards; when the file is written you are done.
| runner | input | output | cache read | cache write | reasoning | total | wall | cost | checks |
|---|---|---|---|---|---|---|---|---|---|
| Claude Code | 3135 | 23.0k | 32.9k | 28.7k | n/a | 87.7k | 4m 26s | $0.90 | 13/13 |
| Zevyn CLI · Claude engine | 1856 | 19.0k | 31.6k | 22.2k | n/a | 74.7k | 3m 17s | $0.64 | 13/13 |
| Codex CLI | 72.0k | 9327 | 52.7k | n/a | 66 | 81.3k | 2m 55s | — | 13/13 |
| Zevyn CLI · Codex engine | 66.5k | 9268 | 51.7k | n/a | 330 | 75.8k | 2m 54s | — | 13/13 |
Grand Finale
particle physics with showmanship — bursts, reflections, a finale
the exact prompt every runner received
Build a night fireworks show over a bay on HTML5 canvas: rockets launch with glowing trails and explode into at least FOUR distinct burst types (peony sphere, drooping willow, ring, and crackling glitter) using real particle physics — gravity, drag, and fade-out. Below, dark water REFLECTS the bursts (blurred and dimmed); behind, a city skyline silhouette with tiny lit windows and a starfield. The show runs automatically with varied timing and builds to a dense multi-launch FINALE roughly every 20 seconds. Clicking anywhere launches a rocket that bursts at the click point, and an intensity slider (1-10) scales the launch rate. requestAnimationFrame, fills the window, resizes correctly. Requirements that override everything else: write a SINGLE file named exactly `index.html` in the current working directory and create NO other files. The file must be fully self-contained — no external images, fonts, stylesheets, scripts, fetches, or network requests of any kind. Do not ask questions and do not explain the code afterwards; when the file is written you are done.
| runner | input | output | cache read | cache write | reasoning | total | wall | cost | checks |
|---|---|---|---|---|---|---|---|---|---|
| Claude Code | 3135 | 21.3k | 32.9k | 27.0k | n/a | 84.3k | 4m 50s | $0.84 | 13/13 |
| Zevyn CLI · Claude engine | 1810 | 10.2k | 31.6k | 13.7k | n/a | 57.3k | 1m 51s | $0.37 | 13/13 |
| Codex CLI | 46.1k | 6257 | 23.2k | n/a | 37 | 52.4k | 2m 0s | — | 13/13 |
| Zevyn CLI · Codex engine | 62.8k | 7079 | 49.7k | n/a | 57 | 69.9k | 2m 33s | — | 13/13 |
Clockwork Orrery
brass, gears and planets — mechanical motion done right
the exact prompt every runner received
Build an animated clockwork orrery (a mechanical model of the solar system) in a brass-and-mahogany steampunk style, drawn entirely with code (SVG or canvas): at least SIX interlocking gears with visible drawn teeth where MESHING gears counter-rotate at speeds inversely proportional to their tooth counts, articulated brass arms carrying at least four planets (plus one moon) around a golden sun, a swinging pendulum, and an engraved base plate with rivets and ornamental flourishes. Include a speed control (slider or crank, 0.2x-5x) and a pause/resume control. Smooth animation via requestAnimationFrame or SMIL; fills the window; resizes correctly. Requirements that override everything else: write a SINGLE file named exactly `index.html` in the current working directory and create NO other files. The file must be fully self-contained — no external images, fonts, stylesheets, scripts, fetches, or network requests of any kind. Do not ask questions and do not explain the code afterwards; when the file is written you are done.
| runner | input | output | cache read | cache write | reasoning | total | wall | cost | checks |
|---|---|---|---|---|---|---|---|---|---|
| Claude Code | 3145 | 45.9k | 360.7k | 52.1k | n/a | 461.8k | 10m 25s | $1.86 | 12/12 |
| Zevyn CLI · Claude engine | 1992 | 62.1k | 191.8k | 66.0k | n/a | 321.9k | 13m 53s | $2.07 | 12/12 |
| Codex CLI | 66.8k | 6718 | 35.8k | n/a | 185 | 73.5k | 2m 8s | — | 12/12 |
| Zevyn CLI · Codex engine | 62.9k | 7104 | 49.7k | n/a | 99 | 70.0k | 2m 16s | — | 12/12 |
Pocket Synth
a playable instrument — oscillators, envelopes, a step sequencer
the exact prompt every runner received
Build a playable retro hardware-style synthesizer: a clickable on-screen piano keyboard spanning at least TWO octaves (white and black keys, also playable from the computer keyboard with a visible key map), voice controls (waveform selector across sine/square/sawtooth/triangle, attack and release envelope sliders, a filter cutoff slider, master volume), and a 16-step drum sequencer with at least kick, snare and hi-hat rows — all sound synthesized with WebAudio, no audio files. Include play/stop transport with a tempo slider (60-180 BPM), a live oscilloscope or spectrum visualizer on canvas driven by an AnalyserNode, and a lovingly styled hardware-panel look (wood side panels, brushed metal, LED step indicators). Pressed keys and the running step must light up. Requirements that override everything else: write a SINGLE file named exactly `index.html` in the current working directory and create NO other files. The file must be fully self-contained — no external images, fonts, stylesheets, scripts, fetches, or network requests of any kind. Do not ask questions and do not explain the code afterwards; when the file is written you are done.
| runner | input | output | cache read | cache write | reasoning | total | wall | cost | checks |
|---|---|---|---|---|---|---|---|---|---|
| Claude Code | 3135 | 17.6k | 32.9k | 23.4k | n/a | 77.0k | 3m 7s | $0.71 | 12/12 |
| Zevyn CLI · Claude engine | 1810 | 16.7k | 31.6k | 20.2k | n/a | 70.3k | 3m 6s | $0.57 | 12/12 |
| Codex CLI | 71.6k | 9287 | 52.7k | n/a | 107 | 80.9k | 2m 54s | — | 12/12 |
| Zevyn CLI · Codex engine | 44.6k | 8333 | 30.8k | n/a | 25 | 52.9k | 2m 37s | — | 12/12 |
Neon Breakout
an arcade classic with juice — angle physics, particles, glow
the exact prompt every runner received
Build a polished neon-arcade Breakout game on HTML5 canvas: a paddle controlled by BOTH mouse movement and left/right arrow keys, a glowing ball whose bounce angle depends on where it strikes the paddle, and at least SIX rows of bricks with per-row neon colors where the top rows take two hits (with a visible damage state). Add particle debris when bricks shatter, a subtle screen shake on impacts, a trailing glow on the ball, a HUD with score, lives (3) and level, falling powerups (wider paddle and multiball), and win/game-over screens with a restart control. The ball launches from the paddle on click or Space; before launch the scene must already be alive (ball riding the paddle, bricks pulsing). requestAnimationFrame, fills the window, resizes correctly. Requirements that override everything else: write a SINGLE file named exactly `index.html` in the current working directory and create NO other files. The file must be fully self-contained — no external images, fonts, stylesheets, scripts, fetches, or network requests of any kind. Do not ask questions and do not explain the code afterwards; when the file is written you are done.
| runner | input | output | cache read | cache write | reasoning | total | wall | cost | checks |
|---|---|---|---|---|---|---|---|---|---|
| Claude Code | 2975 | 14.7k | 32.9k | 20.3k | n/a | 70.9k | 2m 24s | $0.60 | 14/14 |
| Zevyn CLI · Claude engine | 1856 | 16.3k | 31.6k | 19.8k | n/a | 69.6k | 2m 42s | $0.56 | 14/14 |
| Codex CLI | 47.1k | 7099 | 30.8k | n/a | 0 | 54.2k | 2m 16s | — | 14/14 |
| Zevyn CLI · Codex engine | 43.0k | 6664 | 30.8k | n/a | 0 | 49.7k | 2m 6s | — | 14/14 |
Data Detective
Terminal-Bench-style: the numbers live in a seeded CSV — go read it
the exact prompt every runner received
The current directory contains `sales.csv` — six months of 2025 order data for the fictional gadget maker "Klein & Sons" with columns order_id,month,product,region,units,unit_price_eur. INSPECT THE FILE and build a beautiful self-contained dashboard (index.html) that presents YOUR analysis of it: (1) a KPI row showing total revenue in euros, the best-selling product by revenue, the strongest region by revenue, and the best month by revenue — these numbers MUST be computed from the actual file, and the revenue total shown as an exact integer; (2) a revenue-by-month chart; (3) a revenue-by-product breakdown; (4) one insight you found interesting, stated in one sentence. Embed the aggregated data inline (hand-built SVG or canvas charts, no libraries). Leave `sales.csv` untouched; write only `index.html` and create no other files. The file must be fully self-contained with no network requests. Do not ask questions; when the file is written you are done.
| runner | input | output | cache read | cache write | reasoning | total | wall | cost | checks |
|---|---|---|---|---|---|---|---|---|---|
| Claude Code | 3416 | 25.9k | 587.8k | 45.5k | n/a | 662.6k | 6m 4s | $1.41 | 13/13 |
| Zevyn CLI · Claude engine | 2001 | 13.0k | 283.6k | 22.0k | n/a | 320.6k | 2m 52s | $0.62 | 13/13 |
| Codex CLI | 147.2k | 7870 | 118.3k | n/a | 591 | 155.1k | 2m 39s | — | 13/13 |
| Zevyn CLI · Codex engine | 155.1k | 8730 | 129.4k | n/a | 975 | 163.8k | 2m 56s | — | 13/13 |
Incident Room
Terminal-Bench-style: a credential-stuffing attack hides in a seeded access log
the exact prompt every runner received
The current directory contains `access.log` — one day of Apache-style access logs (30 Jun 2026) for the fictional SaaS "Meridian Outfitters". Something bad happened that night. PARSE THE FILE and build a beautiful self-contained incident-report dashboard (index.html) presenting YOUR forensic findings: (1) a KPI row with the total number of requests in the log, the attacker IP address (the source of the failed logins), the EXACT count of failed (HTTP 401) POST /api/login attempts, and the hour (UTC) the attack happened; (2) a requests-per-hour timeline chart that makes the attack window visually obvious; (3) a short reconstructed timeline of the attacker's behavior including what they did BEFORE the burst; (4) one recommended mitigation, stated in a sentence. All numbers MUST be computed from the actual file. Hand-built SVG or canvas charts only, no libraries. Leave `access.log` untouched; write only `index.html` and create no other files. The file must be fully self-contained with no network requests. Do not ask questions; when the file is written you are done.
| runner | input | output | cache read | cache write | reasoning | total | wall | cost | checks |
|---|---|---|---|---|---|---|---|---|---|
| Claude Code | 3280 | 14.7k | 278.3k | 29.1k | n/a | 325.4k | 3m 14s | $0.82 | 13/13 |
| Zevyn CLI · Claude engine | 2082 | 16.6k | 200.2k | 28.4k | n/a | 247.3k | 3m 52s | $0.70 | 13/13 |
| Codex CLI | 148.9k | 8440 | 111.2k | n/a | 947 | 157.3k | 2m 46s | — | 13/13 |
| Zevyn CLI · Codex engine | 175.9k | 8049 | 139.5k | n/a | 662 | 184.0k | 2m 46s | — | 13/13 |
Lander Rescue
SWE-bench-style: a seeded game with four bugs — fix it, don’t rewrite it
the exact prompt every runner received
The current directory contains `index.html` — a small "Orbital Lander" canvas game that is BROKEN: it never boots and has several distinct bugs. Debug and FIX IT IN PLACE so the game fully works: the scene renders and animates, Start/Reset works, holding SPACE fires the thruster, gravity pulls the lander down, and landing gently on the green pad wins while crashing shows the crash message. Do NOT rewrite the game from scratch and do NOT restructure working code — find the bugs and repair them, keeping the existing functions, IDs and the ZV-RESCUE-BASE marker comment exactly where they are. You may fix small cosmetic defects you notice. Keep it one self-contained file, create no other files, no network requests. Do not ask questions; when the game works you are done.
| runner | input | output | cache read | cache write | reasoning | total | wall | cost | checks |
|---|---|---|---|---|---|---|---|---|---|
| Claude Code | 3143 | 6123 | 137.6k | 13.6k | n/a | 160.4k | 1m 44s | $0.37 | 14/14 |
| Zevyn CLI · Claude engine | 1995 | 11.0k | 161.1k | 17.0k | n/a | 191.1k | 2m 34s | $0.47 | 14/14 |
| Codex CLI | 273.8k | 5404 | 234.4k | n/a | 1314 | 279.2k | 2m 34s | — | 14/14 |
| Zevyn CLI · Codex engine | 202.6k | 7469 | 181.4k | n/a | 1296 | 210.0k | 2m 37s | — | 14/14 |
Broken Ledger
SWE-bench-style: the dashboard boots wrong AND computes wrong — fix the numbers
the exact prompt every runner received
The current directory contains `index.html` — the "Meridian Ledger" FY2025 finance dashboard. It is BROKEN: it renders nothing, and even once it renders, several of its computed figures are WRONG (they disagree with the LEDGER data in the file and with each other). Debug and FIX IT IN PLACE so the page renders and every KPI, chart bar and table row is arithmetically correct and mutually consistent. Do NOT rewrite the page and do NOT restructure working code — find the bugs and repair them, keeping the existing functions, IDs, styling and the ZV-LEDGER-BASE marker comment exactly where they are. Keep it one self-contained file, create no other files, no network requests. Do not ask questions; when the dashboard is correct you are done.
| runner | input | output | cache read | cache write | reasoning | total | wall | cost | checks |
|---|---|---|---|---|---|---|---|---|---|
| Claude Code | 3146 | 4675 | 161.7k | 12.9k | n/a | 182.5k | 67s | $0.34 | 16/16 |
| Zevyn CLI · Claude engine | 1860 | 3828 | 76.8k | 10.1k | n/a | 92.6k | 48s | $0.21 | 16/16 |
| Codex CLI | 225.3k | 4727 | 198.4k | n/a | 407 | 230.1k | 1m 55s | — | 16/16 |
| Zevyn CLI · Codex engine | 58.3k | 2494 | 42.5k | n/a | 282 | 60.8k | 53s | — | 16/16 |
One run is an anecdote. Here are 10.
Every visual task, rebuilt 10 times by each Claude-engine runner from the same prompt. A single bad build says nothing — the spread does. Click any thumbnail to open that exact build live.
Living Reef
an ecosystem, not a screensaver — schooling fish, light, life
Grand Finale
particle physics with showmanship — bursts, reflections, a finale
Clockwork Orrery
brass, gears and planets — mechanical motion done right
How this benchmark works
Identical, isolated runs
Every runner gets the byte-identical prompt and workspace — fresh temp directories, some seeded Terminal-Bench-style (a CSV whose aggregates cannot be guessed; a broken game whose seeded version provably fails the runtime checks). No repo context, no retries, no cherry-picking; models and CLI versions are pinned and recorded.
Nothing is estimated
Tokens are captured exactly as each CLI reports them; cost appears only where a CLI prints it. Provider token semantics differ, so the honest comparison is within a family: Zevyn vs plain Claude Code, Zevyn vs plain Codex.
Builds are executed, not trusted
Each artifact runs headlessly (jsdom with pumped animation frames and a button click) and again in real Chrome for the screenshot. Uncaught errors and blank renders fail checks — a page that compiles but doesn't work is a FAIL, and the report says why.
pnpm build # in zevyn-agent/ node bench/run.mjs # 24 builds, 4 runners x 6 tasks node bench/posters.mjs bench/results/<label> # real-Chrome screenshots + errors node bench/export-site.mjs bench/results/<label> # publish to this page
The harness is a single command, machine-readable, and fails loudly (grep-able BENCH: PASS|FAIL, tri-state exit code) — built to be run by AI agents as much as by people.





























































































