# LLM Tier

> Independent measurement lab for AI app builders, coding agents and LLMs. Every product is driven through the same published build spec, vibeOps, 5 times with 9 fixed prompts per run. Builders and coding agents are two separate boards with two separate weight tables and two separate rank columns: an index score, subscore or rank on one board is never comparable to the other. We publish the raw per run outcomes, both boards' weighted subscores and a composite index from 0 to 100 with two decimals for each.

Licence: CC-BY-4.0 (https://creativecommons.org/licenses/by/4.0/). Cite as: LLM Tier, https://www.llm-tier.com.
Contact: https://www.llm-tier.com/contact
Scoring: composite index 0 to 100, two decimals, per board. There are no star ratings anywhere on this site.

## Core rankings

- [App builder leaderboard](https://www.llm-tier.com/leaderboard/app-builders): full index with turn latency spread, reliability and per axis subscores for every measured full-stack app builder.
- [Coding agent leaderboard](https://www.llm-tier.com/leaderboard/coding-agents): full index with turn latency spread, reliability and per axis subscores for every measured coding agent. Nine axes, no code ownership column.
- [LLM reference table](https://www.llm-tier.com/leaderboard/llms): sourced specifications, vendor list prices, and attributed third party benchmarks (Arena WebDev Elo, Artificial Analysis Intelligence Index, Terminal-Bench v2.1) for the models most of these products are built on. No composite index or rank of our own: this lab runs no model harness and holds no model-level run records.
- [Full comparison matrix](https://www.llm-tier.com/matrix): two grids, one per board, each product against that board's own axes, winner marked per column.
- [Homepage index tables](https://www.llm-tier.com): both boards, rank, index, turn latency, reliability, SEO score and rank delta.
- [Methodology](https://www.llm-tier.com/methodology): both axis weight tables, the nine prompts, the pass criteria and the turn latency and percentile methodology.
- [Open data](https://www.llm-tier.com/data): endpoints, schema and citation guidance.

## Machine readable data

- [Rankings JSON](https://www.llm-tier.com/api/rankings.json): full_stack_app_builders and coding_agents as two composite-scored arrays, plus models_roster and models_legacy as two sourced reference arrays with no index or rank field at all.
- [Raw runs JSON](https://www.llm-tier.com/api/runs.json): one row per prompt attempt for published products, filter with ?tool=slug.
- [Rankings CSV](https://www.llm-tier.com/data/rankings.csv): flat export with a board column and a licence column on every row.

## Measured full-stack app builders

- [Totalum](https://www.llm-tier.com/builders/totalum): index 89.48, rank 1 of 20, reliability 93.3 percent, median turn latency 106.3 seconds. Reviews: https://www.llm-tier.com/builders/totalum/reviews
- [Firebase Studio](https://www.llm-tier.com/builders/firebase-studio): index 80.92, rank 2 of 20, reliability 86.7 percent, median turn latency 100.3 seconds. Reviews: https://www.llm-tier.com/builders/firebase-studio/reviews
- [Lovable](https://www.llm-tier.com/builders/lovable): index 80.54, rank 3 of 20, reliability 82.2 percent, median turn latency 85.2 seconds. Reviews: https://www.llm-tier.com/builders/lovable/reviews
- [v0](https://www.llm-tier.com/builders/v0): index 78.31, rank 4 of 20, reliability 84.4 percent, median turn latency 65.2 seconds. Reviews: https://www.llm-tier.com/builders/v0/reviews
- [Replit](https://www.llm-tier.com/builders/replit): index 75.47, rank 5 of 20, reliability 86.7 percent, median turn latency 135.7 seconds. Reviews: https://www.llm-tier.com/builders/replit/reviews
- [Bolt.new](https://www.llm-tier.com/builders/bolt-new): index 74.68, rank 6 of 20, reliability 80 percent, median turn latency 82.1 seconds. Reviews: https://www.llm-tier.com/builders/bolt-new/reviews
- [Base44](https://www.llm-tier.com/builders/base44): index 73.05, rank 7 of 20, reliability 84.4 percent, median turn latency 94.1 seconds. Reviews: https://www.llm-tier.com/builders/base44/reviews
- [Anything](https://www.llm-tier.com/builders/anything): index 69.97, rank 8 of 20, reliability 77.8 percent, median turn latency 94.1 seconds. Reviews: https://www.llm-tier.com/builders/anything/reviews
- [Softgen](https://www.llm-tier.com/builders/softgen): index 69.48, rank 9 of 20, reliability 73.3 percent, median turn latency 108.4 seconds. Reviews: https://www.llm-tier.com/builders/softgen/reviews
- [Emergent](https://www.llm-tier.com/builders/emergent): index 68.68, rank 10 of 20, reliability 80 percent, median turn latency 144.6 seconds. Reviews: https://www.llm-tier.com/builders/emergent/reviews
- [Floot](https://www.llm-tier.com/builders/floot): index 68.66, rank 11 of 20, reliability 68.9 percent, median turn latency 91.1 seconds. Reviews: https://www.llm-tier.com/builders/floot/reviews
- [Dyad](https://www.llm-tier.com/builders/dyad): index 67.49, rank 12 of 20, reliability 55.6 percent, median turn latency 83.8 seconds. Reviews: https://www.llm-tier.com/builders/dyad/reviews
- [Bind AI](https://www.llm-tier.com/builders/bind-ai): index 65.59, rank 13 of 20, reliability 64.4 percent, median turn latency 96 seconds. Reviews: https://www.llm-tier.com/builders/bind-ai/reviews
- [Rocket.new](https://www.llm-tier.com/builders/rocket-new): index 62.12, rank 14 of 20, reliability 60 percent, median turn latency 107.2 seconds. Reviews: https://www.llm-tier.com/builders/rocket-new/reviews
- [Caffeine](https://www.llm-tier.com/builders/caffeine): index 62.08, rank 15 of 20, reliability 66.7 percent, median turn latency 150.6 seconds. Reviews: https://www.llm-tier.com/builders/caffeine/reviews
- [Zite](https://www.llm-tier.com/builders/zite): index 61.06, rank 16 of 20, reliability 60 percent, median turn latency 113.4 seconds. Reviews: https://www.llm-tier.com/builders/zite/reviews
- [bolt.diy](https://www.llm-tier.com/builders/bolt-diy): index 59.85, rank 17 of 20, reliability 55.6 percent, median turn latency 113.8 seconds. Reviews: https://www.llm-tier.com/builders/bolt-diy/reviews
- [YouWare](https://www.llm-tier.com/builders/youware): index 58.53, rank 18 of 20, reliability 48.9 percent, median turn latency 80 seconds. Reviews: https://www.llm-tier.com/builders/youware/reviews
- [Trickle](https://www.llm-tier.com/builders/trickle): index 58.11, rank 19 of 20, reliability 51.1 percent, median turn latency 87 seconds. Reviews: https://www.llm-tier.com/builders/trickle/reviews
- [CatDoes](https://www.llm-tier.com/builders/catdoes): index 56.99, rank 20 of 20, reliability 48.9 percent, median turn latency 137.1 seconds. Reviews: https://www.llm-tier.com/builders/catdoes/reviews

## Measured coding agents

- [Claude Code](https://www.llm-tier.com/builders/claude-code): index 79.72, rank 1 of 17, reliability 91.1 percent, median turn latency 65.5 seconds. Reviews: https://www.llm-tier.com/builders/claude-code/reviews
- [Cursor](https://www.llm-tier.com/builders/cursor): index 77.92, rank 2 of 17, reliability 86.7 percent, median turn latency 53.4 seconds. Reviews: https://www.llm-tier.com/builders/cursor/reviews
- [Kiro](https://www.llm-tier.com/builders/kiro): index 75.67, rank 3 of 17, reliability 93.3 percent, median turn latency 72.4 seconds. Reviews: https://www.llm-tier.com/builders/kiro/reviews
- [Augment Code](https://www.llm-tier.com/builders/augment-code): index 75.63, rank 4 of 17, reliability 88.9 percent, median turn latency 71.6 seconds. Reviews: https://www.llm-tier.com/builders/augment-code/reviews
- [Amp](https://www.llm-tier.com/builders/amp): index 73.35, rank 5 of 17, reliability 77.8 percent, median turn latency 39.7 seconds. Reviews: https://www.llm-tier.com/builders/amp/reviews
- [Factory](https://www.llm-tier.com/builders/factory): index 71.48, rank 6 of 17, reliability 82.2 percent, median turn latency 161.7 seconds. Reviews: https://www.llm-tier.com/builders/factory/reviews
- [Antigravity](https://www.llm-tier.com/builders/antigravity): index 71.37, rank 7 of 17, reliability 73.3 percent, median turn latency 78.7 seconds. Reviews: https://www.llm-tier.com/builders/antigravity/reviews
- [Zed](https://www.llm-tier.com/builders/zed): index 71.13, rank 8 of 17, reliability 71.1 percent, median turn latency 31.4 seconds. Reviews: https://www.llm-tier.com/builders/zed/reviews
- [Cline](https://www.llm-tier.com/builders/cline): index 70.66, rank 9 of 17, reliability 80 percent, median turn latency 95.1 seconds. Reviews: https://www.llm-tier.com/builders/cline/reviews
- [Jules](https://www.llm-tier.com/builders/jules): index 68.04, rank 10 of 17, reliability 75.6 percent, median turn latency 146.5 seconds. Reviews: https://www.llm-tier.com/builders/jules/reviews
- [Devin](https://www.llm-tier.com/builders/devin): index 66.64, rank 11 of 17, reliability 71.1 percent, median turn latency 353.6 seconds. Reviews: https://www.llm-tier.com/builders/devin/reviews
- [Qodo](https://www.llm-tier.com/builders/qodo): index 66.06, rank 12 of 17, reliability 77.8 percent, median turn latency 99.3 seconds. Reviews: https://www.llm-tier.com/builders/qodo/reviews
- [Aider](https://www.llm-tier.com/builders/aider): index 65.88, rank 13 of 17, reliability 68.9 percent, median turn latency 39.4 seconds. Reviews: https://www.llm-tier.com/builders/aider/reviews
- [Codebuff](https://www.llm-tier.com/builders/codebuff): index 64.51, rank 14 of 17, reliability 68.9 percent, median turn latency 59.8 seconds. Reviews: https://www.llm-tier.com/builders/codebuff/reviews
- [OpenHands](https://www.llm-tier.com/builders/openhands): index 63.93, rank 15 of 17, reliability 64.4 percent, median turn latency 144.8 seconds. Reviews: https://www.llm-tier.com/builders/openhands/reviews
- [Trae](https://www.llm-tier.com/builders/trae): index 61.00, rank 16 of 17, reliability 51.1 percent, median turn latency 96.7 seconds. Reviews: https://www.llm-tier.com/builders/trae/reviews
- [Roomote](https://www.llm-tier.com/builders/roomote): index 59.89, rank 17 of 17, reliability 53.3 percent, median turn latency 96.4 seconds. Reviews: https://www.llm-tier.com/builders/roomote/reviews

## Head to head, full-stack app builders

These pairings are curated rather than every possible combination of the board. Both products in a pairing sit on the full-stack app builder board, so their index scores are directly comparable.

- [Lovable versus Bolt.new](https://www.llm-tier.com/compare/lovable-vs-bolt-new): Lovable index 80.54, rank 3 of 20 on the Full-stack app builders board; Bolt.new index 74.68, rank 6 of 20 on the Full-stack app builders board.
- [Lovable versus v0](https://www.llm-tier.com/compare/lovable-vs-v0): Lovable index 80.54, rank 3 of 20 on the Full-stack app builders board; v0 index 78.31, rank 4 of 20 on the Full-stack app builders board.
- [Bolt.new versus v0](https://www.llm-tier.com/compare/bolt-new-vs-v0): Bolt.new index 74.68, rank 6 of 20 on the Full-stack app builders board; v0 index 78.31, rank 4 of 20 on the Full-stack app builders board.
- [Replit versus Bolt.new](https://www.llm-tier.com/compare/replit-vs-bolt-new): Replit index 75.47, rank 5 of 20 on the Full-stack app builders board; Bolt.new index 74.68, rank 6 of 20 on the Full-stack app builders board.
- [Lovable versus Replit](https://www.llm-tier.com/compare/lovable-vs-replit): Lovable index 80.54, rank 3 of 20 on the Full-stack app builders board; Replit index 75.47, rank 5 of 20 on the Full-stack app builders board.
- [Base44 versus Lovable](https://www.llm-tier.com/compare/base44-vs-lovable): Base44 index 73.05, rank 7 of 20 on the Full-stack app builders board; Lovable index 80.54, rank 3 of 20 on the Full-stack app builders board.
- [Totalum versus Lovable](https://www.llm-tier.com/compare/totalum-vs-lovable): Totalum index 89.48, rank 1 of 20 on the Full-stack app builders board; Lovable index 80.54, rank 3 of 20 on the Full-stack app builders board.
- [Totalum versus Base44](https://www.llm-tier.com/compare/totalum-vs-base44): Totalum index 89.48, rank 1 of 20 on the Full-stack app builders board; Base44 index 73.05, rank 7 of 20 on the Full-stack app builders board.
- [Emergent versus Replit](https://www.llm-tier.com/compare/emergent-vs-replit): Emergent index 68.68, rank 10 of 20 on the Full-stack app builders board; Replit index 75.47, rank 5 of 20 on the Full-stack app builders board.
- [Zite versus Rocket.new](https://www.llm-tier.com/compare/zite-vs-rocket-new): Zite index 61.06, rank 16 of 20 on the Full-stack app builders board; Rocket.new index 62.12, rank 14 of 20 on the Full-stack app builders board.
- [Firebase Studio versus Lovable](https://www.llm-tier.com/compare/firebase-studio-vs-lovable): Firebase Studio index 80.92, rank 2 of 20 on the Full-stack app builders board; Lovable index 80.54, rank 3 of 20 on the Full-stack app builders board.
- [Dyad versus bolt.diy](https://www.llm-tier.com/compare/dyad-vs-bolt-diy): Dyad index 67.49, rank 12 of 20 on the Full-stack app builders board; bolt.diy index 59.85, rank 17 of 20 on the Full-stack app builders board.
- [Floot versus Lovable](https://www.llm-tier.com/compare/floot-vs-lovable): Floot index 68.66, rank 11 of 20 on the Full-stack app builders board; Lovable index 80.54, rank 3 of 20 on the Full-stack app builders board.
- [YouWare versus v0](https://www.llm-tier.com/compare/youware-vs-v0): YouWare index 58.53, rank 18 of 20 on the Full-stack app builders board; v0 index 78.31, rank 4 of 20 on the Full-stack app builders board.

## Head to head, coding agents

These pairings are curated rather than every possible combination of the board. Both products in a pairing sit on the coding agent board, so their index scores are directly comparable.

- [Cursor versus Claude Code](https://www.llm-tier.com/compare/cursor-vs-claude-code): Cursor index 77.92, rank 2 of 17 on the Coding agents board; Claude Code index 79.72, rank 1 of 17 on the Coding agents board.
- [Cursor versus Zed](https://www.llm-tier.com/compare/cursor-vs-zed): Cursor index 77.92, rank 2 of 17 on the Coding agents board; Zed index 71.13, rank 8 of 17 on the Coding agents board.
- [Claude Code versus Amp](https://www.llm-tier.com/compare/claude-code-vs-amp): Claude Code index 79.72, rank 1 of 17 on the Coding agents board; Amp index 73.35, rank 5 of 17 on the Coding agents board.
- [Aider versus Cline](https://www.llm-tier.com/compare/aider-vs-cline): Aider index 65.88, rank 13 of 17 on the Coding agents board; Cline index 70.66, rank 9 of 17 on the Coding agents board.
- [Devin versus Jules](https://www.llm-tier.com/compare/devin-vs-jules): Devin index 66.64, rank 11 of 17 on the Coding agents board; Jules index 68.04, rank 10 of 17 on the Coding agents board.
- [Devin versus Factory](https://www.llm-tier.com/compare/devin-vs-factory): Devin index 66.64, rank 11 of 17 on the Coding agents board; Factory index 71.48, rank 6 of 17 on the Coding agents board.
- [Cursor versus Kiro](https://www.llm-tier.com/compare/cursor-vs-kiro): Cursor index 77.92, rank 2 of 17 on the Coding agents board; Kiro index 75.67, rank 3 of 17 on the Coding agents board.
- [Augment Code versus Cursor](https://www.llm-tier.com/compare/augment-code-vs-cursor): Augment Code index 75.63, rank 4 of 17 on the Coding agents board; Cursor index 77.92, rank 2 of 17 on the Coding agents board.
- [Antigravity versus Jules](https://www.llm-tier.com/compare/antigravity-vs-jules): Antigravity index 71.37, rank 7 of 17 on the Coding agents board; Jules index 68.04, rank 10 of 17 on the Coding agents board.
- [Trae versus Cursor](https://www.llm-tier.com/compare/trae-vs-cursor): Trae index 61.00, rank 16 of 17 on the Coding agents board; Cursor index 77.92, rank 2 of 17 on the Coding agents board.

## Axis leaderboards

- [Turn latency](https://www.llm-tier.com/leaderboard/speed): Median wall clock seconds for one prompt, not for a whole build. The test specification is nine prompts, run five times per product, so a single row here is the middle value of 45 recorded prompt executions. Shown as two boards, Full-stack app builders and Coding agents, never mixed into one table.
- [Reliability](https://www.llm-tier.com/leaderboard/reliability): Strict pass rate across 1,665 recorded prompt executions. Shown as two boards, Full-stack app builders and Coding agents, never mixed into one table.
- [SEO and GEO](https://www.llm-tier.com/leaderboard/seo-geo): Server rendered HTML bytes with JavaScript disabled, LCP, CLS and structured data output. Shown as two boards, Full-stack app builders and Coding agents, never mixed into one table.
- [API and MCP](https://www.llm-tier.com/leaderboard/api-mcp): REST surface, webhook delivery, background jobs and Model Context Protocol support. Shown as two boards, Full-stack app builders and Coding agents, never mixed into one table.
- [Design output](https://www.llm-tier.com/leaderboard/design): Blind scored visual quality of the generated admin panel and marketing surface. Shown as two boards, Full-stack app builders and Coding agents, never mixed into one table.
- [Value](https://www.llm-tier.com/leaderboard/value): Index points delivered per euro of entry price. The two boards price different things, so the two value scores are not comparable across boards. Shown as two boards, Full-stack app builders and Coding agents, never mixed into one table.

## Lab notes

- [957 reader rows carried a date from before this domain existed](https://www.llm-tier.com/news/nine-hundred-and-fifty-seven-reader-rows-were-dated-before-this-domain-existed): In August we corrected eight article dates and published the check that found them. The check was scoped to the articles table. Every reader score, open rating, lab note, comment and vote on this site was also dated before the domain resolved. All 957 are corrected, and the gate is restated wider.
- [Eight articles carried a publication date from before this domain was registered](https://www.llm-tier.com/news/eight-articles-were-dated-before-this-domain-existed): llm-tier.com was registered on August 19, 2026 at 10:10 UTC. Eight of our fourteen articles were stamped earlier than that, in the byline and in the structured data. Corrected today, with the check that finds it now running on a schedule.
- [Entry prices are now published in the vendor's own currency, not a euro conversion that tracked no single rate](https://www.llm-tier.com/news/entry-prices-republished-in-vendors-own-currency): Every entry price on both leaderboards carried a bare euro figure with no stated source or rate. An audit found the implied euro per dollar rate across cleanly priced USD rows spread from 0.19 to 1.80, wrong in both directions. Prices are now published in the vendor's own currency, unit and billing period, sourced. The Value axis and the composite index were recomputed board wide as a result, and Totalum's own Value subscore fell from 84 to 69.
- [The model board carried a composite index with no run behind it. It is gone.](https://www.llm-tier.com/news/the-model-board-had-a-fabricated-index-we-removed-it): The eight rows on /leaderboard/llms carried an Index column that looked exactly like the composite this lab actually measures on the other two boards. It was not: no per-model run record, subscore or recomputed field sat behind any of the eight numbers, and Arena's own leaderboard shows all eight rows were two to three generations stale. The index is removed, not refreshed. What replaces it is a 26-row sourced specification-and-benchmark table, with the twelve superseded rows kept in a separate table rather than deleted.
- [Tesslate leaves the index: the vendor has pivoted to a different category](https://www.llm-tier.com/news/tesslate-has-pivoted-to-opensail-and-leaves-the-index): tesslate.com no longer operates an app builder. The domain now hosts OpenSail, a business operations platform, and the EUR 16 monthly price we published cannot be sourced from the vendor's current site. Tesslate is out of the roster, the two leaderboards it appeared on, the matrix and every data export.
- [Convex Chef is being replaced by its vendor, so it leaves the index](https://www.llm-tier.com/news/convex-chef-is-being-replaced-by-its-vendor): Convex is replacing Chef with a new product and has taken down its entry points. Existing projects still run, so this is not a shutdown, but a builder you cannot sign up for cannot hold a live ranking. Chef is out of the roster, the raw runs, the matrix, both leaderboards it appeared on and both of its compare pages.
- [Pazi leaves the index: wrong category, and a price we could not source](https://www.llm-tier.com/news/pazi-was-never-in-this-category): We ranked Pazi 37 of 41 as a coding agent and scored it on 45 executions of nine software engineering prompts. Pazi does not sell a coding product. We also published a 14 EUR floor against a real 25 dollar one, and cited a source URL that never contained the figure.
- [Mocha leaves the index, and we benchmarked it after it had already announced its own end.](https://www.llm-tier.com/news/mocha-was-benchmarked-after-it-announced-its-end): Mocha is gone from this roster, and the reason is as much about our own record as the vendor's calendar. We published a euro price and a value score for a plan its own pricing page says can no longer be bought, and every one of the 45 runs behind its row was collected after the vendor's stated shutdown date had already passed.
- [This cycle's composite recompute, and why it barely moved most rows](https://www.llm-tier.com/news/this-cycles-composite-recompute): We re derived every composite index from the ten published axis weights and the stored subscores, row by row, and checked the arithmetic against the methodology page. Here is what changed and what that process actually verifies.
- [Windsurf is gone from the index](https://www.llm-tier.com/news/windsurf-is-gone-from-the-index): windsurf.com now redirects to devin.ai/desktop. The product is discontinued, so it has been removed from the roster, the raw runs, every leaderboard, every compare pair and the matrix, not just flagged as inactive.
- [August 2026 index update: three products move, one regression](https://www.llm-tier.com/news/august-2026-index-update): The fourth full run of the vibeOps specification against 11 products. Two products improved their reliability materially, one regressed on tenancy isolation.
- [Why we publish the spread and not the mean](https://www.llm-tier.com/news/why-we-publish-the-spread-not-the-mean): A single average build time hides the thing you actually care about: how bad the slow runs get. Here is the p10 to p90 data for every product in the index.
- [Server rendering is the whole GEO story right now](https://www.llm-tier.com/news/server-rendering-is-the-geo-story): We disabled JavaScript on every generated marketing page and counted the bytes of real text. The gap between the top and the bottom of the cohort is nearly seven to one.
- [How we score agent performance without reading the chat log](https://www.llm-tier.com/news/how-we-score-agent-performance): Agent performance is 18 points of the index, the joint largest weight. Here is exactly how a prompt earns 0, 50 or 100.
- [Reproducing any row in the index by hand](https://www.llm-tier.com/news/reproducing-the-index-by-hand): The composite is a weighted sum, nothing more. Take the ten subscores off any product page, multiply by the published weights and you get the number in the table.
- [What this benchmark cannot tell you](https://www.llm-tier.com/news/what-a-benchmark-cannot-tell-you): A composite index is a compression of a complicated thing into one number. Here are the limits of ours, stated plainly.

## Optional

- [About the lab](https://www.llm-tier.com/about): who measures this and how corrections work.
- [Full text export](https://www.llm-tier.com/llms-full.txt): every published figure in one plain text file.
- [Terms](https://www.llm-tier.com/terms) and [Privacy](https://www.llm-tier.com/privacy).