Downloads / open data / CC BY 4.0
Data downloads
The whole dataset, public and unauthenticated. Take the numbers, check our arithmetic, disagree with our weights. Attribution to LLM Tier with a link is the only condition.
Measured: 15 Aug 2026 09:11 UTCLicence: CC-BY-4.0
The figures below come from a measurement campaign that ran as five daily repetitions of the full specification against every product, from 11 Aug 2026 to 15 Aug 2026, and measured_at carries the end of the final recorded execution. The generated_at field on each export is the file serialisation time, not the measurement time; use measured_at for that.
Product rows
37
20 full-stack app builders, 17 coding agents, two boards
Model rows
26
Sourced reference table, no index of our own. Plus 12 superseded rows
Run rows
1,710
One per prompt execution, published products only
Reader index rows
16
Products past the 3 voter threshold, exported separately from the measured rows
Licensed CC-BY-4.0. Attribution required, commercial use allowed. The ranking exports contain measured figures only: reader votes are published in their own endpoint so nothing consuming the measured data can pick them up by accident.
Endpoints
All served from this domain, no key, no sign up. Measured data and voted data are separate files.
| Endpoint | Format | Contents |
|---|---|---|
| /api/rankings.json | application/json | Four arrays, never one mixed ordering. full_stack_app_builders (20 rows) and coding_agents (17 rows) each carry that board's own composite index, its own subscores, speed percentiles, pass counts, public page measurements and a pricing object with the vendor's own price_display figure plus a separate, internal price_eur_normalised used only for the Value axis; a coding agent row has nine subscores, not ten, because code ownership is not scored there. models_roster (26 rows) and models_legacy (12 rows) carry no index and no rank at all: every field is a vendor-published fact or an attributed third party benchmark result (Arena WebDev, Artificial Analysis, Terminal-Bench v2.1), or explicitly null where no source states one. Includes a licence field, the generation timestamp, a separate measured_at timestamp for when the builder/agent figures were actually taken, both boards' weight tables and index definitions in full, and a top level fx block naming the rate, date and source used for price_eur_normalised. |
| /data/rankings.csv | text/csv | Both boards' rows from the JSON above as one flat CSV with a header line, each row tagged with a board column and grouped by board rather than sorted by one shared index. A coding agent row leaves score_code_ownership blank rather than printing a stale figure. Each row carries price_display (the vendor's own currency, unit and billing period), price_amount, price_currency, price_unit and price_eur_normalised (internal, Value axis only) plus fx_rate_usd_per_eur and fx_rate_date so the conversion is never separated from the figure it applies to. Model rows are not included in this export; use /api/rankings.json for the sourced model reference table. The licence is carried in a column so it survives a copy and paste. |
| /api/runs.json | application/json | The raw per run data: one object per prompt execution with the product slug, run number, prompt index, start time, duration in seconds, outcome and the reviewer note. Filter to one product with ?tool=slug. |
| /data/runs.csv | text/csv | The same raw per run data as a flat CSV. The published turn latency percentiles and pass rate are computed from these rows, so this file lets you recompute those two figures yourself. The remaining subscores, seven on the coding agent board and eight on the full-stack app builder board, come from separate axis review, not from this file. Filter to one product with ?tool=slug. |
| /api/reader-index.json | application/json | The reader index: voted axis means per product, the number of distinct voters, the share of index weight they cover and the signed gap against the lab index. Published separately from the rankings so a consumer of the measured data never picks up voted data by accident. |
| /data/reader-index.csv | text/csv | The reader index as a flat CSV, one row per product with the lab index in the adjacent column for comparison. |
Ranking field reference
Field is the rankings.csv column. JSON path is where the same value lives in the nested /api/rankings.json payload.
| Field (CSV) | JSON path | Type | Meaning |
|---|---|---|---|
| board | full_stack_app_builders[].board / coding_agents[].board | string | Which board this row is measured on: fullstack-builder or coding-agent. Never mixed into one ordering |
| slug | [].slug | string | Stable product identifier, also the URL segment |
| name | [].name | string | Product name as the vendor writes it |
| rank | [].rank | integer | Position in this board's own composite index, 1 is highest. Not comparable across boards |
| rank_delta | [].rank_delta | integer | Rank change against the previous cycle, within the same board. Stored but not rendered anywhere on the site for cycle 04, since the roster was re-based from one merged ranking into two class boards and the two cycles' ranks are not on the same scale; deltas resume in cycle 05 |
| official_url | [].official_url | string | The vendor's own product page |
| page_url | [].page_url | string | This site's detail page for the row |
| index_score | [].index_score | number | This board's composite index, 0 to 100, two decimals |
| not exported | [].index_score_recomputed | number | The same composite recomputed live from that board's own subscores, published next to index_score so a reader can check them against each other |
| score_agent_perf | [].subscores.agent_perf | number | Agent performance subscore, 0 to 100 |
| score_reliability | [].subscores.reliability | number | Reliability subscore, 0 to 100 |
| score_scalability | [].subscores.scalability | number | Scalability subscore, 0 to 100 |
| score_seo_geo | [].subscores.seo_geo | number | SEO and GEO subscore, 0 to 100 |
| score_api_mcp | [].subscores.api_mcp | number | API and MCP subscore, 0 to 100 |
| score_integrations | [].subscores.integrations | number | Integrations subscore, 0 to 100 |
| score_design | [].subscores.design | number | Design output subscore, 0 to 100 |
| score_speed | [].subscores.speed | number | Turn latency subscore, 0 to 100 |
| score_value | [].subscores.value | number | Value subscore, 0 to 100 |
| score_code_ownership | full_stack_app_builders[].subscores.code_ownership | number | Code ownership subscore, 0 to 100. Blank on coding_agents rows: that axis is not scored on this board |
| speed_seconds_p10 | [].turn_latency_seconds.p10 | number | Tenth percentile wall clock seconds per prompt, interpolated over all 45 executions regardless of outcome |
| speed_seconds_median | [].turn_latency_seconds.median | number | Median wall clock seconds per prompt |
| speed_seconds_p90 | [].turn_latency_seconds.p90 | number | Ninetieth percentile wall clock seconds per prompt |
| full_spec_minutes_derived | [].turn_latency_seconds.full_spec_minutes_derived | number | Nine times the per prompt median, in minutes. Derived, not separately measured; see /methodology |
| runs_completed | [].runs.completed | integer | Executions that produced a usable artifact, pass plus partial |
| runs_failed | [].runs.failed | integer | Executions that failed outright |
| reliability_pct | [].runs.reliability_pct | number | Strict pass rate as a percentage of all executions |
| html_bytes_no_js | [].shipped_output.html_bytes_no_js | integer | Bytes of real text on the generated public page with JavaScript disabled |
| lcp_ms | [].shipped_output.lcp_ms | integer | Largest contentful paint in milliseconds, throttled |
| cls | [].shipped_output.cls | number | Cumulative layout shift on load |
| not exported | [].pricing.summary | string | Short, human readable price summary shown on the site |
| price_display | [].pricing.price_display | string | Published price string in the vendor's own currency, with unit and billing period. Never a currency conversion. "Not published by the vendor" where none exists |
| price_amount | [].pricing.price_amount | number | Native numeric price amount in price_currency. Null where no price is published |
| price_currency | [].pricing.price_currency | string | USD or EUR, the vendor's own billing currency. Null where no price is published |
| price_unit | [].pricing.price_unit | string | account, user, seat or team. Null where no price is published |
| price_eur_normalised | [].pricing.price_eur_normalised | number | Internal only: the entry price converted to euros at the fixed rate in the top level fx block, used solely to compute the Value axis. Not a published price and never rendered as one. Null where no vendor price is published at all |
| price_source_url | [].pricing.price_source_url | string | The vendor page this price figure was read from |
| price_fetched_at | [].pricing.price_fetched_at | ISO 8601 date | Date the price figure was read from the vendor page |
| not exported | [].pricing.price_note | string | Per row qualifier on the published price, present only where an exception needs recording, such as a vendor that names no currency or a tier structure that the price string cannot carry |
| last_run_at | [].last_run_at | ISO 8601 | Timestamp of the last completed measurement run |
| next_run_at | [].next_run_at | ISO 8601 | Always null under the current cadence, not a live schedule. See /methodology for how the cadence works |
| not exported | data.measured_at | ISO 8601 | When the published figures were actually measured, separate from generated_at which is only this file's serialisation time |
| fx_rate_usd_per_eur | data.fx.rate_usd_per_eur | number | The fixed rate used to compute price_eur_normalised on every row: 1.1699 US dollars to the euro. Not a live rate |
| fx_rate_date | data.fx.rate_date | ISO 8601 date | Date of the fx rate above: 2026-08-21, European Central Bank euro foreign exchange reference rate |
| not exported | data.full_stack_app_builders_index_definition | string | States that the builders composite is the weighted sum of ten subscores |
| not exported | data.coding_agents_index_definition | string | States that the coding agents composite is the weighted sum of nine subscores, code ownership excluded, and is not comparable to the builders composite |
| not exported | data.models_index_definition | string | States plainly that there is no models index: no harness is run, no run records are held, so no composite is published for models_roster or models_legacy |
| not exported | data.full_stack_app_builders_weights / data.coding_agents_weights | array | Both boards' weight tables in full, the same figures published on /methodology |
| not exported | data.counts | object | Row counts for full_stack_app_builders, coding_agents, models_roster, models_legacy and published_products |
Model reference field reference
Fields on each models_roster / models_legacy row. No index or rank column: see data.models_index_definition.
| Field (JSON) | Type | Meaning |
|---|---|---|
| slug | string | Stable identifier, also the URL anchor on /leaderboard/llms |
| name / vendor | string | Model and vendor name as the vendor writes it |
| status / tier | string | current, preview, legacy or open_weight; flagship, workhorse, budget or coding |
| licence | string | e.g. Proprietary, MIT, Apache 2.0, Modified MIT |
| context_window_text / max_output_text | string | Vendor-published figure, or the exact reason it is not published. Text, not a number, because several vendors simply do not state one |
| price_in_text / price_out_text | string | Vendor list price per million tokens in US dollars, or an explicit not-published note. Never sourced from an aggregator |
| modality_text | string | What the model accepts and returns, per the vendor's own page |
| arena_webdev | object | elo, rank, votes and a preliminary flag, all from arena.ai's WebDev leaderboard, dated 21 Aug 2026. Null fields mean the model is not present on that board |
| bench_aa_index_v4_1_1 | string | Artificial Analysis Intelligence Index v4.1.1, may carry an effort level. Null if not measured |
| bench_terminal_bench_v2_1 | string | Terminal-Bench v2.1 pass@1, may carry an effort level. Null if not measured |
| release_note | string | Release/version date if known, plus any caveat the board prints for that row |
| source_urls | string | Semicolon separated list of the vendor and benchmark pages this row was sourced from |
Shape of the JSON payload
Abridged, but every key name and container type below is real.
{
"ok": true,
"data": {
"publisher": "LLM Tier",
"source": "https://www.llm-tier.com",
"licence": "CC-BY-4.0",
"licence_url": "https://creativecommons.org/licenses/by/4.0/",
"attribution": "LLM Tier, https://www.llm-tier.com",
"generated_at": "2026-08-22T09:14:00.000Z",
"measured_at": "2026-08-15T09:11:37.500Z",
"full_stack_app_builders_index_definition": "Composite index ... weighted sum of ten subscores ...",
"coding_agents_index_definition": "Composite index ... weighted sum of nine subscores, code ownership excluded ...",
"models_index_definition": "There is no models index. This lab runs no language-model harness ...",
"turn_latency_definition": "Median wall clock seconds for one prompt, not for a whole build ...",
"turn_latency_derived_minutes_caveat": "Derived, not separately measured ...",
"fx": { "rate_usd_per_eur": 1.1699, "rate_date": "2026-08-21", "source": "European Central Bank euro foreign exchange reference rates", "note": "Used only for pricing.price_eur_normalised ..." },
"boards_not_comparable_note": "full_stack_app_builders and coding_agents are two separate measurements ...",
"full_stack_app_builders_weights": [
{ "key": "agent_perf", "label": "Agent performance", "weight_percent": 18 },
{ "key": "...", "label": "...", "weight_percent": "..." }
],
"coding_agents_weights": [
{ "key": "agent_perf", "label": "Agent performance", "weight_percent": 18.55 },
{ "key": "...", "label": "...", "weight_percent": "..." }
],
"counts": { "full_stack_app_builders": 20, "coding_agents": 17, "models_roster": 26, "models_legacy": 12, "published_products": 37 },
"full_stack_app_builders": [
{
"rank": 1, "rank_delta": 0, "slug": "...", "name": "...",
"board": "fullstack-builder", "official_url": "...", "page_url": "...",
"index_score": 00.00, "index_score_recomputed": 00.00,
"subscores": { "agent_perf": 00, "code_ownership": 00, "...": "..." },
"turn_latency_seconds": { "p10": 00, "median": 00, "p90": 00, "full_spec_minutes_derived": 0.0 },
"runs": { "completed": 00, "failed": 0, "reliability_pct": 00 },
"shipped_output": { "html_bytes_no_js": 0000, "lcp_ms": 0000, "cls": 0.00 },
"pricing": { "summary": "...", "price_display": "USD 25 / month", "price_amount": 25, "price_currency": "USD", "price_unit": "account", "price_source_url": "...", "price_fetched_at": "2026-08-21", "price_note": null, "price_eur_normalised": 21.37 },
"last_run_at": "2026-08-15T09:11:37.500Z", "next_run_at": null
}
],
"coding_agents": [
{
"rank": 1, "rank_delta": 0, "slug": "...", "name": "...",
"board": "coding-agent", "official_url": "...", "page_url": "...",
"index_score": 00.00, "index_score_recomputed": 00.00,
"subscores": { "agent_perf": 00, "...": "..." },
"turn_latency_seconds": { "p10": 00, "median": 00, "p90": 00, "full_spec_minutes_derived": 0.0 },
"runs": { "completed": 00, "failed": 0, "reliability_pct": 00 },
"shipped_output": { "html_bytes_no_js": 0000, "lcp_ms": 0000, "cls": 0.00 },
"pricing": { "summary": "...", "price_display": "USD 20 / seat / month", "price_amount": 20, "price_currency": "USD", "price_unit": "seat", "price_source_url": "...", "price_fetched_at": "2026-08-21", "price_note": null, "price_eur_normalised": 17.1 },
"last_run_at": "2026-08-15T09:11:37.500Z", "next_run_at": null
}
],
"models_roster": [
{
"slug": "...", "name": "...", "vendor": "...", "status": "current", "tier": "flagship",
"licence": "...", "context_window_text": "...", "max_output_text": "...",
"price_in_text": "...", "price_out_text": "...", "modality_text": "...",
"arena_webdev": { "elo": 1691, "rank": 1, "votes": 8116, "preliminary": false },
"bench_aa_index_v4_1_1": "...", "bench_terminal_bench_v2_1": "...",
"release_note": "...", "source_urls": "...", "page_url": "..."
}
],
"models_legacy": [
{ "slug": "...", "name": "...", "vendor": "...", "status": "legacy", "...": "same shape as models_roster" }
]
}
}note: subscores omits code_ownership entirely on coding_agents rows, rather than printing a stale or zero value. No key in this payload ever orders full_stack_app_builders against coding_agents, and no key under models_roster or models_legacy is an index or a rank of any kind.
Citing the lab
For a figure in prose, name the source and the cycle: LLM Tier, August 2026 cycle. For a table or a chart, add a link to the page the numbers came from. For a redistributed dataset, keep the licence and attribution fields in the payload intact.
If you are building something on this data and want a heads up before the shape changes, use the contact form. Breaking changes are announced in the lab notes first, and additive fields are not treated as breaking.