How is the LLM index scored?

The model index is a 0 to 100 composite of agentic tool use, sustained multi file code editing, long context recall, instruction and schema adherence and refusal behaviour under the same harness for every model. It is a separate index from the app builder composite and the two numbers are not comparable.

Are the prices current?

Prices are published list prices per million tokens, read from the vendor pricing page on the fetch date recorded for the cycle. Negotiated, batch and cached input rates are excluded because they are not publicly verifiable.

Why do cheap models score close to expensive ones?

Because the gap on many practical tasks is now smaller than the gap in price. The index does not reward cost, so read the price columns next to the index rather than assuming the ordering reflects value.

Leaderboard / large language models

LLM leaderboard 2026

The models underneath most of the products in the app builder index. Same harness for every model, one composite index on a 0 to 100 scale, list prices per million tokens next to it.

Index leader

Claude Opus 4.5

Composite 89.10

Cheapest output

$0.42

DeepSeek V3.2 per million output tokens

Widest context

1,000,000

Gemini 3 Pro tokens in one request

8 models measured

Sorted by composite index descending. Prices are list, per million tokens, in US dollars.

LLM Tier composite index for large language models
#Model	Vendor	Index	Context	In $/MTok	Out $/MTok	Lab note
1Claude Opus 4.5	Anthropic	89.10	200,000	5.00	25.00	Best sustained multi file editing in the coding harness. Most expensive output tokens in the set.
2GPT-5.1	OpenAI	88.40	400,000	1.25	10.00	Highest agentic tool use score in our harness. Long context recall degrades less than the cohort mean.
3Gemini 3 Pro	Google	86.90	1,000,000	2.00	12.00	Largest usable context in the set and the strongest result on the document grounded subtest.
4Claude Sonnet 4.5	Anthropic	85.60	200,000	3.00	15.00	Close to the frontier on code tasks at a third of the flagship output price.
5Grok 4	xAI	82.30	256,000	3.00	15.00	Fast first token and strong reasoning traces. Weaker on strict JSON schema adherence.
6Qwen3 Max	Alibaba	79.70	262,144	1.20	6.00	Strong multilingual performance and the best non English result in the specification writing subtest.
7DeepSeek V3.2	DeepSeek	78.50	128,000	0.28	0.42	By far the best index points per euro. Tool calling needs stricter prompting than the frontier models.
8Llama 4 Maverick	Meta	74.80	1,000,000	0.22	0.85	Open weights, so it is the only model here you can run on your own hardware without a licence negotiation.

The model index and the app builder index are separate measurements and are not comparable to each other. A product built on the top model does not inherit its score.

Why we run a separate model index

Almost every product in the app builder index is a harness around a frontier model. When a model ships an update, the products built on it change behaviour without shipping anything themselves. That makes the model layer worth measuring on its own, otherwise a product looks like it improved when in fact its supplier did.

The model harness is smaller and more specific than the app builder specification. It tests agentic tool use over a fixed tool set, sustained editing across a repository rather than single file completion, recall at three quarters of the advertised context window, adherence to a strict JSON schema under adversarial input, and refusal behaviour on ambiguous but legitimate requests. It does not test conversation quality, creative writing or multimodal input, because none of those predict how well a product built on the model will complete the vibeOps specification.

Prices sit next to the index rather than inside it. Cost matters enormously in practice and it changes faster than capability, so folding it into the score would make the ordering unstable for reasons that have nothing to do with the models.