Lab notes
News, methodology and analysis
| Date | Category | Title | Summary | Read |
|---|---|---|---|---|
| 19 Aug 2026 | methodology | This cycle's composite recompute, and why it barely moved most rows | We re derived every composite index from the ten published axis weights and the stored subscores, row by row, and checked the arithmetic against the methodology page. Here is what changed and what that process actually verifies. | 2 min |
| 18 Aug 2026 | news | Windsurf is gone from the index | windsurf.com now redirects to devin.ai/desktop. The product is discontinued, so it has been removed from the roster, the raw runs, every leaderboard, every compare pair and the matrix, not just flagged as inactive. | 2 min |
| 15 Aug 2026 | news | August 2026 index update: three products move, one regression | The fourth full run of the vibeOps specification against 11 products. Two products improved their reliability materially, one regressed on tenancy isolation. | 1 min |
| 14 Aug 2026 | methodology | Why we publish the spread and not the mean | A single average build time hides the thing you actually care about: how bad the slow runs get. Here is the p10 to p90 data for every product in the index. | 2 min |
| 12 Aug 2026 | analysis | Server rendering is the whole GEO story right now | We disabled JavaScript on every generated marketing page and counted the bytes of real text. The gap between the top and the bottom of the cohort is nearly seven to one. | 1 min |
| 09 Aug 2026 | methodology | How we score agent performance without reading the chat log | Agent performance is 18 points of the index, the joint largest weight. Here is exactly how a prompt earns 0, 50 or 100. | 1 min |
| 07 Aug 2026 | methodology | Reproducing any row in the index by hand | The composite is a weighted sum, nothing more. Take the ten subscores off any product page, multiply by the published weights and you get the number in the table. | 1 min |
| 03 Aug 2026 | analysis | What this benchmark cannot tell you | A composite index is a compression of a complicated thing into one number. Here are the limits of ours, stated plainly. | 1 min |
methodology
4 published
19 Aug 2026
This cycle's composite recompute, and why it barely moved most rows
We re derived every composite index from the ten published axis weights and the stored subscores, row by row, and checked the arithmetic against the methodology page. Here is what changed and what that process actually verifies.
14 Aug 2026
Why we publish the spread and not the mean
A single average build time hides the thing you actually care about: how bad the slow runs get. Here is the p10 to p90 data for every product in the index.
09 Aug 2026
How we score agent performance without reading the chat log
Agent performance is 18 points of the index, the joint largest weight. Here is exactly how a prompt earns 0, 50 or 100.
07 Aug 2026
Reproducing any row in the index by hand
The composite is a weighted sum, nothing more. Take the ten subscores off any product page, multiply by the published weights and you get the number in the table.
news
2 published
18 Aug 2026
Windsurf is gone from the index
windsurf.com now redirects to devin.ai/desktop. The product is discontinued, so it has been removed from the roster, the raw runs, every leaderboard, every compare pair and the matrix, not just flagged as inactive.
15 Aug 2026
August 2026 index update: three products move, one regression
The fourth full run of the vibeOps specification against 11 products. Two products improved their reliability materially, one regressed on tenancy isolation.
analysis
2 published
12 Aug 2026
Server rendering is the whole GEO story right now
We disabled JavaScript on every generated marketing page and counted the bytes of real text. The gap between the top and the bottom of the cohort is nearly seven to one.
03 Aug 2026
What this benchmark cannot tell you
A composite index is a compression of a complicated thing into one number. Here are the limits of ours, stated plainly.