news / 15 Aug 2026 / 1 min read

August 2026 index update: three products move, one regression

The fourth full run of the vibeOps specification against 11 products. Two products improved their reliability materially, one regressed on tenancy isolation.

By Daniel Osei | Updated 19 Aug 2026

What changed this cycle

We re ran the full vibeOps specification, 5 runs of 9 prompts against each of the 11 measured products, between 11 and 14 August 2026. That is 495 recorded prompt executions.

Three products moved by more than one rank position. The largest single move came from a product that shipped a durable job queue between cycles, which is worth real points because prompt 6 tests idempotency and restart survival directly.

The regression worth reading about

One product now fails the prompt 3 cross tenant read test in 2 of 5 runs. In those runs a member of organisation A could read a project belonging to organisation B through the generated list endpoint. This is not a subjective scoring matter. It is a defect, it is reproducible, and it is recorded in the raw runs data with the note preserved.

We contacted the vendor before publication and gave the standard 5 working days to respond. The result stands as measured.

Method notes for this cycle

No weights changed this cycle. The weight table has been stable since the May 2026 revision and any change is announced before the run, never after the numbers are in.

Two products changed their pricing between the previous cycle and this one, so the value axis moved for both. Prices are read from the public pricing page on the first day of the run window and the fetch date is recorded on each product page.

More lab notes