Opinion page / measured product / rank 25
Devin Reviews and Ratings (2026)
Reader ratings for Devin, aggregated on the same 0 to 100 scale we use for the lab index, with our own commentary underneath. If you are looking for Devin opiniones, reviews or a rating you can check, the measurements behind every claim are on the Devin benchmark page.
Reader index
not rated yet
2 voters, 18 axis scores. Never blended into the lab index.
Lab index
66.36
Measured on the vibeOps spec
Strict pass rate
71.1%
32 clean and 6 failed executions
Median per prompt
353.6 s
p90 651.4 s
Reader index
One score per reader per axis on a 0 to 100 scale, editable at any time.
Reader index
not rated yet
Lab index
66.36
2 readers have scored 100% of the index weight so far. A reader index is published at 3 voters and 50% coverage.
| Axis | Weight | Lab | Readers | Votes | Reader mean |
|---|---|---|---|---|---|
| Agent performance | 18% | 78.0 | 63.5 | 2 | |
| Reliability | 18% | 71.1 | 62.0 | 2 | |
| Scalability | 13% | 80.0 | 65.5 | 2 | |
| SEO and GEO | 12% | 54.0 | 44.0 | 2 | |
| API and MCP | 9% | 72.0 | 60.5 | 2 | |
| Integrations | 8% | 74.0 | 65.5 | 2 | |
| Design output | 7% | 60.0 | 54.0 | 1 | |
| Speed | 7% | 8.9 | 2.0 | 2 | |
| Value | 5% | 52.0 | 26.0 | 1 | |
| Code ownership | 3% | 94.0 | 75.5 | 2 |
The reader index is a weighted mean of reader submitted axis scores using the published index weights, renormalised over the axes readers have scored. It is never blended into the lab index. How this works.
Score Devin
Your scores feed the reader column only. Submitting an axis again edits your existing score rather than adding a second vote.
Reader scores need an account so one person cannot vote twice. Create a reader account or sign in.
Reader scores are published in a separate reader index column. They never move the measured lab index.
Reader lab notes on Devin
Written by readers with an account. Sorted by upvotes, then by date.
It finished unattended, and the bill made the decision for us
Jide Okonkwo | 12 Aug 2026
The thing it does that nothing else here does is finish while nobody is watching. We gave it a bounded task on a Friday and had a pull request with passing tests on Monday, no intervention. That is real and it is worth money. The problem is how much money. One full pass of a spec this size consumed most of a starter allowance, and once we projected the team using it the way we use the interactive agents, the monthly number was several times what we pay for everything else combined. I am scoring value at the floor and agent performance well above the middle, which I think is the honest shape of it.
David Novotny | 13 Aug 2026
We reached the same place. It is a specialist tool for work that is genuinely unattended, and using it as a general agent is how the invoice gets away from you.
Write a lab note
What you built, what broke, what you measured. Short opinions belong in the open rating form further down.
Open rating distribution
Open ratings submitted without an account. Neither the lab index nor the reader index is part of this average.
| Band | Share | Ratings |
|---|---|---|
| 90 to 100 | 0 | |
| 80 to 89 | 0 | |
| 70 to 79 | 0 | |
| 60 to 69 | 0 | |
| 0 to 59 | 0 |
Lab commentary
Our reading of the numbers, kept separate from the reader ratings.
The slowest median in the index by a wide margin and the most expensive route through the spec, because it works in its own environment and takes the long way through every task. What it buys is unattended completion: it opened a pull request with passing tests without a human in the loop in four of five runs. Judged on the published axes the cost weighs heavily against it, which is why the value subscore is the lowest in the agent group.
On the measured axes, Devin is strongest at code ownership with a subscore of 94.0 and weakest at speed at 8.9. Because speed carries 7 of the 100 index points, that weakness costs it roughly 6.38 points against a perfect result on that axis alone.
Reader ratings and lab measurements answer different questions. A reader rating carries the thing a harness cannot capture: whether the product was pleasant to work with over weeks, how support behaved, whether the bill matched the plan. The lab index carries the thing an opinion cannot: 45 timed executions of the same specification with the failures counted. Read them side by side and treat a large gap between the two as the interesting signal.
0 open ratings
Newest first, no account needed. Moderated for spam and vendor astroturfing, not for sentiment.
No reader ratings published yet. Yours would be the first.
Add an open rating
No account needed. For a score that counts toward the reader index, use the axis form above.