Head to head / curated pair / cycle 04
Devin vs Jules
Both products were given the identical 9 prompt vibeOps specification, 5 times each. Nothing on this page is an impression, all of it is a measurement from the same harness.
The slowest median in the index by a wide margin and the most expensive route through the spec, because it works in its own environment and takes the long way through every task. What it buys is unattended completion: it opened a pull request with passing tests without a human in the loop in four of five runs. Judged on the published axes the cost weighs heavily against it, which is why the value subscore is the lowest in the agent group.
An asynchronous agent: you hand it a task against a repository and come back to a proposed change. Judged on wall clock that is expensive, and it posts one of the slower medians in the index, but the comparison is not quite like for like since nobody sits and watches it. Change quality was good and the tests it added were real tests rather than assertions that always pass.
Index gap
2.33
Jules leads
Axes won
3 / 7
Devin versus Jules, of ten axes
Faster median
Jules
146.5 s per prompt
Higher pass rate
Jules
75.6%
Every measurement, side by side
Cyan marks the better figure on each row. The delta column is the first product minus the second.
| Measurement | Devin | Jules | Delta |
|---|---|---|---|
| Composite index | 66.36 | 68.69 | -2.33 |
| Agent performance (weight 18) | 78.0 | 80.0 | -2.00 |
| Reliability (weight 18) | 71.1 | 75.6 | -4.50 |
| Scalability (weight 13) | 80.0 | 74.0 | +6.00 |
| SEO and GEO (weight 12) | 54.0 | 55.0 | -1.00 |
| API and MCP (weight 9) | 72.0 | 66.0 | +6.00 |
| Integrations (weight 8) | 74.0 | 70.0 | +4.00 |
| Design output (weight 7) | 60.0 | 62.0 | -2.00 |
| Speed (weight 7) | 8.9 | 21.4 | -12.50 |
| Value (weight 5) | 52.0 | 84.0 | -32.00 |
| Code ownership (weight 3) | 94.0 | 96.0 | -2.00 |
| Median seconds per prompt | 353.6 | 146.5 | +207.10 |
| p10 seconds | 230.2 | 84.4 | +145.80 |
| p90 seconds | 651.4 | 292.7 | +358.70 |
| Executions failed | 6 | 4 | +2.00 |
| HTML bytes, no JS | 20.0 KB | 21.0 KB | -1024.00 |
| LCP milliseconds | 1,900 | 1,860 | +40.00 |
| CLS | 0.050 | 0.040 | +0.01 |
| Entry price EUR | 18.00 | 0.00 | +18.00 |
Screenshot diff
Both landing pages, captured by the lab on 14 Aug 2026 at 1440 by 900. Stored locally, never hotlinked.
Devin, captured 14 Aug 2026.
Jules, captured 14 Aug 2026.
Speed spread on a shared scale
Devin
230.2 / 353.6 / 651.4 seconds
Jules
84.4 / 146.5 / 292.7 seconds
Axis by axis, who wins
Devin wins 3 of 10
- Scalability80.0
- API and MCP72.0
- Integrations74.0
Jules wins 7 of 10
- Agent performance80.0
- Reliability75.6
- SEO and GEO55.0
- Design output62.0
- Speed21.4
- Value84.0
- Code ownership96.0
Verdict
Jules takes the composite by 2.33 points. It wins on the weighted total, but the axis table is where the real decision sits: Devin takes 3 axes and Jules takes 7.
If the work in front of you is mostly interface, weight the design and speed rows. If it is a product with an API, webhooks and background jobs, weight agent performance, reliability and the API and MCP row, which together carry 45 of the 100 index points. If the project has to outlive its vendor, read the code ownership row first and treat everything else as secondary.
Both products in this comparison were measured in the same cycle, on the same specification, by the same harness. The per run data for each is on its product page and in /api/runs.json.