Head to head / curated pair / cycle 04
Devin vs Factory
Both products were given the identical 9 prompt vibeOps specification, 5 times each. Nothing on this page is an impression, all of it is a measurement from the same harness.
The slowest median in the index by a wide margin and the most expensive route through the spec, because it works in its own environment and takes the long way through every task. What it buys is unattended completion: it opened a pull request with passing tests without a human in the loop in four of five runs. Judged on the published axes the cost weighs heavily against it, which is why the value subscore is the lowest in the agent group.
Built for teams rather than individuals, and the difference is visible in the operational prompts: it attached the change to a ticket, wrote a migration plan and produced a rollback path without being asked. It is slow and it is priced for organisations, so an individual will find better value elsewhere in this group. The API subscore is the second strongest in the agent group.
Index gap
5.16
Factory leads
Axes won
1 / 9
Devin versus Factory, of ten axes
Faster median
Factory
161.7 s per prompt
Higher pass rate
Factory
82.2%
Every measurement, side by side
Cyan marks the better figure on each row. The delta column is the first product minus the second.
| Measurement | Devin | Factory | Delta |
|---|---|---|---|
| Composite index | 66.36 | 71.52 | -5.16 |
| Agent performance (weight 18) | 78.0 | 81.0 | -3.00 |
| Reliability (weight 18) | 71.1 | 82.2 | -11.10 |
| Scalability (weight 13) | 80.0 | 82.0 | -2.00 |
| SEO and GEO (weight 12) | 54.0 | 56.0 | -2.00 |
| API and MCP (weight 9) | 72.0 | 76.0 | -4.00 |
| Integrations (weight 8) | 74.0 | 78.0 | -4.00 |
| Design output (weight 7) | 60.0 | 62.0 | -2.00 |
| Speed (weight 7) | 8.9 | 19.4 | -10.50 |
| Value (weight 5) | 52.0 | 64.0 | -12.00 |
| Code ownership (weight 3) | 94.0 | 93.0 | +1.00 |
| Median seconds per prompt | 353.6 | 161.7 | +191.90 |
| p10 seconds | 230.2 | 84.8 | +145.40 |
| p90 seconds | 651.4 | 350.8 | +300.60 |
| Executions failed | 6 | 3 | +3.00 |
| HTML bytes, no JS | 20.0 KB | 19.0 KB | +1024.00 |
| LCP milliseconds | 1,900 | 1,940 | -40.00 |
| CLS | 0.050 | 0.050 | +0.00 |
| Entry price EUR | 18.00 | 36.00 | -18.00 |
Screenshot diff
Both landing pages, captured by the lab on 14 Aug 2026 at 1440 by 900. Stored locally, never hotlinked.
Devin, captured 14 Aug 2026.
Factory, captured 14 Aug 2026.
Speed spread on a shared scale
Devin
230.2 / 353.6 / 651.4 seconds
Factory
84.8 / 161.7 / 350.8 seconds
Axis by axis, who wins
Devin wins 1 of 10
- Code ownership94.0
Factory wins 9 of 10
- Agent performance81.0
- Reliability82.2
- Scalability82.0
- SEO and GEO56.0
- API and MCP76.0
- Integrations78.0
- Design output62.0
- Speed19.4
- Value64.0
Verdict
Factory takes the composite by 5.16 points. It wins on the weighted total, but the axis table is where the real decision sits: Devin takes 1 axes and Factory takes 9.
If the work in front of you is mostly interface, weight the design and speed rows. If it is a product with an API, webhooks and background jobs, weight agent performance, reliability and the API and MCP row, which together carry 45 of the 100 index points. If the project has to outlive its vendor, read the code ownership row first and treat everything else as secondary.
Both products in this comparison were measured in the same cycle, on the same specification, by the same harness. The per run data for each is on its product page and in /api/runs.json.